Negotiation Martin Beer, School of Computing & Management Sciences, Sheffield Hallam University, Sheffield, United Kingdom

Introduction Principles of Negotiation –Game Theoretic Approaches Evaluation Criteria Voting Auctions General Equilibrium Markets Contract Nets –Heuristic-based Negotiation –Argumentation-based Negotiation

Principles of Negotiation Negotiation = Interaction amongst Agents based on communication for the purpose of coming to an agreement Distributed conflict resolution Decision Making Proposal accepted, refined, criticized or refuted

Principles of Negotiation Collectively Motivated Agents Common Goals Coordination to achieve Common goal Self-interested Agents Own Goals Coordination for Coherent Behaviour Coordination Distributed Search through a space of possible solutions

Negotiation Includes A communication language A negotiation Protocol A decision Process by which an agent decides upon its –Position –Concessions –Criteria for agreement etc.

Negotiation Strategies Negotiation can take a number of different forms –Single or multi-party negotiation –May include a single shot message by each party or a conversation with several messages going back and forth

Negotiation Techniques Game Theoretic Negotiation –Evaluation Criteria –Voting Auctions –General Equilibrium Market Mechanisms –Contract Nets Heuristic-based Negotiation Argument-based negotiation

Game Theoretic Negotiation Evaluation Criteria –Criteria to evaluate negotiation protocols among self- interested agents –Agents are supposed to behave rationally –Rational behavior = an agent prefers a greater utility (payoff) than a smaller one –Payoff maximization Individual payoffs Group payoffs Social welfare –Social Welfare The sum of agents’ utilities (payoffs) in a given solution Measures the global good of the agents Problem: How do we compare utilities?

Utility Utility is the means by which we optimise the results of the negotiation Utility often equates to price, this may not always be the case. It may be –Speed of response –“closeness” –Some combination of functions etc. Any function that can be readily computed can be used.

Pareto Efficiency A solution x –i.e. a payoff vector p(x 1, … x n ) is Pareto efficient (i.e. Pareto optimal) if there is no other solution x’ such that at least one agent is better off in x’ than in x and no agent is worse off in x’ than in x. Measures global good –Does not require utility comparison Social Welfare  Pereto Efficiency

Individual Rationality (IR) IR of an agent participation = The agent’s payoff in the negotiated solution is no less than the payoff that the agent would get by not participating in the negotiation A mechanism is IR if the participation is IR for all agents

Stability A protocol is stable if once the agents arrived at a solution they do not deviate from it Dominant Strategy –The agent is best off using a specific strategy no matter what strategies the other agents use Nash Equilibrium –The strategy profile S * A = –For each I, S * I is the agent’s best strategy given that the other agents use strategies P –Problems No Nash Equilibrium Multiple Nash Equilibria Guarantees stability only in the beginning of the game

Prisoners Dilemma Two prisoners are collectively charged with a crime and held in separate cells, with no way of meeting or communicating. They are told that: –If one confesses and the other does not, the confessor will be freed, and the other will be jailed for three years –If both confess, then each will be jailed for two years Both prisoners know that if neither confesses, then they will each be jailed for one year.

Prisoner’s Dilemma – Possible Outcomes Note – 4 is immediate release Player j CooperateDefect Player iCooperate4,41,4 Defect4,13,3

Prisoner’s Dilemma – Strategic Considerations The individual rational action is to defect This guarantees a payoff of at least 2 whereas cooperating guarantees a payoff of at most 1 Logic Says that this is not the best alternative – if both cooperate the payoff is 3 This is the fundamental problem with multi-agent systems It appears to imply that cooperation will not occur with self-interested agents Can we get over this? Yes – by repeating the problem many times

Computational Efficiency To achieve perfect rationality –The number of options to consider is too big –Sometimes no algorithm finds the optimal solution Bounded Rationality –Limits the time/computation for options consideration –Prunes the search space –Imposes restrictions on the types of spaces

Voting Truthful Voters –Rank feasible social outcomes based on agents’ individual ranking of those outcomes –A – set of n agents –O – set of m feasible outcomes –Each agent has a preference relation < I : O x O, asymmetric and transitive Social Choice Rule –Input: the agent preference relations (< 1, …. < n ) –Output: elements of O sorted according to the input – gives the social preferences relation < *

Properties of the Social Choice Rule A social preference ordering <* should exist for all possible inputs (individual preferences) <* should be defined for every pair (o,o’)  O <* should be asymmetric & transitive over O The outcomes should be Pareto efficient If  I  A, o< I o’ then o <* o’ The scheme is independent of irrelevant alternatives If  I  A, < and <‘ satisfy o < I o’ and o <’ I o’ then the social ranking of o and o’ is the same in these two situations No agent should be a dictator in the sense that o < I o’ implies o<* o’ for all preferences of the other agents

Arrow’s Impossibility Theorem No Social choice rule satisfies all of the six conditions listed on the last slide Alternatives –Binary Protocol Alternatives are voted pair-wise The chosen alternative depends on the agenda That is the order of the pairings This is the method used in parliamentary debates –Borda Protocol Assigns an alternative |O| points for the highest preference, |O| -1 points for the second and so on ….. The counts are summed across the voters and the alternative with the highest count becomes the social choice Winner turns loser and loser turns winner if the lowest ranked alternative is removed

Auctions - Theory The auctioneer wants to sell an item at the highest possible payment and the bidders want to acquire the item at the lowest possible price A centralised protocol includes –One auctioneer –Several bidders The auctioneer introduces the item for sale, which can be a combination of other items, or have multiple attributes Bidders make offers. This may be repeated for several times, depending on the auction type The auctioneer determines the winner Auction Characteristics –Simple protocols –Centralised –Allows collusion “behind the scenes” –May favour auctioneer

Auction Settings Private Value Auctions –The value to a bidder agent depends on its private preferences –Assumed to be known exactly Common Value Auctions –The good’s value depends entirely on other agents’ valuation Correlated Value Auctions –The good’s value depends on internal and external valuations

English (First-price Open Cry) Auction –Each bidder announces openly its bid –When no bidder is willing to raise anyone, the auction ends –The highest bidder wins the item at the price of the bid Strategy –In private value auctions the dominant strategy is to always bid a small amount more than the current highest bid and stop when the private value is reached –In correlated value auctions the bidder increases the price at a constant rate or at a rate it thinks appropriate

First-Price Sealed-Bid Auction Each bidder submits one bid without knowing the other’s bids The highest Bidder wins the item and pays the amount of their bid Strategy –No dominant strategy –Bid less than its true valuation but it is dependant on the other agents bids which are not known

Dutch (descending) Auction The auctioneer continuously lowers the price until one of the bidders takes the item at the current price Strategy –Strategically equivalent to the first-price sealed-bid auction –Efficient for real time

Vickery (Second-Price Sealed Bid) Auctions Each bidder submits one bid without knowing the others’ bids The highest bidder wins but pays the price of the second bid Strategy –The bidder dominant strategy is to bid its true valuation

All-Pay Auctions Each participating bidder has to pay the amount of their bid (or some other amount) to the auctioneer

Problems with Auction Protocols They are not collusion proof Lying Auctioneer –Problem with Vickery Auction –Problem with English Auction – use skills that bid in the auction to increase bidders’ valuation of the item –The auctioneer bids the highest second price to obtain its reservation price – may lead to the auctioneer keeping the item –Common value auctions suffer from the winner’s curse: agents should bid less than their valuation prices (as winning the auction means its valuation is too high) –Interrelated auctions – the bidder may lie about the value of an item to get a combination of items at its valuation price

General Equilibrium Markets General Equilibrium Theory =A macroeconomic theory A set of goods available at different prices Two types of agents – Consumers & Producers The producer’s profits are profits are divided among the consumers according to predetermined proportions that need not be equal The producers’ profits are divided among consumers according the shares they ‘own’ Prices may change and the agents may change their consumption and production plans but –Actual production and consumption only occur when the market has reached a general equilibrium

Contract Nets General Equilibrium Market Mechanisms use –Global prices –A centralised mediator Drawbacks –Not all prices are global –Bottleneck of the mediator –Mediator – point of failure –Agents have no direct control over the agents to which they send information Need a more distributed solution Task allocation via negotiation – Contract Net –A kind of bridge between game theoretic negotiation and heuristic-based one –Formal model for making bids and awarding decisions

Contract Net Protocol –The agents suggest contracts to each other and make their accepting/rejecting decisions based on marginal cost calculations –The agent can take the roles of both Contractor and Contractee –It can also contract out tasks that it received earlier via another contract –The agents do not know the tasks and cost functions of other agents –Task allocation improves with each step - hill climbing in the space of task allocations where the high-metric of the hill is social welfare –It is an anytime algorithm Contracting can be terminated anytime The worth of each agent’s solution increases monotonically  Social Function increases monotonically

Contract Net Problem –Task allocation stuck in a local equilibrium = no contract net is individually rational and the task allocation is not globally optimal Possible solution –Different contract types O – one task C – cluster contracts S – swap contracts M – multi-agent contracts –For each of the four contract types there exists task allocations \for which there is an IR contract under one type but no IR contracts under the other three types –Under all four contract types there are initial task allocations for which no IR sequence of contracts will lead to the optimal solution (social welfare)

Contract Net Main differences as compared to game theoretic negotiation –An agent may reject an IR contract –An agent may accept a non-IR contract –The order of accepting IR contracts may lead to different pay offs –Each contract is made by evaluating just a single contract instead of doing look ahead in the future Untruthful Agents –An agent may lie about its marginal costs –An agent may lie about what tasks it ahs Hide tasks Phantom tasks Decoy tasks –Sometimes lying may be beneficial

Heuristic-based Negotiation Produce good rather than optimal solution Heuristic-based negotiation –Computational approximations of game theoretic techniques –Informal negotiation models No central mediator Utterances are private between negotiating agents The protocol does not prescribe an optimal course of action Central concern: the agent’s decision making heuristically during the course of negotiation

Argumentation-based Negotiation Arguments used to persuade the party to accept a negotiation proposal Different types of agents Each Argument type defines preconditions for its usage. If the preconditions are met, then the agent may use the argument The agent needs a strategy to decide which argument to use Most of the times assumes the BDI model

Argumentation-based Negotiation Strategies –Appeal to past promise –Promise a future reward –Appeal to self interest –Threat