1. Reaching Agreements II

2. 2 What utility does a deal give an agent? Given encounter  T 1,T 2  in task domain  T,{1,2},c  We define the utility of a deal  for agent k as U k (  ) = c (T k )  c (  ) The difference between The cost of agent k doing the tasks T k it was originally assigned The cost of the tasks it is assigned in  Pure deal  =  T 1,T 2  is called the conflict deal –Because no-one agrees to execute tasks other than its own Utility of Deals

3. 3 Utility of Deals in the Delivery Domain The stand-alone cost –For agent 1 is 1 –For agent 2 is 3 Deal  {a},{b}  gives –Agent 1 no utility –Agent 2 utility of 1 Deal  {a,b},{}  gives –Agent 1 utility of -2 –Agent 2 utility of 3 Distribution Point 21 Warehouse a Warehouse b

4. 4 Dominating Deals For two pure deals  1 and  2,  1 dominates  2, denoted as  1 >  2, if, and only if, –For all i  {1,2}, U i (  1 )  U i (  2 ) deal  1 is at least as good as  2 for all agents… –There is an i  {1,2}, U i (  1 ) > U i (  2 ) deal  1 is better for some agent(s)…  1 weakly dominates  2, denoted as  1   2, iff –For all i  {1,2}, U i (  1 )  U i (  2 )  1 is equivalent to  2, denoted as  1   2, iff –For all i  {1,2}, U i (  1 ) = U i (  2 )

5. In our delivery domain, we have the following relationships between deals:  {},{a,b}    {a},{a,b}    {a,b},{a,b}   {a},{b}    {a},{a,b}    {a,b},{a,b}  5 Dominating Deals in Delivery Domain

6. 6 The Negotiation Set A deal  is called individual rational if    A deal  is called Pareto optimal if there does not exist another deal  ’ such that  ’   The set of all deals that are individual rational and Pareto optimal is called the negotiation set. The negotiation set for the delivery domain is: {  {},{a,b} ,  {a},{b}  } –N.B.: We don’t have  {a,b},{}  because it is not individual rational for agent 1

7. 7 Monotonic Concession Protocol (1) Negotiation proceeds in a series of rounds On the first round, both agents simultaneously propose a deal from the negotiation set An agreement is reached –If the agents propose deals  1 and  2 such that either U 1 (  2 )  U 2 (  1 ) or U 2 (  1 )  U 1 (  2 ) That is, an agent finds that the deal proposed by the other is as good as or better than the proposal it made If an agreement is reached, the agreement deal: –If both agent’s offers match or exceed those of the other agent, then one of the proposals is selected at random –If only one proposal exceeds or matches the other’s proposal, then this is the agreement deal

8. 8 Monotonic Concession Protocol (2) If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals –In round u + 1, no agent is allowed to make a proposal less preferred by the other agent than deal proposed at round u If neither agent makes a concession in round u > 0, then negotiation terminates, with the conflict deal

9. 9 Monotonic Concession Protocol (3) For example, –If agent 1 offers a deal that gives agent 2 a utility of 14, –While 2 proposes a deal that gives himself utility of 11, –Then the agreement will be 1’s offer. If both offer such a deal… –An arbitrary choice is made between them. If neither agent offers such a deal… –The negotiation continues to another round. In each round, –An agent may propose the same utility to the other or –Offer more than it did in the previous round.

10. 10 Monotonic Concession Protocol (4) Finally, if neither agent concedes in a round, the agreement reached is the conflict deal. The monotonic concession protocol over pure deals in any encounter in any task environment will terminate after a finite number of rounds. A 2 ’s best deal A 1 ’s best deal

11. 11 Negotiation Strategy Given a negotiation protocol, A negotiation strategy is a function from the history of the negotiation (run of the negotiation process until now) to the current offer consistent with protocol. An example of a strategy: –If the other agent conceded the last n rounds, I will concede with a probability of 1/n+1. We assume that an agent’s strategy doesn’t change during negotiation.

12. 12 Choosing a Negotiation Strategy What negotiation strategy should be adopted? Agents using the monotonic concession protocol can run into conflict if neither concedes during a round; –The deal is then the conflict deal… This can happen even if there are better deals for both agents –Such a result is not efficient! However, to avoid an inefficient outcome, an agent may concede at every step –This is not stable! –The best response to this strategy is to never concede!

13. 13 An Initial Strategy Consider this strategy: –Start by making an offer that is best for me. –Make a minimal concession in the next round. –Then, use the rule: if my opponent has conceded x% of the time, I will concede with probability (100-x)%. This, however, is not stable! Consider 2 agents: –They start by conceding, but as negotiation proceeds, the chance of further concessions diminishes. –Reaching conflict when they can do better is inefficient, and if an agent knows the other is using this strategy the best it can do is never concede.

14. 14 Risk Evaluation One way of thinking about which agent should concede is to consider how much each has to lose by running into conflict. An agent that has made many concessions has less to lose if conflict is reached. This is the inverse of the initial strategy considered: –The more concessions you’ve already made, the less likely you will make the next concession.

15. 15 For each round t, and each agent i, let  i t be the deal offer made by i in round t. The degree of willingness to risk conflict is Risk i t = More formally: Zeuthen Risk Evaluation (1) utility agent i loses by conceding and accepting agent j ’s offer utility agent i loses by not conceding and causing a conflict H G Zeuthen 1, if U i (  i t ) = 0 U i (  i t )  U i (  j t )otherwise U i (  i t ) Risk i t =

16. 16 If t is not the last round in the negotiation protocol, then the risk is always between 0 and 1, for both agents. As Risk 1 t increases, agent 1 has less to lose from a conflict deal, and will be more willing to concede. Let’s consider the strategy where the agent with a smaller risk will make the next concession –Call this the Zeuthen Strategy. Zeuthen Risk Evaluation (2)

17. 17 The Zeuthen Strategy In round 1: –Offer the deal that is best for you among all those in the negotiation set. In round t >1: –Calculate your risk Risk i t and the risk of your opponent Risk j t –If your risk is smaller than or equal to that of your opponent, then make an offer that involves the minimal sufficient concession from your point of view. –Otherwise, offer the same deal that you offered previously

18. 18 Efficiency of the Zeuthen Strategy Two agents using the Zeuthen strategy will not run into conflict. Further, they will agree on a deal that maximises the product of their utilities; i.e. it is Pareto Optimal.

19. 19 Stability of the Zeuthen Strategy A negotiation strategy s is in symmetric Nash equilibrium –If assuming that agent 1 uses s, –Agent 2 can do no better by using a strategy other than s. The Zeuthen Strategy is not in equilibrium. If two agents are one step from agreement: –It is sufficient for one of them to concede. –If one agent knows the other is using the Zeuthen Strategy, it could diverge from the strategy and gain benefit from the other conceding.

20. 20 Last Round in Zeuthen Strategy Suppose 2 agents, each must deliver to warehouse a. Negotiation set = {  {},{a} ,  {a},{}  } Agent 1 starts by offering  {},{a}  Agent 2 starts by offering  {a},{}  If they follow the strategy they must concede so that 1 offers  {a},{}) and 2 offers  {},{a} . –Tossing a coin would be the only solution. However, –If an agent doesn’t concede (breaks from the strategy) and the other does conform to the strategy, –Then it will receive a utility of 1.

21. 21 Zeuthen Last Round: A Game of Chicken 50% chance of winning the fair coin toss. Other agent delivers both containers. Agent 1 Don’t Concede Concede Don’t Concede 0000 0101 Concede 1010 0.5 Agent 2

22. 22 Summary (1) Monotonic concession protocol is –Symmetric –Distributed –Guaranteed to terminate, It also fits well with our intuition about negotiations However –Termination depends on common knowledge about the negotiation set. The Zeuthen strategy –Ensures that agents move toward agreement and, –Because it uses the concept of risk, can be seen as fair. –Agents maximise the product of their utilities.

23. 23 Summary (2) The Zeuthen strategy has problems in the last step. –If an agreement will be reached when one agent concedes, it becomes a “game of chicken” –This means that agents must have additional strategies for the last step. Therefore, there is a finite and positive probability that conflict will occur, leading to inefficient solutions.

24. 24 Summary (3) The Zeuthen strategy is not particularly simple. –Agents need to compute the entire negotiation set –This could be very large… It also involves many steps to reach agreement –A potentially high communication overhead Finally, and most importantly, –An agent needs to know the other agent’s utility function

25. 25 An Introduction to Multi-Agent Systems, M. Wooldridge, John Wiley & Sons, 2002. Chapter 7. Recommended Reading