Download presentation

Presentation is loading. Please wait.

Published byMark Lilly Modified about 1 year ago

1
SIF8072 Distributed Artificial Intelligence and Intelligent Agents Lecture 2: Multi-agent Interactions Lecturer: Sobah Abbas Petersen

2
2 Lecture Outline 1.Multi-agent Systems 2.Utility and Preferences 3.Game Theory and Payoff Matrices 4.Strategies 5.Negotiation - Auctions 6.Summary

3
3 References Wooldridge: ”Introduction to MAS” –Multi-agent Interactions: Chapters 6 –Auctions: Chapter 7

4
4 Interactions ”The world functions through interacting agents. Each person pursues his/her own goals through encounters with other people or machines.” ”Rules of Encouter” by Rosenchein and Zlotskin, 1994

5
5 Example 1 Two students decide to work together on their exercises. They have to decide upon a time. One prefers to work on Thursday afternoons after the lecture while the other prefers to work on Friday morning. How do they decide upon a time to do the work?

6
6 Example 2 A friend invites you out for a drink and the cinema tonight. But your favourite TV program is on tonight. You think: –It would be nice to go out with my friend, but it’s cheaper to watch TV. –If you stay at home and watch TV, you might not have a chance to go out with your friend for a long time. –I can always record the program and watch it afterwards. –I can invite my friend home.

7
7 Multi-agent Systems (MAS) Contains a number of agents which: –interact with one another through communication –are able to act in an environment –have different ”spheres of influence” –may be linked by other relationships, e.g. organisational It is important to understand the type of interaction. Each agent can be assumed to be self-interested: –has its own preferences and desires about how the world should be.

8
8 Multi-agent Systems (MAS) Environment Sphere of influence Multi-agent System Agent Interaction Organisational relationship

9
9 Utilities and Preferences Assume we have 2 agents: Ag = {i,j}. Assume ={ 1, 2,….} is the set of ”outcomes” that agents have preferences over. We capture preferences by utility functions: u i : IR u j : IR Utility functions lead to preference orderings over outcomes: ≥ i ’ means u i ( ) ≥ u i ( ’) > i ’ means u i ( ) > u i ( ’)

10
10 What is Utility? Utility is not money, but a useful analogy Typical relationship between utility and money: Utility Money

11
11 Multi-agent Encounters 1 Need a model of the environment in which the agents will act. Agents simultaneously choose an action and, as a result, an outcome in will result. Actual outcome depends on a combination of actions. Environment behaviour given by state transformer function: ( reference: p31 of textbook ) : A c A c Agent i’s actionAgent j’s action

12
12 Multi-agent Encounters 2 Assume that each agent has two possible actions: 1.C: cooperate 2.D: defect Let Ac = {C,D}

13
13 State Transformer Funtions Environment sensitive to actions of both agents: (D,D)= 1 (D,C)= 2 (C,D)= 3 (C,C)= 4 Environment where neither agent has any influence: (D,D)= 1 (D,C)= 1 (C,D)= 1 (C,C)= 1 Environment controlled by j: (D,D)= 1 (D,C)= 2 (C,D)= 1 (C,C)= 2 Let Ac = {C,D}

14
14 Agent’s Preference Consider the case where both agents influence the outcome and they have the following utility functions : u i ( 1 )=1 u i ( 2 )=1 u i ( 3 )=4 u i ( 4 )=4 u j ( 1 )=1 u j ( 2 )=4 u j ( 3 )=1 u j ( 4 )=4 u i (D,D)=1 u i (D,C)=1 u i (C,D)=4 u i (C,C)=4 u j (D,D)=1 u j (D,C)=4 u j (C,D)=1 u j (C,C)=4 Then, agent i ’s preferences are: C,C i C,D i D,C i D,D Agent i preferes all outcomes that arise through C over all outcomes that arise through D

15
15 Payoff Matrices We can characterise the previous scenario in a payoff matrix e.g. Top right cell: i cooperates, j defects Defect Coop DefectCoop j i Agent i is the column player (payoff received by i shown in top right of each cell) Agent j is the row player

16
16 Game Theory A mathematical theory that studies interactions about self-interested agents. Essential elements of a game are: –Players (2 or more) –Some choice of action (strategy) –One or more outcomes (someone wins, someone loses) –Information Suitable for situations where the other agent’s (player’s) behaviour matters.

17
17 The Prisoner’s Dilemma 1 2 men are collectively charged with a crime and held in separate cells. They have no way of communicating with each other or making an agreement. They are told: –if one confesses and the other does not, confessor will be freed and the other jailed for 3 years. –if both confess, then each will be jailed for 2 years. –If neither confess, then each will be jailed for 1 year. Confessing => defecting (D) Not confessing => cooperating (C) If you were one of the prisoners, what would you do? Discuss your answer with your neighbour.

18
18 The Prisoner’s Dilemma 2 Payoff matrix for Prisoner’s Dilemma: Defect Coop DefectCoop j i Top left: If both defect, punishment for mutual defection. Top right: if i cooperates and j defects, i gets sucker’s payoff of 1 while j gets 4. Bottom left: if j cooperates and i defects, j gets sucker’s payoff of 1 while i gets 4. Bottom right: Reward for mutual cooperation. ã Numbers in the payoff matrix reflect how good an outcome is for the agent. e.g. u i (D,D)=2 u i (D,C)=4 u i (C,D)=1 u i (C,C)=3 u j (D,D)=2 u j (D,C)=1 u j (C,D)=4 u j (C,C)=3

19
19 The Prisoner’s Dilemma 3 The individual rational agent will defect! –This guarantees a payoff of no worse than 2 –Cooperating guarantees a payoff of at most 1 Defection is the best response to all possible strategies –Both agents defect and get a payoff = 2. If both agents cooperate, they will each get payoff = 3. –(The other prisoner is my twin!) Can we recover cooperation? The Iterated Prisoner’s Dilemma

20
20 Let’s take a minute….. How can we apply the Prisoner’s Dilemma to real situations? e.g. Arms races – nuclear weapons compliance treaty between two countries. Can you think of other situations?

21
21 Strategies ”A strategy is the way an agent behaves in an interaction”. (Ref: Rosenchein and Zlotskin, 1994) –From game theory: strategies are actions of agents (Ac) When 2 agents encounter, important question: What should I do?

22
22 Dominance Given any particular strategy s (e.g. C or D), there will be a number of outcomes. We say that s 1 dominates s 2 if every outcome possible by i playing s 1 is preferred over every outcome possible by i playing s 2.

23
23 Nash Equilibrium 2 strategies s 1 and s 2 are in Nash Equilibrium if: –Under the assumption that agent i plays s 1, agent j can do no better than play s 2 ; –Under the assumption that agent j plays s 2, agent i can do no better than play s 1 ; Neither agant has any incentive to deviate from a Nash Equilibrium. Unfortunately: –Not every interaction scenario has a Nash Equilibrium. –Some interaction scenarios have more than one Nash Equilibrium.

24
24 Nash Equilibrium - Example The Battle of the Sexes Conflict between a man and a woman, where the man wants to go to a Prize Fight and the woman wants to go to a Ballet They are deeply in love. So, they would make a sacrifice to be with each other. 2 Nash Equilibria Strategy combination (Prize Fight, Prize Fight) Strategy combination (Ballet, Ballet) Woman Prize Fight Ballet Prize Fight Ballet Man Ref: ”Games and Information, E. Rasmussen, 2001

25
25 Let’s play a little game….. Guess half the average Choose a number between 0 and 100. Your aim is to choose a number that is closest to half the average of the numbers chosen by all the students. What is your number?

26
26 Competitive and zero-sum Interactions One agent can only get a more preferred outcome at the expense of the other agent strictly competitive. Zero-sum encounters –u i ( ) + u j ( ) = 0, for all . –e.g. A football game where only one team can win.

27
27 Assumptions in Game Theory All Players behave rationally –Not always the case with all agents! Each player knows the rule. Payoffs are known and fixed. These are limitations!

28
28 Multi-agent Interaction: Summary MAS: a number of agents which interact with one another through communication. An agent’s action results in an outcome in the environment. Utility functions are used for preference orderings. Game theory – a mathematical theory that studies interactions among agents. An agent’s action is a strategy: –Dominant –Nash Equilibrium

29
29 Negotiation ”The process of several agents searching for an agreement” e.g. about price. Reaching consensus ”Rules of Encouter” by Rosenchein and Zlotskin, 1994

30
30 Auction: Example 1 Several millions of $ paid for art at auction houses such as Sotheby’s. Ears 2 u, Vincent!

31
31 Auction: Example 2 Online Auctions You want to buy some exciting video games. You see that there are some available on eBay. You register at eBay and offer a bid for some of these games.

32
32 Auctions An Auction takes place between an auctioneer and a collection of bidders. Goal is for the auctioneer to allocate the goods to one of the bidders. In most settings, the auctioneer desires to maximise the price; bidders desire to minimise the price. auctioneer bidders auctioneer bidder Price

33
33 Auction Parameters Value of goodsPrivate, public/common, Correlated Winner determinationFirst price, second price Bids may beOpen cry, Sealed Bidding may beOne shot, ascending, descending

34
34 English Auctions English auctions are: –First price –Open cry –Ascending Dominant strategy: successively bid a small amount more than the highest current bid until it reaches the valuation, then withdraw. Susceptible to Winners curse –Winner is the one who overvalues the goods on offer and may end up paying more than its worth. auctioneer Bidder 1 Price Bidder x

35
35 Dutch Auctions Dutch auctions are: –Open cry –Descending Auctioneer starts at an artificially high price. Then continually lowers the offer price until an agent makes a bid which is equal to the current offer price. Dominant strategy: None Susceptible to Winners curse auctioneer Bidder Price auctioneer

36
36 First-price Sealed-bid Auctions One shot auction Single round, where bidders submit a sealed-bid for the good. Good is awarded to agent that made the highest bid. Winner pays price of highest bid. Best strategy: bid less than true value. auctioneer Bidders

37
37 Vickrey Auctions Vickrey auctions are: –second-price –sealed-bids Good is awarded to agent that made the highest bid. Winner pays price of second highest bid. Best strategy: bid the true value. Susceptible to anti-social behaviour

38
38 Lies and Collusions Lies: –By the bidders (e.g. In Vickrey auctions) –By the auctioneer (shills, in Vickrey auction) Collusion of bidders –Coalition of bidders where they agree beforehand to put forward artificially low bids for the good on offer. When the good is obtained, the bidders can then get the true value of the good and share the profits.

39
39 Limitations of Auctions Only concerned with the allocation of goods; Not adequate for settling agreements that concerns matters of mutual interest. Negotiation

40
40 Let’s take a minute…… Can you think of any auctions that you have come across? How about offering your notebook to the highest bidder at the end of the year….. Discuss with your neighbour.

41
41 …..Selecting a Bid

42
42 Auctions: Summary An Auction takes place between an auctioneer and a collection of bidders. In most settings, the auctioneer desires to maximise the price; bidders desire to minimise the price. Types of Auctions: –English auction –Dutch auction –First-price sealed bids –Vickrey (Second-price sealed bids) Useful for allocating goods. But too simple for many other settings.

43
43 Next Lecture: Negotiation Will be based on: ”Reaching Agreements”, Chapter 7 in Wooldridge: ”Introduction to MultiAgent Systems” Coordination – Working together, Chapter 9

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google