1. Mobile Computing Group An Introduction to Game Theory part I Vangelis Angelakis 14 / 10 / 2004

2. Introduction TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Game theory is the mathematically founded study of conflict and cooperation. Game theoretic concepts apply when the decisions/actions of independent agents affect the interests of others Agents may be individuals, groups, firms, “intelligent” devices Game theory provides them with a methodology for structuring and analyzing problems of strategic choice. By formally modeling a situation as a game requires to enumerate the players, their preferences and their strategic options, The decision maker is given a clearer and broader view of the situation in hand. Game theory is the mathematically founded study of conflict and cooperation. Game theoretic concepts apply when the decisions/actions of independent agents affect the interests of others Agents may be individuals, groups, firms, “intelligent” devices Game theory provides them with a methodology for structuring and analyzing problems of strategic choice. By formally modeling a situation as a game requires to enumerate the players, their preferences and their strategic options, The decision maker is given a clearer and broader view of the situation in hand.

3. A little history lesson TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 1838:The first formal game theoretic analysis –the study of a duopoly by A. Cournot. 1921:E. Borel suggests a formal theory of games 1928:“Theory of parlour games” by von Neumann 1944:“Theory of Games and Economic behaviour” by von Neumann & O. Morgenstern –basic terminology & problem setup is standardized in this book. 1951:J. Nash proves that finite games always have an equilibrium… 1950’s-60’s:applications to war and (international) politics 1970’s:revolutionalized the economic theory, applications in sociology,psychology, evolutionary biology 1990’s:E/M spectrum auctions design 1994: Nash is awarded the Nobel prize in economics 1838:The first formal game theoretic analysis –the study of a duopoly by A. Cournot. 1921:E. Borel suggests a formal theory of games 1928:“Theory of parlour games” by von Neumann 1944:“Theory of Games and Economic behaviour” by von Neumann & O. Morgenstern –basic terminology & problem setup is standardized in this book. 1951:J. Nash proves that finite games always have an equilibrium… 1950’s-60’s:applications to war and (international) politics 1970’s:revolutionalized the economic theory, applications in sociology,psychology, evolutionary biology 1990’s:E/M spectrum auctions design 1994: Nash is awarded the Nobel prize in economics

4. Defining Games TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 A Game is a formal model of an interactive situation The formal definition of a game declares: The players Their preferences Their information The strategic actions that are available to them How these actions influence the outcome Games with single players are characterized decision problems Two “schools” of game theory are formed depending on the focus of games and the granularity of the game description A Game is a formal model of an interactive situation The formal definition of a game declares: The players Their preferences Their information The strategic actions that are available to them How these actions influence the outcome Games with single players are characterized decision problems Two “schools” of game theory are formed depending on the focus of games and the granularity of the game description

5. Cooperative Games TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 A high level description declaring: -payoffs of each potential coalition that can play a game -not the formation process of the coalition Hence, it is not defined as a game in which players actually do cooperate, but as a game in which any cooperation is enforced by an outside party. This outside party involvement and the possible bargaining process that takes place do not belong to the cooperative game description. Cooperative game theory investigates coalitional games with respect to relative power held by the players and how a successful coalition should divide its gains. A high level description declaring: -payoffs of each potential coalition that can play a game -not the formation process of the coalition Hence, it is not defined as a game in which players actually do cooperate, but as a game in which any cooperation is enforced by an outside party. This outside party involvement and the possible bargaining process that takes place do not belong to the cooperative game description. Cooperative game theory investigates coalitional games with respect to relative power held by the players and how a successful coalition should divide its gains.

6. Non-cooperative Games TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Players make choices of their own interest Order and timing may be crucial to determine the outcome of a game In a non-cooperative game players are unable to make enforceable contracts outside of those specifically modelled in the game. Hence, it is not defined as games in which players do not cooperate, but as games in which any cooperation must be self- enforcing. Cooperation arises, when it is in the best interest of players Players make choices of their own interest Order and timing may be crucial to determine the outcome of a game In a non-cooperative game players are unable to make enforceable contracts outside of those specifically modelled in the game. Hence, it is not defined as games in which players do not cooperate, but as games in which any cooperation must be self- enforcing. Cooperation arises, when it is in the best interest of players

7. Assumptions TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Strategic Behavior Being aware of your opponents existence and act trying to anticipate/counter their moves is called strategic behavior Rationality rational players always chose actions that give them an outcome they most prefer, given what their opponents will do. Predict how a game will be played or Advice how to best play in a game against rational opponents Rationality is an assumption under common knowledge Some define games as: The interaction among a group of rational agents who behave strategically Strategic Behavior Being aware of your opponents existence and act trying to anticipate/counter their moves is called strategic behavior Rationality rational players always chose actions that give them an outcome they most prefer, given what their opponents will do. Predict how a game will be played or Advice how to best play in a game against rational opponents Rationality is an assumption under common knowledge Some define games as: The interaction among a group of rational agents who behave strategically

8. Strategic & Extensive Form Games TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Strategic form (normal form) A strategic form game can be a single round of a repeated game Time invariant (no turns) game is played in the blink of an eye List each players strategies List outcomes that result from each possible combination of moves The outcome of a game is the payoff each player gets The Payoff is a numerical value, also called Utility. Extensive form (game tree) The complete description of how a game is played as time goes by + Order of players + Information of players at each point in time More on extensive forms in part II Strategic form (normal form) A strategic form game can be a single round of a repeated game Time invariant (no turns) game is played in the blink of an eye List each players strategies List outcomes that result from each possible combination of moves The outcome of a game is the payoff each player gets The Payoff is a numerical value, also called Utility. Extensive form (game tree) The complete description of how a game is played as time goes by + Order of players + Information of players at each point in time More on extensive forms in part II

9. Dominance TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Assume a rational player having strategies A and B. If A has a higher utility than B no mater what strategic combination other players make, then strategy A is said to dominate strategy B. Rational players do not play dominated strategies. In some games examining available strategies and eliminating dominated strategies results in only one credible strategy for a rational player. Assume a rational player having strategies A and B. If A has a higher utility than B no mater what strategic combination other players make, then strategy A is said to dominate strategy B. Rational players do not play dominated strategies. In some games examining available strategies and eliminating dominated strategies results in only one credible strategy for a rational player.

10. Prisoners’ Dilemma TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 A bank is robbed and the robbers escape the police… A pair of suspects are arrested later in the day and are held in separate cells. Each is told that: -if one alone confesses the robbery he will be granted pardon and the other goes in prison for a long time. -if both confess then both go in prison for a short time -if both deny then both go in prison for a very short time (let’s say they will be charged only with arms possession) What action will each prisoner take? A bank is robbed and the robbers escape the police… A pair of suspects are arrested later in the day and are held in separate cells. Each is told that: -if one alone confesses the robbery he will be granted pardon and the other goes in prison for a long time. -if both confess then both go in prison for a short time -if both deny then both go in prison for a very short time (let’s say they will be charged only with arms possession) What action will each prisoner take?

11. Prisoners’ Dilemma TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I DenyConfess Deny 2222 3030 Confess 0303 1111 Elimination of dominated strategies

12. Prisoners’ Dilemma TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 The individually rational outcome is worse for both players… Arms races, environmental pollution, etc are modeled by prisoners’ dilemmas… The solution to a single game is a max-min strategy, a better-play-it-safe strategy In a repeated game patterns for cooperation among the players arise (“suspects” will actually play the “deny-deny” move…) A repeated game allows for a strategy to be dependent on past moves, thus allowing for reputation effects and retribution. In infinitely repeated games, trigger strategies such as tit-for-tat encourage cooperation. The individually rational outcome is worse for both players… Arms races, environmental pollution, etc are modeled by prisoners’ dilemmas… The solution to a single game is a max-min strategy, a better-play-it-safe strategy In a repeated game patterns for cooperation among the players arise (“suspects” will actually play the “deny-deny” move…) A repeated game allows for a strategy to be dependent on past moves, thus allowing for reputation effects and retribution. In infinitely repeated games, trigger strategies such as tit-for-tat encourage cooperation.

13. Quality Selection TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Customer VoIP Provider BuyDon’t buy High 2222 1010 Low 0303 1111

14. The Nash Equilibrium TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 A set of strategies such that no player has incentive to unilaterally change her action. Players are in equilibrium if a change in strategies by any one of them would lead that player to earn less than if she remained with her current strategy. A set of strategies such that no player has incentive to unilaterally change her action. Players are in equilibrium if a change in strategies by any one of them would lead that player to earn less than if she remained with her current strategy.

15. Revisiting Quality Selection TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 Customer VoIP Provider BuyDon’t buy High 2222 1010 Low 0303 1111 1 No dominant Strategies ! – Two Nash Equilibria

16. Equilibrium selection TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 In games with multiple Nash equilibria a theory of strategic interaction should guide players to the “most reasonable” equilibrium Large number of papers focused with equilibrium refinements that attempt to derive conditions making an equilibrium more convincing than the another. In games with multiple Nash equilibria a theory of strategic interaction should guide players to the “most reasonable” equilibrium Large number of papers focused with equilibrium refinements that attempt to derive conditions making an equilibrium more convincing than the another. Custome r VoIP Provider BuyDon’t buy High 2222 1010 Low 0101 1111 For example, in the previous game the “High- buy” equilibrium yields higher utility for both players, so it can easily be argued that they will be self-coordinated to it.

17. Prisoners’ Dilemma Nash Equilibrium TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I DenyConfess Deny 2222 3030 Confess 0303 1111 Single Strategy combination rising from the process of dominated strategies elimination is a Nash Equilibrium

18. Mixed Strategies TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 So far a player chose deterministically among one of her strategies. Such selections are named selections from pure strategies. Randomizing one’s choice, by selecting among her pure strategies with a certain probability is called a mixed strategy. Equilibrium is now defined by a mixed strategy for each player so that none can gain on average by unilaterally deviating. Nash proved (1951) that under mixed strategies any game in strategic-form has an equilibrium Such is therefore the game theorists recommendation in the case when an equilibrium in pure strategies does not exist. So far a player chose deterministically among one of her strategies. Such selections are named selections from pure strategies. Randomizing one’s choice, by selecting among her pure strategies with a certain probability is called a mixed strategy. Equilibrium is now defined by a mixed strategy for each player so that none can gain on average by unilaterally deviating. Nash proved (1951) that under mixed strategies any game in strategic-form has an equilibrium Such is therefore the game theorists recommendation in the case when an equilibrium in pure strategies does not exist.

19. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6

20. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6 Mixed strategy for I: chose Inspect with a probability. (giving a sufficient change to getting caught should deter II from choosing Cheat) Mixed strategy for I: chose Inspect with a probability. (giving a sufficient change to getting caught should deter II from choosing Cheat) Inspect with p = 0.01 then II receives a utility of: 0 under Comply 0.99 x 10 + 0.01 x (-90) = 9 under Cheat (has incentive to Cheat –inspection not too often) Inspect with p = 0.2 then II receives a utility of: 0 under Comply 0.8 x 10 + 0.2 x (-90) = -10 under Cheat (incentive to always Comply –inspections too often) Inspect with p = 0.01 then II receives a utility of: 0 under Comply 0.99 x 10 + 0.01 x (-90) = 9 under Cheat (has incentive to Cheat –inspection not too often) Inspect with p = 0.2 then II receives a utility of: 0 under Comply 0.8 x 10 + 0.2 x (-90) = -10 under Cheat (incentive to always Comply –inspections too often)

21. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6 So if player I randomizes poorly she leads player II to selecting a pure strategy. Player I must make player II indifferent to achieve equilibrium So if player I randomizes poorly she leads player II to selecting a pure strategy. Player I must make player II indifferent to achieve equilibrium Inspect with p = 0.1 then II receives a utility of: 0 under Comply 0.9 x 10 + 0.1 x (-90) = 0 under Cheat Player II is rational so he knows player I will mix strategies only if I is indifferent too. How can II make I indifferent? Cheat with p = 0.2 then I receives a utility of: 0.8 x 0 + 0.2 x (-10) = -2 under Don’t Inspect 0.8 x (-1) + 0.2 x (-6) = -2 under Inspect Inspect with p = 0.1 then II receives a utility of: 0 under Comply 0.9 x 10 + 0.1 x (-90) = 0 under Cheat Player II is rational so he knows player I will mix strategies only if I is indifferent too. How can II make I indifferent? Cheat with p = 0.2 then I receives a utility of: 0.8 x 0 + 0.2 x (-10) = -2 under Don’t Inspect 0.8 x (-1) + 0.2 x (-6) = -2 under Inspect

22. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6 Equilibrium yields an expected payoff of 0 for player II -2 for player I … Equilibrium yields an expected payoff of 0 for player II -2 for player I … Does randomizing make sense for II ??? “ok, why don’t I just always chose comply since I got nothing to win anyways?” Let’s assume player II always chooses Comply under this rationale, then I needs not randomize, just purely chose Don’t Inspect so player II’s strategy becomes sub-optimal (irrational). With no incentive to select one strategy over the other a player can mix strategies and only thus is the equilibrium reached Does randomizing make sense for II ??? “ok, why don’t I just always chose comply since I got nothing to win anyways?” Let’s assume player II always chooses Comply under this rationale, then I needs not randomize, just purely chose Don’t Inspect so player II’s strategy becomes sub-optimal (irrational). With no incentive to select one strategy over the other a player can mix strategies and only thus is the equilibrium reached

23. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6 We saw that the probabilities for mixing one’s strategies depend on opponent's payoffs… It could be reasoned by I that : “We just have the penalty for Cheating disgustingly high and II will be deterred to select Cheat” We saw that II’s payoffs determine the probabilities that will make I be indifferent… It could be reasoned by I that : “We just have the penalty for Cheating disgustingly high and II will be deterred to select Cheat” We saw that II’s payoffs determine the probabilities that will make I be indifferent…

24. Compliance Inspections TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003 II I ComplyCheat Don’t Inspect 0000 10 -10 Inspect 0 -90 -6 One could see this game as an evolutionary game: It is the interaction between an organization who chooses Don’t Inspect and Inspect for certain fractions of a large number of people. One could see this game as an evolutionary game: It is the interaction between an organization who chooses Don’t Inspect and Inspect for certain fractions of a large number of people. Player II’s actions Comply and Cheat are each chosen by a certain fraction of people involved in these interactions. If these fractions would deviate from the equilibrium probabilities, the group whose strategies do better would increase. For example: if player I chooses Inspect too often (relative to the penalty for a cheater who is caught), the fraction of cheaters will decrease, which in turn makes Don’t Inspect a better strategy In this dynamic process, the long-term averages of the fractions approximate the equilibrium probabilities Player II’s actions Comply and Cheat are each chosen by a certain fraction of people involved in these interactions. If these fractions would deviate from the equilibrium probabilities, the group whose strategies do better would increase. For example: if player I chooses Inspect too often (relative to the penalty for a cheater who is caught), the fraction of cheaters will decrease, which in turn makes Don’t Inspect a better strategy In this dynamic process, the long-term averages of the fractions approximate the equilibrium probabilities

25. TNL - Mobile Computing Group Angelakis Vangelis 14/10/2003