Game Theory and Cooperation Jeff Schank UC Davis

Origin of Game Theory John von Neumann and Oskar Morgenstern (1944) Theory of Games and Economic Behavior, Princeton University Press. John F. Nash, Jr. (1950) Equilibrium Points in N-Person Games. PNAS, 36: 48–49. John Maynard Smith and George R. Price (1973) The logic of animal conflict. Nature, 246: 15–8.

What is a Game? A game involves at least two players, not necessarily human For example: Player 1 = a person, Player 2 = a stock market; Player 1 = an animal, Player 2 = an animal (possibly from different species) Player 1 = a firm, Player 2 = consumers Each player has strategies whose payoff depends on the strategy used by (an)other player(s) Payoffs can be in terms of money, resources, happiness, or more generally what economists call utility It can be very difficult to precisely calculate the payoffs for a strategy The best cases for calculating payoffs precisely involve money Although calculating payoffs for strategies are often difficult or effectively impossible, if we can calculate or estimate payoffs, then the theory of games can help us find the best or at least good strategies

Prisoner’s Dilemma

–1, –1 –1, -1 –5, –.5 –.5, –5 –3, –3

Normal Form of the Prisoner’s Dilemma Don’t Confess (cooperate) Confess (defect) –1, –1 –5, –.5 –.5, –5 –3, –3

Explicit Assumptions There are choices of strategies available to both players The success of a chosen strategy depends on the choice of strategy by an opponent Players behave rationally: each player chooses the strategy that maximizes their individual payoff

Nash Equilibrium A pure-strategy Nash equilibrium is a choice of strategies by all players such that no player can unilaterally improve his/her payoff by changing strategies Calculation: Assume: Strategies are fixed Question: If each player is given the strategy of the other, can that player improve his/her payoff by switching? Solution: If so, then the strategy set is not a Nash Equilibrium, otherwise it is

Normal Form of the Prisoner’s Dilemma Don’t Confess (cooperate) Confess (defect) –1, –1 –5, –.5 –.5, –5 –3, –3 Suppose Player 1 and 2 both choose “Don’t Confess” Player 1 knows that player 2 didn’t confess and sees that she can improve her payoff by changing from don’t confess to confess Player 2 know that player 1 didn’t confess and sees that he can improve his payoff by changing from don’t confess to confess Thus, [Don’t Confess, Don’t Confess] is not a Nash equilibrium The same reasoning leads to the conclusion that [Confess, Confess] is a Nash equilibrium

Stag Hunt Game

Normal Form of the Stag Hunt Game Cooperate Defect 2, 2 0, 1 1, 0 1, 1 Suppose Player 1 and 2 both choose “Cooperate” Player 1 knows that player 2 cooperated and sees that she cannot improve her payoff by changing from cooperate to defect Player 2 knows that player 1 cooperated and sees that he cannot improve his payoff by changing from cooperate to defect Thus, [Cooperate, Cooperate] is a Nash equilibrium Is [Defect, Defect] a Nash equilibrium? Yes it is, so the problem is if individuals do not initially cooperate, how can cooperation evolve?

Hawk-Dove Game

Normal Form of the Hawk-Dove Game –2, –2 4, 0 0, 4 1, 1 Suppose Player 1 and 2 both choose “Hawk” Player 1 knows that player 2 played Hawk and sees that she can improve her payoff by changing from Hawk to Dove Player 2 knows that player 1 played Hawk and sees that he can improve his payoff by changing from Hawk to Dove Thus, [Hawk, Hawk] is not a Nash equilibrium Is [Dove, Dove] a Nash equilibrium? Yes, player 1 know that player 2 played Dove and sees that she can improve her payoff by changing from Dove to Hawk. The same logic applies to player 2. Thus, [Dove, Dove] is not a Nash equilibrium. Is there a Nash solution to this game?

Normal Form of the Hawk-Dove Game (V–C)/2, (V–C)/2 V, 0 0, V V/2, V/2 Suppose Player 1 and 2 both choose “Hawk” Player 1 knows that player 2 played Hawk and sees that she can improve her payoff by changing from Hawk to Dove if C > V Player 2 knows that player 1 played Hawk and sees that he can improve his payoff by changing from Hawk to Dove if C > V Thus, [Hawk, Hawk] is not a Nash equilibrium Is [Dove, Dove] a Nash equilibrium?

Calculating a Mixed Strategy for the Hawk-Dove Game

Calculating a Mixed Strategy for the Hawk-Dove Game

Different Games Model Different Aspects of Cooperation Prisoner’s Dilemma: Models the problem of defecting/cheating in cooperative behavior Stag Hunt: Models the problem of coordinating behavior to achieve cooperation Hawk-Dove: Models the problem of competing for vs. cooperating/sharing resources

Repeated Games What happens if such games are played repeatedly? What if a population of individuals played games such as the Prisoner’s Dilemma, Stag Hunt, or Hawk-Dove games, would they cooperate? These are more difficult questions to answer

Evolutionary Game Theory We can answer these questions in evolutionary contexts (biological or cultural) by asking whether strategy I (e.g., cooperate or dove) can be invaded by strategy J (e.g., defect or hawk) in small numbers (e.g., by mutation or strategy switching)? If not, strategy I is evolutionarily stable

Maynard-Smith & Price (1973) They introduced two conditions for a strategy to be evolutionarily stable (ESS) The first is the strict Nash equilibrium condition: E(I, I) > E(J, I), which means that the expected payoff for strategy I against itself is greater than the expected payoff of any strategy J ≠ I against I Or if E(I, I) = E(J, I)  E(I , J) > E(J, J), which means that if J does as well against I as I against itself, then I much do better against J than J does against itself for J ≠ I

In General, Cooperation is Not an ESS Assuming that any player in a population can play any other player (random assortment assumption, like in Kalick and Hamilton’s Mate Choice model) Cooperation in the Prisoner’s Dilemma is not an ESS Cooperation in the Stag Hunt game may be an ESS depending on initial conditions and/or on the size of the cooperative payoff In the Hawk-Dove Game, a mixed ESS only exists if the cost of fighting is greater than the reward (i.e. C > V)

Implicit Assumptions Infinite populations Disembodiment assumptions: No spatial structure No movement strategies No individual characteristics such as personality

Prisoner’s Dilemma Next consider the Prisoner’s dilemma with a similar agent-based model The well-known solution is to defect We allow agents to play with each other in space

What happens if we make the Sucker’s payoff (S) worse?

The Worse it is, the Better it is for Cooperation Smaldino PE, Schank JC, McElreath R (2013) Increased costs of cooperation help cooperators in the long run. The American Naturalist, 181(4), 451–463.

The Worse it is, the Better it is for Cooperation Smaldino PE, Schank JC, McElreath R (2013) Increased costs of cooperation help cooperators in the long run. The American Naturalist, 181(4), 451–463.

What if we allow agents to move in space?

Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

Hypothetical Interpretation Behavioral Syndromes (similar to personality): Behavioral correlations across contexts For example, an animal may be bold when foraging, predators are presence, and in mating contexts Or an animal may be shy when foraging, predators are presence, and in mating contexts Random movement strategies can represent behavioral syndromes For example, SP and ZZ could be interpreted as bold behavior strategies Whereas, CY, CH, and TC are shy behavioral strategies

Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

Conclusions Game theory is a powerful theoretical tool for understanding cooperation and the conditions under which it can occur Game theory, however, makes an assumption, in the context of cooperation, that can limit its application: Players are disembodied By using agent-based models, we can investigate embodied agents and discover that in many cases, stable game-theoretic solutions depend on embodiment and context