Unit III: The Evolution of Cooperation Can Selfishness Save the Environment? Repeated Games: the Folk Theorem Evolutionary Games A Tournament How to Promote Cooperation 3/30

Can Selfishness Save the Environment? The Problem of Cooperation The Tragedy of the Global Commons? Common Resource Game We Play a Game Repeated Games Discounting The Folk Theorem

How can a number of individuals, each behaving as a utility maximizer, come to behave as a group and maximize joint utility? The Problem of Cooperation

C 6,6 0,5 D 5,0 1,1 The Problem of Cooperation C D Players may fail to cooperate (i.e., fail to maximize joint payoffs), because they lack information. If each has reason to believe the other will cooperate, the problem is solved! Assurance Game Prisoner’s Dilemma

C 6,6 0,5 D 5,0 1,1 The Problem of Cooperation C D C 3,3 0,5 D 5,0 1,1 C D Assurance Game Prisoner’s Dilemma

C 6,6 0,5 D 5,0 1,1 The Problem of Cooperation C D C 3,3 0,5 D 5,0 1,1 C D Easy In the Prisoner’s Dilemma, there is no belief that will lead the players to cooperate. Rather than a problem of information, this is a problem of incentives. Cooperation is both inaccessible and unstable. Prisoner’s Dilemma

The problem of cooperation arises in several important contexts, including public goods: everyone can enjoy the good even if they don’t pay for it, e.g., nat’l defense, public tv. common (property) resources: raw or natural resources that are own by everyone (or no one), e.g., clean air, clean water, biodiversity. Can Selfishness Save the Environment? Problem of Cooperation – subject to the Free-rider problem – undersupplied by a voluntary contribution scheme – subject to the “Tragedy of the Commons” (Hardin, 1968) – overconsumed (depleted)

Can Selfishness Save the Environment? Arguments to the effect that “polluting is wrong” are less likely to be effective than measures that get the incentives right over the long run (Ridley & Low, 1993). Our Common Future: Transboundary pollution, ozone depletion, nuclear proliferation, global warming, loss of biodiversity, deforestation, overfishing are all the consequences of continuing economic growth and development (Brundtland, 1987). Negative externalities Tragedy of the Global Commons?

Consider a country deciding on its optimal level of economic growth (X), in the presence of a negative externality (e.g., transboundary pollution, etc.). National Utility is a positive function of own growth and negative function of overall growth (X,X’): National Utility Own Choice of X All Choices of X P(X,X’) = a(X) – b(X,X’) + c Alternatively: X can be the voluntary contribution level in the case of a public good (bad); or the consumption level in the case of a common resource.

Common Resource Game Two fishermen fish from a single lake. Each year, there are a fixed number of fish in the lake and two periods during the year that they can be harvested, spring and fall. Each fisherman consumes all the fish he catches each period, and their identical preferences are described by the following consumption function: U i = C s C f where C s = spring catch; C f = fall catch. Each spring, each fisherman decides how many fish to remove from the lake. In the fall, the remaining fish are equally divided between the two.

Common Resource Game Consider two fishermen deciding how many fish to remove from a commonly owned pond. There are Y fish in the pond. Period 1 each fishery chooses to consume (c 1, c 2 ). Period 2remaining fish are equally divided (Y – (c 1 +c 2 ))/2). c 1 = (Y – c 2 )/2 Y/3c 1 c 2 Y/3 c 2 = (Y – c 1 )/2 U i = c t c t+1 C t = Today’s consumption C t+1 = Tomorrow’s ‘’

Common Resource Game Consider two fishermen deciding how many fish to remove from a commonly owned pond. There are Y fish in the pond. Period 1 each fishery chooses to consume (c 1, c 2 ). Period 2remaining fish are equally divided (Y – (c 1 +c 2 ))/2). c 1 = (Y – c 2 )/2 Y/4Y/3 c 1 c 2 Y/3 Y/4 c 2 = (Y – c 1 )/2 Social Optimality: c 1 = c 2 = Y/4

Common Resource Game Consider two fishermen deciding how many fish to remove from a commonly owned pond. There are Y fish in the pond. Period 1 each fishery chooses to consume (c 1, c 2 ). Period 2remaining fish are equally divided (Y – (c 1 +c 2 ))/2). c 1 = (Y – c 2 )/2 Y/4Y/3 c 1 c 2 Y/3 Y/4 c 2 = (Y – c 1 )/2 If there are 12 fish in the pond, each will consume (Y/3) 4 in the spring and 2 in the fall in a NE. Both would be better off consuming (Y/4) 3 in the fall, leaving 3 for each in the spring.

C 9, 9 7.5,10 D 10,7.5 8, 8 Common Resource Game C D A Prisoner’s Dilemma What would happen if the game were repeated? C = 3 in the spring D = 4 “ “

We Play a Game At each round of the game, you will have the chance to contribute to a public good (e.g., national defense; public tv). The game is repeated for several rounds, and payoffs are calculated as follows: 1 pt. for each contribution made by anyone. + 3 pts. for each round you don’t contribute. See Holt and Laury, JEP 1997:

Payoffs = 1 pt. for each contribution by anyone; 10 pts. for each round you don’. You play: Contribution Rate (n-1) 0%10 … 50 … 90100% Contribute Don’t Assume n = 30 We Play a Game Payoff: 1 pt. for each contribution made by anyone. + 3 pts. for each round you don’t contribute. n-person Prisoner’s Dilemma: Don’t Contribute is a dominant strategy. But if none Contribute, the outcome is inefficient.

We Play a Game Public Goods Games Round No. of Contributions Contributions RoundContributionsRoundContributions … Data from N = 20. Communication was allowed between rounds 7 and 8.

We Play a Game Public Goods Games Typically, contribution rates: 40-60% in one-shot games & first round of repeated games <30% on announced final round Decrease with group size Increase with “learning”

Repeated Games Examples of Repeated Prisoner’s Dilemma Overfishing Transboundary pollution Cartel enforcement Labor union Public goods The Tragedy of the Global Commons Free-rider Problems

Repeated Games Some Questions: What happens when a game is repeated? Can threats and promises about the future influence behavior in the present? Cheap talk Finitely repeated games: Backward induction Indefinitely repeated games: Trigger strategies

The Evolution of Cooperation Under what conditions will cooperation emerge in world of egoists without central authority? Axelrod uses an experimental method – the indefinitely repeated PD tournament – to investigate a series of questions: Can a cooperative strategy gain a foothold in a population of rational egoists? Can it survive better than its uncooperative rivals? Can it resist invasion and eventually dominate the system?

The Evolution of Cooperation The Indefinitely Repeated Prisoner’s Dilemma Tournament Axelrod (1980a,b, Journal of Conflict Resolution). A group of scholars were invited to design strategies to play indefinitely repeated prisoner’s dilemmas in a round robin tournament. Contestants submitted computer programs that select an action, Cooperate or Defect, in each round of the game, and each entry was matched against every other, itself, and a control, RANDOM.

The Indefinitely Repeated Prisoner’s Dilemma Tournament Axelrod (1980a,b, Journal of Conflict Resolution). Contestants did not know the length of the games. (The first tournament lasted 200 rounds; the second varied probabilistically with an average of 151.) The first tournament had 14 entrants, including game theorists, mathematicians, psychologists, political scientists, and others. Results were published and new entrants solicited. The second tournament included 62 entrants... The Evolution of Cooperation

The Indefinitely Repeated Prisoner’s Dilemma Tournament TIT FOR TAT won both tournaments! TFT cooperates in the first round, and then does whatever the opponent did in the previous round. TFT “was the simplest of all submitted programs and it turned out to be the best!” (31). TFT was submitted by Anatol Rapoport to both tournaments, even after contestants could learn from the results of the first. The Evolution of Cooperation

The Indefinitely Repeated Prisoner’s Dilemma Tournament TIT FOR TAT won both tournaments! In addition, Axelrod provides a “theory of cooperation” based on his analysis of the repeated prisoner’s dilemma game. In particular, if the “shadow of the future” looms large, then players may have an incentive to cooperate. A cooperative strategy such as TFT is “collectively stable.” He also offers an evolutionary argument, i.e., TFT wins in an evolutionary competition in which payoffs play the role of reproductive rates. The Evolution of Cooperation

The Indefinitely Repeated Prisoner’s Dilemma Tournament This result has been so influential that “some authors use TIT FOR TAT as though it were a synonym for a self-enforcing, cooperative agreement” (Binmore, 1992, p. 433). And many have taken these results to have shown that TFT is the “best way to play” in IRPD.  While TFT won these, will it win every tournament?  Is showing that TFT is collectively stable equivalent to predicting a winner in the computer tournaments?  Is TFT evolutionarily stable? The Evolution of Cooperation

Class Tournament Imagine a population of strategies matched in pairs to play repeated PD, where outcomes determine the number of offspring each leaves to the next generation. –In each generation, each strategy is matched against every other, itself, and RANDOM. –Between generations, the strategies reproduce, where the chance of successful reproduction (“fitness”) is determined by the payoffs (i.e., payoffs play the role of reproductive rates). Then, strategies that do better than average will grow as a share of the population and those that do worse than average will eventually die-out...

Repeated Games Some Questions: What happens when a game is repeated? Can threats and promises about the future influence behavior in the present? Cheap talk Finitely repeated games: Backward induction Indefinitely repeated games: Trigger strategies

Can threats and promises about future actions influence behavior in the present? Consider the following game, played 2X: C 3,3 0,5 D 5,0 1,1 Repeated Games C D See Gibbons:

Repeated Games Draw the extensive form game: (3,3) (0,5)(5,0) (1,1) (6,6) (3,8) (8,3) (4,4) (3,8)(0,10)(5,5)(1,6)(8,3) (5,5)(10,0) (6,1) (4,4) (1,6) (6,1) (2,2)

Repeated Games Now, consider three repeated game strategies: D (ALWAYS DEFECT): Defect on every move. C(ALWAYS COOPERATE):Cooperate on every move. T(TRIGGER): Cooperate on the first move, then cooperate after the other cooperates. If the other defects, then defect forever.

Repeated Games If the game is played twice, the V(alue) to a player using ALWAYS DEFECT (D) against an opponent using ALWAYS DEFECT(D) is: V (D/D) = = 2, and so on... V (C/C) =3 + 3 =6 V (T/T)=3 + 3 = 6 V (D/C)=5 + 5 =10 V (D/T)=5 + 1 = 6 V (C/D)=0 + 0 =0 V (C/T)=3 + 3 =6 V (T/D)=0 + 1 =1 V (T/C)=3 + 3 =6

Repeated Games And 3x: V (D/D) = = 3 V (C/C) = = 9 V (T/T)= = 9 V (D/C)= =15 V (D/T)= = 7 V (C/D)= =0 V (C/T)= = 9 V (T/D)= =2 V (T/C)= = 9

Repeated Games Time average payoffs: n=3 V (D/D) = = 3 /3= 1 V (C/C) = = 9/3= 3 V (T/T)= = 9/3= 3 V (D/C)= =15/3= 5 V (D/T)= = 7/3= 7/3 V (C/D)= =0/3= 0 V (C/T)= = 9/3= 3 V (T/D)= =2/3 = 2/3 V (T/C)= = 9/3= 3

Repeated Games Time average payoffs: n V (D/D) = /n= 1 V (C/C) = /n= 3 V (T/T)= /n= 3 V (D/C)= /n= 5 V (D/T)= /n= 1 +  V (C/D)= /n= 0 V (C/T)= … /n= 3 V (T/D)= /n = 1 -  V (T/C)= /n= 3

Repeated Games Now draw the matrix form of this game: 1x T3,3 0,5 3,3 C 3,3 0,53,3 D 5,0 1,15,0 C D T

Repeated Games T 3,3 1-  1+  3,3 C 3,3 0,5 3,3 D 5,0 1,1 1+ ,1-  C D T If the game is repeated, ALWAYS DEFECT is no longer dominant. Time Average Payoffs

Repeated Games T 3,3 1-  1+  3,3 C 3,3 0,5 3,3 D 5,0 1,1 1+ ,1-  C D T … and TRIGGER achieves “a NE with itself.”

Repeated Games Time Average Payoffs T(emptation)> R(eward)> P(unishment)> S(ucker) T R,R P-  P +  R,R C R,R S,T R,R D T,S P,P P + , P -  C D T

Discounting The discount parameter, , is the weight of the next payoff relative to the current payoff. In a indefinitely repeated game,  can also be interpreted as the likelihood of the game continuing for another round (so that the expected number of moves per game is 1/(1-  )). The V(alue) to someone using ALWAYS DEFECT (D) when playing with someone using TRIGGER (T) is the sum of T for the first move,  P for the second,  2 P for the third, and so on (Axelrod: 13-4): V (D/T) = T +  P +  2 P + … “The Shadow of the Future”

Discounting Writing this as V (D/T) = T +  P +   2 P +..., we have the following: V (D/D) = P +  P +  2 P + … = P/(1-  ) V (C/C) =R +  R +  2 R + … = R/(1-  ) V (T/T)=R +  R +  2 R + … = R/(1-  ) V (D/C)=T +  T +  2 T + … = T/(1-  ) V (D/T)=T +  P +  2 P + … = T+  P/(1-  ) V (C/D)=S +  S +  2 S + … = S/(1-  ) V (C/T)=R +  R +  2 R + … = R/(1-  ) V (T/D)=S +  P +  2 P + … = S+  P/(1-  ) V (T/C)=R +  R +  2 R + … = R/(1-  )

T C D Discounted Payoffs T > R > P > S 0 >  > 1 R /(1-  ) S /(1-  ) R /(1-  ) R /(1-  ) T /(1-  ) R /(1-  ) T /(1-  ) P /(1-  ) T +  P /(1-  ) S /(1-  ) P /(1-  ) S +  P /(1-  ) Discounting C D T R /(1-  ) S +  P /(1-  ) R /(1-  ) R /(1-  ) T +  P /(1-  ) R /(1-  )

T C D Discounted Payoffs T > R > P > S 0 >  > 1 T weakly dominates C R /(1-  ) S /(1-  ) R /(1-  ) R /(1-  ) T /(1-  ) R /(1-  ) T /(1-  ) P /(1-  ) T +  P /(1-  ) S /(1-  ) P /(1-  ) S +  P /(1-  ) Discounting C D T R /(1-  ) S +  P /(1-  ) R /(1-  ) R /(1-  ) T +  P /(1-  ) R /(1-  )

Discounting Now consider what happens to these values as  varies (from 0-1): V (D/D) = P +  P +  2 P + … = P/(1-  ) V (C/C) =R +  R +  2 R + … = R/(1-  ) V (T/T)=R +  R +  2 R + … = R/(1-  ) V (D/C)=T +  T +  2 T + … = T/(1-  ) V (D/T)=T +  P +  2 P + … = T+  P/(1-  ) V (C/D)=S +  S +  2 S + … = S/(1-  ) V (C/T)=R +  R +  2 R + … = R/(1-  ) V (T/D)=S +  P +  2 P + … = S+  P/(1-  ) V (T/C)=R +  R +  2 R + … = R/(1-  )

Discounting Now consider what happens to these values as  varies (from 0-1): V (D/D) = P +  P +  2 P + … = P/(1-  ) V (C/C) =R +  R +  2 R + … = R/(1-  ) V (T/T)=R +  R +  2 R + … = R/(1-  ) V (D/C)=T +  T +  2 T + … = T/(1-  ) V (D/T)=T +  P +  2 P + … = T+  P/(1-  ) V (C/D)=S +  S +  2 S + … = S/(1-  ) V (C/T)=R +  R +  2 R + … = R/(1-  ) V (T/D)=S +  P +  2 P + … = S+  P/(1-  ) V (T/C)=R +  R +  2 R + … = R/(1-  )

Discounting Now consider what happens to these values as  varies (from 0-1): V (D/D) = P +  P +  2 P + … = P+  P/(1-  ) V (C/C) =R +  R +  2 R + … = R/(1-  ) V (T/T)=R +  R +  2 R + … = R/(1-  ) V (D/C)=T +  T +  2 T + … = T/(1-  ) V (D/T)=T +  P +  2 P + … = T+  P/(1-  ) V (C/D)=S +  S +  2 S + … = S/(1-  ) V (C /T) = R +  R +  2 R + … = R/(1-  ) V (T/D)=S +  P +  2 P + … = S+  P/(1-  ) V (T/C)=R +  R +  2 R + … = R/(1-  ) V(D/D) > V(T/D) D is a best response to D

Discounting Now consider what happens to these values as  varies (from 0-1): V (D/D) = P +  P +  2 P + … = P+  P/(1-  ) V (C/C) =R +  R +  2 R + … = R/(1-  ) V (T/T)=R +  R +  2 R + … = R/(1-  ) V (D/C)=T +  T +  2 T + … = T/(1-  ) V (D/T)=T +  P +  2 P + … = T+  P/(1-  ) V (C/D)=S +  S +  2 S + … = S/(1-  ) V (C/T)=R +  R +  2 R + … = R/(1-  ) V (T/D)=S +  P +  2 P + … = S+  P/(1-  ) V (T/C)=R +  R +  2 R + … = R/(1-  ) ?

Discounting Now consider what happens to these values as  varies (from 0-1): For all values of  : V(D/T) > V(D/D) > V(T/D) V(T/T) > V(D/D) > V(T/D) Is there a value of  s.t., V(D/T) = V(T/T)? Call this  *. If  <  *, the following ordering hold: V(D/T) > V(T/T) > V(D/D) > V(T/D) D is dominant: GAME SOLVED V(D/T) = V(T/T) T+  P(1-  ) = R/(1-  ) T-  t+  P = R T-R =  (T-P)   * = (T-R)/(T-P) ?

Discounting Now consider what happens to these values as  varies (from 0-1): For all values of  : V(D/T) > V(D/D) > V(T/D) V(T/T) > V(D/D) > V(T/D) Is there a value of  s.t., V(D/T) = V(T/T)? Call this  *.  * = (T-R)/(T-P) If  >  *, the following ordering hold: V(T/T) > V(D/T) > V(D/D) > V(T/D) D is a best response to D; T is a best response to T; multiple NE.

Discounting V(T/T) = R/(1-  )  * 1 V TRV TR Graphically: The V(alue) to a player using ALWAYS DEFECT (D) against TRIGGER (T), and the V(T/T) as a function of the discount parameter (  ) V(D/T) = T +  P/(1-  )

The Folk Theorem (R,R) (T,S) (S,T) (P,P) The payoff set of the repeated PD is the convex closure of the points [( T,S ); ( R,R ); ( S,T ); ( P,P )].

The Folk Theorem (R,R) (T,S) (S,T) (P,P) The shaded area is the set of payoffs that Pareto-dominate the one-shot NE ( P,P ).

The Folk Theorem (R,R) (T,S) (S,T) (P,P) Theorem: Any payoff that pareto- dominates the one-shot NE can be supported in a SPNE of the repeated game, if the discount parameter is sufficiently high.

The Folk Theorem (R,R) (T,S) (S,T) (P,P) In other words, in the repeated game, if the future matters “enough” i.e., (  >  * ), there are zillions of equilibria!

The theorem tells us that in general, repeated games give rise to a very large set of Nash equilibria. In the repeated PD, these are pareto-rankable, i.e., some are efficient and some are not. In this context, evolution can be seen as a process that selects for repeated game strategies with efficient payoffs. “Survival of the Fittest” The Folk Theorem

Next Time 4/6 Evolutionary Games. Axelrod, Ch. 5: Gintis Chs. 7, 9: ;