Game Theory “Доверяй, Но Проверяй” (“Trust, but Verify”) - Russian Proverb (Ronald Reagan) Topic 5 Repeated Games

Repeated Interaction Review  Simultaneous games  Put yourself in your opponent’s shoes  Iterative reasoning  Sequential games  Look forward and reason back  Sequentially rational reasoning Outline:  What if interaction is repeated?  What strategies can lead players to cooperate? Mike Shor 2

The Prisoner’s Dilemma Mike Shor 3 Firm 2 LowHigh Firm 1 Low 54, 54 72, 47 High 47, 72 60, 60 Equilibrium: $54 K Cooperation: $60 K

Prisoner’s Dilemma Private rationality  collective irrationality  The equilibrium that arises from using dominant strategies is worse for every player than the outcome that would arise if every player used her dominated strategy instead Goal:  To sustain mutually beneficial cooperative outcome overcoming incentives to cheat (A note about tacit collusion) Mike Shor 4

Finite Interaction (Silly Theoretical Trickery) Suppose the market relationship lasts for only T periods Use backward induction (rollback) T th period: no incentive to cooperate  No future loss to worry about in last period T-1 th period: no incentive to cooperate  No cooperation in T th period in any case  No opportunity cost to cheating in period T-1 Unraveling: logic goes back to period 1 Mike Shor 5

Finite Interaction Cooperation is impossible if the relationship between players is for a fixed and known length of time. But, people think forward if …  Game length uncertain  Game length unknown  Game length too long Mike Shor 6

Long-Term Interaction No last period, so no rollback Use history-dependent strategies Trigger strategies:  Begin by cooperating  Cooperate as long as the rivals do  Upon observing a defection: immediately revert to a period of punishment of specified length in which everyone plays non- cooperatively Mike Shor 7

Two Trigger Strategies Grim trigger strategy  Cooperate until a rival deviates  Once a deviation occurs, play non-cooperatively for the rest of the game Tit-for-tat  Cooperate if your rival cooperated in the most recent period  Cheat if your rival cheated in the most recent period Mike Shor 8

Trigger Strategy Extremes Mike Shor 9 Tit-for-Tat is  most forgiving  shortest memory  proportional  credible but lacks deterrence Tit-for-tat answers: “Is cooperation easy?” Grim trigger is  least forgiving  longest memory  MAD  adequate deterrence but lacks credibility Grim trigger answers: “Is cooperation possible?”

Why Cooperate (Against GTS)? Cooperate if the present value of cooperation is greater than the present value of defection Cooperate:60 today, 60 next year, 60 … 60 Defect:72 today, 54 next year, 54 … 54 Mike Shor 10 LowHigh Firm 1 Low 54, 54 72, 47 High 47, 72 60, 60 Firm 2

Payoff Stream (GTS) Mike Shor tt+1t+2t+3 defect time profit 60 cooperate

Discounting Discounting:  $1 tomorrow is worth less than $1 today r is the interest rate  Invest $1 today  get $(1+r) next year  Want $1 next year  invest $1/(1+r) today  Annuity paying $1 today and $1 every year is worth $1+1/r Mike Shor

Aside: Infinite Discounting Why? Mike Shor 13

Cooperate against GTS Cooperate if Cooperation is sustainable using grim trigger strategies as long as r < 50% Or … as long as $1 invested today does not return more than $1.50 next period Mike Shor 14 PV(defection) 72…54…54…54… /r 12 6/12 = 50% PV(cooperation) 60…60…60…60… /r 6/r r >>>><>>>><

Payoff Stream (TFT) Mike Shor tt+1t+2t+3time profit 60 cooperate defect once defect 54

Cooperate against TFT Cooperate if Much harder to sustain than grim trigger Cooperation may not be likely Mike Shor 16 PV(defection) PV(defect once) 72…47…60…60… /(1+r) r 1/12 = 8.3% PV(cooperation) 60…60…60…60… /(1+r) 13/(1+r) 13 r > and > <

Trigger Strategies Grim Trigger and Tit-for-Tat are extremes Balance two goals: Deterrence  GTS is adequate punishment  Tit-for-tat might be too little Credibility  GTS hurts the punisher too much  Tit-for-tat is credible Mike Shor 17

Axelrod’s Simulation R. Axelrod, The Evolution of Cooperation Prisoner’s Dilemma repeated 200 times Economists submitted strategies Pairs of strategies competed Winner: Tit-for-Tat Reasons:  Forgiving, Nice, Provocable, Clear Mike Shor 18

Main Ideas from Axelrod Not necessarily tit-for-tat  Doesn’t always work Don’t be envious Don’t be the first to cheat Reciprocate opponent’s behavior  Cooperation and defection Don’t be too clever Mike Shor 19

Finite Interaction (Theoretical Aside) Unraveling prevents cooperation if the number of periods is fixed and known Probabilistic termination  The “game” continues to the next period with some probability p: Equivalent to infinite game  $1 next year is worth now  Value of future ={ value if there is a future }  { probability of a future }  Effective interest rate: Mike Shor 20

Subgame Perfection Grim trigger strategy is subgame perfect Tit for tat is not  Consider, in the first stage, one player cooperates and the other does not  Tit for tat would have them alternating forever Mike Shor 21

Folk Theorem We saw in the prisoner’s dilemma that  Always cooperate can be an equilibrium  Sufficient punishment, low interest rate  Always defect can be an equilibrium  Insufficient punishment, high interest rate Mike Shor 22

Folk Theorem  Almost any outcome that, on average, yields at least the mutual defection payoff for both players, can be sustained as a subgame perfect equilibrium of the infinitely repeated prisoner’s dilemma (for sufficiently low interest rates) Mike Shor 23

The Prisoner’s Dilemma Mike Shor 24 Firm 2 LowHigh Firm 1 Low 54, 54 72, 47 High 47, 72 60, 60 Equilibrium: $54 K Cooperation: $60 K

Folk Theorem Mike Shor  72  6047 Firm 1 Firm All feasible payoffs All subgame perfect payoffs

Summary Cooperation  Struggle between high profits today and a lasting relationship into the future Deterrence  A clear, provocable policy of punishment Credibility  Must incorporate forgiveness Looking ahead:  How to be credible? Mike Shor 26