1. 1 Perfect Correlated Equilibria in Stopping Games Yuval Heller Tel-Aviv University (Part of my Ph.D. thesis supervised by Eilon Solan) http://www.tau.ac.il/~helleryu/ 3 rd Israeli Game Theory Conference December 2008

2. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Summary 2 Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction

3. 3 Stopping Games (Undiscounted, Multi-player, Discrete time)  Finite set of players: I  Unknown state variable:     (state space) Filtration: F =( F n  )  At each stage n the players receive a symmetric partial information about the state : F n (  )

4. 4 Stopping Games (undiscounted, multi-player, discrete time)  Stage 1 - everyone is active  Stage n: All active players simultaneously declare whether they stop or continue A player that stops become passive for the rest of the game Player’s payoff depends on the history of players’ actions while he has been active and on the state variable

5. 5 Literature: 2-player zero-sum Stopping Games  Dynkin (1969) – introduction, value where simultaneous stops are not allowed  Neveu (1975) – value when each player prefers the other to stop  Rosenberg, Solan & Vieille (2001) – use of randomized strategies, value with payoffs’ integrability

6. 6 Literature: 2-player non-zerosum Stopping Games  Existence of approximate Nash equilibrium when the payoffs have a special structure: Morimoto (86), Mamer (87), Ohtsubo (87, 91), Nowak & Szajowski (99), Neumann, Ramsey & Szajowski (02)  Recently, Shmaya & Solan (04) proved existence assuming only integrability  Multi-player stopping games: no existence results

7. 7 Stopping Games - Applications  Most applications in the literature: Payoffs: Specific assumptions, such as monotony Discount factor 2 players Multi-player variations are natural

8. 8 Struggle of survival in a declining market  At each turn, each firm loses money  A firm can stay or exit the market for good Partial production is inefficient Market is more profitable with less firms  Which firms survive? What is the exit order?  Ghemawat & Nalebuff (1985)...  Steel market in 70’s and 80’s

9. 9 Research & Development  Race for developing a patent  At each turn, continue spending money on research or leave the race  The first firm to complete the patent earns a lot Stochastic function of spent money  Fudenberg & Tirole (1985)…

10. 10 War of attrition  Attrition wars among animals: Becoming the leader (alpha-male) Territory Maynard-Smith (1982), Nalebuff & Riley (1985)…  2 nd price auctions where all bidders pay Krishna & Morgan (1997)….  Political Sciences – lobbying Bulow & Klemperer (2001)

11. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Summary 11 Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction

12. 12 Perfect Equilibrium  Nash equilibrium may be sustained by non-credible threats of punishment Punisher receives a low payoff  The stronger concept of perfect equilibrium (Selten, 1965, 1975) has been studied. Examples: Fine & Li (1989): uniqueness in discounted 2-player games with monotone payoffs Mashiah-Yaakovi (2008) – existence of (  )-perfect equilibrium when simultaneous stops aren’t allowed

13. 13 Correlated Equilibrium  Aumann (1974): An equilibrium in an extended game with a correlation device Device D sends each player i a private signal m i  M i (M=  i M i ) before the game starts according to    (M) The extended game G(D)  Consistent with Bayesian decision making (Aumann, 87)  Other appealing properties: computability, linear equations, closed and convex set

14. 14 Correlated Equilibrium in Sequential Games  Two main versions: Normal-form: signals are sent only before the game starts Extensive-form: signals are sent at each stage  Equilibrium: normal-form  extensive-form  Correlation among players is natural in many setups: Countries negotiate actions Firms choose strategies based on market’s history A manager coordinates the actions of his workers

15. 15 Normal-Form Correlation (1)  Sometimes players may coordinate before play starts but coordination along the play is costly / impossible:  Example (1) - war of attrition in nature: Commonly modeled as stopping games Coordination before play starts is implemented by evolution of phenotype roles E.g.: Shmida & Peleg, 1997

16. 16 Normal-Form Correlation (2)  Example (2) - News playing among day traders: Monthly employment report will be published at noon Several minutes elapse before market adjusts New information gradually arrives during that time Quick trading can be profitable See e.g., Christie-David, Chaudhry & Khan (2002) Traders of a firm can coordinate their actions in advance Coordination along the play is costly (time limit) Traders may have different payoffs

17. 17 (  )-Perfect Correlated Equilibrium   – A bound for the probability of: An event E   Correlation device sends a signal in M’  M    >0 – A bound for the maximal profit a player can earn by deviating at any stage and after any history, conditioned on that    E and m  M’  Extending the definitions for finite games: Myerson (1986), Dhillon & Mertens (1996)

18. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Summary 18 Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction

19. 19 Main Result  For every    >0, a multi-player stopping game admits a normal-form uniform perfect correlated (   )-equilibrium with a universal correlation device Uniform: An approximate equilibrium in any long enough finite game and in any discounted game with high enough discount factor Universal device – doesn’t depend on game payoffs Corollary: Uniform perfect correlated equilibrium payoff

20. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Reductions Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 20 Summary

21. 1.Terminating games: game terminates at the first stop 2.Tree-like games (Shmaya & Solan, 03) : for every n, F n is finite A finite collection of matrix payoffs 3.Deep enough in the tree: with high probability any matrix payoff either: Repeats infinitely often Never occurs 21 Reductions

22. 22 Reductions  Reductions require 2 properties from the equilibrium ( ,  -unrevealing - expected payoff of each player “almost” doesn’t change With probability of at-least 1- , changes by less than  Universal - The correlation device D(G, ,  ) depends only on |I| and    D(G, ,  )=D(|I|,  )

23. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 23 Summary Reductions

24. 24 Games on Finite Trees  Equivalent to an absorbing game: A stochastic game with a single non-absorbing state. 2 special properties: Recursive game – Payoff in non-absorbing states is 0 Single non-absorbing action profile

25. 25 Games on Finite Trees  An adaptation of a result of Solan & Vohra (2002):  A game on a finite tree has one of the following: 1.Non-absorbing equilibrium (game never stops) 2.Stationary absorbing equilibrium. Adaptations: Perfection Limit minimal per-round terminating probability 3.A special distribution: allows to construct a correlated  -equilibrium. Adaptations: unrevealing, universal device

26. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 26 Summary Reductions

27. 27 Ramsey Theorem (1930)  A finite set of colors  Each 2 integers (k,n) are colored by c(k,n)  There is an infinite sequence of integers k 1 <k 2 <k 3 <… such that: c(k 1,k 2 ) =c(k i,k j ) for all i<j 0123456789101112 k1k1

28. 28 Stochastic Variation of Ramsey Theorem (Shmaya & Solan, 04)  Coloring each finite sub-tree.  There is an increasing sequence of stopping times:  1 1-  Low probability 11

29. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 29 Summary Reductions

30. 30 Equilibrium Construction  Each finite tree is colored according to: Whether it has a non-absorbing perfect equilibrium, an absorbing perfect equilibrium, or a special distribution The equilibrium payoff The maximal payoffs when a player stops alone  If c implies that each game on finite tree has a perfect equilibrium, concatenate the equilibria to obtain an approximate perfect equilibrium of G

31. 31 Equilibrium Construction  Last case: c implies that a special distribution exists  This allow to construct an approximate unrevealing perfect correlated equilibrium with a universal correlation device An adaptation of the protocol of Solan and Vohra (01)

32. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 32 Summary Reductions

33. 33 Summary and Future Research  Summary: every multi-player stopping game admits an approximate normal-form uniform perfect correlated equilibrium with a universal correlation device  Future research: Using this notion of equilibrium in the study of other dynamic games Structure of uniform perfect correlated equilibrium payoffs in specific applications

34. 34 Questions & Comments?  Y. Heller (2008), Perfect correlated equilibria in stopping games, mimeo. http://www.tau.ac.il/~helleryu/

35. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 35 Summary Reductions

36. 36 Reduction to Terminating Games  Proposition: Every game that stops immediately admits a ( ,  )- unrevealing perfect correlated (   )-equilibrium with a universal correlation device Every stopping game admits the same kind of (   )-equilibrium

37. 37 Proof Outline  Induction on the number of players  Given a stopping game G, we define an auxiliary terminating game G’: The payoff to I \ S when a coalition S stops is the equilibrium payoff in the induced stopping game G’ admits an unrevealing perfect correlated (   )-equilibrium with a universal correlation device Concatenation gives such an equilibrium in G

38. 38 Tree-like Games  Shmaya & Solan (2002) showed that any stopping game can be approximated by a tree-like stopping game, with the same set of approximate equilibria Small perturbations of the payoffs don’t change the set of approximate equilibria we can assume that the payoff process has a finite range Each set F n  F n can be identified with a node in a tree

39. Tree-like Games – Shmaya & Solan’s Proof Outline  k th partition: Discretization of the game: Depth: k Precision:     Refinement of all previous partitions Defines the k th approximating game on a tree  The game on finite tree that begins on m and ends on l will be played on the m+l approximating game

40. 40 Tree-like Games F4F4F4F4 F1F1 F2F2 F4F4

41. 41 Deep Enough in the Tree FnFn v1v1 v1v1 v1v1 v1v1 v 1, v 3, v 5 occur infinitely often, all other v  V do not occur at all v1v1 v1v1 v1v1 G (F n ): The induced game that begins at the node F n

42. 42 Lemma - Induced Games  Let: G - a terminating game,  - a stopping time Every induced game G (F n ), where F n is in the range of , admits an unrevealing perfect correlated (   )- equilibrium with a universal correlation device G admits the same kind of (C·   )-equilibrium Corollary: We can assume to be “deep enough”

43. Proof Outline  Until   the players follow an equilibrium in a finite stopping game with absorbing states {F  } with payoffs {x F   equilibria payoffs of G (F  )  After  players follow (   )-equilibrium of G (F   )  Relying on that the equilibrium is unrevealing and with a universal correlation device  Illustration…. 43

44. Proof Outline FF 44 x1x1 x2x2 x3x3 V(|I|,  ) - universal correlation device x4x4 x5x5 x6x6 x7x7

45. 45 Games on Finite Trees  g i : maximal payoff player i can get by stopping alone  The special distribution  over (nodes · players): A stopping player i and a node with maximal payoff (R i i,n =g i ) The distribution gives each player i at-least g i Each stopping player has a punisher j that stops when R j i,n <g i Allows to construct a correlated  -equilibrium

46. 46 Stationary Absorbing Equilibrium: Adaptations  Perfection - using a perturbed tree with  probability to ignore players’ requests to stop  Limiting the minimal per-round terminating probability  (adapting the methods of Shmaya & Solan, 2004) If there is a player i with a payoff below g i, then  can’t be too small or player i stops when his payoff is g i Otherwise either case 3 applies, or there is a node where at-least 2 players stop with a non-negligible probability Recursive trimming of such nodes gives the needed limit

47. 47  Last case: c implies that a special distribution  exists  Let   i k be the k-th time that player i’s maximal payoff occur with the requirement   i k >   j k-1 for all i, j Using the fact that we are “deep enough” in the tree  An approximate unrevealing perfect correlated equilibrium with a universal correlation device is constructed as follows… Equilibrium Construction: Protocol Description `

48. 48 Equilibrium Construction: Protocol Description  A quitter i’ is secretly chosen according to the special distribution  A number l’ is chosen uniformly in {1,T’}  i’ receives the signal l’  A number l is chosen uniformly in {l’+1,l’+T} 1<<T<<T‘  The punisher of i’ receives the signal l  Each other player receives a signal l+1  Approximate unrevealing perfect correlated equilibrium: each player stops at  l (when l is his signal) modulo 1+T+T’

49. Introduction: Stopping games perfect correlated (  )-equilibrium Main Result Proof Outline Finite trees & absorbing games Stochastic variation of Ramsey theorem Equilibrium construction 49 Summary Reductions