1. Game Dynamics Out of Sync Michael Schapira (Yale University and UC Berkeley) Joint work with Aaron D. Jaggard and Rebecca N. Wright

2. Motivation: Internet Routing Establish routes between Autonomous Systems (ASes). Currently handled by the Border Gateway Protocol (BGP). AT&T Qwest Comcast Sprint

3. Internet Routing as a Game [Levin-S-Zohar] Internet routing is a game! –players = ASes –players’ types = preferences over routes –strategies = outgoing edges BGP = Best-Response Dynamics –each AS constantly selects its best available route to each destination –… until a “stable state” (= PNE) is reached.

4. But… Challenge I: No synchronization of players’ actions –players can best-reply simultaneously. –players can best-reply based on outdated information. Challenge II: Are players incentivized to follow best-response dynamics? –Can a player benefit from not best-replying? this talk [Nisan-S-Valiant-Zohar]

5. Game Dynamics and Asynchrony Dynamic environments –Internet protocols –large-scale markets –social networks –multi-processor computer architectures Game theory provides useful tools to analyze these interactions, but…. … has so far primarily concentrated on synchronous environments (simultaneous, sequential).

6. 2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player Illustration

7. 2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player

8. But… 2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player

9. Model for asynchronous game dynamics Impossibility result Circumventing our impossibility result Complexity of asynchronous game dynamics Directions for future research Agenda

10. n nodes 1,…,n Node i has action space A i –A=A 1 … A n –A -i =A 1 … A i-1 A i+1 … A n Node i has reaction function f i :A → A i –f=(f 1,…,f n ) Simple Model: Nodes Interacting

11. Infinite sequence of discrete time steps t=1,… Initial state a 0, Schedule  :{1,…} → 2 [n] –fair schedule The (a 0,  )-dynamics –Start at the initial state a 0 –In each time step t let the nodes in  (t) react. Simple Model: Dynamics

12. Defn: an action profile a=(a 1,…,a n ) is a stable state if f i (a)=a i for all i. –that is, a is a fixed point of f. Defn: The system is convergent if the (a 0,  )-dynamics converges to a stable state for all choices of a 0 and (fair) . Simple Model: Convergence

13. Defn: f is node independent if, for each node i, f i :A -i → A i Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent. Can be generalized to reaction functions that are –randomized –bounded-recall –non-stationary Guaranteed Convergence?

14. Internet protocols –Internet routing [Sami-S-Zohar] –congestion control [Godfrey-S-Zohar-Shenker] Best-response dynamics –with consistent tie-breaking –orthogonal to the results of Hart and Mas-Colell Diffusion of technologies in social networks –2 technologies {A,B}. Each node wants to be consistent with the majority of its neighbours. Circuit design Applications

15. Example 1: (node-dependent reactions) Each f i is such that for every a=(a 1,…,a n ) it holds that f i (a)=a i. “Tightness” of Our Result

16. Example 1: (node dependent reactions) Each f i is such that for every a=(a 1,…,a n ) it holds that f i (a)=a i. Example 2: (unbounded recall) –2 nodes, 1 and 2, each with action space {a,b}. –Node 2 wants to match node 1’s action. –Node 1 selects b if node 2 changed its action from a to b in the past, and a otherwise. –What happens at the initial state (b,a)? “Tightness” of Our Result

17. Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent. Interesting connections to fundamental results in distributed computing theory. –the Fischer-Lynch-Patterson impossibility result for consensus protocols (1983) But, neither result is a special case of the other. Proving Our Result

18. The Dynamics Graph action vector a S =(a S 1,… a S n ) knowledge vector b S =(b S 1,… b S n ) State R knowledge transition i-transition State T State S 1.a T :=a S 2.b T :=a S 1.a R :=a S except a R i :=f i (b S ) 2.b R :=b S

19. The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics

20. The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0

21. The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0 1-transition 3-transitionk-transition

22. The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0 1-transition 3-transitionk-transition 2-transition3-transitionk-transition

23. Defn: A state S in the dynamics graph is stable if every outgoing edge from S leads to S. Defn: A fair path in the dynamics graph is a path that (1) for each i, contains an i-transition; and (2) also contains a knowledge transition. Stability and Fairness

24. Defn: The attractor region of a stable state S are all states from which any (long enough) fair path reaches S. Attractor Regions

25. Claim: A fair cycle in the dynamics graph implies an oscillation in our model. Proposition: If there are multiple stable states then there are states in the dynamics graph that are not in any attractor region (“neutral states”). Proof Sketch (Cont.)

26. Colour each attractor region in a different colour – red, blue, etc. Colour the neutral states in purple. Colouring States

27. Key idea: We show that from every purple state there is a fair path that leads to another purple state. The number of purple states is finite and so this implies a fair cycle. Creating Oscillations

28. Lemma: There cannot be two edges leading from a purple state to two non-purple states that do not have the same colour. Intuition: We can swap the order of activations without affecting the outcome. Proof Sketch (Cont.)    ?  : different transitions

29. Fix a purple state p. Let R be a “maximal” fair path from p to another purple state. Proof Sketch (Cont.) p … … q R

30. Let  be a transition that is not on R. Observe that  at q takes us to a non- purple state. p … … q R  Proof Sketch (Cont.)

31. Because q is purple it must have a fair path to a non-purple non-red state. p … … q R  … … u Proof Sketch (Cont.)

32. Now, we prove that  at u must take us to a red state --- a contradiction! p … … q R  … … u  Proof Sketch (Cont.)

33. Our result holds for randomized reaction functions. –adversarially-chosen schedule What if the schedule is randomized? –our impossibility result breaks … –… but no general possibility result either Circumventing Our Impossibility Result: Randomness

34. Defn: A schedule  is r-fair if each node is activated at least once within every r consecutive time steps. Can we prove our impossibility result for schedules that are r-fair? If so, for what values of r? We present positive and negative results. Circumventing Our Impossibility Result: r-Fair Schedules

35. Thm: Determining if a system with n nodes, each with two actions, is convergent requires exponential communication (in n). The proof requires reaction functions to be of exponential size. Combinatorial proof: a “Snake in the Box” construction Complexity Results

36. What if the reaction functions can be succinctly described? Thm: Determining if a system with n nodes is convergent is PSPACE- Complete. Hence, there is no “short” characterization of asynchronous convergence! Complexity Results

37. Other notions of asynchrony Other reaction functions –fictitious play, regret minimization –Observation: regret minimization is much more resilient to asynchrony (different framework…). Other restrictions on schedules –random schedules –r-fair schedules –more Directions for Future Research

38. THANK YOU