1. The Computational Complexity of Finding Nash Equilibria Edith Elkind Intelligence, Agents, Multimedia group (IAM) School of Electronics and CS U. of Southampton

2. Games and Strategies Games: strategic interactions between rational entities Solution concepts: what’s going to happen? –dominant strategies –Nash equilibrium –…. Can it be computed? –if your computer cannot find it, the market probably cannot either

3. Matrix (Normal Form) Games 20 01 1 0 0 3 Row player: Column player: 0 1 0101 0 1 finite set of players {1, …, n} each player has k actions (pure strategies): 1, …, k payoffs of the i th player: P i : {1, …, k} n → R

4. Nash Equilibrium 20 01 1 0 0 3 Row player: Column player: 0 1 0101 0 1 Nash equilibrium: a strategy profile such that noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies: –(0, 0) and (1, 1) are both NE

5. Pure vs. Mixed Strategies 1 1 1 1 Row player: Column player: H T HTHT H T NE in pure strategies may not exist! –“matching pennies” Mixed strategy: a probability distribution over actions –50% tail, 50% head

6. Existence of NE Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies can we find one in poly-time?

7. Plan of the Talk 2 players, k actions n players, 2 actions other cool stuff

8. 2 (r=const) players, k actions Input representation: –2 players: two k x k matrices –r players: r k x k x … x k matrices poly-size for constant r Output representation: –for 2 players all NE are in Q –but not for 3 and more players… Checking for pure NE: easy –at most k 2 strategy profiles

9. 2 players, k actions: mixed NE Naïve approaches: exp(k) Simplex-like approach (Lemke-Howson algorithm): –works well in practice –exp(k) in the worst case (2004) Is it time to give up? –maybe the problem is NP-hard?

10. Is Finding NE NP-hard? Reminder: a problem P is NP-hard if you can reduce 3-SAT to it: –“yes”-instance 3-SAT → “yes”-instance of P –“no”-instance 3-SAT → “no”-instance of P Problem: each instance of NASH is a “yes”-instance! –every game has a NE Formally: if NASH is NP-hard then NP = coNP Need: complexity theory for total search problems

11. Reducibility Among Search Problems S associates x in X with a solution set S(x) Total search problem: for any x, S(x) is not empty S: X Y T: X’ Y’ If T is easy, so is S S is reducible to T if: –f, g easy to compute –g(T(f(x))) is in S(x) f g

12. Completeness Results? Can we prove that any total search problem is reducible to r-NASH? Not really: the class T of all total search problems is a semantic class –not known how to find complete problems for these Want to pick a large subclass S of T s.t. –S includes some natural problems –there are problems that are complete for S –in particular, r-NASH is complete for S

13. Input: Boolean circuits S (Successor), P (Predecessor): –n inputs, n outputs –S(0 n ) ≠ 0 n, P(0 n ) = 0 n Output: x ≠ 0 n s.t. –S(P(x)) ≠ x or P(S(x)) ≠ x Intuition: G=(V, E): –V =  n ; –E = {(x,y) | y=S(x), x=P(y)} END OF THE LINE 00000 01011 11001 01011

14. PPAD PPAD: Polynomial Parity Argument, Directed version PPAD is the class of all search problems that are reducible to END OF THE LINE search problem solution circuits S, T “end of the line” f g

15. r-NASH is in PPAD Proof on Nash’s theorem: –existence of NE reduces to Brouwer’s fixpoint theorem –Brouwer’s fixpoint theorem reduces to Sperner’s lemma –Sperner’s lemma is proven by a parity argument (similar to END OF THE LINE) Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)

16. Brouwer’s Fixpoint Theorem Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint. Nash  Brouwer proof sketch: –set of all strategy profiles → simplex –mapping: (s 1, …, s n ) → (s 1 +  1, …, s n +  n ), where  i is a shift in the direction of best response to (s 1, …, s i-1, s i+1, …, s n ) –NE is a point where noone wants to deviate, i.e., a fixpoint

17. Proper coloring: –vertices on BC are not blue –vertices on AC are not green –vertices on AB are not yellow Sperner’s Lemma: there exists a trichromatic triangle Brouwer’s theorem  Sperner’s Lemma: –x is blue if the grad(F) at x points away from A, etc. –trichromatic triangle “has no direction” –repeat at increased resolution… Sperner’s Lemma A B C

18. Reductions (Papadimitriou 1994) END OF THE LINE is PPAD-complete TRICHROMATIC TRIANGLE is PPAD-complete 3D-BROUWER is PPAD-complete r-NASH is in PPAD

19. r-NASH vs 3D BROUWER Existence of NE follows from Brouwer’s fixpoint theorem NE are special cases of Brouwer’s fixpoints –just how special? Can any fixpoint be represented as a NE of a game? –Is there a reduction from 3D BROUWER to r-NASH?

20. Hardness Reductions: the Timeline 3D-BROUWER is PPAD-complete (Papadimitriou, 1994) 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005) 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005) 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)

21. n players, 2 actions representation: payoffs to each player for every action profile (vector in {0, 1} n ): n2 n numbers graphical games: –players are vertices of a graph –V’s payoff depends on actions of W in N (V) U V –n players, max degree d => n2 d+1 numbers T U V W t=0, u=0, v=0, w=0: 12 t=1, u=0, v=0, w=0: 31 …. t=1, u=1, v=1, w=1: -6 W’s payoffs (16 cases):

22. Algorithms: What Was Known Bounded-degree trees: –Exp-time algorithm/poly-time approximation algorithm to find all NE (Kearns, Littman, Singh, UAI 2001) –??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001) Heuristics for graphs with cycles

23. Our Results (E., Goldberg, Goldberg’06) Algorithm in NIPS’01 paper is incorrect (does not always output a NE) We fix the NIPS’01 algorithm, but… –our algorithm runs in poly-time on paths –with a trick, also on cycles There is a graph of pathwidth 2 on which our algorithm runs in exp time –true for all algorithms that use the basic approach of the UAI’01 paper

24. Warm-up: 2-player 2-action games 20 01 1 0 0 3 Row player: Column player: 0 1 0101 0 1 Suppose R plays 1 w.p. r EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3 1-r > 3r iff r < ¼ Suppose C plays 1 w.p. c EP(R) from playing 0: (1-c)*2 EP(R) from playing 1: c*1 (1-c)*2 > c iff c < 2/3 1/4 1 r BR(C) c 1 2/3 BR(R) mixed NE: r=1/4, c=2/3

25. Potential best response: v is a PBR to w iff when W plays w, there is a NE for T V in which V plays v. upstream pass: construct PBR V (w) from PBR U1 (v), PBR U2 (v) and PBR U3 (v) downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves Algorithm for Trees (KLS’01) TVTV W V U1U1 U2U2 U3U3 v w

26. Computing PBR on a Path E0 = EP(V) from playing 0: (1-u)(1-w)*v 000 +(1-u)w*v 001 +u(1-w)*v 100 +uw*v 101 = auw+bu+cw+d E1 = EP(V) from playing 1: (1-u)(1-w)*v 010 +(1-u)w*v 011 +u(1-w)*v 110 +uw*v 111 = a’uw+b’u+c’w+d’ E0 = E1 iff w = (Au+B)/(Cu+D) = f(u) UVW.51 1 u v v 1 1.1.9 w (v, u) → (f(u), v) PBR U (v) PBR V (w)

27. Trees: too many segments vvw u t v v1v1 v2v2 v1v1 v2v2 v1v1 v2v2 KLS (NIPS’01): can “trim” PBR Incorrect! W V TU (v,t), (v,u) → (f(u,t), v) u2u2 u1u1 t2t2 t1t1

28. Solutions? Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis –the number of segments/rectangles in each PBR is O(n 2 ) –running time O(n 3 ) Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments –O(n 2 ) running time

29. Extension to trees? V0V0 V1V1 V2V2 V n-1 VnVn U1U1 T1T1 U n-1 T2T2 U2U2 T n-1 TnTn UnUn

30. Graphical games: hardness results NP-hard? –no: total search problem PPAD-hard? –yes! –in fact, this is how the hardness result for 4-player games was obtained (Goldberg, Papadimitriou, Aug 2005)

31. Equivalences: GP’05 r-player game G NE of G deg 3 graphical game G’ NE of G’ f g d 2 -player game G’ NE of G’ deg d graphical game G NE of G f g

32. Combining Reductions: GP’05 r-player game G NE of G 9-player game G’ NE of G’ f g Finding NE in a 4-player game is as hard as finding NE in a r-player game for any constant r X 4

33. PPAD-hardness: missing details 3D-Brouwer is PPAD-complete (Papadimitriou, 1994) 4-NASH is as hard as deg 3-GG (Goldberg, Papadimitriou, Aug 2005) deg 3-GG is PPAD-complete and hence 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005) 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005) 2-NASH is PPAD-complete !!! (Chen, Deng, Dec 2005)

34. NE with special properties Pure NE: easy for constant number of players NP-hard for general graphical games –even if max degree = 3 –NP vs. PPAD: pure NE may not exist! poly-time on trees (KLS algorithm) –also on graphs with bounded treewidth

35. Welfare-Maximizing NE 20 01 1 0 0 3 Row player: Column player: 0 1 0101 0 1 Nash equilibria: (0, 0): total payoff is 3 (1, 1): total payoff is 4 (1/4, 2/3): total payoff is 17/12 not all NE are created equal…

36. Algorithms for Good NE 2-player games: checking for NE with total payoff > T is NP-hard (Gilboa Zemel 89, Conitzer, Sandholm 03) Graphical games:  for any algebraic , deg(  ) = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays   approximation algorithms for any  (E., Goldberg, Goldberg 07)

37. Approximate NE  -Nash equilibrium: a strategy profile such that noone can gain >  by deviating Graphical games on trees: poly-time algorithms for any  (KLS’01) 2-player games ( utilities in [0, 1] ): –PPAD-complete for  =O(1/n) –Approximation for constant  : 0.5WINE’06 (Dec 2006) 0.382 ( =1-1/  ) ACM EC’07 (June 2007) 0.364WINE’07 (Dec 2007) 0.339WINE’07 (Dec 2007)

38. Conclusions Computational aspects of game-theoretic questions are crucial Lots of cool open problems –computing NE in graphical games on trees –finding  -Nash in 2-player games for small  A rich set of techniques Talk to me if you want to know more…

39. Mixed strategies and payoffs Payoff matrices: the row player plays a = (a 1, …, a n ) the column player plays b = (b 1, …, b n ) expected payoff of R when playing i: (R i, *, b) expected payoff of C when playing i: (C *, j, a) R 11 R 12 … R 1n R 21 R 22 … R 2n … R n1 R n2 … R nn C 11 C 12 … C 1n C 21 C 22 … C 2n … C n1 C n2 … C nn R:C:

40. if 1 st player’s strategy a supported on I  N a i ≠ 0 iff i  I 2 nd player’s strategy b supported on J  N b j ≠ 0 iff j  J then I  BR(b): (b, R i, * ) ≥ (b, R k, * ) for all i  I, k  N J  BR(a): (a, C *, j ) ≥ (a, C *, k ) for all j  J, k  N –LP on variables a 1, …, a n, b 1, …, b n –solutions to LP ↔ Nash equilibria running time: 2 2k poly(k) 2 players, k actions: support guessing linear inequalities!

41. Reminder: 2-player 2-action games 20 01 1 0 0 3 Row player: Column player: 0 1 0101 0 1 Suppose R plays 1 w.p. r EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3 1-r > 3r iff r < ¼ Suppose C plays 1 w.p. c EP(R) from playing 0: (1-c)*2 EP(R) from playing 1: c*1 (1-c)*2 > c iff c < 2/3 1/4 1 r BR(C) c 1 2/3 BR(R) mixed NE: r=1/4, c=2/3

42. Computing PBR on a path f(u) = (au+b)/(cu+d) a, b, c, d are determined by V’s payoffs UVW.51 1 u v v 1 1.1.9 w (v, u) → (f(u), v) + “tails” PBR U (v) PBR V (w)