1. Potential games, Congestion games Computational game theory Spring 2010 Adapting slides by Michal Feldman TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.:

2. Potential games Definition: a game is an ordinal potential game if there exists  :S1×…×Sn  R, s.t.  i,s i,s -i,s i ’, c i (s i,s -i ) > c i (s i ’,s -i ) IFF  (s i,s -i ) >  (s i ’,s -i ) Note: G is an exact potential game if c i (s i,s -i ) - c i (s i ’,s -i ) =  (s i,s -i ) -  (s i ’,s -i ) Example: max-cut is an exact potential game, where  is the cut size – Unfortunately,  is not always so natural

3. Potential games Lemma: a game is a potential game IFF local improvements always terminate proof: – Define a directed graph with a node for each possible pure strategy profile – Directed edge (u,v) means v (which differs from u only in the strategy of a single player, i) is a (strictly) better action for i, given the strategies of the other players – A potential function exists IFF graph does not contains cycles If cycle exists, no potential function; e.g., (a,b,c,a) means f(a)<f(b)<f(c)<f(a) If no cycles exist, can easily define a ordinal potential function WHY?

4. Examples direction of local improvement 1,-1-1,1 1,-1 0,53,3 1,15,0 0,02,2 3,30,0 Matching pennies Prisoner’s dilemma Coordination game C D DC col row Which are potential games? Exact potential games? Are the potential functions unique? 0,02,1 1,20,0 Battle of the sexes

5. Properties of potential games Admit a pure strategy Nash equilibrium Best-response dynamics converge to NE Price of stability is bounded

6. Existence of a pure NE Theorem: every potential game admits a pure NE Proof: we show that the profile minimizing  is a NE – Let s be pure profile minimizing  – Suppose it is not a NE, so i can improve by deviating to a new profile s’ –  (s’) -  (s) = c i (s’) – c i (s) < 0 – Thus  (s’) <  (s), contradicting s minimizes  More generally, the set of pure-strategy Nash equilibria is exactly the set of local minima of the potential function –Local minimum = no player can improve the potential function by herself

7. Best-response dynamics converge to a NE Best-response dynamics: –Start with any strategy profile –If a player is not best-responding, switch that player’s strategy to a better response (must decrease potential) –Terminate when no player can improve (thus a NE) –Alas, no guarantee on the convergence rate

8. 8 Multicast (and non-multicast) Routing Multicast routing: Given a directed graph G = (V, E) with edge costs c e  0, a source node s, and k agents located at terminal nodes t 1, …, t k. Agent j must construct a path P j from node s to its terminal t j. Routing: Given a directed graph G = (V, E) with edge costs c e  0, and k agents seeking to connect s j,t j pairs, Agent j must construct a path P j from node s j to its terminal t j. Fair share: If x agents use edge e, they each pay c e / x. Slides on cost sharing based on slides by Kevin Wayne. Copyright @ 2005 Pearson-Addison Wesley. All rights reserved.

9. 9 Multicast Routing : Shapley price sharing (fair cost sharing) outer 2 middle 4 1 pays 5 + 1 5/2 + 1 middle4 1 outer middle outer 8 2 pays 8 5/2 + 1 5 + 1 s t1t1 v t2t2 4 8 11 5

10. 10 Nash Equilibrium Example: – Two agents start with outer paths. – Agent 1 has no incentive to switch paths (since 4 5 + 1). – Once this happens, agent 1 prefers middle path (since 4 > 5/2 + 1). – Both agents using middle path is a Nash equilibrium. s t1t1 v t2t2 4 8 11 5

11. Recall price of anarchy and stability Price of anarchy (poa)=cost of worst NE / cost of OPT Price of stability (pos)=cost of best NE / cost of OPT

12. Socially Optimum Social optimum: Minimizes total costs of all agents. Observation: In general, there can be many Nash equilibria. Even when it is unique, it does not necessarily equal the social optimum. s t1t1 v t2t2 355 1 1 Social optimum = 7 Unique Nash equilibrium = 8 s t k 1 +  Social optimum = 1 +  Nash equilibrium A = 1 +  Nash equilibrium B = k k agents pos=1, poa=kpos=poa=8/7

13. Price of anarchy Claim: poa ≤ k Proof: – Let N be the worst NE – Suppose by contradiction c(N) > k OPT – Then, there exists a player i s.t. c i (N) > OPT – But i can deviate to OPT (by paying OPT alone), contradicting that N is a NE Note: bound is tight (lower bound in prev. slide)

14. 14 Price of Stability What is price of stability in multicast routing? Lower bound of log k: s t2t2 t3t3 tktk t1t1... 11/2 1/31/k 0 000 1 +  1 + 1/2 + … + 1/k Social optimum: Everyone Takes bottom paths. Unique Nash equilibrium: Everyone takes top paths. Price of stability: H(k) / (1 +  ). upper bound will follow..

15. 15 Finding a potential function Attempt 1: Let  (s) =  j=1 k cost(t i ) be the potential function. A problem: The potential might increase when some agent improve. Example: When all 3 agents use the right path, each pays 4/3 and the potential (total cost) is 4. After one agent moves to the left path the potential increases to 5. s t 4 1 3 agents

16. 16 Finding a potential function Consider a set of paths P 1, …, P k. – Let x e denote the number of paths that use edge e. – Let  (P 1, …, P k ) =  e  E c e · H(x e ) be a potential function. – Consider agent j switching from path P j to path P j '. – Change in agent j’s cost: H(0) = 0,

17. 17 Potential function –  increases by –  decreases by – Thus, net change in  is identical to net change in player j’s cost

18. 18 Bounding the Price of Stability Claim: Let C(P 1, …, P k ) denote the total cost of selecting paths P 1, …, P k. For any set of paths P 1, …, P k, we have Proof: Let x e denote the number of paths containing edge e. – Let E + denote set of edges that belong to at least one of the paths.

19. 19 Bounding the Price of Stability Theorem: There is a Nash equilibrium for which the total cost to all agents exceeds that of the social optimum by at most a factor of H(k) (i.e., price of stability ≤ H(k)). Proof: – Let (P 1 *, …, P k * ) denote set of socially optimal paths. – Run best-response dyn algorithm starting from P *. – Since  is monotone decreasing  (P 1, …, P k )   (P 1 *, …, P k * ). previous claim applied to P previous claim applied to P*

20. Local search and PLS (polynomial local search) Local optimization problem: find a local optimum (i.e., no improvement in neighborhood Local optimization problem is in PLS if exists an oracle that for every instance and solution s decides if s is a local optimum; if not returns a better solution s’ in neighborhood of s Finding NE in potential games is in PLS – Define neighborhood of a profile s to be profiles obtained by deviation of a single player – s is local optimum for c(s) =  (s) iff s is a NE

21. Congestion games [Rosenthal 73] There is a set of resources R Agent i’s set of actions (pure strategies) A i is a subset of 2 R, representing which subsets of resources would meet her needs –Note: different agents may need different resources There exist cost functions c r : {1, 2, 3, …} →  such that agent i’s cost for a = (a i, a -i ) is Σ r  a i c r (n r (a)) –n r (a) is the number of agents that chose r as one of their resources in the profile a

22. Example: multicast routing Resources = edges Each resource r has a cost c r Player 1’s action set: {{A}, {C,D}} Player 2’s action set: {{B}, {C,E}} For all resources r, c r (n r (a)) = c r / n r (a) s t1t1 v t2t2 E 8 11 5 A 4 C D B

23. Every congestion game is an exact potential game Use potential  (a) = Σ r Σ 1 ≤ i ≤ nr(a) c r (i) –One interpretation: the sum of the costs that the agents would have received if each agent were unaffected by all later agents Why is this a correct potential function? Suppose an agent changes action: stop using some resources (R-), start using others (R+) increase in the agent’s cost equals Σ r  R+ c r (n r (a) + 1) - Σ r  R- c r (n r (a)) This is exactly the change in the potential function above –Conclusion: congestion games are exact potential games

24. Other Congestion games (s i,t i ) connectivity, atomic flow, cost = latency, all flow one unit.

25. Computational Game Theory: Network Creation Game Arbitrary Payments (Not a congestion game) Credit to Slides To Eva Tardos Modified/Corrupted/Added to by Michal Feldman and Amos Fiat

26. Network Creation Game – Arbitrary Cost partition G = (V,E) is an undirected graph with edge costs c(e). There are k players. Each player i has a source s i and a sink t i he wants to have connected. s1s1 t3t3 t1t1 t2t2 s2s2 s3s3

27. Model (cont’) Player i picks payment p i (e) for each edge e. e is bought if total payments ≥ c(e). Note: any player can use bought edges s1s1 t3t3 t1t1 t2t2 s2s2 s3s3

28. The Game Each player i has only 2 concerns: 1) Must be a bought path from s i to t i s1s1 t3t3 t1t1 t2t2 s2s2 s3s3 bought edges

29. The Game Each player i has only 2 concerns: 1) Must be a bought path from s i to t i 2) Given this requirement, i wants to pays as little as possible. s1s1 t3t3 t1t1 t2t2 s2s2 s3s3

30. Nash Equilibrium A Nash Equilibium (NE) is set of payments for players such that no player wants to deviate. Note: player i doesn’t care whether other players connect. s1s1 t3t3 t1t1 t2t2 s2s2 s3s3

31. An Example One NE: Each player pays 1/k to top edge. Another NE: Each player pays 1 to bottom edge. Note: No notion of “fairness”; many NE that pay unevenly for the cheap edge. s 1 …s k t 1 …t k c(e) = 1 c(e) = k

32. Three Observations 1) The bought edges in a NE form a forest. 2) Players only contribute to edges on their s i -t i path in this forest. 3) The total payment for any edge e is either c(e) or 0.

33. Example 2: No Nash s1s1 t1t1 t2t2 s2s2 all edges cost 1 a b c d

34. Example 2: No Nash s1s1 t1t1 t2t2 s2s2 We know that any NE must be a tree: WLOG assume the tree is a,b,c. all edges cost 1 a b c d

35. Example 2: No Pure Nash s1s1 t1t1 t2t2 s2s2 We know that any NE must be a tree: WLOG assume the tree is a,b,c. Only player 1 can contribute to a. all edges cost 1 a b c d

36. Example 2: No Pure Nash s1s1 t1t1 t2t2 s2s2 We know that any NE must be a tree: WLOG assume the tree is a,b,c. Only player 1 can contribute to a. Only player 2 can contribute to c. all edges cost 1 a b c d

37. Example 2: No Pure Nash s1s1 t1t1 t2t2 s2s2 We know that any NE must be a tree: WLOG assume the tree is a,b,c. Only player 1 can contribute to a. Only player 2 can contribute to c. Neither player can contribute to b, since d is tempting deviation. all edges cost 1 a b c d