Download presentation

Presentation is loading. Please wait.

Published byRebekah Bement Modified about 1 year ago

1
Price of Total Anarchy June 2008 Slides by Israel Shalom Based on “Regret Minimization and the Price of Total Anarchy” By Avrim Blum, MohammadTaghi Hajiaghayi, Katrina Ligett and Aaron Roth 1

2
Agenda Preliminaries Game Theory Basics Regret Minimization Hotelling games Valid games Atomic congestion games Algorithmic efficiency 2

3
Games in Strategic Form The game has players Each player has his available pure strategies marks the strategy profiles Individual utility (payoff) functions 3

4
Games in Strategic Form – cont’d Examples: Rock, Paper, Scissors Prisoner’s Dilemma RockPaperScissors Rock0, 0-1, 11, -1 Paper1,-10,0-1, 1 Scissors-1,11, -10, 0 DenyConfess Deny1, 15, 0 Confess0, 53, 3 4

5
Mixed Strategies Users can play “mixed strategies” as well – a probability distribution over, we mark this as marks the mixed strategy profiles The payoffs are now defined as the expected value of over the randomness of the players Sometimes marked by 5

6
Best Response and Nash Equilibria Lowercase letters will usually denote elements: ,, … We denote by the selected strategies of the players other than i ( ) A strategy is best response to if for all : A strategy profile is a Nash Equilibrium if for all i, is a best response to. Pure equilibria might exist, but in every game there is at least one mixed Nash Equilbrium. 6

7
Nash Equilibria Examples: Rock, Paper, Scissors Mixed equilibrium: ([1/3,1/3,1/3], [1/3, 1/3, 1/3]) Prisoner’s Dilemma Pure equilibrium (Confess, Confess) RockPaperScissors Rock0, 0-1, 11, -1 Paper1,-10,0-1, 1 Scissors-1,11, -10, 0 DenyConfess Deny1, 15, 0 Confess0, 53, 3 7

8
Social Optimum Sometimes, we’ll define a social utility (welfare) function, similar to payoffs: Choices that would make sense: For mixed strategies, we’ll look for the expected value (analogous to payoff in mixed strategies) Socially optimum strategy profile (and OPT) are: We are assuming a maximizing game throughout, the minimization is analogous 8

9
Price of Anarchy Let mark all the Nash Equilibria in the game The price of anarchy is defined as the ratio of the worst NE to optimum: 9

10
Price of Anarchy Prisoner’s Dilemma Notice that the fraction is flipped (minimization game) DenyConfess Deny1, 15, 0 Confess0, 53, 3 10 OPT = 2 N = 6

11
Regret Minimization Let mark the strategy profiles in T steps We define the regret of player i in a maximization game: Intuitively, this is “how much i could gain more in average had he played a single strategy throughout the game” 11

12
Regret Minimization When a player i uses a regret-minimizing algorithm, for any sequence, we have the property Where: vanishes as marks the number of steps before The expectancy is over the algorithm’s randomness In other words, the expected value of regret vanishes Notice that this is for maximizing games 12

13
Regret Minimization This implies that for any sequence, if player i is regret-minimizing, then: The price of total anarchy is defined as: Where max is taken over, that are play profiles with regret-minimization property 13

14
Regret and NE Notice that when playing a Nash Equilibrium, all players will have zero-regret If there’s a better “constant” response, we can improve by moving to it Therefore, the price of total anarchy in any game consists an upper bound for the price of anarchy Regret-minimizing strategies NE 14

15
Advantages of Regret Minimization Computational Nash Equilibria are hard (PPAD-hard) to calculate – even for small action spaces There are efficient regret minimization algorithms for polynomial number of actions Motivational No particular reason for players to converge down to NE There might be multiple equilibria, and agents may individually prefer different ones Byzantine players’ actions are not taken into account in NE Regret-minimization considers only local information, much more practical 15

16
Agenda Preliminaries Hotelling games Definition POA/POTA Generalization Valid games Atomic congestion games Algorithmic efficiency 16

17
Hotelling - Game Definition Souvenir stand owners in Paris: There are tourists every day, they buy from whichever stand they find first Each stand owner wishes to maximize his own sales We want “fairness”, the social welfare function is the minimum of the total sales made. Formally: We have an n-vertex graph. Each seller locates himself at a vertex Each day, a tourist in each vertex, goes to the closest seller If there is a “tie” between the sellers, they split the gains Minimum utility: 17

18
Hotelling - Optimum Solution Notice that the sum of payoffs is always exactly n Therefore, the social optimum is achieved when all players have equal payoffs This can happen if all players play on the same vertex Therefore 18

19
Hotelling – POA Theorem 3.1 The price of anarchy in the Hotelling game is (2k – 2)/k Proof We are to show that all players gain at least n/(2k – 2) Assume the contrary, that player i gains less than that in S Consider player i “leaving” the game. The total payoff is still n, so the average payoff for players is now n/(k-1) There must be at least one player h gaining at least the average, playing the vertex v h Player i can assure n/(2k – 2) by moving to v h Contradiction to Nash equilibrium 19

20
Theorem 3.1 – cont’d We are left with showing tightness Consider a game with k-1 stars k-1 players play at centers of their own stars, and player k plays uniformly over all the star centers This is NE The randomizing player earns n/(2k - 2) 1 2 3 k-1 20

21
Hotelling – POTA Let be the strategy of playing an arbitrary strategy from strategies in. Define Notice that, since when player i is removed, the rest have average payoff of Lemma 3.4 For all i, for all,. (Trivial for t = u) 21

22
Lemma 3.4 - Proof Consider a -player game Each player other than i replicated twice: once as time- t player and once as time- u player, with strategies and. Average payoff is If player i replaces a time- t player, that’s his expected payoff If we further remove time-t players, we only improve 22

23
The Imaginary Game, n=10, k=4 23 time-t players time-u players replacing time-t player in imaginary ≤E replacing time-t & removing other time-t players =E =

24
Lemma 3.4 – cont’d 24 Same argument holds for replacing u-player:

25
Hotelling – POTA Theorem 3.2 Each regret minimizing player has at least n/(2k-2) payoff Proof Provided a sequence of T plays, select a random time u The average expected payoff if we played throughout is: Averaging over different u, we reach: 25

26
Hotelling – POTA We reached The second term is non-negative due to Lemma 3.4 There is value for u that achieves the average For that u, if player i mixes between, he’ll achieve A regret minimizing player achieves this expected payoff 26

27
Hotelling – POTA Corollary: The price of total anarchy in the Hotelling game is (2k-2)/k, matching the price of anarchy Notice that in the we haven’t made any assumptions about how other players behave, so the proof holds even in the presence of Byzantine players making arbitrary (or adversarial) decisions! 27

28
Generalized Hotelling Game Notice that in the proof we have used only three features of the hotelling game: Constant sum – the sum of utilities is constant Symmetric – the “names” of the stand owners don’t matter Monotone – any player can “leave” the game and the sum does not change We call such games with the “fairness” social utility generalized Hotelling games. Theorem 3.6: In any k -player generalized Hotelling game, the price of total anarchy among regret minimizing players is (2k-2)/k even in the presence of arbitrarily many Byzantine players. 28

29
Non-Convergence Consider the game with: Players {0, …, k-1} k-1 n -vertex stars, with centers at v 0, …, v k-2 and isolated vertex v k-1 Consider Each player’s payoff No single vertex has expected payoff more than No regrets However, this is not Nash! Players at the isolated vertex will deviate! 1 2 3 k-1 k 29

30
Break? 30

31
Agenda Preliminaries Hotelling games Valid games Definition Market sharing game POA/POTA Byzantine players Atomic congestion games Algorithmic efficiency 31

32
Valid Games – Definitions Consider a k -player maximization game For each player, there is a groundset of actions V i Player i plays from some feasible set Definitions Let The discrete derivative of at in the direction is The function is said to be submodular if for This should remind us “concavity” – decreasing marginal utility 32

33
Submodularity 33 Adding something to a smaller set makes a bigger difference A B V car house villa high-def jacuzzi

34
Valid Games – Definitions We will notate as the strategies of players with index smaller than i. We will also use both this and as complete strategies (as in apply over them), meaning that the remaining players play the empty set Definition 4.2: A game with private utility functions and social utility function is valid if: is submodular For all i, s: - private fairness For all s: - social fairness 34

35
Valid Games – Example Market sharing game (Goemans et. al., 2005) Players are ISP’s Markets are towns Each market has price and value Each player can “enter” the market he has an edge towards, with budget constraint Player’s payoff per market is the value divided by entrances Sum social utility Or – sum of values at entered markets 5 3 9 2 players markets 35

36
Valid Games – Price of Anarchy Vetta, 2002: In a valid game, if is a NE strategy, and is the optimal strategy then: Corollary: if is non-decreasing, then we have POA 2 (The derivatives are always positive) Theorem 4.3, Corollary 4.2 (no proofs) POTA matches POA in valid games (up to ) 36

37
Valid Games – Byzantine Players Theorem 4.5 In a valid game with nondecreasing social welfare, if k players minimize regret with while the Byzantine players play strategies the average social welfare is: Proof. Assume the contrary, 37

38
Theorem 4.5 – cont’d (non-decreasing) (gradually inserting) (submodularity) (private fairness) 38

39
Gradual Insertation 39 A B car house villa jacuzzi

40
Theorem 4.5 – cont’d (summarizing) (assumption – the first term is less than half) (social fairness) (rearranging sum) 40

41
Theorem 4.5 – cont’d At least one player must match that, so for him we have Contradictory to regret minimization! Note that it’s compared to the old OPT (without the Byzantine players) But it’s fair – Byzantine players may be acting even against their own interest – we can’t say anything about them 41

42
Agenda Preliminaries Hotelling games Valid games Atomic congestion games Definition Sum social utility – POTA Makespan utility – Lower bounds Algorithmic efficiency 42

43
Congestion Games A congestion game is a minimization game, with k players For each player, there is a set of facilities V i Player i plays from some feasible set In weighted games, player i has a weight w i For unweighted games, we assume w i = 1 The load on facility e is defined as Each facility e has an associated latency function f e Player i playing a i experiences cost 43

44
Atomic Congestion Games We’ll consider a specific kind of congestion game Unweighted Linear latencies – We will use sum social utility: Previously known results: POA for pure strategies is 2.5 (Awerbuch et. al., 2005) POA for mixed strategies is also 2.5 (Chirstodoulou and Koutsoupias, 2007) Theorem 5.1: POTA in this setting is 2.5 This asserts the previously known results! 44

45
Theorem 5.1 – Proof Let be the optimal play Since we have no regret, for all i Summarizing for each player, and rearranging sum: Or more simply: 45

46
Theorem 5.1 – cont’d Geometric mean is smaller than arithmetic mean, so: Recall our equation (1) (2) (3) 46

47
Theorem 5.1 – cont’d Multiplying both sides by two: Further relaxing the inequality: We’re done! 47

48
Parallel Link Congestion Game Consider n identical links and k weighted players Each player selects which link to use (single link) Each player pays the sum of the weights on the link 48

49
Parallel Link Congestion Games – cont’d Claim: In Parallel link congestion game with social cost function as the maximum expected job latency, POTA is 2 Proof. Rescale the weights, so that OPT = 1 Total weight is less than n, weights are less than 1 Total latency in T plays is Tn, at least one link e* with latency less than T in total, average latency - l(e*) ≤ 1 Regret minimizing player will be competitive to moving to e* We expect at most l(e*) +w i ≤ 2 49

50
Parallel Links Congestion Game – cont’d The unweighted case with the sum social utility, is called “load balancing game” It’s a specific case of the discussed before, thus we will have POA and POTA of 2.5 If k >> n (a likely case) and the server speeds are relatively bounded, we can say even more Theorem 5.6 (no proof): In this formation, POTA is 1 + o(1) Corollary 5.7: In this formation, POA is 1 + o(1), even for mixed strategies 50

51
Parallel Link Congestion Games – cont’d Usually, we consider the makespan social utility function – the load on the most loaded link Why doesn’t our argument from before hold? Because E[max{X}] > max{E[X]} The POA for 2-link games is 3/2 (Koutsopias and Papadimitriou, 1999) The POA for n-link games is (Koutsopias, Marvronikolas, Spirakis, 1999) Theorem 5.4 (no proof) POTA for this game with two links is 3/2. Theorem 5.5: POTA for this game with n links is. 51

52
Theorem 5.5 – Proof Sketch Consider n links, n players, unit weights. OPT = 1 Resembles what we did in Hotelling (for non- convergence): Split the players into groups At time t, group t mod plays at link 1, while the rest play in different nodes – get average latency of close to 1 This minimizes regret – for any fixed link, the player will need to share the link most of the times (latency ~2) Still, at each time, link 1 has a whole group – maximum latency of Notice that this holds even for unweighted players! 52

53
Theorem 5.5 – Proof Sketch 53 spots single spot load = load = 0 spots load=1

54
Agenda Preliminaries Hotelling games Valid games Atomic congestion games Algorithmic efficiency 54

55
Algorithmic Efficiency Weighted Majority Algorithm (Littlestone, Warmuth) Initialize for all i Update at time t, where is a small tradeoff parameter (0.01) and is the loss at time t-1 Expects regret over time T Explained in “Algorithmic Game Theory”, chapter 4 Polynomial in the number of strategies (Hotelling, Congestion games) Not as good in Valid games (the strategies are exponential to the size of groundset) We’re assuming a minimizing game with [0,1] loss and n strategies. 55

56
Questions? 56

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google