Download presentation

Presentation is loading. Please wait.

Published byCasey Catlett Modified over 2 years ago

1
On the Approximation Performance of Fictitious Play in Finite Games Paul W. GoldbergU. Liverpool Rahul Savani U. Liverpool Troels Bjerre Sørensen U. Warwick Carmine VentreU. Liverpool

2
Penalty-kick practice shoot on the right, left or center of the goal dive on the right, left or center of the goal players actions Scenario: Every day two friends meet to practice penalty-kicks

3
Penalty-kick game 0,11,0 0,11,0 0,1 R C L R C L Q: How would a goalie “learn” to play this game?

4
Fictitious play [Brown, 51] FP rule: Best respond to the empirical distribution of play of the opponent. 0, 11,0 0,11,0 0,1 R C L R CL 1/10 2/10 7/10 dives on the L Is it a “good” choice? Ie, is it a good algorithm strategically? RCCLLLLLLL Days12345678910 ‘s action

5
Where does the name come from? FP can also be seen as an algorithm for playing the game just once 1. Simulate what would happen in the repeated version of the game up to some predetermined round r 2. Output the empirical distribution In the above example for r=10, the empirical distribution of is: R wp 1/10, C wp 2/10, L wp 7/10 FP is a very simple iterative algorithm Sometimes, advocated to model bounded rationality Is FP strategically “good”?

6
Fictitious play and Nash equilibria The empirical distribution of play defined by FP converges to Nash equilibria for constant-sum games [Robinson, 51] non-degenerate 2 × 2 games [Miyasawa, 61] 2 × n games [Berger, 05]... but it does not converge in general [Shapley, 64] 0, 11,0 0,11,0 0,1 R C L R CL 0,1+ Ɛ 1,0 0,1+ Ɛ 1,0 0,1+ Ɛ R C L R CL

7
Fictitious play and approximate NEs Analysis of the strategic performances of FP done by means of approximate NEs NE = no incentive to deviate Ɛ - NE = little ( Ɛ ) incentive to deviate Concept which assumed relevance given the PPAD- hardness of computing exact NEs [Daskalakis, Goldberg & Papadimitriou, 06] + [Chen & Deng, 06] Payoffs normalized to [0,1] and additive approximation [Conitzer, 09] proves: For any game, Ɛ ≤ (r+1)/(2r) at round r There exists an infinite game for which Ɛ = (r+1)/2r

8
Approximation guarantee of FP round players’ actions By FP rule, s i is a best response to the mixture of the first i-1 actions 123...i-1i...r-1r a1a1 a2a2 a3a3...a i-1 aiai... a r-1 arar s1s1 s2s2 s3s3...s i-1 sisi... s r-1 srsr Ɛ = 0 Ɛ = 1 Ɛ for playing s i is (r-i+1)/r 2 Ɛ of FP is

9
Approximation for finite games? Re-using strategies may guarantee a significantly better approximation of FP Experimentally, Shapley’s game (for which FP does not converge) has Ɛ ≈ 1/4 round players’ actions 123...i-1i...r-1r a1a1 a2a2 a3a3...a i-1 aiai... a r-1 arar s1s1 s2s2 s3s3...s i-1 sisi... sisi srsr Ɛ = 1 Ɛ = 0 Ɛ for playing s i at round i is less than (r-i+1)/r 2

10
Our contribution We define a class of 4n × 4n symmetric games, n being a parameter, for which we show that FP fails to obtain any constant Ɛ < ½ Specifically, we prove a lower bound of ½ - O(1/n 1-δ ) for any δ > 0 We also give a “matching” upper bound of ½ - O(1/n)

11
The game: row player’s payoff matrix n=5 α>1, β<1 Blank entries stand for a 0 Column player’s payoff matrix is the transpose of the above Players share the same sequence of actions (simpler analysis)

12
The role of α and β α>1, β<1 Blank entries stand for a 0 Column player’s payoff matrix is the transpose of the above Players share the same sequence of actions (simpler analysis)

13
The last block and the induction

14
Next ideas of the analysis α and β govern the ratio between the probabilities of two consecutive actions Ratios are such that probabilities increase in geometric progression last n different actions played occupy all but an exponentially-small fraction of the probability mass best response has payoff around β (1-1/2 n ) ≈ 1- O(1/n 1-δ ) probability distribution does not allocate much probability to any individual strategy payoff from FP distribution is around α/2 ≈ 1/2

15
Upper bound Ɛ for an action is defined by its last occurrence in the sequence The maximum Ɛ is given by the sequence m 1,..., m 1,..., m n,..., m n round players’ actions 123...i-1i...r-1r a1a1 a2a2 a3a3...a i-1 aiai... a r-1 arar s1s1 s2s2 s3s3...s i-1 sisi... sisi srsr r/n

16
Conclusions FP is not good from a strategic point of view in terms of approximation guarantee to NEs for finite games There is a class of finite games for which a cyclic behavior persists which leads to a poor guarantee (independently of the number of iterations) Ie, fully rational player has always beaten his bounded rational friend

17
Open problems Is ½ a limit to the approximation performance obtainable by simple or decentralized algorithms? Cf. algorithm of [Daskalakis, Mehta & Papadimitriou, 09] vs more complex centralized algorithms achieving a ratio better than a half Consider more general class of algorithms E.g., uncoupled dynamics defined by [Hart & Mas- Colell, 03] + [Hart & Mas-Colell, 06]

Similar presentations

Presentation is loading. Please wait....

OK

Learning in games Vincent Conitzer

Learning in games Vincent Conitzer

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on famous indian entrepreneurs in Ppt on synthesis and degradation of purines and pyrimidines paired Ppt on the coordinate plane A ppt on refraction of light Download ppt on oxidation and reduction chemistry Ppt on contract labour act 1970 india Ppt on input devices and output devices Ppt on eye os and od Ppt on electrical power generation system using railway track slides Ppt on projectile motion in physics