Preprocessing Techniques for Computing Nash Equilibria Vincent Conitzer Duke University Based on: Conitzer and Sandholm. A Generalized Strategy Eliminability Criterion and Computational Methods for Applying It. AAAI-05 Conitzer and Sandholm. A Technique for Reducing Normal-Form Games to Compute a Nash Equilibrium. AAMAS-06

Computing Nash equilibria in (2-player) normal-form games Computing one Nash equilibrium is PPAD-complete –Daskalakis, Goldberg, & Papadimitriou ECCC05; Chen & Deng FOCS06 Determining whether a Nash equilibrium with a certain property exists is typically NP-complete –Is there an equilibrium with player 1s utility / each players utility / average utility > k? Even hard to approximate –Is there an equilibrium that puts positive / zero probability on pure strategy s? –Etc. –Gilboa & Zemel GEB 89; Conitzer & Sandholm IJCAI03/extended draft All known algorithms take exponential time –Even Lemke-Howson [Savani & von Stengel FOCS04/Econometrica06]

Preprocessing games We are solving for an equilibrium (optimal equilibrium, maybe) When can we shrink the game and solve the shrunk game instead? If a strategy is (strictly) dominated, can throw it out If the game has only a small nonzero component, can focus on that We will see generalizations of both of these

A class of hard games Sandholm, Gilpin, Conitzer AAAI05 0, 20, 33, 00, 00, 20, 00, 2 0, 0 0, 30, 23, 00, 2 2, 42, 0 4, 22, 03, 3 2, 0 2, 42, 04, 2 0, 23, 00, 30, 00, 20, 00, 2 0, 0 3, 00, 20, 30, 2 4, 22, 0 3, 32, 02, 4 1/

Eliminability concepts Dominance: strategy always does worse than some other (mixed) strategy - strong argument - local reasoning - easy to compute - often does not apply Nash equilibrium: strategy does not appear in support of any Nash equilibrium -weaker argument - global reasoning - hard to compute - applies more often 3, 22, 3 3, 2 4, 00, Is there something in between that combines good aspects of both? Yes! [Conitzer & Sandholm AAAI05] 3, 22, 3 3, 2 2, 02, 1.5

Definition as game between attacker and defender Stage 1: Defender specifies probabilities on E strategies (e r * must get > 0) 3, 00, 30, 2 sr4sr4 0, 33, 00, 2 sr3sr3 2, 0 2, 2 sr2sr2 2, 0 2, 2 sr1sr1 sc4sc4 sc3sc3 sc2sc2 sc1sc Stage 2: Attacker chooses one of the E strategies with positive probability to attack and chooses (possibly mixed) attacking strategy attacked attacking Stage 3: Defender chooses on which (non-E) strategy to place the remainder of the probability –If attacking outperforms attacked, attacker wins attacked attacking e r * = s r 3, E r = {s r 3, s r 4 }, E c = {s c 3, s c 4 } 3, 00, 30, 2 sr4sr4 0, 33, 00, 2 sr3sr3 2, 0 2, 2 sr2sr2 2, 0 2, 2 sr1sr1 sc4sc4 sc3sc3 sc2sc2 sc1sc1 3, 00, 30, 2 sr4sr4 0, 33, 00, 2 sr3sr3 2, 0 2, 2 sr2sr2 2, 0 2, 2 sr1sr1 sc4sc4 sc3sc3 sc2sc2 sc1sc1

A spectrum of elimination power The larger the E i sets, the more strategies are eliminable If the E i sets include all strategies, then a strategy is eliminable if and only if no Nash equilibrium places positive probability on it If the E i sets are empty (with the exception of e r *) then e r * is eliminable if and only if it is dominated dominance Nash equilibrium larger E i sets

Alternative definition Stage 1: Defender specifies probabilities on E sets (e r * must get > 0) Stage 2: Attacker chooses one of the E strategies with positive probability to attack Stage 3: Defender distributes the remainder of the probability (not on E) attacked attacked Stage 4: Attacker chooses attacking strategy –If attacking outperforms attacked, attacker wins 0.05 attacked attacking e r * = s r 3, E r = {s r 3, s r 4 }, E c = {s c 3, s c 4 }

Equivalence Theorem. The alternative definition is equivalent to the original one. Proof based on duality (more specifically, Minimax Theorem [von Neumann 1927] )

Mixed integer programming approach (using alternative definition) Continuous variables: p i (e i ), p i e -i (s i ), binary: b i (e i ) maximize p r (e r *) subject to –for both i, for any e i E i, Σp -i (e -i ) + Σp -i e i (s -i ) = 1 –for both i, for any e i E i, p i (e i ) b i (e i ) –for both i, for any e i E i and any d i S i, Σp -i (e -i )(u i (e i, e -i )-u i (d i, e -i )) + Σp -i e i (s -i )(u i (e i, s -i )-u i (d i, s -i )) (b i (e i )-1)U i U i is the maximum difference between two of player is utilities Number of binary variables = |E r | + |E c | –Exponential only in this!

Eliminating strategies in the hard game 0, 20, 33, 00, 00, 20, 00, 2 0, 0 0, 30, 23, 00, 2 2, 42, 0 4, 22, 03, 3 2, 0 2, 42, 04, 2 0, 23, 00, 30, 00, 20, 00, 2 0, 0 3, 00, 20, 30, 2 4, 22, 0 3, 32, 02, 4 1/ ErEr EcEc

Another preprocessing technique for computing a Nash equilibrium [Conitzer & Sandholm AAMAS06] a l, d ml …a 2, d m2 a 1, d m1 ……… a l, d 2l …a 2, d 22 a 1, d 21 a l, d 1l …a 2, d 12 a 1, d 11 c kn, b k …c k2, b k c k1, b k ……… c 2n, b 2 …c 22, b 2 c 21, b 2 c 1n, b 1 …c 12, b 1 c 11, b 1 G π r, π c a l, Σ i p G (s i )d 1l …a 2, Σ i p G (s i )d i2 a 1,Σ i p G (s i )d i1 Σ j p G (t j )c kj, b k … Σ j p G (t j )c 2j, b 2 Σ j p G (t j )c 1j, b 1 G H H

Required structure on original game O a l, d ml …a 2, d m2 a 1, d m1 smsm ………… a l, d 2l …a 2, d 22 a 1, d 21 s2s2 a l, d 1l …a 2, d 12 a 1, d 11 s1s1 c kn, b k …c k2, b k c k1, b k ukuk ………… c 2n, b 2 …c 22, b 2 c 21, b 2 u2u2 c 1n, b 1 …c 12, b 1 c 11, b 1 u1u1 tntn …t2t2 t1t1 vlvl …v2v2 v1v1 That is: against any fixed v j, all the s i give the row player the same utility a j against any fixed u i, all the t j give the column player the same utility b i H G

Solve for equilibrium of G (recursively) smsm … s2s2 s1s1 tntn …t2t2 t1t1 Obtain –Equilibrium distributions p G (s i ), p G (t j ) –Players expected payoffs in equilibrium π r, π c G

Reduced game R π r, π c a l, Σ i p G (s i )d 1l …a 2, Σ i p G (s i )d i2 a 1,Σ i p G (s i )d i1 s Σ j p G (t j )c kj, b k ukuk …… Σ j p G (t j )c 2j, b 2 u2u2 Σ j p G (t j )c 1j, b 1 u1u1 tvlvl …v2v2 v1v1 Expected payoffs when row player plays the equilibrium of G, column player plays v i Expected payoffs when both players play the equilibrium of G Theorem. p R (u i ), p R (s)p G (s i ); p R (v j ), p R (t)p G (t j ) constitutes a Nash equilibrium of original game. H

Example v1v1 t1t1 t2t2 u1u1 2, 20, 32, 3 s1s1 1, 24, 00, 4 s2s2 1, 40, 44, 0 t1t1 t2t2 s1s1 0, 4 s2s2 4, v1v1 t u1u1 2, 21, 3 s 2,

A more difficult example = the game that we solved before! v 1 = b 2 t 1 = b 1 t 2 = b 3 u 1 = a 2 2, 20, 32, 3 s 1 = a 1 1, 24, 00, 4 s 2 = a 3 1, 40, 44, 0 b1b1 b2b2 b3b3 a1a1 1, 20, 4 a2a2 0, 32, 22, 3 a3a3 0, 41, 44, 0 But how (in general) do we find the correct labeling of the strategies as u i, s i, v j, t j ? Can it be done in polynomial time?

Lets try to use satisfiability b1b1 b2b2 b3b3 a1a1 4, 01, 20, 4 a2a2 0, 32, 22, 3 a3a3 0, 41, 44, 0 Say that v(σ) = true if we label σ as one of the s i or t j (that is, we put it in G) If a 1, a 2 are both in G, then b 1 must also be in G because a 1, a 2 get different payoffs against b 1 Equivalently, v(a 1 ) and v(a 2 ) v(b 1 ) –or (-v(a 1 ) or -v(a 2 ) or v(b 1 )) Theorem: satisfaction of all such clauses the condition is satisfied

Clauses for the example b1b1 b2b2 b3b3 a1a1 4, 01, 20, 4 a2a2 0, 32, 22, 3 a3a3 0, 41, 44, 0 v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 1 ) and v(b 3 ) v(a 2 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(b 1 ) and v(b 2 ) v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 2 ) and v(a 3 ) Complete characterization of solutions: –Set at most one variable to true for each player (does not reduce game) –Set all variables to true (G = whole game!) –Only nontrivial solution: set v(a 1 ), v(a 3 ), v(b 1 ), v(b 3 ) to true

Simple algorithm Algorithm to find nontrivial solution: –Start with any two variables for the same agent set to true –Follow the implications –If all variables set to true, start with next pair of variables

Solving the example with the algorithm (pass 1) b1b1 b2b2 b3b3 a1a1 4, 01, 20, 4 a2a2 0, 32, 22, 3 a3a3 0, 41, 44, 0 v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 1 ) and v(b 3 ) v(a 2 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(b 1 ) and v(b 2 ) v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 2 ) and v(a 3 ) Variables set to true: v(a 1 ) v(a 2 ) v(a 3 ) v(b 1 )v(b 2 )v(b 3 )

Solving the example with the algorithm (pass 2) b1b1 b2b2 b3b3 a1a1 4, 01, 20, 4 a2a2 0, 32, 22, 3 a3a3 0, 41, 44, 0 v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 1 ) and v(b 3 ) v(a 2 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(b 1 ) and v(b 2 ) v(a 1 ) and v(a 2 ) v(b 1 ) and v(b 3 ) v(a 1 ) and v(a 3 ) v(b 2 ) and v(b 3 ) v(a 1 ) and v(a 2 ) and v(a 3 ) Variables set to true: v(a 1 ) v(a 3 )v(b 1 )v(b 3 )

Algorithm complexity Theorem. Requires at most O((#rows+#columns) 4 ) clause applications –That is, quadratic if the game is square Can improve in practice by caching previous results

Preprocessing the hard game 2, 44, 23, 3 2, 44, 2 3, 32, 4 0, 21.5, 1.50, 2 2, 42, 04, 23, 3 2, 02, 44, 2 2, 03, 32, 4 0, 20, 33, 00, 00, 20, 00, 2 0, 0 0, 30, 23, 00, 2 2, 42, 0 4, 22, 03, 3 2, 0 2, 42, 04, 2 0, 23, 00, 30, 00, 20, 00, 2 0, 0 3, 00, 20, 30, 2 4, 22, 0 3, 32, 02, 4 0, 33, 0 0, 3 3, 00, 0 1.5, 1.5 3, 00, 30, 0 1.5, 1.50, 0 1.5, 1.5 0, 33, 0 0, 3 3, 00, 0 0, 33, 0 0, 30, 0 3, 00, 3 1/2 1/

Conclusions Generalized strategy eliminability criterion [AAAI05] –Parameterized definition –At one extreme setting, dominance –At other extreme, whether a strategy is in the support of any Nash –Efficiently computable for settings close to dominance Technique for recursively solving subcomponent [AAMAS06] –Subcomponents solution can be used to shrink original game –Efficient algorithm for finding subcomponent Other techniques? Generalization to extensive form? Thank you for your attention!

Search-based approaches Suppose we know the support X i of each player is mixed strategy in equilibrium Then, we have a simple linear feasibility problem: –for both i, for any s i X i, Σp -i (s -i )u i (s i, s -i ) = u i –for both i, for any s i S i - X i, Σp -i (s -i )u i (s i, s -i ) u i Thus, we can search over supports –This is the basic idea underlying methods in Dickhaut & Kaplan 91; Porter, Nudelman, Shoham AAAI04; Sandholm, Gilpin, Conitzer AAAI05 Dominated strategies can be eliminated –A type of preprocessing –What other preprocessing techniques exist?