Winning concurrent reachability games requires doubly-exponential patience Michal Koucký IM AS CR, Prague Kristoffer Arnsfelt Hansen, Peter Bro Miltersen Aarhus U., Denmark

2 Example Player 1 chooses A  {t,h} Player 1 chooses A  {t,h} Player 2 chooses B  {t,h} Player 2 chooses B  {t,h}If A = B then move one level up, A = B then move one level up, A  B = t then move to 1 st level, A  B = t then move to 1 st level, A  B = h then Player 1 loses. A  B = h then Player 1 loses. Entrance fee: $15 Win: $20 W

3 Entrance fee: $15 Win: $20 Observation: To break even, you need at least ¾ probability to win. Good news: you can win with probability arbitrary close to 1. Bad news: the expected time to win the game with probability at least ¾ is years (one move per day). … the age of universe: years

4 Concurrent reachability games [de Alfaro, Henzinger, Kupferman ’98, Everett ’57] Two players play on a graph of states. At each step they simultaneously (independently) pick one of possible actions each and based on a transition table move to the next state. … … … …

5 Goals:Player 1 wants to reach a specific state or states. Player 2 wants to prevent Player 1 from reaching these states. Strategy of a player: Memory-less (non-adaptive) – π : states  actions. Memory-less (non-adaptive) – π : states  actions. Adaptive – π : history  actions. Adaptive – π : history  actions. Probabilistic strategy: π gives a probability distribution of possible actions.  Patience of a memory-less strategy π = 1/min non-zero prob. in π … [Everett ’57]

6 Winning starting states: Sure – Player 1 has a winning strategy that never fails. Sure – Player 1 has a winning strategy that never fails. Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 – . Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 – .

7 Purgatory n Player 1 chooses A  {t,h} Player 1 chooses A  {t,h} Player 2 chooses B  {t,h} Player 2 chooses B  {t,h}If A = B then move one level up, A = B then move one level up, A  B = t then move to 1 st level, A  B = t then move to 1 st level, A  B = h then move to state H. A  B = h then move to state H. P n n … H

8 Our results Thm:1) For every 0 1/  2 n-2. 2) For every l 2 2 n-l-2. Thm:For every 0 61 actions in total, both players have  -optimal strategies with patience 61 actions in total, both players have  -optimal strategies with patience < 1/  2 42m.

9 Thm:1) For every 0 t then the expected time to win the game by any  ’-optimal strategy of Player 1 can be forced to be Ω( t ).  patience ~ expected time to win  patience ~ expected time to win All the results essentially hold also for adaptive strategies All the results essentially hold also for adaptive strategies Recall: the expected time to win Purgatory 7 with probability at least ¾ is years (one move per day).

10 Algorithmic consequences Three algorithmic questions: 1. What are *-SURE states?  PTIME [dAHK] 2. What are the winning probabilities of different states?  PSPACE [EY] 3. What is the (  -)optimal strategy?  EXP-EXP-TIME upper-bound [CdAH,…]  EXP-SPACE lower-bound [our results] Cor: Any algorithm that manipulates winning strategies in explicit representation must use exponential space. … explicit representation: integer fractions

11 Purgatory n p i – probability of playing t in state i in  -optimal strategy of Player 1. p i – probability of playing t in state i in  -optimal strategy of Player 1. Claim: 1) 0< p i < 1, for all i. 2) p i < , for all i. 3) p 1 ≤ p 2. p 3 … p n 4) p i ≤ p i+1. p i+2 … p n P n n … 1\2th tlevel+1loss hlevel=1level+1 p n p n-1 p3p3p2p2p1p1p3p3p2p2p1p1 Player 2 plays h Player 2 plays t  Player 2 plays h t t t t t

12 Open problems Generic algorithm for  - optimal strategy with symbolic representation? Generic algorithm for  - optimal strategy with symbolic representation? How to redefine the game to be more realistic? How to redefine the game to be more realistic?

13 Goals:Player 1 wants to reach a specific state or states. Player 2 wants to prevent Player 1 from reaching these states. Winning starting states: Sure – Player 1 has a winning strategy that never fails. Sure – Player 1 has a winning strategy that never fails. Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. Almost-Sure – Player 1 has a randomized strategy that reaches goal with probability 1. Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 – . Limit-Sure – For every  > 0, Player 1 has a strategy that reaches goal with probability at least 1 – .