Krishnendu Chatterjee1 Partial-information Games with Reachability Objectives Krishnendu Chatterjee Formal Methods for Robotics and Automation July 15, 2011 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A

Krishnendu Chatterjee2 Games on Graphs  Games on graphs.  Perfect-information graph games  Zermelo’s theorem about Chess in 1913  From every configuration  Either player 1 can enforce a win.  Or player 2 can enforce a win.  Or both players can enforce a draw.

Krishnendu Chatterjee3 Chess: Games on Graph  Chess is a game on graph.  Configuration graph.

Krishnendu Chatterjee4 Graphs vs. Games Two interacting players in games: Player 1 (Box) vs Player 2 (Diamond).

Krishnendu Chatterjee5 Game Graph

Krishnendu Chatterjee6 Game Graphs  A game graph G= ((S,E), (S 1, S 2 ))  Player 1 states (or vertices) S 1 and similarly player 2 states S 2, and (S 1, S 2 ) partitions S.  E is the set of edges.  E(s) out-going edges from s, and assume E(s) non- empty for all s.  Game played by moving tokens: when player 1 state, then player 1 chooses the out-going edge, and if player 2 state, player 2 chooses the outgoing edge.

Krishnendu Chatterjee7 Game Example

Krishnendu Chatterjee8 Game Example

Krishnendu Chatterjee9 Game Example

Krishnendu Chatterjee10 Strategies  Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges.  ¾ : S * S 1  D(S).  ¼ : S * S 2 ! D(S).

Krishnendu Chatterjee11 Strategies  Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges.  ¾ : S * S 1 ! D(S).  History dependent and randomized.  History independent: depends only current state (memoryless or positional).  ¾ : S 1 ! D(S)  Deterministic: no randomization (pure strategies).  ¾ : S * S 1 ! S  Deterministic and memoryless: no memory and no randomization (pure and memoryless and is the simplest class).  ¾ : S 1 ! S  Same notations for player 2 strategies ¼.

Krishnendu Chatterjee12 Objectives  Objectives are subsets of infinite paths, i.e., Ã µ S !.  Reachability: there is a set of good vertices (example check-mate) and goal is to reach them. Formally, for a set T if vertices or states, the objective is the set of paths that visit the target T at least once.

Krishnendu Chatterjee13 Applications: Verification and Control of Systems  Verification and control of systems  Environment  Controller M satisfies property ( Ã ) E C

Krishnendu Chatterjee14 Applications: Verification and Control of Systems  Verification and control of systems  Question: does there exists a controller that against all environment ensures the property. M satisfies property ( Ã ) EC ||

Krishnendu Chatterjee15 Applications: Systems for Specification  Synthesis of systems from specification  Input/Output signals.  Automata over I/O that specifies the desired set of behaviors.  Can the input player present input such that no matter how the output player plays the generated sequence of I/O signals is accepted by automata ?  Deterministic automata: Games.

Krishnendu Chatterjee16 -synthesis [Church, Ramadge/Wonham, Pnueli/Rosner] -model checking of open systems -receptiveness [Dill, Abadi/Lamport] -semantics of interaction [Abramsky] -non-emptiness of tree automata [Rabin, Gurevich/ Harrington] -behavioral type systems and interface automata [deAlfaro/ Henzinger] -model-based testing [Gurevich/Veanes et al.] -etc. Game Models Applications

Krishnendu Chatterjee17 Reachability Games  Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

Krishnendu Chatterjee18 Reachability Games  Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

Krishnendu Chatterjee19 Reachability Games  Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T

Krishnendu Chatterjee20 Reachability Games  Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X.  Fix-point X T

Krishnendu Chatterjee21 Reachability Games  Sure winning: requires least fixpoint computation. R(1) R R(0)

Krishnendu Chatterjee22 Reachability Games  Winning set for a partition: Determinacy  Player 1 wins: then no matter what player 2 does, certainly reach the target.  Player 2 wins: then no matter what player 1 does, the target is never reached.  Memoryless winning strategies.  Can be computed in linear time [Beeri 81, Immerman 81].

Krishnendu Chatterjee23 Chess Theorem  Zermelo’s Theorem Win 1 Win 2 Both draw

Krishnendu Chatterjee24 Partial-information Graph Games

Krishnendu Chatterjee25 Why Partial-information  Perfect-information: controller knows everything about the system.  This is often unrealistic in the design of reactive systems because systems have internal state not visible to controller (private variables) noisy sensors entail uncertainties on the state of the game  Partial-observation Hidden variables = imperfect information. Sensor uncertainty = imperfect information.

Krishnendu Chatterjee26 Partial-information Games  A PIG G =(L, A, , O) is as follows  L is a finite set of locations (or states).  A is a finite set of input letters (or actions).   µ L £ A £ L non-deterministic transition relation that for a state and an action gives the possible next states.  O is the set of observations and is a partition of the state space. The observation represents what is observable.  Perfect-information: O={{l} | l 2 L}.

Krishnendu Chatterjee27 PIG: Example a,b a ba b

Krishnendu Chatterjee28 New Solution Concepts  Sure winning: winning with certainty (in perfect information setting determinacy).  Almost-sure winning: win with probability 1.  Limit-sure winning: win with probability arbitrary close to 1.  We will illustrate the solution concepts with card games.

Krishnendu Chatterjee29 Card Game 1  Step 1: Player 2 selects a card from the deck of 52 cards and moves it from the deck (player 1 does not know the card).  Step 2:  Step 2 a: Player 2 shuffles the deck.  Step 2 b: Player 1 selects a card and view it.  Step 2 c: Player 1 makes a guess of the secret card or goes back to Step 2 a.  Player 1 wins if the guess is correct.

Krishnendu Chatterjee30 Card Game 1  Player 1 can win with probability 1: goes back to Step 2 a until all 51 cards are seen, and plays uniformly at random for choosing cards till then.  Player 1 cannot win with certainty: there are cases (though with probability 0) such that all cards are not seen. Then player 1 either never makes a guess or makes a wrong guess with positive probability.

Krishnendu Chatterjee31 Card Game 2  Step 1: Player 2 selects a new card from an exactly same deck and puts is in the deck of 52 cards (player 1 does not know the new card). So the deck has 53 cards with one duplicate.  Step 2:  Step 2 a: Player 2 shuffles the deck.  Step 2 b: Player 1 selects a card and view it.  Step 2 c: Player 1 makes a guess of the secret duplicate card or goes back to Step 2 a.  Player 1 wins if the guess is correct.

Krishnendu Chatterjee32 Card Game 2  Player 1 can win with probability arbitrary close to 1: goes back to Step 2 a for a long time and then choose the card with highest frequency (choosing uniformly at random till the choice is made).  Player 1 cannot win probability 1, there is a tiny chance that not the duplicate card has the highest frequency, but can win with probability arbitrary close to 1, (i.e., for all ² >0, player 1 can win with probability 1- ², in other words the limit is 1).

Krishnendu Chatterjee33 Sure winning for Reachability  Result from [Reif 79]  Memory is required.  Exponential memory required.  Subset construction: what subsets of states (knowledge) player 1 can be. Reduction to exponential size perfect-information games.  EXPTIME-complete.

Krishnendu Chatterjee34 Partial-information Games a a a a b b a b a a b b In starting play a. In yellow play a and b at random. In purple: if last was yellow then a if last was starting, then b. Requires both randomization and memory

Krishnendu Chatterjee35 Almost-sure winning for Reachability  Result from [CDHR 06, CHDR 07]  Standard subset construction fails: as it captures only sure winning, and not same as almost-sure winning.  More involved subset construction is required.  EXPTIME-complete.

Krishnendu Chatterjee36 Almost-sure winning for Reachability  Brief Idea of the proof.  Suppose GOD gives us the winning knowledge sets.  Given a knowledge cannot play an action that is unsafe (takes out of the winning knowledge set).  Must play safe actions.  Play all safe actions uniformly at random.

Krishnendu Chatterjee37 Almost-sure winning for Reachability  Brief Idea of the proof.  Suppose GOD gives us the winning knowledge sets.  What GOD can do we can do by non- determinism.  This gives NEXPTIME algorithm, for EXPTIME algorithm see [CDHR 07].

Krishnendu Chatterjee38 Summary: Theory of Graph Games Winning Mode/ Game Graphs SureAlmost-sureLimit-sure Turn-based Games (CHESS) Linear time (PTIME-complete) Linear-time (PTIME-complete) Linear-time (PTIME-complete) Partial-information Games (POKER) EXPTIME-complete

Krishnendu Chatterjee39 Limit-sure winning for Reachability  Limit-sure winning for reachability is undecidable [GO 10, CH 10].  Reduction from the Post-correspondence problem (PCP).

Krishnendu Chatterjee40 Limit-sure winning for Reachability  The undecidable result is first proved for probabilistic automata.  Probabilistic automata are special case of blind stochastic games.  For partial-information games, stochastic transitions can be simulated by deterministic transition [CDGH 10].  Consequently we obtain the undecidability result.

Krishnendu Chatterjee41 Summary: Theory of Graph Games Winning Mode/ Game Graphs SureAlmost-sureLimit-sure Turn-based Games (CHESS) Linear time (PTIME-complete) Linear-time (PTIME-complete) Linear-time (PTIME-complete) Partial-information Games (POKER) EXPTIME-complete Undecidable

Krishnendu Chatterjee42 Conclusion and Related Problems  Theory of graph games  Perfect-information and partial-information games.  Different solution concepts and different complexity.  Several algorithmic questions open.  Partial-observation stochastic games with pure strategies (Working manuscript of CD 11)  Many surprising results (such as non-elementary complete memory bounds).

Krishnendu Chatterjee43 Conclusion and Related Problems  Partial information games  Problem with clear practical motivation.  Challenging to establish the right frontier of complexity.  Important generalization of perfect-information games.  Unfortunately, undecidable and also high complexity.  Looking for new definitions (of sub-classes).

Krishnendu Chatterjee44 Collaborators  Laurent Doyen  Hugo Gimbert  Thomas A. Henzinger  Jean-Francois Raskin

Krishnendu Chatterjee45 Thank you ! Questions ? The end