2. 10/19/2004TCSS435A Isabelle Bichindaritz1 Game and Tree Searching

3. 10/19/2004TCSS435A Isabelle Bichindaritz2 Learning Objectives Introduction to games Optimal decisions in games Alpha-Beta pruning Real-time decisions

4. 10/19/2004TCSS435A Isabelle Bichindaritz3 Introduction to Games Games as Search Problems –Frameworks: two-player, multi-player; zero-sum; perfect information –Minimax algorithm Perfect decisions Imperfect decisions (based upon static evaluation function) –Issues Quiescence Horizon effect –Need for pruning (alpha-beta pruning)

5. 10/19/2004TCSS435A Isabelle Bichindaritz4 Introduction to Games Perfect Play –General framework(s) –What could agent do with perfect info? Resource Limits –Search ply –Static evaluation: from heuristic search to heuristic game tree search –Examples Tic-tac-toe, connect four, checkers, connect-five Chess, go Games with Uncertainty –Explicit: games of chance (e.g., backgammon, Monopoly, blackjack) –Implicit

6. 10/19/2004TCSS435A Isabelle Bichindaritz5 Games versus Search Problems Unpredictable Opponent –Games are adversarial search problems –Solution is strategy, contingency plan –Time limits Unlikely to find goal Must approximate Plan of Attack –Algorithm for perfect play (J. von Neumann, 1944) –Finite horizon, approximate evaluation (C. Zuse, 1945; C. Shannon, 1950, A. Samuel, 1952-1957) –First chess program (Turing, 1951) –Pruning to allow deeper search (J. McCarthy, 1956)

7. 10/19/2004TCSS435A Isabelle Bichindaritz6 Types of Games Information: Can Know (Observe) –… outcomes of actions / moves? –… moves committed by opponent? Uncertainty –Deterministic vs. nondeterministic outcomes –Thought exercise: sources of nondeterminism?

8. 10/19/2004TCSS435A Isabelle Bichindaritz7 Minimax Games with two players MIN and MAX are a search problem with: –Initial state –Successor function –Terminal test / state –Utility function (objective / payoff) Win / loss / draw(chess) +1 / -1 / 0(chess) -192 … +192 (backgammon)

9. 10/19/2004TCSS435A Isabelle Bichindaritz8 Minimax

10. 10/19/2004TCSS435A Isabelle Bichindaritz9 Minimax Perfect play for deterministic, perfect-information games Choose move to position with highest minimax value = best achievable payoff against best play Simple example: 2-ply game, a ply being a half-move Game ends after one move each by MAX and MIN (one move deep)

11. 10/19/2004TCSS435A Isabelle Bichindaritz10 Minimax

12. 10/19/2004TCSS435A Isabelle Bichindaritz11 Minimax Algorithm: Decision and Evaluation  what’s this?

13. 10/19/2004TCSS435A Isabelle Bichindaritz12 Properties of Minimax Complete?

14. 10/19/2004TCSS435A Isabelle Bichindaritz13 Properties of Minimax Complete? –… yes, provided following are finite: Number of possible legal moves (generative breadth of tree) “Length of game” (depth of tree) – more specifically? A finite strategy can exist even in an infinite tree! –Perfect vs. imperfect information? Q: What search is perfect minimax analogous to? A: Bottom-up breadth-first

15. 10/19/2004TCSS435A Isabelle Bichindaritz14 Properties of Minimax Complete? –… yes, provided the tree is finite: Number of possible legal moves (generative breadth of tree) “Length of game” (depth of tree) – more specifically? Optimal?

16. 10/19/2004TCSS435A Isabelle Bichindaritz15 Properties of Minimax Complete? –… yes, provided the tree is finite Optimal? –… yes, provided perfect info (evaluation function) and opponent is optimal! –… otherwise, guaranteed if evaluation function is correct Time Complexity?

17. 10/19/2004TCSS435A Isabelle Bichindaritz16 Properties of Minimax Complete? –… yes, provided the tree is finite: Optimal? –… yes, provided perfect info (evaluation function) and opponent is optimal! Time Complexity? –Depth of tree: m –Legal moves at each point: b –O(b m ) – NB, m  100, b  35 for chess! Space Complexity?

18. 10/19/2004TCSS435A Isabelle Bichindaritz17 Properties of Minimax Complete? –… yes, provided the tree is finite: Optimal? –… yes, provided perfect info (evaluation function) and opponent is optimal! Time Complexity? –Depth of tree: m –Legal moves at each point: b –O(b m ) – NB, m  100, b  35 for chess! Space Complexity? O(bm) – why?

19. 10/19/2004TCSS435A Isabelle Bichindaritz18 Resource Limits

20. 10/19/2004TCSS435A Isabelle Bichindaritz19 Static Evaluation Function Example: Chess

21. 10/19/2004TCSS435A Isabelle Bichindaritz20 Do Exact Values Matter?

22. 10/19/2004TCSS435A Isabelle Bichindaritz21 Cutting Off Search

23. 10/19/2004TCSS435A Isabelle Bichindaritz22 Cutting Off Search Issues –Quiescence Play has “settled down” Evaluation function unlikely to exhibit wild swings in value in near future –Horizon effect “Stalling for time” Postpones inevitable win or damaging move by opponent See: Figure 6.9 R&N Solutions? –Quiescence search: expand non-quiescent positions further –No general solution to horizon problem at present

24. 10/19/2004TCSS435A Isabelle Bichindaritz23 Why Prune?

25. 10/19/2004TCSS435A Isabelle Bichindaritz24 Game Tree Search Perfect Play (TBD from Perfect Information) –Common framework: two-player (or multi-player), zero-sum –Deterministic or known probability distribution function, totally observable –Minimax algorithm given ability to search for arbitrary ply Assumptions about –… game? –… opponent? Specifications and Ramifications –Combinatorics: finite depth (m), breadth (b) of search –Resource limits: need for static evaluation function Practical Issues –Effects of depth cutoff –Need for pruning

26. 10/19/2004TCSS435A Isabelle Bichindaritz25  -  Pruning Properties of  -  pruning –Pruning does not affect final result –Good move ordering improves effectiveness of pruning –With “perfect ordering,” time complexity = O(b m/2 ) Depth of tree: m Legal moves at each point: b Effective branching factor becomes b 1/2 Can easily reach depth 8 and play good chess –A simple example of the value of reasoning about which computations are relevant ( a form of metareasoning)

27. 10/19/2004TCSS435A Isabelle Bichindaritz26  -  Pruning  is the value of the best (i.e. highest value) choice we have found so far at any choice point along the path for MAX  is the value of the best (i.e. lowest value) choice we have found so far at any choice point along the path for MIN.

28. 10/19/2004TCSS435A Isabelle Bichindaritz27 Alpha-Beta (  -  ) Pruning: Definitions

29. 10/19/2004TCSS435A Isabelle Bichindaritz28 Alpha-Beta (  -  ) Pruning Example ≥ 3 3 3 MAX MIN MAX 128 ≤ 2 2145 ≤ 14 2 ≤52

30. 10/19/2004TCSS435A Isabelle Bichindaritz29 ≥ 3 3 3 MAX MIN MAX 128 ≤ 2 2 145 ≤ 14 2 ≤52 Alpha-Beta (  -  ) Pruning Example ,  here? What are ,  values here?

31. 10/19/2004TCSS435A Isabelle Bichindaritz30 Alpha-Beta (  -  ) Pruning: Modified Minimax Algorithm

32. 10/19/2004TCSS435A Isabelle Bichindaritz31 Alpha-Beta (  -  ) Pruning Properties Perfect Play (TBD from Perfect Information) –Common framework: two-player (or multi-player), zero-sum –Deterministic or known probability distribution function, totally observable –Minimax algorithm given ability to search to arbitrary ply Assumptions about –… game? –… opponent? Specifications and Ramifications –Combinatorics: finite depth (m), breadth (b) of search –Resource limits: need for static evaluation function Practical Issues –Effects of depth cutoff –Need for pruning