1. Game tree search Chapter 6 (6.1 to 6.3 and 6.6) cover games. 6.6 covers state of the art game players in particular. 6.5 covers games that involve uncertainty. Now, on to what we learned last time....

11. Let’s practice by computing all the game theoretic values for nodes in this tree. A A A A B B B

12. A A A A B B B 0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

13. Let’s practice by computing all the game theoretic values for nodes in this tree. A A A A B B B 0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2 0 3 2 0 1 -5 -3 2

14. A A A A B B B 0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2 0 3 2 0 1 -5 -3 2 0 2 0 1 -5 2 0 2 1 2 0 1 1 Question: if both players play rationally, what path will be followed through this tree?

15. A A A A B B B 0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2 0 3 2 0 1 -5 -3 2 0 2 0 1 -5 2 0 2 1 2 0 1 1

22. Question: what happens if there are loops in the tree?

23. A s1s2 s3 14 128 β = 8 24 α = 2, then 4, then.... s4 s5 B 911 2 Example 1: We are currently expanding possible moves for player A, from left to right. Which of the node expansions above could we prune, and why?

24. A s1s2 s3 14 128 β = 8 24 α = 9 s4 s5 B 911 2 Once we discover a node with value ‘9’, there is no need to expand the nodes to the right!

25. B s1s2 s3 6 27 α = 7 93 β = 9, then 3, then.... s4 s5 A 42 8 Example 2: We are currently expanding possible moves for player B, from left to right. Which of the node expansions above could we prune, and why?

26. B s1s2 s3 6 27 α = 7 93 β = 3 s4 s5 A 42 8 Once we discover a node with value ‘3’, there is no need to expand the nodes to the right!

27. Which computations could we have avoided here? Assuming we expand nodes left to right? A A A A B B B 0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2 0 3 2 0 1 -5 -3 2 0 2 0 1 -5 2 0 2 1 2 0 1 1

28. A A A A B B B

29. Effectiveness of alpha beta pruning With no pruning, you have to explore O(b D ) nodes, which makes the run time of a search with pruning the same as plain minimax. If, however, the move ordering for the search is optimal (meaning the best moves are searched first), the number of nodes we need to search using alpha beta pruning O(b D/2 ). That means you can, in theory, search twice as deep! In Deep Blue, they found that alpha beta pruning meant the average branching factor at each node was about 6 instead of 35.

30. This is an example of the best case scenario for alpha beta pruning. The effective branching factor of the first layer is B. The effective branching of the second is 1. The effective layer of the third is B. And so on.... A A B

31. Rational Opponents This, however, all assumes that your opponent is rational e.g., will choose moves that minimize your score Storing your strategy is a potential issue: you must store “decisions” for each node you can reach by playing optimally if your opponent has unique rational choices, this is a single branch through game tree if there are “ties”, opponent could choose any one of the “tied” moves: which means you must store a strategy for each subtree What if your opponent doesn’t play rationally? will it affect the quality of the outcome? will your stored strategies work?

33. Heuristic evaluation functions in games

34. Some examples of heuristic evaluation functions Example for tic tac toe: h(n) = [# of 3 lengths that are left open for player A] - [# of 3 lengths that are left open for player B]. Alan Turing’s function for chess: h(n) = A(n)/B(n) where A(n) is the sum of the point value for player A’s pieces and B(n) is the sum for player B. Most evaluation functions are specified as a weighted sum of features: h(n) = w 1 *feat 1 (n) + w 2 *feat 2 (n) +... w i *feat i (n). Deep Blue used about 6000 features in its evaluation function.

35. Heuristic evaluation functions in games Think of a few games and suggest some heuristics for estimating the “goodness” of a position chess? checkers? your favorite video game? “find the last parking spot”?

36. An example of realtime game tree search 1.We run A* (or our favorite search algorithm) until we are forced to make a move or run out of memory. Note: no leaves are goals yet. 2. We use evaluation function f(n) to decide which path looks best (let’s say it is the red one). 3. We take the first step along the best path (red), by actually making that move. 4. We restart search at the node we reach by making that move. (We may actually cache the results of the relevant part of first search tree if it’s hanging around, as it would with A*).

37. Some issues in real game searches  How far should we search in our game tree to determine the value of a node, if we only have a fixed amount of time?  What if we stop our search at a level in the search tree where subsequent moves dramatically change our evaluation?  What if our opponent pushes this level off of the search horizon?  Often, it makes sense to make the depth we search to dynamically decided.

38. Question: is there an alpha beta version you can use to search this tree?