1. Game Trees ctd.. 15-211 Fundamental Data Structures and Algorithms Ananda Guna April 27, 2006

2. In this lecture..  Computer chess  Backtracking  Game Trees  Minimax and NegaMax  Alpha-beta pruning  Iterative Deepening  Nim and Amazons X O X O X O X O O

3. Computer Chess

4. Chiptest, Deep Thought Timeline  A VLSI design class project evolves into F.H.Hsu move-generator chip  A productive CS-TG results in ChipTest, about 6 weeks before the ACM computer chess championship  Chiptest participates and plays interesting (illegal) moves  Chiptest-M wins ACM CCC  Redesign becomes DT  DT participates in human chess championships (in addition to CCC)  DT wins second Fredkin Prize ($100K)  DT is wiped out by Kasparov Story continues @ IBM

5. Opinions: Is Computer Chess AI? From Hans Moravec’s Book “Robot”

6. How does Kasparov won?  Even the best Chess grandmasters say they only look 4 or 5 moves ahead each turn. Deep Junior looks up about 18-25 moves ahead. How does it lose!?  Kasparov has an unbelievable evaluation function. He is able to assess strategic advantages much better than programs can (although this is getting less true).  The moral, the evaluation function plays a large role in how well your program can play.

7. Programming Chess  There are 10 120 possible chess boards (Shannon, 1949)  Early 70s Atkin and Slate wrote chess 4.5  Read "Chess Skill in Man and Machine", which is in the library  They developed the basic techniques that is the foundation for most chess engines  Hash Tables  Move ordering  Iterative Deepening  Etc..

8. Hash Tables  Remember hash value of positions as different sequences of moves can lead to the same position  What to keep in the table  May be obvious from context which positions are more useful  Keep the position in which more search time has been invested  alpha-beta and hashing is a little tricky

9. Move Ordering  Alpha-beta algorithm correctly evaluates a position as mini-max  The performance of Alpha-Beta is affected by other factors  The order in which the moves are tried can have an enormous impact on the efficiency of the search  Estimates of the quality of a move may sometimes be available  Heuristic estimate of the cost of searching one move versus another may be available

10. Iterative Deepening  Depth can be a variable (based on available time)  Starting from current position, evaluate to successively deeper and deeper depths  Use the results of shallower evaluations to control the move ordering  Benefit of having a good move ordering makes up for the expense of doing all the shallower searches  Another advantage of iterative deepening is that if the search at a depth D is taking too much time, then the program can stop and fall back on the shallower evaluation

11. Tricks  Many tricks and heuristics have been added to chess program, including this tiny subset:  Opening Books  Avoiding mistakes from earlier games  Endgame databases (Ken Thompson)  Singular Extensions  Think ahead  Contempt factor  Strategic time control

12. Backtracking X O X O X O X O O CD B A

13. Backtracking  An algorithm-design technique  “Organized brute force”  Enumerate the space of possible answers  Do this in an organized way  Useful…  … when a problem is too hard to be solved directly  Maze traversal  8-queens problem  … when we have limited time and can accept a good but potentially not optimal solution  Game playing  Planning

14. Basic backtracking  Develop answers by identifying a set of decisions.  Knights tour  if the chess piece the knight could make a tour around the board, thereby visiting every square once and just once  On a 6x6 board there are 9,862 different closed tours  Tried 4,056,367,434 moves

15. Basic backtracking  Develop answers by identifying a set of decisions.  Pure backtracking  Decisions can be ok or impossible.  Heuristic backtracking  Decisions can have goodness. They can also be impossible.  Bad to sacrifice the queen to take a pawn  Bad not to block in tic-tac-toe  Good to pack a large object into the box

16. 8-Queen Problem  Arrange 8-queens on a 8x8 chess board such that no queen can take another 4-Queen problem ………. ……..

17. Backtracking technique  When you reach a dead-end  Withdraw the most recent choice  Undo its consequences  Is there a new choice? If so, try that If not, you are at another dead-end A B C

18. No optimality guarantees  If there are multiple solutions, simple backtracking will find one of them, but not necessarily the optimal.  No guarantees that we’ll reach a solution quickly.

19. Backtracking Summary  Organized brute force  Formulate problem so that answer amounts to taking a set of successive decisions  When stuck, go back, try the remaining alternatives  Does not give optimal solutions

20. Game Trees

21. Decision trees: Planning  Nodes of the tree  Denote states in the problem space  The path from leaf to root  Denotes a solution  A sequence of states  Edges of the tree  Denote legal “moves” that lead to new states X O X O X O X O O

22. Mini-Max Algorithm  The Mini-Max algorithm is applied in two player games  tic-tac-toe, checkers, chess, Amazons, Nim etc..  Properties of these games  Can be described by a set of rules  Finite state space  Can look ahead to see what moves are possible  Full information games  Each player knows all the possible moves of the opponent

23. Min-Max Algorithm  Each terminal position has some value  Terminal position is a position where the game is over or a place where a value can be assigned to a board (base case)  In TicTacToe a game may be over with a tie, win or loss  The value of a non-terminal position P, is given by  v(P) = max - v(P') P' in S(P)  Where S(P) is the set of all successor positions of P  Minus sign is there because the other player is moving into position P’

24. Nega-Max - pseudo code Nega-max(P){ if P is terminal, return v(P) m = -  for each P' in S(P) v = -(Nega-max(P')) if m < v then m = v return m }

25. How fast is mini-max?  Minimax is pretty slow even for a modest depth.  It is basically a brute force search.  What is the running time?  Each level of the tree has some average b moves per level. We have d levels. So the running time is _____

26. Quiz Time  It is your turn. You have n possible moves you can make. Each subsequent stage the number of moves is reduced by 50%. You want to analyze the decision tree up to d levels. How many moves must be analyzed before you decide to make a move with mini-max?

27. A Game  Consider the following two player game:  The game starts with a pile of n pebbles.  Players alternate taking pebbles from the pile  Each turn, you are allowed to take either 1, 2, 5, or sqrt(x) pebbles from the pile where x is the current number of pebbles in the pile.  The game continues until there are no more pebbles.  The person who takes the last pebble loses.  Let F(x) be a function that returns 1 if the current player will win with a pile of x pebbles  F(x)=-1 if the current player loses.

28. Problem ctd…  Case 1:  Suppose you are allowed to take only one pebble at a time. Write a recursive definition of F(x) that maximizes your advantage and minimizes your opponents  Case 2:  Suppose you are now allowed to take 1,3,5 or sqrt(x) from the pile. Write a recursive definition that maximizes your advantage and minimizes your opponents

30. Alpha-Beta Pruning Reducing search complexity

31. Alpha-Beta Pruning

32. Alpha Beta speedup  Alpha Beta is always a win:  Returns the same result as Minimax,  Is minor modification to Minimax  Claim: The optimal Alpha Beta search tree is O( b d/2 ) nodes or the square root of the number of nodes in the regular Minimax tree.  Can enable twice the depth  In chess branching is about 38. In practice Alpha Beta reduces it to about 6 and enables 10 ply searches on a PC.  Question: How many nodes need to be analyzed in chess  If mini-max is used w/o Alpha-Beta?  If mini-max is used w/ Alpha-Beta?

33. Alpha Beta Pruning Theorem: Let v(P) be the value of a position P. Let X be the value of a call to AB(, ). Then one of the following holds: v(P) ≤  and X ≤   < v(P) <  and X = v(P)  ≤ v(p) and X ≥  Suppose we take a chance and reduce the size of the infinite window. What might happen?

34. Aspiration Window:  Suppose you had information that value of a position was probably close to 2 (say from the result of shallower search)  Instead of starting with an infinite window, start with an “aspiration window” around 2 (e.g., (1.5, 2.5)) to get more pruning.  If the result is in that range you are done.  If outside the range you don’t know the exact value, only a bound. Repeat with a different range.

35. Heuristics

36. Heuristic search techniques  Alpha-beta is one way to prune the game tree… Heuristic = aid to problem-solving

37. Move Ordering  Explore decisions in order of likely success to get early alpha beta pruning  Guide search with estimator functions that correlate with likely search outcomes or  track which moves tend to cause beta cutoff  Heuristic estimate of the cost (time) of searching one move versus another  Search the easiest move first

38. Timed moves  Not uncommon to have a limited time to make a move. May need to produce a move in say 2 minutes.  How do we ensure that we have a good move before the timer goes off?

39. Iterative Deepening  Evaluate moves to successively deeper and deeper depths:  Start with 1-ply search and get best move(s). Fast  Next do 2-ply search using the previous best moves to order the search.  Continue to increase depth of search.  If some depth takes too long, fall back to previous results (timed moves).

40. Iterative Deepening  Save best moves of shallower searches to control move ordering.  Need to search the same part of the tree multiple times but improved move ordering more than makes up for this redundancy  Difficulty:  Time control: each iteration needs about 6x more time  What to do when time runs out?

41. Transposition Tables  Minimax and Alpha Beta (implicitly) build a tree. But what is the underlying structure of a game?  Different sequences of moves can lead to the same position.  Several game positions may be functionally equivalent (e.g. symmetric positions).  Use a Hash Table to save the known results

42. Transposition Tables  Memoize: hash table of board positions to get  Value for the node  Upper Bound, Lower Bound, or Exact Value  Be extremely careful with alpha beta as may only know a bound at that position.  Best move at the position  Useful for move ordering for greater pruning!  Which positions to save?  Sometimes obvious from context  Ones in which more search time has been invested  Collisions: Simply overwrite

43. Limited Depth Search Problems  Horizon effect: if the bad news over the search depth  White queen takes knight, and the evaluation function will report that white is up a knight  One level down, black has the reply pawn takes queen  Quiescence search: Look for tactical moves by the opponent from a depth node

44. Games Nim and Amazons

45. Game of Nim  Arbitrary number of piles. Players alternate. A player can take any number of coins from a pile. Player who take the last coin (or pile) wins  Let’s start with a simple configuration of Nim and use Minimax to select a move.  Our initial configuration consists of three piles, with 1, 2, and 3 pennies in each pile.  We can represent this configuration compactly by writing it as (1,2,3). Each position in this list represents the number of pennies in that stack. Order does not matter (I can just rearrange the stacks).

46. Drawing the Game Tree The first thing we need to take care of is drawing the game tree. (1,2,3) (2,3)(1,1,3)(1,2,2)(1,3)(1,1,2)(1,2) One level of the tree. Whose move is it now?

47. Nim Game Tree (1,2,3) (2,3) (1,1,3) (1,3) (1,2,2) (1,2,1) (1,2) (2,2) (1,3) (3) (2) (1,2) (1,1,2) (1,1) (1) (1,1,1) (2,1) (2) (1) (3) (1,1,1) (1,1) (2) (1,1) (1) (1)

48. Nim Game Graph (1,2,3) (2,3)(1,1,3)(1,2,2)(1,3)(1,1,2)(1,2) (2,2)(1,2)(2)(1,3)(3)(1,1,2)(1,1,1)(1,1)(1) (3)(2)(1)(1,1)(1,2)(1,1,1) (2)(1)loss(1,1) (1)win loss win Us Them Us Them

49. Game of Amazons

50. Game of Amazons  Invented in 1988 by Argentinian Walter Zamkauskas.  Several programming competitions and yearly championships.  Distantly related to Go.  Active area in combinatorial game theory.

51. Amazons Board  Chess board 10 x 10  4 White and 4 Black chess Queens (Amazons) and Arrows  Starting configuration  White moves first

52. Amazons Rules  Each move consists of two steps: 1.Move an amazon of own color. 2.This amazon has to throw an arrow to an empty square where it stays.  Amazons and arrows move as a chess Queen as long as no obstacle blocks the way (amazon or arrow)  Players alternate moves.  Player who makes last move wins.

53. Amazons challenges  Absence of opening theory  Branching factor of more than 1000  Often 20 reasonable moves  Need for deep variations  Opening book >30,000 moves

54. A MAZONG  World’s best computer player by Jens Lieberum  Distinguishes three phases of the game:  Opening at the beginning  Middle game  Filling phase at the end See http://jenslieberum.de/amazong/amazong.html http://jenslieberum.de/amazong

55. Amazons Opening  Main goals in the opening  Obtain a good distribution of amazons  Build large regions of potential territory  Trap opponent’s amazons