State Space 4 Chapter 4 Adversarial Games

Two Flavors Games of Perfect Information Each player knows everything that can be known Chess, Othello Games of Imperfect Information Player’s have partial knowledge Poker: dispute is settled by revealing the contents of one’s hand Two Flavors

Two Approaches to Perfect Information Games Use simple heuristics and search many nodes Use sophisticated heuristics and search few nodes Cost of calculating the heuristics might outweigh the cost of opening many nodes The closer h is to h*, the better informed it is. But information can be expensive Two Approaches to Perfect Information Games

A Model: MiniMax on exhaustively searchable graphs Two Players min: tries to achieve an outcome of 0 max: tries to achieve an outcome of 1 A Model: MiniMax on exhaustively searchable graphs

You are max at node A MAX A C D Min B 1 Max G E F Min K 1 J H I Max O C D Min B 1 Max G E F Min K 1 J H I Max O N 1 1 L M P Q R Min 1

Wins Max would like to end game at F,J,N,Q,L Min would like to end game at D,H,P,R,M Propagating Scores: A first pass Min’s Turn at Node I Go to M to win So assign I a 0 Max’s turn at node O Go to Q to win So assign 1 to node O Wins

If faced with two labeled choices, you would choose 0 (if min) or 1 (if max) Assume you’re opponent will play the same way Conclusion

Propagating Scores For each unlabeled node in the tree If it’s max’s turn, give it the max score of its children If it’s min’s turn, give it the min score of its children Now label the tree Conclusion: Max must choose C at the first move or lose the game Propagating Scores

7 coins 2 players Players divide coins into two piles at each move, such that Piles have an unequal number of coins No pile is empty Play ends when a player can no longer move. At that point the other player wins and his/her score bubbles up Nim

Start of Game min 7 4,3 5,2 6,1 5,1,1 4,2,1 P. 152 Luger.

Problem For games of any complexity, you can’t search the whole tree Solution Look ahead a fixed number of plys (levels) Evaluate according to some heuristic estimate Develop a criterion for pruning subtrees that don’t require examination αβ Pruning

Instead of representing wins, numbers represent the relative goodness of nodes. At any junction Max chooses highest Min chooses lowest Higher means better for max Lower means better for min Recast

What should Max do? 8 Max g 4 8 j k 9 t n m z 4 8 12 r p q Min 7 3 9 beta n m z alpha 4 8 12 r p q Min 7 3 9

Situation Max’s turn at node g Left subtree of g has been explored If max chooses j, min will choose m So the best max can do by going left is 8. Call this α Now Examine K and its left subtree n with a value of 4 If max chooses k, the worst min can do is 4. Why? T may be < 4. If it is min will choose it. If not, min will choose 4 So the worst min can do, if max goes right is 4. Call this β Situation

Question Must max expand the rst(K)? No. min is guaranteed 4. But if max chose j, min is guaranteed 8 So max is better off by going left More formally: Val(k) = min(4, val(t)) <= 4 Val(g) = max(8,val(k)) = max(8, min(4,val(t)) = 8 Question

Max Principle If you’re Max: Search can be stopped below any min node where β <= α of its max ancestor Max Principle

What should Min do? Min 4 k 9 4 n t d e p q r Beta 4 3 7 9 alpha 3

Situation Min’s turn at node k Left subtree of k has been explored If min chooses n, max will choose d So the best min can do by going left is 4. Call this β Now examine T and its left subtree P with a value of 7 If min chooses T, the worst max can do is 7. Why? Q or R may be > 7. If either is Max will choose one of them. If not, max will choose 7. So the worst max can do, if min goes right is 7. Call this α Situation

Question Should min explore Q and R No Max is guaranteed 7 if Min chooses T But if min chooses N, max gets only 4 Val(T) = max(7,val(Q), val(R)) >= 7 Val(k) = min(4,val(T)) = 4 Question

Min Principle If you’re min: Search can be stopped below any max node where α >= β of its min ancestor Min Principle

To Summarize Max’s turn Max principle Min’s turn Min principle β is min’s guaranteed score (the worst min can do) α is best max can do Max principle Search can be stopped below any min node where β <= α of its max ancestor Min’s turn α is max’s guaranteed score (the worst max can do) β is best min can do Min principle Search can be stopped below any max node where α >= β of its min ancestor To Summarize

On White Board A Few Examples

AB Prune (Nilsson, p. 205) Similarity between 2 and 2’ Arguments for min/min principles (Slides 15, 19) AB Prune (Nilsson, p. 205)

Need for A-B prune Combinatorial Explosion Number of nodes produced by a full search of the chess state space: 10120 Not just a big number Comparable to Number of molecules in the universe Number of nanoseconds since the big bang Need for A-B prune

Best Case Performance Call ND the number of terminal nodes D the depth of the search space ND the number of terminal nodes B the branching factor Best case AB performance: ND = 2BD/2 – 1 for even D ND = 2B(D+1)/2 + B(D-1)/2 for odd D Best Case Performance

Suppose B = 5, D = 6 w/out alpha/beta ND= 56 = 15625 w/alpha/beta ND= 2* 53 - 1 = 249 Approx. 1.6% of the terminal nodes that would have been examined without AB prune Example

Average Performance AB prune reduces branching factor B to B3/4 Suppose B = 5, Bab = 3.34 Suppose D = 6 Then ND = 56 = 15625 NDAB = 3.346 = 1388 ≈ 8.8% without AB prune Average Performance

Clear relationship between branching factor and the size of the search space Let T = number of nodes in a full binary tree T = 20 + 21 + 22 + … + 2D-1 = 2D – 1 Easily proved through induction Binary Trees

T = B0 +B1 + B2 + … + BD = B(BD - 1)/(B-1) + 1 Also provable through induction Extend to arbitrary B

To Sum Up D is the depth of the search space B is the average number of descendants at each level Size of search space = B(BD -1)/(B-1) + 1 Grows very fast As branching factor increases As depth increases Combinatorial Explosion: search space grows too fast to be exhaustively searched But we want to search deeply (large D) Conclusion: reduce B through AB pruning To Sum Up