Adversarial Games

Two Flavors  Perfect Information –everything that can be known is known –Chess, Othello  Imperfect Information –Player’s have each have partial knowledge –Poker: dispute is settled by revealing the contents of one’s hand

Two Approaches to Perfect Information Games  Use simple heuristics and search many nodes  Use sophisticated heuristics and search few nodes Cost of calculating the heuristics might outweigh the cost of opening many nodes The closer h is to h*, the better informed it is. But information can be expensive

MiniMax on exhaustively searchable graphs Two Players  min: tries to achieve an outcome of 0  max: tries to achieve an outcome of 1

You are max at node A Expand the entire search space B A H F CD K G O E J N PQ R I LM MAX Max Min Max

Wins  Max: F,J,N,Q,L  Min: D,H,P,R,M Propagating Scores: A first pass  Min’s Turn a Node I –Go to M to win –So assign I a 0  Max’s turn at node O –Go to Q to win –So assign 1 to node O

Conclusion 1. If faced with two labeled choices, you would choose 0 (if min) or 1 (if max) 2. Assume you’re opponent will play the same way

Propagating Scores For each unlabeled node in the tree  If it’s max’s turn, give it the max score of its children  If it’s min’s turn, give it the min score of its children Now label the tree Conclusion: Max must choose C at the first move or lose the game

Nim  7 coins  2 players  Players divide coins into two piles at each move, such that –Piles have an unequal number of coins –No pile is empty  Play ends when a player can no longer move

Start of Game 7 6,1 5,2 4,3 min Complete the game to see that min wins only if max makes a mistake at 6,1 or 5,2 5,1,1 4,2,1

α-β Pruning Problem  For games of any complexity, you can’t search the whole tree Solution  Look ahead a fixed number of plys (levels)  Evaluate according to some heuristic estimate  Develop a criterion for pruning subtrees that don’t require examination

Recast Instead of representing wins, numbers represent the relative goodness of nodes. At any junction  Max chooses highest  Min chooses lowest Higher means better for max Lower means better for min

Example: Max’s Turn j mz g k n t pq r Max Min Alpha beta

Situation Max’s turn at node g  Left subtree of g has been explored  If max chooses j, min will choose m  So the best max can do by going left is 8. Call this α Now Examine K and its left subtree n with a value of 4  If max chooses k, the worst min can do is 4.  Because, T may be < 4. If it is min will choose it. If not, min will choose 4  So the worst min can do, if max goes right is 4. Call this β

Question Should max expand the right subtree of k. No. Because min is guaranteed 4. But if max chose j, min is only guaranteed 8. Val(k) = min(4, val(t)) <= 4 Val(g) = max(8,val(k)) = max(8, min(4,val(t)) = max(8, min(4,val(t)) = 8 = 8

Leads to Max Principle Search can be stopped below any min node where β <= α of its max ancestor

Example: Min’s Turn n de k t pqr min Min alpha Beta

Situation Min’s turn at node k  Left subtree of k has been explored  If min chooses n, max will choose d  So the best min can do by going left is 4. Call this β Now examine T and its left subtree P with a value of 7  If min chooses T, the worst max can do is 7.  Because, Q or R may be > 7. If it is Max will choose it. If not, min will choose 7.  So the worst max can do, if min goes right is 7. Call this α

Question  Should min explore Q and R  No  Max is guaranteed 7 if min chooses T  But if min chooses N, max gets only 4  Val(T) = max(7,val(Q), val(R)) >= 7  Val(k) = min(4,val(T)) = 4

Leads to min principle Search can be stopped below any max node where α >= β of its min ancestor

To Summarize  Max’s turn –β is min’s guaranteed score (the worst min can do) –α is best max can do  Max principle –Search can be stopped below any min node where β <= α of its max ancestor  Min’s turn –α is max’s guaranteed score (the worst max can do) –β is best min can do  Min principle –Search can be stopped below any max node where α >= β of its min ancestor

Efficiency  Suppose a tree has depth, D, and branching factor B D = 2, B = 2: Terminal Nodes = 2 2 In general, Terminal Nodes = B D

Ordering  Ordering of nodes in a tree clearly affects the number than can be pruned using alpha/beta.  Call N D the number of terminal nodes  Can be shown that with alpha/beta best case performance is: N D = 2B D/2 – 1 for even D N D = 2B (D+1)/2 + B (D-1)/2 for odd D

Example Suppose B = 5, D = 6 w/out alpha/beta N D = 5 6 = w/alpha/beta N D = 2* = 249 Approx. 1.6% of worst case

Reduction in Branching Factor

Average Performance On average, the peformance reduces B to B 3/4 Suppose B = 5, B ab = 3.34 Suppose D = 6 Then N D = 5 6 = N DAB = = 1388 ≈ 8.8% of worst case

Combinatorial Explosion  Key Idea: branching factor makes optimal solution intranctable –Sum of Subsets Problem  2 –Traveling Salesperson  (N + 1)/2 –8 puzzle  2.67 –15 puzzle  Approximately 4 Let B = average branching factor Let T = total nodes Let D = depth of search Then T = B + B 2 + B 3 + … + B D = B(B D – 1)/(B – 1) + 1

Sum of Subsets Given a set, S, of positive integers, find all subsets whose sum is m. E.G. S = {7,11,13,24} m = 31 Solutions S-1 = {7,11,13} S-2 = {7,24}

Problem Representation Solution is a sequence of 1s and 0s, indicating that elements of S have been chosen or not. Rep of S-1 (1,1,1,0) Rep of S-2 (1,0,0,1) State space is a tree where left turn indicates a 1 and right turn indicates a 0

Partial Space S-1 Three left turns and a right turn to get to S-1 is the sequence (1,1,1,0) Clearly the branching factor is 2

TSP Can also be represented as a state space search. Suppose 4 cities 4 Choices at level 0 3 Choices at level 1 2 Choices at level 2 1 Choice at level 3 T = 4*3*2*1 = 4 P 4 = 4!/(4-4)! = 4!

Branching Factor B_F = (sum of choices at each level)/#of levels = ( )/4 = 2.5 = ( )/4 = 2.5 Clearly this increases with the size of the tour: (1+2+3+…+n)/n = (n(n+1)/2)/n = (n+1)/2

Relationship between branching factor and nodes in the tree Whenever the branching factor >= 2, we have an exponentially complex problem Let T = number of nodes in a full binary tree T = … + 2 d-1 = 2 d – 1 Easily proved through induction

Replace 2 by branching factor, B T = B 0 +B 1 + B 2 + … + B L = B(B L -1)/(B-1) + 1 = B(B L -1)/(B-1) + 1 Where L is d-1, d being the depth of the tree Proof Basis: T = B(B 0 – 1)/(B-1) + 1 = 1 Inductive hypothesis: T = B 0 +B 1 + B 2 + … + B L = B(B L -1)/(B-1) + 1

Show that this is true at level L+1 That is, Show = B 0 +B 1 + B 2 + … + B L+1 = B(B L+1 -1)/(B-1) + 1 B 0 +B 1 + B 2 + … + B L + B L+1 = B(B L -1)/(B-1) + B L = (B(B L -1) + B L+1 (B-1))/(B-1) + 1 %common D = (B( (B L -1) + B L (B-1))/(B-1) + 1 %factor B out = (B(B L -1 + B L+1 –B L )/(B-1) + 1 %multiply b = B(B L+1 -1)/(B-1) + 1 %subtract Which is what we were trying to prove

Two Concepts 1. B – average number of descendents that emerge from any state in the space 2. Total nodes = B(B L -1)/(B-1) + 1 Where L is the deepest level (or, the depth of the search)

Another Problem: 8 Puzzle ABC A: 4 moves for blank * 1 position = 4 B: 2 moves for blank * 4 positions = 8 C: 3 moves for blank * 4 positions = 12 B = ( ) /( ) = 2.67 Does the branching factor of larger (15, 24) puzzles approach 4 as the puzzles get larger?