1. CSCI 4310 Lecture 6: Adversarial Tree Search

2. Book Winston Chapter 6

3. Adversaries Many games involve two players Competing Agents Previous Trees were exploring search space

4. Adversaries Now, alternating levels of the tree represent each player P 1 decision P 2 decision P 1 decision

5. Adversaries Making choices based on scoring function for each node Ideally, fully expand the tree, follow the choice that leads you to the best subtree

6. Adversaries A completed game

7. Scoring path outcomes Domain specific We need expert advice Tic-tac-toe is easy Maximize winning possibilities X This position has a score of 3 for X

8. Scoring path outcomes Other games can become much more difficult or no apparent best strategy. X This position has a score of 4 for X

9. Scoring path outcomes Tuning parameters are important Piece count – often underperforms Not effective in chess, unless you are playing against a chimp Positional strategy – more involved Works well in Othello Moriarty paper on NN Othello play Computers crush the best human Othello players

10. Scoring path outcomes Make it fast – we will do this often Every potential game (along a path we have not pruned) will need to be evaluated by score function

11. Minimax P 1 is maximizing Maximizer (p1) must take p2’s decisions into account Min = 2Min = 1 P 1 decision - MAX P 2 decision - MIN P 1 decision - MAX s g1 =2s g2 =7s g3 =1s g4 =8

12. Minimax In practice, we limit the search depth Assuming, if we are p max, other player will minimize Can lead to poor play against poor players Assuming (if I am maximizing) that the opponent is minimizing is sometimes risky

13. Zugzwang from wikipediawikipedia A situation where someone is forced to act, but would prefer not to. In chess, zugzwang occurs when one player is put at a disadvantage because he has to make a move.

14.  -  pruning We do not need to evaluate every possibility to the leaves of the tree Book: If you have an idea that is surely bad, do not take time to see how truly awful it is

15.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 872 DFS starts by exploring leftmost child of each node Scoring function assigns values to children

16.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 872 Maximizer will choose 8 at this node

17.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 89 8729 This is already better for maximizer than the 8 of the sibling nodes. But…

18.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 89 8729 The minimizer will never choose this path over the 8. We need not score these nodes

19.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 89 8729 Minimizer one level above only cares if this is < 8.

20.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894 4 8729241 It is - after the first node (value = 2), so we proceed Minimizer will choose 4 here – will maximizer at root?

21.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 8945 4 8729241135 We need to score the leftmost branch of the center subtree for a baseline

22.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 89459 4 872924113539 Once we hit 9, minimizer at level above will never choose this node – no need to evaluate siblings

23.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 89459 4 872924113539 Minimizer one level above wants a max of 4 from the children of this node

24.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396 Oops – we can ignore siblings

25.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396 Oops – we can ignore siblings

26.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396 Maximizer at root wants ≥ 6, which means…

27.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396 Minimum of these siblings must be ≥ 6 Which means…

28.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396123 Maximum of these groups must be ≥ 6 Didn’t work, so…

29.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 894596 4 5 8729241135396123 Discontinue further evaluation

30.  -  pruning MAX LEVEL MIN LEVEL MAX LEVEL 8945963 4 5 3 8729241135396123 Maximizer will follow middle path. Alpha-Beta prunes 11/27 40% This minimax search is evaluating 3 levels

31.  -  pruning Effectiveness of alpha-beta pruning dependent on the ordering of successor nodes in the minimax DFS. We can get lucky from the perspective of min or max.

32.  -  pruning If we manage to get the perfect ordering of successors Minimax O(b d ) Branching factor b Search depth d Minimax with alpha-beta Ideal branching factor  b O(b d/2 ) perfect O(b 3d/4 ) realistic Computational savings may allow larger d in simulations

33.  -  pruning Recursive algorithm on p. 110 Or, a nice explanation with application to Othello Or, a nice explanation with application to Othello

34. Horizon effect Not seeing the big picture Going for quick points may lead to a state that was worse than the original even with the extra points We did not search deep enough Did not have time

35. Horizon effect Problem occurs in many algorithms Attempting to avoid local optima may be considered the defining characteristic of many algorithms Neural networks can be overtrained GA’s can converge too soon Greedy search problems Etc.

36. Horizon effect P. 114 describes heuristics that attempt to mitigate the horizon effect

37. McAllester’s conspiracy number Nodes that, if changed, could cause different choices higher in the search are expanded for further evaluation David Allen McAllester, Conspiracy numbers for min-max search, Artificial Intelligence, v.35 n.3, p.287-310, July 1988