1. Search CSE391 -2005 1 When you can’t use A* Hill-climbing Simulated Annealing Other strategies 2 person- games

2. Search CSE391 -2005 2 Hill-climbing Just keep current state Generate successors If best successor is better than current state, move to it. Otherwise, you’re stuck – local maxima.

3. Search CSE391 -2005 3 N-queens NxN board N queens Place the queens on the board so that all queens are safe. No queen is on the same row, column or diagonal as another queen.

4. Search CSE391 -2005 4 8 queens Initial state: no queens Successor function: add a queen to any empty square Goal: are all queens safe? 64 x 63 x 62 ….= 3 x 10 14 possible sequences

5. Search CSE391 -2005 5 8-queens – almost a solution, 4 steps

6. Search CSE391 -2005 6 8 queens Initial state: no queens Successor function: add a queen to a safe empty square Goal: are all queens safe? 2057 possible sequences

7. Search CSE391 -2005 7 Applying hill climbing to 8 queens Initial state: 8 queens placed randomly Successor function: move queen in its column (8 x 7 = 56 successors) 8 8 Goal state: all queens are safe Works 14% of the time, gets stuck 86%

8. Search CSE391 -2005 8 8-queens – a random start state

9. Search CSE391 -2005 9 8-queens – h = # of pairs of queens attacking each other 18121413 1214 16131512141216 14121813151214 1514 13161316 1417151416 17161815 181415 1416 14 131712141218

10. Search CSE391 -2005 10 8-queens – almost a solution, 4 steps Almost a solution - 5 moves

11. Search CSE391 -2005 11 Applying hill climbing to 8 queens Initial state: 8 queens placed randomly Successor function: move queen in its column (8 x 7 = 56 successors) 8 8 or allow up to 100 sideways moves… Goal state: all queens are safe Works 94% of the time, gets stuck 6%

12. Search CSE391 -2005 12 Hill-climbing Local maxima: a peak which is higher than each of its neighbors but lower than the global maximum. Ridge: a sequence of local maxima Plateau: function value is flat –Flat local maxima, no uphill exit –Shoulder, can find uphill path

13. Search CSE391 -2005 13 Hill-climbing Just keep current state Generate successors If best successor is better than current state, move to it. Otherwise, you’re stuck – local maxima. Random restart.

14. Search CSE391 -2005 14 Simulated annealing Modeled after “annealing” in metallurgy –Heated metal cools slowly so that crystals can form Closer to a purely random walk Starts by picking random moves (high temp) As the “temperature” cools, shifts to preferring “best” moves. Applications: –VLSI layouts, factory scheduling, airline scheduling

15. Search CSE391 -2005 15 Other strategies Local beam search –Same as hill-climbing, but keeps k nodes in memory instead of just 1. Genetic algorithms –Generate a successor by combining two parent states, instead of just modifying one. –Apply random mutations –Evaluate with “fitness function”

16. Search CSE391 -2005 16 2 person games – more complex Opponent introduces non-determinism –Minimax, alpha-beta Space and time limitations can introduce inaccessibility

17. Search CSE391 -2005 17 Problem Formulation Initial state Operators Terminal test (goal state) Utility function (payoff) –Minimax: back up from terminal state, high values for max, low values for min

18. Search CSE391 -2005 18 Tic-Tac-Toe Initial state –Representation? matrix Successor functions –Placing x’s and o’s Goal states –Explicit Utility function - Minimax x o x o x x o x x x x o

19. Search CSE391 -2005 19 Straightforward Mimimax Utility function (payoff) –Minimax: back up from terminal state, high values for max, low values for min

20. Search CSE391 -2005 20 x x x o x x o x x o x x o x o x x o x o x x o x x o x o x o x o x x o x o x x o x o x o x x o x x x o x x o o x x o x o x x o x o Tic-Tac-Toe x x o x o x o x x x o o x 0 0 00 1 0 1 0 1 0 1 1

21. Search CSE391 -2005 21 A More “Intelligent” Minimax Backing up wins and losses –Requires entire search tree – Often infeasible Can use heuristic estimates, e(p), instead –pick “best next move” based on limited search –After opponent’s move, extend search further and estimate again

22. Search CSE391 -2005 22 Tic-Tac-Toe Evaluation Function If p is a win for MAX then e(p) = If p is a win for MIN then e(p) = - Else h(n) = ( X’s # of complete open rows,columns, diagonals - O’s # of complete open rows, columns,diagonals) e(p) = 6 - 4 = 2 o x

23. Search CSE391 -2005 23 Heuristic Minimax for Tic-Tac-Toe x o x o x x o ox ox x o x x o xoxo x o xo xo x o x o x o x 6-5=1 5-5=0 4-5=-1 5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2 5-4=1 6-4=2 -2 1 1

24. Search CSE391 -2005 24 o x o x o x o x x o x x o x o x o x o x o o x x o x o o x x o x o x x o o x o x o o x x o x o x o o x o x o x o x o o o x o x o x x o o x x o o o x x o x o x 4-2=2 3-2=1 5-2=3 3-2=1 4-2=2 3-2=1 4-3=1 3-3=0 5-3=2 3-3= 0 4-3=1 4-3=1 4-2=2 4-2=2 5-2=3 3-2=1 4-2=2 4-2=2 4-3=1 4-3=1 3-3= 0 1 0 1 0 1 Minimax - 2

25. Search CSE391 -2005 25 Minimax-3 o o x 2-1=1 3-1=2 - = - 2-2=0 2-2=0 3-2=1 -=- 2-1=1 2-1=1 2-1=1 1 - - 1 x o o x o o x x o o o x x x o o x X o x o o x x o x o o x o x o o x o x o o x x x o o o o x x o o o x o o o x x x o o x x x o o o x x x o o o x x x o o o x x x - -

26. Search CSE391 -2005 26 Alpha/Beta The value of a MAX node = current largest final backed-up value of its successors The value of a MIN node = current smallest final backed-up value of its successors

27. Search CSE391 -2005 27 Alpha/Beta Pruning Stop search below any MIN node where Stop search below any MAX node where

28. Search CSE391 -2005 28 Alpha/Beta for Tic-Tac-Toe x o x o x x o ox ox x o x x o xoxo x o xo xo x o x o x o x 6-5=1 5-5=0 4-5=-1 5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2 5-4=1 6-4=2 MIN=-1, BETA=-1 MIN=-2, BETA=-1 MIN=1, BETA=1 MAX=1, ALPHA= -1 X X X X ALPHA= 1

29. Search CSE391 -2005 29 Alpha/Beta Pruning o o x 2-1=1 3-1=2 - = - 2-2=0 2-2=0 3-2=1 x o o x o o x x o o o x x x o o x X o x o o x x o x o o x o x o o x o x o o x x x o o o o x x o o o x XX X o o x x x o o o x x x X X X - = - MIN= -, BETA= - MIN= - BETA= - MIN=1, BETA=1 MAX=1 ALPHA=1