1. Heuristic search, page 1 CSI 4106, Winter 2005 Heuristic search Points Definitions Best-first search Hill climbing Problems with hill climbing An example: local and global heuristic functions Simulated annealing The A* procedure Means-ends analysis

2. Heuristic search, page 2 CSI 4106, Winter 2005 Definitions Heuristics (Greek heuriskein = find, discover): "the study of the methods and rules of discovery and invention". We use our knowledge of the problem to consider some (not all) successors of the current state (preferably just one, as with an oracle). This means pruning the state space, gaining speed, but perhaps missing the solution! In chess: consider one (apparently best) move, maybe a few -- but not all possible legal moves. In the travelling salesman problem: select one nearest city, give up complete search (the greedy technique). This gives us, in polynomial time, an approximate solution of the inherently exponential problem; it can be proven that the approximation error is bounded.

3. Heuristic search, page 3 CSI 4106, Winter 2005 Definitions (2) For heuristic search to work, we must be able to rank the children of a node. A heuristic function takes a state and returns a numeric value -- a composite assessment of this state. We then choose a child with the best score (this could be a maximum or minimum). A heuristic function can help gain or lose a lot, but finding the right function is not always easy. The 8-puzzle: how many misplaced tiles? how many slots away from the correct place? and so on. Water jugs: ??? Chess: no simple counting of pieces is adequate.

4. Heuristic search, page 4 CSI 4106, Winter 2005 Definitions (3) The principal gain -- often spectacular -- is the reduction of the state space. For example, the full tree for Tic-Tac-Toe has 9! leaves. If we consider symmetries, the tree becomes six times smaller, but it is still quite large. With a fairly simple heuristic function we can get the tree down to 40 states. (More on this when we discuss games.) Heuristics can also help speed up exhaustive, blind search, such as depth-first and breadth- first search.

5. Heuristic search, page 5 CSI 4106, Winter 2005 Best-first search The algorithm select a heuristic function (e.g., distance to the goal); put the initial node(s) on the open list; repeat select N, the best node on the open list; succeed if N is a goal node; otherwise put N on the closed list and add N's children to the open list; until we succeed or the open list becomes empty (we fail); A closed node reached on a different path is made open. NOTE: "the best" only means "currently appearing the best"...

6. Heuristic search, page 6 CSI 4106, Winter 2005 Hill climbing This is a greedy algorithm: go as high up as possible as fast as possible, without looking around too much. The algorithm select a heuristic function; set C, the current node, to the highest-valued initial node; loop select N, the highest-valued child of C; return C if its value is better than the value of N; otherwise set C to N;

7. Heuristic search, page 7 CSI 4106, Winter 2005 Problems with hill climbing 1.Local maximum, or the foothill problem: there is a peak, but it is lower than the highest peak in the whole space. 2.The plateau problem: all local moves are equally unpromising, and all peaks seem far away. 3.The ridge problem: almost every move takes us down. Random-restart hill climbing is a series of hill- climbing searches with a randomly selected start node whenever the current search gets stuck. See also simulated annealing -- in a moment.

8. Heuristic search, page 8 CSI 4106, Winter 2005 A hill climbing example

9. Heuristic search, page 9 CSI 4106, Winter 2005 A hill climbing example (2) A local heuristic function Count +1 for every block that sits on the correct thing. The goal state has the value +8. Count -1 for every block that sits on an incorrect thing. In the initial state blocks C, D, E, F, G, H count +1 each. Blocks A, B count -1 each, for the total of +4. Move 1 gives the value +6 (A is now on the correct support). Moves 2a and 2b both give +4 (B and H are wrongly situated). This means we have a local maximum of +6.

10. Heuristic search, page 10 CSI 4106, Winter 2005 A hill climbing example (3) A global heuristic function Count +N for every block that sits on a correct stack of N things. The goal state has the value +28. Count -N for every block that sits on an incorrect stack of N things. That is, there is a large penalty for blocks tied up in a wrong structure. In the initial state C, D, E, F, G, H count -1, -2, -3, -4, -5, -6. A counts -7, for the total of -28. continued

11. Heuristic search, page 11 CSI 4106, Winter 2005 A hill climbing example (4) Move 1 gives the value -21 (A is now on the correct support). Move 2a gives -16, because C, D, E, F, G, H count -1, -2, -3, -4, -5, -1. Move 2b gives -15, because C, D, E, F, G count -1, -2, -3, -4, -5. There is no local maximum! Moral: sometimes changing the heuristic function is all we need.

12. Heuristic search, page 12 CSI 4106, Winter 2005 Simulated annealing The intuition for this search method, an improvement on hill-climbing, is the process of gradually cooling a liquid. There is a schedule of "temperatures", changing in time. Reaching the "freezing point" (temperature 0) stops the process. Instead of a random restart, we use a more systematic method.

13. Heuristic search, page 13 CSI 4106, Winter 2005 Simulated annealing (2) The algorithm, one of the versions select a heuristic function f; set C, the current node, to any initial node; set t, the current time, to 0; let schedule(x) be a table of "temperatures”; loop t = t + 1; if schedule(t) = 0, return C; select at random N, any child of C; if f(N) > f(C), set C to N, otherwise set C to N with probability e (f(N)-f(C))/schedule(t) ;

14. Heuristic search, page 14 CSI 4106, Winter 2005 A few other search ideas Stochastic beam search: select w nodes at random; nodes with higher values have a higher probability of selection. Genetic algorithms: generate nodes like in stochastic beam search, but from two parents rather than from one. (This topic is worthy of a separate lecture...)

15. Heuristic search, page 15 CSI 4106, Winter 2005 The A* procedure Hill-climbing (and its clever versions) may miss an optimal solution. Here is a search method that ensures optimality of the solution. The algorithm keep a list of partial paths (initially root to root, length 0); repeat succeed if the first path P reaches the goal node; otherwise remove path P from the list; extend P in all possible ways, add new paths to the list; sort the list by the sum of two values: the real cost of P till now, and an estimate of the remaining distance; prune the list by leaving only the shortest path for each node reached so far; until success or the list of paths becomes empty;

16. Heuristic search, page 16 CSI 4106, Winter 2005 The A* procedure (2) A* requires a lower-bound estimate of the distance of any node to the goal node. A heuristic that never overestimates is also called optimistic or admissible. We consider three functions with values ≥ 0: g(n) is the actual cost of reaching node n, h(n) is the actual unknown remaining cost, h'(n) is the optimistic estimate of h(n).

17. Heuristic search, page 17 CSI 4106, Winter 2005 The A* procedure (3) Here is a sketchy proof of the optimality of the goal node found by A*. Let m be such a non-optimal goal: g(m) > g(n k ) and let n 0, n 1,..., n k be a path leading to n k. Since m is a goal, h(m) = 0 and thus h'(m) = 0. Every step of A* selects the next n i in such a way that g(n i ) + h'(n i ) ≤ g(n i ) + h(n i ) = g(n k ) < g(m) = g(m) + h'(m) In other words, g(n i ) + h'(n i ) < g(m) + h'(m), so that A* cannot have selected m.

18. Heuristic search, page 18 CSI 4106, Winter 2005 The A* procedure (4) A simple search problem: S is the start node, G is the goal node, the real distances are shown. A lower-bound estimate of the distance to G could be as follows (note that we do not need it for F): A B C D E F G S 2 3 4 4 4 4 5 5 3.5 A B C D E F G S 5.8 3.4 9.2 11.5 10.1 7.1 3.5

19. Heuristic search, page 19 CSI 4106, Winter 2005 The A* procedure (5) Hill-climbing happens to succeed here: S A D A B F G S E D E 10.19.2 10.1 11.57.1 5.8 9.23.5 7.10.0

20. Heuristic search, page 20 CSI 4106, Winter 2005 The A* procedure (6) A*, step 1 S A D 3+10.14+9.2

21. Heuristic search, page 21 CSI 4106, Winter 2005 The A* procedure (7) A*, step 2 S A D B D 13.113.2 3+4+5.8 3+5+9.2

22. Heuristic search, page 22 CSI 4106, Winter 2005 The A* procedure (8) A*, step 3 S A D B C D E 13.113.2 12.8 17.2 3+4+4+3.4 3+4+5+7.1

23. Heuristic search, page 23 CSI 4106, Winter 2005 The A* procedure (9) A*, step 4 S A D B C E D A E 13.113.2 12.8 17.24+5+10.1 14.4 19.14+2+7.1

24. Heuristic search, page 24 CSI 4106, Winter 2005 The A* procedure (10) A*, step 5 S A D B C E F D A E B 13.113.2 12.8 17.219.1 14.4 19.113.1 4+2+5+5.84+2+4+3.5

25. Heuristic search, page 25 CSI 4106, Winter 2005 The A* procedure (11) A*, step 6 S A D B C E F G D A E B 13.113.2 12.8 17.219.1 14.4 19.113.1 16.813.5

26. Heuristic search, page 26 CSI 4106, Winter 2005 The A* procedure (12) The 8-puzzle with optimistic heuristics. A,..., N: states; m = misplaced; d = depth; f(x) = m(x) + d(x).

27. Heuristic search, page 27 CSI 4106, Winter 2005 Means-ends analysis The idea is due to Newell and Simon (1957): work by reducing the difference between states, and so approaching the goal state. There are procedures, indexed by differences, that change states. The General Problem Solver algorithm set C, the current node, to any initial node; loop succeed if the goal state has been reached, otherwise find the difference between C and the goal state; choose a procedure that reduces this difference, apply it to C to produce the new current state;

28. Heuristic search, page 28 CSI 4106, Winter 2005 Means-ends analysis (2) An example: planning a trip. miles PlaneTrainCarTaxiWalk > 1000X 100-1000XX 10-100XX < 1X PrerequisiteAt planeAt trainAt carAt taxi DistanceDistance P r o c e d u r e