1. Informed Search

2. S B AD E C F G 1 20 2 3 48 61 1 straight-line distances h(S-G)=10 h(A-G)=7 h(D-G)=1 h(F-G)=1 h(B-G)=10 h(E-G)=8 h(C-G)=20 The graph above shows the step-costs for different paths going from the start (S) to the goal (G). 1.Draw the search tree for this problem. 2.Give the order in which the tree is searched (e.g. S-C-B...-G), solution path, solution cost, and no. of nodes expanded for the following search algorithms: a.Breadth-first search, b.Depth first search, c.Uniform cost search, d.Iterative deepening search e.Greedy best-first search f.A* search.

3. Dealing with hard problems For large problems, A* may require too much space Two variations conserve memory: IDA* and SMA* IDA* -- iterative deepening A* -- uses successive iteration with growing limits on f, e.g. –A* but don’t consider any node n where f(n) >10 –A* but don’t consider any node n where f(n) >20 –A* but don’t consider any node n where f(n) >30,... SMA* -- Simplified Memory-Bounded A* –uses a queue of restricted size to limit memory use.

4. IDA* Iterative deepening: Repeat depth-first search within increasing depth limit IDA*: Repeat depth-first search within increasing f-limit Iterative-deepening-A* (IDA*) works as follows: At each iteration, perform a depth-first search, cutting off a branch when its total cost (g + h) exceeds a given threshold. This threshold starts at the estimate of the cost of the initial state, and increases for each iteration of the algorithm. At each iteration, the threshold used for the next iteration is the minimum cost of all values that exceeded the current threshold.

5. f-values form relief Start f0 f1 f2 f3 IDA*: Repeat depth-first within f-limit increasing: f0, f1, f2, f3,...

6. Simple Memory-bounded A* (SMA*) 24+0=24 A BG CD EF H J I K 0+12=12 10+5=15 20+5=25 30+5=35 20+0=20 30+0=30 8+5=13 16+2=18 24+0=2424+5=29 108 816 88 f = g+h (Example with 3-node memory) Progress of SMA*. Each node is labeled with its current f-cost. Values in parentheses show the value of the best forgotten descendant.  = goal Search space maximal depth is 3, since memory limit is 3. This branch is now useless. best forgotten node A 12 A B 15 A BG 13 1513 H  A G 18 13[15] A G 24[  ] I 15[15] 24 A BG 15 24  A B C 15[24] 15 25 A B D 8 20 20[24] 20[  ] best estimated solution so far for that node

7. SMA* pseudocode function SMA*(problem) returns a solution sequence inputs: problem, a problem static: Queue, a queue of nodes ordered by f-cost Queue  MAKE-QUEUE({MAKE-NODE(INITIAL-STATE[problem])}) loop do if Queue is empty then return failure n  deepest least-f-cost node in Queue if GOAL-TEST(n) then return success s  NEXT-SUCCESSOR(n) if s is not a goal and is at maximum depth then f(s)   else f(s)  MAX(f(n),g(s)+h(s)) if all of n’s successors have been generated then update n’s f-cost and those of its ancestors if necessary if SUCCESSORS(n) all in memory then remove n from Queue if memory is full then delete shallowest, highest-f-cost node in Queue remove it from its parent’s successor list insert its parent on Queue if necessary insert s in Queue end

8. Iterative improvement In many optimization problems, path is irrelevant; the goal state itself is the solution. Then, state space = space of “complete” configurations. Algorithm goal: - find configuration satisfying constraints (e.g., n-queens) In such cases, can use iterative improvement algorithms: keep a single “current” state, and try to improve it. 8

9. Local search “incremental improvement” search approach to search involves starting with an initial guess at a solution and gradually improving it until it is one Some examples: –Generate-and-Test –Hill climbing –Simulated annealing

10. Generate-and-Test Algorithm 1.Generate a possible solution. 2.Test to see if this is actually a solution. 3.Quit if a solution has been found. Otherwise, return to step 1. 10

11. Generate-and-Test Acceptable for simple problems. Inefficient for problems with large space. 11

12. Hill Climbing Is a variant of generate-and test in which feedback from the test procedure is used to help the generator decide which direction to move in search space. The test function is augmented with a heuristic function that provides an estimate of how close a given state is to the goal state. Computation of heuristic function can be done with negligible amount of calculation. Hill climbing is often used when a good heuristic function is available for evaluating states but when no other useful knowledge is available.

13. Hill Climbing Searching for a goal state = Climbing to the top of a hill Generate-and-test + direction to move. Heuristic function to estimate how close a given state is to a goal state. 13

14. Simple Hill Climbing Algorithm 1.Evaluate the initial state. 2.Loop until a solution is found or there are no new operators left to be applied:  Select and apply a new operator  Evaluate the new state: goal  quit better than current state  new current state 14

15. Steepest-Ascent Hill Climbing (Gradient Search) Considers all the moves from the current state. Selects the best one as the next state. 15

16. Steepest-Ascent Hill Climbing (Gradient Search) Algorithm 1.Evaluate the initial state. 2.Loop until a solution is found or a complete iteration produces no change to current state:  SUCC = a state such that any possible successor of the current state will be better than SUCC (the worst state).  For each operator that applies to the current state, evaluate the new state: goal  quit better than SUCC  set SUCC to this state  SUCC is better than the current state  set the current state to SUCC. 16

17. Hill climbing on a surface of states Height Defined by Evaluation Function

18. Hill-climbing search If there exists a successor s for the current state n such that –h(s) < h(n) –h(s) <= h(t) for all the successors t of n, then move from n to s. Otherwise, halt at n. Looks one step ahead to determine if any successor is better than the current state; if there is, move to the best successor. Like Greedy search in that it uses h, but doesn’t allow backtracking or jumping to an alternative path since it doesn’t “remember” where it has been.

19. Hill Climbing Extended the current path with a successor node which is closer to the solution than the end of the current path If our goal is to get to the top of a hill, then always take a step the leads you up Simple hill climbing – take any upward step Steepest ascent hill climbing – consider all possible steps, and take the one that goes up the most

20. Hill climbing example 283 164 7 5 283 14 765 23 184 765 13 84 765 2 3 184 765 2 13 8 4 765 2 start goal -5 h = -3 h = -2 h = -1 h = 0 h = -4 -5 -4 -3 -2 f(n) = -(number of tiles out of place) 13 8 4 765 2

21. Use the Manhattan Distance heuristic to find a solution to the 8-puzzle. 123 48 76 5 123 456 78 goal start

22. 5 64 3 4 2 1 3 3 0 2 We can use heuristics to guide search. In this hill climbing example, the Manhattan Distance heuristic helps us quickly find a solution to the 8- puzzle. h(n) 123 45 78 6 123 45 78 6 13 425 78 6 12 453 78 6 123 456 78 123 45 78 6 123 485 7 6 123 485 7 6 123 485 76 12 483 76 5 123 48 76 5 goal 123 456 78

23. Example of a local optimum 125 74 863 4 123 8 765 125 74 863 25 174 863 125 874 63 -4 -5 -4 0 startgoal

24. Hill-climbing This simple policy has three well-known drawbacks: 1. Local Maxima: a local maximum as opposed to global maximum. 2. Plateaus: An area of the search space where evaluation function is flat, thus requiring random walk. 3. Ridge: Where there are steep slopes and the search direction is not towards the top but towards the side. (a) (b) (c) Local maxima, Plateaus and ridge situation for Hill Climbing

25. Hill-climbing In each of the previous cases (local maxima, plateaus & ridge), the algorithm reaches a point at which no progress is being made. A solution is to do a random-restart hill-climbing - where random initial states are generated, running each until it halts or makes no discernible progress. The best result is then chosen. Random-restart hill-climbing

26. Simulated Annealing A alternative to a random-restart hill-climbing when stuck on a local maximum is to do a ‘reverse walk’ to escape the local maximum. This is the idea of simulated annealing.

27. Simulated annealing: basic idea From current state, pick a random successor state; If it has better value than current state, then “accept the transition,” that is, use successor state as current state; Otherwise, do not give up, but instead flip a coin and accept the transition with a given probability (that is lower as the successor is worse). So we accept to sometimes “un-optimize” the value function a little with a non-zero probability. 27

28. Simulated annealing Kirkpatrick et al. 1983: Simulated annealing is a general method for making likely the escape from local minima by allowing jumps to higher energy states. The analogy here is with the process of annealing used by a craftsman in forging a sword from an alloy. He heats the metal, then slowly cools it as he hammers the blade into shape. –If he cools the blade too quickly the metal will form patches of different composition; –If the metal is cooled slowly while it is shaped, the constituent metals will form a uniform alloy. 28

29. Simulated Annealing Step 1: Initialize – Start with a random initial placement. Initialize a very high “temperature”. Step 2: Move – disturb the placement through a defined move. Step 3: Calculate score – calculate the change in the score due to the move made. Step 4: Choose – Depending on the change in score, accept or reject the move. The prob of acceptance depending on the current “temperature”. Step 5: Update and repeat– Update the temperature value by lowering the temperature. Go back to Step 2. The process is done until “Freezing Point” is reached.

30. Simulated annealing algorithm Idea: Escape local extrema by allowing “bad moves,” but gradually decrease their size and frequency. 30 Note: goal here is to maximize E. -

31. Simulated annealing algorithm Idea: Escape local extrema by allowing “bad moves,” but gradually decrease their size and frequency. 31 Algorithm when goal is to minimize E. < - -

32. Differences The algorithm for simulated annealing is slightly different from the simple-hill climbing procedure. The three differences are: –The annealing schedule must be maintained –Moves to worse states may be accepted –It is good idea to maintain, in addition to the current state, the best state found so far.

33. Summary Best-first search = general search, where the minimum-cost nodes (according to some measure) are expanded first. Greedy search = best-first with the estimated cost to reach the goal as a heuristic measure. - Generally faster than uninformed search - not optimal - not complete. A* search = best-first with measure = path cost so far + estimated path cost to goal. - combines advantages of uniform-cost and greedy searches - complete, optimal and optimally efficient - space complexity still exponential –IDA* and SMA* reduce the memory requirements of A*. 33

34. Summary Time complexity of heuristic algorithms depend on quality of heuristic function. Good heuristics can sometimes be constructed by examining the problem definition or by generalizing from experience with the problem class. Iterative improvement algorithms keep only a single state in memory. Can get stuck in local extrema; simulated annealing provides a way to escape local extrema, and is complete and optimal given a slow enough cooling schedule. 34