1. CS 415 – A.I. Slide Set 6

2. Chapter 4 – Heuristic Search Heuristic – the study of the methods and rules of discovery and invention State Space Heuristics –  Formalized as rules for choosing those branches in a state space that are most likely to lead to an acceptable problem solution Apply Heuristics When: 1. A problem is ambiguous and may not have an EXACT solution 2. The computational cost of finding an exact solution is prohibitive

3. Simplifying Tic-Tac-Toe Brute Force – 9! total states  First use symmetry to reduce the number of states (see next slide)‏  7x12 states  Simple Heuristic  Put X where it has the most winning possibilities  See Figure 4.2 and Figure 4.3  This “prunes” the search space  In the first level, pick one of three, ignore the subtrees for the other two  Approx. 25 total states to consider

4. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.1First three levels of the tic-tac-toe state space reduced by symmetry 2

5. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.2The “most wins” heuristic applied to the first children in tic-tac-toe. 3

6. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.3Heuristically reduced state space for tic-tac-toe. 4

7. Hill-Climbing and Dynamic Programming Hill-Climbing  Expand the current state and evaluate its children  Pick the best child Mountain climbing  Go as steep as you can for as long as you can. Problem:  Keeps no backtracking info  Becomes stuck at “local maxima”  Example:  8-square problem

8. Dynamic Programming  Don’t solve subproblems multiple times  Instead, keep track of the solutions to these subproblems  Memoizing – subproblem solution caching Two Examples 1. Finding an optimal global alignment 2. Finding the minimum edit distance between two strings

9. Optimal Global Alignment Small Example  String #1  BAADDCABDDA  String #2  BBADCBA Rules  Cannot change order of respective elements  Can have spaces between elements Possible solutions  How do we figure out optimal solution?

10. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.5The initialization stage and first step in completing the array for character alignment using dynamic programming. 6 Let’s create a matrix The value of each element reflects the global alignment success to that point in the matching process Our “cost” scheme 1 – if an element has to be shifted along the shorter string for better alignment Cost recorded in col 1 – if a new character is inserted Cost recorded in row 2 – different characters, shift and insert cost 2 0 – if elements are identical

11. The forward stage Fill the array from the upper left corner  The value of (x,y) is a function (min cost) of either  (x-1,y), (x-1,y-1), or (x, y-1)‏  If there is a match  Add 0 to (x-1,y-1)‏  If there is no match  Add 2 to (x-1,y-1)‏  If we shift  Add 1 to the previous column  If we insert a character  Add 1 to the previous row

12. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.6The completed array reflecting the maximum alignment information for the strings. 7

13. The backward stage Once the matrix is filled  From the best alignment count, we produce a specific alignment of characters  Begin at the lower right-hand corner  Move back through the matrix  Each step, select one of the immediate state’s predecessors  (previous diagonal, row, or column)‏  Choose the minimum

14. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.7A completed backward component of the dynamic programming example giving one (of several possible) string alignments. 8

15. Example #2 – Min Edit Distance Spell Checker  What words from our dictionary best approximate a word we do not recognize (misspelled word)‏  We need to know the “distance” between two words Minimum edit distance  The number of insertions, deletions and replacements to turn the source word into the target word

16. intention – source word execution – target word Our “cost”  1 for a character insertion or deletion  2 for a replacement (deletion + insertion)‏

17. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.8Initialization of minimum edit difference matrix between intention and execution (adapted from Jurafsky and Martin, 2000). 9 Beginning state a sequence of insertions is needed beginning at null to make either word look like the other (2,2) is cost 2 because a replacement is required to make an i be an e

18. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Intention ntentiondelete I, cost 1 etentionreplace n with e, cost 2 exentionreplace t with x, cost 2 exenution insert u, cost 1 execution replace n with c, cost 2 Complete array of minimum edit difference between intention and execution (adapted from Jurafsky and Martin, 2000) (of several possible) string alignments. 11 The value at each (x,y) is the cost of minimum editing to that point, plus the minimum cost of either an insertion, deletion, or replacement In other words, cost of (x,y) is the minimum of Cost of (x-1,y) + insertion Cost of (x-1,y-1) + replacement Cost of (x, y-1) + deletion Finally, the backward part will select a list of optimal changes Bold text near the diagonal Not strictly required for spell-check case

19. Why Dynamic Programming? What’s the cost of DP?  n 2 where n is the size of the longest string  n 3 in the worst case if we need to consider other row/col values to determine the current state  What is the cost of exhaustive search?  Between 2 n and 3 n  Useful heuristics for DP  Usually the solution lies near the horizontal  Prune the considerations if the cost passes a predetermined threshold

20. Best-First Search Algorithm Need to have a good evaluation function  Avoid  Local maxima  Dead-ends  Anomalies in search space Return to general form of state space search algorithm  Lists  open – states we haven’t explored  closed – states we have explored  Now, order states in open according to some heuristic  Thus, each loop considers the most “promising” state first

21. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 12 Keeps track of ancestor information to ensure that we can check to see if state already reached and so we can return the chosen path at the end Can make use of a “priority queue”

22. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 Fig 4.10Heuristic search of a hypothetical state space. 13 Notice how states are selected for next examination.

23. A trace of the execution of best_first_search for Figure 4.4 Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 14

24. Implementing Heuristic Evaluation Functions Evaluate the performance of heuristics for solving 8-puzzle. Ideas? 1. Count number of tiles out of place  Doesn’t consider how far out of place they are 2. Get a total distance out of place  But what if tiles are right next to each other but out of order  Very difficult to switch tiles

25. Examining these Heuristics Fig 4.12The start state, first moves, and goal state for an example-8 puzzle. Which seems closest?

26. What the Heuristics Give Us Fig 4.14Three heuristics applied to states in the 8-puzzle.

27. What if two states have (nearly) the same heuristic evaluation? The heuristic f applied to states in the 8-puzzle.

28. How Good is a Heuristic? Evaluated along several dimensions  Goals?  Guaranteed Solution?  Shortest Path?  Shortest path heuristics are said to be admissible  Better heuristics  Concept of informedness  When a state is found is there a guarantee that a better state won’t be found later in the search?  Concept of monotonicity

29. Admissibility Measures Admissible – guaranteed to find a minimal path to a solution if it exists  Example: BFS Use the function f(n) = g(n) + h(n)‏  f(n) = evaluation function  g(n) = actual length from any state n to the start state  h(n) = heuristic estimate from any state n to the goal

30. Using f(n) to discuss admissibility  f(n) estimates the total cost of the path from the start state through n to the goal state Define f*(n) = g*(n) + h*(n)‏  All the same, but now only dealing with the shortest path from start to n and from n to goal  f*(n) is often called an oracle  Often do not exist for most real problems  Still we want f to be a close estimate to f*

31. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 5

32. Examples BFS is an A* algorithm in which  f(n) = g(n) + 0  The decision for considering a state is based solely on the distance from the start state Several Heuristics from 8-Puzzle are A*  Can’t always compute value of h*(n), but we can bound a heuristic cost above by the actual shortest-path cost

33. A* Examples for 8-puzzle Counting the number of tiles out of place is always less than or equal to the number of moves to move them to their goal  Thus: heuristic is admissible The sum of the direct distances of tiles out of place is also less than or equal to the minimum actual path  Thus: heuristic is admissible

34. Monotonicity The definition of A* does not require that g(n) = g*(n)‏  So, A* algorithms may initially reach non-goal states along sub-optimal paths before eventually finding an optimal path Monotonicity – property that guarantees that the heuristic is “locally optimal”  Consistently follows only the optimal path

35. Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 6

36. Monotonicity is possible when:  Search space is everywhere locally consistent with the heuristic employed  The difference between a heuristic measure of a state and any of its successors is bound by the actual cost of moving from predecessor to successor states As the path is followed when using a monotone heuristic  value of f is monotonically nondecreasing (hence the name)‏

37. Monotonicity and Admissibility All monotone heuristics are admissible  In fact, monotonicity is a refinement of admissibility Not all admissible heuristics are monotone

38. Relatively Better Heuristics Informedness

39. Example BFS is equivalent to A* with heuristic h1 such that h1(x)=0 for all states x  This is always less than h*(x)‏ Lets call the number of tiles out of place, h2  This is also less than h* But, we have h1<=h2<=h*  Thus, h2 is “more informed” than h1 Additionally, we can argue that calculation of direct distance of out of place tiles is more informed than h2