1. Computer Science CPSC 322 Lecture 5 Heuristic Search and A* (Ch: 3.6, 3.6.1) Slide 1

2. Announcements Assignment 1 posted last Friday in Connect Due Friday, Jan 29, noon Please carefully read and follow the instructions on cover sheet If you have not done them yet, do all the “Graph Searching exercises” available at http://www.aispace.org/exercises.shtml http://www.aispace.org/exercises.shtml Suggestion: look at solutions only after you have tried hard to solve them! 2

3. Lecture Overview Recap of Lecture 4 Best First Search A* Admissible heuristic Completeness and Optimality Branch and Bound Slide 3

4. A. Check what node on a path is the goal B. Initialize the frontier C. Add/remove paths from the frontier D. Check if a state is a goal Search Strategies are different with respect to how they: 4

5. A. Check what node on a path is the goal B. Initialize the frontier C. Add/remove paths from the frontier D. Check if a state is a goal Search Strategies are different with respect to how they: 5

6. DFS Depth-First Search, DFS explores each path on the frontier until its end (or until a goal is found) before considering any other path. the frontier is a last-in-first-out stack 6

7. Breadth-first search (BFS) BFS explores all paths of length l on the frontier, before looking at path of length l + 1 The frontier is a first-in-first-out queue 7

8. DFS vs. BFS How can we achieve an acceptable (linear) space complexity while maintaining completeness and optimality? Key Idea: re-compute elements of the frontier rather than saving them. CompleteOptimalTimeSpace DFS NO O(b m ) BFS YES O(b m ) ????? YES O(b m ) 8

9. depth = 1 depth = 2 depth = 3... Iterative Deepening DFS (IDS) in a Nutshell Use DFS to look for solutions at depth 1, then 2, then 3, etc – For depth D, ignore any paths with longer length – Depth-bounded depth-first search If no goal re-start from scratch and get to depth 2 If no goal re-start from scratch and get to depth 3 If no goal re-start from scratch and get to depth 4 9

10. Further Analysis of Iterative Deepening DFS (IDS) Space complexity: it does DFS Complete? Optimal? 10

11. DFS vs. BFS But what if we have costs associated with search actions (arcs in the search space? CompleteOptimalTimeSpace DFS NO O(b m ) BFS YES O(b m ) IDS YES O(b m ) 11

12. Search with Costs In this setting we usually want to find the solution that minimizes cost Def.: The cost of a path is the sum of the costs of its arcs Def.: A search algorithm is optimal if when it finds a solution, it is the best one: it has the lowest path cost Slide 12

13. Lowest-cost-first search finds the path with the lowest cost to a goal node At each stage, it selects the path with the lowest cost on the frontier. The frontier is implemented as a priority queue ordered by path cost. Lowest-Cost-First Search (LCFS) 13

14. Summary of Uninformed Search CompleteOptimalTimeSpace DFSNNO(b m )O(mb) BFSYY (shortest) O(b m ) IDSYY (shortest) O(b m )O(mb) LCFSY Costs > э > 0 Y (Least Cost) Costs >=0 O(b m ) Slide 14

15. How to Make Search More Informed? Def.: A search heuristic h(n) is an estimate of the cost of the optimal (cheapest) path from node n to a goal node. Estimate: h(n1) Estimate: h(n2) Estimate: h(n3) n3 n2 n1 h can be extended to paths: h(  n 0,…,n k  )=h(n k ) h(n) should leverage readily obtainable information (easy to compute) about a node. Slide 15

16. Example: finding routes Possible h(n)? the straight-line (Euclidian) distance between source and goal node (here Bucharest) 16

17. Lecture Overview Recap of Lecture 4 Best First Search A* Admissible heuristic Completeness and Optimality Branch and Bound Slide 17

18. Idea: always choose the path on the frontier with the smallest h value. BestFS treats the frontier as a priority queue ordered by h. Greedy approach: expand path whose last node seems closest to the goal - chose the solution that is locally the best. Best First Search (BestFS) Let’s see how this works in AIspace: in the Search Applet toolbar select the “Vancouver Neighborhood Graph” problem set “Search Options -> Search Algorithms” to “Best-First ”. select “Show Node Heuristics” under “View” compare number of nodes expanded by BestFS and LCFS 18

19. Complete? See the “misleading heuristics demo” example in AISPACE Optimal? Try AISPACE example “ex-best.txt” from schedule page (save it and then load using “load from file” option) Time Complexity and space complexity Analysis of BestFS 19

20. Complete? Analysis of BestFS 20 A. YES B. NO

21. Complete? Analysis of BestFS 21 B. NO See the “misleading heuristics demo” example in AISPACE

22. Optimal? Analysis of BestFS 22 A. YES B. NO

23. Optimal? Analysis of BestFS 23 B. NO Try this example in AISPACE or example “ex-best.txt” from schedule page (save it and then load using “load from file” option)

24. Time and space Complexity: O(b m ) - Worst case (bad h(n)): has to explore all nodes and keep related partial paths on the frontier Analysis of BestFS Why would one want to use Best First Search ? Because if the heuristics is good if can find the solution very fast. For another example, see Aispace, Delivery problem graph with C1 linked to o123 (cost 3.0) 24

25. Complete? See the “misleading heuristics demo” example in AISPACE Optimal? Try AISPACE example “ex-best.txt” from schedule page (save it and then load using “load from file” option) Time Complexity: O(b m ) - Worst case: has to explore all nodes Space Complexity: O(b m ) Analysis of BestFS Why would one want to use Best Search then? Because if the heuristics is good if can find the solution very fast. For another example, see Aispace, Delivery problem graph with C1 linked to o123 (cost 3.0) 25

26. Always choose the path on the frontier with the smallest h value. BestFS treats the frontier as a priority queue ordered by h. Can get to the goal pretty fast if it has a good h but… It is not complete, nor optimal still has time and space worst-case complexity of O(b m ) Best First Search (BestFS) 26

27. Thus, having estimates of the distance to the goal can speed things up a lot, but by itself it can also mislead the search (i.e. Best First Search) On the other hand, taking only path costs into account allows LCSF to find the optimal solution, but the search process is still uniformed as far as distance to the goal goes. What’s Next? 27

28. How can we more effectively use h(p) and cost(p)? Shall we select from the frontier the path p with: A.Lowest cost(p) – h(p) B.Highest cost(p) – h(p) C.Highest cost(p)+h(p) D. Lowest cost(p)+h(p) 28

29. How can we more effectively use h(p) and cost(p)? Shall we select from the frontier the path p with: A.Lowest cost(p) – h(p) B.Highest cost(p) – h(p) C.Highest cost(p)+h(p) D. Lowest cost(p)+h(p) 29

30. Lecture Overview Recap of Lecture 4 Best First Search A* Admissible heuristic Completeness and Optimality Branch and Bound Slide 30

31. A * search takes into account both the cost of the path to a node c(p) the heuristic value of that path h(p). Let f(p) = c(p) + h(p). f(p) is an estimate of the cost of a path from the start to a goal via p. A* Search c(p) h(p) f(p ) A* always chooses the path on the frontier with the lowest estimated distance from the start to a goal node constrained to go via that path. 31

32. F-value of ubc  kd  jb? A.6 C.10 D. 11 B. 9 Computing f-valeues 32

33. F-value of ubc  kd  jb? C.10 Computing f-valeues 33

34. 34

35. Compare A* and LCFS on the Vancouver graph A * search takes into account both the cost of the path to a node c(p) the heuristic value of that path h(p). Let f(p) = c(p) + h(p). f(p) is an estimate of the cost of a path from the start to a goal via p. A* Search c(p) h(p) f(p ) A* always chooses the path on the frontier with the lowest estimated distance from the start to a goal node constrained to go via that path. 35

36. A* is complete (finds a solution, if one exists) and optimal (finds the optimal path to a goal) if the branching factor is finite arc costs are > > 0 h(n) is admissible Optimality of A* 36

37. Lecture Overview Recap of Lecture 4 Best First Search A* Admissible heuristic Completeness and Optimality Branch and Bound Slide 37

38. Admissibility of a heuristic Def.: Let c(n) denote the cost of the optimal path from node n to any goal node. A search heuristic h(n) is called admissible if h(n) ≤ c(n) for all nodes n, i.e. if for all nodes it is an underestimate of the cost to any goal. Example: is the straight- line distance (SLD) admissible? 38

39. Admissibility of a heuristic Def.: Let c(n) denote the cost of the optimal path from node n to any goal node. A search heuristic h(n) is called admissible if h(n) ≤ c(n) for all nodes n, i.e. if for all nodes it is an underestimate of the cost to any goal. Example: is the straight- line distance admissible? - The shortest distance between two points is a line. YES 39

40. Admissibility of a heuristic Def.: Let c(n) denote the cost of the optimal path from node n to any goal node. A search heuristic h(n) is called admissible if h(n) ≤ c(n) for all nodes n, i.e. if for all nodes it is an underestimate of the cost to any goal. example: the goal is Urzizeni (red box), but all we know is the straight- line distances (sld) to Bucharest (green box) Possible h(n) = sld(n, Bucharest) + cost(Bucharest, Urzineni) Admissible? A. Yes C. It depends B. No 40

41. Admissibility of a heuristic Def.: Let c(n) denote the cost of the optimal path from node n to any goal node. A search heuristic h(n) is called admissible if h(n) ≤ c(n) for all nodes n, i.e. if for all nodes it is an underestimate of the cost to any goal. example: the goal is Urzizeni (red box), but all we know is the straight- line distances to Bucharest (green box) NO Possible h(n) = sld(n, Bucharest) + cost(Bucharest, Urzineni) Admissible? 41 Actual cost of going from Vastul to Urzineni is shorter than this estimate

42. Example 2: grid world Search problem: robot has to find a route from start to goal location G on a grid with obstacles Actions: move up, down, left, right from tile to tile Cost : number of moves Possible h(n)? -Manhattan distance (L 1 distance) to the goal G: sum of the (absolute) difference of their coordinates -Admissible? 1 2 3 4 5 6 G 43214321 Slide 42

43. Example 2: grid world Search problem: robot has to find a route from start to goal location G on a grid with obstacles Actions: move up, down, left, right from tile to tile Cost : number of moves Possible h(n)? -Manhattan distance (L 1 distance) to the goal G: sum of the (absolute) difference of their coordinates -Admissible? Yes. Manhattan distance is the shortest path between any two tiles of the grid given the actions available and no walls. Including the walls will force the agent to take some extra steps to avoid them 1 2 3 4 5 6 G 43214321 Slide 43

44. Heuristic Function for 8-puzzle An admissible heuristics for the 8-puzzle is? A.Number of misplaced tiles plus number of correctly place tiles B.Number of misplaced tiles C.Number of correctly placed tiles D.None of the above 44

45. Example 3: Eight Puzzle Admissible h(n): Number of Misplaced Tiles: One needs at least that many moves to get the board in the goal state Slide 45

46. Example 3: Eight Puzzle Another possible h(n): Sum of number of moves between each tile's current position and its goal position (we can move over other tiles in the grid) Sum ( 12345678 Slide 46

47. Example 3: Eight Puzzle Another possible h(n): Sum of number of moves between each tile's current position and its goal position sum ( 2 3 3 2 4 2 0 2) = 18 Admissible? 12345678 A. Yes C. It depends B. No Slide 47

48. Example 3: Eight Puzzle Another possible h(n): Sum of number of moves between each tile's current position and its goal position sum 2 3 3 2 4 2 0 2 = 18 Admissible? YES! One needs to make at least as many moves to get to the goal state when constrained by the grid structure 12345678 Slide 48

49. How to Construct an Admissible Heuristic Identify relaxed version of the problem: -where one or more constraints have been dropped -problem with fewer restrictions on the actions Grid world: the agent can Driver: the agent can 8 puzzle: -“number of misplaced tiles”: tiles can -“sum of moves between current and goal position”: tiles can Why does this lead to an admissible heuristic? -The problem gets \ - 49

50. How to Construct an Admissible Heuristic Identify relaxed version of the problem: -where one or more constraints have been dropped -problem with fewer restrictions on the actions Grid world: the agent can move through walls Driver: the agent can move straight 8 puzzle: -“number of misplaced tiles”: tiles can move everywhere and occupy same spot as others -“sum of moves between current and goal position”: tiles can occupy same spot as others Why does this lead to an admissible heuristic? -The problem only gets easier! -Less costly to solve it 50

51. How to Construct a Heuristic (cont.) You should identify constraints which, when dropped, make the problem easy to solve important because heuristics are not useful if they're as hard to solve as the original problem! This was the case in our examples Robot: allowing the agent to move through walls. Optimal solution to this relaxed problem is Manhattan distance Driver: allowing the agent to move straight. Optimal solution to this relaxed problem is straight-line distance 8puzzle: tiles can move anywhere. Optimal solution to this relaxed problem is number of misplaced tiles 51

52. A* is complete (finds a solution, if one exists) and optimal (finds the optimal path to a goal) if the branching factor is finite arc costs are > > 0 h(n) is admissible Back to A* Slide 52

53. Lecture Overview Recap of Lecture 4 Best First Search A* Admissible heuristic Completeness and Optimality Branch and Bound Slide 53

54. S G p* p 1. f(p*) = c(p*) + h(p*) is Proof Let - p be a subpath of an optimal path p*. - c( ) be the actual path cost of a path (> 0) - f( ) be the f-values of a path - h() be the heuristic value of a path (≥ 0) We can show that If h() is admissible, f(p) ≤ f(p*) 54 A = c(p*) B ≤ c(p*) C ≥ c(p*) D None of the above p* Proof

55. S G p* p Proof Let - p be a subpath of an optimal path p*. - c( ) be the actual path cost of a path (> 0) - f( ) be the f-values of a path - h() be the heuristic value of a path (≥ 0) We can show that If h() is admissible, f(p) ≤ f(p*) 55 p* Proof 1. f(p*) = c(p*) + h(p*) = c(p*) Because h is admissible, thus cannot be > 0 at the goal, must be h(p*) = 0

56. S G p* p Proof Let - p be a subpath of an optimal path p*. - c( ) be the actual path cost of a path (> 0) - f( ) be the f-values of a path - h() be the heuristic value of a path (≥ 0) We can show that If h() is admissible, f(p) ≤ f(p*) 56 p* Proof 1. f(p*) = c(p*) + h(p*) = c(p*) Because h is admissible, thus cannot be > 0 at the goal, must be h(p*) = 0 2. f(p) = c(p) + h(p) is A = c(p*) B ≤ c(p*) C ≥ c(p*) D None of the above

57. S G p* p Proof Let - p be a subpath of an optimal path p*. - c( ) be the actual path cost of a path (> 0) - f( ) be the f-values of a path - h() be the heuristic value of a path (≥ 0) We can show that If h() is admissible, f(p) ≤ f(p*) 57 p* Proof 1. f(p*) = c(p*) + h(p*) = c(p*) Because h is admissible, thus cannot be > 0 at the goal, must be h(p*) = 0 2. f(p) = c(p) + h(p) ≤ c(p*) = f(p*) Because h(p) does not overestimate the cost of getting from p to the goal. Thus f(p) ≤ f(p*) For every subpath of an optimal path p*

58. Apply basic properties of search algorithms: - completeness, optimality, time and space complexity CompleteOptimalTimeSpace DFSN (Y if no cycles) NO(b m )O(mb) BFSYYO(b m ) IDSYYO(b m )O(mb) LCFS (when arc costs available) Y Costs > Y Costs >=0 O(b m ) Best First (when h available) A* (when arc costs > and h admissible) 58 Learning Goal for Search

59. Apply basic properties of search algorithms: - completeness, optimality, time and space complexity CompleteOptimalTimeSpace DFSN (Y if no cycles) NO(b m )O(mb) BFSYYO(b m ) IDSYYO(b m )O(mb) LCFS (when arc costs available) Y Costs > Y Costs >=0 O(b m ) Best First (when h available) NNO(b m ) 59 Learning Goal for Search

60. Apply basic properties of search algorithms: completeness, optimality time and space complexity Select the most appropriate search algorithms for specific problems. Depth-First Search vs. Bredth-First Search vs. Iterative Deepening vs. Least-Cost-First Search, Best First, Define/read/write/trace/debug different search algorithms Construct heuristic functions and discuss their admissibility for specific search problems Learning Goals for Search (up to today) Slide 60

61. Do practice excercises 3C and 3D Read Ch 3.7 (More sophisticated search) TODO Start working on Assignment 1! You can already do many of the questions 61

62. Branch-and-Bound Search What does allow A* to do better than the other search algorithms we have seen? What is the biggest problem with A*? Possible Solution: 62

63. Branch-and-Bound Search What does allow A* to do better than the other search algorithms we have seen? Arc cost and h(h) combined in f(n) What is the biggest problem with A*? space Possible Solution: Combine DFS with f 63

64. Branch-and-Bound Search One way to combine DFS with heuristic guidance Follows exactly the same search path as depth-first search But to ensure optimality, it does not stop at the first solution found It continues, after recording upper bound on solution cost upper bound: UB = cost of the best solution found so far Initialized to  or any overestimate of optimal solution cost When a path p is selected for expansion: Compute lower bound LB(p) = f(p) = cost(p) + h(p) - If LB(p)  UB, remove p from frontier without expanding it (pruning) - Else expand p, adding all of its neighbors to the frontier 64

65. Example Arc cost = 1 h(n) = 0 for every n UB = ∞ Solution! UB = ? Before expanding a path p, check its f value f(p): Expand only if f(p) < UB 65 JUST T O SIMPLY THE EXAMPLE

66. Example Arc cost = 1 h(n) = 0 for every n UB = 5 f= 5 Prune! 66 Before expanding a path p, check its f value f(p): Expand only if f(p) < UB

67. Example Before expanding a path p, check its f value f(p): Expand only if f(p) < UB Arc cost = 1 h(n) = 0 for every n UB = 5 f = 5 Prune! f = 5 Prune! Solution! UB =? 67

68. Example Before expanding a path p, check its f value f(p): Expand only if f(p) < UB Arc cost = 1 h(n) = 0 for every n UB = 3 f = 3 Prune! 68

69. Example Before expanding a path p, check its f value f(p): Expand only if f(p) < UB Arc cost = 1 h(n) = 0 for every n UB = 3 f = 3 Prune! f = 3 Prune! 69

70. Branch-and-Bound Analysis Complete ? Optimal: Space complexity: Time complexity: ……. 70

71. Is Branch-and-Bound optimal? D. Only if there are no cycles A. YES, with no further conditions C. Only if h(n) is admissible B. NO Branch-and-Bound Analysis 71

72. Is Branch-and-Bound optimal? D. Only if there are no cycles A. YES, with no further conditions C. Only if h(n) is admissible. Otherwise, when checking LB(p)  UB, if the answer is yes but h(p) is an overestimate of the actual cost of p, we remove a possibly optomal solution B. NO Branch-and-Bound Analysis 72

73. Branch-and-Bound Analysis Complete ? (..even when there are cycles) B. NO A. YES C. It depends 73

74. Branch-and-Bound Analysis Complete ? (..even when there are cycles) IT DEPENDS on whether we can initialize UB to a finite value, i.e. we have a reliable overestimate of the solution cost. If we don`t, we need to use ∞, and BBA can be caught in a cycle 74

75. Branch-and-Bound Analysis Complete ? Same as DFS: can’t handle cycles/infinite graphs. But complete if initialized with some finite U Optimal: YES, if h(n) is admissible Space complexity: Branch & Bound has the same space complexity as…. this is a big improvement over …………..! Time complexity: ……. 75

76. Branch-and-Bound Analysis Complete ? Same as DFS: can’t handle cycles/infinite graphs. But complete if initialized with some finite U Optimal: YES, if h(n) is admissible Space complexity: Branch & Bound has the same space complexity as DFS this is a big improvement over A*! Time complexity: O(b m ). 76