1. Instructor: Eyal Amir Grad TAs: Wen Pu, Yonatan Bisk Undergrad TAs: Sam Johnson, Nikhil Johri CS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #4

2. When to Use Search Techniques? The search space is small, and –There is no other available techniques, or –It is not worth the effort to develop a more efficient technique The search space is large, and –There is no other available techniques, and –There exist “good” heuristics

3. Models, |=, math proofs Alpha |= Beta –Alpha, Beta – propositional formulas –M |= Alpha “M models Alpha” means “Alpha evaluated to TRUE in model M” Math. Proofs: example –A  B |= A –(another one later in today’s class)

4. Search Algorithms Blind search – BFS, DFS, ID, uniform cost –no notion concept of the “right direction” –can only recognize goal once it’s achieved Heuristic search – we have rough idea of how good various states are, and use this knowledge to guide our search

5. Types of heuristic search Best First –A* is a special case –BFS is a special case ID A* –ID is a special case Hill climbing Simulated Annealing

6. A* Example: 8-Puzzle 0+41+5 1+3 3+3 3+4 3+24+15+2 5+0 2+3 2+4 2+3 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

7. ID A*: 8-Puzzle Example 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4

8. 4 4 6 6 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

9. 4 4 6 Cutoff=4 6 5 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

10. 4 4 6 Cutoff=4 6 5 5 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

11. 4 4 6 Cutoff=4 6 5 56 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

12. 4 6 Cutoff=5 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

13. 4 4 6 Cutoff=5 6 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

14. 4 4 6 Cutoff=5 6 5 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

15. 4 4 6 Cutoff=5 6 57 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

16. 4 4 6 Cutoff=5 6 5 7 5 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

17. 4 4 6 Cutoff=5 6 5 7 55 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

18. 4 4 6 Cutoff=5 6 5 7 55 ID A*: 8-Puzzle Example f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

19. Hill climbing example 283 164 7 5 283 14 765 23 184 765 13 84 765 2 3 184 765 2 13 8 4 765 2 start goal -5 h = -3 h = -2 h = -1 h = 0 h = -4 -5 -4 -3 -2 f(n) = -(number of tiles out of place)

20. Best-first search Idea: use an evaluation function f(n) for each node n Expand unexpanded node n with min f(n) Implementation: FRINGE is queue sorted by decreasing order of desirability –Greedy search –A* search

21. Greedy Search h(n) – a ‘heuristic’ function estimating the distance to the goal Greedy Best First: expand argmin_n h(n) thus,f(v) = h(v)

22. Informed Search Add domain-specific information to select the best path along which to continue searching h(n) = estimated cost (or distance) of minimal cost path from n to a goal state. The heuristic function is an estimate, based on domain-specific information that is computable from the current state description, of how close we are to a goal

23. Robot Navigation

24. 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 f(N) = h(N), with h(N) = Manhattan distance to the goal

25. Robot Navigation 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 f(N) = h(N), with h(N) = Manhattan distance to the goal 7 0 What happened???

26. Greedy Search f(N) = h(N)  greedy best-first Is it complete? If we eliminate endless loops, yes Is it optimal?

27. More informed search Our goal is not to minimize the distance from the current head of our path to the goal, we want to minimize the overall length of the path to the goal! Let g(N) be the cost of the best path found so far between the initial node and N f(N) = g(N) + h(N)

28. Robot Navigation f(N) = g(N)+h(N), with h(N) = Manhattan distance to goal 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 7+0 6+1 8+1 7+0 7+2 6+17+2 6+1 8+1 7+2 8+37+26+3 5+4 4+5 3+6 2+78+37+4 6+5 5+6 6+35+62+73+8 4+7 5+64+7 3+8 4+73+8 2+9 3+10 2+9 3+82+9 1+10 0+11

29. Can we Prove Anything? If the state space is finite and we avoid repeated states, the search is complete, but in general is not optimal Proof: ? If the state space is finite and we do not avoid repeated states, the search is in general not complete

30. Admissible heuristic Let h*(N) be the true cost of the optimal path from N to a goal node Heuristic h(N) is admissible if: 0  h(N)  h*(N) An admissible heuristic is always optimistic

31. A* Search Evaluation function: f(N) = g(N) + h(N) where: –g(N) is the cost of the best path found so far to N –h(N) is an admissible heuristic Then, best-first search with this evaluation function is called A* search Important AI algorithm developed by Fikes and Nilsson in early 70s. Originally used in Shakey robot.

32. Completeness & Optimality of A* Claim 1: If there is a path from the initial to a goal node, A* (with no removal of repeated states) terminates by finding the best path, hence is: –complete –optimal requirements: –0 <   c(N,N’) - c(N,N’) – cost of going from N to N’

33. Completeness of A* Theorem: If there is a finite path from the initial state to a goal node, A* will find it.

34. Proof of Completeness Intuition (not math. Proof): Let g be the cost of a best path to a goal node No path in search tree can get longer than g/ , before the goal node is expanded

35. Optimality of A* Theorem: If h(n) is admissable, then A* is optimal (finds an optimal path).

36. Proof of Optimality G1G1 N G2G2 f(G 1 ) = g(G 1 ) f(N) = g(N) + h(N)  g(N) + h*(N) f(G 1 )  g(N) + h(N)  g(N) + h*(N) Cost of best path to a goal thru N

37. Example of Evaluation Function f(N) = (sum of distances of each tile to its goal) + 3 x (sum of score functions for each tile) where score function for a non-central tile is 2 if it is not followed by the correct tile in clockwise order and 0 otherwise 123 456 78 12 3 4 5 67 8 N goal f(N) = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 3x(2 + 2 + 2 + 2 + 2 + 0 + 2) = 49

38. Heuristic Function Function h(N) that estimates the cost of the cheapest path from node N to goal node. Example: 8-puzzle 123 456 78 12 3 4 5 67 8 N goal h(N) = number of misplaced tiles = 6

39. Heuristic Function Function h(N) that estimate the cost of the cheapest path from node N to goal node. Example: 8-puzzle 123 456 78 12 3 4 5 67 8 N goal h(N) = sum of the distances of every tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13

40. 8-Puzzle 45 5 3 3 4 34 4 212 0 3 4 3 f(N) = h(N) = number of misplaced tiles

41. 8-Puzzle 0+41+5 1+3 3+3 3+4 3+24+15+2 5+0 2+3 2+4 2+3 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

42. 8-Puzzle 56 6 4 4 212 0 5 5 3 f(N) = h(N) =  distances of tiles to goal

43. 8-Puzzle 123 456 78 12 3 4 5 67 8 Ngoal h 1 (N) = number of misplaced tiles = 6 is admissible h 2 (N) = sum of distances of each tile to goal = 13 is admissible h 3 (N) = (sum of distances of each tile to goal) + 3 x (sum of score functions for each tile) = 49 is not admissible

44. 8-Puzzle 0+41+5 1+3 3+3 3+4 3+24+15+2 5+0 2+3 2+4 2+3 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

45. Robot navigation Cost of one horizontal/vertical step = 1 Cost of one diagonal step =  2 f(N) = g(N) + h(N), with h(N) = straight-line distance from N to goal

46. About Repeated States NN1N1 SS1S1 f(N)=g(N)+h(N) f(N 1 )=g(N 1 )+h(N 1 ) g(N 1 ) < g(N) h(N) < h(N 1 ) f(N) < f(N 1 ) N2N2 f(N 2 )=g(N 2 )+h(N) g(N 2 ) < g(N)

47. Consistent Heuristic The admissible heuristic h is consistent (or satisfies the monotone restriction) if for every node N and every successor N’ of N: h(N)  c(N,N’) + h(N’) (triangle inequality) N N’ h(N) h(N’) c(N,N’)

48. 8-Puzzle 123 456 78 12 3 4 5 67 8 Ngoal h 1 (N) = number of misplaced tiles h 2 (N) = sum of distances of each tile to goal are both consistent

49. Robot navigation Cost of one horizontal/vertical step = 1 Cost of one diagonal step =  2 h(N) = straight-line distance to the goal is consistent

50. Claims If h is consistent, then the function f along any path is non-decreasing: f(N) = g(N) + h(N) f(N’) = g(N) +c(N,N’) + h(N’) N N’ h(N) h(N’) c(N,N’)

51. Claims If h is consistent, then the function f along any path is non-decreasing: f(N) = g(N) + h(N) f(N’) = g(N) +c(N,N’) + h(N’) h(N)  c(N,N’) + h(N’) f(N)  f(N’) N N’ h(N) h(N’) c(N,N’)

52. Claims If h is consistent, then the function f along any path is non-decreasing: f(N) = g(N) + h(N) f(N’) = g(N) +c(N,N’) + h(N’) h(N)  c(N,N’) + h(N’) f(N)  f(N’) If h is consistent, then whenever A* expands a node it has already found an optimal path to the state associated with this node N N’ h(N) h(N’) c(N,N’)

53. Avoiding Repeated States in A* If the heuristic h is consistent, then: Let CLOSED be the list of states associated with expanded nodes When a new node N is generated: –If its state is in CLOSED, then discard N –If it has the same state as another node in the fringe, then discard the node with the largest f

54. Complexity of Consistent A* Let s be the size of the state space Let r be the maximal number of states that can be attained in one step from any state Assume that the time needed to test if a state is in CLOSED is O(1) The time complexity of A* is O(s r log s)

55. Heuristic Accuracy h(N) = 0 for all nodes is admissible and consistent. Hence, breadth-first is a special case of A* !!! Let h 1 and h 2 be two admissible and consistent heuristics such that for all nodes N: h 1 (N)  h 2 (N). Then, every node expanded by A* using h 2 is also expanded by A* using h 1. h 2 is more informed than h 1

56. Iterative Deepening A* (IDA*) Use f(N) = g(N) + h(N) with admissible and consistent h Each iteration is depth-first with cutoff on the value of f of expanded nodes

57. 8-Puzzle 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4

58. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4 6

59. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4 6 5

60. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4 6 5 5

61. 4 8-Puzzle 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4 6 5 56

62. 8-Puzzle 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5

63. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6

64. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6 5

65. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6 57

66. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6 5 7 5

67. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6 5 7 55

68. 8-Puzzle 4 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=5 6 5 7 55

69. About Heuristics Heuristics are intended to orient the search along promising paths The time spent computing heuristics must be recovered by a better search After all, a heuristic function could consist of solving the problem; then it would perfectly guide the search Deciding which node to expand is sometimes called meta-reasoning Heuristics may not always look like numbers and may involve large amount of knowledge

70. What’s the Issue? Search is an iterative local procedure Good heuristics should provide some global look-ahead (at low computational cost)

71. Another approach… for optimization problems –rather than constructing an optimal solution from scratch, start with a suboptimal solution and iteratively improve it Local Search Algorithms –Hill-climbing or Gradient descent –Potential Fields –Simulated Annealing –Genetic Algorithms, others…

72. Hill-climbing search If there exists a successor s for the current state n such that –h(s) < h(n) –h(s) <= h(t) for all the successors t of n, then move from n to s. Otherwise, halt at n. Looks one step ahead to determine if any successor is better than the current state; if there is, move to the best successor. Similar to Greedy search in that it uses h, but does not allow backtracking or jumping to an alternative path since it doesn’t “remember” where it has been. Not complete since the search will terminate at "local minima," "plateaus," and "ridges."

73. Hill climbing on a surface of states Height Defined by Evaluation Function

74. Hill climbing Steepest descent (~ greedy best-first with no search)  may get stuck into local minimum

75. Robot Navigation f(N) = h(N) = straight distance to the goal Local-minimum problem

76. Examples of problems with HC applet

77. Drawbacks of hill climbing Problems: –Local Maxima: peaks that aren’t the highest point in the space –Plateaus: the space has a broad flat region that gives the search algorithm no direction (random walk) –Ridges: flat like a plateau, but with dropoffs to the sides; steps to the North, East, South and West may go down, but a step to the NW may go up. Remedy: –Introduce randomness Random restart. Some problem spaces are great for hill climbing and others are terrible.

78. What’s the Issue? Search is an iterative local procedure Good heuristics should provide some global look-ahead (at low computational cost)

79. Hill climbing example 283 164 7 5 283 14 765 23 184 765 13 84 765 2 3 184 765 2 13 8 4 765 2 start goal -5 h = -3 h = -2 h = -1 h = 0 h = -4 -5 -4 -3 -2 f(n) = -(number of tiles out of place)

80. Example of a local maximum 125 74 863 125 74 863 125 74 863 125 74 863 125 74 863 -3 -4 0 startgoal

81. Potential Fields Idea: modify the heuristic function Goal is gravity well, drawing the robot toward it Obstacles have repelling fields, pushing the robot away from them This causes robot to “slide” around obstacles Potential field defined as sum of attractor field which get higher as you get closer to the goal and the indivual obstacle repelling field (often fixed radius that increases exponentially closer to the obstacle)

82. Does it always work? No. But, it often works very well in practice Advantage #1: can search a very large search space without maintaining fringe of possiblities Scales well to high dimensions, where no other methods work Example: motion planning Advantage #2: local method. Can be done online

83. Example: RoboSoccer All robots have same field: attracted to the ball Repulsive potential to other players Kicking field: attractive potential to the ball and local repulsive potential if clase to the ball, but not facing the direction of the opponent’s goal. Result is tangent, player goes around the ball. Single team: kicking field + repulsive field to avoid hitting other players + player position fields (paraboilic if outside your area of the field, 0 inside). Player nearest to the ball has the largest attractive coefficient, avoids all players crowding the ball. Two teams: identical potential fields.

84. Simulated annealing Simulated annealing (SA) exploits an analogy between the way in which a metal cools and freezes into a minimum-energy crystalline structure (the annealing process) and the search for a minimum [or maximum] in a more general system. SA can avoid becoming trapped at local minima. SA uses a random search that accepts changes that increase objective function f, as well as some that decrease it. SA uses a control parameter T, which by analogy with the original application is known as the system “temperature.” T starts out high and gradually decreases toward 0.

85. Simulated annealing (cont.) A “bad” move from A to B is accepted with a probability (f(B)-f(A)/T) e The higher the temperature, the more likely it is that a bad move can be made. As T tends to zero, this probability tends to zero, and SA becomes more like hill climbing If T is lowered slowly enough, SA is complete and admissible.

86. The simulated annealing algorithm

87. Summary: Local Search Algorithms Steepest descent (~ greedy best-first with no search)  may get stuck into local minimum Better Heuristics: Potential Fields Simulated annealing Genetic algorithms

88. When to Use Search Techniques? The search space is small, and –There is no other available techniques, or –It is not worth the effort to develop a more efficient technique The search space is large, and –There is no other available techniques, and –There exist “good” heuristics

89. Summary Heuristic function Best-first search Admissible heuristic and A* A* is complete and optimal Consistent heuristic and repeated states Heuristic accuracy IDA*

90. Modified Search Algorithm 1.INSERT(initial-node,FRINGE) 2.Repeat: If FRINGE is empty then return failure n  REMOVE(FRINGE) s  STATE(n) If GOAL?(s) then return path or goal state For every state s’ in SUCCESSORS(s)  Create a node n’  INSERT(n’,FRINGE)