1. CMSC 671 Fall 2010 Thu 9/9/10 Uninformed Search (summary) Informed Search Functional Programming (revisited) Prof. Laura Zavala, laura.zavala@umbc.edu, ITE 373, 410-455-8775laura.zavala@umbc.edu

2. Functional Programming

3. Cool Things About Lisp Functions as objects (pass a function as an argument) Lambda expressions (construct a function on the fly) Lists as first-class objects Program as data Macros (smart expansion of expressions) Symbol manipulation

4. Functional Programming Decomposes a problem into a set of functions Has its roots in Lambda Calculus Formal system for function definition, function application and recursion. Functions as objects (pass a function as an argument) Lambda expressions (construct a function on the fly) Functions don't have any internal state (global and local variables) Recursion Truly functional programs are highly parallelizable

5. Functional Programming Languages  A focus on LISt Processing (Lisp).  Built-in map(), reduce(), and filter(), and the operator lambda.  Lisp, Scheme, Haskell, ML, OCAML, Clean, Mercury, Erlang, and a few others.  Python – Procedural, object-oriented, FP  Clojure - a dynamically-typed, functional programming language that runs on the JVM and provides interoperability with Java.

6. Lisp Map Function  Apply a function to all elements of a list, and return the resulting list  Lisp has a whole family of map functions  map, maplist, mapcar, etc. > mapcar #’(lambda (x) (+ x 10) ‘(1 2 3)) (11 12 13) > mapcar #’list ‘(a b c) ‘(11 2 3 4)) ((A 1) (B 2) (C 3))

7. Lisp Reduce Function  Boiling down a sequence into a single value  The function must be a function of two arguments > reduce #’fn ‘(a b c d)) is equivalent to > (fn (fn (fn ‘a ‘b) ‘c) ‘d) > reduce # ’ + ‘(1 2 3)) (6)

8. Uninformed Search (summary)

9. Building goal-based agents To build a goal-based agent we need to answer the following questions:  What is the goal to be achieved?  What are the actions?  What relevant information is necessary to encode in order to describe the state of the world, describe the available transitions, and solve the problem? Initial state Goal state Actions

10. 8 puzzle  State: 3 x 3 array configuration of the tiles on the board.  Operators: Blank Left, Blank Right, Blank Up, Blank Down.  Initial State: A particular configuration of the board.  Goal: A particular configuration of the board.

11. 8 puzzle – Search Tree

12. Study this map of Romania. Note the distances between cities and try and calculate the shortest route between Arad and Bucharest. Arad Bucharest Oradea Zerind Faragas Neamt Iasi Vaslui Hirsova Eforie Urziceni Giurgui Pitesti Sibiu Dobreta Craiova Rimnicu Mehadia Timisoara Lugoj 87 92 142 86 98 86 211 101 90 99 151 71 75 140 118 111 70 75 120 138 146 97 80 140 80 97 101 Sibiu Rimnicu Pitesti Optimal route is (140+80+97+101) = 418 miles http://www.cs.nott.ac.uk/~ajp/

13. Search Tree http://www.ics.uci.edu/~smyth/

14. Search Tree http://www.ics.uci.edu/~smyth/

15. Search Tree http://www.ics.uci.edu/~smyth/

16. Evaluating search strategies  Completeness  Guarantees finding a solution whenever one exists  Time complexity  How long (worst or average case) does it take to find a solution? Usually measured in terms of the number of nodes expanded  Space complexity  How much space (memory) is used by the algorithm? Usually measured in terms of the maximum size of the “nodes” list during the search  Optimality/Admissibility  If a solution is found, is it guaranteed to be an optimal one? That is, is it the one with minimum cost?

17. Uninformed Search Methods

18. Breadth-First vs Depth-First (DFS) Breadth-First Exponential time and space O(b d ) Optimal if costs are the same Depth-First Exponential time O(b d ) Linear space O(bd) May not terminate Uniform-Cost (UCS)Iterative Deepening solves the infinite-path problemcomplete and optimal

19. Bi-directional search  Alternate searching from the start state toward the goal and from the goal state toward the start.  Stop when the frontiers intersect.  Works well only when there are unique start and goal states.  Requires the ability to generate “predecessor” states.  Can (sometimes) lead to finding a solution more quickly.

20. Uninformed Search Results

21. Example for illustrating uninformed search strategies S CBA D G E 3 1 8 15 20 5 3 7

22. Depth-First Search Expanded node Nodes list { S 0 } S 0 { A 3 B 1 C 8 } A 3 { D 6 E 10 G 18 B 1 C 8 } D 6 { E 10 G 18 B 1 C 8 } E 10 { G 18 B 1 C 8 } G 18 { B 1 C 8 } S CBA D G E 3 1 8 15 20 5 3 7 Solution path found is S A G, cost 18 Number of nodes expanded (including goal node) = 5

23. Breadth-First Search Expanded node Nodes list { S 0 } S 0 { A 3 B 1 C 8 } A 3 { B 1 C 8 D 6 E 10 G 18 } B 1 { C 8 D 6 E 10 G 18 G 21 } C 8 { D 6 E 10 G 18 G 21 G 13 } D 6 { E 10 G 18 G 21 G 13 } E 10 { G 18 G 21 G 13 } G 18 { G 21 G 13 } Solution path found is S A G, cost 18 Number of nodes expanded (including goal node) = 7 S CBA D G E 3 1 8 15 20 5 3 7

24. Uniform-Cost Search Expanded node Nodes list { S 0 } S 0 { B 1 A 3 C 8 } B 1 { A 3 C 8 G 21 } A 3 { D 6 C 8 E 10 G 18 G 21 } D 6 { C 8 E 10 G 18 G 1 } C 8 { E 10 G 13 G 18 G 21 } E 10 { G 13 G 18 G 21 } G 13 { G 18 G 21 } Solution path found is S B G, cost 13 Number of nodes expanded (including goal node) = 7 S CBA D G E 3 1 8 15 20 5 3 7

25. How they perform  Depth-First Search:  Expanded nodes: S A D E G  Solution found: S A G (cost 18)  Breadth-First Search:  Expanded nodes: S A B C D E G  Solution found: S A G (cost 18)  Uniform-Cost Search:  Expanded nodes: S A D B C E G  Solution found: S B G (cost 13) This is the only uninformed search that worries about costs.  Iterative-Deepening Search:  nodes expanded: S S A B C S A D E G  Solution found: S A G (cost 18)

26. Bi-directional search  Alternate searching from the start state toward the goal and from the goal state toward the start.  Stop when the frontiers intersect.  Works well only when there are unique start and goal states.  Requires the ability to generate “predecessor” states.  Can (sometimes) lead to finding a solution more quickly.

27. Comparing Search Strategies

28. Avoiding Repeated States  In increasing order of effectiveness in reducing size of state space and with increasing computational costs: 1. Do not return to the state you just came from. 2. Do not create paths with cycles in them. 3. Do not generate any state that was ever created before.  Net effect depends on frequency of “loops” in state space.

29. Informed Search Methods

30. Informed search strategy  Uses problem-specific knowledge to assess whether one non-goal state is “more- promising” than another.  The strategy and knowledge used for the assessment is known as heuristic.

31. Heuristic Merriam-Webster's Online Dictionary Heuristic (pron. \hyu-’ris-tik\): adj. [from Greek heuriskein to discover.] involving or serving as an aid to learning, discovery, or problem- solving by experimental and especially trial-and-error methods The Free On-line Dictionary of Computing (15Feb98) heuristic 1. A rule of thumb, simplification or educated guess that reduces or limits the search for solutions in domains that are difficult and poorly understood. Unlike algorithms, heuristics do not guarantee feasible solutions and are often used with no theoretical guarantee. 2. approximation algorithm. From WordNet (r) 1.6 heuristic adj 1: (computer science) relating to or using a heuristic rule 2: of or relating to a general formulation that serves to guide investigation [ant: algorithmic] n : a commonsense rule (or set of rules) intended to increase the probability of solving some problem [syn: heuristic rule, heuristic program]

32. Heuristic function  Define a heuristic function h(n) that estimates the “goodness” of a node n.  Specifically, h(n) = estimated cost (or distance) of minimal cost path from n to a goal state.  The heuristic function is an estimate of how close we are to a goal, based on domain- specific information that is computable from the current state description.

33. Examples of Heuristics  Examples:  Missionaries and Cannibals: Number of people on starting river bank  8-puzzle: Number of tiles out of place  8-puzzle: Sum of distances each tile is from its goal position (Manhattan Distance)  In general:  h(n) ≥ 0 for all nodes n  h(n) = 0 implies that n is a goal node  h(n) = ∞ implies that n is a dead-end that can never lead to a goal

34. Best-first search  Order nodes on the nodes list by increasing value of an evaluation function f (n)  f (n) incorporates domain-specific information in some way.  Incorportes a heuristic function h(n) as a component of f.  This is a generic way of referring to the class of informed methods.  We get different searches depending on the evaluation function f (n)

35. Greedy search  Evaluates nodes by using just the heuristic function f (n) = h(n).  Selects node to expand believed to be closest (hence “greedy”) to a goal node (i.e., select node with smallest f value)  Not complete  Not admissible

36. Greedy Search 253 176 h = Straight Line Distance

37. Algorithm A*  Use as an evaluation function f (n) = g(n) + h(n)  g(n) = minimal-cost path from the start state to state n.  The g(n) term adds a “breadth- first” component to the evaluation function.  Ranks nodes on search frontier by estimated cost of solution from start node through the given node to goal. S BA D G 1 5 8 3 1 5 C 1 9 4 5 8 9 g(d)=4 h(d)=9 C is chosen next to expand

38. Evaluating  Admissible heuristic – Never overestimates the cost to reach the goal  SLD – Shortest path between any two points is a straight line, so it can not be an overestimate.  Consistency – estimated cost of reaching goal from n is no greater than step cost n to n’ plus estimated cost of reaching goal from n’  h(n) <= c(n,a,n’) + h(n’)

39. Algorithm A*  Complete whenever the branching factor is finite, and every operator has a fixed positive cost  Optimally efficient for any given consistent heuristic S BA D G 1 5 8 3 1 5 C 1 9 4 5 8 9 g(d)=4 h(d)=9 C is chosen next to expand

40. A* in action  Remember that the A* search pattern is the union of two evaluation functions. In the following demonstration the A* search pattern evaluates the cost of the path so far (UCS), together with an admissible heuristic function based on the shortest line distance (SLD) between between the initial state and the goal location, such that  h SLD (n) = straight line distance between n and the goal location.  The following is table of the straight line distances between some of the major Romanian cities and and the goal state, Bucharest.

41. Straight Line Distances to Bucharest TownSLD Arad366 Bucharest0 Craiova160 Dobreta242 Eforie161 Fagaras178 Giurgiu77 Hirsova151 Iasi226 Lugoj244 TownSLD Mehadai241 Neamt234 Oradea380 Pitesti98 Rimnicu193 Sibiu253 Timisoara329 Urziceni80 Vaslui199 Zerind374 We can use straight line distances as an admissible heuristic as they will never overestimate the cost to the goal: there is no shorter distance between two cities than the straight line distance.

42. Press space to see an A* search of the Romanian map featured in the previous slide. Note: Throughout the animation all nodes are labelled with f(n) = g(n) + h(n). However,we will be using the abbreviations f, g and h to make the notation simpler Oradea Zerind Fagaras Pitesti Sibiu Craiova Rimnicu Timisoara Bucharest Arad We begin with the initial state of Arad. The cost of reaching Arad from Arad (or g value) is 0 miles. The straight line distance from Arad to Bucharest (or h value) is 366 miles. This gives us a total value of ( f = g + h ) 366 miles. Press space to expand the initial state of Arad. F= 0 + 366 F= 366 F= 75 + 374 F= 449 F= 140 + 253 F= 393 F= 118 + 329 F= 447 Once Arad is expanded we look for the node with the lowest cost. Sibiu has the lowest value for f. (The cost to reach Sibiu from Arad is 140 miles, and the straight line distance from Sibiu to the goal state is 253 miles. This gives a total of 393 miles). Press space to move to this node and expand it. We now expand Sibiu (that is, we expand the node with the lowest value of f ). Press space to continue the search. F= 239 + 178 F= 417 F= 291 + 380 F= 671 F= 220 + 193 F= 413 We now expand Rimnicu (that is, we expand the node with the lowest value of f ). Press space to continue the search. F= 317 + 98 F= 415 F= 366 + 160 F= 526 Once Rimnicu is expanded we look for the node with the lowest cost. As you can see, Pitesti has the lowest value for f. (The cost to reach Pitesti from Arad is 317 miles, and the straight line distance from Pitesti to the goal state is 98 miles. This gives a total of 415 miles). Press space to move to this node and expand it. We now expand Pitesti (that is, we expand the node with the lowest value of f ). Press space to continue the search. We have just expanded a node (Pitesti) that revealed Bucharest, but it has a cost of 418. If there is any other lower cost node (and in this case there is one cheaper node, Fagaras, with a cost of 417) then we need to expand it in case it leads to a better solution to Bucharest than the 418 solution we have already found. Press space to continue. F= 418 + 0 F= 418 In actual fact, the algorithm will not really recognise that we have found Bucharest. It just keeps expanding the lowest cost nodes (based on f ) until it finds a goal state AND it has the lowest value of f. So, we must now move to Fagaras and expand it. Press space to continue. We now expand Fagaras (that is, we expand the node with the lowest value of f ). Press space to continue the search. Bucharest (2) F= 450 + 0 F= 450 Once Fagaras is expanded we look for the lowest cost node. As you can see, we now have two Bucharest nodes. One of these nodes ( Arad – Sibiu – Rimnicu – Pitesti – Bucharest ) has an f value of 418. The other node (Arad – Sibiu – Fagaras – Bucharest (2) ) has an f value of 450. We therefore move to the first Bucharest node and expand it. Press space to continue Bucharest We have now arrived at Bucharest. As this is the lowest cost node AND the goal state we can terminate the search. If you look back over the slides you will see that the solution returned by the A* search pattern ( Arad – Sibiu – Rimnicu – Pitesti – Bucharest ), is in fact the optimal solution. http://www.cs.nott.ac.uk/~ajp/

43. Features of an A* Search  A* is optimal and complete, but it is not all good news. It can be shown that the number of nodes that are searched is still exponential to the length of the search space for most problems. This has implications not only for the time taken to perform the search but also the space required. Of these two problems the search complexity is more serious.  If you examine the animation on the previous slide you will notice an interesting phenomenon. Along any path from the root, the f-cost never decreases. This is no accident. It holds true for all admissible heuristics. A heuristic for which it holds is said to exhibit monotonicity.  Because A* expands the leaf nodes of lowest f, we can see that an A* search fans out from the start node, adding nodes in concentric bands of increasing f-cost. These bands can be modelled as contours in the state space.

44. Dealing with hard problems  For large problems, A* often requires too much space.  Two variations conserve memory: IDA* and SMA*  IDA* -- iterative deepening A*  uses successive iteration with growing limits on f. For example,  A* but don’t consider any node n where f (n) > 10  A* but don’t consider any node n where f (n) > 20  A* but don’t consider any node n where f (n) > 30,...  SMA* -- Simplified Memory-Bounded A*  uses a queue of restricted size to limit memory use.  throws away the “oldest” worst solution.

45. What’s a good heuristic?  If h 1 (n) < h 2 (n) ≤ h*(n) for all n, h 2 is better than (dominates) h 1  Relaxing the problem: remove constraints to create a (much) easier problem; use the solution cost for this problem as the heuristic function  Combining heuristics: take the max of several admissible heuristics: still have an admissible heuristic, and it’s better!  Identify good features, then use a learning algorithm to find a heuristic function: also may lose admissibility

46. Summary: Informed search  Best-first search is general search where the minimum-cost nodes (according to some measure) are expanded first.  Greedy search uses minimal estimated cost h(n) to the goal state as measure. This reduces the search time, but the algorithm is neither complete nor optimal.  A* search combines uniform-cost search and greedy search: f (n) = g(n) + h(n). A* handles state repetitions and h(n) never overestimates.  A* is complete and optimal, but space complexity is high.  The time complexity depends on the quality of the heuristic function.  IDA* and SMA* reduce the memory requirements of A*.

47. Thanks for coming -- see you next Tuesday!