1. Brian Williams, Fall 041 Analysis of Uninformed Search Methods Brian C. Williams 16.410-13 Sep 21 st, 2004 Slides adapted from: 6.034 Tomas Lozano Perez, Russell and Norvig AIMA

2. Brian Williams, Fall 042 Assignments Remember: Problem Set #2: Simple Scheme and Search due Monday, September 27th, 2004. Reading: –Uninformed search: more of AIMA Ch. 3

3. Brian Williams, Fall 043 Outline Recap Analysis –Depth-first search –Breadth-first search Iterative deepening

4. Brian Williams, Fall 044 Complex missions must carefully: Plan complex sequences of actions Schedule tight resources Monitor and diagnose behavior Repair or reconfigure hardware.  Most AI problems, like these, may be formulated as state space search.

5. Brian Williams, Fall 045 Astronaut Goose Grain Fox Grain Fox Astronaut Goose Astronaut Grain Fox Goose Astronaut Fox Grain Astronaut Goose Grain Fox Astronaut Goose Grain Fox Grain Astronaut Goose Fox Astronaut Goose Grain Fox Astronaut Goose Grain Astronaut Goose Fox Grain Goose Fox Astronaut Grain Goose Grain Astronaut Fox Astronaut Grain Goose Fox Astronaut Fox Goose Grain Goose Grain Fox Astronaut Goose Grain Fox Formulate Goal Formulate Problem –States –Operators Generate Solution –Sequence of states Problem Solving: Formulate Graph and Find Paths

6. Brian Williams, Fall 046 Elements of Algorithm Design Description: (last Wednesday) –stylized pseudo code, sufficient to analyze and implement the algorithm –Implementation (last Monday).

7. Brian Williams, Fall 047 Depth First Search (DFS) S D BA CG CG D CG S D BA CG CG D CG Breadth First Search (BFS) Depth-first: Add path extensions to front of Q Pick first element of Q Breadth-first: Add path extensions to back of Q Pick first element of Q

8. Brian Williams, Fall 048 C S B G A D Issue: Starting at S and moving top to bottom, will depth-first search ever reach G?

9. Brian Williams, Fall 049 C S B G A D Q 1(S) 2(A S) (B S) 3(C A S) (D A S) (B S) 4(D A S) (B S) 5 (C D A S)(G D A S) (B S) 6(G D A S)(B S) 1 2 3 4 Depth-First C visited multiple times Multiple paths to C, D & G How much wasted effort can be incurred in the worst case? Effort can be wasted in more mild cases

10. Brian Williams, Fall 0410 How Do We Avoid Repeat Visits? Idea: Keep track of nodes already visited. Do not place visited nodes on Q. Does this maintain correctness? Any goal reachable from a node that was visited a second time would be reachable from that node the first time. Does it always improve efficiency? Visits only a subset of the original paths, such that each node appears at most once at the head of a path in Q.

11. Brian Williams, Fall 0411 How Do We Modify Simple Search Algorithm Let Q be a list of partial paths, Let S be the start node and Let G be the Goal node. 1.Initialize Q with partial path (S) as only entry; 2.If Q is empty, fail. Else, pick some partial path N from Q 3.If head(N) = G, return N (goal reached!) 4.Else a)Remove N from Q b)Find all children of head(N) and create all the one-step extensions of N to each child. c)Add to Q all the extended paths; d)Go to step 2.

12. Brian Williams, Fall 0412 Simple Search Algorithm Let Q be a list of partial paths, Let S be the start node and Let G be the Goal node. 1.Initialize Q with partial path (S) as only entry; set Visited = ( ) 2.If Q is empty, fail. Else, pick some partial path N from Q 3.If head(N) = G, return N (goal reached!) 4.Else a)Remove N from Q b)Find all children of head(N) not in Visited and create all the one-step extensions of N to each child. c)Add to Q all the extended paths; d)Add children of head(N) to Visited e)Go to step 2.

13. Brian Williams, Fall 0413 Note: Testing for the Goal Algorithm stops (in step 3) when head(N) = G. Could have performed test in step 6, as each extended path is added to Q. But, performing test in step 6 will be incorrect for optimal search, discussed later.  We chose step 3 to maintain uniformity with these future searches.

14. Brian Williams, Fall 0414 Outline Analysis –Depth-first search –Breadth-first search Iterative deepening

15. Brian Williams, Fall 0415 Elements of Algorithm Design Description: (last Wednesday) –stylized pseudo code, sufficient to analyze and implement the algorithm –Implementation (last Monday). Analysis: (today) Soundness: –when a solution is returned, is it guaranteed to be correct? Completeness: –is the algorithm guaranteed to find a solution when there is one? Time complexity: –how long does it take to find a solution? Space complexity: –how much memory does it need to perform search?

16. Brian Williams, Fall 0416 Characterizing Search Algorithms d = 1 Level 1 Level 2 Level 0 b = 3 b = maximum branching factor, number of children d = depth of the shallowest goal node m = maximum length of any path in the state space m = 2

17. Brian Williams, Fall 0417 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first Breadth-first Which is better, depth-first or breadth-first? Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D CG C S B G A D

18. Brian Williams, Fall 0418 Worst Case Time for Depth-first Worst case time T is proportional to number of nodes visited T dfs = [b m + … b + 1]*c dfs where cdfs is time per node b * T dfs = [b m+1 + b m + … b]*c dfs Solve recurrence [b – 1] * T dfs = [b m+1 – 1]*c dfs T dfs = [b m+1 – 1] / [b – 1] *c dfs Level 1 Level 2 Level 0 b*1 b*b b*b m-1... Level m 1

19. Brian Williams, Fall 0419 Cost Using Order Notation Worst case time T is proportional to number of nodes visited Level 1 Level 2 Level 0 b*1 b*b b*b m-1... Order Notation T = O(e) if T ≤ c * e for some constant c T dfs = [b m +1 – 1] / [b – 1] *c dfs = O(b m+1 ) ~ O(b m )for large b 1

20. Brian Williams, Fall 0420 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~ b m Breadth-first Which is better, depth-first or breadth-first? Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D CG C S B G A D

21. Brian Williams, Fall 0421 Worst Case Space for Depth-first Worst case space S dfs is proportional to maximal length of Q Level 1 Level m Level 0

22. Brian Williams, Fall 0422 Worst Case Space for Depth-first Worst case space S dfs is proportional to maximal length of Q If a node is queued its parent and siblings have been queued, and its parent dequeued.  S dfs ≥ [(b-1)*m+1] *c dfs where cdfs is space per node The children of at most one sibling is expanded at each level.  S dfs = [(b-1)*m+1] *c dfs S dfs = O(b*m) Level 1 Level m Level 0 b-1 b...

23. Brian Williams, Fall 0423 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m Breadth-first Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q Which is better, depth-first or breadth-first? S D BA CG CG D CG C S B G A D

24. Brian Williams, Fall 0424 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m No Breadth-first Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q Which is better, depth-first or breadth-first? S D BA CG CG D CG C S B G A D

25. Brian Williams, Fall 0425 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q Which is better, depth-first or breadth-first? S D BA CG CG D CG C S B G A D

26. Brian Williams, Fall 0426 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first S D BA CG CG D C G Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q Which is better, depth-first or breadth-first? C S B G A D

27. Brian Williams, Fall 0427 Worst Case Time for Breadth-first Worst case time T is proportional to number of nodes visited Level 1 Level d Level 0 Level d+1 Level m... Consider case where solution is at level d:

28. Brian Williams, Fall 0428 Worst Case Time for Breadth-first Worst case time T is proportional to number of nodes visited Level 1 Level d Level 0 Consider case where solution is at level d: T bfs = [b d+1 + b d + … b + 1 - b]*c bfs ~ O(b d+1 ) for large b Level d+1 b b d+1 - b... 1 bdbd Level m...

29. Brian Williams, Fall 0429 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first ~b d+1 Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G Which is better, depth-first or breadth-first? C S B G A D

30. Brian Williams, Fall 0430 Worst Case Space for Breadth-first Level 1 Level d Level 0 Level d+1 Worst case space S dfs is proportional to maximal length of Q

31. Brian Williams, Fall 0431 Worst Case Space for Breadth-first Level 1 Level d Level 0 S bfs = [b d+1 - b + 1]*c bfs = O(b d+1 ) Level d+1 b b d+1 - b... 1 bdbd Worst case space S dfs is proportional to maximal length of Q

32. Brian Williams, Fall 0432 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first ~b d+1 b d+1 Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G Which is better, depth-first or breadth-first? C S B G A D

33. Brian Williams, Fall 0433 Breadth-first Finds Shortest Path G Level 1 Level d Level 0 G Level d+1 First reached Nodes visited earlier can’t include G Assuming each edge is length 1, other paths to G must be at least as long as first found

34. Brian Williams, Fall 0434 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first ~b d+1 b d+1 Yes unit lngth Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G Which is better, depth-first or breadth-first? C S B G A D

35. Brian Williams, Fall 0435 Cost and Performance Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first ~b d+1 b d+1 Yes unit lngth Yes Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G Which is better, depth-first or breadth-first? C S B G A D

36. Brian Williams, Fall 0436 The Worst of The Worst Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first b m+1 b*m NoYes for finite graph Breadth-first b m+1 bmbm Yes unit lngth Yes Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G Which is better, depth-first or breadth-first? C S B G A D Assume d = m in the worst case, and call both m. Take the conservative estimate: b m + … 1 = O(b m +1)

37. Brian Williams, Fall 0437 Growth for Best First Search b = 10; 10,000 nodes/sec; 1000 bytes/node DepthNodesTimeMemory 21,100.11 seconds1 megabyte 4111,10011 seconds106 megabytes 610 7 19 minutes10 gigabytes 810 9 31 hours1 terabyte 1010 11 129 days101 terabytes 1210 13 35 years10 petabytes 1410 15 3,523 years1 exabyte For best first search, which runs out first – time or memory?

38. Brian Williams, Fall 0438 How Do We Get The Best of Both Worlds? Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~b m b*m NoYes for finite graph Breadth-first ~b d+1 b d+1 Yes unit lngth Yes Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q S D BA CG CG D C G C S B G A D

39. Brian Williams, Fall 0439 Outline Analysis Iterative deepening

40. Brian Williams, Fall 0440 Iterative Deepening (IDS) S D BA CG CG D CG Level 1 Level 2 Level 0 Level 3 Idea: Explore tree in breadth-first order, using depth-first search.  Search tree to depth 1, ….

41. Brian Williams, Fall 0441 Iterative Deepening (IDS) S D BA CG CG D CG Level 1 Level 2 Level 0 Level 3 Idea: Explore tree in breadth-first order, using depth-first search.  Search tree to depth 1, then 2, ….

42. Brian Williams, Fall 0442 Iterative Deepening (IDS) S D BA CG CG D CG Idea: Explore tree in breadth-first order, using depth-first search.  Search tree to depth 1, then 2, then 3…. Level 1 Level 2 Level 0 Level 3

43. Brian Williams, Fall 0443 Speed of Iterative Deepening Compare speed of BFS vs IDS: T bfs = 1+b + b 2 +... b d + (b d+1 – b)= O(b d+2 ) T ids = (d + 1)1 + (d)b + (d - 1)b 2 +... b d = O(b d+1 )  Iterative deepening performs better than breadth-first! S D BA CG CG D CG d*b 1*b d... d+1 2*b d-1 Level 1 Level 2 Level 0 Level d

44. Brian Williams, Fall 0444 Summary Most problem solving tasks may be encoded as state space search. Basic data structures for search are graphs and search trees. Depth-first and breadth-first search may be framed, among others, as instances of a generic search strategy. Cycle detection is required to achieve efficiency and completeness. Complexity analysis shows that breadth-first is preferred in terms of optimality and time, while depth-first is preferred in terms of space. Iterative deepening draws the best from depth-first and breadth-first search.