1. CSC-470: ARTIFICIAL INTELLIGENCE
Lecturer
Muhammad Tariq Siddique
Lecture 3: Uninformed Search

2. Outline Uninformed search Search strategies Problem formulation
Uniform Search Strategies
Depth First Search
Breadth First Search
Uniform cost Search
Depth-Limit Search
Iterative Deepening Search
Bidirectional Search

3. Search Strategies
A strategy is defined by picking the order of node expansion
Strategies are evaluated along the following dimensions:
completeness – does it always find a solution if one exists?
optimality – does it always find a least-cost solution?
time complexity – number of nodes generated/expanded
space complexity – maximum number of nodes in memory
Time and space complexity are measured in terms of
b – maximum branching factor of the search tree
d – depth of the least-cost solution
m – maximum depth of the state space (may be infinite)

4. Search Strategies
The effectiveness of a search algorithm may only consist of search cost (depends on time complexity) OR
It may also include memory usage.
Total cost, which combine the search cost and the path cost of the solution found.

5. Search Strategies
For example: For the path finding problem from Arad to Bucharest, the search cost is the amount of time by the search and the solution cost is the total length of the path in Kilometers. Total cost = KM + Milliseconds. (officially no exchange rate) So KM convert into milliseconds by estimating the car average speed (b/c time is what the agent cares about). Thus it enable the agent to find an optimal tradeoff point for further computation to find a shorter path.

6. Uninformed Search Strategies
Uninformed (blind) strategies use only the information available in the problem definition
vs. informed search techniques which might have additional information (e.g. a compass).
Breadth-first search
Depth-first search
Uniform-cost search
Depth-limited search
Iterative deepening search
Bidirectional Search
All uninformed search can generate successors and distinguish a goal state from a nongoal state.

7. Breadth-First Search (BFS)
Root node is expanded first  successor of root node  their successors
All nodes are expanded at a given depth in the search tree before any nodes at the next level are expanded
Easy to implement by using FIFO queue

8. Breadth-First Search (BFS)
Successors: B,C,D D B C
Initial state G H I J E F K N O P Q R L M
Goal state S T U
Fringe: A (FIFO)
Visited:

9. Breadth-First Search (BFS)
Next node D B C
Successors: E,F G H I J E F K N O P Q R L M S T U
Fringe: B,C,D (FIFO)
Visited: A

10. Breadth-First Search (BFS)
Next node D
Successors: G,H B C G H I J E F K N O P Q R L M S T U
Fringe: C,D,E,F (FIFO)
Visited: A, B

11. Breadth-First Search (BFS)
Next node
Successors: I,J D B C G H I J E F R K N O P Q L M S T U
Fringe: D,E,F,G,H (FIFO)
Visited: A, B, C

12. Breadth-First Search (BFS)
Successors: K,L D B C
Next node G H I J E F K N O P Q R L M S T U
Fringe: E,F,G,H,I,J (FIFO)
Visited: A, B, C, D

13. Breadth-First Search (BFS)
Successors: M D B C
Next node G H I J E F K N O P Q R L M S T U
Fringe: F,G,H,I,J,K,L (FIFO)
Visited: A, B, C, D, E

14. Breadth-First Search (BFS)
Successors: N D B C
Next node G H I J E F P K N O Q R L M U S T
Fringe: G,H,I,J,K,L,M (FIFO)
Visited: A, B, C, D, E, F

15. Breadth-First Search (BFS)
Successors: O D B C
Next node G H I J E F K N O P Q R L M U S T
Fringe: H,I,J,K,L,M,N (FIFO)
Visited: A, B, C, D, E, F, G

16. Breadth-First Search (BFS)
Successors: P,Q D B C
Next node G H I J E F K N O P Q R L M S T U
Fringe: I,J,K,L,M,N,O (FIFO)
Visited: A, B, C, D, E, F, G, H

17. Breadth-First Search (BFS)
Successors: R D B C
Next node G H I J E F K N O P Q R L M S T U
Fringe: J,K,L,M,N,O,P,Q (FIFO)
Visited: A, B, C, D, E, F, G, H, I

18. Breadth-First Search (BFS)
Successors: S D B C G H I J E F
Next node K N O P Q R L M S T U
Fringe: K,L,M,N,O,P,Q,R (FIFO)
Visited: A, B, C, D, E, F, G, H, I, J

19. Breadth-First Search (BFS)
Successors: T D B C G H I J E F
Next node K L N O P Q R M S T U
Fringe: L,M,N,O,P,Q,R,S (FIFO)
Visited: A, B, C, D, E, F, G, H, I, J, K

20. Breadth-First Search (BFS)
Successors: D B C G H I J E F K N O P Q R L M
Next node S T U
Fringe: M,N,O,P,Q,R,S,T (FIFO)
Visited: A, B, C, D, E, F, G, H, I, J, K, L

21. Breadth-First Search (BFS)
Goal state achieved
Successors: D B C G H I J E F K O P Q R L M N
Next node S T U
Fringe: N,O,P,Q,R,S,T (FIFO)
Visited: A, B, C, D, E, F, G, H, I, J, K, L, M

22. Evaluating Breadth-First Search
Observations
Very systematic
If there is a solution breadth first search is guaranteed to find it
If there are several solutions then breadth first search will always find the shallowest goal state first and if the cost of a solution is a non- decreasing function of the depth then it will always find the cheapest solution

23. Properties of Breadth-First Search
Completeness: Yes, if b is finite
Time complexity: 1+b+b2+…+bd = O(b d), i.e., exponential in d
Space complexity: O(b d)
Optimality: Yes (assuming cost = 1 per step)
b – maximum branching factor of the search tree
d – depth of the least-cost solution
m – maximum depth of the state space (may be infinite)

24. Breadth-First Search Search Pattern: spread before dive Fringe in red
Visited in blue
Size of fringe
O(bd)
exponential 
root
outline of tree …

25. Exponential Growth
Time and memory requirements for breadth-first search, assuming a branching factor of 10, 100 bytes per node and searching 1000 nodes/second

26. Breadth-First Search (BFS)
Although breadth-first search is optimal when all step costs are equal -
O(bd) is scary for both time and memory required
The memory requirements are a bigger problem for breadth-first search than is the execution time.
Time is a problem, it will take 3500 years to find a solution with depth 14.

27. Depth-First Search (DFS)
Expands the deepest node in the current frontier of the search tree until the nodes have no successors, then backs up to the next deepest node that still has unexplored successors.
Use LIFO queue
The most recently generated node is chosen for expansion, the deepest unexpanded node
Use recursive

28. Depth-First Search (DFS)
Successors: B,C,D D B C
Initial state G H I J E F K N O P Q R L M S T U
Goal state
Fringe: A (LIFO)
Visited:

29. Depth-First Search (DFS)
Successors: E,F D B C G H I J E F K N O P Q R L M S T U
Visited: A
Fringe: B,C,D (LIFO)

30. Depth-First Search (DFS)
Successors: K,L D B C G H I J E F K N O P Q R L M S T U
Visited: A, B
Fringe: E,F,C,D (LIFO)

31. Depth-First Search (DFS)
Successors: S D B C G H I J E F K N O P Q R L M S T U
Fringe: K,L,F,C,D (LIFO)
Visited: A, B, E

32. Depth-First Search (DFS)
Successors: D B C G H I J E F K N O P Q R L M S T U
Visited: A, B, E, K
Fringe: S,L,F,C,D (LIFO)

33. Depth-First Search (DFS)
Successors: T D B C G H I J E F K N O P Q R L M S T U
Visited: A, B, E, K, S
Fringe: L,F,C,D (LIFO)
Backtracking

34. Depth-First Search (DFS)
Successors: D B C G H I J E F K N O P Q R L M S T U
Fringe: T,F,C,D (LIFO)
Visited: A, B, E, K, S, L

35. Depth-First Search (DFS)
Successors: M D B C G H I J E F K N O P Q R L M S T U
Visited: A, B, E, K, S, L, T
Fringe: F,C,D (LIFO)
Backtracking

36. Depth-First Search (DFS)
Successors: B C G H I J E F K N O P Q R L M S T U
Fringe: M,C,D (LIFO)
Visited: A, B, E, K, S, L, T, F

37. Depth-First Search (DFS)
Successors: G,H D B C G H I J E F K N O P Q R L M S T U
Visited: A, B, E, K, S, L, T, F, M
Fringe: C,D (LIFO)
Backtracking

38. Depth-First Search (DFS)
Successors: N B C G H I J E F K N O P Q R L M S T U
Fringe: G,H,D (LIFO)
Visited: A, B, E, K, S, L, T, F, M, C

39. Depth-First Search (DFS)
Goal state achieved
Successors: D B C
Finished search G H I J E F K N O P Q R L M U S T
Fringe: N,H,D (LIFO)
Visited: A, B, E, K, S, L, T, F, M, C, G

40. Depth-First Search - Observations
Only needs to store the path from the root to the leaf node as well as the unexpanded nodes. For a state space with a branching factor of b and a maximum depth of m, DFS requires storage of bm nodes
Time complexity for DFS is bm in the worst case
If DFS goes down a infinite branch it will not terminate if it does not find a goal state.
If it does find a solution there may be a better solution at a lower level in the tree. Therefore, depth first search is neither complete nor optimal.

41. Depth-First Search (DFS)
Image from Russel & Norvig, AIMA, 2003
Keeps O(bm) nodes in memory.
Requires a depth limit to avoid infinite paths (limit is 4 in the figure).
Incomplete (is not guaranteed to find a solution)
Not optimal
Linear space complexity (good)
Exponential time complexity
Black nodes are removed from memory
b = branching factor, m = depth

42. Depth-First Search (DFS)
Search Pattern: dive before spread
Fringe in red
Visited in blue
Size of fringe
O(bd)
linear 
root …
outline of tree

43. Depth-First Search (DFS)
Much better than BFS in space complexity
Graph-search : no advantage
A depth-first tree search – need to store only a single path from root to leaf, and remaining unexpanded sibling nodes . Once expanded, it can be removed from memory.

44. Properties of Depth-First Search
Completeness: No, fails in infinite state-space
(yes if finite state space)
Time complexity: O(b m)
Space complexity: O(bm)
Optimality: No
Remember:
b = branching factor
m = max depth of search tree

45. Avoiding repeated states
In increasing order of effectiveness and computational overhead:
do not return to state we come from, i.e., expand function will skip possible successors that are in same state as node’s parent.
do not create paths with cycles, i.e., expand function will skip possible successors that are in same state as any of node’s ancestors.
do not generate any state that was ever generated before, by keeping track (in memory) of every state generated, unless the cost of reaching that state is lower than last time we reached it.

46. Comparing BFS and DFS
BFS: better if the branching factor of the graph is small
Pros: No dead end, Complete and optimum
Cons: time & memory consuming
DFS: better if tree is deep but a solution is known to be shallow in the graph
Pros: small amount of memory, accidentally can find solution in short time
Cons: incomplete possibility (tree-search allow redundant path)

47. Uniform-Cost Search (UCS)
Extension from breadth-first search
Instead of expanding the shallowest node, expand the node with the lowest path cost g(n)
Using priority queue ordered by g
Uniformed cost search care about total cost of the search rather than number of steps the paths has.
May go to infinite loop if it ever expand a node with cost zero leading back to the same state.

48. Uniform-Cost Search (Dijkstra, 1959)
Let g(n) be path cost of node n.
Expand least-cost unexpanded node
Implementation:
QueueingFn = insert in order of increasing path length
Arad
140
Zerind
Sibiu
Timisoara 75
118 75
140
118
Arad
Oradea 75 71
Arad
Lugoj
118
111
150
146
236
229

49. Uniform Cost Search (UCS)5 2
[2]
[5] 1 4 1 7
[6]
[3]
[9]
[9]
Goal state 4 5
[x] = g(n)
path cost of node n
[7]
[8]

50. Uniform Cost Search (UCS)2 5
[5]
[2]

51. Uniform Cost Search (UCS)5 2
[2]
[5] 1 7
[3]
[9]

52. Uniform Cost Search (UCS)5 2
[2]
[5] 1 7
[3]
[9] 4 5
[7]
[8]

53. Uniform Cost Search (UCS)2 5 1 7 4
[5]
[2]
[9]
[3]
[7]
[8]
[6]

54. Uniform Cost Search (UCS)2 5 1 7 4
[5]
[2]
[9]
[3]
[7]
[8]
Goal state path cost g(n)=[6]

55. Uniform Cost Search (UCS)2 5 1 7 4
[5]
[2]
[9]
[3]
[7]
[8]
[6]

56. Properties of Uniform-Cost Search
Complete: Yes if arc costs > 0.
Optimal: Yes
Time:
Space:
Note: Breadth first is equivalent to Uniform-Cost Search if step cost is equal.
g = goal
n = node

57. Depth-Limited Search (DLS)
Is a depth-first search with a predetermined depth limit l
Implementation: Nodes at depth l are treated as if they have no successors.
Complete: if cutoff chosen appropriately then it is guaranteed to find a solution.
Optimal: it does not guarantee to find the least-cost solution (If choosing l >d )

58. Depth-Limited Search (DLS)
Given the following state space (tree search), give the sequence of visited nodes when using DLS (Limit = 2):
Limit = 0 A B C E D F G H I J K L O M N
Limit = 1
Limit = 2

59. Depth-Limited Search (DLS)B C D E
Limit = 2

60. Depth-Limited Search (DLS)
A,B, A B C E D F G
Limit = 2

61. Depth-Limited Search (DLS)
A,B,F, A B C E D F G
Limit = 2

62. Depth-Limited Search (DLS)
A,B,F, G, A B C D E
Limit = 2 F G

63. Depth-Limited Search (DLS)
A,B,F, G, C, A B C E D F G H
Limit = 2

64. Depth-Limited Search (DLS)
A,B,F, G,
C,H, A B C D E
Limit = 2 F G H

65. Depth-Limited Search (DLS)
A,B,F, G,
C,H, D, A B C D E
Limit = 2 F G H I J

66. Depth-Limited Search (DLS)
A,B,F, G,
C,H,
D,I A B C D E
Limit = 2 F G H I J

67. Depth-Limited Search (DLS)
A,B,F, G,
C,H,
D,I J, A B C D E
Limit = 2 F G H I J

68. Depth-Limited Search (DLS)
A,B,F, G,
C,H,
D,I J, E A B C D E
Limit = 2 F G H I J

69. Depth-Limited Search (DLS)
A,B,F, G,
C,H,
D,I J,
E, Failure A B C D E
Limit = 2 F G H I J

70. Depth-Limited Search (DLS)
DLS algorithm returns Failure (no solution)
The reason is that the goal is beyond the limit (Limit =2): the goal depth is (d=4) A B C E D F G H I J K L O M N
Limit = 2

71. How to choose depth limit l
E.g. Romania case, 20 cities
The longest is 19, so l can be 19
From study, cities can be reached by other cities by 9 steps (diameter)
If diameter is known, more efficient depth-limited search
Depth limit search can be terminated by
Failure  no solution
Cutoff  no solution within the depth limit

72. Iterative Deepening Search
It’s a Depth First Search, but it does it one level at a time, gradually increasing the limit, until a goal is found.
Combine the benefits of depth-first and breadth-first search
like DFS, modest memory requirements O(bd)
like BFS, it is complete when branching factor is finite, and optimal when the path cost is a nondecreasing function of the dept of the node

73. Iterative Deepening Search
May seem wasteful because states are generated multiple times
but actually not very costly, because nodes at the bottom level are generated only once
In practice, however, the overhead of these multiple expansions is small, because most of the nodes are towards leaves (bottom) of the search tree:
thus, the nodes that are evaluated several times (towards top of tree) are in relatively small number.
Iterative depending is the preferred uninformed search method when the search space is large and the depth of the solution is unknown

74. Iterative Deepening Search l =0

75. Iterative Deepening Search l =1

76. Iterative Deepening Search l =2

77. Iterative Deepening Search l =3

78. Iterative Deepening Search
Combines the best of breadth-first and depth-first search strategies.
Completeness: Yes,
Time complexity: O(b d)
Space complexity: O(bd)
Optimality: Yes, if step cost = 1

79. Properties of Iterative Deepening Search
(b finite)
Time? d b1 + (d-1)b2 + … + bd = O(bd)
Space? O(bd)
Optimal? Yes, if step costs identical

80. Bidirectional Search
Both search forward from initial state, and backwards from goal.
Stop when the two searches meet in the middle.
Motivation: bd/2 + bd/2 is much less than bd
Implementation
Replace the goal test with a check to see whether the frontiers of the two searches intersect, if yes  solution is found
Goal
Start

81. Bidirectional Search S Forward Backwards A D B D A E C E E B B F 11 DG 14 17 15 15 13 G C G F 19 19 17 G 25

82. Bidirectional Search Not always optimal, even if both searches are BFS
Check when each node is expanded or selected for expansion
Can be implemented using BFS or iterative deepening (but at least one frontier needs to be kept in memory)
Significant weakness
Space requirement
Time Complexity is good

83. Bidirectional Search Problem: how do we search backwards from goal??
predecessor of node n = all nodes that have n as successor
this may not always be easy to compute!
if several goal states, apply predecessor function to them just as we applied successor (only works well if goals are explicitly known; may be difficult if goals only characterized implicitly).
for bidirectional search to work well, there must be an efficient way to check whether a given node belongs to the other search tree.
select a given search algorithm for each half.

84. Bidirectional Search Completeness: Yes,
Time complexity: 2*O(b d/2) = O(b d/2)
Space complexity: O(b m/2)
Optimality: Yes
To avoid one by one comparison, we need a hash table of size O(b m/2)
If hash table is used, the cost of comparison is O(1)

85. Comparison Uninformed Search Strategies

86. Conclusions
Interesting problems can be cast as search problems, but you’ve got to set up state space
Problem formulation usually requires abstracting away real-world details to define a state space that can be explored using computer algorithms.
Once problem is formulated in abstract form, complexity analysis helps us picking out best algorithm to solve problem.
Variety of uninformed search strategies; difference lies in method used to pick node that will be further expanded.

87. Reading Assignments
Problem Solving Agents. What is a problem? Problem formulation, a general search algorithm. Read Chapter 3 from Russell & Norvig
Chapter 3
Exercises 3.1, 3.3,3.7