1. Artificial Intelligence
Search Problem

2. Solving problems by searching

3. Search Problem
Search is a problem-solving technique to explores successive stages in problem-solving process.

4. Search Space
We need to define a space to search in to find a problem solution
To successfully design and implement search algorithm, we must be able to analyze and predict its behavior.

5. State Space Search
One tool to analyze the search space is to represent it as space graph, so by use graph theory we analyze the problem and solution of it.

6. Graph Theory
A graph consists of a set of nodes and a set of arcs or links connecting pairs of nodes.
River2
Island1
Island2
River1

7. Graph structure Nodes = {a, b, c, d, e}
Arcs = {(a,b), (a,d), (b,c),….} d b c e a

8. Tree
A tree is a graph in which two nodes have at most one path between them.
The tree has a root. a b c d e f g h i j

9. Space representation
In the space representation of a problem, the nodes of a graph correspond to partial problem solution states and arcs correspond to steps in a problem-solving process

10. Example
Let the game of Tic-Tac-toe 1 2 3 8 4 7 6 5

11. 1 2 3 8 4 7 6 5 1 2 3 7 4 6 5 8 1 4 3 8 7 6 5 2 1 4 3 7 8 6 5 2 1 4 3 7 6 5 8 2 1 3 7 4 6 5 8 2 1 3 7 4 6 5 8 2 1 4 3 7 6 5 8 2 1 4 3 5 7 6 8 2

12. A simple example: traveling on a graph2 C 9 B 2 3
goal state F A
start state E D 4 3 4

13. Search tree search tree nodes and states are not the same thing!
state = A,
cost = 0
state = B,
cost = 3
state = D,
cost = 3
state = C,
cost = 5
state = F,
cost = 12
goal state!
state = A,
cost = 7
search tree nodes and states are not the same thing!

14. . . . . . . Full search tree goal state! goal state! state = A,
cost = 0
state = B,
cost = 3
state = D,
cost = 3
state = C,
cost = 5
state = F,
cost = 12
state = E,
cost = 7
goal state!
state = A,
cost = 7
state = F,
cost = 11
goal state!
state = B,
cost = 10
state = D,
cost = 10 . . . . . .

15. Problem types Deterministic, fully observable  single-state problem
Solution is a sequence of states
Non-observable  sensorless problem
Problem-solving may have no idea where it is; solution is a sequence
Nondeterministic and/or partially observable
Unknown state space

16. Algorithm types There are two kinds of search algorithm Complete
guaranteed to find solution or prove there is none
Incomplete
may not find a solution even when it exists
often more efficient (or there would be no point)

17. Comparing Searching Algorithms: Will it find a solution? the best one?
Def. : A search algorithm is complete if
whenever there is at least one solution, the
algorithm is guaranteed to find it within a finite
amount of time.
Def.: A search algorithm is optimal if
when it finds a solution, it is the best one

18. Comparing Searching Algorithms: Complexity
Branching factor b of a node is the number of arcs going out
of the node
Def.: The time complexity of a search algorithm is
the worst- case amount of time it will take to run,
expressed in terms of
maximum path length m
maximum branching factor b.
Def.: The space complexity of a search algorithm is
the worst-case amount of memory that the algorithm will use (i.e., the maximum number of nodes on the frontier), also expressed in terms of m and b.

19. Example: the 8-puzzle. Given: a board situation for the 8-puzzle:1 3 8 2 7 5 4 6 1 2 3 5 6 7 4 8
Problem: find a sequence of moves that transform this board situation in a desired goal situation:

20. State Space representation
In the space representation of a problem, the nodes of a graph correspond to partial problem solution states and arcs correspond to steps (action) in a problem-solving process

21. Sizes of State Spaces Tic-Tac-Toe 39 8 Puzzle 105 .01 seconds
Brute-Force Search Time (10 million nodes/second)
Problem
Nodes
Tic-Tac-Toe 39
8 Puzzle seconds
23 Rubik’s Cube seconds
15 Puzzle days
33 Rubik’s Cube ,000 years
24 Puzzle billion years
Checkers
Chess

22. Key concepts in search Set of states that we can be in
Including an initial state…
… and goal states (equivalently, a goal test)
For every state, a set of actions that we can take
Each action results in a new state
Given a state, produces all states that can be reached from it
Cost function that determines the cost of each action (or path = sequence of actions)
Solution: path from initial state to a goal state
Optimal solution: solution with minimal cost

23. Search problem components
Initial state
Actions
Transition model
What is the result of performing a given action in a given state?
Goal state
Path cost
Assume that it is a sum of nonnegative step costs
The optimal solution is the sequence of actions that gives the lowest path cost for reaching the goal
Initial
state
Goal state

24. Example: Route Finding
Initial state
City journey starts in
Operators
Driving from city to city
Goal test
Is current location the destination city?
Liverpool
Leeds
Nottingham
Manchester
Birmingham
London

25. State space representation (salesman)
the list of cities that are already visited
Ex.: ( NewYork, Boston )
Initial state:
Ex.: ( NewYork )
Rules:
add 1 city to the list that is not yet a member
add the first city if you already have 5 members
Goal criterion:
first and last city are equal

26. Example: The 8-puzzle states? locations of tiles
actions? move blank left, right, up, down
goal? = goal state (given)
path cost? 1 per move

27. Search Example: Remove 5 Sticks Problem
Remove exactly 5 of the 17 sticks so the resulting figure forms exactly 3 squares

28. Example: robotic assembly
states?: real-valued coordinates of robot joint angles parts of the object to be assembled
actions?: continuous motions of robot joints
goal test?: complete assembly
path cost?: time to execute

29. 1 2 3 8 4 7 6 5 1 3 7 4 6 5 8 2 1 4 3 7 6 5 8 2 1 4 3 7 8 6 5 2 1 4 3 7 6 5 8 2 1 3 7 4 6 5 8 2 1 3 7 4 6 5 8 2 4 3 1 7 6 5 8 2 1 4 3 5 7 6 8 2

30. A State Space Graph What is the corresponding search tree? GOAL d b p
START
GOAL d b p q c e h a f r t u v
What is the corresponding search tree?

31. Example: Chess Problem: develop a program that plays chess1 2 3 4 5 6 7 8
A B C D E F G H
1. A way to represent board situations
Ex.:
List:
(( king_black, 8, C),
( knight_black, 7, B),
( pawn_black, 7, G),
( pawn_black, 5, F),
( pawn_white, 2, H),
( king_white, 1, E))

32. Chess search tree ~15 Move 1 Move 2 ~ (15)2 ~ (15)3 Move 3
Need very efficient search techniques to find good
paths in such combinatorial trees.

33. independence of states:
Ex.: Blocks world problem.
Initially: C is on A and B is on the table.
Rules: to move any free block to another or to the table
Goal: A is on B and B is on C. A C B
Goal: A on B and B on C
Goal: A on B
Goal: B on C
AND-OR-tree?
AND

34. Search in State Spaces
Effects of moving a block (illustration and list-structure iconic model notation)

35. Avoiding Repeated States
In increasing order of effectiveness in reducing size of state space and with increasing computational costs:
1. Do not return to the state you just came from.
2. Do not create paths with cycles in them.
3. Do not generate any state that was ever created before.
Net effect depends on frequency of “loops” in state space.

36. Forward versus backward reasoning:
Initial states
Goal states
Forward reasoning (or Data-driven): from initial states to goal states.

37. Forward versus backward reasoning:
Initial states
Goal states
Backward reasoning (or backward chaining / goal-driven ): from goal states to initial states.

38. Data-Driven search
It is called forward chaining
The problem solver begins with the given facts and a set of legal moves or rules for changing state to arrive to the goal.

39. Goal-Driven Search
Take the goal that we want to solve and see what rules or legal moves could be used to generate this goal.
So we move backward.

40. Search Implementation
In both types of moving search, we must find the path from start state to a goal.
We use goal-driven search if
The goal is given in the problem
There exist a large number of rules
Problem data are not given

41. Search Implementation
The data-driven search is used if
All or most data are given
There are a large number of potential goals
It is difficult to form a goal

42. Criteria: Sometimes equivalent:1 3 8 2 7 5 4 6
In this case: even the same rules !!
Sometimes equivalent:
Sometimes: no way to start from the goal states
because there are too many (Ex.: chess)
because you can’t (easily) formulate the rules in 2 directions.

43. General Search Considerations
Given initial state, operators and goal test
Can you give the agent additional information?
Uninformed search strategies
Have no additional information
Informed search strategies
Uses problem specific information
Heuristic measure (Guess how far from goal)

44. Uninformed search strategies
Breadth-first search
Depth-first search
Bidirectional search
Depth-bounded depth first search
like depth first but set limit on depth of search in tree
Iterative Deepening search
use depth-bounded search but iteratively increase limit

45. Breadth-first search:G
Move downwards, level by level, until goal is reached.
It explores the space in a level-by-level fashion.

46. Breadth-First Search (BFS)‏
Expand the shallowest node first:
Examine states one step away from the initial states
Examine states two steps away from the initial states
and so on
Goal

47. Breadth-first search
BFS is complete: if a solution exists, one will be found
Implementation:
fringe is a FIFO queue, i.e., new successors go at end

48. Breadth-first search Implementation:
fringe is a FIFO queue, i.e., new successors go at end

49. Breadth-first search Implementation:
fringe is a FIFO queue, i.e., new successors go at end

50. Breadth-first search Implementation:
fringe is a FIFO queue, i.e., new successors go at end

51. Breadth-First Search (BFS)
generalSearch(problem, queue) 5 2 9 6 4 1 7
S start A E D F B G
goal C H
# of nodes tested: 0, expanded: 0
expnd. node
Frontier list
{S}
CLIKC ANIM: 1. generalSearch, 2. graph, 3. table, 4. # expanded

52. Analysis of BFS Def. : A search algorithm is complete if
whenever there is at least one solution, the
algorithm is guaranteed to find it within a finite
amount of time.
Is BFS complete?
Yes
If a solution exists at level l, the path to it will be explored before any other path of length l + 1
impossible to fall into an infinite cycle
see this in AISpace by loading “Cyclic Graph Examples” or by adding a cycle to “Simple Tree”

53. Analysis of BFS Def.: A search algorithm is optimal if
when it finds a solution, it is the best one
Is BFS optimal?
Yes
E.g., two goal nodes: red boxes
Any goal at level l (e.g. red box N 7) will be reached before goals at lower levels

54. Analysis of BFS Def.: The time complexity of a search algorithm is
the worst-case amount of time it will take to run,
expressed in terms of
maximum path length m
maximum forward branching factor b.
What is BFS’s time complexity, in terms of m and b ?
O(bm)
Like DFS, in the worst case BFS must examine every node in the tree
E.g., single goal node -> red box

55. Analysis of BFS
Def.: The space complexity of a search algorithm is the
worst case amount of memory that the algorithm will use
(i.e., the maximal number of nodes on the frontier), expressed in terms of
maximum path length m
maximum forward branching factor b.
What is BFS’s space complexity, in terms of m and b ?
BFS must keep paths to all the nodes al level m
O(bm)

56. Using Breadth-first Search
When is BFS appropriate?
space is not a problem
it's necessary to find the solution with the fewest arcs
When there are some shallow solutions
there may be infinite paths
When is BFS inappropriate?
space is limited
all solutions tend to be located deep in the tree
the branching factor is very large

57. Depth-First Order
When a state is examined, all of its children and their descendants are examined before any of its siblings.
Not complete (might cycle through nongoal states)
Depth- First order goes deeper whenever this is possible.

58. Depth-First Search Expand the deepest node first
Select a direction, go deep to the end
Slightly change the end
Slightly change the end some more…
Use a Stack to order nodes on the Frontier
Goal

59. Depth-first search = Chronological backtracking
Select a child
convention: left-to-right
Repeatedly go to next child, as long as possible.
Return to left-over alternatives (higher-up) only when needed. A B C E F D G

60. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

61. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

62. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

63. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

64. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

65. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

66. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

67. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

68. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

69. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

70. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

71. Depth-first search Implementation:
fringe = LIFO queue, i.e., put successors at front

72. Depth-First Search (DFS)
generalSearch(problem, stack)
S start
# of nodes tested: 0, expanded: 0
expnd. node
Frontier
{S} 5 2 4 A B C
CLICK DESC: 1. interior nodes, 2. leaf nodes, 3. arcs 9 4 6 2 D E 6 G
goal 1 F 7 H

73. Analysis of DFS Is DFS complete? Is DFS optimal?.
Is DFS optimal?
What is the time complexity, if the maximum path length is m and the maximum branching factor is b ?
What is the space complexity?
We will look at the answers in AISpace (but see next few slides for a summary of what we do)

74. Analysis of DFS
Def. : A search algorithm is complete if whenever there is at least one
solution, the algorithm is guaranteed to find it within a finite
amount of time.
Is DFS complete? No
If there are cycles in the graph, DFS may get “stuck” in one of them
see this in AISpace by loading “Cyclic Graph Examples” or by adding a cycle to “Simple Tree”
e.g., click on “Create” tab, create a new edge from N7 to N1, go back to “Solve” and see what happens

75. Analysis of DFS Def.: A search algorithm is optimal if
when it finds a solution, it is the best one (e.g., the shortest)
Is DFS optimal? No
It can “stumble” on longer solution paths before it gets to shorter ones.
E.g., goal nodes: red boxes
see this in AISpace by loading “Extended Tree Graph” and set N6 as a goal
e.g., click on “Create” tab, right-click on N6 and select “set as a goal node”

76. Analysis of DFS Def.: The time complexity of a search algorithm is
the worst-case amount of time it will take to run,
expressed in terms of
maximum path length m
maximum forward branching factor b.
What is DFS’s time complexity, in terms of m and b ?
O(bm)
In the worst case, must examine every node in the tree
E.g., single goal node -> red box

77. Analysis of DFS
Def.: The space complexity of a search algorithm is the
worst-case amount of memory that the algorithm will use
(i.e., the maximum number of nodes on the frontier), expressed in terms of
maximum path length m
maximum forward branching factor b.
What is DFS’s space complexity, in terms of m and b ?
for every node in the path currently explored, DFS maintains a path to its unexplored siblings in the search tree
Alternative paths that DFS needs to explore
The longest possible path is m, with a maximum of b-1 alterative paths per node
O(bm)
See how this
works in

78. Analysis of DFS (cont.) DFS is appropriate when. Space is restricted
Many solutions, with long path length
It is a poor method when
There are cycles in the graph
There are sparse solutions at shallow depth
There is heuristic knowledge indicating when one path is better than another

79. Press space to see a BFS of the example node set
Initial state A A B C D E F
Goal state G H I J K L L M N O P Q R S T U V W X Y Z
Press space to see a BFS of the example node set

80. A A B B C C D D E E F F G G H H I I J J K K L L L L L L M N O P Q R S
This node is then expanded to reveal further (unexpanded) nodes. Press space
Node A is removed from the queue. Each revealed node is added to the END of the queue. Press space to continue the search.
We begin with our initial state: the node labeled A. Press space to continue
Node B is expanded then removed from the queue. The revealed nodes are added to the END of the queue. Press space.
We then backtrack to expand node C, and the process continues. Press space
The search then moves to the first node in the queue. Press space to continue. A A B B C C D D E E F F G G H H I I J J K K L L L L L L M N O P
Node L is located and the search returns a solution. Press space to end. Q R S T U
Press space to begin the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Press space to continue the search
Size of Queue: 0
Size of Queue: 10
Size of Queue: 9
Size of Queue: 6
Size of Queue: 8
Size of Queue: 1
Size of Queue: 0
Size of Queue: 7
Size of Queue: 5
Queue: Empty
Queue: G, H, I, J, K, L, M, N, O, P
Queue: A
Queue: I, J, K, L, M, N, O, P, Q, R
Queue: L, M, N, O, P, Q, R, S, T, U
Queue: Empty
Queue: E, F, G, H, I, J, K, L
Queue: D, E, F, G, H, I, J
Queue: J, K, L, M, N, O, P, Q, R, S
Queue: K, L, M, N, O, P, Q, R, S, T
Queue: F, G, H, I, J, K, L, M, N
Queue: B, C, D, E, F
Queue: H, I, J, K, L, M, N, O, P, Q
Queue: C, D, E, F, G, H
Nodes expanded: 8
Nodes expanded: 9
Nodes expanded: 10
Nodes expanded: 11
Nodes expanded: 7
Nodes expanded: 4
Nodes expanded: 5
Nodes expanded: 3
Nodes expanded: 6
Nodes expanded: 2
Nodes expanded: 0
Nodes expanded: 1
Current Action: Backtracking
Current Action: Backtracking
Current Action: Expanding
Current Action:
Current Action: Expanding
Current Action: Expanding
Current Action: Backtracking
Current Action: Backtracking
Current Action: Backtracking
Current Action: Expanding
Current Action: Expanding
Current Action: Expanding
FINISHED SEARCH
Current Action: Backtracking
Current Action: Backtracking
Current Action: Backtracking
Current Action: Backtracking
Current Action: Expanding
Current Action: Expanding
Current Action: Expanding
Current Action: Backtracking
Current Action: Expanding
Current level: 2
Current level: 1
Current level: 0
Current level: 0
Current level: 1
Current level: 1
Current level: 1
Current level: 0
Current level: 0
Current level: 1
Current level: n/a
Current level: 0
Current level: 1
Current level: 0
Current level: 1
Current level: 2
Current level: 2
Current level: 1
Current level: 1
Current level: 0
Current level: 1
Current level: 2
Current level: 1
Current level: 1
Current level: 2
Current level: 1
Current level: 2
Current level: 0
BREADTH-FIRST SEARCH PATTERN

81. Aside: Internet Search
Typically human search will be “incomplete”,
E.g. finding information on the internet before google, etc
look at a few web pages,
if no success then give up

82. Iterative deepening search

83. Iterative deepening search

84. Iterative deepening search

85. Iterative deepening search

86. Example
Determine whether data-driven or goal-driven and depth-first or breadth-first would be preferable for solving each of the following
Diagnosing mechanical problems in an automobile
You have met a person who claims to be your distant cousin, with a common ancestor named John. You like to verify her claim
A theorem prover for plane geometry

87. Example A program for examining sonar readings and interpreting them
An expert system that will help a human classify plants by species, genus,etc.

88. Any path, versus shortest path, versus best path:
Ex.: Traveling salesperson problem:
Find a sequence of cities ABCDEA such that the total distance is MINIMAL.
Boston
Miami
NewYork
SanFrancisco
Dallas
3000
250
1450
1200
1700
3300
2900
1500
1600

89. Bi-directional search
IF you are able to EXPLICITLY describe the GOAL state, AND you have BOTH rules for FORWARD reasoning AND BACKWARD reasoning
Compute the tree both from the start-node and from a goal node, until these meet.
Start
Goal

90. Example S 1 5 8 A B C 3 7 9 4 5 D E G How are nodes expanded by A B C
How are nodes expanded by
Depth First Search
Breadth First Search
Iterative Deepening
Are the solutions the same? D E G

91. Example Search Problem
A genetics professor
Wants to name her new baby boy
Using only the letters D,N & A
Search through possible strings (states)
D,DN,DNNA,NA,AND,DNAN, etc.
3 operators: add D, N or A onto end of string
Initial state is an empty string
Goal test
Look up state in a book of boys’ names, e.g. DAN

92. G(n) = The cost of each move as the distance between each town H(n) = The Straight Line Distance between any town and town M. 10 40 12 5 15 20 23 M L K H J I G F E D C B A A 45 E 32 I 12 M B 20 F 23 J 5 C 34 G 15 K 40 D 25 H 10 L

94. Consider the following search problem
Consider the following search problem. Assume a state is represented as an integer, that the initial state is the number 1, and that the two successors of a state n are the states 2n and 2n+1. For example, the successors of 1 are 2 and 3, the successors of 2 are 4 and 5, the successors of 3 are 6 and 7, etc. Assumes the goal state is the number 12. Consider the following heuristics for evaluating the state n where the goal state is g
h1(n) = |n-g| & h2(n) = (g – n) if (n  g) and h2 (n) =  if (n >g)
Show the search trees generated for each of the following strategies for the initial state 1 and the goal state 12, numbering the nodes in the order expanded.
Depth-first search b) Breadth-first search
c) beast-first with heuristic h1 d) A* with heuristic (h1+h2)
If any of these strategies get lost and never find the goal, then show the few steps and say "FAILS"