Lirong Xia Basic search Friday, Jan 24, 2014

TA OH –Joe Johnson (mainly in charge of the project): Wed 12:30-1:30 pm, Amos Eaton 217 –Hongzhao Huang (everything else): Thur 12:30-1:30 pm, Amos Eaton 125 Project0 due on next Tuesday Jan 28 midnight –LMS working! –2 points –submission instruction changed, if you have uploaded project 0, please re- submit it Sign up on piazza by Jan 28 midnight (1 point) Try your best to use piazza for Q/A You are encouraged to stop me and ask questions in class 1 Reminder

2 Last class

3 Welcome back to the earth

Rational agents Search problems –State space graph vs search trees Uninformed search –Depth first search (DFS) algorithm –Breadth first search (BFS) algorithm 4 Today’s schedule

An agent is an entity that perceives and acts. A rational agent selects actions that maximize its utility function. Characteristics of the percepts, environment, and action space dictate techniques for selecting rational actions. Rational Agents 5 Environment Agent Sensors Actuators Actions Percepts

Pacman as an Agent 6 Environment ? Agent Sensors Actuators Percepts Actions

Reflex vs goal-based Agents 7 Reflex agents: –Act on how the world IS –Do not consider the future consequences of their actions Goal-based agents: –Plan ahead –Act on how the world WOULD BE –Must have a model of how the world evolves in response to actions We will be interested in goal-based agents in this class

8 When goal = search for something

9 More general than you may think

10 Can we teach agents how to search for all of these?

11 WARNING: leisure time is over! Let’s do science

Search Problems 12 A search problem consists of: –A state space …… –A successor function (with actions, costs) –A start state and a goal test A solution is a sequence of actions (a plan) which transforms the start state to a goal state

What’s in a State Space? 13 A search state keeps only the details needed (abstraction) The world state specifies every last detail of the environment Problem: Pathing States: (x,y) location Actions: NSEW Successor: adjacent locations Goal test: is (x,y) = END Problem: Eat-All-Dots States: {(x,y), dot booleans} Actions: NSEW Successor: updated location and dot booleans Goal test: dots all false

State Space Sizes? 14 World state: Agent positions: 120 Food count: 30 Ghost positions: 12 Agent facing: NSEW How many World states? States for pathing? States for eat-all-dots?

State Space Graphs 15 State space graph: A mathematical representation of a search problem For every search problem, there’s a corresponding state space graph The successor function is represented by arcs We can rarely build this graph is memory (so we don’t) Ridiculously tiny search graph for a tiny search problem

Search Trees 16 A search tree: This is a “what if” tree of plans and outcomes Start state at the root node Children correspond to successors Nodes contain states, correspond to PLANS to those states For most problems, we can never actually build the whole tree

Generic search algorithm Fringe = set of nodes generated but not expanded = nodes we know we still have to explore fringe := {node corresponding to the initial state} loop: –if fringe empty, declare failure –choose and remove a node v from fringe –check if v’s state s is a goal state; if so, declare success –if not, expand v, insert resulting nodes into fringe Key question in search: Which of the generated nodes do we expand next?

Example 2: Capital Region 18 State space: Cities Successor function: Roads: Go to adjacent city with cost = dist Start state: Cohoes Goal test: Is state == Delmar Solution? Cohoes Latham Troy Loudonville Albany Guilderland Rensselaer Delmar

Another Search Tree 19 Search: Expand out possible plans Maintain a fringe of unexpanded plans Try to expand as few tree nodes as possible Cohoes LathamTroy LoudonvilleTroyLoudonvilleCohoes Latham CLaTA Lo

State Graphs vs. Search Trees 20 State graphs: a representation of the search problem –each node is an abstract of the state of the world Search tree: a tool that helps us to find the solution –each node represents an entire path in the graph –tree nodes are constructed on demand and we construct as little as possible State graph Search trees

States vs. Nodes 21 Nodes in state space graphs are problem states: Represent an abstracted state of the world Have successors, can be goal/non-goal, have multiple predecessors Nodes in search trees are plans Represent a plan (sequence of actions) which results in the node’s state Have a problem state and one parent, a path length, a depth and a cost The same problem state may be achieved by multiple search tree nodes Problem StatesSearch Nodes

Uninformed search Uninformed search: given a state, we only know whether it is a goal state or not Cannot say one nongoal state looks better than another nongoal state Can only traverse state space blindly in hope of somehow hitting a goal state at some point –Also called blind search –Blind does not imply unsystematic!

Breadth-first search

Implementing Breadth-first search Fringe can be maintained as a First-In-First-Out (FIFO) queue fringe := {node corresponding to initial state} loop: –if fringe empty, declare failure –choose and remove the top node v from fringe –check if v’s state s is a goal state; if so, declare success –if not, expand v, insert resulting nodes into fringe

25 Example Start a b c d e GOAL

Properties of breadth-first search Ignores the cost May expand more nodes than necessary BFS is complete: if a solution exists, one will be found BFS finds a shallowest solution –Not necessarily an optimal solution If every node has b successors (the branching factor), shallowest solution is at depth d, then fringe size will be at least b d at some point –This much space (and time) required 

Never expand a node whose state has been visited Fringe can be maintained as a First-In-First-Out (FIFO) queue (class Queue in util.py) Maintain a set of visited states fringe := {node corresponding to initial state} loop: –if fringe empty, declare failure –choose and remove the top node v from fringe –check if v’s state s is a goal state; if so, declare success –if v’s state has been visited before, skip –if not, expand v, insert resulting nodes whose states have not been visited into fringe This is the BFS you should implement in project 1 27 Fixed BFS

Depth-first search

Implementing Depth-first search Fringe can be maintained as a Last-In-First-Out (LIFO) queue (class Stack in util.py) fringe := {node corresponding to initial state} loop: –if fringe empty, declare failure –choose and remove the top node v from fringe –check if v’s state s is a goal state; if so, declare success –if not, expand v, insert resulting nodes into fringe

30 Example Start a b c d e GOAL

Properties of depth-first search Ignores cost Not complete (might cycle through nongoal states) If solution found, generally not optimal/shallowest If every node has b successors (the branching factor), and we search to at most depth m, fringe is at most bm –Much better space requirement –Actually, generally don’t even need to store all of fringe Time: still need to look at every node –b m + b m-1 + … + 1 (for b>1, O(b m )) –Inevitable for uninformed search methods…

Never expand a node whose state has been visited Fringe can be maintained as a Last-In-First-Out (LIFO) queue (class Stack in util.py) Maintain a set of visited states fringe := {node corresponding to initial state} loop: –if fringe empty, declare failure –choose and remove the top node v from fringe –check if v’s state s is a goal state; if so, declare success –if v’s state has been visited before, skip –if not, expand v, insert resulting nodes whose states have not been visited into fringe This is the DFS you should implement in project 1 32 Fixed DFS

Read the instructions on course website and the comments in search.py first Q1: DFS –LIFO Q2: BFS –FIFO Due in two weeks (Feb 7) Check util.py for LIFO and FIFO implementation Use piazza for Q/A 33 You can start to work on Project 1 now

The auto-grader is very strict –0 point for expanding more-than-needed states –no partial credit Hint 1: do not include print "Start:", problem.getStartState() in your formal submission –comment out all debuging commands Hint 2: remember to check if a state has been visited before Hint 3: return a path from start to goal. You should pass the local test before submission (details and instructions on project 1 website) 34 Dodging the bullets

TA OH –Joe Johnson (mainly in charge of the project): Wed 12:30-1:30 pm, Amos Eaton 217 –Hongzhao Huang (everything else): Thur 12:30-1:30 pm, Amos Eaton 125 Project0 due on next Tuesday Jan 28 midnight –LMS working! –2 points –submission instruction changed, if you have uploaded project 0, please re-submit it Project1 due on Feb 7 midnight Sign up on piazza by Jan 28 midnight (1 point) Try your best to use piazza for Q/A You are encouraged to stop me and ask questions in class 35 Reminder

Iterative Deepening 36 Iterative deepening: BFS using DFS as a subroutine: Do a DFS which only searches for paths of length 1 or less. If “1” failed, do a DFS which only searches for paths of length 2 or less. If “2” failed, do a DFS which only searches for paths of length 3 or less. …and so on. AlgorithmCompleteOptimalTimeSpace DFS w/Path Checking YN BFSYN* IDYN*

Costs on Actions 37 Notice that BFS finds the shortest path in terms of number of transitions. It does not find the least-cost path. We will quickly cover an algorithm which does find the least-cost path.

Uniform Cost Search 38 Expand cheapest node first: Fringe is a priority queue (priority: cumulative cost)

Priority Queue Refresher 39 A priority queue is a data structure in which you can insert and retrieve (key, value) pairs with the following operations: You can decrease a key’s priority by pushing it again Unlike a regular queue, insertions aren’t constant time, usually We’ll need priority queues for cost-sensitive search methods pq.push(key, value)Inserts (key, value) into the queue. pq.pop() returns the key with the lowest value, and removes it from the queue

Uniform Cost Search 40 AlgorithmCompleteOptimalTimeSpace DFS w/Path Checking YN BFSYN UCSY*Y * UCS can fail if actions can get arbitrarily cheap

What will UCS do for this graph? What does this mean for completeness? Uniform Cost Search 41

Remember: explores increasing cost contours The good: UCS is complete and optimal! The bad: Explores options in every “direction” No information about goal location Uniform Cost Search 42

Any estimate of how close a state is to a goal Designed for a particular search problem Examples: Manhattan distance, Euclidean distance Search Heuristics 43

Heuristics 44 Cohoes Latham Troy Loudonville Albany Guilderland Rensselaer Delmar Albany4.4 Cohoes12.4 Delmar0 Guilderland7.1 Latham9.3 Loudonville7.0 Rensselaer4.8 Troy10.2 Straight-line distance to Delmar

Best First / Greedy Search 45 Expand the node that seems closest… What can go wrong? Cohoes LathamTroy LoudonvilleTroyLoudonvilleCohoes Latham ( ) ( ) ( )( ) ( ) ( )( ) ( ) LathamTroyAlbany ( )( ) ( ) RensselaerDelmar ( ) LathamTroyAlbany ( )

Best First / Greedy Search 46 Expand the node that seems closest… What can go wrong? Cohoes LathamTroy LoudonvilleTroyCohoes (9.3) (10.2) (12.4) (10.2) (7) LathamTroyAlbany (4.4) (10.2) (9.3) RensselaerDelmar (0) (4.8) LoudonvilleGuilderlandTroy (10.2)(7) (7.1)

Best First / Greedy Search 47 A common case: Best-first takes you straight to the (wrong) goal Worst-case: like a badly- guided DFS in the worst case Can explore everything Can get stuck in loops if no cycle checking Like DFS in completeness (finite states w/ cycle checking)

Search Gone Wrong? 48

Extra Work? 49 Failure to detect repeated states can cause exponentially more work (why?)

Graph Search 50 In BFS, for example, we shouldn’t bother expanding the circled node (why?)

Graph Search 51 Very simple fix: never expand a state type twice Can this wreck completeness? Why or why not? How about optimality? Why or why not?

Some Hints 52 Graph search is almost always better than tree search (when not?) Implement your closed list as a dict or set! Nodes are conceptually paths, but better to represent with a state, cost, last action, and reference to the parent node

Best First Greedy Search 53 AlgorithmCompleteOptimalTimeSpace Greedy Best- First search Y*N What do we need to do to make it complete? Can we make it optimal? Next class!