Graph-based Planning Brian C. Williams Sept. 25 th & 30 th, J/6.834J

Outline  Introduction  The Graph Plan Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with Failure

Graph Plan  Graphplan was developed in 1995 by Avrim Blum and Merrick Furst, at CMU.  Recent implementations by other researchers have extended the capability of Graphplan to reason with temporally extended actions, metrics and non-atomic preconditions and effects.

Approach: Graph Plan 1.Constructs compact encoding of state space from operators and initial state, which prunes many invalid plans – Plan Graph. 2.Generates plan by searching for a consistent subgraph that achieves the goals. Proposition Init State Action Time 1 Proposition Time 1 Action Time 2

Plan graphs focus towards valid plans  Plan graphs exclude many plans that are infeasible.  Plans that do not satisfy the initial or goal state.  Plans with operators that threaten each other.  Plan graphs are constructed in polynomial time and are of polynomial in size.  The plan graph does not eliminate all infeasible plans.  Plan generation still requires focused search.

Example: Graph and Solution noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Outline  Introduction  The Graph Plan Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with failure

8 GraphPlan Planning Problem State A consistent conjunction of propositions (positive literals) E.g., (and (cleanhands) ( quiet) (dinner) (present)(noGarbage)) Doesn’t commit to the truth of all propositions Initial State Problem state at time i = 0 E.g., (and (cleanHands) (quiet)) Goal State A partial state, represented by a conjunction of literals. E.g., (and (noGarbage) (dinner) (present)) The planner must put the system in a final state that satisfies the conjunction.

9 GraphPlan Planning Problem Actions E.g., (:operator carry :precondition :effect (:and (noGarbage) (not (cleanHands))) Preconditions: propositions that must be true to apply operator. A conjunction of propositions (no negated propositions). Effects: Propositions that operator changes, given preconditions. A conjunction of propositions (adds) and their negation (deletes). carry noGarb cleanH Parameterized schemas and objects are used to compactly encode actions. Add Edge Delete Edge

10 GraphPlan Planning Problem (:operator cook :precondition (cleanHands) :effect (dinner)) (:operator carry :precondition :effect (:and (noGarbage) (not (cleanHands))) Action Execution at time i: If all propositions of :precondition appear in the state at i, Then state at i+1 is created from state at i, by adding to i, all “add” propositions in :effects, removing from i, all “delete” propositions in :effects. carry noGarb cleanH cook dinner cleanHands

11 GraphPlan Planning Problem Noops Every proposition has a no-op action, which maintains it from time i to i+1. E.g., (:operator noop-P :precondition (P) :effect (P)) Noop-P P P

Example: Dinner Date Problem Initial Conditions: (and (cleanHands) (quiet)) Goal: (and (noGarbage) (dinner) (present)) Actions: (:operator carry :precondition :effect (and (noGarbage) (not (cleanHands))) (:operator dolly :precondition :effect (and (noGarbage) (not (quiet))) (:operator cook :precondition (cleanHands) :effect (dinner)) (:operator wrap :precondition (quiet) :effect (present)) + noops

dinner present cook wrap carry cleanH quiet noGarb cleanH dinner present Prop at 1 Action at 1 Prop at 2 Action at 2 Prop at 2 noop-dinner noop-present Sets of concurrent actions performed at each time [i] Uses model of concurrency as interleaving. If actions a and b occur at time i, then it must be valid to perform either a followed by b, OR b followed by a. A Plan in GraphPlan

A Complete Consistent Plan Given that the initial state holds at time 0, a plan is a solution iff: Complete: The preconditions of every operator at time i, is satisfied by a proposition at time i. The goal propositions all hold in the final state. Consistent: The operators at any time i can be executed in any order, without one of these operators undoing the: preconditions of another operator at time i. effects of another operator at time i.

dinner present cook wrap carry cleanH quiet noGarb cleanH dinner present Prop at 1 Action at 1 Prop at 2 Action at 2 Prop at 3 (noop dinner) (noop present) Example of a Complete Consistent Plan Initial Conditions: (and (cleanHands) (quiet)) Goal: (and (noGarbage) (dinner) (present))

GraphPlan Algorithm  Phase 1 – Plan Graph Expansion  Creates graph encoding pairwise consistency and reachability of actions and propositions from initial state.  Graph includes, as a subset, all plans that are complete and consistent.  Phase 2 - Solution Extraction  Graph treated as a kind of constraint satisfaction problem (CSP).  Selects whether or not to perform each action at each time point, by assigning CSP variables and testing consistency.

Outline  Introduction  The Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with failure

Graph Properties A Plan graph  compactly encodes the space of consistent plans,  while pruning... 1.partial states and actions at each time i that are not reachable from the initial state. 2.pairs of actions and propositions that are mutually inconsistent at time i. 3.plans that cannot reach the goals.

Constructing the planning graph…(Reachability)  Initial proposition layer  Contains propositions in initial state.

Example: Initial State, Layer 0 cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Constructing the planning graph…(Reachability)  Initial proposition layer  Contains propositions in initial state  Action layer i  If all action’s preconditions are consistent in i-1  Then add action to layer i  Proposition layer i+1  For each action at layer i  Add all its effects at layer i+1

Example: Add Actions and Effects noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Constructing the planning graph…(Reachability)  Initial proposition layer  Write down just the initial conditions  Action layer i  If all action’s preconditions appear consistent in i-1  Then add action to layer i  Proposition layer i+1  For each action at layer i  Add all its effects at layer i+1  Repeat until all goal propositions appear

Can a solution exist? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Do all goal propositions appear?

Constructing the planning graph…(Consistency)  Initial proposition layer  Write down just the initial conditions  Action layer i  If action’s preconditions appear consistent in i-1 (non-mutex)  Then add action to layer i  Proposition layer i+1  For each action at layer i  Add all its effects at layer i+1  Identify mutual exclusions  Actions in layer i  Propositions in layer i + 1  Repeat until all goal propositions appear non-mutex

26 Mutual Exclusion: Actions  Actions A,B are mutually exclusive at level i if no valid plan could possibly contain both at i:  They Interfere  A deletes B’s preconditions, or  A deletes B’s effects, or  Vice versa or  OR....

What causes the exclusion? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop

noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet What causes the exclusion? 1 Prop 1 Action 2 Prop 2 Action 3 Prop

noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet What causes the exclusion? 1 Prop 1 Action 2 Prop 2 Action 3 Prop

noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet What causes the exclusion? 1 Prop 1 Action 2 Prop 2 Action 3 Prop

31 Mutual Exclusion: Actions  Actions A,B are mutually exclusive at level i if no valid plan could possibly contain both at i:  They Interfere  A deletes B’s preconditions, or  A deletes B’s effects, or  Vice versa or  They have competing needs:  A & B have inconsistent preconditions

Layer 0: complete action mutexs noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop

33 Mutual Exclusion: Proposition Layer Propositions P,Q are inconsistent at i  if no valid plan could possibly contain both at i  If at i, all ways to achieve P exclude all ways to achieve Q P Q A1 A2 MNMN

Layer 1: add proposition mutexs noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop None!

Can a solution exist? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Do all goal propositions appear non-mutex?

Outline  Introduction  The Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with failure

37 Graphplan  Create plan graph level 1 from initial state  Loop  If goal  propositions of the highest level (nonmutex)  Then search graph for solution  If solution found, then return and terminate  Else extend graph one more level A kind of double search: forward direction checks necessary (but insufficient) conditions for a solution,... Backward search verifies...

38 Searching for a Solution Recursively find consistent actions achieving all goals at t, t-1,... :  For each goal proposition G at time t  For each action A making G true at t  If A isn’t mutex with previously chosen action at t,  Then select it  Finally,  If no action of G works,  Then backtrack on previous G.  Finally  If action found for each goal at time t,  Then recurse on preconditions of actions selected, t-1,  Else backtrack.

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Select action achieving Goal noGarb

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Select action achieving Goal dinner, Consistent with carry Backtrack on noGarb

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Select action dolly Backtrack on noGarb

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Select action achieving Goal dinner, Consistent with dolly

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Select action achieving Goal present, Consistent w dolly, cook

Searching for a solution noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 1 Prop 1 Action 2 Prop 2 Action 3 Prop Recurse on preconditions of dolly, cook & wrap All satisfied by initial state

Avoiding redundancy  No-ops are always favoured.  guarantees that the plan will not contain redundant actions.

Suppose the Plan Graph was Extended noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Can a valid plan use carry? noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Using carry at level 2 is valid noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Using carry at level 2 is valid noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 1 Prop 1 Action 2 Prop 2 Action 3 Prop

Outline  Introduction  The Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with failure

Plan Graph Properties  Propositions monotonically increase  once they are added to a layer they are never removed in successive layers;  Mutexes monotonically decrease  once a mutex has decayed it can never reappear;  Memoized sets (to be discussed) monotonically decrease  if a goal set is achievable at a layer Then it will be achievable at all successive layers.  The graph will eventually reach a fix point, that is, a level where facts and mutexes no longer change.

Fix point Example: Door Domain A B

Move from room A to room B  pre: robot is in A and the door is open  add: robot is in B  del: robot is in A Open door  pre: door is closed  add: door is open  del: door is closed Close door  pre: door is open  add: door is closed  del: door is open

noop Move Open noop Close noop In(B) In(A) Closed Opened Layer 4 N Move Open In(A) Closed Layer 1 Open noop In(A) Closed Opened Layer 2 In(B) noop Move Open noop Close In(A) Closed Opened Layer 3 Layer 4 is the fixed point (called level out) of the graph

Graph Search Properties  Graphplan may need to expand well beyond the fix point to find a solution.  Why?

Gripper Example Move from one room to another  pre: robot in the first room  add: robot in the second room  del: robot in the first room Pick a ball up in a gripper  pre: robot has a free gripper and is in same room as ball  add: robot holding the ball  del: gripper free, ball in room. Drop a ball in a room  pre: robot holding the ball, robot in destination room  add: ball in destination room, gripper free.  del: ball being held.

Gripper Example  In Gripper, the fix point occurs at layer 4,  all facts concerning the locations of the balls and the robot are pairwise non-mutex after 4 steps.  The distance to the solution layer depends on the number of balls to be moved.  For large numbers of balls Graphplan searches copies of the same layers repeatedly.  For example, for 30 balls  the solution layer is at layer 59,  54 layers contain identical facts, actions and mutexes.

Search Properties (cont)  Plans do not contain redundant steps.  By preferring No-ops  Graphplan guarantees parallel optimality.  The plan will take as short a time as possible.  Graphplan doesn’t guarantee sequential optimality  Might be possible to achieve all goals at some layer with fewer actions at that layer.  Graphplan returns failure if and only if no plan exists.

Outline  Introduction  The Planning Problem  Graph Construction  Solution Extraction  Properties  Termination with failure

Simple Termination  If the fix point is reached and either:  One of the goals is not asserted or  Two of the goals are mutex Then Graphplan can return "No solution" without any search at all.  Else there may be higher order exclusions which Graphplan cannot detect  Requires more sophisticated additional test for termination.

Why Continue After the Fix Point?  facts, actions and mutexes no longer change after the fix point,  N-ary exclusions DO change.  New layers add time to the graph.  Time allows actions to be spaced so that N-ary mutexes have time to decay.  While new things can happen between two successive layers progress is being made.  Track n-ary mutexes.  Terminate on their fix point.

Memoization and Termination  A goal set at a layer may be unsolvable  Example, in Gripper.  If a goal set at layer k cannot be achieved, Then that set is memoized at layer k.  To prevent wasted search effort  Checks each new goal set at k against memos of k for infeasibility.  If an additional layer is built and searching it creates no new memos, then the problem is unsolvable.

Recap: Graph Plan  Graphplan was developed in 1995 by Avrim Blum and Merrick Furst, at CMU.  Graphplan searches a compact encoding of the state space from the operators and initial state, which prunes many invalid plans, violating reachability and mutual exclusion.  Recent implementations by other researchers have extended the capability of Graphplan to reason with temporally extended actions, metrics and non-atomic preconditions and effects.

Appendix

Termination Test  If the Graph leveled off at layer n, and a search stage, t >n, has passed in which the memoized sets at n+1 are the same as at layer n, Then Graphplan can output "No Solution". Layer n Layer n + 1 Layer t S t n = S t n+1 Where S m k = the sets of goals found unsolvable at layer k during search from m

Termination Property  Claim: Graphplan returns with failure iff the problem is unsolvable.  Proof: If the problem is unsolvable Graphplan returns with failure. The number of sets found unsolvable at layer n from layer t will never be smaller than the number at n from layer t+1, and there is a finite maximum number of goal sets. This means that, if the problem is unsolvable, eventually two successive layers will contain the same memoized sets.

If Graphplan outputs "No Solution," then the problem is unsolvable.  Suppose the fix point is at layer n and Graphplan has completed an unsuccessful search from layer t > n.  A plan to achieve any set unsolvable at layer n+1 must, one step earlier, achieve some set unsolvable at layer n.  Suppose Graphplan returns "No Solution" but the problem is solvable:  If S t n = S t n+1 then S' and S'' must both be in S t n+1. This means that some set in S t n+1 will need to be achieved in n+1. this situation is contradictory. Layer n Layer N + 1 Layer q S’ in S t n S’’ in S t n+1 Layer q + 1 Solution Layer q S’’ S’

Presentation Issues  Look for and eliminate repetition about the function of the graph.  Fix door as an example of fixed point.  Fix gripper as an example of expanding beyond the fixed point.  Give example where not preferring noops produces redundancy.  Give example that is parallel optimal but not sequentially optimal.