Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics.

Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics Done

Add…  The issue of time—dense vs. integer  Rintanen’s complexity issue—R.C. with the same action..  Non-RC plans can be compiled 1-1  A huge modeling jump

A [d] s e o s e Essence of Temporal Planning

No Temporal Gap  Classical + Scheduling  Forbidding temporal gap implies  All effects at one time  Before-conditions meet effects  After-conditions meet effects  Unique transition per action A [d] * pre eff Essence of Temporal Planning A * B * C * D *

Approaches for MTP  In theory, pretty much every one of the approaches we saw for classical planning can be (and have been) extended to MTP (with varying degrees of scalability)  There are some interesting tradeoffs  PO planners are easiest to extend to support the concurrency needed for durative actions  Have harder time handling resources (because resource consumption depends on exactly what actions occurred before this time point)  Progression planners easiest to extend to support resource consuming actions  But harder time handling concurrency (need to consider “advancing clock” as a separate option in addition to applying one of the actions)

Temporal Planning via Plan Space Planning  Instead of constraints w.r.t steps (s<s’ or P@s), we will have constraints on time points (t – t’ = 4 etc.).

at_fuse_box@G} Cross_cellar GI At_fusebox Have_light@ t1 t2 t2-t1 =10 t1 < tG tI < t1 Have_light@t1

at_fuse_box@G} Cross_cellar GI At_fusebox Have_light@ t1 t2 t2-t1 =10 t1 < tG tI < t1 T4<tG T4-t3=15 T3<t1 T4<t3 V t1<t4 Have_light@t1 Burn_match t3t4 ~have-light The ~have_light effect at t4 can violate the causal link! Resolve by Adding T4<t3 V t1<t4

at_fuse_box@G} Cross_cellar GI At_fusebox Have_light@ t1 t2 t2-t1 =10 t1 < tG tI < t1 t4<tG t4-t3=15 t3<t1 t4<t3 V t1<t4 t3<t2 t4<t3 V t2<t4 Have_light@t1 Burn_match t3t4 ~have-light To work on have_light@, we can either --support the whole interval directly by adding a causal link > --or first split to two subintervals and work on supporting have-light on both intervals Notice that zeno allows arbitrary slack between the two actions

PO (Partial Order) Search [Zeno; 1994] Split the Interval into Multiple overlapping intervals Involves Posting temporal Constraints, and Durative goals Involves LPsolving over Linear constraints (temporal constraints Are linear too); Waits for nonlinear constraints To become linear.

More on Temporal planning by plan-space planners (Zeno)  The “accommodation” to complexity that Zeno makes by refusing to handle nonlinear constraints (waiting instead until they become linear) is sort of hilarious given it doesn’t care much about heuristic control otherwise  Basically Zeno is trying to keep the “per-node” cost of the search down (and if you do nonlinear constraint consistency check, even that is quite hard)  Of course, we know now that there is no obvious reason to believe that reducing the per-node cost will, ipso facto, also lead to reduction in overall search.  The idea of “goal reduction” by splitting a temporal subgoal to multiple subintervals is used only in Zeno, and helps it support a temporal goal over a long duration with multiple actions. Neat idea.  Zeno doesn’t have much of a problem handling arbitrary concurrency—since we are only posting constraints on temporal variables denoting the start points of the various actions. In particular, Zeno does not force either right or left alignment of actions.  In addition to Zeno, IxTeT is another influential metric temporal planner that uses plan-space planning idea.

Temporal Planning with Progression?  All the progression planners we looked at were able to produce sequential plans.  Can progression be used to produce concurrent plans?  Sub Question: Can progression be used to produce parallel plans?

Producing parallel plans with progression?  The naïve idea is to project over subsets of non-interfering actions (rather than single actions).  Problem: Exponential branching factor  A better idea: Consider “fattening” as well as “lengthening” the current partial plan as two options.  We start by representing the state of a partial plan prefix as [S, {A1…Ak}] where S is the current state, and {A1..Ak} are the mutually non-interfering actions that we have already committed to applying at S.  Notice that this is just a generalization of the normal progression state, in which the action set {A1..Ak} will be a singleton  Given a state [S,{A1..Ak}] to expand, we have (backtrackable) choices:  Fatten: Consider applying another action B in state S [One branch for each possible action B]  For this to be feasible, B should be applicable in Si and B should not be interfering with A1..Ak. The resulting state will be {S; {A1…Ak}+B}  Advance: Consider advancing the state. If S’ is the state resulting from application of {A1..Ak} to S, then generate the state {S’; {}}  Notice that  Fattening is only done at the current state (once advancing is done, the current state changes. So any new fattening will be done at the new state.  Normal progression does an automatic advance after each fatten (which means you will have only one action at each step)

Generating concurrent plans is similar to generating parallel plans… almost..  We will continue to consider fattening at the current state, and advancing to the next state  Several issues:  Actions have durations. So the state information should include the actions we committed to (but haven’t completed)  How much do we advance?  Worst case: Advance time to the “next time point” (could be bad news if we have dense time!)  Clever idea: Advance time to the “next happening” –where the state changes

State-Space Search: Search is through time-stamped states Search states should have information about -- what conditions hold at the current time slice (P,M below) -- what actions have we already committed to put into the plan ( ,Q below) S=(P,M, ,Q,t) Set of predicates pi and the time of their last achievement t i < t. Set of functions represent resource values. Set of protected persistent conditions (could be binary or resource conds). Event queue (contains resource as well as binary fluent events). Time stamp of S. In the initial state, P,M, non-empty Q non-empty if we have exogenous events

Search Algorithm (cont.)  Goal Satisfaction : S=(P,M, ,Q,t)  G if   G either:    P, t j < t i and no event in Q deletes p i.   e  Q that adds p i at time t e < t i.  Action Application : Action A is applicable in S if:  All instantaneous preconditions of A are satisfied by P and M.  A’s effects do not interfere with  and Q.  No event in Q interferes with persistent preconditions of A.  A does not lead to concurrent resource change  When A is applied to S:  P is updated according to A’s instantaneous effects.  Persistent preconditions of A are put in   Delayed effects of A are put in Q. S=(P,M, ,Q,t) [TLplan; Sapa; 2001—talk given 9/12/01]

Decision Epochs: Limiting the places where clock can be advanced  To support concurrency, we need to consider advancing the clock  How far to advance the clock?  One popular strategy is to advance the clock to the time of the next earliest event in the event queue; since this is the least advance needed to make changes to P and M of S.  At this point, all the events happening at that time point are transferred from Q to P and M (to signify that they have happened) Light-match Cross-cellar ~have-light 15 10 In the cellar plan above, the clock, If advanced, will be advanced to 15, Where an event (~have-light will occur) This means cross-cellar can either be done At 0 or 15 (and the latter makes no sense) Cross-cellar

Interference  Clearly an overkill

Let current state S be P:{have_light@0; at_steps@0}; Q:{~have_light@15} t: 0 (presumably after doing the light-candle action) Applying cross_cellar to this state gives S’= P:{have_light@0; crossing@0};  :{have_light, } Q:{at_fuse-box@10;~have_light@15} t: 0 Light-match Cross-cellar 15 10 Time-stamp

Short matches  No epoch available  “middle of nowhere”  Decision Epoch Planning is incomplete! !!! Wow!

Decision Epoch Planning: DEP  Only start actions after events  Choose  Start an action  Advance epoch  Temporally Simple  Complete, suboptimal  Temporally Expressive  Incomplete, suboptimal Salvaging DEP A [3] B [2] light-match [8] fix-fuse [4]

Generalized DEP: DEP+  Also end actions after events  Choose  Start an action  End an action  Advance epoch  Temporally Simple  Complete, optimal  Temporally Expressive  Incomplete, suboptimal Salvaging DEP A [3] B [2]

Wow!  Temporally Simple  Classical + Scheduling  Winners incomplete for all Temporally Expressive Languages  Most/all benchmarks are classical! !!!

State of the Art: Incomplete or Slow  Metric-FF, MIPS, SGPlan, SAPA, TP4, TPG, HSP*,...  Guarantees only for temporally simple languages  Can solve some concurrent problems  Light-match, but not short-match  Difficult to detect  ZENO, IxTeT, VHPOP, LPGP,...  Complete  Slow !!!

Interleaving-Space: TEMPO  Delay dispatch decisions until afterwards  Choose  Start an action  End an action  Make a scheduling decision  Solve temporal constraints  Temporally Simple  Complete, Optimal  Temporally Expressive  Complete, Optimal Salvaging State-space Temporal Planning light fix match fuse fix light fuse fix light match fuse fix light

Slides beyond this not explicitly discussed in the class

Regression Search is similar…  In the case of regression over durative actions too, the main generalization we need is differentiating the “advancement of clock” and “application of a relevant action”  Can use same state representation S=(P,M, ,Q,t) with the semantics that  P and M are binary and resource subgoals needed at current time point  Q are the subgoals needed at earlier time points   are subgoals to be protected over specific intervals  We can either add an action to support something in P or Q, or push the clock backward before considering subgoals  If we push the clock backward, we push it to the time of the latest subgoal in Q  TP4 uses a slightly different representation (with State and Action information) [TP4; 1999] A2:X A3:W A1:Y Q RWXyRWXy We can either work On R at t inf or R and Q At t inf -D(A 3 ) To work on have_light@, we can either --support the whole interval directly with one action --or first split to two subintervals and work on supporting have-light on both intervals

Let current state S be P:{at_fuse_box@0} t: 0 Regressing cross_cellar over this state gives S’= P:{};  :{have_light, } Q:{have_light@ -10;at_stairs@-10} t: 0 Cross_cellar Have_light Notice that in contrast to progression, Regression will align the end points of Concurrent actions…(e.g. when we put in Light-match to support have-light)

S’= P:{};  :{have_light, } Q:{have_light@-10;at_stairs@-10} t: 0 If we now decide to support the subgoal in Q Using light-match S’’=P:{} Q:{have-match@-15;at_stairs@-10}  :{have_light, } t: 0 Cross_cellar Have_light Notice that in contrast to progression, Regression will align the end points of Concurrent actions…(e.g. when we put in Light-match to support have-light) Cross_cellar Have_light Light-match

Tradeoffs: Progression/Regression/PO Planning for metric/temporal planning  Compared to PO, both progression and regression do a less than complete job of handling concurrency (e.g. slacks may have to be handled through post-processing).  Progression planners have the advantage that the exact amount of a resource is known at any given state. So, complex resource constraints are easier to verify. PO (and to some extent regression), will have to verify this by posting and then verifying resource constraints.  Currently, SAPA (a progression planner) does better than TP4 (a regression planner). Both do oodles better than Zeno/IxTET. However  TP4 could be possibly improved significantly by giving up the insistence on admissible heuristics  Zeno (and IxTET) could benefit by adapting ideas from RePOP.

When is Temporal Planning Really Temporal? William Cushing Subbarao Kambhampati Special thanks to: J. Benton, Menkes van den Briel Mausam Daniel Weld

Temporal Planning  Plan-space  Extended planning graph  Reduction to ILP  State-space  Competition winners  Reachability heuristics  Infinite number of time points  Decision Epochs  Restrict start times to events Introduction name [duration] start-pre end-pre over-pre start-eff end-eff light-match [8] fix-fuse [4] M - match L - light F - fuse

Troubling Questions  What do/should the IPCs measure? Essence of Temporal Planning  Required Concurrency  Temporally Simple  Temporally Expressive  Can Decision Epoch Planning be fixed?  No.  But!  DEP+  “Less” incomplete  TEMPO  Reachability heuristics Overview ≈ Classical ≈ Harder

Required Concurrency  Temporally Simple Languages  Concurrency never necessary  …but can be exploited for quality  Temporally Expressive Languages  Can specify problems such that concurrency is needed Essence of Temporal Planning

Temporal Action Languages name [duration] Start-pre End-pre Over-pre Start-eff End-eff Essence of Temporal Planning name [duration] Over-pre End-eff

Temporal Action Languages  Temporally Simple  Rescheduling is possible  MIPS, SGPlan, LPG, …  Sequential planning is complete – “optimal” ?  TGP, yes  In general, yes  Temporally Expressive   Temporal Gap A [d] s e o s e Essence of Temporal Planning

(Minimal) Temporally Expressive Languages  Temporal Gap  Before-condition and effect  After-condition and effect  Two effects  Temporally Simple  No Temporal Gap Essence of Temporal Planning

No Temporal Gap  Classical + Scheduling  Forbidding temporal gap implies  All effects at one time  Before-conditions meet effects  After-conditions meet effects  Unique transition per action  Theorem: Every concurrent plan is an O(n) rescheduling of a sequential plan  And vice versa A [d] * pre eff Essence of Temporal Planning A * B * C * D *

Conclusions  Required concurrency is the essence of temporal planning  Otherwise classical planner + O(n) scheduling suffices  Simple test for required concurrency: Temporal gap  Decision epoch planning is fundamentally incomplete  But DEP+ may solve most real-world problems  Complete state-space temporal planning: TEMPO  Allows leveraging of state-based reachability heuristics  !!!!!

Evaluating Temporal Planning Domains William Cushing Subbarao Kambhampati Kartik Talamadupula Daniel Weld Mausam

Competition winners are incomplete fix-fuse light-match L F M L -M ^ -L  How incomplete?  What should the IPC measure?  Epoch  A time at which an event happens  Decision Epoch Planning  Only start actions after epochs Temporal Planning Required Concurrency

How deep is the problem?  Required Concurrency  Languages  Incomplete for temporally expressive languages  Complete for temporally simple languages

(Minimal) Temporally Expressive Languages  Temporal Gap  Before-condition and effect  After-condition and effect  Two effects  Temporally Simple  No Temporal Gap

Required Concurrency  Inherently sequential is easy  Timestamps (with support for arithmetic)  Loose integration with a PERT scheduler  TGP, LPG-td, SGPlan, MIPS, …  Required concurrency is hard  The plan space is larger  The scheduling sub-problem is harder  Sub-problem optimality principle  State of the art is VHPOP, LPGP, CRIKEY  TEMPO, reduction to CSP

The International Planning Competition  Benchmarks must not require (much) concurrency  How much?  None at all  How do we show it?  Use temporal gap?  Problem: “every” action has temporal gap

Solution: Decompile temporal gap  (navigate ?rover ?alpha ?omega)  Pre: (at start (at ?rover ?alpha))  Eff: (and  (at start (not (at ?rover ?alpha)))  (at end (at ?rover ?omega)))  (navigate ?rover ?alpha ?omega)  (over all (=> (at ?rover) ?alpha ?omega))

Causal Structure and Concurrency A * B D * C * light-match fix-fuse light-match fix-fuse A B C D * Inherently Sequential Inherently Concurrent

Navigate’s sequential structure navigate communicate navigate ??

Technique: Start-time Sequentialization light-match fix-fuse A B A B  Do not want to enumerate plans!  Nor every sequentialization!  Start-time sequentialization  Fixed attempt  Suffices for benchmarks (not necessary)  End-time sequentialization  Critical-time sequentialization  Start times of containing actions in same order as all dependencies

Element Safety  Y < X  S(A(Y)) > S(A(X))  Threat-free  X supports Z,  Y threatens Z  Interaction-free  Z supports Y  X threatens Y  Link-free  Y supports X A B A B A B

Benchmarks never require concurrency  Durative change on m.v. fluents is safe  Unbounded resources are safe  “The Perils of Discrete Resource Models”  ICAPS workshop on IPC  A few special cases  (at end (calibrated ?c ?r))  Document…  http://rakaposhi.eas.asu.edu/is-benchmarks.html http://rakaposhi.eas.asu.edu/is-benchmarks.html  Forthcoming (:durative-action navigate :parameters (?x - rover ?y - waypoint ?z - waypoint) :duration (= ?duration 5) :condition (and ;;(at start (at ?x ?y)) ;; MV Fluent ;;(at start (>= (energy ?x) 8)) ;; Resource Consumption (over all (can_traverse ?x ?y ?z)) (at start (available ?x)) (over all (visible ?y ?z)) ) :effect (and ;;(at start (decrease (energy ?x) 8)) ;; Resource Consumption (over all (consume (energy ?x) 8)) ;; Resource Consumption ;;(at start (not (at ?x ?y))) ;; MV Fluent ;;(at end (at ?x ?z)))) ;; MV Fluent (over all (-> (at ?x) ?y ?z)) ;; MV Fluent )) ;;(at ?x - rover ?y - waypoint) (at ?x - rover ) - waypoint

Only RC due to Modeling Bugs  1: drop  1.1: drop  2.05: sample  …  (and (full ?s) (empty ?s))  1: recharge  1.1: recharge  1.2: recharge  …  (>= (energy ?x) (* k (capacity ?x)))

Syntactic Sugar for avoiding Errors  Action drop (store)  full(store) == true at start  full(store) := false at end  Should be at start  empty(store) := true at end  Explicit resources  amount(store) :consume 1  space(store) :produce 1  Explicit durative change + m.v. fluents  amount(store) == full => empty

Temporal Machine Shop  Benchmarks lack required concurrency  Real world lacks required concurrency?  (:durative-action fire-kiln :parameters (?k - kiln) :duration (= ?duration 20) :effect (and (over all (lend (firing ?k))) (over all (–> (ready ?k) true false))  (:durative-action bake-ceramic :parameters (?p - piece ?k - kiln) :duration (= ?duration (bake-time ?p)) :condition (and (over all (firing ?k)) (over all (shaped ?p))) :effect (over all (–> (baked ?p) false true)))

Real world required concurrency  (and (lifted bowl-left) (lifted bowl-right))  Spray-oil (during milling)  Heat-beaker (while adding chemicals)  Ventilate-room (while drying glue)  …

Lessons for the Competition  Competitors tune for the benchmarks  Most of the competitors simplify to TGP  Either required concurrency is important  Benchmarks should test it  Or it isn’t  Language should be inherently sequential  PDDL spec. highlights light-match  RC occurs in the real world  Might need processes, continuous effects

Conclusion  Required Concurrency separates easy and hard temporal planning  The easy case allows offloading to a scheduler  Still an intriguing problem  Simplify the language – push the classical track  The hard case forces temporal reasoning by the planner  Real world required concurrency is frequent  PDDL 2.1.3 was designed for required concurrency  But the benchmarks fell through  Analysis of domains is hard  Automatable? Embeddable within a search?  Domain modeling is very hard  Durative change  Resources Evaluating Temporal Planning Domains ICAPS 2007 When is Temporal Planning Really Temporal? IJCAI 2007 The Perils of Discrete Resource Models ICAPS 2007, IPC workshop

Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics.

Similar presentations

Presentation on theme: "Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics.

Similar presentations

Presentation on theme: "Temporal Planning  action models  Using PDDL2.1 standard  how to model the search  Progression; Regression; PO planning  how to extract good heuristics."— Presentation transcript:

Similar presentations

About project

Feedback