1. Top 5 Worst Times For A Conference Talk 1.Last Day 2.Last Session of Last Day 3.Last Talk of Last Session of Last Day 4.Last Talk of Last Session of Last Day, after Best Paper Award 5.Last Talk of Last Session of Last Day, after Best Paper Award on Same Topic

2. Heuristic Guidance Measures For Conformant Planning Daniel Bryce & Subbarao Kambhampati Dept of Computer Science & Engineering Arizona State University ICAPS-04

3. Talk Outline Contributions Search Heuristic Computation –Single, Unioned Graph –Multiple Graphs –Single, Labeled Graph System Architecture Empirical Results Applications to Contingent Planning!!! Conclusion & Future Work –Applications to Stochastic Planning!!!

4. Contributions What should belief space search distance estimates measure? –Previous approaches to heuristics do not reflect true nature of distances in belief space planning Cardinality: MBP planners State to State plans: GPT planner State to State plan overlap How do we compute these measures efficiently? – (Concentration of Talk)

5. Search Belief States represented as formulas –Belief State contains all states consistent with the formula –Use Conjunctive Normal Form Actions have (Un)Conditional Effects and Enabling Preconditions –All conditions and effects are formulas Disjunctive Preconditions and Non-deterministic Effects A* Regression Search in Belief Space –Terminates when Initial Belief State Entails the Search Belief State

6. Planning Graph Heuristic Computation Heuristics –BFS –Cardinality –Max, Sum, Level, Relaxed Plans Planning Graph Structures –Single, unioned planning graph (SG) –Multiple, independent planning graphs (MG) –Single, labeled planning graph (LUG) [Bryce, et. al, 2004] – AAAI MDP workshop

7. Using a Single, Unioned Graph P M Q M R M P Q R M A1 A2 A3 Q R M K L A4 G A5 P A1 A2 A3 Q R M K L P G A4 K A1 P M Heuristic Estimate = 2 Not effective Lose world specific support information Union literals from all initial states into a conjunctive initial graph level Minimal implementation

8. Using Multiple Graphs P M A1 P M K P M K A4 G R M A3 R M L R M L G A5 P M Q M R M Q M A2 Q M K Q K A4 G M G K A1 M P G A4 K A2 Q M G A5 L A3 R M Same-world Mutexes Memory Intensive Heuristic Computation Can be costly Unioning these graphs a priori would give much savings …

9. Using a Single, Labeled Graph (joint work with David E. Smith) P Q R A1 A2 A3 P Q R M L A1 A2 A3 P Q R L A5 Action Labels: Conjunction of Labels of Supporting Literals Literal Labels: Disjunction of Labels Of Supporting Actions P M Q M R M K A4 G K A1 A2 A3 P Q R M G A5 A4 L K A1 A2 A3 P Q R M Heuristic Value = 5 Memory Efficient Cheap Heuristics Scalable Extensible Benefits from BDD’s ~Q & ~R ~P & ~R ~P & ~Q (~P & ~R) V (~Q & ~R) (~P & ~R) V (~Q & ~R) V (~P & ~Q) M True Label Key Labels signify possible worlds under which a literal holds

10. System Architecture A* Search Engine (HSP-r) Heuristics Planning Graph(s) (IPP) Belief States Labels (CUDD) Model Checker (NuSMV) Off – The - ShelfCustom IPC PDDL Parser Searches Guided By Input for Condense Validates Extracted From

11. Sum and Relaxed Plan Are Best for a single Graph Relaxed Plan is Best Multiple Or Label Graphs Label Graph using mutexes With relaxed plan is best overall

12. Relaxed Plan is Best for a single Graph Sum is Best for Multiple Graphs Label Graph using mutexes With relaxed plan is best overall

13. Cardinality does well Multiple Graph Union Relaxed Plan scales Label Graph Relaxed Plan Does best

14. OptimalApproaches scale poorly Cardinality approaches are faster But quality suffers Relaxed Plan approaches Scale better with time approximate to cardinality And quality comparable to optimal

15. OptimalApproaches scale poorly Cardinality approaches are faster But quality suffers Relaxed Plan approaches Scale better with time approximate to cardinality And quality comparable to optimal

16. Contingent Planning Progression Planner – PBSP –LAO* type search -- Non-Deterministic & Partially Observable –Build Planning Graph to compute heuristic for each Belief State No Mutexes Computed Added Observational Actions to Domains

17. OptimalApproaches scale poorly Cardinality approaches are faster And scale better But quality suffers by two orders of magnitude Relaxed Plan approaches Scale better than optimal approaches and have Comparable quality

18. Conclusions & Future Work Conclusion –Distance Estimations using “overlap” are more informed than cardinality and max state to state heuristics –Multiple Planning Graphs give good heuristics, but are costly Labeled Planning graphs reduce cost –Planning Graph Heuristics help control plan length while scaling to difficult problems More details in: –TR at: http://rakaposhi.eas.asu.edu/belief-searchhttp://rakaposhi.eas.asu.edu/belief-search Conformant, Contingent – all planning graph types –AAAI-04 MDP workshop Labeled Planning Graph for conformant planning Future Work –Stochastic Planning

19. Stochastic Planning Stochastic Planning Problem Non-Deterministic Planner (PBSP or CAltAlt) Deterministic Planner (UCPOP) New Approach Buridan Seed Stochastic Plan Relaxation Of Instance Convert Solution to Stochastic Plan Deterministic Plan Non- Deterministic Plan Stochastic Plan Local Search To Improve Probability of Satisfaction A seed non- deterministic plan is likely to reflect physics of a stochastic planning problem better than a seed deterministic plan. Can use Relaxed Plans that are greedy On Probability by Using Probability in Planning Graph (similar to PGraphPlan)

20. Regression Search Example Actions: A1: M P => K A2: M Q => K A3: M R => L A4: K => G A5: L => G Initially: (P V Q V R) & (~P V ~Q) & (~P V ~R) & (~Q V ~R) & M Goal State: G G (G V K) (G V K V L) A4 A1 (G V K V L V P) & M A2 A5 A3 G or K must be true before A4 For G to be true after A4 (G V K V L V P V Q) & M (G V K V L V P V Q V R) & M Each Clause is Satisfied by a Clause in the Initial Clausal State -- Done! (5 actions) Initially: (P V Q V R) & (~P V ~Q) & (~P V ~R) & (~Q V ~R) & M Clausal States compactly represent disjunction to sets of uncertain literals – Yet, still need heuristics for the search (G V K V L V P V Q V R) & M Enabling precondition Must be true before A1 was applied

21. Distance Estimates Cardinality Max State to StateState to State Overlap – Belief state to Belief state 4 2 3 5 3 6 4 4 min 7  7 max 7 union 10

22. Cardinality does well Multiple Graph Union Relaxed Plan scales Label Graph Relaxed Plan Does best, mutexes do help

23. OptimalApproaches scale poorly Cardinality approaches are faster And scale better But quality suffers by an order of magnitude Relaxed Plan approaches Scale better than optimal approaches, but have quality comparable to optimal