1. Planning as Satisfiability CS672

2. 2 Outline 0. Overview of Planning 1. Modeling and Solving Planning Problems as SAT - SATPLAN 2. Improved Encodings using Graph Analysis - BLACKBOX 3. Improved Encodings using Compiled Control Knowledge

3. 3 Overview of Planning Find a sequence of operators that transform an initial state to a goal state State = complete truth assignment to a set of variables (fluents) Goal = partial truth assignment (set of states) Action = a partial function State  State specified by three sets of variables: precondition, add list, delete list

4. 4 Abdundance of Negative Complexity Results I. Domain-independent planning: PSPACE- complete (Chapman 1987; Bylander 1991; Backstrom 1993) II. Domain-dependent planning: NP-complete (Chenoweth 1991; Gupta and Nau 1992) III. Approximate planning: NP-complete (Selman 1994)

5. 5 Planning as Inference Planning as first-order theorem proving (Green 1969) computationally infeasible STRIPS (Fikes & Nilsson 1971) very hard Partial-order planning (modal truth criteria) (Tate 1977, Chapman 1985, McAllester 1991, Smith & Peot 1993) can be more efficient, but still hard (Minton, Bresina, & Drummond 1994) SATPLAN: planning as propositional reasoning

6. 6 Part 1: Modeling and Solving Planning Problems as SAT

7. 7 SAT Encodings Planning Problem -> Propositional CNF by axiom schemas Discrete time, modeled by integers state predicates: indexed by time at which they hold action predicates: indexed by time at which action begins each action takes 1 time step many actions may occur at the same step

8. 8 Encoding Conventions Actions imply preconditions and effects fly(x,y,i)  at(x,i) & route(x,y) & at(y,i+1) Conflicting actions cannot occur at same time (A deletes a precondition of B) fly(x,y,i) & y  z   fly(x,z,i) If something changes, an action must have caused it (Explanatory Frame Axioms) at(x,i) &  at(x,i+1)   y. route(x,y) & fly(x,y,i) Initial and final states hold at(NY,0) &... & at(LA,9) &...

9. 9 Modeling Tricks Can often dramatically reduce size of problem by modeling techniques move(x,y,z,i) requires n 4 vars pickup(x,y,i), putdown(x,z,i) requires 2n 3 vars State-based encodings: eliminate all action variables (“compile away”) at(x,i)  at(x,i+1)   y. route(x,y) & at(y,i+1) at(x,i) & x  y   at(y,i)

10. 10 Solution to a Planning Problem A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators Easy to convert back to a STRIPS-style plan

11. 11 SATPLAN axiom schemas instantiated propositional clauses satisfying model plan mapping length problem description SAT engine(s) instantiate interpret

12. 12 SAT Algorithms Systematic Search DP (Davis Putnam Logemann Loveland) backtrack search + unit propagation satz (Chu Min Li) - variable selection by forward checking: max unit props relsat (Bayardo) - dependency directed backtracking: add new clauses at dead-ends Local Search Inspired by Mins-Conflict algorithm (Adorf, Johnson, Minton, Philips, & Laird) GSAT (Selman), Walksat (Selman, Kautz & Cohen) greedy local search + noise to escape minima

13. 13 Planning Benchmark Test Set Extension of Graphplan test set blocks world - up to 18 blocks, 10 19 states logistics - complex, highly-parallel transportation domain. Logistics.d: 2,165 possible actions per time slot 10 16 legal configurations (2 2000 states) optimal solution contains 74 distinct actions over 14 time slots Problems of this size never previously handled by state-space planning systems

14. 14 Scaling Up Logistics Planning 0.01 0.1 1 10 100 1000 10000 rocket.arocket.b log.blog.alog.c log.d log solution time Graphplan DP DP/Satz Walksat

15. 15 Randomized Restarts Solution: randomize the systematic solver Add noise to the heuristic branching (variable choice) function Cutoff and restart search after a fixed number of backtracks In practice: rapid restarts with low cutoff can dramatically improve performance (Gomes 1996, Gomes, Kautz, and Selman 1997, 1998)

16. 16 Increased Predictability 0.01 0.1 1 10 100 1000 10000 rocket.arocket.b log.blog.alog.c log.d log solution time Satz Satz/Rand

17. 17 What SATPLAN Shows General propositional theorem provers can compete with state of the art specialized planning systems New, highly tuned variations of DP surprising powerful –result of sharing ideas and code in large SAT/CSP research community –specialized engines can catch up, but by then new general techniques Radically new stochastic approaches to SAT can provide very low exponential scaling –2+ orders magnitude speedup on hard benchmark problems

18. 18 Why SATPLAN Works More flexible than forward or backward chaining Systematic: most unit propagation at most highly constrained states Stochastic: iterative repair Randomized algorithms less likely to get trapped along bad paths

19. 19 Part 2: Improved Encodings by Graph Analysis: The BLACKBOX Planner

20. 20 Graphplan Planning as graph search (Blum & Furst 1995) Set new paradigm for planning Like SATPLAN... Two phases: instantiation of propositional structure, followed by search Unlike SATPLAN... Interleaves instantiation and pruning of plan graph Employs specialized search engine Graphplan - better instantiation SATPLAN - better search

21. 21 Graph Pruning Graphplan instantiates in a forward direction, pruning unreachable nodes conflicting actions are mutex if all actions that add two facts are mutex, the facts are mutex if the preconditions for an action are mutex, the action is unreachable! In logical terms: limited application of negative binary propagation given:  P V  Q, P V R V S V... infer:  Q V R V S V...

22. 22 The Plan Graph Facts Actions... Facts Actions... preconditionsadd effects mutually exclusive delete effects

23. 23 Translation of Plan Graph Fact  Act1  Act2 Act1  Pre1  Pre2 ¬Act1  ¬Act2 Act1 Act2 Fact Pre1 Pre2

24. 24 General Limited Inference Generated wff can be further simplified by consistency propagation techniques Compact (Crawford & Auton 1996) unit propagation: is Wff inconsistant by resolution against unit clauses? O(n) failed literal rule: is Wff + { P } inconsistant by unit propagation? O(n 2 ) binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation? O(n 3 ) Complements domain specific limited inference Discovers hidden local structure!

25. 25 General Limited Inference

26. 26 Blackbox STRIPS Plan Graph Mutex computation CNF General Stochastic / Systematic SAT engines Solution Simplifier Translator CNF

27. 27 Blackbox Results

28. 28 Applicability When is the BlackBox approach not a good idea? when domain too large for propositional planning approaches when long sequential plans are needed when solution time dominated by reachability analysis (plan-graph generation), not extraction when optimal or near optimal planning not necessary

29. 29 Part 3: Improved Encodings: Compiling Control Knowledge

30. 30 Kinds of Control Knowledge About domain itself a truck is only in one location About good plans do not remove a package from its destination location About how to search plan air routes before land routes

31. 31 Expressing Knowledge Such information is traditionally incorporated in the planning algorithm itself –or in a special programming language Instead: use additional declarative axioms –(Bacchus 1995; Kautz 1998; Chen, Kautz, & Selman 1999) Problem instance: operator axioms + initial and goal axioms + control axioms Control knowledge  constraints on search and solution spaces Independent of any search engine strategy

32. 32 Axiomatic Control Knowledge State Invariant: A truck is at only one location at(truck,loc1,i) & loc1  loc2   at(truck,loc2,i) Optimality: Do not return a package to a location at(pkg,loc,i) &  at(pkg,loc,i+1) & i<j   at(pkg,loc,j) Simplifying Assumption: Once a truck is loaded, it should immediately move  in(pkg,truck,i) & in(pkg,truck,i+1) & at(truck,loc,i+1)   at(truck,loc,i+2)

33. 33 Adding Control Kx to SATPLAN Problem Specification Axioms Control Knowledge Axioms Instantiated Clauses SAT Simplifier SAT Engine SAT “Core” As control knowledge increases, Core shrinks!

34. 34 Tradeoffs of Control Knowledge If the planning domain is inherently intractable, how can any amount of control knowledge make planning tractable? by reducing solution quality optimal planning - NP-Hard non-optimal - (maybe) Polynomial Issue: speed / quality tradeoff Case study: Control Knowledge in TLPLAN and BlackBox TLPLAN (Bacchus 1996): simple forward- chaining search with strong control rules

35. 35 TLPlan Temporal Logic Control Formula

36. 36 I. Rules involves only static information II. Rules depends on the current state III. Rules depends on the current state and requires dynamic user-defined predicates Temporal Logic for Control ( at(obj1, loc1) => at(obj1, loc1) )

37. 37 a Category I Control Rules a Do NOT unload an object from an airplane unless the object is at its goal destination GoalInitial a SFOORLNYC

38. 38 Pruning the Planning Graph Category I Rules Facts Actions... Facts Actions...

39. 39 Effect of Graph Pruning

40. 40 Category II Control Rules a ORLNYC Do NOT move an airplane if there is an object in the airplane that needs to be unloaded at that location. SFO

41. 41 Control by Adding Constraints Control Rules Planning FormulaConstraints Clauses

42. 42 Blackbox with Control Knowledge (Logistics domain)

43. 43 Comparison Blackbox and TLPlan ( Plan Length) Comparison between Blackbox and TLPlan (Parallel Plan Length)

44. 44 Comparison between Blackbox and TLPlan (Running Time)

45. 45 Comparison TLPlan (without Control): Intractable. TLPlan (with Control): fastest, but limited parallelism Blackbox (without Control): slower, high parallelism Blackbox (with Control): faster, high parallelism

46. 46 Summary Easy to encode domain-specific knowledge in the planning as satisfiablity frame Key to order-of-magnitude scaling Propositional logic, temporal logic,... Can be applied before/after SAT encoding Can control time / quality tradeoff Power of underlying SAT engines gives option of finding higher quality solutions Heuristics are independent from the SAT engine Can use same axioms for radically different problem solvers

47. 47 How to Generate Control Kx Introspection Try to capture “obvious” inferences that are hard to deduce EBL (Minton, Kambhampati) Generalize trace of previous problem solving Static analysis (Smith, Etzioni, Knoblock, Peot) Analyze operators Inductive Logic Programming (Huang, Selman, Kautz) Find rules that hold for a set of previous high-quality solution plans

48. 48 Conclusions Propositional approaches to Open-Loop planning using general SAT engines are highly competitive with specialized planning algorithms Synergy with Plan Graph approaches Can effectively employ purely declarative control knowledge Biggest limitation: domains where number of objects is too large to instantiate