1. Reviving Integer Programming Approaches for AI Planning: A Branch-and-Cut Framework Thomas Vossen Leeds School of Business University of Colorado at Boulder Boulder, CO vossen@colorado.edu Subbarao Kambhampati Department of Computer Science Arizona State University Tempe, AZ rao@asu.edu Menkes van den Briel Department of Industrial Engineering Arizona State University Tempe, AZ menkes@asu.edu

2. Compilation approaches for AI planning Integer Programming (IP) Propositional Satisfiability (SAT) Constraint Satisfaction (CSP) Planning Very little focus on IP There were no IP-based planners in IPCs 1998, 2000, 2002

3. Background Why should we pursue integer programming? –Support for optimality –Support for numeric constraints –IP has shown to be a great success in solving large scale optimization problems

4. Reviving integer programming 1.State variables are modeled by an appropriately defined network –The resulting IP formulation can be interpreted as a network flow problem with additional side constraints 2.The common notion of parallelism based on planning graphs is generalized –This reduces the plan length of the solution plan 3.Action ordering constraints are dynamically generated when needed by a branch-and-cut algorithm –Branch-and-cut has been extremely successful for a large number of optimization problems

5. Example: State change flow network LOC1LOC2

6. Example: State change flow network I I G AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 t = 1t = 2t = 3 LOC1LOC2 0123 Package1 Truck1

7. Example: State change flow network I I G AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 t = 1t = 2t = 3 LOC1LOC2 0123 Package1 Truck1

8. Action effects STRIPS (binary-valued) SAS+ (multi-valued) LOAD(Package1,Loc1) PRE: Package1_AT_LOC1, Truck1_AT_LOC1 DEL: Package1_AT_LOC1 ADD: Package1_IN_TRUCK1 LOAD(Package1,Loc1) State change effects Package1: AT_LOC1  IN_TRUCK1 Prevail conditions Truck1: AT_LOC1  AT_LOC1 LOC1 Prevail condition State change effect

9. Example: state change flow network I I G AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 t = 1t = 2t = 3 LOC1LOC2 Action effects link multiple networks together Package1 Truck1 LOAD(Package1)  DRIVE(Truck1,Loc1,Loc2)  UNLOAD(Package1)

10. Single state change (1SC) formulation Constraints –State changes (network flow), for all c  C  g  C y c f,g,t = 1{f  I}for f  D c  h  C y c g,h,t+1 =  f  C y c g,h,t for f  D c, 1  t < T  f  C y c f,g,T = 1for g  G –Effect implications, for all c  C, 1  t  T  a  A:(f,g)  SCa x a,t = y c f,g,t for f, g  D c, f  g x a,t  y c f,f,t for a  A, (f,f)  PRa

11. Reviving integer programming 1.State variables are modeled by an appropriately defined network –The resulting IP formulation can be interpreted as a network flow problem with additional side constraints 2.The common notion of parallelism based on planning graphs is generalized –This reduces the plan length of the solution plan 3.Action ordering constraints are dynamically generated when needed by a branch-and-cut algorithm –Branch-and-cut has been extremely successful for a large number of optimization problems

12. Generalized parallelism Common notion of parallelism (Graphplan) –Actions can be executed in the same plan step as long as they do not interfere with each other (no action deletes the precondition or add effect of another action) –In an actual plan it is possible to arrange independent parallel actions in any order with exactly the same outcome Alternative concept of parallelism –Allow actions to be executed in the same plan step as long as there exists a feasible ordering of the actions such that: All the actions preconditions are met Within each state variable and for each plan step, the value is changed at most once

13. Generalized parallelism Increase in parallelism requires “generalized” state changes Graphplan parallelismAlternative parallelism

14. Example: Generalized parallelism AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 Package1 Truck1 Action effect implications LOAD(Package1,Truck1) Increase in parallelism requires “generalized” state changes LOAD(Package1,Truck1) DRIVE(Truck1,Loc1,Loc2)

15. Implied precedences A1,A2 A3 A4 A1 A2 A3A4 Implied precedence graph

16. Implied precedences Cycle elimination constraints need to ensure feasible orderings A1,A2 A3 A4 A1 A2 A3A4 Implied precedence graph

17. Generalized single state change (G1SC) formulation Constraints –State changes (network flow), for all c  C  g  C y c f,g,t = 1{f  I}for f  D c  h  C y c g,h,t+1 =  f  C y c g,h,t for f  D c, 1  t  T  f  C y c f,g,T = 1for g  G  D c –Effect implications, for all c  C, 1  t  T  a  A:(f,f)  SCa x a,t = y c f,g,t for f, g  D c, f  g, x a,t  y c f,f,t +  g  Dc:f≠g (y c g,f,t + y c f,g,t ) for a  A, (f,f)  PRa –Ordering (Cycle elimination) constraints  a  V(  ) x a,t  |V(  )| – 1for all cycles  G, 1  t  T

18. Reviving integer programming 1.State variables are modeled by an appropriately defined network –The resulting IP formulation can be interpreted as a network flow problem with additional side constraints 2.The common notion of parallelism based on planning graphs is generalized –This reduces the plan length of the solution plan 3.Action ordering constraints are dynamically generated when needed by a branch-and-cut algorithm –Branch-and-cut has been extremely successful for a large number of optimization problems

19. Branch-and-cut algorithm Branch-and-cut (row generation with branch-and- bound) –Classes of valid inequalities are left out of the LP relaxation because there are too many constraints to handle efficiently, and most of them will not be binding in an optimal solution anyway –Branching occurs when no violated inequalities in a class are found and the LP solution does not satisfy the integrality constraints

20. Reviving integer programming 1.State variables are modeled by an appropriately defined network –The resulting IP formulation can be interpreted as a network flow problem with additional side constraints 2.The common notion of parallelism based on planning graphs is generalized –This reduces the plan length of the solution plan 3.Action ordering constraints are dynamically generated when needed by a branch-and-cut algorithm –Branch-and-cut has been extremely successful for a large number of optimization problems

21. Further generalizations Allow actions to be executed in the same plan step as long as there exists a feasible ordering of the actions such that –All the actions preconditions are met –For each state variable, more than one state change is allowed, e.g. “state change paths” I I G AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 t = 1t = 2 00’11’ Package1 Truck1

22. Further generalizations There exists a feasible ordering of the actions such that –All the actions preconditions are met –For each state variable, more than one state change is allowed, e.g. “state change paths” I G I AT_LOC1 AT_LOC2 IN_TRUCK1 AT_LOC1 AT_LOC2 t = 1 00’ Package1 Truck1 LOAD(Package1)  DRIVE(Truck1,Loc1,Loc2)  UNLOAD(Package1)

23. State change path (kSC) formulation Constraints –State changes (network flow), for all c  C  g  C,f  g y c g,f,1 + 1 {f  I} = y c f,f,1 y c f,f,1 =  g  C,f  g y c f,g,1 + z c f,1 for f  C  g  C,f  g y c g,f,1 + z c f,t-1 = y c f,f,t y c f,f,t =  g  C,f  g y c f,g,t + z c f,t for f  C, 1  t  T – 1  g  C,f  g y c g,f,T + z c f,T-1 = y c f,f,T y c f,f,T =  g  C,f  g y c f,g,T + z c f,T for f  C –Effect implications, for all c  C, 1  t  T  a  A:(f,f)  SCa x a,t = y c f,g,t for f, g  D c, f  g, x a,t  y c f,f,t +  g  Dc:f≠g (y c g,f,t + y c f,g,t ) for a  A, (f,f)  PRa –Ordering (Cycle elimination) constraints  a  V(  ) x a,t  |V(  )| – 1for all cycles  G, 1  t  T

24. Experimental Setup Four domains from the International Planning Competition –Blocks-world, Logistics, Driverlog, Zenotravel –Used Fast Downward preprocessor (translates PDDL to multi-valued state variables) Compared results of 1SC, G1SC, and kSC to SATPLAN04 (SAT4) –State-of-the art SAT-based planner –First prize in the IPC-2004 Resources –2.67GHz Linux machine, 1Gb memory –ILOG CPLEX 8.1 –30 minute maximum for each problem

25. Performance Quality

27. BlocksLogisticsDriverlogZenotravel Results: plan length

28. Summary and conclusions First integer programming formulation that is competitive with state-of-the art SAT-based planner: –Uses multi valued states and represent state changes as flows in an appropriately defined network –Uses generalized notion of parallelism in a branch-and-cut algorithm Future work –Optimal planning (minimum number of actions, minimum cost plans, minimum resource usage, minimum makespan) –Expand the scope of planning problems, apply integer programming techniques to planning with resources and temporal planning