1. Formulations and Reformulations in Integer Programming Michael Trick Carnegie Mellon University Workshop on Modeling and Reformulation, CP 2004

2. Goals Provide a perspective on what makes a “good” integer programming formulation for a problem Give examples on automatic versus manual reformulation of problems Outline some challenges in the automatic reformulation of integer programs (and perhaps constraint programs?)

3. Outline Quick review of key concepts in integer programming Two models  Truck-route contracting  Traveling Tournament Problem General Comments

4. Integer Program (IP) Minimize cx Subject to Ax=b l<=x<=u some or all of x j integral X: variables Linear objective Linear constraints Makes things hard!

5. Rules of the Game Must put in that form! Seems limiting, but 50 years of experience gives “tricks of the trade” Many formulations for same problem

6. Simple example Variables x, y both binary (0-1) variables Formulate requirement that x can be 1 only if y is 1 Formulation 1: x ≤ y; x,y  {0,1} Formulation 2: x ≤ 20y; x,y  {0,1} Are they different? Do we care which we use?

7. Differences From a modeling point of view, they are the same: they both correctly model the given requirement From an algorithmic point of view, they may be different, depending on algorithm used

8. Solving Integer Programming problems Most common method is some form of branch and bound  Use linear relaxation to bound objective value  Branch on fractional values in linear relaxation solution  Stop branching when subproblem is Infeasible Integer Fathomed (cannot be better than best found so far)

9. Linear Relaxation Minimize cx Subject to Ax=b l<=x<=u some or all of x j integral X: variables Linear objective Linear constraints Makes things hard!

10. Illustration

11. Linear Relaxation

12. Key is linear relaxation If linear relaxation is very different from integer program then  Choose wrong variables to branch on  Fathoming will be done less often

13. Ideal Formulation gives convex hull of feasible integer points

14. Simple example (binary variables) x ≤ y x ≤ 20 y x y x y

15. Fundamental Mantra of Integer Programming Formulations Use formulations with good linear relaxations! Other issues in formulations: avoiding symmetry issues, keeping problem size down, scaling, etc. that will not be covered here This guideline is quite misleading!

16. Model 1: Truck Route Contracting Real application Highly simplified version (which shows everything I learned) AB D: 8, A: 12, $150, C: 100 D: 9, A: 1, $250, C: 80 D: 10, A: 2, $200, C: 125 TRUCK DATA D: Departure Time A: Arrival Time $: Cost C: Capacity Sample Package Size: 10 Time Available: 9 Time Needed: 2 Problem: Purchase trucks sufficient to move all packages on time

17. Model Variables: y(i) = 1 if truck i purchased, 0 else x(j,i) = 1 if package j on i, 0 else Objective: Minimize truck costs Constraints: Packages fit on assigned truck Use only paid for trucks Every package on some truck No partial trucks or package splitting

18. Formulation: declarations model "Transportation Planning" uses "mmxprs" declarations TRUCKS = 1..10 PACKAGES = 1..20 capacity: array(TRUCKS) of real size: array(PACKAGES) of real cost: array(TRUCKS) of real can_use: array(PACKAGES,TRUCKS) of real x: array(PACKAGES,TRUCKS) of mpvar y: array(TRUCKS) of mpvar end-declarations capacity:= [100,200,100,200,100,200,100,200,100,200] size := [17,21,54,45,87,34,23,45,12,43, 54,39,31,26,75,48,16,32,45,55] cost := [1,1.8,1,1.8,1,1.8,1,1.8,1,1.8] can_use:=[0-1 matrix whether package can go on truck]

19. Formulation: Constraints Total := sum(i in TRUCKS) cost(i)*y(i) forall(i in TRUCKS) sum(j in PACKAGES) size(j)*x(j,i) <= capacity(i) ! (1) Packages fit forall (i in TRUCKS) sum (j in PACKAGES) x(j,i) <= NUM_PACKAGE*y(i) ! (2) use only ! paid for trucks forall (j in PACKAGES) sum(i in TRUCKS) can_use(j,i)*x(j,i) = 1 ! (3) every ! package on truck forall (i in TRUCKS) y(i) is_binary ! (4) no partial trucks forall (i in TRUCKS, j in PACKAGES) x(j,i) is_binary ! (5) no package splitting minimize(Total) end-model

20. “Improving the Formulation” Every integer programming will immediately spot the improvements: forall (i in TRUCKS) sum (j in PACKAGES) x(j,i) <= NUM_PACKAGE*y(i) ! (2) use only ! paid for trucks should be replaced with forall (i in TRUCKS, j in PACKAGES) x(j,i) <= y(i) !(2’) tighter formulation which we saw as “tighter” (though bigger)

21. Other improvements Integer programmers are good at spotting opportunities: forall(i in TRUCKS) sum(j in PACKAGES) size(j)*x(j,i) <= capacity(i) ! (1) Packages fit Can be strengthened with forall(i in TRUCKS) sum(j in PACKAGES) size(j)*x(j,i) <= capacity(i)*y(i) ! (1’) Packages fit

22. Results Weak Formulation: 11.2 sec, 31,825 nodes Strong Formulation: 22.1 sec, 50,631 nodes WHAT HAPPENED?

23. Automatic versus Manual Reformulations XPRESS-MP (ILOG’s CPLEX will work the same) “knows” about this form of tightening. It will do it automatically In fact, it will do it “better”, only including constraints that the linear relaxation points to as relevant Automatic reformulation trumps manual reformulation in this case!

24. Naïve code If you use a naïve code that doesn’t understand this, then tightened formulation is critical: Weak formulation: Unsolved after 3600 seconds (gap is 1.22 – 8.4) Strong formulation: 1851 seconds, 2.4 million nodes But who would use such a code for real work?

25. Gets more confusing Consider the constraint sum(i in TRUCKS) capacity(i)*y(i) >= sum (j in PACKAGES)size(j) ! (6) Have sufficient capacity Such a constraint does not tighten the formulation (it is a linear combination of existing constraints): fundamental mantra says don’t add. Solution time:.1 seconds, 1 node

26. What happened XPRESS (and other sophisticated codes) knows a lot about “knapsack” constraints and does automatic tightening on those Can’ identify knapsack constraint, but once identified by user, can tighten (a lot!).

27. Summary of model 1 Standard tightening methods by user makes things slower Creative addition of constraint that does not appear to tighten relaxation makes things much faster

28. Model 2: Traveling Tournament Problem Given an n by n distance matrix D= [d(i,j)] and an integer k find a double round robin (every team plays at every other team) schedule such that:  The total distance traveled by the teams is minimized (teams are assumed to start at home and must return home at the end of the tournament), and  No team is away more than k consecutive games, or home more than k consecutive games. (For the instances that follow, an additional constraint that if i is at j in slot t, then j is not at i in t+1.)

29. Sample Instance NL6: Six teams from the National League of (American) Major League Baseball. Distances: 0 745 665 929 605 521 745 0 80 337 1090 315 665 80 0 380 1020 257 929 337 380 0 1380 408 605 1090 1020 1380 0 1010 521 315 257 408 1010 0 k is 3

30. Sample Solution Distance: 23916 (Easton May 7, 1999) Slot ATL NYM PHI MON FLA PIT 0 FLA @PIT @MON PHI @ATL NYM 1 NYM @ATL FLA @PIT @PHI MON 2 PIT @FLA MON @PHI NYM @ATL 3 @PHI MON ATL @NYM PIT @FLA 4 @MON FLA @PIT ATL @NYM PHI 5 @PIT @PHI NYM FLA @MON ATL 6 PHI @MON @ATL NYM @PIT FLA 7 MON PIT @FLA @ATL PHI @NYM 8 @NYM ATL PIT @FLA MON @PHI 9 @FLA PHI @NYM PIT ATL @MON

31. Simple Problem, yes? NL12. 12 teams Feasible Solution: 143655 (Rottembourg and Laburthe May 2001), 138850 (Larichi, Lapierre, and Laporte July 8 2002), 125803 (Cardemil, July 2 2002), 119990 (Dorrepaal July 16, 2002), 119012 (Zhang, August 19 2002), 118955 (Cardemil, November 1 2002), 114153 (Van Hentenryck January 14, 2003), 113090 (Van Hentenryck February 26, 2003), 112800 (Van Hentenryck June 26, 2003), 112684 (Langford February 16, 2004), 112549 (Langford February 27, 2004), 112298 (Langford March 12, 2004), 111248 (Van Hentenryck May 13, 2004). Lower Bound: 107483 (Waalewign August 2001)

32. Formulation as IP Straightforward formulation is possible: plays(i,j,t) = 1 if i at j in slot t Need auxiliary variables location (i,j,t) = 1 if i in location j in slot t follows(i,j,k,t) = 1 I travels from j to k after slot t

33. Formulation Rest of formulation in paper (pages 9 and 10 in proceedings) Result is a mess  N=6  After 1800 seconds gap is 5434 – 25650 (optimal is 23,916) Anything XPRESS is doing is not helping enough!

34. Reformulation Sample Variables: @NY@MON @PHI @NY H H H X1 X2 X3 Y1 Y2H

35. Constraints One thing per time: X1+X2+Y1+Y2  1 @NY@MON @PHI H H H X1 X2 Y1 Y2H

36. Constraints No Away followed by Away X1+X3  1 @MON @PHI @NY X2 X3

37. Rest of formulation Rest of formulation is straightforward (in proceedings, looking more complicated than it needs to) Result: initial relaxation (for n=6) 21,624.7 Optimal: 4136 seconds, 66,000 nodes

38. Strengthening the Constraints Stronger: X1+X2+X3+Y2  1 @NY@MON @PHI @NY H X1 X2 X3 Y2H

39. Result Initial relaxation same, solution time a little longer What happened: “Strengthening” is type of clique inequality, known by XPRESS Without clique inequalities: unsolved after more than 36,000 seconds

40. Conclusions for Model 2 Initial formulation almost hopeless Manual reformulation needed to redefine variables Then, automatic reformulation can improve results tremendously

41. Questions What is the role of manual versus automatic reformulation?  Model 1: manual needed to identify hidden constraint  Model 2: manual needed to redefine the variables Is this an ever-moving line, or are some aspects intrinsically difficult to determine? How can software be developed to better  Do automatic reformulation  Provide flexibility to experiment with different reformulations/reformulation levels

42. Resources Introduction to Integer Programming (by Bob Bosch and me) and this talk  Will be at http://mat.tepper.cmu.edu/trickhttp://mat.tepper.cmu.edu/trick XPRESS-MP and ILOG’s OPL Studio provide great software to experiment with