1. Integer Programming 3 Brief Review of Branch and Bound
Cutting plane techniques for getting improved bounds
MIT and James Orlin © 2003

2. Review from Last Lecture
Investment budget = $14,000
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
xj binary for j = 1 to 6
MIT and James Orlin © 2003

3. Branch and Bound The incumbent solution has value 43 1 x1 = 0 x1 = 1 244
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1 8 9 10 11 43 8 43 9 43 10 43 11 44 12 - 13 - 14 15 16 17 44 18 19 38
MIT and James Orlin © 2003

4. On Bounds from Linear Programming
We found an incumbent integer solution with a value of 43.
The best LP solution value is between 44 and 45.
Is there a way that we could have directly established an upper bound between 43 and 44? Perhaps we could form a “better linear program.”
The closer the LP is to the Integer Program, the better.
Note: not all LP formulations are the same in terms of “closeness”
MIT and James Orlin © 2003

5. Overview of cutting planes
Try to get better bounds so that more nodes of the branch and bound tree can be fathomed, and the fathoming can be done earlier.
Recall: we eliminate a node j if the LP bound for j is no more than the value of the incumbent.
The lower the bound (for a maximization problem) the better
This lecture will provide several approaches for obtaining bounds.
MIT and James Orlin © 2003

6. Goals of this lecture
Illustrate how valuable obtaining better LP’s can be
Illustrate techniques for obtaining better bounds
set packing
capital budgeting
traveling salesman problem
general integer programs
MIT and James Orlin © 2003

7. What is the maximum number of diamonds that can be packed on a Chinese checkerboard.
Special Thanks to Professor Andreas Schulz for this example.
MIT and James Orlin © 2003

8. The diamonds are not permitted to overlap, or even to share a single circle. What is the best packing you can find?
MIT and James Orlin © 2003

9. Is this the best possible?
MIT and James Orlin © 2003

10. Set Packing Problem Let D be the collection of diamonds
Let O be the pairs of diamonds that overlap.
(d, d’)  O, implies that diamonds d and d’ have a point in common
Decision variables: xd for d  D
xd = 1 if diamond d is selected
xd = 0 if diamond d is not selected.
MIT and James Orlin © 2003

11. How many decision variables are there?
For each black circle, one can hang a yellow diamond from it.
There are 88 black circles.
And there are 88 green and red diamonds.
So, there are 88 possible yellow diamonds.
for a total of 264 diamonds.
MIT and James Orlin © 2003

12. The First Integer Programming Formulation
This problem is known as the set packing problem.
The optimal objective value for the LP relaxation is 132.
The optimal objective value for this IP is 27.
MIT and James Orlin © 2003

13. For many dots, there are 12 diamonds that go through the dot.
Heading towards an improved IP formulation. 2 5 6 8 7 1 3 4
MIT and James Orlin © 2003

14. Getting an improved LP
We say that S is an overlapping set of diamonds, if every two diamonds in S overlap.
x1 + x2  1
x1 + x3  1
x1 + x4  1
x2 + x3  1
x2 + x4  1
x3 + x4  1
x1, x2, x3, x4  {0,1}
Conclusion: x1 + x2 + x3 + x4  1
for each overlapping set S.
MIT and James Orlin © 2003

15. An improved integer program
The optimal objective value for this IP is 27
Next: a proof that the at most 27 diamonds can be packed.
The optimal objective value for the LP relaxation is 27.5
MIT and James Orlin © 2003

16. The weight of a diamond is the sum of its node weights.1
1/2
1/3
1/6
The weight of a diamond is the sum of its node weights.
MIT and James Orlin © 2003

17. The weight of a diamond is the sum of its node weights.1
1/2
1/3
1/6
The weight of a diamond is the sum of its node weights.
Each diamond has weight at least 1.
MIT and James Orlin © 2003

18. The weight of 27 independent diamonds is at least 27.
Node Weight 1
1/2
1/3
1/6 18 1 24 6
The weight of 27 independent diamonds is at least 27.
The weight of 28 independent diamonds would be at least 28.
But the total weight of all nodes is 27.5
MIT and James Orlin © 2003

19. No solution has more than 27 diamonds.
Node Weight 1
1/2
1/3
1/6
No solution has more than 27 diamonds.
The weights were the shadow prices for the LP
MIT and James Orlin © 2003

20. The Optimal Solution Again
MIT and James Orlin © 2003

21. Set Packing Problem Putting items into storage Manufacturing
How many car doors pack into a metal sheet?
How many rolls of paper can be cut from a giant roll?
Closely related to fire station problem: set covering.
MIT and James Orlin © 2003

22. Review There may be different ways of formulating an IP.
Each way gives the same IP
The LPs may be very different
Some LPs have different bounds
NEXT: The geometry of Integer Programs and Linear Programs
MIT and James Orlin © 2003

23. Using Cutting Planes y P x
Optimum (integer) solution P
Optimum fractional (i.e. infeasible) solution x
Example. Minimize x + 10y subject to x, y are in P x, y integer

24. Using Cutting Planes y P x
Idea: add constraints that eliminate fractional solutions to the LP without eliminating any integer solutions. y P
add “y 1”
add “y  x –1” x
Example. Minimize x + 10y subject to x, y are in P x, y integer
These constraints were obtained by inspection. We will develop techniques later.

25. Using Cutting Planes y P x
If we add exactly the right inequalities, then every corner point of the LP will be integer, and the IP can be solved by solving the LP y
Optimum (integer) solution
We call this minimal LP, the convex hull of the IP solutions. P x
Example. Minimize x + 10y subject to x, y are in P x, y integer
For large problems, these constraints are hard to find.

26. More on adding constraints
The tightest possible constraints are very useful, and are called facets.
Suppose that we are maximizing, and zLP is the opt for the LP relaxation, and zIP is the opt for the IP. Then zIP  zLP
Ideally, we want zIP to be close to zLP. This is GREAT for branch and bound.
Adding lots of valid inequalities can be very helpful.
It has no effect on zIP.
It can reduce zLP significantly.
MIT and James Orlin © 2003

27. “Pure” Cutting Plane Technique
Instead of partitioning the feasible region, the (pure) cutting plane technique works with a single LP
It adds cutting planes (valid linear programming inequalities) to this LP iteratively.
At each iteration the feasible region is successively reduced until an integer optimal is found by solving the LP.
In practice, it is also used as part of branch and bound. The essential idea is finding valid “cuts” or inequalities.
MIT and James Orlin © 2003

28. Where do these cuts come from?
Two approaches
Problem specific
Set Packing
Capital Budgeting (knapsack)
Traveling Salesman Problem
LP-based approach, that works for general integer programs
Gomory cutting planes
MIT and James Orlin © 2003

29. Problem Specific The capital budgeting (knapsack) problem
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
xj binary for j = 1 to 6
MIT and James Orlin © 2003

30. The LP Relaxation The capital budgeting (knapsack) problem
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
0  xj  1 for j = 1 to 6
The optimal solution: x1 = 1, x2 = 3/7, x3 = x4 = 0, x5 = 0, x6 = 1
We say that a subset S is a cover if the sum of its weights is more than 14 (more than the budget)
What are some covers of this capital budgeting problem?
MIT and James Orlin © 2003

31. Getting constraints from covers
5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
Consider the cover S = {1, 2, 3}
We have the constraint 5x1 + 7x2 + 4x3  14
We can obtain the constraint x1 + x2 + x3  2.
This is a cover constraint
This constraint is “stronger”
It does not eliminate any IP solutions, but it cuts off fractional LP solutions.
In general, for each cover S,we obtain the constraint SjS xj  |S| - 1
MIT and James Orlin © 2003

32. The LP Relaxation The capital budgeting (knapsack) problem
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
0  xj  1 for j = 1 to 6
The optimal solution: x1 = 1, x2 = 3/7, x3 = x4 = 0, x5 = 0, x6 = z = 44 3/7
5x1 + 7x2 + 6x6  
x1 + x2 + x6  2
Our approach: Given the linear solution, try to find a violated cover constraint.
A violated cover is {1, 2, 6}
Excel
MIT and James Orlin © 2003

33. After one cut maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
x1 + x x6  2
0  xj  1 for j = 1 to 6
The optimal solution: x1 = 0, x2 = 1, x3 = 1/ x4 = 0, x5 = 0, x6 = z = 44
Exercise: find a violated cover.
Excel
MIT and James Orlin © 2003

34. After two cuts maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
x1 + x x6  2
x2 + x x6  2
0  xj  1 for j = 1 to 6
The optimal solution: x1 = 1/3, x2 = 1, x3 = 1/ x4 = 0, x5 = 0, x6 = z = 44
Exercise: Find a violated cover:
Excel
MIT and James Orlin © 2003

35. Obtaining a stronger constraint
5x1 + 7x2 + 4x3 + 6x6  14
At most 2 of x1, x2, x3, x6 can be 1.
So x1 + x2 + x3 + x6  2.
MIT and James Orlin © 2003

36. After “three” cuts maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
x1 + x x6  2
x2 + x x6  2
x1 + x2 + x x6  2
0  xj  1 for j = 1 to 6
Note: the new cuts dominates the other cuts.
Excel
MIT and James Orlin © 2003

37. We eliminate the redundant constraints
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
x1 + x2 + x x6  2
0  xj  1 for j = 1 to 6
The optimal solution: x1 = 0, x2 = 1, x3 = 0, x4 = 0, x5 = 1/4, x6 = z = 43 3/4
So, z*  43
Excel
MIT and James Orlin © 2003

38. Summary for knapsack problem
We could find some simple valid inequalities that showed that z*  43. This is the optimal objective value.
It took 3 cuts
Had we been smarter it would have taken 1 cut
We had a simple approach for finding cuts.
This does not find all of the cuts.
Recall, it took 25 nodes of a branch and bound tree
In fact, researchers have found cutting plane techniques to be necessary to solve large integer programs (usually as a way of getting better bounds.)
MIT and James Orlin © 2003

39. Traveling Salesman Problem (TSP)
What is a minimum length tour that visits each point?
MIT and James Orlin © 2003

40. Comments on the TSP Very well studied problem
It’s often the problem for testing out new algorithmic ideas
NP-complete (it is intrinsically difficult in some technical sense)
Large instances have been solved optimally (5000 cities and larger)
Very large instances have been solved approximately (10 million cities to within a couple of percent of optimum.)
We will formulate it by adding constraints that look like cuts
MIT and James Orlin © 2003

41. The TSP as an IP, almost Are these constraints enough?
Let xe = 1 if arc e is in the tour xe = 0 otherwise
Let A(i) = arcs incident to node i
Minimize Se ce xe
subject to SeA(i) xe = 2
xe is binary
Are these constraints enough?
MIT and James Orlin © 2003

42. Subtour: a cycle that passes through a strict subset of cities
Fact: Any integer solution with exactly two arcs incident to every node is the union of cycles. Why? 3 2 4 7 9
Improved IP/LP: for each possible subtour, add a constraints that makes the subtour infeasible for the IP. These are called subtour breaking constraints.
MIT and James Orlin © 2003

43. A subtour breaking constraint
Let S be any proper subset of nodes, e.g., S = {2, 3, 4, 7, 9}. 7 2 9 3 4
Observations:
A subtour that includes all nodes of S has |S| arcs
2. A tour for the entire network has at most |S| - 1 arcs with two endpoints in S.
This ensures that the set S will not have a subtour going through all five nodes.
MIT and James Orlin © 2003

44. A traveling salesman tour7 2 9 3 4
There are at most 4 arcs of any tour incident to 2, 3, 4, 7, 9
MIT and James Orlin © 2003

45. Formulation of the TSP as an IP
Minimize Se ce xe
subject to Se inc to i xe = 2
Se in S xe  |S| - 1
(subtour breaking constraints)
xe is binary
Exponentially many constraints, too many to include in an IP or an LP
In practice:
Include only some of the constraints. Solve the LP.
If a subtour appears as part of the LP solution, add a new subtour elimination constraint to the LP, and solve again.
MIT and James Orlin © 2003

46. More on the TSP
The IP formulation is a very good formulation good in the following sense: for practical problem the LP bound is usually 1% to 2% from the optimal TSP tour length. (The LP bound is close.)
One can add even more complex constraints to get a better LP formulation, and people do. (and it helps)
MIT and James Orlin © 2003

47. Gomory Cuts: a method of generating cuts using LP tableaus
Consider the following constraint in an integer program x1 x2 x4 x3
1 3/5
4 1/5
2/5 3 =
9 4/5
Focus on the fractions.
 3/5 x1 + 1/5 x2 +2/5 x4 = Integer + 4/5
 3/5 x1 + 1/5 x2 + 2/5 x4  4/5
 x x x4  4.
MIT and James Orlin © 2003

48. Gomory Cuts What did we rely on to obtain the Gomory cut?x1 x2 x3 x4
1 3/5
4 1/5 3
2/5
9 4/5 =
A single constraint with a fractional right hand side
All coefficients of the constraint were positive.
All variables must be integral
MIT and James Orlin © 2003

49. What to do if coefficients are negativex1 x2 x4 x3
1 3/5
-4 3/5
-2/5 -3 =
-1 1/5
Rewrite so that the coefficients of the fractional parts are positive.
1 3/5
-5 +2/5 -3
-1 +3/5
-2 +4/5 =
 3/5 x1 + 2/5 x2 +3/5 x4 = integer + 4/5 .
 3/5 x1 + 2/5 x2 +3/5 x4  4/5 .
 x x x4 
MIT and James Orlin © 2003

50. In general   Let fr(a) be the positive fractional part of a ;
fr(a) = a - a
fr(2 3/5) = /5 – 2 = 3/5
fr(-2 3/5) = /5 – (-3) = 2/5 x1 x2 x3 x4
1 3/5
-4 3/5 -3
-2/5
-1 1/5 =
3/5
2/5
+3/5 
+4/5 x1 x2 x3 x4
ai1
ai2
ai3
ai4
bi =
fr(ai1)
fr(ai2)
fr(ai3)
fr(ai4) 
fr(bi)
MIT and James Orlin © 2003

51. How to generate cutting planes?
In general
After pivoting, find a basic variable that is fractional.
Write a Gomory cut. (It is an inequality constraint, leading to a new slack variable).
Note: the only coefficients that are fractional correspond to non-basic variables (why?)
The Gomory cut makes the previous basic feasible solution infeasible. (why?)
Resolve the LP with the new constraint, and iterate.
MIT and James Orlin © 2003

52. Integer Programming Summary
Dramatically improves the modeling capability
Economic indivisibilities
Logical constraints
Modeling non-linearities
Not as easy to model.
Not as easy to solve.
MIT and James Orlin © 2003

53. IP Solution Techniques Summary
Branch and Bound
very general and adaptive
used in practice (e.g. Excel Solver)
Implicit Enumeration
branch and bound technique for binary IPs
Cutting planes
clever way of improving bounding
active area for research, theoretical and applied
MIT and James Orlin © 2003

54. Example: Fire company location.
Consider locating fire companies in different districts.
Objective: place fire companies so that each district either has a fire company in it, or one that is adjacent, and so as to minimize cost.
MIT and James Orlin © 2003

55. Example for the Fire Station Problem
Let xj = 1 if a fire station is placed in district j. xj = 0 otherwise. 1 2 3 5 6 7
Let cj = cost of putting a fire station is district j. 4 8 9
With partners, formulate the fire station problem. 11 12 13 10 14 15 16
MIT and James Orlin © 2003

56. Set covering problem set S= {1, …, m} of items to be covered
districts that need to have a fire station or be next to a district with one
n subsets of S
For each possible location j for a fire station, the subset is district j plus the list of districts that are adjacent to district j.
aij = 1 if district i is adjacent to district j or if i = j.
Minimize
subject to
MIT and James Orlin © 2003

57. Covering constraints are common
Fleet routing for airlines:
Assigning airplanes to flight legs.
Each flight must be included
Assigning crews to airplanes
Each plane must be assigned a crew
Warehouse location
Each retailer needs to be served by some warehouse
MIT and James Orlin © 2003

58. Independent Set (set packing)
What is the maximum number of districts no two of which have a common border? 1 2 3 5 6 7 4
With your partner, formulate the independent set problem. List all constraints containing x10. 8 9 11 12 13 10 14 15 16
MIT and James Orlin © 2003

59. Set Packing constraints arise often in manufacturing and logistics
Cutting shapes out of sheet metal
Manufacturing many items at once which share resources (how much work can be done in parallel?)
MIT and James Orlin © 2003