1. Computational Methods for Management and Economics Carla Gomes
Module 10b
Integer Programming
Solving Integer Programming Models
(slides adapted from Hillier and Hillier’s and Orlin’s)

2. The Challenges of Rounding
Rounded Solution may not be feasible.
Rounded solution may not be close to optimal.
There can be many rounded solutions.
Example: Consider a problem with 30 variables that are non-integer in the LP-solution. How many possible rounded solutions are there?
The solution to the LP-relaxation shown on the graph is approximately (3.8, 4.9).
None of the possible rounded solutions, (3, 4), (4, 4), (3, 5), or (4, 5), are even feasible.
The optimal solution at (1, 3) is not even close to the LP-relaxation solution.
There are 230, or approximately 1 billion rounded solutions to a problem with 30 variables that are non-integer.

3. How Integer Programs are Solved
This slide and the next can be used to intuitively explain the branch-and-bound procedure for solving integer programs.
The first step is to solve the LP relaxation. For this problem (as can be seen graphically), the optimal solution to the LP-relaxation is approximately (3.6, 4.3).
Since neither variable is integer, the next step is to branch and bound.
Two subproblems are created. In the first subproblem, the constraint x1 ≤ 3 is added. In the second subproblem, the constraint x1 ≥ 4 is added. In a feasible solution, x1 must either be ≤3 or ≥4, so the optimal solution must lie in one of these subproblems (we have not eliminated any feasible solutions).
The next slide shows the two subproblems.

4. How Integer Programs are Solved
The left subproblem (call it subproblem #1, with x1 ≤ 3) has an optimal solution of (3, 4.3). This solution is not integer either, so it is split into two subproblems (#1a and #1b), one with the added constraint x2≤4, and one with the added constraint x2≥5. Subproblem #1a would have a solution of (3,4) which is feasible and integer. Subproblem #1b would have no feasible solutions. (3,4) becomes the “incumbent” solution (the best feasible solution found so far).
The right subproblem (subproblem #2) has a solution of (4, 3.2). This solution is not integer either, so it is split into two subproblems (#2a and #2b), one with the added constraint x2≤3, and one with the added constraint x2≥4. Subproblem #2a would have a solution of (4.1,3) which is not integer. However, it has a lower objective function value than the “incumbent” found in Subproblem #1a (3, 4), so we can eliminate it from further consideration. Subproblem #2b would have no feasible solutions.
Therefore, since all other subproblems have been eliminated, the “incumbent” (3,4) is the optimal solution.
The main point: We (or the Solver) have to solve SEVEN LP’s for this simple two-variable problem. With more variables, the number of potential subproblems can explode. This helps explain why integer programs are so difficult to solve.

5. Overview of Techniques for Solving Integer Programs
Enumeration Techniques
Complete Enumeration
list all “solutions” and choose the best
Branch and Bound
Implicitly search all solutions, but cleverly eliminate the vast majority before they are even searched
Implicit Enumeration
Branch and Bound applied to binary variables
Cutting Plane Techniques
Use LP to solve integer programs by adding constraints to eliminate the fractional solutions.

6. Branch and Bound
It is the starting point for all solution techniques for integer programming
Lots of research has been carried out over the past 40 years to make it more and more efficient
But, it is an art form to make it efficient. (We shall get a sense why.)
Integer programming is intrinsically difficult.

7. Capital Budgeting Example
Investment budget = $14,000
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
xj binary for j = 1 to 6

8. Complete Enumeration
Systematically considers all possible values of the decision variables.
If there are n binary variables, there are 2n different ways.
Usual idea: iteratively break the problem in two. At the first iteration, we consider separately the case that x1 = 0 and x1 = 1.

9. An Enumeration Tree Original problem x1 = 0 x1 = 1 x2 = 0 x2 = 1

10. On complete enumeration
Suppose that we could evaluate 1 billion solutions per second.
Let n = number of binary variables
Solutions times (approx.)
n = 30, second
n = 40, minutes
n = days
n = years

11. On complete enumeration
Suppose that we could evaluate 1 trillion solutions per second, and instantaneously eliminate % of all solutions as not worth considering
Let n = number of binary variables
Solutions times
n = 70, second
n = 80, minutes
n = days
n = years

12. Branch and Bound Solve the linear program at the node
The essential idea: search the enumeration tree, but at each node
Solve the linear program at the node
Eliminate the subtree (fathom it) if
The solution is integer (there is no need to go further- why?) or
The best solution in the subtree cannot be as good as the best available solution (the incumbent- how does that happen?) or
There is no feasible solution

13. Branch and Bound Node 1 is the original LP Relaxation 1
44 3/7
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
0  xj  1 for j = 1 to 6
Solution at node 1:
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
The IP cannot have value higher than 44 3/7.

14. Branch and Bound
Node 2 is the original LP Relaxation plus the constraint x1 = 0. 1
44 3/7 2
x1 = 0 44
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
0  xj  1 for j = 1 to 6, x1 = 0
Solution at node 2: x1 = 0 x2 = x3 = 1/4 x4 = x5 = 0 x6 = z = 44

15. Branch and Bound
Node 3 is the original LP Relaxation plus the constraint x1 = 1. 1
44 3/7 3
x1 = 1 2
x1 = 0 44
44 3/7
The solution at node 1 was
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Note: it was the best solution with no constraint on x1. So, it is also the solution for node 3. (If you add a constraint, and the old optimal solution is feasible, then it is still optimal.)

16. Branch and Bound
Node 4 is the original LP Relaxation plus the constraints x1 = 0, x2 = 0. 1 1
44 3/7
x1 = 0
x1 = 1 2 3 44
44 3/7 4
x2 = 0 42 4
Solution at node 4: z = 42
Our first incumbent solution!
No solution in the subtree can have a value better than 42.
No further searching from node 4 because there cannot be a better integer solution.

17. Branch and Bound The incumbent solution has value 42 1 1 x1 = 0 x1 = 1
The incumbent is the best solution on hand. 1 1
44 3/7
x1 = 0
x1 = 1 2 3 44
44 3/7 4
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44
44 1/3
We next solved the LP’s associated with nodes 5, 6, and 7
No new integer solutions were found.
We would eliminate (fatham) a subtree if we were guaranteed that no solution in the subtree were better than the incumbent.

18. Branch and Bound The incumbent solution has value 42 1 x1 = 0 x1 = 1 2
44 3/7
x1 = 0
x1 = 1 2 3 44
44 3/7 4
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44
44 1/3 8
x3 = 0 9
x3 = 1 10
x3 = 0 11
x3 = 1 12
x3 = 0 13
x3 = 1 13
43.75
43.5
43.25
43.8
44.3 -
We next solved the LP’s associated with nodes 8 -13

19. Summary so far We have solved 13 different linear programs so far.
One integer solution found
One subtree fathomed (pruned) because the solution was integer (node 4)
One subtree fathomed because the solution was infeasible (node 13)
No subtrees fathomed because of the bound

20. Branch and Bound The incumbent solution has value 42 1 x1 = 0 x1 = 1 2
44 3/7
x1 = 0
x1 = 1 2 3 44
44 3/7
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44
44 1/3 8
x3 = 0 9
x3 = 1 10
x3 = 0 11
x3 = 1 12
x3 = 0
x3 = 1
43.75
43.5
43.25
43.8
44.3 - 13
43.75
42.66
We next solved the LP’s associated with the next nodes.
We can fathom the node with z = Why?

21. Getting a better bound
The bound at each node is obtained by solving an LP.
But all costs are integer, and so the objective value of each integer solution is integer. So, the best integer solution has an integer objective value.
If the best integer valued solution for a node is at most 42.66, then we know the best bound is at most 42.
Other bounds can also be rounded down.

22. Branch and Bound The incumbent solution has value 42 1 x1 = 0 x1 = 1 2
44 3/7
x1 = 0
x1 = 1 2 3 44
44 3/7
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44
44 1/3 8
x3 = 0 9
x3 = 1 10
x3 = 0 11
x3 = 1 12
x3 = 0
x3 = 1
43.75
43.5
43.25
43.8
44.3
-. 13
43.75
42.66

23. Branch and Bound The incumbent solution has value 42 1 x1 = 0 x1 = 1 244
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44 8
x3 = 0 9
x3 = 1 10
x3 = 0 11
x3 = 1 12
x3 = 0
x3 = 1 43 43 43 43 44 - 13 43 42

24. Branch and Bound The incumbent solution has value 42 1 x1 = 0 x1 = 1 244
The incumbent solution has value 42 1
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44 8
x3 = 0 9
x3 = 1 10
x3 = 0 11
x3 = 1 12
x3 = 0
x3 = 1 43 43 43 43 44 - 13 43 42 43 42 43 43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43

25. Branch and Bound The new incumbent solution has value 43 1 x1 = 044
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1 8 9 10 11 43 8 43 9 43 10 43 11 44 12 - 13 43 42 43 42 43 43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43

26. Branch and Bound The new incumbent solution has value 43 1 x1 = 044
The new incumbent solution has value 43 1
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1 8 9 10 11 43 8 43 9 43 10 43 11 44 12 - 13
If we had found this incumbent earlier, we could have saved some searching.

27. Finishing Up The new incumbent solution has value 43 1 x1 = 0 x1 = 1 244
The new incumbent solution has value 43 1
x1 = 0
x1 = 1 2 3 44 44
x2 = 0 5
x2 = 1 6
x2 = 0 7
x2 = 1 42 4 44 44 44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1 8 9 10 11 43 8 43 9 43 10 43 11 44 12 - 13 - 14 15 16 17 44 18 19 38

28. Lessons Learned Branch and Bound can speed up the search
Only 25 nodes (linear programs) were evaluated
Other nodes were fathomed
Obtaining a good incumbent earlier can be valuable
only 19 nodes would have been evaluated.
Solve linear programs faster, because we start with an excellent or optimal solution
uses a technique called the dual simplex method
Obtaining better bounds can be valuable.
We sometimes use properties that are obvious to us, such as the fact that integer solutions have integer solution values

29. Branch and Bound Notation: z* = optimal integer solution value
Subdivision: a node of the B&B Tree
Incumbent: the best solution on hand
zI: value of the incumbent
zLP: value of the LP relaxation of the current node
Children of a node: the two problems created for a node, e.g., by saying xj = 1 or xj = 0.
LIST: the collection of active (not fathomed) nodes, with no active children.
NOTE: zI  z*

30. Illustrating the definitions
The incumbent solution has value 42 1 2 3
x1 = 0
x1 = 1
44 3/7 44 4
x2 = 0 42 5
x2 = 1 6 7
44 1/3 8
x3 = 0 9
x3 = 1 10 11 12 13
43.75
43.5
43.25
43.8
44.3 -
z* = 43 = optimal integer solution value. (We found it later in the search)
Incumbent is zI = It is the optimal solution for the subdivision 4.
The zLP values for each subdivision are next to the nodes.

31. Illustrating the definitions
The incumbent solution has value 42 1 2 3
x1 = 0
x1 = 1
44 3/7 44 4
x2 = 0 42 5
x2 = 1 6 7
44 1/3 8
x3 = 0 9
x3 = 1 10 11 12 13
43.75
43.5
43.25
43.8
44.3 -
The children of node (subdivision) 1 are nodes 2 and 3. The children of node 3 are nodes 6 and 7.
LIST = { 8, 9, 10, 11, 12 } = unfathomed nodes with no active children

32. Branch and Bound Algorithm
INITIALIZE LIST = {original problem} Incumbent: =  zI = -
SELECT:
If LIST = , then the Incumbent is optimal if it exists, and the problem is infeasible if no incumbent exists;
else, let S be a node (subdivision) from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value 1
44 3/7
e.g., S = {1}
e.g., S = {13} 13 - 13
CASE zLP = - (the LP is infeasible)
Remove S from LIST (fathom it)
Return to SELECT

33. Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a node from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 2. - < zLP  zI.
That is, the LP is dominated by the incumbent.
Then remove S from LIST (fathom it)
Return to SELECT
e.g., the incumbent has value 43, and node 14 is selected. zLP = 43. 8 43 14 42 14

34. Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 3. zI < zLP and xLP is integral.
That is, the LP solution is integral and dominates the incumbent.
Then Incumbent := xLP; zI := zLP
Remove S from LIST (fathomed by integrality)
Return to SELECT
e.g., node 4 was selected, and the solution to the LP was integer-valued. 42 4 4

35. Branch and Bound Algorithm
INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 4. zI < zLP and xLP is not integral.
There is not enough information to fathom S
Remove S from LIST
Add the children of S to LIST
Return to SELECT
e.g., select node 3. 3
x1 = 1
44 3/7 6
x2 = 0 7
x2 = 1
List := List – 3 + {6,7}

36. Different Selection Rules are Possible
Rule of Thumb 1: Don’t let LIST get too big (the solutions must be stored). So, prefer nodes that are further down in the tree.
Rule of Thumb 2: Pick a node of LIST that is likely to lead to an improved incumbent. Sometimes special heuristics are used to come up with a good incumbent.

37. Branching
One does not have to have the B&B tree be symmetric, and one does not select subtrees by considering variables in order.
Choosing how to branch so as to reduce running time is largely “art” and based on experience. X
= 0 X
= 1 1 1 X
= 0 X
= 1 X 8
= 1
= 0 2 2 X
= 0 X
= 0
= 1 3
= 0 3 X 3 X 3
= 1
= 1 X
= 1 X
= 0 X X 3 3 3 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

38. Different Branching Rules are Possible
Branching: determining children for a node. There are many choices.
Rule of thumb 1: if it appears clear that xj = 1 in an optimal solution, it is often good to branch on xj = 0 vs xj = 1.
The hope is that a subdivision with xj = 0 can be pruned.
Rule of thumb 2: branching on important variables is worthwhile

39. Different Bounding Techniques are Possible
We use the bound obtained by dropping the integrality constraints (LP relaxation). There are other choices.
Key tradeoff for bounds: time to obtain a bound vs quality of the bound.
If one can obtain a bound much quicker, sometimes we would be willing to get a bound that is worse
It usually is worthwhile to get a bound that is better, so long as it doesn’t take too long.

40. What if the variables are general integer variables?
One can choose children as follows:
child 1: x1  3 (or xj  k)
child 2 x1  4 (or xj  k+1)
How would one choose the variable j and the value k
A common choice would be to take a fractional value from xLP. e.g., if x7 = 5.62, then we may branch on x7  5 and x7  6.
Other choices are also possible.

42. Summary
Branch and Bound is the standard way of solving IPs to optimality.
There is art to making it work well in practice.
Much of the art is built into state-of-the-art solvers such as CPLEX.