1. EE 553 Integer Programming
J. McCalley

2. Solving IPs How to solve it? Minimize f(x1, x2, x3) Subject to
h(x1, x2, x3)=c
g(x1, x2, x3)≤b
x1, x2, x3 are binary (1 or 0)
How to solve it?
Two immediate ideas:
Check every possible solution (exhaustive enumeration),
A huge computational burden.
2. Solve it as an LP and then round to closest binary value.
Under such an approximation, optimality cannot guaranteed.

3. Solving IPs: Tree search methods
Tree-search methods conceptualize the problem as a huge tree of solutions, and then they try to do some smart things to avoid searching the entire tree.
Most popular IP (and MIP) solver today is a tree-search method called branch and bound. CPLEX uses this method, in combination with cutting planes.
We will study the branch and bound method.

4. Solving IPs: Branch & Bound

5. Solving IPs: Branch & Bound
Definitions
Predecessor problem
Successor problem
Problem Pj is predecessor to Problem Pk and
Problem Pk is successor to Problem Pj if:
they are identical with the exception
that one continuous-valued variable in Pj is constrained to be integer in Pk. Pj Pk

6. Example Solution (as an LP) using CPLEX yields
“Zeta”
Pose a problem P1 to be exactly like P0 except that we will constrain x1≤1.
Solution (as an LP) using CPLEX yields
What do you expect the value of x1 to be in the optimal solution?
Because the solution without the constraint x1≤1 wanted of x1, we can be sure that the solution with the constraint x1≤1 will want as much of x1 as it can get, i.e., it will want x1=1.
Solution (as an LP) using CPLEX yields
It worked: x1 did in fact become integer. In fact, x2 became integer as well, but this is by coincidence.

7. Example
Let’s consider the same problem but with (x1,x2) constrained to be integers.
x1<1
But what about P2? What is P2?
Is the P1 solution we obtained, which by chance is a feasible solution to IP1, also optimal to IP1?
P1 has to be as good as or better than its successors because these will be more heavily constrained, so we need not look at P1 successors.

8. Example P2 is the P0 problem with x1>2.
It is the remaining part of the space we need to search.
x1<1
x1>2
Use CPLEX to solve P2. (see solution on the tree). Observe P2 is infeasible!
(infeasible to the IP, that is, because x2 is non-integer).
So can we draw any useful conclusion from this observation?
1. The P2 solution, , is better than our best feasible solution so far, 65.0 (65.0 establishes a lower boundsolution is no less than 65).
YES!
But why?
2. Although P2 is infeasible, we can add constraints and find feasible solutions in successor problems.
3. We are not sure any of those successor problems will be “better than our best” (of 65), but because >65, we know it is worth trying.

9. Yes, Why? (In more detail…) Compare the objective function value of P2, , with the objective function value of P1, 65. Since we are maximizing, the objective function of P2 is better. But the P2 solution is not feasible. But we can constrain x2 so that we get a feasible solution. Whether that feasible solution will have better objective function value we do not know.
Example
x1<1
x1>2
What we do know is, because the objective function value of P2 ( ) is better than the objective function value of P1 (65), it is worthwhile to check it. Although the objective function value of successor problems to P2 can only get worse (lower), they might be better than P1, and if we can find a successor (or a successor’s successor,…) that is feasible, it might be better than our best current feasible solution, which is P1.

10. Example
x1<1
x1>2
Question: What if the P2 solution would have been 64?
Would you have searched its successor nodes?
NO! Why not?
Because successor nodes, even if feasible, would necessarily be more constrained than P2 and therefore no better than its solution. Since we already have a feasible solution of 65, and P2 successors could be no better than 64, there is no use searching them. Again, the value of 65 establishes a lower bound on the problem solution.

11. Example x1<1 x1>2 x2<2 x2>3
But because P2 solution is better than our current best feasible solution, we should pursue P2 successor problems.
Example
So what constraint should we add to P2?
x1<1
x1>2
x2<2
x2>3
Our choices are x2<2 and x2>3.
Let’s try x2<2, using P3.
The P3 solution is not feasible. Should we branch further?
YES!
1. The P3 solution, 68.2, is better than our best feasible solution so far, 65.0.
2. Although P2 is infeasible, we can add constraints and find feasible solutions in successor problems.
3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.2>65, we know it is worth trying.

12. Example x1>2 x1<1 x2>3 x2<2 x1<2 x1>3
So what constraint should we add to P3?
Example
Our choices are x1<2 and x1>3.
Let’s try x1<2, using P4.
x1<1
x1>2
x2>3
x2<2
x1<2
x1>3
Should we branch further?
No! Why?
Two reasons, either one of which is enough:
The P4 solution is feasible! And so we will not find another better feasible solution that is successor to P4.
The P4 objective is 58, worse than our best (65). So P4 and any further successor nodes are of no interest.

13. Example x1>2 x1<1 x2>3 x2<2 x1<2 x1>3
Now what? We have to decide on going back to P3 or P2. Choose P3.
Example
We impose x1>3, using P5.
x1<1
x1>2
x2>3
x2<2
x1<2
x1>3
Should we branch further?
Yes! Why?
1. The P5 solution, , is better than our best feasible solution so far, 65.0.
2. Although P5 is infeasible, we can add constraints and find feasible solutions in successor problems.
3. We are not sure any of those successor problems will be “better than our best” (of 65), but because >65, we know it is worth trying.

14. Example x1>2 x1<1 x2>3 x2<2 x1<2 x1>3
So what constraint should we add to P5?
Example
Our choices are x2<1 and x2>2.
Let’s try x2<1, using P6.
x1<1
x1>2
x2>3
x2<2
x1<2
x1>3
Should we branch further?
x2<1
x2>2
Yes! Why?
1. The P6 solution, 68.1, is better than our best feasible solution so far, 65.0.
2. Although P6 is infeasible, we can add constraints and find feasible solutions in successor problems.
3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.1>65, we know it is worth trying.

15. Example x1>2 x1<1 x2>3 x2<2 x1<2 x1>3
So what constraint should we add to P6?
Example
Our choices are x1<3 and x1>4.
Let’s try x1<3, using P7.
x1<1
x1>2
x2>3
x2<2
x1<2
x1>3
Should we branch further?
x2<1
x2>2
No! Why?
Two reasons, either one of which is enough:
The P7 solution is feasible! We will not find another better successor to P7
x1<3
x1>4
Now let’s try here.
P7 objective is 63, worse than our best (65), so P7’s successor nodes are of no interest.

16. Example x1>2 x1<1 x2>3 x2<2 x1<2
So now add the x1>4 constraint to P6, to obtain P8.
Example
x1<1
x1>2
x2>3
x2<2
x1<2
Should we branch further?
x1>3
No! Why?
The P8 solution is feasible! We will not find another better successor to P8
x2<1
x2>2
Note the P8 objective is 68, which is better than our best (65)! So P8 solution becomes our new best, i.e., it becomes our new lower bound on the solution. That is, the objective at the solution must be at least 68.
x1<3
x1>4

17. Example Question: Do we need to check the other branch to P5 and P2?
Answer: Yes! Why?
x1<1
x1>2
x2>3
Because the objective value for P5 and P2 at greater than our bound of 68, so a successor node could be better than 68 as well.
x2<2
x1<2
x1>3
x2<1
x2>2
x1<3
x1>4

18. Example x1>2 x1<1 x2>3 x2<2 x1<2 x1>3 x2<1
P9: infeasible
to the LP
x1<2
x1>3
x2<1
x2>2
P10: infeasible to the LP
x1<3
x1>4

19. Central ideas to branch & bound
Branch, force integrality on one variable by adding a constraint to an LP-relaxation:
Use LP-relaxation to decide how to branch. Each branch adds a constraint to previous LP-relaxation to enforce integrality on one variable that was not integer in the predecessor solution.
Bound, continue branching only if the objective function value of the current solution is better than the objective function value of the best feasible solution obtained so far:
Maintain the best feasible solution obtained so far as a bound on tree-paths that should still be searched.
If any tree node has an objective value less optimal than the identified bound, no further searching from that node is necessary, since adding constraints can never improve an objective.
If any tree node has an objective value more optimal than the identified bound, then additional searching from that node is necessary.

20. Using CPLEX to solve MIPS directly
1. Created CPLEX source within a text file called mip.lp as follows:
2. Used WinSCP to port the file to
server (linux-7.ece.iastate.edu).
4. Called cplex.
5. Typed read mip.lp to read in
problem statement.
6. Typed mipopt to call the MIP-solver. The result was as follows…
maximize
17 x x2
subject to
10 x1 + 7 x2 <= 40
x1 + x2 <= 5
Bounds
0<= x1 <= 1000
0<= x2 <= 1000
Integer
x1 x2
end

21. Using CPLEX to solve MIPS directly
1. Created CPLEX source within a text file called mip.lp as follows:
7. Typed
display solution variables - The result was:
maximize
17 x x2
subject to
10 x1 + 7 x2 <= 40
x1 + x2 <= 5
Bounds
0<= x1 <= 1000
0<= x2 <= 1000
Integer
x1 x2
end
Variable Name Solution Value x
All other variables in the range 1-2 are 0.

22. Depth vs breadth x1<1 x1>2 x2<2 x2>3
Recall this point in our procedure:
Depth vs breadth
The question can be posed like this:
Depth: Do we continue from P3, requiring x1≤2, for example? or
Breadth: Do we go back to P2 to examine its other branch, x2≥3?
x1<1
x1>2
x2<2
x2>3
For high-dimensional IPs, it is usually the case that feasible solutions are more likely to occur deep in a tree than at nodes near the root.
Finding multiple feasible solutions early in B&B is important because it tightens the bound (in the above example, it increases the bound), and therefore enables termination of branching at more nodes (and therefore decreases computation). One can see that this will be the case if we consider the bound before we find a feasible solution: the bound is infinite! (-∞ for maximization problems and +∞ for minimization problems).

23. Branching variable selection
For high-dimensional IPs, is there any benefit for selecting one non-integer variable over another when branching?
In our problem, except initially, there was never a decision to make in this way because there was never more than one non-integer variable.
Branching variable selection
x1<1
x1>2
x2>3
x2<2
P9: infeasible
x1<2
x1>3
A rich research question.
For specific problems, you can pre-specify an ordering of the variables required to be integer. Good orderings become apparent based on experience with running the algorithm or based on physical understanding, e.g., largest unit.
x2<1
x2>2
P10: infeasible
x1<3
x1>4

24. Mixed integer problems
Our example was a pure integer problem. What about a MIP?
Mixed integer problems
x1<1
x1>2
x2>3
x2<2
P9: infeasible
The solution to IP2 is obtained as soon as we solve P1 and P2.
x1<2
x1>3
x2<1
Conclusion: We can easily solve MIP within our LP-relaxation branch and bound scheme by simply allowing the non-integer variables to remain relaxed.
x2>2
P10: infeasible
x1<3
x1>4