1. EMGT 6412/MATH 6665 Mathematical Programming Spring 2016
Decomposition Principle and Column Generation
Dincer Konur
Engineering Management and Systems Engineering

2. Outline Decomposition Principle Column Generation Chapter 7
Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
Column Generation
Cutting Stock Problem
Pricing Problem Reformulation
Column Generation Scheme
Chapter 7

3. Outline Decomposition Principle Column Generation Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
Column Generation
Cutting Stock Problem
Pricing Problem Reformulation
Column Generation Scheme

4. Decomposition Principle
Linear programming models can have too many columns (decision variables) or rows (constraints)
Or we might have special structures in the model that can help us to solve the model more efficiently
The idea of decomposition is to partition the model into parts with special structures so that a more efficient solution method can be developed

5. Decomposition Principle
Duality and reduced costs:
Consider the following model:
B is optimum for P if
Recall
Let then means
So a basic feasible solution B is optimum if the associated dual solution is feasible

6. Decomposition Principle
Consider the following LP model:
Since X is a bounded polyhedral set, then any point x in X can be represented as a convex combination of the finite number of extreme points of X
Let be the extreme points of X

7. Decomposition Principle
Substituting x into the model, we have:
Master Problem
Since t, the number of extreme points of the set X, is usually very large, attempting to explicitly enumerate all the extreme points and explicitly solving this problem is an onerous task

8. Decomposition Principle
In variables we have master problem:
B is optimum if

9. Decomposition Principle
Recall the optimality conditions:
B is optimum if
and

10. Decomposition Principle
Recall the optimality conditions:
B is optimum if
This problem is feasible

11. Decomposition Principle
Solving ?
t is very large still and enumerating all extreme points xj corresponding to is hard
will be at an extreme point of X
X is bounded and not empty

12. Decomposition Principle
If the optimum objective function value is =0, then the current basic feasible solution is optimum
If the optimum objective function value is >0, then the current basic feasible solution is not optimum
Then enters the basis

13. Decomposition Algorithm
Initialization: Determine a feasible subset of the extreme points of X, say x1,x2,…,xr such that r<t
Restricted Master Problem:
Solve the restricted master problem and let w and α be the optimum dual variables associated with 7.1 and 7.2 r r r r

14. Decomposition Algorithm
Sub-problem:
Solve the sub-problem (pricing problem, column generator) and let xr+1 be its solution
If the objective function value is <= 0, stop, current solution to the restricted master problem is optimum to the original problem
If the objective function value is >0, add column to the restricted master problem and extend it

15. Decomposition Algorithm
Column to be added: xr+1
Calculate cxr+1 and Axr+1
Extended Restricted Master Problem:
Solve the new restricted master problem and let w and α be the optimum dual variables associated with 7.1 and 7.2
r+1
r+1
r+1
r+1

16. Decomposition Algorithm Example

17. Decomposition Algorithm Example
The master problem:
Initialization:
Ax1 = [0,0]t
cx1 = 0

18. Decomposition Algorithm Example
Restricted Master Problem:
optimum solution
0λ1
(0,0)tλ1 λ1
optimum dual λ1

19. Decomposition Algorithm Example
First pricing problem:
optimum solution
Extend master problem by adding x2

20. Decomposition Algorithm Example
Extending Restricted Master Problem:
cx2 = -17/2
cx1 = 0
c=[-2,-1,-1,1]
Ax2 = [5,7/2]
Ax1 = [0,0]t
optimum solution
0*λ1-17/2*λ2
0*λ *λ2 +s1=2
0*λ1 +7/2*λ2 +s2=3
λ λ =1
λ1>=0, λ2 >=0
optimum dual

21. Decomposition Algorithm Example
Second pricing problem:
optimum solution
Extend master problem by adding x3

22. Decomposition Algorithm Example
Extending Restricted Master Problem:
cx3 = -5/2
cx2 = -17/2
cx1 = 0
Ax3 = [0,5/2]
Ax2 = [5,7/2]
Ax1 = [0,0]t
c=[-2,-1,-1,1]
optimum solution
0*λ1-17/2*λ2 -5/2*λ3
0*λ *λ s1=2
0*λ1 +7/2*λ2 + 5/2* λ3 +s2=3
λ λ λ =1
λ1>=0, λ2 >=0, λ3 >=0
optimum dual

23. Decomposition Algorithm Example
Third pricing problem:
optimum solution
Extend master problem by adding x4

24. Decomposition Algorithm Example
Extending Restricted Master Problem:
cx4 = -11/2
cx3 = -5/2
cx2 = -17/2
cx1 = 0
Ax4 = [2,7/2]
Ax3 = [0,5/2]
Ax2 = [5,7/2]
Ax1 = [0,0]t
c=[-2,-1,-1,1]
optimum solution
0*λ1-17/2*λ2 -5/2*λ3-11/2*λ4
0*λ *λ *λ s1=2
0*λ1 +7/2*λ2 + 5/2* λ3 +7/2*λ4 +s2=3
λ λ λ λ4 =1
λ1>=0, λ2 >=0, λ3 >=0, λ4 >=0
optimum dual

25. Decomposition Algorithm Example
Fourth pricing problem:
optimum solution
Current solution is optimum!! Finally!!!!

26. Decomposition Algorithm Example
The optimum solution:
x*= =
Columns generated:
Ax4 = [2,7/2] t
Ax3 = [0,5/2] t
Ax2 = [5,7/2] t
Ax1 = [0,0]t
cx4 = -11/2
cx3 = -5/2
cx2 = -17/2
cx1 = 0

27. Outline Decomposition Principle Column Generation Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
Column Generation
Cutting Stock Problem
Pricing Problem Reformulation
Column Generation Scheme

28. Column Generation
When a linear programming problem has too many decision variables (columns), it is impractical to solve the large scale model
Sometimes, one might need to calculate the cost for each column, which might be another optimization problem (even not, it requires computational time)
Most of the times, majority of the columns will not be in the optimum basis
Column generation aims at solving linear programming model without a need to enumerate all of the columns explicitly

29. Cutting Stock Problem Recall the cutting stock problem:
Let a standard metal sheet be of length
Suppose that the demand for metal sheets of length is equal to
A standard metal sheet can be cut in different patterns to satisfy the demand
Let j denote a pattern and a pattern can be defined by the number of metal sheets of length in pattern j

30. Cutting Stock Problem For instance
For instance, when you cut one standard metal sheet in Pattern 3, it will produce 2 metal sheets of 5in by 10in and 1 metal sheet of 15in by 10in. So
a13=2, a23=0, a33=1

31. Cutting Stock Problem Cutting stock problem is then:
Assuming you know all of the patterns, j=1,…,m
But m is very very large…
is the average number of cut sizes
in each of the patterns

32. Cutting Stock Problem
Solving the linear relaxation is important, it provides bounds, and can be used in branch-and-bound
Linear relaxation:
Still large scale so not easy to solve!
Master
Problem (MP)

33. Cutting Stock Problem
Let’s consider only a subset of the patterns, j=1,…,k where k<<<m
Initially, you can consider n patterns where each pattern only consists of
Restricted
Master
Problem (RMP)

34. Cutting Stock Problem RMP is easy to solve Not large scale
If we can show that there is no other pattern that can improve the current objective function value, the solution to the RMP is optimum for MP
Recall that a solution is optimum if for all patterns.. So if solution of
is <=0, then we have the optimum solution…
Pricing Problem

35. Cutting Stock Problem
Let be the optimum dual values for the solution of the current RMP
Ideally, we want to find the pattern with the most negative reduced cost
Still too many variables…m is large

36. Cutting Stock Problem A pattern is defined by
So, we need that will solve the pricing problem
Let be the number of parts of length cut from the standard metal sheet
Then a pattern is feasible as long as

37. Cutting Stock Problem Reformulating the pricing problem:
This one has n variables
But they are integer
This is an integer-knapsack problem
Dynamic solution in pseudo-polynomial time
Pricing Problem

38. Cutting Stock Problem
Let the optimum solution to pricing problem be given by
If , then the solution of the current RMP is optimum for MP
Else, we generated a new pattern
Let
Extend the RMP
Restricted
Master
Problem (RMP)

39. Column Generation Suppose you have a large scale LP (MP)
Consider only a subset of the columns and formulate the Restricted Master Problem
Solve the Restricted Master Problem (RMP)
Solve the pricing problem
If the negative reduced cost, stop, solution of the current RMP is optimum for the MP
Else, extend the current RMP by including the new column

40. Column Generation Notes about column generation:
Pricing problem should be relatively easy to solve
It should be solved over and over again
Knapsack problem, shortest-path problem..
Pricing problem is not always needed to be solved to optimality
It is ok to determine a column with negative reduced cost
Use heuristic methods first to find a good column,
If cannot find a good column, then solve the pricing problem exactly
It is ok to add multiple columns simultanesouly

41. Next time
Integer programming