EMGT 6412/MATH 6665 Mathematical Programming Spring 2016 Decomposition Principle and Column Generation Dincer Konur Engineering Management and Systems Engineering

Outline Decomposition Principle Column Generation Chapter 7 Master Problem Optimality of Restricted Master Problem Sub-problem Decomposition Example Column Generation Cutting Stock Problem Pricing Problem Reformulation Column Generation Scheme Chapter 7

Outline Decomposition Principle Column Generation Master Problem Optimality of Restricted Master Problem Sub-problem Decomposition Example Column Generation Cutting Stock Problem Pricing Problem Reformulation Column Generation Scheme

Decomposition Principle Linear programming models can have too many columns (decision variables) or rows (constraints) Or we might have special structures in the model that can help us to solve the model more efficiently The idea of decomposition is to partition the model into parts with special structures so that a more efficient solution method can be developed

Decomposition Principle Duality and reduced costs: Consider the following model: B is optimum for P if Recall Let then means So a basic feasible solution B is optimum if the associated dual solution is feasible

Decomposition Principle Consider the following LP model: Since X is a bounded polyhedral set, then any point x in X can be represented as a convex combination of the finite number of extreme points of X Let be the extreme points of X

Decomposition Principle Substituting x into the model, we have: Master Problem Since t, the number of extreme points of the set X, is usually very large, attempting to explicitly enumerate all the extreme points and explicitly solving this problem is an onerous task

Decomposition Principle In variables we have master problem: B is optimum if

Decomposition Principle Recall the optimality conditions: B is optimum if and

Decomposition Principle Recall the optimality conditions: B is optimum if This problem is feasible

Decomposition Principle Solving ? t is very large still and enumerating all extreme points xj corresponding to is hard will be at an extreme point of X X is bounded and not empty

Decomposition Principle If the optimum objective function value is =0, then the current basic feasible solution is optimum If the optimum objective function value is >0, then the current basic feasible solution is not optimum Then enters the basis

Decomposition Algorithm Initialization: Determine a feasible subset of the extreme points of X, say x1,x2,…,xr such that r<t Restricted Master Problem: Solve the restricted master problem and let w and α be the optimum dual variables associated with 7.1 and 7.2 r r r r

Decomposition Algorithm Sub-problem: Solve the sub-problem (pricing problem, column generator) and let xr+1 be its solution If the objective function value is <= 0, stop, current solution to the restricted master problem is optimum to the original problem If the objective function value is >0, add column to the restricted master problem and extend it

Decomposition Algorithm Column to be added: xr+1 Calculate cxr+1 and Axr+1 Extended Restricted Master Problem: Solve the new restricted master problem and let w and α be the optimum dual variables associated with 7.1 and 7.2 r+1 r+1 r+1 r+1

Decomposition Algorithm Example

Decomposition Algorithm Example The master problem: Initialization: Ax1 = [0,0]t cx1 = 0

Decomposition Algorithm Example Restricted Master Problem: optimum solution 0λ1 (0,0)tλ1 λ1 optimum dual λ1

Decomposition Algorithm Example First pricing problem: optimum solution Extend master problem by adding x2

Decomposition Algorithm Example Extending Restricted Master Problem: cx2 = -17/2 cx1 = 0 c=[-2,-1,-1,1] Ax2 = [5,7/2] Ax1 = [0,0]t optimum solution 0*λ1-17/2*λ2 0*λ1 + 5*λ2 +s1=2 0*λ1 +7/2*λ2 +s2=3 λ1 + λ2 =1 λ1>=0, λ2 >=0 optimum dual

Decomposition Algorithm Example Second pricing problem: optimum solution Extend master problem by adding x3

Decomposition Algorithm Example Extending Restricted Master Problem: cx3 = -5/2 cx2 = -17/2 cx1 = 0 Ax3 = [0,5/2] Ax2 = [5,7/2] Ax1 = [0,0]t c=[-2,-1,-1,1] optimum solution 0*λ1-17/2*λ2 -5/2*λ3 0*λ1 + 5*λ2 +s1=2 0*λ1 +7/2*λ2 + 5/2* λ3 +s2=3 λ1 + λ2 + λ3 =1 λ1>=0, λ2 >=0, λ3 >=0 optimum dual

Decomposition Algorithm Example Third pricing problem: optimum solution Extend master problem by adding x4

Decomposition Algorithm Example Extending Restricted Master Problem: cx4 = -11/2 cx3 = -5/2 cx2 = -17/2 cx1 = 0 Ax4 = [2,7/2] Ax3 = [0,5/2] Ax2 = [5,7/2] Ax1 = [0,0]t c=[-2,-1,-1,1] optimum solution 0*λ1-17/2*λ2 -5/2*λ3-11/2*λ4 0*λ1 + 5*λ2 +2*λ4 +s1=2 0*λ1 +7/2*λ2 + 5/2* λ3 +7/2*λ4 +s2=3 λ1 + λ2 + λ3 + λ4 =1 λ1>=0, λ2 >=0, λ3 >=0, λ4 >=0 optimum dual

Decomposition Algorithm Example Fourth pricing problem: optimum solution Current solution is optimum!! Finally!!!!

Decomposition Algorithm Example The optimum solution: x*= = Columns generated: Ax4 = [2,7/2] t Ax3 = [0,5/2] t Ax2 = [5,7/2] t Ax1 = [0,0]t cx4 = -11/2 cx3 = -5/2 cx2 = -17/2 cx1 = 0

Outline Decomposition Principle Column Generation Master Problem Optimality of Restricted Master Problem Sub-problem Decomposition Example Column Generation Cutting Stock Problem Pricing Problem Reformulation Column Generation Scheme

Column Generation When a linear programming problem has too many decision variables (columns), it is impractical to solve the large scale model Sometimes, one might need to calculate the cost for each column, which might be another optimization problem (even not, it requires computational time) Most of the times, majority of the columns will not be in the optimum basis Column generation aims at solving linear programming model without a need to enumerate all of the columns explicitly

Cutting Stock Problem Recall the cutting stock problem: Let a standard metal sheet be of length Suppose that the demand for metal sheets of length is equal to A standard metal sheet can be cut in different patterns to satisfy the demand Let j denote a pattern and a pattern can be defined by the number of metal sheets of length in pattern j

Cutting Stock Problem For instance For instance, when you cut one standard metal sheet in Pattern 3, it will produce 2 metal sheets of 5in by 10in and 1 metal sheet of 15in by 10in. So a13=2, a23=0, a33=1

Cutting Stock Problem Cutting stock problem is then: Assuming you know all of the patterns, j=1,…,m But m is very very large… is the average number of cut sizes in each of the patterns

Cutting Stock Problem Solving the linear relaxation is important, it provides bounds, and can be used in branch-and-bound Linear relaxation: Still large scale so not easy to solve! Master Problem (MP)

Cutting Stock Problem Let’s consider only a subset of the patterns, j=1,…,k where k<<<m Initially, you can consider n patterns where each pattern only consists of Restricted Master Problem (RMP)

Cutting Stock Problem RMP is easy to solve Not large scale If we can show that there is no other pattern that can improve the current objective function value, the solution to the RMP is optimum for MP Recall that a solution is optimum if for all patterns.. So if solution of is <=0, then we have the optimum solution… Pricing Problem

Cutting Stock Problem Let be the optimum dual values for the solution of the current RMP Ideally, we want to find the pattern with the most negative reduced cost Still too many variables…m is large

Cutting Stock Problem A pattern is defined by So, we need that will solve the pricing problem Let be the number of parts of length cut from the standard metal sheet Then a pattern is feasible as long as

Cutting Stock Problem Reformulating the pricing problem: This one has n variables But they are integer This is an integer-knapsack problem Dynamic solution in pseudo-polynomial time Pricing Problem

Cutting Stock Problem Let the optimum solution to pricing problem be given by If , then the solution of the current RMP is optimum for MP Else, we generated a new pattern Let Extend the RMP Restricted Master Problem (RMP)

Column Generation Suppose you have a large scale LP (MP) Consider only a subset of the columns and formulate the Restricted Master Problem Solve the Restricted Master Problem (RMP) Solve the pricing problem If the negative reduced cost, stop, solution of the current RMP is optimum for the MP Else, extend the current RMP by including the new column

Column Generation Notes about column generation: Pricing problem should be relatively easy to solve It should be solved over and over again Knapsack problem, shortest-path problem.. Pricing problem is not always needed to be solved to optimality It is ok to determine a column with negative reduced cost Use heuristic methods first to find a good column, If cannot find a good column, then solve the pricing problem exactly It is ok to add multiple columns simultanesouly

Next time Integer programming