Tuesday, March 5 Duality – The art of obtaining bounds – weak and strong duality Handouts: Lecture Notes

2 Bounds One of the great contributions of optimization theory (and math programming) is the providing of upper bounds for maximization problems We can prove that solutions are optimal For other problems, we can bound the distance from optimality

3 A 4-variable linear program David has minerals that he will mix together and sell for profit. The minerals all contain some gold content, and he wants to ensure that the mixture has 3% gold, and each bag will weigh 1 kilogram. Mineral 1: 2% gold, $3 profit/kilo Mineral 2: 3% gold, $4 profit/kilo Mineral 3: 4% gold, $6 profit/kilo Mineral 4: 5% gold, $8 profit/kilo maximize z = 3x 1 + 4x 2 +6x 3 + 8x 4 subject to x 1 + x 2 + x 3 + x 4 = 1 2x 1 + 3x 2 +4x 3 + 5x 4 = 3 x 1, x 2, x 3, x 4 0

4 A 4-variable linear program maximize z = 3x 1 + 4x 2 +6x 3 + 8x 4 subject to x 1 + x 2 + x 3 + x 4 = 1 2x 1 + 3x 2 +4x 3 + 5x 4 = 3 x 1, x 2, x 3, x 4 0

5 Obtaining a Bound Subtract 8 times constraint 1 from the objective function. -z – 5 x 1 – 4 x 2 – 2 x 3 = -8 z + 5 x 1 + 4x x 3 = 8 Does this show that z 8? YES!

6 Obtaining a Second Bound: Treat the operation as pricing out Subtract 3 * constraint 1 and subtract constraint 2 from the objective function. -z – 2x 1 – 2 x 2 – 1x 3 = -6 z + 2 x 1 + 2x x 3 = 6 Thus z 6! Which bound is better: 6 or 8? Prices

7 Obtaining the Best Bound: Formulate the problem as an LP Prices A: 3 - y 1 - 2y 2 0 y 1 + 2y 2 3 B: 4 - y 1 - 3y 2 0 y 1 + 3y 2 4 C: 6 - y 1 - 4y 2 0 y 1 + 4y 2 6 D: 8 - y 1 - 5y 2 0 y 1 + 5y 2 8 minimize y 1 + 3y 2

8 The problem that we formed is called the dual problem minimize y 1 + 3y 2 Subject to y 1 + 2y 2 y 1 + 3y 2 y 1 + 5y 2 y 1 + 4y 2 y 1 a nd y 2 are unconstrained in sign

9 Summary of previous slides If reduced costs are non-positive then we have an upper bound on the objective value The problem of finding the least upper bound is a linear program, and is referred to as the dual of the original linear program. To do: express duality in general notation. To do: show that the shadow prices solve the dual, and the bound is the optimal solution to the original problem

10 PRIMAL PROBLEM: maximize subject to z = 3x 1 + 4x 2 +6x 3 + 8x 4 x 1 + x 2 + x 3 + x 4 = 1 2x 1 + 3x 2 +4x 3 + 5x 4 = 3 x 1, x 2, x 3, x 4 0 maximize y 1 + 3y 2 subject to y 1 + 2y 2 y 1 + 3y 2 y 1 + 4y 2 y 1 + 5y 2 Observation 1. The constraint matrix in the primal is the transpose of the constraint matrix in the dual. Observation 2. The RHS coefficients in the primal become the cost coefficients in the dual. DUSL PROBLEM:

11 PRIMAL PROBLEM: maximize subject to z = 3x 1 + 4x 2 +6x 3 + 8x 4 x 1 + x 2 + x 3 + x 4 = 1 2x 1 + 3x 2 +4x 3 + 5x 4 = 3 x 1, x 2, x 3, x 4 0 DUSL PROBLEM: maximize y 1 + 3y 2 subject to y 1 + 2y 2 y 1 + 3y 2 y 1 + 4y 2 y 1 + 5y 2 Observation 3. The cost coefficients in the primal become the RHS coefficients in the dual. Observation 4. The primal (in this case) is a max problem with equality constraints and non-negative variables The dual (in this case) is a minimization problem with constraints and variables unconstrained in sign.

12 PRIMAL PROBLEM (in standard form): max z = c 1 x 1 + c 2 x 2 + c 3 x 3 + … + c n x n s.t. a 11 x 1 + a 12 x 2 + a 13 x 3 + … + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 + … + a 2n x n = b 2 … a m1 x 1 + a m2 x 2 + a m3 x 3 + … + a mn x n = bm x j 0 for j = 1 to n. DUAL PROBLEM: max v = ??? s.t. ??? What is the dual problem in terms of the notation given above?

13 PRIMAL PROBLEM (in standard form): max z = c 1 x 1 + c 2 x 2 + c 3 x 3 + … + c n x n s.t. a 11 x 1 + a 12 x 2 + a 13 x 3 + … + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 + … + a 2n x n = b 2 … a m1 x 1 + a m2 x 2 + a m3 x 3 + … + a mn x n = b m x j 0 for j = 1 to n. DUAL PROBLEM: max v = b 1 y 1 + b 2 y 2 + b 3 y 3 + … +b n y n s.t. a 11 y 1 + a 21 y 2 + a 31 y 3 + … + a m1 y m c 1 a 12 y 1 + a 22 y 2 + a 32 y 3 + … + a m2 y m c 2 … a 1n y 1 + a 2n y 2 + a 3n y 3 + … + a mn y m c n

14 Theorem. Suppose that x is any feasible solution to the primal, and y is any feasible solution to the dual. Then j=1..n c j x j i=1..n y i b i (Max Min) Proof. j=1..n c j x j i=1...n j=1...m (y i a ij ) x i j=1...m i=1...n y i (a ij x j ) j=1...m y i b i Weak Duality Theorem

15 Unboundedness Property Theorem. Suppose that the primal (dual) problem has an unbounded solution. Then the dual (primal) problem has no feasible solution. Proof. Suppose that y was feasible for the dual. Then every solution to the primal problem is unbounded above by j=1...m y i b i

16 Strong Duality Theorem Theorem. If the primal problem has a finite optimal solution value, then so does the dual problem, and these two values are the same.

17 Strong Duality Illustrated Prices Optimal dual solution: y 1 = –1/3, y 2 = 5/3, v = 14/3 Optimal primal solution: x 1 = 2/3, x 2 = 0, x 3 = 0, x 4 = 1/3 z = 14/3

18 Strong Duality Illustrated: the final tableau Optimal primal solution: x 1 = 2/3, x 2 = 0, x 3 = 0, x 4 = 1/3 v = 14/3 Observation The cost coeffients satisfy complementary slackness

19 Summary If we take an LP in standard form (max), we can formulate a dual problem Weak Duality: Each solution of the dual gives an upper bound on the maximum objective value for the primal Strong duality: if there are feasible solutions to the primal and dual, then the optimal objective value for both problems is the same.

20 Shadow prices solve the dual! Theorem. Suppose that the primal problem has a finite optimal solution value. Then the shadow prices for the primal problem form an optimal solution to the dual problem. (And the objective values are the same.) Davids Mineral Problem

21 FACT: Optimal simplex multipliers are shadow prices. Shadow prices

22 Summary The dual problem to a maximizing LP provides upper bounds on the optimal objective function The maximum solution value for the primal is the same as the minimum solution value for the dual The shadow prices are optimal for the dual LP Next: alternative optimality conditions

23 Primal Problem max Σj=1..m cj xj s.t. Σj=1..n aij xj = bi for i = 1 to m xj 0 for all j = 1 to n Dual Problem min j=1..m yi bi s.t. i=1..m yi aij cj for j = 1 to n Optimality Condition 1. A solution x* is optimal for the primal problem if it is a basic feasible solution and the tableau satisfies the optimality conditions. Optimality Condition 2. A solution x* is optimal for the primal problem if it is feasible and if there is a feasible solution y* for the dual with j=1..m cj x*j = j=1..m y*i bi

24 Primal Problem max Σj=1..m cj xj s.t. Σj=1..n aij xj = bi for i = 1 to m xj 0 for all j = 1 to n Dual Problem min Σj=1..m yi bi s.t. Σi=1..m yi aij cj for j = 1 to n Complementary Slackness Conditions. – Suppose that y is feasible for the dual, and let cj= cj - Σi=1..myi aij. Suppose x is feasible for the primal. Theorem (complementary slackness). x and y are optimal for the primal and dual if and only if cj xj = 0 for all j.

25 Complementary Slackness Illustrated prices

26 Next Lecture on Duality Dual linear programs in general form Optimality Conditions in general form. Illustrating Duality with 2-person 0-sum game theory

27 Coverage for first midterm (Closed book exam) Formulations 2D graphing and finding opt. solution Setting up an LP in standard form The simplex algorithm starting with a bfs Phase 1 of the simplex algorithm Interpreting sensitivity analysis, including shadow prices, and ranges, and reduced costs Pricing out Using tableaus to determine shadow prices, reduced costs, and ranges