1. EMGT 6412/MATH 6665 Mathematical Programming Spring 2016
Duality
Dincer Konur
Engineering Management and Systems Engineering

2. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation
Chapter 6

3. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

4. Dual Formulation For every LP we solve
There is another associated LP being solved
It has important relationships and implications for the original LP
Original LP is the “primal” problem
Associated LP is the “dual” problem

5. Formulation Dual formulation in canonical form: 𝒄: 1𝑥𝑛 vector
𝑨: 𝑚𝑥𝑛 matrix
𝒃: 𝑚𝑥1 vector
n decision variables
m constraints (excluding )
𝒘: 1𝑥𝑚 vector
𝒃: 𝑚𝑥1 vector
𝒄: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
m decision variables
n constraints (excluding )

6. Dual Formulation
Example 6.1:

7. Dual Formulation Dual formulation in standard form: 𝒄: 1𝑥𝑛 vector
𝑨: 𝑚𝑥𝑛 matrix
𝒃: 𝑚𝑥1 vector
n decision variables
m constraints (excluding )
𝒘: 1𝑥𝑚 vector
𝒃: 𝑚𝑥1 vector
𝒄: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
m decision variables
n constraints

8. Dual Formulation
Example 6.2:

9. Dual Formulation
Standard or canonical, they are the same:

10. Dual Formulation
Dual of the dual is primal

11. Dual Formulation
Mixed form of formats:

12. Dual Formulation
Example 6.3:

13. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

14. Weak Duality Consider the duality in canonical form:
Feasible solutions
Multiply with w0 from left
Multiply with xo from right

15. Weak Duality Illustration: Since this primal is a maximization problem
Weak duality: If x is a feasible solution for the primal problem and y is a feasible solution for the dual problem, then cx ≤ yb.

16. Weak Duality Illustration: <= <=
Suppose we have a primal solution (X1, X2)
Suppose we have a dual solution (y1,y2,y3) So
y1x1+2y2x2+y3(3x1+2x2)<=
x1(y1+3y3)+x2(2y2+2y3)>=
<=
<=

17. Weak Duality Implications of weak duality:
The objective function value for any feasible solution to the primal minimization problem is always greater than or equal to the objective function value for any feasible solution to the dual maximization problem.
the objective value of any feasible solution of the primal minimization problem gives an upper bound on the optimal objective of the dual maximization problem.
Similarly, the objective value of any feasible solution of the dual maximization problem gives a lower bound on the optimal objective of the primal minimization problem.

18. Weak Duality Implications of weak duality:
If and are feasible solutions to the primal and dual problems, respectively, such that then they are optimal solutions to their respective problems.
If either problem has an unbounded objective value, then the other problem possesses no feasible solution.
Unboundedness in one problem implies infeasibility in the other problem.
Infeasibility in one problem DOES NOT necessarily imply unboundedness in the other.
See Example 6.4

19. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

20. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions:
Farka’s Lemma:

21. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions:
Farka’s Lemma alternatively says:

22. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions:
Farka’s Lemma Part 1:

23. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions:
Farka’s Lemma Part 2:

24. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions:
Farka’s Lemma Part 2:

25. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions (necessary and sufficient):
Farka’s lemma

26. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions (necessary and sufficient):

27. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions (necessary and sufficient):

28. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions (necessary and sufficient):

29. KKT Conditions and Duality
About the relation with KKT optimality conditions:
KKT Optimality Conditions (necessary and sufficient):

30. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

31. Strong Duality About the relation with KKT optimality conditions:
From KKT conditions
From weak duality
If one problem possesses an optimal solution, then both problems possess optimal solutions and the two optimal objective values are equal

32. Strong Duality Implications of strong duality:
If P is unbounded, D is infeasible
If D is unbounded, P is infeasible
If P is infeasible
D can be unbounded? Possible!
D can have optimum solution? No!!
D can be infeasible? Possible!
From strong duality

33. Strong Duality
Fundamental theory of duality: I O U
Yes

34. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

35. Complementary Slackness
If and only if

36. Complementary Slackness

37. Complementary Slackness
At optimality:
If a variable in one problem is positive, then the corresponding constraint in the other problem must be tight.
If a constraint in one problem is not tight, then the corresponding variable in the other problem must be zero.

38. Complementary Slackness
Using complementary slackness to solve primal
Example 6.5 from the book
Since the dual has only two variables, we may solve it graphically

39. Complementary Slackness
Using complementary slackness to solve primal
Example 6.5 from the book
Obj. value=5
From fundamental duality theorem, we know that there is an optimal solution to the primal with the same objective function value  =
< < <
= >=

40. Complementary Slackness
Another example:
x1*=2 and x2*=6.
Let’s get the optimum dual solution
The objective function values should be the same
3x1*+5x2*=4y1*+12y2*+18y3*  36=4y1*+12y2*+18y3*
If a constraint is not ‘tight’ then the associated variable is 0
x1*<=4  2<4  not tight  so y1*=0
2x2*<=12  12=12  tight  so nothing about y2* so far
3x1*+2x2*<=18  18=18  tight  so nothing about y3* so far
If a variable is not zero then the associated constraint is tight
x1*=2>0  so first constraint  y1*+3y3*=3
x2*=6>0  so second constraint  2y2*+2y3* =5
Optimum solution to primal is x1*=2 and x2*=6.

41. Complementary Slackness
Another example:
So we have three unknowns three equations
 4y1*+12y2*+18y3*=36
 y1*+3y3*=3
 2y2*+2y3* =5
Furthermore, we already know that
 y1*=0
So, we can solve for y1*, y2*, y3*

42. Complementary Slackness
Prove that there exists an optimum solution to the following LP model such that only one of the variables is non-negative
Minimize 𝑖=1 𝑛 𝑎 𝑖 𝑥 𝑖
subject to 𝑖=1 𝑛 𝑏 𝑖 𝑥 𝑖 ≥𝐵
where ai>=0, bi>=0, and B>=0
𝑥 𝑖 ≥0 𝑖=1,2,…,𝑛

43. Outline Dual Formulation Primal-Dual Relationships
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
Economic Interpretation

44. Economic Interpretation
Consider

45. Economic Interpretation
Consider
To illustrate, if the ith constraint represents a demand for production of at least bi units of the ith product and ex represents the total cost of production, then wi* is the incremental cost of producing one more unit of the ith product.

46. Economic Interpretation

47. Economic Interpretation
Instead of trying to control the operation of the firm to obtain the most desirable mix of activities, suppose that we agree to pay the firm unit prices for each of the m outputs. We also stipulate that these prices announced by the firm must be fair.
Since is the number of units of output i produced by one unit of activity j and is the unit price of output i, then can be interpreted as the unit price of activity j consistent with the prices
Therefore, we tell the firm that the implicit price of activity j , namely should not exceed the actual cost cj.
Within these constraints, the firm would like to choose a set of prices that maximizes its return

48. Economic Interpretation
Fundamental duality theory states that, given optimum exists, minimum production cost of the buyer is equal to the maximum revenue of the seller
Complementary slackness states:
If the optimal level of activities that meets all demand requirements automatically produces an excess of product i, then the incremental cost associated with marginally increasing bi is naturally zero.
If the total revenue generated via the items produced by a unit level of activity j is less than the associated production cost, then the level of activity j should be zero at optimality.

49. Economic Interpretation
Consider an example:
Wyndor Glass Co. produces windows and glass doors
Plant 1 makes aluminum frames and hardware
Plant 2 makes wood frames
Plant 3 produces glass and assembles products
Company introducing two new products
Product 1: 8 ft. glass door with aluminum frame
Product 2: 4 x 6 ft. double-hung, wood-framed window
Problem: What mix of products would be most profitable?
Assuming company could sell as much of either product as could be produced

50. Economic Interpretation
The primal problem is:
Objective function of the dual:
biyi can thereby be interpreted as the current contribution to profit by having bi units of resource i available for the primal problem.

51. Economic Interpretation
Shadow price: Given an optimal solution and the corresponding value of the objective function for a linear programming model, the shadow price for a functional constraint is the rate at which the value of the objective function changes by 1 unit change on the right-hand-side
The dual variable yi is interpreted as the contribution to profit per unit of resource i, when the current set of basic variables is used to obtain the primal solution.
So yi are shadow prices

52. Economic Interpretation
Consider the Wyndor problem…
If we produce 1 unit of doors, we will consume 1 hour from Plant 1 and 3 hours from Plant 3
So we could have gained y1+3y3 to the profit if we have not produced 1 unit of doors
Also recall that 1 unit of doors has a profit of c1=3
So if the contribution of the resources used by 1 door is greater than the profit we will get from 1 door, we would not produce,
Therefore, we have:
If y1+3y3 >3  x1=0
Recall from complementary slackness, if the constraint is not tight, then the associated variable should be 0

53. Economic Interpretation
Now let’s look at the shadow prices
Let’s consider resource 2
If y2>0, this means that having additional one unit of resource 2 (i.e., 1 more hour in Plant 2) will increase our profit
If we are at the optimum solution, then we should have used all of the hours in Plant 2, otherwise, we would not be in the optimum solution as using some more of the available time in Plant 2, we could increase our profit..
Therefore, if y2>0, we should currently be using all the time in plant 2, that is, we should have 2x2=16
Recall from complementary slackness, if a variable is not zero, then the associated constraint should be tight

54. Next time So far, we have learned the basics of linear programming…
How can we use these to solve large scale problems?
How can we use these to solver integer/mixed-integer models?
Decomposition principle
Column generation
Branch and bound
Cutting Planes
Branch and cut
Branch and price
Bender’s decomposition