EMGT 6412/MATH 6665 Mathematical Programming Spring 2016 Duality Dincer Konur Engineering Management and Systems Engineering

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation Chapter 6

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

Dual Formulation For every LP we solve There is another associated LP being solved It has important relationships and implications for the original LP Original LP is the “primal” problem Associated LP is the “dual” problem

Formulation Dual formulation in canonical form: 𝒄: 1𝑥𝑛 vector 𝑨: 𝑚𝑥𝑛 matrix 𝒃: 𝑚𝑥1 vector n decision variables m constraints (excluding ) 𝒘: 1𝑥𝑚 vector 𝒃: 𝑚𝑥1 vector 𝒄: 𝑛𝑥1 vector 𝑨: 𝑚𝑥𝑛 matrix m decision variables n constraints (excluding )

Dual Formulation Example 6.1:

Dual Formulation Dual formulation in standard form: 𝒄: 1𝑥𝑛 vector 𝑨: 𝑚𝑥𝑛 matrix 𝒃: 𝑚𝑥1 vector n decision variables m constraints (excluding ) 𝒘: 1𝑥𝑚 vector 𝒃: 𝑚𝑥1 vector 𝒄: 𝑛𝑥1 vector 𝑨: 𝑚𝑥𝑛 matrix m decision variables n constraints

Dual Formulation Example 6.2:

Dual Formulation Standard or canonical, they are the same:

Dual Formulation Dual of the dual is primal

Dual Formulation Mixed form of formats:

Dual Formulation Example 6.3:

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

Weak Duality Consider the duality in canonical form: Feasible solutions Multiply with w0 from left Multiply with xo from right

Weak Duality Illustration: Since this primal is a maximization problem Weak duality: If x is a feasible solution for the primal problem and y is a feasible solution for the dual problem, then cx ≤ yb.

Weak Duality Illustration: <= <= Suppose we have a primal solution (X1, X2) Suppose we have a dual solution (y1,y2,y3) So y1x1+2y2x2+y3(3x1+2x2)<= x1(y1+3y3)+x2(2y2+2y3)>= <= <=

Weak Duality Implications of weak duality: The objective function value for any feasible solution to the primal minimization problem is always greater than or equal to the objective function value for any feasible solution to the dual maximization problem. the objective value of any feasible solution of the primal minimization problem gives an upper bound on the optimal objective of the dual maximization problem. Similarly, the objective value of any feasible solution of the dual maximization problem gives a lower bound on the optimal objective of the primal minimization problem.

Weak Duality Implications of weak duality: If and are feasible solutions to the primal and dual problems, respectively, such that then they are optimal solutions to their respective problems. If either problem has an unbounded objective value, then the other problem possesses no feasible solution. Unboundedness in one problem implies infeasibility in the other problem. Infeasibility in one problem DOES NOT necessarily imply unboundedness in the other. See Example 6.4

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions: Farka’s Lemma:

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions: Farka’s Lemma alternatively says:

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions: Farka’s Lemma Part 1:

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions: Farka’s Lemma Part 2:

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions: Farka’s Lemma Part 2:

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions (necessary and sufficient): Farka’s lemma

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions (necessary and sufficient):

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions (necessary and sufficient):

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions (necessary and sufficient):

KKT Conditions and Duality About the relation with KKT optimality conditions: KKT Optimality Conditions (necessary and sufficient):

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

Strong Duality About the relation with KKT optimality conditions: From KKT conditions From weak duality If one problem possesses an optimal solution, then both problems possess optimal solutions and the two optimal objective values are equal

Strong Duality Implications of strong duality: If P is unbounded, D is infeasible If D is unbounded, P is infeasible If P is infeasible D can be unbounded? Possible! D can have optimum solution? No!! D can be infeasible? Possible! From strong duality

Strong Duality Fundamental theory of duality: I O U Yes

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

Complementary Slackness If and only if

Complementary Slackness

Complementary Slackness At optimality: If a variable in one problem is positive, then the corresponding constraint in the other problem must be tight. If a constraint in one problem is not tight, then the corresponding variable in the other problem must be zero.

Complementary Slackness Using complementary slackness to solve primal Example 6.5 from the book Since the dual has only two variables, we may solve it graphically

Complementary Slackness Using complementary slackness to solve primal Example 6.5 from the book Obj. value=5 From fundamental duality theorem, we know that there is an optimal solution to the primal with the same objective function value  = < < < = >=

Complementary Slackness Another example: x1*=2 and x2*=6. Let’s get the optimum dual solution The objective function values should be the same 3x1*+5x2*=4y1*+12y2*+18y3*  36=4y1*+12y2*+18y3* If a constraint is not ‘tight’ then the associated variable is 0 x1*<=4  2<4  not tight  so y1*=0 2x2*<=12  12=12  tight  so nothing about y2* so far 3x1*+2x2*<=18  18=18  tight  so nothing about y3* so far If a variable is not zero then the associated constraint is tight x1*=2>0  so first constraint  y1*+3y3*=3 x2*=6>0  so second constraint  2y2*+2y3* =5 Optimum solution to primal is x1*=2 and x2*=6.

Complementary Slackness Another example: So we have three unknowns three equations  4y1*+12y2*+18y3*=36  y1*+3y3*=3  2y2*+2y3* =5 Furthermore, we already know that  y1*=0 So, we can solve for y1*, y2*, y3*

Complementary Slackness Prove that there exists an optimum solution to the following LP model such that only one of the variables is non-negative Minimize 𝑖=1 𝑛 𝑎 𝑖 𝑥 𝑖 subject to 𝑖=1 𝑛 𝑏 𝑖 𝑥 𝑖 ≥𝐵 where ai>=0, bi>=0, and B>=0 𝑥 𝑖 ≥0 𝑖=1,2,…,𝑛

Outline Dual Formulation Primal-Dual Relationships Weak Duality KKT Conditions and Duality Strong Duality Complementary Slackness Economic Interpretation

Economic Interpretation Consider

Economic Interpretation Consider To illustrate, if the ith constraint represents a demand for production of at least bi units of the ith product and ex represents the total cost of production, then wi* is the incremental cost of producing one more unit of the ith product.

Economic Interpretation

Economic Interpretation Instead of trying to control the operation of the firm to obtain the most desirable mix of activities, suppose that we agree to pay the firm unit prices for each of the m outputs. We also stipulate that these prices announced by the firm must be fair. Since is the number of units of output i produced by one unit of activity j and is the unit price of output i, then can be interpreted as the unit price of activity j consistent with the prices Therefore, we tell the firm that the implicit price of activity j , namely should not exceed the actual cost cj. Within these constraints, the firm would like to choose a set of prices that maximizes its return

Economic Interpretation Fundamental duality theory states that, given optimum exists, minimum production cost of the buyer is equal to the maximum revenue of the seller Complementary slackness states: If the optimal level of activities that meets all demand requirements automatically produces an excess of product i, then the incremental cost associated with marginally increasing bi is naturally zero. If the total revenue generated via the items produced by a unit level of activity j is less than the associated production cost, then the level of activity j should be zero at optimality.

Economic Interpretation Consider an example: Wyndor Glass Co. produces windows and glass doors Plant 1 makes aluminum frames and hardware Plant 2 makes wood frames Plant 3 produces glass and assembles products Company introducing two new products Product 1: 8 ft. glass door with aluminum frame Product 2: 4 x 6 ft. double-hung, wood-framed window Problem: What mix of products would be most profitable? Assuming company could sell as much of either product as could be produced

Economic Interpretation The primal problem is: Objective function of the dual: biyi can thereby be interpreted as the current contribution to profit by having bi units of resource i available for the primal problem.

Economic Interpretation Shadow price: Given an optimal solution and the corresponding value of the objective function for a linear programming model, the shadow price for a functional constraint is the rate at which the value of the objective function changes by 1 unit change on the right-hand-side The dual variable yi is interpreted as the contribution to profit per unit of resource i, when the current set of basic variables is used to obtain the primal solution. So yi are shadow prices

Economic Interpretation Consider the Wyndor problem… If we produce 1 unit of doors, we will consume 1 hour from Plant 1 and 3 hours from Plant 3 So we could have gained y1+3y3 to the profit if we have not produced 1 unit of doors Also recall that 1 unit of doors has a profit of c1=3 So if the contribution of the resources used by 1 door is greater than the profit we will get from 1 door, we would not produce, Therefore, we have: If y1+3y3 >3  x1=0 Recall from complementary slackness, if the constraint is not tight, then the associated variable should be 0

Economic Interpretation Now let’s look at the shadow prices Let’s consider resource 2 If y2>0, this means that having additional one unit of resource 2 (i.e., 1 more hour in Plant 2) will increase our profit If we are at the optimum solution, then we should have used all of the hours in Plant 2, otherwise, we would not be in the optimum solution as using some more of the available time in Plant 2, we could increase our profit.. Therefore, if y2>0, we should currently be using all the time in plant 2, that is, we should have 2x2=16 Recall from complementary slackness, if a variable is not zero, then the associated constraint should be tight

Next time So far, we have learned the basics of linear programming… How can we use these to solve large scale problems? How can we use these to solver integer/mixed-integer models? Decomposition principle Column generation Branch and bound Cutting Planes Branch and cut Branch and price Bender’s decomposition