1. (iii) Lagrange Multipliers and Kuhn-tucker Conditions D Nagesh Kumar, IISc Introduction to Optimization Water Resources Systems Planning and Management: M2L3

2. Objectives D Nagesh Kumar, IISc 2  To study the optimization with multiple decision variables and equality constraint : Lagrange Multipliers.  To study the optimization with multiple decision variables and inequality constraint : Kuhn-Tucker (KT) conditions Water Resources Systems Planning and Management: M2L3

3. Constrained optimization with equality constraints D Nagesh Kumar, IISc 3 A function of multiple variables, f(x), is to be optimized subject to one or more equality constraints of many variables. These equality constraints, g j (x), may or may not be linear. The problem statement is as follows: Maximize (or minimize) f(X), subject to g j (X) = 0, j = 1, 2, …, m where (1) Water Resources Systems Planning and Management: M2L3

4. Constrained optimization… D Nagesh Kumar, IISc 4  With the condition that ; or else if m > n then the problem becomes an over defined one and there will be no solution. Of the many available methods, the method of constrained variation and the method of using Lagrange multipliers are discussed. Water Resources Systems Planning and Management: M2L3

5. Solution by method of Lagrange multipliers D Nagesh Kumar, IISc 5 Continuing with the same specific case of the optimization problem with n = 2 and m = 1 we define a quantity λ, called the Lagrange multiplier as (2) Using this in the constrained variation of f [ given in the previous lecture] And (2) written as (3) (4) Water Resources Systems Planning and Management: M2L3

6. Solution by method of Lagrange multipliers… D Nagesh Kumar, IISc 6 Also, the constraint equation has to be satisfied at the extreme point (5) Hence equations (2) to (5) represent the necessary conditions for the point [x 1 *, x 2 *] to be an extreme point. λ could be expressed in terms of as well has to be non- zero. These necessary conditions require that at least one of the partial derivatives of g(x 1, x 2 ) be non-zero at an extreme point. Water Resources Systems Planning and Management: M2L3

7. D Nagesh Kumar, IISc 7 The conditions given by equations (2) to (5) can also be generated by constructing a functions L, known as the Lagrangian function, as (6) Alternatively, treating L as a function of x 1,x 2 and, the necessary conditions for its extremum are given by (7) Solution by method of Lagrange multipliers… Water Resources Systems Planning and Management: M2L3

8. Necessary conditions for a general problem D Nagesh Kumar, IISc 8 For a general problem with n variables and m equality constraints the problem is defined as shown earlier Maximize (or minimize) f(X), subject to g j (X) = 0, j = 1, 2, …, m where In this case the Lagrange function, L, will have one Lagrange multiplier j for each constraint as (8) Water Resources Systems Planning and Management: M2L3

9. D Nagesh Kumar, IISc 9 L is now a function of n + m unknowns,, and the necessary conditions for the problem defined above are given by (9) which represent n + m equations in terms of the n + m unknowns, x i and j. The solution to this set of equations gives us and (10) Necessary conditions for a general problem… Water Resources Systems Planning and Management: M2L3

10. Sufficient conditions for a general problem D Nagesh Kumar, IISc 10 A sufficient condition for f(X) to have a relative minimum at X * is that each root of the polynomial in Є, defined by the following determinant equation be positive. (11) Water Resources Systems Planning and Management: M2L3

11. D Nagesh Kumar, IISc 11 where (12) Similarly, a sufficient condition for f(X) to have a relative maximum at X * is that each root of the polynomial in Є, defined by equation (11) be negative. If equation (11), on solving yields roots, some of which are positive and others negative, then the point X * is neither a maximum nor a minimum. Sufficient conditions for a general problem… Water Resources Systems Planning and Management: M2L3

12. Example D Nagesh Kumar, IISc 12 Minimize, Subject to Solution with n = 2 and m = 1 L = or Water Resources Systems Planning and Management: M2L3

13. Example… D Nagesh Kumar, IISc 13 Hence, Water Resources Systems Planning and Management: M2L3

14. Example… D Nagesh Kumar, IISc 14 The determinant becomes or Since is negative, X*, correspond to a maximum Water Resources Systems Planning and Management: M2L3

15. Kuhn – Tucker Conditions D Nagesh Kumar, IISc 15  KT condition: Both necessary and sufficient if the objective function is concave and each constraint is linear or each constraint function is concave, i.e., the problems belongs to a class called the convex programming problems. Water Resources Systems Planning and Management: M2L3

16. Kuhn-Tucker Conditions: Optimization Model Consider the following optimization problem Minimize f(X) subject to g j (X) ≤ 0 for j=1,2,…,p where the decision variable vector X=[x 1,x 2,…,x n ] D Nagesh Kumar, IISc Water Resources Systems Planning and Management: M2L3 16

17. Kuhn-Tucker Conditions Kuhn-Tucker conditions for X * = [x 1 *, x 2 *,... x n * ] to be a local minimum are D Nagesh Kumar, IISc Water Resources Systems Planning and Management: M2L3 17

18. Kuhn Tucker Conditions …  In case of minimization problems, if the constraints are of the form g j (X) ≥ 0, then λ j have to be non-positive  If the problem is one of maximization with the constraints in the form g j (X) ≥ 0, then λ j have to be nonnegative. D Nagesh Kumar, IISc Water Resources Systems Planning and Management: M2L3 18

19. Example 1 D Nagesh Kumar, IISc 19 Minimize subject to Water Resources Systems Planning and Management: M2L3

20. D Nagesh Kumar, IISc 20 Kuhn – Tucker Conditions Example 1… Water Resources Systems Planning and Management: M2L3

21. From (17) either = 0 or, Case 1: = 0  From (14), (15) and (16) we have x 1 = x 2 = and x 3 =  Using these in (18) we get  From (22),, therefore, =0,  Therefore, X * = [ 0, 0, 0 ] This solution set satisfies all of (18) to (21) Example 1… D Nagesh Kumar, IISc 21 Water Resources Systems Planning and Management: M2L3

22. Case 2:  Using (14), (15) and (16), we have or  But conditions (21) and (22) give us and simultaneously, which cannot be possible with Hence the solution set for this optimization problem is X * = [ 0 0 0 ] Example 1… D Nagesh Kumar, IISc 22 Water Resources Systems Planning and Management: M2L3

23. Minimize subject to Example 2 D Nagesh Kumar, IISc 23 Water Resources Systems Planning and Management: M2L3

24. Kuhn – Tucker Conditions Example 2… D Nagesh Kumar, IISc 24 Water Resources Systems Planning and Management: M2L3

25. From (25) either = 0 or, Case 1  From (23) and (24) we have and  Using these in (26) we get  Considering, X * = [ 30, 0]. But this solution set violates (27) and (28)  For, X * = [ 45, 75]. But this solution set violates (27) Example 2… D Nagesh Kumar, IISc 25 Water Resources Systems Planning and Management: M2L3

26. Case 2:  Using in (23) and (24), we have  Substitute (31) in (26), we have  For this to be true, either Example 2… D Nagesh Kumar, IISc 26 Water Resources Systems Planning and Management: M2L3

27.  For,  This solution set violates (27) and (28)  For,  This solution set is satisfying all equations from (27) to (31) and hence the desired  Thus, the solution set for this optimization problem is X * = [ 80, 40 ] Example 2… D Nagesh Kumar, IISc 27 Water Resources Systems Planning and Management: M2L3

28. BIBLIOGRAPHY / FURTHER READING 1. Rao S.S., Engineering Optimization – Theory and Practice, Fourth Edition, John Wiley and Sons, 2009. 2. Ravindran A., D.T. Phillips and J.J. Solberg, Operations Research – Principles and Practice, John Wiley & Sons, New York, 2001. 3. Taha H.A., Operations Research – An Introduction, 8 th edition, Pearson Education India, 2008. 4. Vedula S., and P.P. Mujumdar, Water Resources Systems: Modelling Techniques and Analysis, Tata McGraw Hill, New Delhi, 2005. D Nagesh Kumar, IISc 28 Water Resources Systems Planning and Management: M2L3

29. Thank You D Nagesh Kumar, IISc Water Resources Systems Planning and Management: M2L3