Case 2 cont’d, regular pt? 9 Regular Pt? 1. pt feasible, YES 2. active constraint gradients independent Are active constraint gradients independent i.e. parallel? Determinant of Constraint gradients non-singular? Case 2 results in a KKT point!
LaGrange Multiplier Method May produce a KKT point A KKT point is a CANDIDATE minimum It may not be a local Min If a point fails KKT conditions, we cannot guarantee anything…. The point may still be a minimum. We need a SUFFICIENT condition 14
15 Convex set: All pts in feasible region on a straight line(s). Convex sets Non-convex set Pts on line are not in feasible region
16 Multiple variables Fig 4.21 What if it were an equality constraint? misprint
17. Figure 4.22 Convex function f(x)=x 2 Bowl that holds water.
Test for Convex Function 19 Difficult to use above definition! However, Thm 4.8 pg 163: If the Hessian matrix of the function is PD ro PSD at all points in the set S, then it is convex. PD… “strictly” convex, otherwise PSD… “convex”
Theorem 4.9 20 Given: S is convex if: 1. h i are linear 2. g j are convex i.e. H g PD or PSD When f(x) and S are convex= “convex programming problem”
“Sufficient” Theorem 4.10, pg 165 21 The first-order KKT conditions are Necessary and Sufficient for a GLOBAL minimum….if: 1. f(x) is convex H f (x) Positive definite 2. x is defined as a convex feasible set S Equality constraints must be linear Inequality constraints must be convex HINT: linear functions are convex!
Summary LaGrange multipliers are the instantaneous rate of change in f(x) w.r.t. relaxing a constraint. Equality constraints may need tightening rather than loosening Convex sets assure contiguity and or the smoothness of f(x) KKT pt of a convex programming problem is a GLOBAL MINIMUM! 22