1. OR II GSLM 52800 1

2. Outline
inequality constraint 2

3. NLP with Inequality Constraints
min f(x)
s.t.
hi(x) = 0 for i = 1, …, p
gj(x)  0 for j = 1, …, m
in matrix form
h(x) = 0
g(x) = 0 3

4. Inequality constraint gj(x)  0 is binding at point x0 if gj(x0) = 0
Binding Constraints
Inequality constraint gj(x)  0 is binding at point x0 if gj(x0) = 0
=-constraint: always binding 4

5. Regular Points K: the set of binding (inequality) constraints
x* subject to h(x) = 0 & g(x)  0 is a regular point
if Thi(x*), i = 1, …, p, & Tgj(x*), j  K, are linearly independent 5

6. FONC for Inequality Constraints (Karush-Kuhn-Tucker Necessary Conditions; KKT Conditions)
for a regular point x* to be a local min, there exist *  p and *  m such that
In Matrix Form:
f(x*) + *Th(x*) + * Tg(x*) = 0 (stationary condition; dual feasibility)
h(x*) = 0, g(x*)  0 (primal feasibility)
* Tg(x*) = 0 (Complementary slackness conditions)
*  0 (Non-negativity of the dual variables) 6

7. The Convex Cone Formed by Vectors
v1 = (1, 0), v2 = (0, 1)
{(x, y)|v1+v2 , ,   0}
The convex cone
formed by (1, 0) and (0, 1):
{(x, y)|v1+v2 , ,   0} v1 v2
(1, 0)
(0, 1)
The convex cone
formed by v1 and v2:
{(x, y)|v1+v2 , ,   0} 7

8. Geometric Interpretation of the KKT Condition
constraint 1: g1(x)  0
constraint 2: g2(x)  0
g2(x) = 0
g2(x0)
g1(x) = 0
g1(x0)
f (x0)
f (x0) x0
x0 cannot be a minimum 8

9. Geometric Interpretation of the KKT Condition
constraint 2: g2(x)  0
constraint 1: g1(x)  0
g1(x) = 0
g2(x) = 0
g2(x0)
g1(x0) x0
f (x0)
f (x0)
x0 cannot be a minimum 9

10. Geometric Interpretation of the KKT Condition
f (x0)
f (x0)
region decreasing in f
constraint 2: g2(x)  0
constraint 1: g1(x)  0
g1(x) = 0
g2(x) = 0
g2(x0)
g1(x0) x0
x0 cannot be a minimum 10

11. Geometric Interpretation of the KKT Condition
region decreasing in f
f (x0)
f (x0)
constraint 2: g2(x)  0
constraint 1: g1(x)  0
g1(x) = 0
g2(x) = 0
g2(x0)
g1(x0) x0
x0 cannot be a minimum 11

12. Geometric Interpretation of the KKT Condition
f (x0)
f (x0)
region decreasing in f
constraint 2: g2(x)  0
constraint 1: g1(x)  0
g1(x) = 0
g2(x) = 0
g2(x0)
g1(x0) x0
x0 can be a minimum 12

13. A Necessary Condition for x0 to be a Minimum
when f in the convex cone of g1 and g2
f(x0) = 1g1(x0) + 2g2(x0), i.e.,
f(x0) + 1g1(x0) + 2g2(x0) = 0
g1(x) = 0
g2(x) = 0
g2(x0)
g1(x0) x0
f (x0)
f (x0) 13

14. Effect of Convexity
convex objective function, convex feasible region set 
the KKT condition  necessary & sufficient
convex gj & linear hi 14

15. Example of JB
KKT conditions 15

16. Example 10.11 of JB Tf(x) = (4(x1+1), 6(x24)) Tg1(x) = (2x1, 2x2)
possibilities 2
= 0
> 0 1 16

17. Example 10.11 of JB case 1 = 0, 2 = 0 (both constraints not binding)
x1 = -1, x2 = 4, violating 17

18. Example of JB
case 1 > 0, 2 > 0 (both constraints binding)
solving
g2(x0)
g1(x0)
f (x0)
g2(x0)
g1(x0)
f (x0) 18

19. Example of JB
case 1 = 0, 2 > 0 (the second constraint binding)
solving, 2 < 0 19

20. Example of JB
case 1 > 0, 2 = 0 (the first constraint binding)
solving, 20

21. KKT Condition with Non-negativity Constraints
min f(x),
s.t. gi(x)  0, i = 1, …, m,
x  0 21

22. KKT Condition with Non-negativity Constraints
define L(x, ) = f(x) + Tg(x) 22

23. KKT Condition with Non-negativity Constraints
define L(x, ) = f(x) + 1Tg(x) - 2Tx 23

24. Example 10.12 of JB L(x, ) = KKT condition
(a) 2x1 8 +   0, 8x2 16 +   0
(b) x1+ x2 5  0
(c) x1(2x1 8 + ) = 0, x2(8x2 16 + ) = 0
(d) (x1+ x2 5) = 0
(e) x1 0, x2 0,  0   24

25. Example of JB
eight cases, depending on whether x1, x2, and = 0 or > 0
on checking, the case that
x1+x2  5 is binding, x1 > 0, x2 > 0  x = (3.2, 1.8) and  = 1.6
satisfying the KKT condition
regular point
convex objective with linear constraint  KKT being sufficient 25