1. 1 OR II GSLM 52800

2. 2 Outline  separable programming  quadratic programming

3. 3 Separable Programs  a separable NLP if  f and all g j are separable functions  0  x i   i, a finite number

4. 4 Idea of Separable Program  min f(x), s.t.s.t.  g j (x)  0 for j = 1, …, m.  hard NLP but simple LP problems  approximating a separable NL program by a LP  a non-linear function by a piecewise linear one

5. 5 A Fact About Convex Functions  f: a convex function  for any  > 0, possible to find a sequence of piecewise linear convex functions f n such that |f  f n |  

6. 6 Example 6.1  a separable program

7. 7 Example 6.1  approximating by a piecewise linear function  two representations, -form and  -form pointsOABC x1x1 0244.5 y041620.25 1 C B A O 16 9 4 43 2 1

8. 8  Form  a piecewise linear function with (segment) break points  any point = the convex combination of the two break points of the linear segment  i (  0) = the weight of break point i 1 C B A O 20. 25 16 9 4 43 2 1

9. 9 Example 6.1  the program becomes  the last but one type of constraints is non-linear

10. 10 Fact  nonlinear constraint: at most two adjacent i taking non-zero values  possible to have only one i = 1  for convex f and g j : no need to have the non- linear constraint  non-optimal to have more than two non-zero i, or two i not adjacent

11. 11 Fact  non-optimal to have more than two non-zero i, or two i not adjacent  e.g., f being an objective function  any convex combination between two non-adjacent break points being above the piecewise non-linear function  similarly, the point for three or more non-zero I ’s lying above the piecewise non-linear function  think about A = 0.3, B = 0.4, and C = 0.3 1 C B A O 20.25 16 9 4 43 2 1

12. 12 Fact  non-optimal to have more than two non-zero i, or two i not adjacent  e.g., g j being a constraint  g j (0.3A+0.7B)  g j (0.3A+0.7C)  b j  the feasible set of {0.3A+0.7B} is larger than that by {0.3A+0.7C}  the solution from {0.3A+0.7C} cannot be minimum  similar argument for three or more non-zero i ’s lying above the piecewise non-linear function C B A O

13. 13 Example 6.1  the program becomes a linear program

14. 14 Example 6.2: Non-Convex Problem  min f(x),  s.t.1  x  3.  approximating f(x) by a piecewise linear function  y = 0 + 10 A + 6 B  x = 0 + 2 A + 3 B

15. 15 Example 6.2: Non-Convex Problem  adding slack variable s, surplus variable u, and artificial variable a 1 and a 2 :

16. 16 Example 6.2: Non-Convex Problem

17. 17 Example 6.2: Non-Convex Problem

18. 18 Example 6.2: Non-Convex Problem  most negative 0  B in basis  only A qualified to enter, not O

19. 19 Example 6.2: Non-Convex Problem

20. 20 Example 6.2: Non-Convex Problem

21. 21 Example 6.2: Non-Convex Problem

22. 22  Form  again, the last constraint is unnecessary for a convex program 1 C B A O 20.25 16 9 4 43 2 1

23. 23 Quadratic Programming

24. 24 Quadratic Objective Function & Linear Constraints  Langrangian function

25. 25 KKT Conditions  positive definite Q  a convex program  a unique global minimum  the KKT sufficient  otherwise, KKT necessary

26. 26 KKT Conditions  c T + x T Q +  T A  0  Qx + A   y =  c  Ax  b  0  Ax + v = b  x T (c + Qx + A  ) = 0  x T y = 0   T (Ax  b) = 0   T v = 0  x  0,   0, y  0, v  0  solving the set of equations  phase-1 of a linear program

27. 27 Example 7.1 (Example 10.14 of JB)

28. 28 Example 7.1 (Example 10.14 of JB)  KKT conditions  2x 1 +  1 +  2  y 1 = 8,  8x 2 +  1  y 2 = 16,  x 1 + x 2 + v 1 = 5,  x 1 + v 2 = 3.  x 1 y 1 = x 2 y 2 =  1 v 1 =  2 v 2 = 0  x 1, y 1, x 2, y 2,  1, v 1,  2, v 2  0

29. 29 Example 7.1 (Example 10.14 of JB)

30. 30 Example 7.1 (Example 10.14 of JB) Example 7.1 (Example 10.14 of JB)