1. Bi-Parametric Convex Quadratic Optimization Tamás Terlaky Lehigh University Joint work with Alireza Ghaffari-Hadigheh and Oleksandr Romanko RUTCOR 2009: Dedicated to the 80 th Birthday of Professor András Prékopa

2. 2

3. 3 Outline Introduction Quadratic optimization, optimal partition Uni-Parametric quadratic optimization Bi-Parametric quadratic optimization Numerical illustration Fundamental properties Algorithm Conclusions and future work

4. 4 General framework of parametric optimization Multidimensional parameter is introduced into objective function and/or constraints The goal is to find – optimal solution – optimal value function Generalization of sensitivity analysis Applications: multi-objective optimization Introduction: Parametric Optimization

5. 5 Multi-objective optimization: Introduction: Multi-Objective Optimization as Parametric Problem Multi-objective optimization with weighting method: Parametric formulation: f2f2 f*2f*2 f1f1 identify Pareto frontier (all non- dominated solutions) f*1f*1 OBJECTIVE SPACE

6. 6 Convex Quadratic Optimization (QO) problem: Introduction: Quadratic Optimization and Its Parametric Counterpart Bi-Parametric Convex Quadratic Optimization (PQO) problem: Bi-parametric QO generalizes three models: uni-parametric QO

7. 7 Sensity Analysis: Just be careful!

8. 8 Convex Quadratic Optimization problems: Optimal Partition for QO Optimality conditions: PrimalDual Maximally complementary solution: LO: and - strictly complementary solution QO:, but may not hold maximally complementary solution maximizes the number of non-zero coordinates in and IPMs !!!

9. 9 An optimal solution is maximally complementary iff: Optimal Partition for QO The optimal partition of the index set {1, 2,…, n} is The optimal partition is unique!!! Example: for maximally complementary solution with:

10. 10 Uni-Parametric Quadratic Optimization Primal and dual perturbed problems: For some we are given the maximally complementary optimal solution of and with the optimal partition. - invariancy interval The left and right extreme points of the invariancy interval: - transition points

11. 11 Uni-Parametric QO: Optimal Partition in the Neighboring Invariancy Interval How to proceed from the current invariancy interval to the next one? (1) (2) Solve two auxiliary problems z Gengyang zz

12. 12 Uni-param QO: Numerical Illustration type l u B N T  ( ) ----------------------------------------------------------------------------------------------------------------- invariancy interval +3.33333 Inf 3 4 5 1 2 0.00 2 - 0.00 + 0.00 transition point -8.00000 -8.00000 3 5 1 4 2 -0.00 invariancy interval -8.00000 -5.00000 2 3 5 1 4 8.50 2 + 68.00 + 0.00 transition point -5.00000 -5.00000 2 1 3 4 5 -127.50 invariancy interval -5.00000 +0.00000 1 2 3 4 5 4.00 2 + 35.50 - 50.00 transition point +0.00000 +0.00000 1 2 3 4 5 -50.00 invariancy interval +0.00000 +1.73913 1 2 3 4 5 -6.91 2 + 35.50 - 50.00 transition point +1.73913 +1.73913 2 3 4 5 1 -9.15 invariancy interval +1.73913 +3.33333 2 3 4 5 1 -3.60 2 + 24.00 - 40.00 transition point +3.33333 +3.33333 3 4 5 1 2 0.00 Solver output

13. 13 Bi-Parametric Quadratic Optimization Primal and dual perturbed problems: Invariancy regions instead of invariancy intervals Illustrative example:

14. 14 Bi-Parametric Quadratic Optimization Illustrative example Invariancy regions

15. 15 Bi-Parametric Quadratic Optimization Illustrative example: Optimal value function

16. 16 Bi-Parametric Quadratic Optimization Optimal partition is constant on invariancy regions. Invariancy regions that are transition lines or singletons are called trivial regions. Otherwise, they are called non-trivial invariancy regions. The optimal value function is continuous and piecewise bivariate quadratic The boundary of a non-trivial invariancy region consists of a finite number of line segments. The optimal value function is a bivariate quadratic function on invariancy region : Invariancy region is a convex set and its closure is a polyhedron that might be unbounded.

17. 17 Bi-Parametric QO: Algorithm Idea: reduce bi-parametric QO problem to a series of uni-paramteric QO problems with where

18. 18 Bi-Parametric QO: Algorithm Start from, determine the optimal partition Solve where Solve where Now, two points and on the boundary of the invariancy region are known Consider cases and Choose, and

19. 19 Bi-Parametric QO: Algorithm Case

20. 20 Bi-Parametric QO: Algorithm Case  : and

21. 21 Bi-Parametric QO: Algorithm Case  : and

22. 22 Bi-Parametric QO: Algorithm Case  : and  : back to the first or the second case

23. 23 Bi-Parametric QO: Algorithm Invariancy region exploration

24. 24 Bi-Parametric QO: Algorithm Enumerating all invariancy regions cell vertex edge To-be-processed queue Completed queue

25. 25 Conclusions and Future Work Developed an IPM-based technique for solving bi-parametric problems that extends the results of the uni-parametric case allows solving both bi-parametric linear and bi-parametric quadratic optimization problems systematically explores the optimal value surface  Polynomial-time algorithm in the output size  Applications in finance, IMRT, data mining Improving the implementation Extending methodology to Parametric Second Order Conic Optimization Multi-Parametric Quadratic Optimization

26. 26 References A. B. Berkelaar, C. Roos, and T. Terlaky. The optimal set and optimal partition approach to linear and quadratic programming. In Advances in Sensitivity Analysis and Parametric Programming, T. Gal and H. J. Greenberg, eds., Kluwer, Boston, USA, 1997. A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Sensitivity Analysis in Convex Quadratic Optimization: Simultaneous Perturbation of the Objective and Right-Hand-Side Vectors. Algorithmic Operations Research, Vol. 2(2), 2007. A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Bi- Parametric Convex Quadratic Optimization. To appear in Optimization Methods and Software, 2009. A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. On Bi- Parametric Programming in Quadratic Optimization. P roceedings of EurOPT-2008, 2008.