1. Optimization of thermal processes2007/2008 Optimization of thermal processes Maciej Marek Czestochowa University of Technology Institute of Thermal Machinery Lecture 3

2. Optimization of thermal processes2007/2008 Overview of the lecture Multivariable optimization with no constraints Multivariable objective function −Extreme points −Necessary and sufficient conditions for extreme points Differential calculus methods Multivariable optimization with equality constraints −Solution by direct substitution −Lagrange multipliers method

3. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization problem Find which minimizes What about constraints? The constraints are not significant in some of the problems It is instructive to study unconstrained problems first There are powerful methods for constrained optimization problems that use unconstrained minimization techniques

4. Optimization of thermal processes2007/2008 Multivariable objective function minimum Surface plot Contour plot Objective function surfaces

5. Stationary point Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) Necessary condition Just as in the case of single-variable function, this condition is not sufficient: Saddle point

6. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) To formulate sufficient condition we have to introduce matrix H: The Hess matrix The Hessian Sufficient condition for minimum at the extreme point X*: If the Hessian is positive definite, then X* is minimum. What does it mean?

7. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) The matrix H is positive definite when: for every non-zero h...

8. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) With the help of determinants H 1, H 2,..., H n we can formulate the sufficient condition in a more convenient way: −If all the values H 1, H 2,..., H n are positive, then the Hessian is positive definite and the extreme point is minimum −If the sign of H j is for j=1,2,...,n, then the Hessian is negative definite and the extreme point is maximum For instance in the case of two variables (suppose all the derivatives are evaluated at the extreme point X*): Relative minimum at X *

9. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) But if: then there is relative maximum at X * Note, that H 2 >0 in both of the cases. If, on the other hand: then there is a saddle point at X * (H is indefinite).

10. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) EXAMPLE Find the extreme points of the function: Necessary condition Stationary points Hessian What is the nature of the extreme points?

11. Optimization of thermal processes2007/2008 Unconstrained multivariable optimization (differential calculus methods) EXAMPLE contd Point XH1H1 H2H2 Nature of HNature of X (0,0)+4+32Positive definiteRelative minimum (0,-8/3)+4-32IndefiniteSaddle point (-4/3,0)-4-32IndefiniteSaddle point (-4/3,-8/3)-4+32Negative definiteRelative maximum

12. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) Find which minimizes subject to Equality constraints Number of independent variables (degrees of freedom): So there should be: Otherwise, the problem is overdefined (no solution, in general)

13. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) Minimum point with no constraints Minimum with constraint g 1 Minimum with constraint g 2 With both constraints there is no solution!

14. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) EXAMPLE Find the dimensions of a box of largest volume that can be inscribed in a sphere of unit radius.

15. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) EXAMPLE contd Objective function Constraint We can transform this problem into unconstrained optimization problem. So, substituting in f: We need only to maximize f (use classical differential calculus method). Homework : find the extreme point and make sure it is maximum!

16. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) This is the idea of Method of Direct Substitution. Suppose we have n variables and m equality constraints Then there are n-m independent variables Choose a set of m variables and express them in terms of independent variables The new objective function involves only n-m variables and is not subjected to any constraints. Voilà! But it is not always that simple. For many nonlinear constraints it is impossible to express any m variables in terms of the remaining ones.

17. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) Method of Lagrange Multipliers is more general. Here are the basic features for the problem with two variables and one constraint. Minimize subject to Construct a function L (Lagrange function) as Necessary conditions for the extremum are: Lagrange multiplier

18. Optimization of thermal processes2007/2008 Constrained multivariable optimization problem (equality constraints) EXAMPLE Minimize subject to k, a - constants Lagrange function

19. Optimization of thermal processes2007/2008 Thank you for your attention