1. Engineering Optimization
Concepts and Applications
WB 1440
Fred van Keulen
Matthijs Langelaar
CLA H21.1

2. Geometrical interpretation
For single equality constraint: simple geometrical interpretation of Lagrange optimality condition: f h x1 x2
Gradients parallel  tangents parallel  h tangent to isolines
Meaning: h f
For multiple equality constraints, this doesn’t work anymore, because the multipliers define a subspace. Since they can have any sign, there is no interpretation, other than the fact that the gradient must lie in this subspace.

3. Summary f h x1 x2 h f
First order optimality condition for equality constrained problem:
Zero reduced gradient:
Equivalent: stationary Lagrangian:

4. Contents Constrained Optimization: Optimality Criteria
Reduced gradient
Lagrangian
Sufficiency conditions
Inequality constraints
Karush-Kuhn-Tucker (KKT) conditions
Interpretation of Lagrange multipliers
Constrained Optimization: Algorithms

5. Sufficiency?
Until now, only stationary points considered. Does not guarantee minimum! f h h f f h h f
Lagrange condition:
maximum
minimum
Using second order Taylor approximation, with this Hessian, we can formulate the condition for a minimum. f h h f f h h f
minimum
no extremum

6. Constrained Hessian
Sufficiency conditions follow from 2nd order Taylor approximation
Second order information required: constrained Hessian:
obtained by differentiation of the constrained gradient, and second-order constraint perturbation:
with

7. Sufficiency conditions
Via 2nd order Taylor approximation, it follows that at a minimum the following must hold:
(Constrained Hessian positive definite)
and
Lagrangian approach also yields:
with
i.e., the constrained hessian should be positive definite.
Perturbations only in tangent subspace of h!

8. Summary Optimality conditions for equality constrained problem:
1. Necessary condition: stationary point when:
2. Sufficient condition: minimum when (1) and:
on tangent subspace.

9. Example x2 f h x1
1. Necessary condition: stationary point when

10. Contents Constrained Optimization: Optimality Criteria
Reduced gradient
Lagrangian
Sufficiency conditions
Inequality constraints
Karush-Kuhn-Tucker (KKT) conditions
Interpretation of Lagrange multipliers
Constrained Optimization: Algorithms

11. Inequality constrained problems
Consider problem with only inequality constraints:
At optimum, only active constraints matter:
Optimality conditions similar to equality constrained problem

12. Inequality constraints
First order optimality:
Consider feasible local variation around optimum:
(boundary optimum)
(feasible perturbation)

13. Optimality condition Multipliers must be non-negative:x1 x2 f
-f
This interpretation is given in Haftka.
Interpretation: negative gradient (descent direction) lies in cone spanned by positive constraint gradients
-f

14. Optimality condition (2)g2
Feasible cone x2
Feasible direction: g1 f
-f
Descent direction: x1
This interpretation is given in Belegundu.
Equivalent interpretation: no descent direction exists within the cone of feasible directions

15. Examples f f
-f f
-f
-f

16. Optimality condition (3)
Active constraints: Inactive constraints:
Formulation including all inequality constraints:
For regular points, for the non-degenerate case, the multipliers of active constraints are positive, and cannot be zero.
Because all multipliers must be nonnegative, and because inactive constraints have negative g-values, the multipliers related to the constraints must be zero. So therefore the inner product is equivalent to mu_I times g_I.
and
Complementaritycondition

17. Example x2 x1 L m x2 x1 m L
Also used in finite element contact algorithms

18. Mechanical application: contact
Lagrange multipliers also used in:
Contact in multibody dynamics
Contact in finite elements

19. Contents Constrained Optimization: Optimality Criteria
Reduced gradient
Lagrangian
Sufficiency conditions
Inequality constraints
Karush-Kuhn-Tucker (KKT) conditions
Interpretation of Lagrange multipliers
Constrained Optimization: Algorithms

20. Karush-Kuhn-Tucker conditions
Combining Lagrange conditions for equality and inequality constraints yields KKT conditions for general problem:
Lagrangian:
(optimality)
Note, this condition applies only to regular points, I.e. were the constraint gradients are not linearly dependent.
and
(feasibility)
(complementarity)

21. Sufficiency
KKT conditions are necessary conditions for local constrained minima
For sufficiency, consider the sufficiency conditions based on the active constraints:
on tangent subspace of h and active g.
Interpretation: objective and feasible domain locally convex

22. Additional remarks Global optimality:
Globally convex objective function?
And convex feasible domain?
Then KKT point gives global optimum
Pitfall:
Sign conventions for Lagrange multipliers in KKT condition depend on standard form!
Presented theory valid for negative null form

23. Contents Constrained Optimization: Optimality Criteria
Reduced gradient
Lagrangian
Sufficiency conditions
Inequality constraints
Karush-Kuhn-Tucker (KKT) conditions
Interpretation of Lagrange multipliers
Constrained Optimization: Algorithms

24. Significance of multipliers
Consider case where optimization problem depends on parameter a:
Lagrangian:
KKT:
Looking for:

25. Significance of multipliers (2)
Looking for:
KKT:

26. Significance of multipliers (3)
Lagrange multipliers describe the sensitivity of the objective to changes in the constraints:
Similar equations can be derived for multiple constraints and inequalities
Multipliers give “price of raising the constraint”
Note, this makes it logical that at an optimum, multipliers of inequality constraints must be positive!

27. Minimize mass (volume):
Example
A, sy N
Minimize mass (volume): l
Stress constraint:

28. Constraint sensitivity:
Example (2)
Stress constraint:
Constraint sensitivity:
Check: