Engineering Optimization Concepts and Applications WB 1440 Fred van Keulen Matthijs Langelaar CLA H21.1 A.vanKeulen@tudelft.nl

Geometrical interpretation For single equality constraint: simple geometrical interpretation of Lagrange optimality condition: f h x1 x2 Gradients parallel  tangents parallel  h tangent to isolines Meaning: h f For multiple equality constraints, this doesn’t work anymore, because the multipliers define a subspace. Since they can have any sign, there is no interpretation, other than the fact that the gradient must lie in this subspace.

Summary f h x1 x2 h f First order optimality condition for equality constrained problem: Zero reduced gradient: Equivalent: stationary Lagrangian:

Contents Constrained Optimization: Optimality Criteria Reduced gradient Lagrangian Sufficiency conditions Inequality constraints Karush-Kuhn-Tucker (KKT) conditions Interpretation of Lagrange multipliers Constrained Optimization: Algorithms

Sufficiency? Until now, only stationary points considered. Does not guarantee minimum! f h h f f h h f Lagrange condition: maximum minimum Using second order Taylor approximation, with this Hessian, we can formulate the condition for a minimum. f h h f f h h f minimum no extremum

Constrained Hessian Sufficiency conditions follow from 2nd order Taylor approximation Second order information required: constrained Hessian: obtained by differentiation of the constrained gradient, and second-order constraint perturbation: with

Sufficiency conditions Via 2nd order Taylor approximation, it follows that at a minimum the following must hold: (Constrained Hessian positive definite) and Lagrangian approach also yields: with i.e., the constrained hessian should be positive definite. Perturbations only in tangent subspace of h!

Summary Optimality conditions for equality constrained problem: 1. Necessary condition: stationary point when: 2. Sufficient condition: minimum when (1) and: on tangent subspace.

Example x2 f h x1 1. Necessary condition: stationary point when

Contents Constrained Optimization: Optimality Criteria Reduced gradient Lagrangian Sufficiency conditions Inequality constraints Karush-Kuhn-Tucker (KKT) conditions Interpretation of Lagrange multipliers Constrained Optimization: Algorithms

Inequality constrained problems Consider problem with only inequality constraints: At optimum, only active constraints matter: Optimality conditions similar to equality constrained problem

Inequality constraints First order optimality: Consider feasible local variation around optimum: (boundary optimum) (feasible perturbation)

Optimality condition Multipliers must be non-negative: x1 x2 f -f This interpretation is given in Haftka. Interpretation: negative gradient (descent direction) lies in cone spanned by positive constraint gradients -f

Optimality condition (2) g2 Feasible cone x2 Feasible direction: g1 f -f Descent direction: x1 This interpretation is given in Belegundu. Equivalent interpretation: no descent direction exists within the cone of feasible directions

Examples f f -f f -f -f

Optimality condition (3) Active constraints: Inactive constraints: Formulation including all inequality constraints: For regular points, for the non-degenerate case, the multipliers of active constraints are positive, and cannot be zero. Because all multipliers must be nonnegative, and because inactive constraints have negative g-values, the multipliers related to the constraints must be zero. So therefore the inner product is equivalent to mu_I times g_I. and Complementaritycondition

Example x2 x1 L m x2 x1 m L Also used in finite element contact algorithms

Mechanical application: contact Lagrange multipliers also used in: Contact in multibody dynamics Contact in finite elements

Contents Constrained Optimization: Optimality Criteria Reduced gradient Lagrangian Sufficiency conditions Inequality constraints Karush-Kuhn-Tucker (KKT) conditions Interpretation of Lagrange multipliers Constrained Optimization: Algorithms

Karush-Kuhn-Tucker conditions Combining Lagrange conditions for equality and inequality constraints yields KKT conditions for general problem: Lagrangian: (optimality) Note, this condition applies only to regular points, I.e. were the constraint gradients are not linearly dependent. and (feasibility) (complementarity)

Sufficiency KKT conditions are necessary conditions for local constrained minima For sufficiency, consider the sufficiency conditions based on the active constraints: on tangent subspace of h and active g. Interpretation: objective and feasible domain locally convex

Additional remarks Global optimality: Globally convex objective function? And convex feasible domain? Then KKT point gives global optimum Pitfall: Sign conventions for Lagrange multipliers in KKT condition depend on standard form! Presented theory valid for negative null form

Contents Constrained Optimization: Optimality Criteria Reduced gradient Lagrangian Sufficiency conditions Inequality constraints Karush-Kuhn-Tucker (KKT) conditions Interpretation of Lagrange multipliers Constrained Optimization: Algorithms

Significance of multipliers Consider case where optimization problem depends on parameter a: Lagrangian: KKT: Looking for:

Significance of multipliers (2) Looking for: KKT:

Significance of multipliers (3) Lagrange multipliers describe the sensitivity of the objective to changes in the constraints: Similar equations can be derived for multiple constraints and inequalities Multipliers give “price of raising the constraint” Note, this makes it logical that at an optimum, multipliers of inequality constraints must be positive!

Minimize mass (volume): Example A, sy N Minimize mass (volume): l Stress constraint:

Constraint sensitivity: Example (2) Stress constraint: Constraint sensitivity: Check: