# Engineering Optimization

## Presentation on theme: "Engineering Optimization"— Presentation transcript:

Engineering Optimization
Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1

Covered so far … Introduction: Optimization problem: Problem checking:
Negative null form Applications Optimization problem: Definition Characteristics Problem checking: Boundedness Monotonicity analysis Optimization model: Model simplification Approximation

Upcoming topics Optimization problem Definition Checking
Negative null form Model Special topics Sensitivity analysis Topology optimization Linear / convex problems Solution methods Optimality criteria Optimization algorithms Unconstrained problems Constrained problems

Contents Unconstrained problems Transformation methods
Existence of solutions, optimality conditions Nature of stationary points Global optimality

Unconstrained Optimization
Why? Elimination of active constraints  unconstrained problem Develop basic understanding useful for constrained optimization Transformation of constrained problems into unconstrained problems Relevant engineering problems (potential energy minimization) Unconstrained problems are rare in practice, most practical optimum design problems involve restrictions of some sort. Transformation of constrained problems into unconstrained problems

Transforming constrained problem
Reformulation through barrier functions: g Transformation: Also called barrier functions. This is just one of the many possibilities. x f

Transformed problem g f

Transformed problem Barrier functions result in feasible, interior optimum: r = 100 An iterative optimization process is often used, where r is gradually increased. Note, recent research has also produced exact barrier functions. However, this is outside the scope of this course. r = 200 r = 400 r = 800

Penalization Alternative reformulation: penalty functions g
Transformation: f

Penalization (2) Penalty functions result in infeasible, exerior optimum: p = 40 p = 20 p = 10 p = 4

Problem transformation summary
Barrier function Penalty function Need feasible Yes No starting point? Nature of optimum Interior Exterior (feasible) (infeasible) Type of constraints g g, h Exact methods: see e.g.

Unconstrained Optimization
Why? Elimination of active constraints  unconstrained problem Develop basic understanding useful for constrained optimization Transformation of constrained problems into unconstrained problems Relevant engineering problems (potential energy minimization) Unconstrained problems are rare in practice, most practical optimum design problems involve restrictions of some sort.

Unconstrained engineering problem
Example: displacement of loaded structure Fx = 5N Fy = 5N k1 = 8N/cm k2 = 1N/cm 10 cm x1 x2 Equilibrium: minimum potential energy

Unconstrained engineering problem
Potential energy: x1 x2 k1 Fy Fx k2 Equilibrium:

Unconstrained engineering problem
x1 x2 P x2 x1

Contents Unconstrained problems Transformation methods
Existence of solutions, optimality conditions Nature of stationary points Global optimality

Theory for solving unc. problems
Assumptions: Objective continuous and differentiable (C1) Domain closed and bounded (compact) ]a, b[ f

Existence of minima Weierstrass Theorem:
“A continuous function on a compact set has a maximum and a minimum in that set” Sufficient condition for existence! Interior optima only exist for non-monotonic functions

One-dimensional case Calculus: conditions for minimum of f
Derivative zero: Second derivative positive: (necessary) (sufficient) Interpretation through Taylor series: f x +h Emphasize that a Taylor series is a LOCAL approximation. The conditions apply to LOCAL optima. Second order derivative positive means that the gradient is increasing. Points with f’=0 are called stationary points. Stationary points with f’’>0 are local minima. -h

One-dimensional case (2)
Condition for minimum: f”(x) > 0 Other possibilities: f”(x) < 0 maximum f”(x) = 0 ? Check higher order derivatives

Geometrical interpretation
Positive locally increasing f x

One-dimensional case (3)
Possible situations for stationary points (f’ = 0): f x

Example Aspirin pill revisited: worst pill ever!
Maximize dissolving time  minimize surface area r h Meaningful problems for surface minimization of a cylinder with a fixed volume could be e.g. a cooled tank (surface is proportional to heat transfer with environment) or building the cheapest container tank for e.g. oil storage. Equality constraint active  eliminate h

Example (2)

Multidimensional case
Local approximation to multidimensional minimum by multidimensional Taylor series: Gradient Condition for minimum:

Multidimensional case (2)
For minimum, consider second-order approximation: Hessian

Multidimensional case (3)
Second order approximation: These are the optimality conditions for unconstrained optimization. Local minimum: First Order Necessity Condition: Second Order Sufficiency Condition:

Positive definite Hessian positive definite
Test for positive definiteness: Evaluate yTHy for all y (impractical) All eigenvalues li of H positive Sylvester’s rule: all determinants of H and its principal submatrices are positive

Fx Fy k1 k2 ux uy Another loaded structure: Equilibrium: First order necessity condition:

Example (2) Second order sufficiency: H positive definite?
In a separable minimization problem, the minimization can be performed separately for every variable (1-D). Note: H diagonal constant matrix. Separable objective function:

Quadratic functions Polynomial terms up to 2nd order: General form:
For geometrical interpretation of optimality conditions, consider quadratic functions. Note: 2nd order Taylor series is exact

Transformation to symmetric matrix: Symmetric, since:

Optimality conditions: Gradient: Hessian: The gradient is defined by matrix differentiation, see here: or other matrix calculus reference works. Stationary points:

Hessian positive definite
Example Consider Stationary point Hessian positive definite

Example (2) Result: minimum at (1.2, -2.6): f x2 x1

Example: least squares
Least squares fitting: unconstrained optimization problem Stationary point: x f

Contents Unconstrained problems Transformation methods
Existence of solutions, optimality conditions Nature of stationary points Global optimality

Nature of stationary points
Hessian H positive definite: Quadratic form Eigenvalues Local nature: minimum Both definitions of positive definiteness are equivalent Note positive curvatures Eigenvalues can be interpreted as curvatures.

Nature of stationary points (2)
Hessian H negative definite: Quadratic form Eigenvalues Local nature: maximum

Nature of stationary points (3)

Nature of stationary points (4)
Hessian H positive semi-definite: Quadratic form Eigenvalues H singular! Local nature: valley

Nature of stationary points (5)
Hessian H negative semi-definite: Quadratic form Eigenvalues H singular! Local nature: ridge

Stationary point nature summary
Definiteness H Nature x* Positive d. Minimum Positive semi-d. Valley Indefinite Saddlepoint Negative semi-d. Ridge Negative d. Maximum

Structural example Compressed pin-jointed truss with rotational springs: F k1 k2 l Maximum (unstable) Minima (stable) Saddles (unstable)

Contents Unconstrained problems Transformation methods
Existence of solutions, optimality conditions Nature of stationary points Global optimality

Global optimality Optimality conditions for unconstrained problem:
First order necessity: (stationary point) Second order sufficiency: H positive definite at x* Optimality conditions only valid locally: local minimum Specifically, how to be sure a global optimum has been found. When can we be sure x* is a global minimum?

Convex functions Convex function: any line connecting any 2 points on the graph lies above it (or on it): Note difference between strictly convex and “normal” convex. Equivalent: tangent lines/planes stay below the graph H positive (semi-)definite  f locally convex (proof by Taylor approximation)

Convex domains Convex set:
“A set S is convex if for every two points x1, x2 in S, the connecting line also lies completely inside S”

Convexity and global optimality
If: Objective f = (strictly) convex function Feasible domain = convex set (Ok for unconstrained optimization) Stationary point = (unique) global minimum Special case: f, g, h all linear  linear optimization programming More general class: convex optimization

Example Quadratic functions with A positive definite are strictly convex: f x1 x2 Stationary point (1.2, -2.6) must be unique global optimum

Summary optimality conditions
Conditions for local minimum of unconstrained problem: First Order Necessity Condition: Second Order Sufficiency Condition: H positive definite For convex f in convex feasible domain: condition for global minimum: Sufficiency Condition: