Download presentation

Presentation is loading. Please wait.

1
**Engineering Optimization**

Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1

2
**Covered so far … Introduction: Optimization problem: Problem checking:**

Negative null form Applications Optimization problem: Definition Characteristics Problem checking: Boundedness Monotonicity analysis Optimization model: Model simplification Approximation

3
**Upcoming topics Optimization problem Definition Checking**

Negative null form Model Special topics Sensitivity analysis Topology optimization Linear / convex problems Solution methods Optimality criteria Optimization algorithms Unconstrained problems Constrained problems

4
**Contents Unconstrained problems Transformation methods**

Existence of solutions, optimality conditions Nature of stationary points Global optimality

5
**Unconstrained Optimization**

Why? Elimination of active constraints unconstrained problem Develop basic understanding useful for constrained optimization Transformation of constrained problems into unconstrained problems Relevant engineering problems (potential energy minimization) Unconstrained problems are rare in practice, most practical optimum design problems involve restrictions of some sort. Transformation of constrained problems into unconstrained problems

6
**Transforming constrained problem**

Reformulation through barrier functions: g Transformation: Also called barrier functions. This is just one of the many possibilities. x f

7
Transformed problem g f

8
Transformed problem Barrier functions result in feasible, interior optimum: r = 100 An iterative optimization process is often used, where r is gradually increased. Note, recent research has also produced exact barrier functions. However, this is outside the scope of this course. r = 200 r = 400 r = 800

9
**Penalization Alternative reformulation: penalty functions g**

Transformation: f

10
Penalization (2) Penalty functions result in infeasible, exerior optimum: p = 40 p = 20 p = 10 p = 4

11
**Problem transformation summary**

Barrier function Penalty function Need feasible Yes No starting point? Nature of optimum Interior Exterior (feasible) (infeasible) Type of constraints g g, h Exact methods: see e.g.

12
**Unconstrained Optimization**

Why? Elimination of active constraints unconstrained problem Develop basic understanding useful for constrained optimization Transformation of constrained problems into unconstrained problems Relevant engineering problems (potential energy minimization) Unconstrained problems are rare in practice, most practical optimum design problems involve restrictions of some sort.

13
**Unconstrained engineering problem**

Example: displacement of loaded structure Fx = 5N Fy = 5N k1 = 8N/cm k2 = 1N/cm 10 cm x1 x2 Equilibrium: minimum potential energy

14
**Unconstrained engineering problem**

Potential energy: x1 x2 k1 Fy Fx k2 Equilibrium:

15
**Unconstrained engineering problem**

x1 x2 P x2 x1

16
**Contents Unconstrained problems Transformation methods**

Existence of solutions, optimality conditions Nature of stationary points Global optimality

17
**Theory for solving unc. problems**

Assumptions: Objective continuous and differentiable (C1) Domain closed and bounded (compact) ]a, b[ f

18
**Existence of minima Weierstrass Theorem:**

“A continuous function on a compact set has a maximum and a minimum in that set” Sufficient condition for existence! Interior optima only exist for non-monotonic functions

19
**One-dimensional case Calculus: conditions for minimum of f**

Derivative zero: Second derivative positive: (necessary) (sufficient) Interpretation through Taylor series: f x +h Emphasize that a Taylor series is a LOCAL approximation. The conditions apply to LOCAL optima. Second order derivative positive means that the gradient is increasing. Points with f’=0 are called stationary points. Stationary points with f’’>0 are local minima. -h

20
**One-dimensional case (2)**

Condition for minimum: f”(x) > 0 Other possibilities: f”(x) < 0 maximum f”(x) = 0 ? Check higher order derivatives

21
**Geometrical interpretation**

Positive locally increasing f x

22
**One-dimensional case (3)**

Possible situations for stationary points (f’ = 0): f x

23
**Example Aspirin pill revisited: worst pill ever!**

Maximize dissolving time minimize surface area r h Meaningful problems for surface minimization of a cylinder with a fixed volume could be e.g. a cooled tank (surface is proportional to heat transfer with environment) or building the cheapest container tank for e.g. oil storage. Equality constraint active eliminate h

24
Example (2)

25
**Multidimensional case**

Local approximation to multidimensional minimum by multidimensional Taylor series: Gradient Condition for minimum:

26
**Multidimensional case (2)**

For minimum, consider second-order approximation: Hessian

27
**Multidimensional case (3)**

Second order approximation: These are the optimality conditions for unconstrained optimization. Local minimum: First Order Necessity Condition: Second Order Sufficiency Condition:

28
**Positive definite Hessian positive definite**

Test for positive definiteness: Evaluate yTHy for all y (impractical) All eigenvalues li of H positive Sylvester’s rule: all determinants of H and its principal submatrices are positive

29
**Example Another loaded structure: Equilibrium:**

Fx Fy k1 k2 ux uy Another loaded structure: Equilibrium: First order necessity condition:

30
**Example (2) Second order sufficiency: H positive definite?**

In a separable minimization problem, the minimization can be performed separately for every variable (1-D). Note: H diagonal constant matrix. Separable objective function:

31
**Quadratic functions Polynomial terms up to 2nd order: General form:**

For geometrical interpretation of optimality conditions, consider quadratic functions. Note: 2nd order Taylor series is exact

32
**Quadratic functions (2)**

Transformation to symmetric matrix: Symmetric, since:

33
**Quadratic functions (2)**

Optimality conditions: Gradient: Hessian: The gradient is defined by matrix differentiation, see here: or other matrix calculus reference works. Stationary points:

34
**Hessian positive definite**

Example Consider Stationary point Hessian positive definite

35
Example (2) Result: minimum at (1.2, -2.6): f x2 x1

36
**Example: least squares**

Least squares fitting: unconstrained optimization problem Stationary point: x f

37
**Contents Unconstrained problems Transformation methods**

Existence of solutions, optimality conditions Nature of stationary points Global optimality

38
**Nature of stationary points**

Hessian H positive definite: Quadratic form Eigenvalues Local nature: minimum Both definitions of positive definiteness are equivalent Note positive curvatures Eigenvalues can be interpreted as curvatures.

39
**Nature of stationary points (2)**

Hessian H negative definite: Quadratic form Eigenvalues Local nature: maximum

40
**Nature of stationary points (3)**

Hessian H indefinite: Quadratic form Eigenvalues Local nature: saddle point

41
**Nature of stationary points (4)**

Hessian H positive semi-definite: Quadratic form Eigenvalues H singular! Local nature: valley

42
**Nature of stationary points (5)**

Hessian H negative semi-definite: Quadratic form Eigenvalues H singular! Local nature: ridge

43
**Stationary point nature summary**

Definiteness H Nature x* Positive d. Minimum Positive semi-d. Valley Indefinite Saddlepoint Negative semi-d. Ridge Negative d. Maximum

44
Structural example Compressed pin-jointed truss with rotational springs: F k1 k2 l Maximum (unstable) Minima (stable) Saddles (unstable)

45
**Contents Unconstrained problems Transformation methods**

Existence of solutions, optimality conditions Nature of stationary points Global optimality

46
**Global optimality Optimality conditions for unconstrained problem:**

First order necessity: (stationary point) Second order sufficiency: H positive definite at x* Optimality conditions only valid locally: local minimum Specifically, how to be sure a global optimum has been found. When can we be sure x* is a global minimum?

47
Convex functions Convex function: any line connecting any 2 points on the graph lies above it (or on it): Note difference between strictly convex and “normal” convex. Equivalent: tangent lines/planes stay below the graph H positive (semi-)definite f locally convex (proof by Taylor approximation)

48
**Convex domains Convex set:**

“A set S is convex if for every two points x1, x2 in S, the connecting line also lies completely inside S”

49
**Convexity and global optimality**

If: Objective f = (strictly) convex function Feasible domain = convex set (Ok for unconstrained optimization) Stationary point = (unique) global minimum Special case: f, g, h all linear linear optimization programming More general class: convex optimization

50
Example Quadratic functions with A positive definite are strictly convex: f x1 x2 Stationary point (1.2, -2.6) must be unique global optimum

51
**Summary optimality conditions**

Conditions for local minimum of unconstrained problem: First Order Necessity Condition: Second Order Sufficiency Condition: H positive definite For convex f in convex feasible domain: condition for global minimum: Sufficiency Condition:

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google