Download presentation

1
**Engineering Optimization**

Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1

2
**Recap / overview Optimization problem Definition Checking**

Negative null form Model Special topics Linear / convex problems Sensitivity analysis Topology optimization Solution methods Unconstrained problems Constrained problems Optimality criteria Optimality criteria Optimization algorithms Optimization algorithms

3
**Summary optimality conditions**

Conditions for local minimum of unconstrained problem: First Order Necessity Condition: Second Order Sufficiency Condition: H positive definite For convex f in convex feasible domain: condition for global minimum: Sufficiency Condition:

4
**Stationary point nature summary**

Definiteness H Nature x* Positive d. Minimum Positive semi-d. Valley Indefinite Saddlepoint Negative semi-d. Ridge Negative d. Maximum

5
Complex eigenvalues? Question: what is the nature of a stationary point when H has complex eigenvalues? Answer: this situation never occurs, because H is symmetric by definition. Symmetric matrices have real eigenvalues (spectral theory).

6
**Nature of stationary points**

Nature of initial position depends on load (buckling): F k1 k2 l

7
**Nature of stationary points (2)**

8
**Unconstrained optimization algorithms**

Single-variable methods 0th order (involving only f ) 1st order (involving f and f ’ ) 2nd order (involving f, f ’ and f ” ) Multiple variable methods

9
**Why optimization algorithms?**

Optimality conditions often cannot be used: Function not explicitly known (e.g. simulation) Conditions cannot be solved analytically Example: Stationary points:

10
**0th order methods: pro/con**

Weaknesses: (Usually) less efficient than higher order methods (many function evaluations) Strengths: No derivatives needed Work also for discontinuous / non- differentiable functions Easy to program Robust

11
**Minimization with one variable**

Why? Simplest case: good starting point Used in multi-variable methods during line search Setting: f x Model Optimizer Iterative process:

12
**Termination criteria Stop optimization iterations when:**

Solution is sufficiently accurate (check optimality criteria) Progress becomes too slow: Maximum resources have been spent The solution diverges Cycling occurs xa xb

13
**Brute-force approach Simple approach: exhaustive search**

Disadvantage: rather inefficient f x L0 n points: Final interval size = Ln

14
**Basic strategy of 0th order methods for single-variable case**

Find interval [a0, b0] that contains the minimum (bracketing) Iteratively reduce the size of the interval [ak, bk] (sectioning) Approximate the minimum by the minimum of a simple interpolation function over the interval [aN, bN] Sectioning methods: Dichotomous search Fibonacci method Golden section method

15
**Bracketing the minimum**

f x4 = x3+g2D x1 [a0, b0] x2 = x1+D x3 = x2+gD x Starting point x1, stepsize D, expansion parameter g: user-defined

16
Unimodality Bracketing and sectioning methods work best for unimodal functions: “An unimodal function consists of exactly one monotonically increasing and decreasing part”

17
**Dichotomous search Conceptually simple idea:**

Main Entry: di·chot·o·mous Pronunciation: dI-'kät-&-m&s also d&- Function: adjective : dividing into two parts Conceptually simple idea: Try to split interval in half in each step L0 a0 b0 L0/2 d << L0

18
**Dichotomous search (2) Interval size after 1 step (2 evaluations):**

Interval size after m steps (2m evaluations): Proper choice for d :

19
Dichotomous search (3) Example: m = 10 Ideal interval reduction m

20
**Sectioning - Fibonacci**

Situation: minimum bracketed between x1 and x3 : x4 x4 x1 x2 x3 Test new points and reduce interval Optimal point placement?

21
**Optimal sectioning Fibonacci method: optimal sectioning method Given:**

Initial interval [a0, b0] Predefined total number of evaluations N, or: Desired final interval size e

22
**Fibonacci sectioning - basic idea**

Start at final interval and use symmetry and maximum interval reduction: d << IN IN IN-1 = 2IN IN-2 = 3IN IN-3 = 5IN IN-4 = 8IN IN-5 = 13IN Yellow point is point that has been added in the previous iteration. Fibonacci number

23
**Sectioning – Golden Section**

For large N, Fibonacci fraction b converges to golden section ratio f ( …): Golden section method uses this constant interval reduction ratio f f 1

24
**Sectioning - Golden Section**

Origin of golden section: I1 I2 = fI1 I2 = fI1 I3 = fI2 Final interval:

25
**Comparison sectioning methods**

Ideal dichotomous interval reduction Fibonacci Golden section Evaluations N Dichotomous 12 Golden section 9 Fibonacci 8 (Exhaustive 99) Example: reduction to 2% of original interval: Conclusion: Golden section simple and near-optimal

26
**Quadratic interpolation**

Three points of the bracket define interpolating quadratic function: ai+1 bi+1 xnew New point evaluated at minimum of parabola: ai bi For minimum: a > 0! Shift xnew when very close to existing point

27
**Unconstrained optimization algorithms**

Single-variable methods 0th order (involving only f ) 1st order (involving f and f ’ ) 2nd order (involving f, f ’ and f ” ) Multiple variable methods

28
Cubic interpolation Similar to quadratic interpolation, but with 2 points and derivative information: ai bi

29
Bisection method Optimality conditions: minimum at stationary point Root finding of f ’ Similar to sectioning methods, but uses derivative: f f’ Interval is halved in each iteration. Note, this is better than any of the direct methods.

30
**Secant method f ’ Also based on root finding of f ’**

Uses linear interpolation f ’ Interval possibly even more than halved in each iteration. Best.

31
**Unconstrained optimization algorithms**

Single-variable methods 0th order (involving only f ) 1st order (involving f and f ’ ) 2nd order (involving f, f ’ and f ” ) Multiple variable methods

32
**Newton’s method Again, root finding of f ’**

Basis: Taylor approximation of f ’: Linear approximation New guess:

33
**Newton’s method f’ f’ Best convergence of all methods:**

xk+1 xk+1 xk+2 xk xk+2 xk Note, jumping from point to point, not contained in an interval. Dangerous, may diverge. Unless it diverges

34
**Summary single variable methods**

Bracketing + Dichotomous sectioning Fibonacci sectioning Golden ratio sectioning Quadratic interpolation Cubic interpolation Bisection method Secant method Newton method In practice: additional “tricks” needed to deal with: Multimodality Strong fluctuations Round-off errors Divergence 0th order 1st order 2nd order And many, many more!

Similar presentations

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google