Download presentation

Presentation is loading. Please wait.

Published byDamien Daugherty Modified about 1 year ago

1
Introduction to Optimization Anjela Govan North Carolina State University SAMSI NDHS Undergraduate workshop 2006

2
What is Optimization? Optimization is the mathematical discipline which is concerned with finding the maxima and minima of functions, possibly subject to constraints.

3
Where would we use optimization? Architecture Nutrition Electrical circuits Economics Transportation etc.

4
What do we optimize? A real function of n variables with or without constrains

5
Unconstrained optimization

6
Optimization with constraints

7
Lets Optimize Suppose we want to find the minimum of the function

8
Review max-min for R 2 What is special about a local max or a local min of a function f (x)? at local max or local min f ’(x)=0 f ”(x) > 0 if local min f ”(x) < 0 if local max

9
Review max-min for R 3

10
Second Derivative Test Local min, local max, saddle point Gradient of f – vector (d f/ dx, d f /dy, d f /dz ) direction of fastest increase of f Global min/max vs. local min/max

11
Gradient Descent Method Examples Minimize function

12
Gradient Descent Method Examples Use function gd(alpha,x0) Does gd.m converge to a local min? Is there a difference if > 0 vs. < 0? How many iterations does it take to converge to a local min? How do starting points x0 affect number of iterations? Use function gd2(x0) Does gd2.m converge to a local min? How do starting points x0 affect number of iterations and the location of a local minimum?

13
How good are the optimization methods? Starting point Convergence to global min/max. Classes of nice optimization problems Example: f(x,y) = 0.5( x 2 +y 2 ), > 0 Every local min is global min.

14
Other optimization methods Non smooth, non differentiable surfaces can not compute the gradient of f can not use Gradient Method Nelder-Mead Method Others

15
Convex Hull A set C is convex if every point on the line segment connecting x and y is in C. The convex hull for a set of points X is the minimal convex set containing X.

16
Simplex A simplex or n-simplex is the convex hull of a set of (n +1). A simplex is an n- dimensional analogue of a triangle. Example: a 1-simplex is a line segment a 2-simplex is a triangle a 3-simplex is a tetrahedron a 4-simplex is a pentatope

17
Nelder-Mead Method n = number of variables, n+1 points form simplex using these points; convex hull move in direction away from the worst of these points: reflect, expand, contract, shrink Example: 2 variables 3 points simplex is triangle 3 variables 4 points simplex is tetrahedron

18
Nelder-Mead Method – reflect, expand

19
Nelder-Mead Method-reflect, contract

20
A tour of Matlab: Snapshots from the minimization After 0 steps

21
A tour of Matlab: Snapshots from the minimization After 1 steps

22
A tour of Matlab: Snapshots from the minimization After 2 steps

23
A tour of Matlab: Snapshots from the minimization After 3 steps

24
A tour of Matlab: Snapshots from the minimization After 7 steps

25
A tour of Matlab: Snapshots from the minimization After 12 steps

26
A tour of Matlab: Snapshots from the minimization After 30 steps (converged)

27
fminsearch function parameters: q =[C,K] cost function: Minimize cost function [q,cost]= q0,[],time,y_tilde)

28
Our optimization problem In our problem Our function: cost function “lives” in R 3 2 parameters C and K, n=2 Simplex is a triangle

29
Done!

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google