Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Optimization Anjela Govan North Carolina State University SAMSI NDHS Undergraduate workshop 2006.

Similar presentations


Presentation on theme: "Introduction to Optimization Anjela Govan North Carolina State University SAMSI NDHS Undergraduate workshop 2006."— Presentation transcript:

1 Introduction to Optimization Anjela Govan North Carolina State University SAMSI NDHS Undergraduate workshop 2006

2 What is Optimization? Optimization is the mathematical discipline which is concerned with finding the maxima and minima of functions, possibly subject to constraints.

3 Where would we use optimization? Architecture Nutrition Electrical circuits Economics Transportation etc.

4 What do we optimize? A real function of n variables with or without constrains

5 Unconstrained optimization

6 Optimization with constraints

7 Lets Optimize Suppose we want to find the minimum of the function

8 Review max-min for R 2 What is special about a local max or a local min of a function f (x)? at local max or local min f ’(x)=0 f ”(x) > 0 if local min f ”(x) < 0 if local max

9 Review max-min for R 3

10 Second Derivative Test Local min, local max, saddle point Gradient of f – vector (d f/ dx, d f /dy, d f /dz ) direction of fastest increase of f Global min/max vs. local min/max

11 Gradient Descent Method Examples Minimize function

12 Gradient Descent Method Examples Use function gd(alpha,x0)  Does gd.m converge to a local min? Is there a difference if  > 0 vs.  < 0?  How many iterations does it take to converge to a local min? How do starting points x0 affect number of iterations? Use function gd2(x0)  Does gd2.m converge to a local min?  How do starting points x0 affect number of iterations and the location of a local minimum?

13 How good are the optimization methods? Starting point Convergence to global min/max. Classes of nice optimization problems Example: f(x,y) = 0.5(  x 2 +y 2 ),  > 0 Every local min is global min.

14 Other optimization methods Non smooth, non differentiable surfaces  can not compute the gradient of f  can not use Gradient Method Nelder-Mead Method Others

15 Convex Hull A set C is convex if every point on the line segment connecting x and y is in C. The convex hull for a set of points X is the minimal convex set containing X.

16 Simplex A simplex or n-simplex is the convex hull of a set of (n +1). A simplex is an n- dimensional analogue of a triangle. Example:  a 1-simplex is a line segment  a 2-simplex is a triangle  a 3-simplex is a tetrahedron  a 4-simplex is a pentatope

17 Nelder-Mead Method n = number of variables, n+1 points form simplex using these points; convex hull move in direction away from the worst of these points: reflect, expand, contract, shrink Example:  2 variables  3 points  simplex is triangle  3 variables  4 points  simplex is tetrahedron

18 Nelder-Mead Method – reflect, expand

19 Nelder-Mead Method-reflect, contract

20 A tour of Matlab: Snapshots from the minimization After 0 steps

21 A tour of Matlab: Snapshots from the minimization After 1 steps

22 A tour of Matlab: Snapshots from the minimization After 2 steps

23 A tour of Matlab: Snapshots from the minimization After 3 steps

24 A tour of Matlab: Snapshots from the minimization After 7 steps

25 A tour of Matlab: Snapshots from the minimization After 12 steps

26 A tour of Matlab: Snapshots from the minimization After 30 steps (converged)

27 fminsearch function parameters: q =[C,K] cost function: Minimize cost function [q,cost]= q0,[],time,y_tilde)

28 Our optimization problem In our problem  Our function:  cost function “lives” in R 3  2 parameters C and K, n=2  Simplex is a triangle

29 Done!


Download ppt "Introduction to Optimization Anjela Govan North Carolina State University SAMSI NDHS Undergraduate workshop 2006."

Similar presentations


Ads by Google