2 Fminsearch uses Nelder Mead Fminsearch finds minimum of a function of several variables starting from an initial value.Unconstraint nonlinear optimization method, meaning we cannot give upper or lower bounds for parametersDownhill simplex methodGlobal optimization method (finds global minimum)
3 Nelder MeadThe Nelder–Mead technique is a heuristic search method that can converge to non-stationary points. But it is easy to use and will converge for a large class of problems.The Nelder–Mead technique was proposed by John Nelder & Roger Mead (1965).The method uses the concept of a simplex, which is a special polynomium type with N + 1 vertices in N dimensions.Examples of simplexes include a line segment on a line, a triangle on a plane, a tetrahedron in three-dimensional space and so forth.
4 The Nelder-Mead Algorithm Given n+1 vertices xi, i=1… n+1 and associated function values f(xi).Define the following coefficientsR=1 (reflection)K=0.5 (contraction)E=2 (expansion)S=0.5 (shrinkage)
5 The Nelder-Mead Algorithm Sort by function value: Order the vertices to satisfy f1 < f2 < … < fn+1Calculate xm = sum xi (the average of all the points except the worst)Reflection. Compute xr = xm + R(xm-xn+1) and evaluate f(xr). If f1 < fr < fn accept xr and terminate the iteration.
6 The Nelder-Mead Algorithm Expansion. If fr < f1 calculate xe = xm+ K (xr - xm) and evaluate f(xe). If fe < fr, accept xe; otherwise accept xr. Terminate the iteration.
7 The Nelder-Mead Algorithm Contraction. If fr > fn, perform a contraction between xm and the better of xr and xn+1.Outside. If fn < fr < fn+1 calculate xoc= xm+ K (xr - xm) and evaluate f(xoc). If foc< fr, accept xoc and terminate the iteration; otherwise do a shrink.Inside. If fr > fn+1 calculate xic = xm – K (xm- xn+1) and evaluate f(xic). If fic< fn+1 accept xic and terminate the iteration; otherwise do a shrink.
8 The Nelder-Mead Algorithm Shrink. Evaluate f at the n points vi = xi + S (xi-x1), i = 2,….,n+1. The vertices of the simplex at the next iteration are x1, v2, …, vn+1.