6.5 Taylor Series Linearization writing out the Taylor series expansion simplification using the b and a notation expansion for the Gaussian peak example iterative solution of the matrix least-squares errors in the coefficients are given by the a-1 variance/covariance matrix tolerance toward initial guesses improving performance with a second-order expansion 6.5 : 1/8

The Approach Linearize the function locally by Taylor series expansion about a set of guess values for the coefficients. Use the linear least squares matrix method to calculate b and a and solve for a. Use the values of a as the new guess values for another Taylor series expansion Iterate to convergence. 6.5 : 1/8

Linearization of Coefficients Consider a function non-linear in its coefficients, y = f(x,a), where a is a vector of coefficients: a0, a1, ... , an. The function can be linearized by a first-order Taylor series expansion about the true coefficient values. Let the true values be denoted as m0, m1, ... , mn, and be represented by the vector, m. Simplify this expression by using the following variable changes. The linear approximation can now be written in a compact form. 6.5 : 2/8

We have seen this problem before! 6.5 : 1/8

Iterative Matrix Solution By defining, it is possible to recover a matrix formalism, where a-1 is the variance/covariance matrix. If the parameter values at the minimum in chi-square were substituted for m, the vector d would contain all zeros. In practice the parameters are estimated and used to construct an initial guess at the m vector. The expansion is performed around the guesses. The least-squares solution then yields a non-zero value for d. These two vectors are then used to construct a refined set of guesses, m'. The new vector, m', is used as a new set of guesses and the matrix calculation is repeated. The process stops when all dr ~ 0. 6.5 : 3/8

Example Gaussian Linearization Consider the Gaussian example we have been using, where a is a vector containing the coefficients. The function y(x,a) is used to compute Dy given on an earlier slide. The calculation of the a matrix and b vector require the following partial derivatives. 6.5 : 4/8

Iterative Solution The initial guesses used with the grid and gradient search algorithms were used. They appear in the table below as iteration 1. The computed parameters were then used as guesses for the second computation. They are iteration 2. Etc. a Iteration 1 2 3 4 g0 d0 2.000 +0.137 2.137 +0.004 2.141 0.000 g1 d1 51.000 -0.058 50.942 -0.005 50.937 +0.001 50.938 The final coefficients are essentially the same as those obtained with the grid and gradient search algorithms. 6.5 : 5/8

Coefficient Errors The error in the fit is given by the following with a0 = 2.141 and a1 = 50.938. There were eleven data values. The a-1 variance/covariance equation has the following two diagonal elements. The standard deviation of the two coefficients can now be calculated. These values are comparable to those obtained by fitting chi-square space to a two-dimensional parabola: sa0 = 0.034, sa1 = 0.045. 6.5 : 6/8

Tolerance Toward Initial Guesses The Taylor series linearization does not tolerate arbitrarily bad initial guesses. Consider the example Gaussian. If the value of a1 is fixed at 50.938, the smallest value of a0 that converged to the minimum in chi-square space was 0.25. The largest value of a0 that converged was 5.1. Values smaller than 0.25 and larger than 5.1 gave iterative values that diverged. These values are shown below. In contrast, the gradient search converged with values as small as 0.1. Values as large as 1,000 converged, albeit, taking ~1 min. 6.5 : 7/8

Tolerance Toward Initial Guesses The curvature of c2-space dictates whether the Taylor series approach will converge or diverge. You may recall, the elements of the matrix a are essentially second derivatives of c2-space. When those invert in sign, the Taylor series approach leads away from the minimum rather than toward it. 6.5 : 7/8