Chapter 9 Function Approximation 고려대학교 컴퓨터학과 음성정보처리 연구실 2003010594 조영규 2004020594 방규섭 2005020594 정재연

Contents 9.1 Least Squares Approximation 9.2 Continuous Least Squares 9.3 Function Approximation at a Point 9.4 Using Matlab’s Functions 음성정보처리연구실

9.1 Least Squares Approximation Some of the most common methods of approximating data are based on the desire to minimize some measure of the difference between the approximating function and the given data points. The method of least squares seeks to minimize the sum of the squares of the differences between the function value and the data value. Advantages to using the square of the differences at each point Positive differences do not cancel negative differences Differentiation is not difficult Small differences become smaller and large differences are magnified 음성정보처리연구실

9. 1. 1 Linear Least Squares Approximation - Example 9 9.1.1 Linear Least Squares Approximation - Example 9.1 Linear Approximation to Four Points 음성정보처리연구실

9.1.1 Linear Least Squares Approximation - Discussion (1/2) Normal equation for linear least square approximation (1/2) 음성정보처리연구실

9.1.1 Linear Least Squares Approximation - Discussion (2/2) Normal equation for linear least square approximation (2/2) Matlab Function for linear Least Squares Approximation 음성정보처리연구실

Example 9.2 Least Squares Straight Line to Fit Four Data Points 음성정보처리연구실

Example 9.3 Noisy Straight-Line Data 음성정보처리연구실

9.1.2 Quadratic Least Squares Approximation - Discussion (1/2) Normal equation for Quadratic least square approximation 음성정보처리연구실

9.1.2 Quadratic Least Squares Approximation - Discussion (2/2) Matlab Function for Quadratic Least Squares Approximation 음성정보처리연구실

Example 9.6 Oil Reservoir Using the Matlab function for quadratic least squares, we find that the normal equations are Az=b, with The solution is 음성정보처리연구실

9.1.3 Cubic Least Squares Approximation 음성정보처리연구실

Example 9.7 Cubic Least Squares 음성정보처리연구실

Example 9.8 Cubic Least Squares, Continued 음성정보처리연구실

9.1.4 Least Squares Approximation for Other Functional Forms If the data are best fit by an exponential function, it is convenient instead to fit the logarithm of the data by a straight line 음성정보처리연구실

Example 9.10 Least-Squares Approximation of a Reciprocal Relation The plot of the following data suggests that they could be fit by a function of the form y=1/(ax+b) x=[0 0.5 1 1.5 2] y=[1.00 0.50 0.30 0.20 0.20] 음성정보처리연구실

9.2 Continuous Least Squares To approximate the exact value of the function for all point in an interval The summations are replaced by the corresponding integrals To approximate a given function a quadratic function on the interval [0, 1] and [-1, 1], we minimize the following equation. 음성정보처리연구실

Example 9. 12 Continuous Least Squares To find the continuous least squares quadratic approximation to the exponential function on [-1, 1] The coefficient matrix 음성정보처리연구실

Example 9. 12 Continuous Least Squares (cont.) Exponential function 음성정보처리연구실

9.2.1 Continuous Least Squares with Orthogonal Polynomails The set of functions {f0, f1, f2, ….., fn} is linearly independent on the interval [a, b] A linear combination of the functions is the zero function only if all of the coefficients are zero In other words c0f0(x) + c1f1(x) + c2f2(x) + …. + cnfn(x) = 0 for all x in [a, b], then c0=c1=c2=….=cn=0. Example of linearly independent function f0=1, f1=x, f2=x2,…., fn=xn {p0, p1, p2, ….., pn} where pj is a polynomial of degree j {1, sin(x), cos(x), sin(2x), cos(2x),…., sin(nx), cos(nx)} The set of functions {f0, f1, f2, ….., fn} is orthogonal on [a, b] Specially, if the functions are orthogonal and dj =1 for all j, the functions are called orthonormal. 음성정보처리연구실

9.2.2 Gram-Schmidt process How to construct a sequence of polynomials that are arthogonal on the interval [a, b] Conditions Pn be a polynomial of degree n The coefficient of xn in pn be positive For n = 0, 1, 2, …. Starting by taking p0(x)=x>0 To satisfy condition 3 To construct p1(x), we begin by letting q1(x) = x + c1,0p0. q1(x) is orthogonal to p0 음성정보처리연구실

9.2.2 Gram-Schmidt process To satisfy condition 3, we normalize q1(x) To construct p2(x), we begin by letting q2(x) = x2 + c2,1p1(x) + x2,0p0(x). q2(x) is orthogonal to p1(x) Normalizing q2(x) 음성정보처리연구실

9.2.2 Gram-Schmidt process The process continues in the same manner, as we construct each higher degree polynomial in turn. Where, qn(x) = xn + cn,0p0(x) + cn,1p1(x) + ….. + cn,n-1pn-1(x) (The coefficients cn,0, … , cn,n-1 are found so that qn(x) is orthogonal to each of the previously generated polynomials) 음성정보처리연구실

9.2.3 Legendre Polynomials A function of x defined on any finite interval a≤ x≤ b To transformed to a function of t defined -1≤ t≤ 1 The continuous least squares function approximation of Legendre polynomials Legendre polynomials form an orthogonal set on [-1, 1] For normalization An alternative definition of the Legendre polynomials 음성정보처리연구실

9.2.4 Least Squares Approximation with Legendre Polynomials To find the quadratic least squares approximation to f(x) on the interval [-1, 1] , we need to determine the coefficient c0, c1, c2 that minimize Setting 음성정보처리연구실

9.2.4 Least Squares Approximation with Legendre Polynomials (cont.) In a similar manner, equations formed by setting and and The integrations on the left were performed earlier; the result is 음성정보처리연구실

Example 9.13 Least Squares Approximation Using Legendre Polynomials To find the quadratic least squares approximation to f(x) = ex on the interval [-1, 1]in terms of the Legendre polynomials g(x) = c0p0(x) + c1p1(x) + c2p2(x) where p0(x) = 1, p0(x) = x, p0(x) = x2 – 음성정보처리연구실

Example 9.13 Least Squares Approximation Using Legendre Polynomials Exponential function 음성정보처리연구실

Example 9.14 Pade Approximation of the Runge function Its first three derivatives at x =0 are f(0)=1, f′(0)=0, f″(0)=-50, f‴(0)=0 Taylor polynomial of the function (at x=0) To seed a rational-function representation, using k=3, m=1, and n=2 For k=3, the linear system of equation is 음성정보처리연구실

9.4 Using Matlab’s Functions % script for polynomial regression % generate data X = -3 : 0.1 : 3; y1 = sin(x); y2 = cos(2*x); yy = y1 + y2; y = 0.01*round(100*yy) % find least squares 6th-degree polynomial to fit data z = polyfit (x, y, 6) % evaluate t he polynomial p = polyval(z, x); % plot the polynomial plot(x, p) Hold on % plot the data plot (x, y, ‘+’) hold off 음성정보처리연구실