1. Computational Methods CMSC/AMSC/MAPL 460 Polynomial Interpolation Ramani Duraiswami, Dept. of Computer Science

2. Interpolation Given a function at N points, find its value at other point(s) –“within” the points Interpolation, Imputation –“outside” the points Extrapolation, Forecasting How do we extend the value from known points to unknown points? –Have to have prior knowledge about the function –Can do what is convenient Occam’s razor –Parsimony: “One should not increase, beyond what is necessary, the number of entities required to explain anything” –Key in all scientific modeling

3. Function representation: in a “function space”

4. Solving interpolation equations

5. Polynomial interpolation

6. Polynomial Facts A polynomial of degree k has at most k distinct zeroes, unless it is identically zero Sum of two polynomials of degree k is another polynomial of degree at most k. Polynomials can be expressed in many ways –Degree of basis functions is 2, 1 and 0 … –Basis 3 is the power basis or monomial basis –Any basis can be used … often “orthogonal polynomials” are used

7. A polynomial can be expressed in many ways

8. Interpolation Weierstrass theorem Provides a guarantee that polynomials can interpolate any function On the other hand does not tell us how to choose the polynomial Also does not guarantee that the polynomial will actually “interpolate” the function … only that it will be within ε. Does not tell what the degree of the polynomial is

9. Uniqueness of interpolant We know that the polynomial exists Suppose that there are two different polynomials that can interpolate the data Let them be p n-1 and q n-1. So we have p n-1 (x i ) = y i, i=1, …n q n-1 (x i ) = y i, i=1, …n So p n-1 (x i ) -q n-1 (x i )= 0, i=1, …n p n-1 -q n-1 is the difference of two polynomials of degree n-1. It has n zeroes. Recall polynomial of degree k has at most k zeroes, or is the zero polynomial. Here we have more zeroes than degree … so it is the zero polynomial So interpolant is unique

10. Interpolating polynomials in power form Given n values of the function y i at points x i Fit a polynomial P(x) that interpolates data at these points If we have n points to interpolate, then a polynomial of degree n-1is We can write the condition that it interpolate as a linear system

11. Vandermonde matrices Matlab code In matlab Vandermonde matrix is defined in flipped order using the function vander Example … Also there is a function to fit polynomials to data, polyfit Vandermonde matrices are nonsingular if the points are distinct However they can be very poorly conditioned

12. Computing interpolants by hand Suppose we have data at n points and we have fit a polynomial How can we do this fit efficiently? Fit for one point is y=y 1 Fit for two points can be written as y=a(x-x 1 )+b(x-x 2 ) Fit for three points can be written as y=a(x-x 1 )(x-x 2 )+b(x-x 1 )(x-x 3 )+c(x-x 2 )(x-x 3 ) And so on … Advantage: each coefficient can be calculated independent of others –Why? –What is the form of the coefficient computed?

13. Lagrange and Newton forms for interpolations Lagrange and Newton modified these forms further to conveniently compute polynomials

14. Lagrange Polynomials Summation of terms, such that: –Equal to f() at a data point. –Equal to zero at all other data points. –Each term is a n th - degree polynomial

15. Lagrange form x0=1, f(x0)=-5 ;x1=2, f(x1)= -3;x2=3, f(x2)=2; Define L 0 =(x-2)(x-3)/(1-2)(1-3)= (x-2)(x-3)/2 L 1 =(x-1)(x-3)/(2-1)(2-3)= -(x-1)(x-3) L 2 =(x-1)(x-2)/(3-1)(3-2)= (x-1)(x-2)/2 Polynomial is -5 L 0 - 3 L 1 + 2 L 2 -5/2 (x-2)(x-3) + 3 (x-1)(x-3) +(x-1)(x-2)

16. Newton Interpolation Consider our data set of n+1 points y i =f(x i ) at x 0,x 1 …x i,…x n : x n > x 0 Since p n (x) is the unique polynomial p n (x) of order n, write it: f[x i,x j ] is a first divided difference f[x 2,x 1,x 0 ] is a second divided difference, etc. Efficient way of adding points to the interpolation! Used to fit data to a table

17. Newton Interpolation Example x0=1, f(x0)=-5 ;x1=2, f(x1)= -3; x2=3, f(x2)=2;x3=4, f(x3)=4. Build divided difference table f [x0]=-5 f [x1]=-3 f [x0,x1] = (-3+5)/1=2 f [x2]= 2 f [x1,x2] = (2+3)/(3-2)=5 f[x0,x1,x2] = (5-2)/(3-1) = 3/2 f [x3]= 4 f[x2,x3] = 2 f[x1,x2,x3] =-3/2 f[x0,x1,x2,x3]= (3/2+3/2)/(1-4)=-1 To compute Newton form we need f[x0], f[x0,x1], f[x0,x1,x2], f[x0,x1,x2,x3]

18. Newton form Interpolation P(x)=f[x0]+f[x0,x1] (x-x0) + f[x0,x1,x2] (x-x0)(x-x1) + f[x0,x1,x2,x3](x-x0)(x-x1)(x-x2) P(x)=-5+2 (x-1) + 3/2 (x-1)(x-2) -(x-1)(x-2)(x-3)

19. Error Define the error term as: If f(x) is an n th order polynomial p n (x) is of course exact. Otherwise, since there is a perfect match at x 0, x 1,…,x n This function has at least n+1 roots at the interpolation points.

20. Interpolation Errors Suppose we want to measure error at a point x To make polynomial go through x, add to existing polynomial divided difference term. This is the error we make using existing polynomial Comparing with Taylor series

21. Interpolation Errors Looks a bit like Taylor series remainder Recall, first n+1 terms of the Taylor Series is also an n th degree polynomial.

22. Interpolation: the story so far Given a function at N points, find its value at other point(s) So far: polynomial interpolation –Polynomials are guaranteed to approximate any given function in an interval as accurately as we want Different polynomial bases –Monomial or Power basis –Newton and Lagrange basis For a given set of points and function values –interpolating polynomial is unique Interpolation problem requires solution of a linear system –System is dense for Monomial/Power basis –Newton and Lagrange forms allow the direct solution of the polynomial interpolation form –Newton form particularly convenient to add new values Error for interpolation with n points is related to the value of the (n+1) th derivative of the underlying function

23. Polyinterp Lagrange interpolation code –x,y are points and function values –u are points where value is needed function v = polyinterp(x,y,u) n = length(x); v = zeros(size(u)); for k = 1:n %Lagrange function k at u w = ones(size(u)); for j = [1:k-1 k+1:n] w = (u-x(j))./(x(k)-x(j)).*w; end v = v + w*y(k); end Cost: 2 nested loops, so the cost is n 2. k = 5, n = 9 j= [1:k-1 k+1:n] j = 1 2 3 4 6 7 8 9

24. Examples of polynomial interpolation Go to MATLAB demo –Vandermonde –Polynomial interpolation for small set –For larger set See that even for six points we have a problem –In between the data points, (especially in first and last subintervals), function shows excessive variation. –overshoots changes in the data values. –As a result, full-degree polynomial interpolation is hardly ever used for data and curve fitting. However we saw polynomial interpolation works well when degree is low

25. Piecewise linear interpolation Simple idea –Connect straight lines between data points –Any intermediate value read off from straight line The local variable, s, is s = x - x k The first divided difference is δ k = (y k+1 – y k )/(x k+1 –x k ) With these quantities in hand, the interpolant is L(x) = y k + (x – x k ) (y k+1 – y k )/(x k+1 –x k ) = y k + sδ k Linear function that passes through (x k, y k ) and (x k+1, y k+1 )

26. Piecewise linear interpolation Same format as all other interpolants Function diff finds difference of elements in a vector Find appropriate sub-interval Evaluate Jargon: x is called a “knot” for the linear spline interpolant function v = piecelin(x,y,u) %PIECELIN Piecewise linear interpolation. % v = piecelin(x,y,u) finds piecewise linear L(x) % with L(x(j)) = y(j) and returns v(k) = L(u(k)). % First divided difference delta = diff(y)./diff(x); % Find subinterval indices k so that x(k) <= u < x(k+1) n = length(x); k = ones(size(u)); for j = 2:n-1 k(x(j) <= u) = j; end % Evaluate interpolant s = u - x(k); v = y(k) + s.*delta(k);

27. So we can reduce error by choosing small intervals where 2 nd derivative is higher –If we can choose where to sample data –Do more where the “action” is more

28. Piecewise Cubic interpolation While we expect function not to vary, we expect it to also be smooth So we could consider piecewise interpolants of higher degree How many pieces of information do we need to fit a cubic between two points? (x0,y0;x1,y1) –y=a+bx+cx 2 +dx 3 = y0+ b(x-x0) + c (x-x0)^3 + d (x-x1)^3 –4 coefficients –Need 4 pieces of information –2 values at end points –Need 2 more pieces of information –Derivatives?

29. Cubic splines

30. Imposing the continuity conditions

31. Using function continuity

32. First Derivative continuity

33. Second derivative continuity

34. Solving for m n-2 equations in n unknowns

35. Need to add two conditions Usually at end points

36. Solving a cubic spline system Assume natural splines This is a tridiagonal system Can be solved in O(n) operations How? –Do LU and solve –With tridiagonal structure requires O(7n) operations

37. However for Hermite, the derivative needs to be specified Cubic splines, the derivative is not specified but enforced

38. Efficient polynomial evaluation Given a polynomial in power form how many operations does it take to evaluate it? a p x p +  + a 1 x + a 0 Horner’s Rule Horner’s rule (Horner, 1819) recursively evaluates the polynomial a p x p +  + a 1 x + a 0 as: ((  (a p x + a p-1 )x+  )x + a 0. costs p multiplications and p additions, no extra storage. Reduces complexity from O(p 2 ) to O(p)

39. Interpolation: wrap up Interpolation: Given a function at N points, find its value at other point(s) Polynomial interpolation –Monomial, Newton and Lagrange forms Piecewise polynomial interpolation –Linear, Hermite cubic and Cubic Splines Polynomial interpolation is good at low orders However, higher order polynomials “overfit” the data and do not predict the curve well in between interpolation points Cubic Splines are quite good in smoothly interpolating data