1. Lecture 9 Interpolation and Splines

2. Lingo Interpolation – filling in gaps in data Find a function f(x) that 1) goes through all your data points 2) does something sensible in between

3. Lingo Splines – a broad class of ways of performing interpolation (we’ll get to the details, eventually)

4. Find a function f(x) that 1) goes through all your data points (observations) 2) does something sensible in between (prior information) Why not just use least-squares?

5. Remember this? m est = m A + M [ d obs – Gm A ] where M = [G T C d -1 G + C m -1 ] -1 G T C d -1

6. m – a vector of all the points at which you want to estimate the function, including the points for which you have observations d – a vector of just those points where you have observations So the equation Gm=d is very simple, a model parameter equals the data when the corresponding observation is available: … 0 … 0 1 0 … 0 … …mi……mi… …dj……dj… = Just a single “1” per row

7. You then implement a smoothness constraint through minimizing |Dm| 2, where D is some measure of the non-smoothness of m Thus m A = 0 and C m -1 =  2 D T D and C d =I … 0 … 1 -2 1 … 0 … D = One possibility is to use the finite- difference approximation of the second derivative

8. First derivative [dm/dx] i  (1/  x) m i – m i-1  m i – m i-1 Second derivative [d 2 m/dx 2 ] i  [dm/dx] i+1 - [dm/dx] i = m i+1 – m i – m i + m i-1 = m i+1 – 2m i + m i-1

9. example 101 equally spaced along the x-axis So 101 values of the function f(x) 40 of these values measured (the data, d) the rest are unknown Two prior information minimize 2 nd derivative for interior 99 x’s minimize 1 st derivative at left and right x’s (nice to have the same number of priors as unknowns, but not required)

10.  = 10 -6 data result f(x) x

11.  can be chosen by trail and error but usually the result fairly insensitive to , as long as its small

12.  varying over six orders of magnitude log 10 (Total Error) log 10 (  ) x f(x)

13. A purist might say that this is not really interpolation, because the curve goes through the data only in the limit  but for small  ’s the error is extremely small

14. an aside Construct an equation F m = h as follows: G  D d  m = then note [F T F] -1 F T h = [G T G+  2 D T D] -1 G T d so if you want, you can just append  D to the bottom of G and solve by simple least-squares

15. solved via solved via [F T F] -1 F T h [G T G+  2 D T D] -1 G T d exactly the same!

16. another reason to work with F m = h G  D d  m = both G and D, and therefore F, too, are mostly zero (that is, they’re sparse matrices) very efficient algorithms are available for solving Fm=h in the least-squares sense when F is a sparse matrix (note G T G and D T D are not as sparse as G or D and [G T G and D T D] -1 is not sparse at all)

17. 2D Example (here a sparse solver would really be useful, for the number of unknowns is very large)

18. 21 unknowns 21  21=441 unknowns 44 observed data

19. Prior information:  2 f = d 2 f/dx 2 + d 2 f/dy 2 = 0 in interior of the box n  f = 0 on edges of box … a generalization of the 1D case

20. results

21. comparison

22. one limitation of this method is that it is discrete it only gives the unknown function at specific, prescribed values of x i one might prefer to have an analytic formula for the value of the function at any x

23. LINGO an analytic formula that gives the value of the function at any x is called an interpolant

24. high order polynomial something that sound like a good idea but isn’t even though an N-1 polynomal can computed to pass through any N points

25. example: 10 th order polynomial fit to 11 points Big swings not what we hoped for

26. solution simple function e.g. a low order polynomial that is valid in some interval near x i obviously, we need many such polynomials to over the whole x-axis This approach is called a spline

27. we’ve all used one already - linear splines x xixi x i+1 yiyi y i+1 y in this interval y(x) = y i + (y i+1 -y i )  (x-x i )/(x i+1 -x i )

28. cubic splines – somewhat more complicated but a lot nicer … x xixi x i+1 yiyi y i+1 y cubic a+bx+cx 2 +dx 3 in this interval a different cubic in this interval

29. counting up unknowns … x xixi x i+1 yiyi y i+1 y four coefficients a, b, c, d in every interval unknowns N data N-1 intervals 4 coefficients per interval 4(N-1)=4N-4 coefficients total=4N-4 unknowns constraints curve goes thru point at end of its interval 2(N-1)=2N-2 dy/dx match at interior points N-2 constraints d 2 y/dx 2 match at interior points N-2 constraints d 2 y/dx 2 =0 at end points 2 constraints total: 4N-4 constraints

30. formulating the cubic spline problem in an efficient manner f(x) with N observations (x i, f i ) let h i =  x i = x i+1 - x i and  f i = f i+1 -f i S i (x) are cubic polynomials, one for each interval

31. Let the 2 nd derivatives have values d 2 S i /dx 2 =y” i at the left hand end of its interval But since the second derivative is presumed continuous across intervals, d 2 S i /dx 2 =y” i+1 on the right hand side of its interval too. since S i (x) is a cubic, its second derivative is a linear function So within an interval d 2 S i /dx 2 varies linearly d 2 S i /dx 2 = y” i (x j+1 -x)/h j + y” i+1 (x-x j )/h j we’ll wind up solving for these y” i ’s

32. now integrate twice to get S i (x) d 2 S i /dx 2 = y” i (x j+1 -x)/h j + y” i+1 (x-x j )/h j S i (x) = y” i (x j+1 -x) 3 /(6h j ) + y” i+1 (x-x j ) 3 /(6h j ) + c i (x-x i ) + d i (x i+1 -x) where c i and d i are integration constants

33. now choose the integration constants c i and d i such that the the cubic goes through the data points. That is, S i (x i )=f i and S i (x i+1 )=f i+1 this leads to c i = f i+1 /h i – y” i+1 h i /6 d i = f i /h i – y” i h i /6

34. so S i (x) = y” i (x j+1 -x) 3 /(6h j ) + y” i+1 (x-x j ) 3 /(6h j ) + {f j+1 /h i – y” i+1 h i /6}(x-x i ) + {f i /h i – y” i h i /6}(x i+1 -x) where c i and d i are integration constants but we still haven’t implemented the continuity of dS/dx condition …

35. so we compute the derivative S’ i (x) = dS i /dx = y” i (x i+1 -x) 2 /(2h i ) + y” i+1 (x-x i ) 2 /(2h i ) + {f i+1 /h i – y” i+1 h i /6} - {f i /h i – y” i h i /6} = ½y” i (x i+1 -x) 2 h i + ½y” i+1 (x-x i ) 2 /h i +  f i+1 /h i – (y” i+1 -y” i )h i /6

36. now require the first derivative match across intervals: S i-1 ’(x i )=S i ’(x i ) this leads to an equation for the unknown y” i h i-1 y” i-1 + 2(h i +h i-1 )y” i + h i y” i+1 = b j with b j = 6  f i /h i – 6  f i-1 /h i-1

37. the equation for the unknown y” i h i-1 y” i-1 + 2(h i +h i-1 )y” i + h i y” i+1 = b i is just a matrix equation i=1: h 0 y” 0 + 2(h 1 +h 0 )y” 1 + h 1 y” 2 = b 1 i=2: h 1 y” 1 + 2(h 2 +h 1 )y” 2 + h 2 y” 2 = b 2 i=2: h 2 y” 2 + 2(h 3 +h 2 )y” 3 + h 3 y” 3 = b 3 … i=N-1 h N-2 y” N-2 + 2(h N-1 +h N-2 )y” N-1 + h N-1 y” N = b N-1 i=N h N-1 y” N-1 + 2(h N +h N-1 )y” N + h N y” N+1 = b N = we’ll discuss the issue raise by y” 0 and y” N+1 in a moment

38. the matrix equation, with g i =2(h i +h i-1 ), is h 0 g 1 h 1 h 1 g 2 h 2 h 2 g 3 h 3 … h N-2 g N-1 h N-1 h N-1 g N h N y” 0 y” 1 y” 2 y” 3 … y” N y” N+1 b 0 b 1 b 2 b 3 … b N b N+1 = A Tridiagonal Matrix, by the way. Very fast solvers are available …

39. the matrix equation, with g i =2(h i +h i-1 ), is h 0 g 1 h 1 h 1 g 2 h 2 h 2 g 3 h 3 … h N-2 g N-1 h N-1 h N-1 g N h N y” 0 y” 1 y” 2 y” 3 … y” N y” N+1 b 0 b 1 b 2 b 3 … b N b N+1 = I’ve written this as if there were two extra points, one to the left of the first point and one to the right of the last point. Of course, there aren’t. The way to handle this is to prescribe y” 0 and y” N+1 and move them to the r.h.s. of the equation. N N+2

40. moving over these two now-specified unknowns g 1 h 1 h 1 g 2 h 2 h 2 g 3 h 3 … h N-1 g N h N h N g N y” 1 y” 2 y” 3 … y” N b 1 – h 0 y” 0 b 2 b 3 … b N – h N+1 y” N+1 = we can set y” 0 and y” N to whatever we want. A simple choice is zero, in which case the splines are called natural cubic splines N N

41. example of cubic spline interpolation

42. very easy in MatLab new_y = spline(x,y,new_x);