1. Selected from presentations by Jim Ramsay, McGill University, Hongliang Fei, and Brian Quanz Basis Basics

2. 1. Introduction Basis: In Linear Algebra, a basis is a set of vectors satisfying:  Linear combination of the basis can represent every vector in a given vector space;  No element of the set can be represented as a linear combination of the others.

3. In Function Space, Basis is degenerated to a set of basis functions; Each function in the function space can be represented as a linear combination of the basis functions. Example: Quadratic Polynomial bases {1,t,t^2}

4. What are basis functions? We need flexible method for constructing a function f(t) that can track local curvature. We pick a system of K basis functions φ k (t), and call this the basis for f(t). We express f(t) as a weighted sum of these basis functions: f(t) = a 1 φ 1 (t) + a 2 φ 2 (t) + … + a K φ K (t) The coefficients a 1, …, a K determine the shape of the function.

5. What do we want from basis functions? Fast computation of individual basis functions. Flexible: can exhibit the required curvature where needed, but also be nearly linear when appropriate. Fast computation of coefficients a k : possible if matrices of values are diagonal, banded or sparse. Differentiable as required: We make lots of use of derivatives in functional data analysis. Constrained as required, such as periodicity, positivity, monotonicity, asymptotes and etc.

6. What are some commonly used basis functions? Powers: 1, t, t 2, and so on. They are the basis functions for polynomials. These are not very flexible, and are used only for simple problems. Fourier series: 1, sin(ωt), cos(ωt), sin(2ωt), cos(2ωt), and so on for a fixed known frequency ω. These are used for periodic functions. Spline functions: These have now more or less replaced polynomials for non-periodic problems. More explanation follows.

7. What is Basis Expansion? Given data X and transformation Then we model as a linear basis expansion in X, where is a basis function.

8. Why Basis Expansion? In regression problems, f(X) will typically nonlinear in X; Linear model is convenient and easy to interpret; When sample size is very small but attribute size is very large, linear model is all what we can do to avoid over fitting.

9. 2. Piecewise Polynomials and Splines Spline:  In Mathematics, a spline is a special function defined piecewise by polynomials;  In Computer Science, the term spline more frequently refers to a piecewise polynomial (parametric) curve. Simple construction, ease and accuracy of evaluation, capacity to approximate complex shapes through curve fitting and interactive curve design.

10. Assume four knots spline (two boundary knots and two interior knots), also X is one dimensional. Piecewise constant basis: Piecewise Linear Basis:

12. Basis functions: Six functions corresponding to a six- dimensional linear space. Piecewise Cubic Polynomial

14. http://numericalmethods.eng.usf.edu 14 Spline Interpolation Method Slides taken from the lecture by Authors: Autar Kaw, Jai Paul

15. http://numericalmethods.eng.usf.edu15 What is Interpolation ? Given (x 0,y 0 ), (x 1,y 1 ), …… (x n,y n ), find the value of ‘y’ at a value of ‘x’ that is not given.

16. http://numericalmethods.eng.usf.edu16 Interpolants Polynomials are the most common choice of interpolants because they are easy to: Evaluate Differentiate, and Integrate.

17. http://numericalmethods.eng.usf.edu17 Why Splines ?

18. http://numericalmethods.eng.usf.edu18 Why Splines ? Figure : Higher order polynomial interpolation is a bad idea

19. http://numericalmethods.eng.usf.edu19 Linear Interpolation

20. http://numericalmethods.eng.usf.edu20 Linear Interpolation (contd)

21. http://numericalmethods.eng.usf.edu21 Example The upward velocity of a rocket is given as a function of time in Table 1. Find the velocity at t=16 seconds using linear splines. Table Velocity as a function of time Figure. Velocity vs. time data for the rocket example (s) (m/s) 00 10227.04 15362.78 20517.35 22.5602.97 30901.67

22. http://numericalmethods.eng.usf.edu22 Linear Interpolation

23. http://numericalmethods.eng.usf.edu23 Quadratic Interpolation

24. http://numericalmethods.eng.usf.edu24 Quadratic Interpolation (contd)

25. http://numericalmethods.eng.usf.edu25 Quadratic Splines (contd)

26. http://numericalmethods.eng.usf.edu26 Quadratic Splines (contd)

27. http://numericalmethods.eng.usf.edu27 Quadratic Splines (contd)

28. http://numericalmethods.eng.usf.edu28 Quadratic Spline Example The upward velocity of a rocket is given as a function of time. Using quadratic splines a) Find the velocity at t=16 seconds b) Find the acceleration at t=16 seconds c) Find the distance covered between t=11 and t=16 seconds Table Velocity as a function of time Figure. Velocity vs. time data for the rocket example (s) (m/s) 00 10227.04 15362.78 20517.35 22.5602.97 30901.67

29. http://numericalmethods.eng.usf.edu29 Solution Let us set up the equations

30. http://numericalmethods.eng.usf.edu30 Each Spline Goes Through Two Consecutive Data Points

31. http://numericalmethods.eng.usf.edu31 tv(t) sm/s 00 10227.04 15362.78 20517.35 22.5602.97 30901.67 Each Spline Goes Through Two Consecutive Data Points

32. http://numericalmethods.eng.usf.edu32 Derivatives are Continuous at Interior Data Points

33. http://numericalmethods.eng.usf.edu33 Derivatives are continuous at Interior Data Points At t=10 At t=15 At t=20 At t=22.5

34. http://numericalmethods.eng.usf.edu34 Last Equation

35. http://numericalmethods.eng.usf.edu35 Final Set of Equations

36. http://numericalmethods.eng.usf.edu36 Coefficients of Spline iaiai bibi cici 1022.7040 20.88884.92888.88 3−0.135635.66−141.61 41.6048−33.956554.55 50.2088928.86−152.13

37. http://numericalmethods.eng.usf.edu37 Quadratic Spline Interpolation Part 2 of 2 http://numericalmethods.eng.usf.edu http://numericalmethods.eng.usf.edu

38. 38 Final Solution

39. http://numericalmethods.eng.usf.edu39 Velocity at a Particular Point a) Velocity at t=16

40. Quadratic Spline Graph t=a:2:b;

41. Quadratic Spline Graph t=a:0.5:b;

42. Natural Cubic Spline Interpolation The domain of S is an interval [a,b]. S, S’, S’’ are all continuous functions on [a,b]. There are points t i (the knots of S) such that a = t 0 < t 1 <.. t n = b and such that S is a polynomial of degree at most k on each subinterval [t i, t i+1 ]. SPLINE OF DEGREE k = 3 ynyn …y1y1 y0y0 y tntn …t1t1 t0t0 x t i are knots

43. Natural Cubic Spline Interpolation S i (x) is a cubic polynomial that will be used on the subinterval [ x i, x i+1 ].

44. Natural Cubic Spline Interpolation S i (x) = a i x 3 + b i x 2 + c i x + d i 4 Coefficients with n subintervals = 4n equations There are 4 n-2 conditions Interpolation conditions Continuity conditions Natural Conditions S’’(x 0 ) = 0 S’’(x n ) = 0