Selected from presentations by Jim Ramsay, McGill University, Hongliang Fei, and Brian Quanz Basis Basics

1. Introduction Basis: In Linear Algebra, a basis is a set of vectors satisfying:  Linear combination of the basis can represent every vector in a given vector space;  No element of the set can be represented as a linear combination of the others.

In Function Space, Basis is degenerated to a set of basis functions; Each function in the function space can be represented as a linear combination of the basis functions. Example: Quadratic Polynomial bases {1,t,t^2}

What are basis functions? We need flexible method for constructing a function f(t) that can track local curvature. We pick a system of K basis functions φ k (t), and call this the basis for f(t). We express f(t) as a weighted sum of these basis functions: f(t) = a 1 φ 1 (t) + a 2 φ 2 (t) + … + a K φ K (t) The coefficients a 1, …, a K determine the shape of the function.

What do we want from basis functions? Fast computation of individual basis functions. Flexible: can exhibit the required curvature where needed, but also be nearly linear when appropriate. Fast computation of coefficients a k : possible if matrices of values are diagonal, banded or sparse. Differentiable as required: We make lots of use of derivatives in functional data analysis. Constrained as required, such as periodicity, positivity, monotonicity, asymptotes and etc.

What are some commonly used basis functions? Powers: 1, t, t 2, and so on. They are the basis functions for polynomials. These are not very flexible, and are used only for simple problems. Fourier series: 1, sin(ωt), cos(ωt), sin(2ωt), cos(2ωt), and so on for a fixed known frequency ω. These are used for periodic functions. Spline functions: These have now more or less replaced polynomials for non-periodic problems. More explanation follows.

What is Basis Expansion? Given data X and transformation Then we model as a linear basis expansion in X, where is a basis function.

Why Basis Expansion? In regression problems, f(X) will typically nonlinear in X; Linear model is convenient and easy to interpret; When sample size is very small but attribute size is very large, linear model is all what we can do to avoid over fitting.

2. Piecewise Polynomials and Splines Spline:  In Mathematics, a spline is a special function defined piecewise by polynomials;  In Computer Science, the term spline more frequently refers to a piecewise polynomial (parametric) curve. Simple construction, ease and accuracy of evaluation, capacity to approximate complex shapes through curve fitting and interactive curve design.

Assume four knots spline (two boundary knots and two interior knots), also X is one dimensional. Piecewise constant basis: Piecewise Linear Basis:

Basis functions: Six functions corresponding to a six- dimensional linear space. Piecewise Cubic Polynomial

14 Spline Interpolation Method Slides taken from the lecture by Authors: Autar Kaw, Jai Paul

What is Interpolation ? Given (x 0,y 0 ), (x 1,y 1 ), …… (x n,y n ), find the value of ‘y’ at a value of ‘x’ that is not given.

Interpolants Polynomials are the most common choice of interpolants because they are easy to: Evaluate Differentiate, and Integrate.

Why Splines ?

Why Splines ? Figure : Higher order polynomial interpolation is a bad idea

Linear Interpolation

Linear Interpolation (contd)

Example The upward velocity of a rocket is given as a function of time in Table 1. Find the velocity at t=16 seconds using linear splines. Table Velocity as a function of time Figure. Velocity vs. time data for the rocket example (s) (m/s)

Linear Interpolation

Quadratic Interpolation

Quadratic Interpolation (contd)

Quadratic Splines (contd)

Quadratic Splines (contd)

Quadratic Splines (contd)

Quadratic Spline Example The upward velocity of a rocket is given as a function of time. Using quadratic splines a) Find the velocity at t=16 seconds b) Find the acceleration at t=16 seconds c) Find the distance covered between t=11 and t=16 seconds Table Velocity as a function of time Figure. Velocity vs. time data for the rocket example (s) (m/s)

Solution Let us set up the equations

Each Spline Goes Through Two Consecutive Data Points

tv(t) sm/s Each Spline Goes Through Two Consecutive Data Points

Derivatives are Continuous at Interior Data Points

Derivatives are continuous at Interior Data Points At t=10 At t=15 At t=20 At t=22.5

Last Equation

Final Set of Equations

Coefficients of Spline iaiai bibi cici − − − −152.13

Quadratic Spline Interpolation Part 2 of 2

38 Final Solution

Velocity at a Particular Point a) Velocity at t=16

Quadratic Spline Graph t=a:2:b;

Quadratic Spline Graph t=a:0.5:b;

Natural Cubic Spline Interpolation The domain of S is an interval [a,b]. S, S’, S’’ are all continuous functions on [a,b]. There are points t i (the knots of S) such that a = t 0 < t 1 <.. t n = b and such that S is a polynomial of degree at most k on each subinterval [t i, t i+1 ]. SPLINE OF DEGREE k = 3 ynyn …y1y1 y0y0 y tntn …t1t1 t0t0 x t i are knots

Natural Cubic Spline Interpolation S i (x) is a cubic polynomial that will be used on the subinterval [ x i, x i+1 ].

Natural Cubic Spline Interpolation S i (x) = a i x 3 + b i x 2 + c i x + d i 4 Coefficients with n subintervals = 4n equations There are 4 n-2 conditions Interpolation conditions Continuity conditions Natural Conditions S’’(x 0 ) = 0 S’’(x n ) = 0