1. Lecture 15 Orthogonal Functions Fourier Series

2. LGA mean daily temperature time series is there a global warming signal?

3. Model that includes annual variability T(t) = a + bt + A 1 cos(2  f 1 t) + B 1 sin(2  f 1 t) + A 2 cos(2  f 2 t) + B 2 sin(2  f 2 t) + A 3 cos(2  f 3 t) + B 3 sin(2  f 3 t) + … withf 1 = 1 cycle per year f 2 = 2 cycles per year etc

4. Why both sines and cosines? cos{2  f 1 (t-t 0 )}

5. Why both sines and cosines? cos{2  f 1 (t-t 0 )} Cosine does not ‘start’ at t=0 But remember cos(a+b)=cos(a)cos(b)-sin(a)sin(b)

6. cos{2  f 1 (t-t 0 )} = cos(2  f 1 t 0 ) cos(2  f 1 t) – sin(2  f 1 t 0 ) sin(2  f 1 t) = A cos(2  f 1 t) + B sin(2  f 1 t) cos(a+b)=cos(a) cos(b) - sin(a) sin(b)

7. So using both sines and cosines moves the delay, t 0, out of the cosine, and into the coefficients of the sines and cosines. This trick ‘linearizes’ the unknown, t 0. cos{2  f 1 (t-t 0 )} = cos(2  f 1 t 0 ) cos(2  f 1 t) – sin(2  f 1 t 0 ) sin(2  f 1 t) = A cos(2  f 1 t) + B sin(2  f 1 t) cos(a+b)=cos(a) cos(b) - sin(a) sin(b)

8. Why more than one frequency? f 1 = 1 cycle per year f 2 = 2 cycles per year etc Allows us to represent non-sinusoidal shape of annual cycle.

9. cos(ft) 0.3cos(2ft) sum: cos(ft)+0.3cos(2ft) exactly periodic, but shape not exactly sinusoidal

10. Least-squares fit to LGA data (up to f 8 ) data fit constant term, a error of fit, e linear term, bt

11. Statistics of linear term, bt b = 0.31 degrees F per decade  d = [ e T e / N ] 1/2 = 7 deg F C m =  d 2 [G T G] -1  b = [  d 2 C m b,b ] 1/2 = 0.05 degrees F per decade 95% confidence b = 0.31±0.1 degrees F per decade So LGA is warming

12. sines and cosines are “orthogonal” functions T(t) = A 0 + A 1 cos(2  f 1 t) + B 1 sin(2  f 1 t) + A 2 cos(2  f 2 t) + B 2 sin(2  f 2 t) + A 3 cos(2  f 3 t) + B 3 sin(2  f 3 t) + … with f 2 =2f 1, f 3 =3f 1, etc Called a “Fourier Series”

13. 1 cos(2  f 1 t 1 ) sin(2  f 1 t 1 ) cos(2  f 2 t 1 ) sin(2  f 2 t 1 ) … 1 cos(2  f 1 t 2 ) sin(2  f 1 t 2 ) cos(2  f 2 t 2 ) sin(2  f 2 t 2 ) … 1 cos(2  f 1 t 3 ) sin(2  f 1 t 3 ) cos(2  f 2 t 3 ) sin(2  f 2 t 3 ) … 1 cos(2  f 1 t 4 ) sin(2  f 1 t 4 ) cos(2  f 2 t 4 ) sin(2  f 2 t 4 ) … 1 cos(2  f 1 t 5 ) sin(2  f 1 t 5 ) cos(2  f 2 t 5 ) sin(2  f 2 t 5 ) … 1 cos(2  f 1 t 6 ) sin(2  f 1 t 6 ) cos(2  f 2 t 6 ) sin(2  f 2 t 6 ) … … Standard least-squares G matrix G=G=

14. With the proper choice of f 1 the matrix G T G is diagonal dot product of any pair of columns of G is zero columns of G are orthogonal

15. The proper choice of f 1 Suppose the time-series is N data points long, with spacing  t. Then the lowest frequency must be f 1 = 1 / (N  t) one oscillation over the length of the time-series And the highest frequency must be f N/2 = 1 / (2  t) one-half oscillation per sampling interval

16. f 1 = 1 / (2N  t) f 1 = 1 / (2  t) note sine is zero

17. Count of unknowns The constant term, one unknown plus 2 coefficients per frequency, N/2 frequencies so N unknowns minus One unknown since the f N/2 term, which has no sine term equals N unknowns, same as number of data

18. MatLab Code N = 100; % times vector dt = 0.5; tmin = 0.0; t = tmin + dt*[0:N-1]'; tmax = tmin + dt*(N-1); df = 1/(N*dt); % frequency spacing M = N; % number of unknowns same as data G = zeros(N,M); % set up least-squares G matrix G(:,1)=ones(N,1); for p = 2*[1:M/2-1] G(:,p) = cos(pi*p*df*t); G(:,p+1) = sin(pi*p*df*t); end p=M/2; G(:,M) = cos(2*pi*p*df*t);

19. G T G for N=100 [G T G] 11 = [G T G] NN =N Other diagonal elements [G T G] ii =N/2 Off diagonal elements are zero

20. So least-squares solution is m = [G T G] -1 G T d = = diag( N -1, 2/N, … 2/N, N -1 ) G T d NO matrix inversion required!

21. G T G for N=100 Example: Neuse River Hydrograph (100 days) data, d d=Gm with m=[G T G] -1 G T d d=Gm with m=DG T d where D= diag( N -1, 2/N, … 2/N, N -1 )

22. “spectrum” amount of power at different frequencies s i 2 = A i 2 + B i 2 si2si2 fifi fpfp time-series has a lot of energy at frequency f p

23. Spectrum of Neuse data set for N=4380

24. Close up of low frequencies 12 mo 6 mo 4 mo3 mo2 mo Big annual cycle in Neuse hydrograph

25. Error Estimates for Fourier Series Assume uncorrelated, normally-distributed data, d, with variance  d 2 The problem Gm=d is linear, so the unknowns, m, (the coefficients of the cosines and sines, A i and B i ) are also normally-distributed. Since sines and cosines are orthogonal, G T G is diagonal and C m =  d 2 [G T G] -1 is diagonal, too So that m’s have uncorrelated errors. All but the first and last have variance  m 2 = 2  d 2 /N. The spectrum s i 2 =A i 2 +B i 2 is the sum of two uncorrelated, normally distributed random variables and is thus   2 - distributed. The   2 -distribution has a variance of 4, so that  s 2 = 8  d 2 /N

26. Switching to complex numbers nothing different in principle but calculations become easier

27. But first Lets switch to angular frequency measured in radians per second  i = 2  f i Beats writing all those 2  ’s !

28. Remember Euler’s formula exp( i  t ) = cos(  t ) + i sin(  t ) ?

29. exp( i  t ) = cos(  t ) + i sin(  t ) exp( -i  t ) = cos(  t ) - i sin(  t ) cos(  t ) = (1/2) [exp( i  t ) + exp( -i  t )] sin(  t ) = (1/2i) [exp( i  t ) - exp( -i  t )]

30. Let’s compare T(t) = A 0 cos(  0 t) + B 0 sin(  0 t) + A 1 cos(  1 t) + B 1 sin(  1 t) + A 2 cos(  2 t) + B 2 sin(  2 t) + A 3 cos(  3 t) + B 3 sin(  3 t) + … with T(t) =... + C -2 exp(-i  2 t) + C -1 exp(-  1 t) + C 0 exp(i  0 t) + C 1 exp(i  1 t) + C 2 exp(i  2 t) + C 3 exp(i  3 t) + … =1=0 with  p =p    p = -  p First, if T is real, then we must have C -p = C p * Then C -p exp(-  p t) + C p exp(  p t) = (C p r -i C p i ) [ cos(  p t) - i sin(  p t)] + (C p r +i C p i ) [ cos(  p t) + i sin(  p t)] = 2C p r cos(  p t) - 2 C p i sin(-  -p t)] So A p = 2C p r and B p = -2C p i So these two representations are equivalent

31. T(t) =... + C -2 exp(-i  2 t) + C -1 exp(-  1 t) + C 0 exp(i  0 t) + C 1 exp(i  1 t) + C 2 exp(i  2 t) + C 3 exp(i  3 t) + … … exp(-i  2 t 0 ) exp(-i  1 t 0 ) exp(i  0 t 0 ) exp( i  1 t 0 ) exp(i  2 t 0 ) … … exp(-i  2 t 1 ) exp(-i  1 t 1 ) exp(i  0 t 1 ) exp( i  1 t 1 ) exp( i  2 t 1 ) … … exp(-i  2 t 2 ) exp(-i  1 t 2 ) exp(i  0 t 2 ) exp( i  1 t 2 ) exp( i  2 t 2 ) … … exp(-i  2 t 3 ) exp(-i  1 t 3 ) exp(i  0 t 3 ) exp( i  1 t 3 ) exp( i  2 t 3 ) … … exp(-i  2 t 4 ) exp(-i  1 t 4 ) exp(i  0 t 4 ) exp( i  1 t 4 ) exp( i  2 t 4 ) … = Implies a simple form of the equation d=Gm … C -2 C -1 C 0 C 1 C 2 … T 0 T 1 T 2 T 3 T 4 …

32. Least-squares with complex numbers real numbers: given Gm =d minimize E=e T e implies m=[G T G] -1 G T d complex nos: given Gm =d minimize E=e H e where e H = e* T implies m=[G H G] -1 G H d The formula m=[G H G] -1 G H d is not hard to work out using the standard minimization procedure, but we don’t have time to work it out in class. The Hermitian transpose, that is, the transpose of the complex conjugate.

33. … exp(-i  2 t 0 ) exp(-i  1 t 0 ) exp(i  0 t 0 ) exp(i  1 t 0 ) exp(i  2 t 0 ) … … exp(-i  2 t 1 ) exp(-i  1 t 1 ) exp(i  0 t 1 ) exp(i  1 t 1 ) exp(i  2 t 1 ) … … exp(-i  2 t 2 ) exp(-i  1 t 2 ) exp(i  0 t 2 ) exp(i  1 t 2 ) exp(i  2 t 2 ) … … exp(-i  2 t 3 ) exp(-i  1 t 3 ) exp(i  0 t 3 ) exp(i  1 t 3 ) exp(i  2 t 3 ) … … exp(-i  2 t 4 ) exp(-i  1 t 4 ) exp(i  0 t 4 ) exp(i  1 t 4 ) exp(i  2 t 4 ) … = … C -2 C -1 C 0 C 1 C 2 … T 0 T 1 T 2 T 3 T 4 … … exp(i  2 t 0 ) exp(i  2 t 1 ) exp(i  2 t 2 ) exp(i  2 t 3 ) exp(i  2 t 4 ) … … exp(i  1 t 0 ) exp(i  1 t 1 ) exp(i  1 t 2 ) exp(i  1 t 3 ) exp(i  1 t 4 ) … … exp(i  0 t 0 ) exp(i  0 t 1 ) exp(i  0 t 2 ) exp(i  0 t 3 ) exp(i  0 t 4 ) … … exp(-i  1 t 0 ) exp(-i  1 t 1 ) exp(-i  1 t 2 ) exp(-i  1 t 3 ) exp(-i  1 t 4 ) … … exp(-i  2 t 0 ) exp(-i  2 t 1 ) exp(-i  2 t 2 ) exp(-i  2 t 3 ) exp(-i  2 t 4 ) … =N -1 … C -2 C -1 C 0 C 1 C 2 … T 0 T 1 T 2 T 3 T 4 … d=Gm m=N -1 G H m Note C 2   i T i exp(-i   t i ) Note T 2   i C i exp(+i   t 2 )

34. … exp(-i  2 t 0 ) exp(-i  1 t 0 ) exp(i  0 t 0 ) exp(i  1 t 0 ) exp(i  2 t 0 ) … … exp(-i  2 t 1 ) exp(-i  1 t 1 ) exp(i  0 t 1 ) exp(i  1 t 1 ) exp(i  2 t 1 ) … … exp(-i  2 t 2 ) exp(-i  1 t 2 ) exp(i  0 t 2 ) exp(i  1 t 2 ) exp(i  2 t 2 ) … … exp(-i  2 t 3 ) exp(-i  1 t 3 ) exp(i  0 t 3 ) exp(i  1 t 3 ) exp(i  2 t 3 ) … … exp(-i  2 t 4 ) exp(-i  1 t 4 ) exp(i  0 t 4 ) exp(i  1 t 4 ) exp(i  2 t 4 ) … = … C -2 C -1 C 0 C 1 C 2 … T 0 T 1 T 2 T 3 T 4 … … exp(i  2 t 0 ) exp(i  2 t 1 ) exp(i  2 t 2 ) exp(i  2 t 3 ) exp(i  2 t 4 ) … … exp(i  1 t 0 ) exp(i  1 t 1 ) exp(i  1 t 2 ) exp(i  1 t 3 ) exp(i  1 t 4 ) … … exp(i  0 t 0 ) exp(i  0 t 1 ) exp(i  0 t 2 ) exp(i  0 t 3 ) exp(i  0 t 4 ) … … exp(-i  1 t 0 ) exp(-i  1 t 1 ) exp(-i  1 t 2 ) exp(-i  1 t 3 ) exp(-i  1 t 4 ) … … exp(-i  2 t 0 ) exp(-i  2 t 1 ) exp(-i  2 t 2 ) exp(-i  2 t 3 ) exp(-i  2 t 4 ) … … C -2 C -1 C 0 C 1 C 2 … T 0 T 1 T 2 T 3 T 4 … d=Gm M=N -1 G H m Note C 2   i T i exp(-i   t i ) Note T 2   i C i exp(+i   t 2 ) Opposite signs =N -1

35. Discrete Fourier Transform Find the coefficients C given the data, T Equivalent to m = G H d C k =  n=-N/2 N/2 T n exp(±2  ikn/N ) with k=-½N, …, ½N Discrete Inverse Fourier Transform Find the data T given the coefficients, C Equivalent to d = N -1 Gm T n = N -1  k=-N/2 N/2 C k exp( 2  ikn/N ) with n=-½N, …, ½N Warnings: 1) no one can agree on signs 2) no one can agree on normalizations ± Note normalization factor of N -1 has been omitted Note normalization factor of N -1 has been added

36. Counting unknowns frequencies from –(N/2)  to (N/2)  in steps of  So N+1 complex numbers, C p So 2N+2 real and imaginary parts, C p r and C p i But C -p = C p *, so really only N/2+1 unknown complex numbers So N+2 real and imaginary parts, C p r and C p i (p  0) But C 0 i =0 and C N/2 i =0 (always) So N unknowns, matching N data

37. % standard fft setup. The standard implementation of the digital fourier % transform is VERY INFLEXIBLE. Learn these rules: N=256; % you can choose the length N of the time series % in some implementations N can be any positive % integer, but in others it MUST be a % power-or-two. I set it here to 256, which % is two-to-the-eigth-power. dt=1.0; % and you can choose the sampling interval dt % but then the following variables are set tmax=dt*(N-1); % we presume the time series starts at t=0, so % the maximum time is tmax t=dt*[0:N-1]; % time then goes from 0 to (N-1)*dt fmax=1/(2.0*dt); % the maximum frequency in the fft calculation is % called the Nyquist frequency. It is % determined by the two-points-per wavelength % rule df=fmax/(N/2); % the frequency spacing, df, assumes that a N-point % time series is reperesented by an N-point fourier % transform f=df*[0:N/2,-N/2+1:-1]'; % The fourier transform has N values, from a negative % frequency of -(fmax-df) through zero freqency, to % positive frequency of fmax. But note the weird order. The % zero and positive frequencies are put in the first % half of the array and the negative frequencies are % put in the second half.

38. % p is the timeseries whose transform is being computed w0 = 2*pi*fmax/10; % sample p, a simple sinusoid of frequency w0 p = sin(w0*t); % fourier transform using MatLab's fft function. The help function % says that it uses the NEGATIVE sign in the exponential. pt=fft(p); % these are the coefficients, C, of the complex exponential % presumably one would do something with the fourier transform % at this point - apply a filter, for example. But I do nothing. % Inverse fourier transform using the Matlab function ifft. % Help says it uses the POSITIVE sign in the exponential, and that % it has the right normalization that ifft(fft(x))=x. But % BE WARNED, that doesn't mean that the normalization on fft % is 1 and that the normalization on ifft is (1/N), like % I had it in class. You can put any constant, b, in front % of the fft integral, as long as you put 1/b in front of % the IFFT integral. But judging by the Help, I think that % Matlab used b=1. pr=ifft(pt); % this reconstructs the function from the coefficients

39. The fast Fourier transform algorithm The Fourier Transform equation m = [G T G] -1 G T d = diag(N -1,2/N, … 2/N,N -1 ) G T d has N multiplications for each of N unknowns, So N 2 in total. For example, for N=1024, N 2 =1,048,576 But in the special case of N being a power of two, there is an algorithm – the fast fourier transform algorithm - that can compute m in only Nlog 2 N multiplications For example, N=1024, Nlog 2 N=10,240 A substantial savings! MatLab implements it. Use it!