Fourier Transformations

Fourier Transformations
Grad Algorithms Fourier Transformations Sum of Sine/Cosine Functions A Pulse, Linearity, & Periodic Duality Change of Basis Orthogonal Basis Rectangular vs Polar vs Complex Continuous Functions Radio Signal Convolution Linear Filter   * FFT in nlogn Time FFT Code FFT Butterfly JPG Polynomial Basis Evaluating & Interpolating FFT in nlogn Time Roots of Unity Same FFT Code & Butterfly Inverse FFT Polynomial Multiplication Integer Multiplication Jeff Edmonds York University COSC 6111

Fourier Transformation
are a change of basis from the time basis to sine/cosine basis JPG or polynomial basis Purposes: Signal Processing Compressing data (eg images with .jpg) Multiplying integers in n logn loglogn time. Error correcting codes. …. Amazingly once you include complex numbers, the FFT code for sine/cosines and for polynomials are the SAME. The Scientist and Engineer's Guide to Digital Signal Processing By Steven W. Smith, Ph.D.

Sine &Cosine Basis A continuous periodic function t time y(t) Swings, capacitors, and inductors all resonate at a given frequency, which is how the circuit picks out the contribution of a given frequency. Find the contribution of each frequency Purposes: Signal Processing Compressing data (eg images with .jpg) Multiplying integers in n logn loglogn time. Error correcting cods. ….

Sine &Cosine Basis y(x) = x y(x)  2 sin(x) - sin(2x) + 2/3 sin(3x) Surely this can’t be expressed as sum of sines and cosines.

Sine &Cosine Basis y(x) = x2 y(x)  -4 sin(x) + sin(2x) - 4/9 sin(3x)

Sine &Cosine Basis A continuous periodic function t time y(t) A discrete periodic function Time Domain j[0,n-1] y[j] y[] is an array of real values j 1 2 3 4 5 6 7 8 9 10 6

Sine &Cosine Basis A continuous periodic function t time y(t) A discrete periodic function Time Domain j[0,n-1] y[j] The function y is expressed as a linear combination of the basis functions cf and sf. y[] YRe[] YIm[] Y[] 7

Sine &Cosine Basis A continuous periodic function t time y(t) A discrete periodic function Time Domain j[0,n-1] y[j] The function y is expressed as a linear combination of the basis functions cf and sf. YRe [f] is the coefficient of the bases function cf. YIm [f] is the coefficient of the bases function sf. Recall: ei = cos() + i sin() Hence, the coefficients of cosine are subscripted “Real” And of sine with “Imaginary”. 8

Single Pulse Lets consider a few examples and a few properties. Time Domain y Frequency Domain Y Cosine wave Cosine with f=8 Delta function Impulse at Yre[8] One non-zero coefficient in the real frequency domain arises from what in the time domain? One cosine wave of the stated frequency.

Single Pulse Time Domain y Frequency Domain Y Cosine wave Cosine with f=8 Delta function Impulse at Yre[8] In the discrete world, delta has one nonzero value, which is delta(8) = 1. In the continuous world, delta has the value 1/ for a width  for a total area of one.

Single Pulse Time Domain y Frequency Domain Y Cosine wave Cosine with f=4 Delta function Impulse at Yre[4] A different frequency. A different coefficient?

Linearity of FT Time Domain y Frequency Domain Y Scale up the amplitude in the time domain. c 2c -c -2c Cosine wave Cosine with f=4 Scales up the amplitude in the frequency domain. c 2c -c Delta function Impulse at Yre[4] Linearity of FT: If Y=FT(y), then cY=FT(cy)

Linearity of FT Time Domain y Frequency Domain Y Linearity of FT: FT(c1∙y1+c2∙y2) = c1∙FT(y1) + c2∙FT(y2)

Periodic Signals Time Domain y Frequency Domain Y j y[j] j Sine and cosine waves are periodic. The time domain is composed of these. Hence, the time domain is periodic. Strangely, the frequency domain is periodic too.

Periodic Signals Time Domain y Frequency Domain Y One frequency solution Frequency This set of discrete points needs to be interpreted as a continuous function. This is the obvious solution. Leading to this frequency decomposition. This is another. Another. Cos(-) = Cos() = Cos(+2 ) =…

Duality of FT Time Domain y Frequency Domain Y Cosine wave Cosine with f=4 Delta function Impulse at Yre[4] Delta function Impulse at y[4] Ok, we know this. But what if we switch them?

Duality of FT Time Domain y Frequency Domain Y Cosine wave Cosine with f=4 Delta function Impulse at Yre[4] Delta function Impulse at y[4] Cosine wave Cosine with f=4 A single pulse in the time domain is produced from what sum of sines and cosines? Not clear! The curve of the coefficients forms a sine/cosine wave! Strange but true!

[ ][ ] =[ ] Fourier Transformation 1/n Time Domain y
Duality of FT Time Domain y Frequency Domain Y Cosine wave Cosine with f=4 Delta function Impulse at Yre[4] Delta function Impulse at y[4] Cosine wave Cosine with f=4 Arises from the duality of FT and its inverse. [ ][ ] =[ ] s1[1] s1[2] s2[1] s2[2] Y[1] Y[2] 1/n y[1] y[2] Sorry! we do this later.

Duality of FT Time Domain y Frequency Domain Y Cosine wave Cosine with f=4 Delta function Impulse at Yre[4] Delta function Impulse at y[4] Cosine wave Cosine with f=4 Arises from the duality of FT and its inverse. Duality of FT: If Y=FT(y), then y=FT(Y)

Duality of FT Time Domain y Frequency Domain Y Square wave Sinc function How do you get these corner? Sinc function Square wave ? Duality of FT: If Y=FT(y), then y=FT(Y)

Duality of FT Time Domain y Frequency Domain Y Gaussian Duality of FT: If Y=FT(y), then y=FT(Y)

Change of Basis Change of Basis: T([a1,a2,…,ad]) = [A1,A2,…,Ad] Changes the basis used to describe an object. The Time basis of a vector space: A tuple <w1,w2,…,wd> of basis objects Linearly independent Spans the space uniquely "v $[a1,a2,…,ad], v = a1w1+a2w2 +… + adwd The Fourier basis of a vector space: A tuple <W1,W2,…,Wd> of basis objects Linearly independent Spans the space uniquely "v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd 22

Change of Basis Change of Basis: T([a1,a2,…,ad]) = [A1,A2,…,Ad] Changes the basis used to describe an object. "v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd "v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd =[w1,w2] =[ , ] Standard Basis New Basis =[W1,W2] =[ , ] W1[1] W1[2] W2[1] W2[2] v = v = [a1,a2] = [3,2] [A1,A2] = [11/5,32/5] [ ][ ] =[ ] W1[1] W1[2] W2[1] W2[2] a1 a2 A1 A2 23

Change of Basis Change of Basis: T(y[0],y[1],…,y[n-1]) = [YRe[0],…,YIm[n/2]] Changes the basis used to describe an object. "y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In The time basis j’ Ij[j’] zero one j Time Basis =[ , ] =[?,?] =[I1,I2,…] ? = y = y[0]=3 y[1]=2 ? A discrete periodic function j y[j] 24

Change of Basis Change of Basis: T(y[0],y[1],…,y[n-1]) = [YRe[0],…,YIm[n/2]] Changes the basis used to describe an object. "y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In y = YRe[0]∙c1+YIm[0]∙s1+ ,…,YRe[n/2]∙sn/2+YIm[n/2]∙sn/2 Time Basis =[ , ] =[I1,I2,…] y = Fourier Basis =[ , ] =[?,?] =[c1,s1,..] c1 sn/2 cn/2 s1 y = y[0]=3 y[1]=2 ? YRe[0] =11/5 YIm[0] =32/5 A discrete periodic function j y[j] 25

Change of Basis Change of Basis: T(y[0],y[1],…,y[n-1]) = [YRe[0],…,YIm[n/2]] Changes the basis used to describe an object. "y $[y[0],y[1],…,y[n-1]], y = y[0]I1 +y[1]I2 +… + y[n-1]In y = YRe[0]∙c1+YIm[0]∙s1+ ,…,YRe[n/2]∙sn/2+YIm[n/2]∙sn/2 Time Basis =[ , ] =[I1,I2,…] y = Fourier Basis =[ , ] =[c1,s1,..] c1 sn/2 cn/2 s1 y = y[0]=3 y[1]=2 YRe[0] =11/5 YIm[0] =32/5 s1[1] s1[2] s2[1] s2[2] Y[1] Y[2] y[1] y[2] [ ] [ ] =[ ] s1[1] s2[1] s1[2] s2[2] Y[1] Y[2] [ ] [ ] =[ ] y[1] y[2] -1 26

Orthogonal Basis Time Basis =[ , ] =[I1,I2,…] Fourier Basis =[ , ] =[c1,s1,..] This frequency basis vectors are orthogonal and of (almost) uniform length: Evenly spaced points within an integer number of full cycles. Sum of squares 27

Orthogonal Basis Time Basis =[ , ] =[I1,I2,…] Fourier Basis =[ , ] =[c1,s1,..] This frequency basis vectors are orthogonal and of (almost) uniform length: Evenly spaced points within an integer number of full cycles. Sum of products 28

Orthogonal Basis Time Basis =[ , ] =[I1,I2,…] Fourier Basis =[ , ] =[c1,s1,..] This frequency basis vectors are orthogonal and of (almost) uniform length: [ ][ ]=[ ] s1[1] s1[2] s2[1] s2[2] n/2 [ ] = [ ] s1[1] s1[2] s2[1] s2[2] -1 2/n 29

[ ][ ] =[ ] Fourier Transformation =[I1,I2,…] =[c1,s1,..] =[ , ]
Orthogonal Basis Time Basis =[ , ] =[I1,I2,…] Fourier Basis =[ , ] =[c1,s1,..] This frequency basis vectors are orthogonal and of (almost) uniform length: [ ][ ] =[ ] s1[1] s1[2] s2[1] s2[2] Y[1] Y[2] 2/n y[1] y[2] This is the correlation between y and cf 30

Change of Basis Viewing correlation a different way: YRe[0] =11/5 YIm[0] =32/5 y =  y = y[0]=3 y[1]=2 c1 y =  cos() = y∙c1 |y||c1| A1 = |y|cos() = y∙c1 = j y[j]c1[j] This is the correlation between y and cf 31

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Rectangular Notation j There are two basis (hence two coefficients) for each frequency f. n/2 frequencies are considered There are n values in the Time Domain and as needed n in the Frequency Domain. Note 2(n/2 +1)=n+2, because s1 = sn/2 = 0 32

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Rectangular Notation j Any function y in the Time Domain can be expressed as a linear combination of these n basis functions. Inverse Fourier Transform: 33

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Rectangular Notation Polar Notation The contribution from frequency f is: As needed, there are still n in the Frequency Domain. 34

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Complex Notation eθi = cosθ + isinθ i -i -1 1 θ f’(0) = g’(0) Proof: f(θ) = g(θ) f(0) = g(0) f’’(θ) = -f(θ) g’’(θ) = -g(θ) 35

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. There are two basis (hence two coefficients) for each frequency f[0,n/2]. Rectangular Notation Complex Notation There is one basis (hence one coefficients) for each frequency f[0,n-1]. There are n values in the Time Domain. Hence n are needed in the Frequency Domain. eθi = cosθ + isinθ 36

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Rectangular Notation Complex Notation Any function y in the Time Domain can be expressed as a linear combination of these n basis functions. Inverse Fourier Transform: 37

Rectangular vs Polar vs Complex The contribution of a given frequency f can be expressed in different ways. Rectangular Notation Complex Notation Surprisingly the two techniques give the same numbers when the time domain values are real. 38

Continuous Functions A continuous periodic function t time y(t) -½T ½T y(t) is an function from real time to a real values. The basis functions are the same. Except they also need to be continuous. Stretched so that a period of the basis lines up with a period of the signal. If this is the dominate musical note of frequency  = 2/T, then all the other basis functions are its harmonics frequencies: Frequency: Note on the Piano: , 2, 3, 4, 5, 6, ... C G E 39

Continuous Functions A continuous periodic function t time y(t) -½T ½T y(t) is an function from real time to a real values. The third basis, as before will cycle 3 times. Before the signal had n values so we needed n bases functions. Now we need n. Continue for ever. 40

Continuous Functions A continuous periodic function t time y(t) -½T ½T y(t) is an function from real time to a real values. Both the signal and the basis functions repeat every T units. Discrete: (previous defn) Continuous: (new defn) Called a Fourier Sequence. 41

Continuous Functions A continuous periodic function t time y(t) -½T ½T y(t) is an function from real time to a real values. Both the signal and the basis functions repeat every T units. Discrete: (previous defn) Continuous: (new defn) Called a Fourier Sequence. 42

Continuous Functions -½T ½T y(t) A continuous non-periodic function time t We want T. 43

Continuous Functions -½T ½T y(t) A continuous non-periodic function time t We want T. Continuous Periodic: (previous defn) Continuous Non-Periodic: (new defn) Continuous Non-Periodic Complex: (new defn) 44

Continuous Functions These four definitions of Fourier Transformations look very different, but we will argue that they give pretty much the same answer. Discrete: Continuous Periodic: Continuous Non-Periodic: Continuous Non-Periodic Complex: 45

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe()  = (-,) A single square y(t) -T/2 T/2 1 -1 Continuous Non-Periodic: The th real Fourier coefficient is the correlation between 1 and cos(t).

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe() f = 0, 1, 2,3, …, A single square y(t) -T/2 T/2 1 -1 Continuous Periodic: This technique can’t be applied because the signal is not periodic.

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe() f = 0, 1, 2,3, …, YRe[f] A square wave y(t) -T/2 T/2 1 -1 Continuous Periodic:  is a real number f is an integer

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe() f = 0, 1, 2,3, …,n YRe[f] A square wave y(t) -T/2 T/2 1 -1 Discrete:  is a real number f is an integer

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe()  = (-,) A square wave y(t) -T/2 T/2 1 -1 Continuous Non-Periodic: The Y[f] discrete signal at f is replaced at =2f/T with a continuous delta with value Y[f]/dt =  of width dt for a total area of Y[f]. else zero.  is a real number f is an integer

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe()  = (-,) A square wave y(t) -T/2 T/2 1 -1 The Y[f] discrete signal at f is replaced at =2f/T with a continuous delta with value Y[f]/dt =  of width dt for a total area of Y[f]. else zero.

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe()  = (-,) A square wave y(t) -T/2 T/2 1 -1 Continuous Non-Periodic: The Y[f] discrete signal at f is replaced at =2f/T with a continuous delta with value Y[f]/dt =  of width dt for a total area of Y[f]. else zero.

Continuous Functions Time Domain y Frequency Domain Y Sinc function YRe()  = (-,) A square wave y(t) -T/2 T/2 1 -1 Continuous Non-Periodic: The Y[f] discrete signal at f is replaced at =2f/T with a continuous delta with value Y[f]/dt =  of width dt for a total area of Y[f]. else zero. Continuous Periodic: The two techniques are equivalent!

Continuous Functions Time Domain y Frequency Domain Y How do you get these corner? Zero function 2 YIm() -  A square wave y(t) -T/2 T/2 1 -1 Continuous Non-Periodic:

Continuous Functions YIm() Time Domain y Frequency Domain Y y(t) = t Continuous Non-Periodic:

Continuous Functions Time Domain y Frequency Domain Y y(t) = t y(x)  2 sin(x) - sin(2x) + 2/3 sin(3x) Continuous Non-Periodic: Alternatively only do the FT on part of the signal.

Radio Signals Time Domain y Frequency Domain Y Sound Signal ie how far out is the speaker drum at each point in time. Sound is low frequency High frequencies filtered out. This constant function in the time domain translates to a delta at f=0 in the frequency domain Note signal is not centered at zero, but has a constant added to it.

Radio Signals Time Domain y Frequency Domain Y Radio Carrier Signal ie A wave of magnetic field that can travel far. One high frequency signal

Radio Signals Time Domain y Frequency Domain Y Carrier signal Audio Signal (shifted) Audio Signal (shifted &flipped) Modulation: Their product y(i) = y1(i)  y2(i)

Radio Signals Time Domain y Frequency Domain Y Because FT is periodic, a flipped version is in the “negative” frequencies. Carrier signal Audio Signal (shifted) Negative copy (shifted) Modulation: Their product y(i) = y1(i)  y2(i)

 * Convolution Time Domain y Frequency Domain Y Modulation: Their product Y = Y1*Y2 Convolution y = y1 y2

 * Convolution “Proof”: Complex Notation y = y1 y2 Y = Y1*Y2 64

 * Convolution “Proof”: y = y1 y2 Y = Y1*Y2

Linear Filter Filters also use of Convolution: Because x[] y[] Independent of time: x[_+dt] y[_+dt] Linear: c1∙x1[]+c2∙x2[] c1∙y1[]+y2∙x2[] This system takes in a signal and outputs transformed signal.

h[] = Linear Filter Filters also use Convolution: h[] = In order understand this transformation, we put in a single pulse. [] = h[] This response h[] identifies the system.

h[] = Linear Filter Feed in any signal x[] = Sum of contributions from each separate pulse. h[] Sum

h[] = Fourier Transformation Linear Filter Where does y[k] come from? x[] = h[]

Linear Filter x[] = Input x[]*h[] = Output h[] = Impulse Response This takes O(nm) time. Too much! How can we do it faster?

 * Convolution Time Domain y Frequency Domain Y y = y1 y2 Product Y = Y1*Y2 Convolution Duality of FT: If Y=FT(y), then y=FT(Y) Y = X H Product y = x*h Convolution

 * Convolution Time Domain y Frequency Domain Y x[] = Input h[] = Impulse Response Oops Fourier Transform takes O(n2) time. X[] H[] Fast Fourier Transform takes O(nlogn) time! X[]H[] x[]*h[] = Output Multiplication takes O(n) time. Y = X H Product y = x*h Convolution

 * Convolution Time Domain y Frequency Domain Y Impulse Response h[] = H[] = Impulse Response x[] = Input X[] x[]*h[] = Output X[]H[] Multiplying zeros low and high frequencies in input. Filters out low and high frequencies in input. Not clear what system does to input Y = X H Product y = x*h Convolution

FFT nlogn Time Time Domain y Frequency Domain Y Theorem: abcdefgh y[0]=a, y[1]=b, y[2]=c, y[3]=d, … ABCDEFGH Fast Fourier Transform takes O(nlogn) time!

FFT nlogn Time The Fast Fourier Transform (FFT) is a very efficient algorithm for performing a discrete Fourier transform FFT principle first used by Gauss in 18?? (But was not interesting without computers) FFT algorithm published by Cooley & Tukey in 1965 In 1969, the 2048 point analysis of a seismic trace took 13 ½ hours. Using the FFT, the same task on the same machine took 2.4 seconds!

FFT nlogn Time Time Domain y Frequency Domain Y Lemma 1: y = abcd Y = ABCD If y’ = a0b0c0d0 Y’ = ½∙ABCDABCD then Proof 1:

FFT nlogn Time Time Domain y Frequency Domain Y Lemma 1: y = abcd Y = ABCD If y’ = a0b0c0d0 Y’ = ½∙ABCDABCD then “Proof” 2:

FFT nlogn Time Time Domain y Frequency Domain Y 16 32 48 64 Lemma 1: Impulse at f = n/2 16 32 48 64 Impulse at f = 0 ½c16+½c16= 16 32 48 64 ½I16+½I16= Impulse at f = 0 & n/2 1 = = ½∙

FFT nlogn Time Time Domain y Frequency Domain Y Lemma 1: y = abcd Y = ABCD If Interpolate intermediate values Shouldn’t change frequencies. aabbccdd ABCD  = * = ½∙ y’ = a0b0c0d0 Y’ = ½∙ABCDABCD

FFT nlogn Time Time Domain y Frequency Domain Y Lemma 2: y = abcd Y = ABCD If y’ = a0b0c0d0 Y’ = ½ s1 ∙ABCDABCD then Proof: a0b0c0d0 ½∙ABCDABCD By Lemma 1 * Impulse at f = 1 ?  = 1 -1 s1 ? = 0a0b0c0d ½s1∙ABCDABCD The value Y[j] is scaled by B1[j].

Fast Fourier Transformation
Divide & Conquer - Friends - Recursion. Trust your friends to solve any subinstance: as long as smaller and is an instance to the same problem. My instance My friend’s Instance My friend’s Instance

FFT nlogn Time Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: Split into even & odd indexes yeven = aceg yodd = bdfh Yeven = ACEG Yodd = BDFH Recurse Twice on input of half the size. By Lemmas ½∙ACEGACEG ½s1∙BDFHBDFH Copy 0b0d0f0h a0c0e0g0

FFT nlogn Time Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: + = + = ½∙ACEGACEG +½s1∙BDFHBDFH 0b0d0f0h a0c0e0g0 ½∙ACEGACEG ½s1∙BDFHBDFH

FFT nlogn Time Time Domain y Frequency Domain Y FFT Alg: T(n) = 2 T(n/2) + O(n) Time to solve FFT on input of size n. Two recursive friends Time to solve FFT on input of size n/2. Time spent by top stack frame. = O(nlog(n)) Same as merge sort.

FFT Code Time Domain y Frequency Domain Y Y = ABCDEFGH y = abcdefgh FFT Input: FFT Alg: Algorithm FFT(y, , n): Input: y = [y[0],y[1],y[2],…, y[n-1]] (Time Domain) = e2i 1/n (nth root of unity) n = # of samples (2r) Output: Y = [Y[0],Y[1],Y[2],…, Y[n-1]] (Frequency Domain)

FFT Code Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: Split into even & odd indexes yeven = aceg yodd = bdfh Yeven = ACEG Yodd = BDFH Recurse Twice on input of half the size. % Separate even and odd indices yeven = [y[0],y[2],y[4],…, y[n-2]] yodd = [y[1],y[3],y[5],…, y[n-1]] % Recurse Yeven =FFT(yeven, n/2) Yodd =FFT(yodd , n/2)

FFT Code Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: + = ½∙ACEGACEG +½s1∙BDFHBDFH Yeven = ACEG ½∙ACEGACEG ½s1∙BDFHBDFH Yodd = BDFH 1 -1 s1 The value Y[j] is scaled by B1[j]. %Combining For i = 0 to n/2-1 Y[i] = Yeven[i] + i ∙Yodd[i] Y[i+n/2] = Yeven[i] + i+n/2 ∙Yodd[i] Return(Y)

FFT Code Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: + = ½∙ACEGACEG +½s1∙BDFHBDFH Yeven = ACEG ½∙ACEGACEG ½s1∙BDFHBDFH Yodd = BDFH 1 -1 s1 The value Y[j] is scaled by B1[j]. % Recurse Yeven =FFT(yeven, 2, n/2) Yodd =FFT(yodd , 2, n/2) Algorithm FFT(y, , n): Input: = e2i 1/n = e2i 1/n (nth root of unity) 2 = e2i 2/n (n/2 th root of unity)

FFT Code Algorithm FFT(y, , n): Input: y = [y[0],y[1],y[2],…, y[n-1]] (Time Domain) = e2i 1/n (nth root of unity) n = # of samples (2r) Output: Y = [Y[0],Y[1],Y[2],…, Y[n-1]] (Frequency Domain) % Separate even and odd indices yeven = [y[0],y[2],y[4],…, y[n-2]] yodd = [y[1],y[3],y[5],…, y[n-1]] % Recurse Yeven =FFT(yeven, 2, n/2) (2 = e2i 2/n ) Yodd =FFT(yodd , 2, n/2) %Combining For i = 0 to n/2-1 Y[i] = Yeven[i] + i ∙Yodd[i] Y[i+n/2] = Yeven[i] + i+n/2 ∙Yodd[i] Return(Y)

FFT Butterfly Time Domain y Frequency Domain Y FFT Alg: y = abcdefgh FFT Input: + = + = ½∙ACEGACEG +½s1∙BDFHBDFH 0b0d0f0h a0c0e0g0 ½∙ACEGACEG ½s1∙BDFHBDFH

FFT Butterfly O(log(n)) levels

JPG Image Compression JPEG (Image Compression) JPEG is two dimensional Fourier Transform exactly as done before.

JPG Image Compression Each 88 block of values from the image is encoded separately.

JPG Image Compression Each basis function has a coefficient, giving the contribution of this basis function to the image. Each 88 block of values from the image is encoded separately. It is decomposed as a linear combination of basis functions.

JPG Image Compression Because the image domain is indexed by x,y [0,7], so is each basis function. Each 88 block of values from the image is encoded separately. It is decomposed as a linear combination of basis functions.

JPG Image Compression They are indexed similarly by u,v [0,7] Each 88 block of values from the image is encoded separately. It is decomposed as a linear combination of basis functions. Because the image domain is specified by 64 values, the frequency domain must also be specified by 64 values And hence 64 basis functions are needed.

JPG Image Compression Each 88 block of values from the image is encoded separately. It is decomposed as a linear combination of basis functions. Each of the 64 basis functions is a two dimensional cosine.

JPG Image Compression The first basis is constant. Its coefficient gives the average value in within block. Because many images have large blocks of the same colour, this one coefficient gives much of the key information!

JPG Image Compression The second basis “slopes” left to right Its (pos or neg) coefficient gives whether left to right the value tends to increase or decrease.

JPG Image Compression The second basis “slopes” left to right Because many images have have a gradual change in colour, this one coefficient gives more key information!

JPG Image Compression A similar basis for top to bottom.

JPG Image Compression The <0,2> basis is Its coefficient gives whether the value tends to be smaller in the middle. This helps display the horizontal lines in images

JPG Image Compression As seen, the low frequency components of a signal are more important. Removing 90% of the bits from the high frequency components might remove, only 5% of the encoded information.

Polynomial Basis A continuous periodic function t time y(t) A discrete periodic function Time Domain j[0,n-1] y[j] The function y is expressed as a linear combination of the basis functions cf and sf. What we did before y[] YRe[] YIm[] Y[] 106

Polynomial Basis Instead of using sine and cosines as the basis, 107

Polynomial Basis Instead of using sine and cosines as the basis, We use polynomials. 108

Polynomial Basis Change of Basis: T([a1,a2,…,ad]) = [A1,A2,…,Ad] Changes the basis used to describe an object. The Time basis of a vector space: A tuple <w1,w2,…,wd> of basis objects Linearly independent Spans the space uniquely "v $[a1,a2,…,ad], v = a1w1+a2w2 +… + adwd The Polynomial basis of a vector space: A tuple <W1,W2,…,Wd> of basis objects Linearly independent Spans the space uniquely "v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd 109

Polynomial Basis Change of Basis: T([a1,a2,…,ad]) = [A1,A2,…,Ad] Changes the basis used to describe an object. "v $[a1,a2,…,ad], v = a1w1 +a2w2 +… + adwd "v $[A1,A2,…,Ad], v = A1W1+A2W2 +… + AdWd =[w1,w2] =[ , ] Standard Basis New Basis =[W1,W2] =[ , ] W1[1] W1[2] W2[1] W2[2] v = v = [a1,a2] = [3,2] [A1,A2] = [11/5,32/5] [ ][ ] =[ ] W1[1] W1[2] W2[1] W2[2] a1 a2 A1 A2 110

Polynomial Basis Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a1,a2,…,an-1] Changes the basis used to describe an object. f = a0∙B0+a1∙B1+a2∙B2 +… + an-1∙Bn-1 f(x) = a a1x +a2x2 + …+ an-1xn-1 a0 =11/5 a1 =32/5 Polynomial Basis =[ , ] =[B0,B1,..] [a1,a2,…,an-1] = coefficients of the polynomial f = A discrete function x f(x) 111

Polynomial Basis Have you seen Taylor Expansions of a Function? They show that functions f(x) can be expressed by specifying the coefficients of the polynomial. F(x) = a0+a1x +a2x2 +a3x3 + … Eg: f(x) = 1/(1-x) F(x) = 1+x +x2 +x3 + …

Polynomial Basis Have you seen Taylor Expansions of a Function? They show that functions f(x) can be expressed by specifying the coefficients of the polynomial. F(x) = a0+a1x +a2x2 +a3x3 + …

Polynomial Basis Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a1,a2,…,an-1] Changes the basis used to describe an object. "y $[y[0],y[1],…,y[n-1]], y = y[0]I0 +y[1]I1 +… + y[n-1]In-1 The time basis x Ij[x] zero one j Time Basis =[ , ] =[?,?] =[I0,I1,…] f = y[0]=3 y[1]=2 y[j] = the value f(j) of the function at x=j. the value f(j) of the function at x=j. A discrete function x f(x) 115

Polynomial Basis Instead of evaluating the polynomials on j = 0, 1, 2,…, n-1 y[j] = the value f(j) of the function at x=j. We evaluate them on x0, x1, x2, …, xn-1 the value f(j) of the function at x=j. These xj are special fixed values. Later we set xj = e2i j/n 116

Polynomial Basis Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a1,a2,…,an-1] Changes the basis used to describe an object. "y $[y[0],y[1],…,y[n-1]], y = y[0]I0 +y[1]I1 +… + y[n-1]In-1 The time basis x Ij[x] zero one xj Time Basis =[ , ] =[I0,I1,…] =[?,?] f = y[0]=3 y[1]=2 y[j] = the value f(xj) of the function at x=j. the value f(j) of the function at x=j. A discrete function x f(x) x0 x1 x2 x3 x … xn-1 117

Evaluating & Interpolating A Fourier Transform is a change in basis. It changes the representation of a function from the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 x to the value f(xi) at key values xi. x0 x1 x2 x3 x … xn-1 y0 y1 y2 y3 y … yn-1 yi = f(xi) This amounts to evaluating f at these points. 118

Evaluating & Interpolating A Fourier Transform is a change in basis. It changes the representation of a function from the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 (x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (x1)0 (x1)1 (x1)2 (x1)3 … (x1)n-1 (xn-1)0(xn-1)1(xn-1)2 (xn-1)3…(xn-1)n-1 (x2)0 (x2)1 (x2)2 (x2)3 … (x2)n-1 (x3)0 (x3)1 (x3)2 (x3)3 … (x3)n-1 yi = f(xi) Vandermonde matrix Invertible if xi distinct. This amounts to evaluating f at these points. 119

Evaluating & Interpolating An Inverse Fourier Transform is the reverse. It changes the representation of a function x from the value f(xi) at key values xi. x0 x1 x2 x3 x … xn-1 y0 y1 y2 y3 y … yn-1 yi = f(xi) to the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 This amounts to interpolating these points. 120

Evaluating & Interpolating An Inverse Fourier Transform is the reverse. It changes the representation of a function Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points. to the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 This amounts to interpolating these points. 121

Evaluating & Interpolating My output: f(x) = a0+a1x +a2x2 + … + an-1xn-1 My input: (a0,a1,a2,…,an-1) & x n-1 multiplications? f(x) = a0+x(a1+x(a2 + … + x(an-1) … )) n multiplications and n additions. Given n distinct values (x0,x1,x2,…,xn-1). Can they all be done in less than n×O(n) time?

Evaluating & Interpolating My output: f(x) = a0+a1x +a2x2 + … + an-1xn-1 My input: (a0,a1,a2,…,an-1) & x (x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (x1)0 (x1)1 (x1)2 (x1)3 … (x1)n-1 (xn-1)0(xn-1)1(xn-1)2 (xn-1)3…(xn-1)n-1 (x2)0 (x2)1 (x2)2 (x2)3 … (x2)n-1 (x3)0 (x3)1 (x3)2 (x3)3 … (x3)n-1 yi = f(xi) I hope so Given n distinct values (x0,x1,x2,…,xn-1) Can they all be done in less than n×O(n) time?

FFT nlogn Time The Fast Fourier Transform (FFT) is a very efficient algorithm for performing a discrete Fourier transform FFT principle first used by Gauss in 18?? (But was not interesting without computers) FFT algorithm published by Cooley & Tukey in 1965 In 1969, the 2048 point analysis of a seismic trace took 13 ½ hours. Using the FFT, the same task on the same machine took 2.4 seconds!

FFT nlogn Time Divide & Conquer - Friends - Recursion. Trust your friends to solve any subinstance: as long as smaller and is an instance to the same problem. My instance My friend’s Instance My friend’s Instance My friend’s Instance

FFT nlogn Time My output: f(x) = a0+a1x +a2x2 + … + an-1xn-1 My input: (start with one x) (a0,a1,a2,…,an-1) & x 1st friend’s input? feven: (a0,a2,a4,…,an-2) & ? 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) & ? feven(z) = a0+a2z+a4z2+a6z3+ … + an-2zn/2-1 f(x) = a0+a1x +a2x2 + … + an-1xn-1 = a0+a2x2+a4x4 + … + an-2xn-2 + a1x+a3x3+a5x5 + … + an-1xn-1 = a0+a2x2+a4x4 + … + an-2xn-2 + x( a1+a3x2+a5x4 + … + an-1xn-2 ) = feven(x2) + x( fodd(x2) ) f(x) = feven(x2) + x fodd(x2)

FFT nlogn Time My output: f(x) = a0+a1x +a2x2 + … + an-1xn-1 My input: (start with one x) (a0,a1,a2,…,an-1) & x 1st friend’s input? feven: (a0,a2,a4,…,an-2) & x2 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) & x2 2nd friend’s output: yodd = fodd(x2) 1st friend’s output: yeven = feven(x2) My output: f(x) = yeven + x yodd T(n) = 2 T(n/2) + O(1) = O(n) Ok. So it takes O(n) time to evaluate. f(x) = feven(x2) + x fodd(x2)

FFT nlogn Time My input: (a0,a1,a2,…,an-1) (x0,x1,x2,…,xn-1) My output: (y0,y1,y2,…,yn-1) yi = f(xi) 1st friend’s input? feven: (a0,a2,a4,…,an-2) (x02,x12,x22,…,xn-12) 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) (x02,x12,x22,…,xn-12) 2nd friend’s output: i y<odd,i> = fodd(xi2) 1st friend’s output: i y<even,i> = feven(xi2) My output: i f(xi) = y<even,i> + xi y<odd,i> T(n) = 2 T(n/2) + O(n) = O(n log n) Wow! That was easy.

FFT nlogn Time My input: (a0,a1,a2,…,an-1) (x0,x1,x2,…,xn-1) n coefficients n values of x n/2 coefficients 1st friend’s input? feven: (a0,a2,a4,…,an-2) (x02,x12,x22,…,xn-12) Does not meet precondition! Oops

FFT nlogn Time My input: (a0,a1,a2,…,an-1) (x0,x1,x2,…,xn-1) My output: (y0,y1,y2,…,yn-1) yi = f(xi) 1st friend’s input? feven: (a0,a2,a4,…,an-2) (x02,x12,x22,…,xn/2-12) 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) (x02,x12,x22,…,xn/2-12) 3rd friend’s input? feven: (a0,a2,a4,…,an-2) (xn/22,xn/2+12,…,xn-12) 4th friend’s input? fodd: (a1,a3,a5,…,an-1) (xn/22,xn/2+12,…,xn-12) That’s no good. My output: i f(xi) = y<even,i> + xi y<odd,i> T(n) = 4 T(n/2) + O(n) = O(n2)

Roots of Unity The values (x0,x1,x2,…,xn-1) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n/2 distinct values. Eg: …, -3, -2, -1, 1, 2, 3, … square each of them …, 9, 4, 1, 1, 4, 9, … collapse the set 1, 4, 9, … half as many elements.

Roots of Unity My input: (a0,a1,a2,…,an-1) Special (x0,x1,x2,…,xn-1) My output: (y0,y1,y2,…,yn-1) yi = f(xi) 1st friend’s input? feven: (a0,a2,a4,…,an-2) (x02,x12,x22,…,xn-12) n/2 distinct values 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) (x02,x12,x22,…,xn-12) n/2 distinct values 9 2nd friend’s output: i y<odd,i> = fodd(xi2) 1st friend’s output: i y<even,i> = feven(xi2) feven(9) fodd(9) My output: i f(xi) = y<even,i> + xi y<odd,i> T(n) = 2 T(n/2) + O(n) = O(n log n) That’s better 3 f(3) -3 f(-3) feven(9) fodd(9)

Roots of Unity My input: (a0,a1,a2,…,an-1) Special (x0,x1,x2,…,xn-1) My output: (y0,y1,y2,…,yn-1) yi = f(xi) To meet precondition these also need to be special 1st friend’s input? feven: (a0,a2,a4,…,an-2) (x02,x12,x22,…,xn-12) n/2 distinct values 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) (x02,x12,x22,…,xn-12) n/2 distinct values

Roots of Unity The values (x0,x1,x2,…,xn-1) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n/2 distinct values. Which are also special Eg: …, -3, -2, -1, 1, 2, 3, … square each of them …, 9, 4, 1, 1, 4, 9, … collapse the set 1, 4, 9, … square each of them 2, 16, 81, … But these are not special.

Roots of Unity The values (x0,x1,x2,…,xn-1) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n/2 distinct values. Which are also special Eg: -i, -1, 1, i square each of them -1, 1, 1, -1  = -i,-1, 1, i are said to be 4th roots of unity Because 4 = 1 collapse the set -1, 1 square each of them 1, 1 collapse the set 1

Roots of Unity  is said to be an nth root of unity (in a field) if n = 1 (There should be n solutions of this polynomial) Fermat’s Little Theorem: b≠0 bp-1 =mod p says every nonzero element is an nth root of unity when n=p-1.

Roots of Unity  is said to be an nth root of unity (in a field) if n = 1  is said to be a generator of the field if the numbers 1,,2, …,n-1 are all distinct 1,,2, …,n-1 are then special (when n is even) 1st half 2nd half n/2+n/2-1,…,n/2+3,n/2+2,n/2+1,n/2+0 ,0,1,2,3,…,n/2-1 square each of them n+n-2, …, n+6, n+4, n+2, n+0, 0,2,4,6, …,n-2 use n = 1 n-2, …, 6, 4, 2, 0, 0,2,4, 6, …,n-2 collapse the set 0,2,4, 6, …,n-2 We need these to be n/2 special values.

Roots of Unity 16th roots of unity (n/4)2 = n/2 = -1 i These could be Z mod 17 or complex numbers × 164 165 163 166 162 rr 167 161 (n/2)2 = 1 r θ 168 -1 = 160 = 1616 = 1 1615 169 1614 1610 1613 1611 1612 reθi = rcosθ + irsinθ reθi × seαi = (rs)e(θ+α)i (3n/4)2 = n/2 = -1 -i

Roots of Unity Goal: Proof f(θ) = g(θ) f(0) = g(0) f’(0) = g’(0) f’’(θ) = -f(θ) g’’(θ) = -g(θ) Proof by induction (over the reals) that f(θ) = g(θ) f(θ) g(θ) For this θ, f(θ) = g(θ) and f’(θ) = g’(θ) For next θ+, f(θ+) = g(θ+) f’’(θ) = -f(θ) = -g(θ) =g’’(θ) For next θ+, f’(θ+) = g’(θ+)

Roots of Unity g(θ) = rcosθ + irsinθ f(θ) = reθi Goal: Proof f(θ) = g(θ) g(0) = rcos0 + irsin0 = r f(0) = re0i = r f(0) = g(0) g’(θ) = -rsinθ + ircosθ f’(θ) = ireθi g’(0) = -rsin0 + ircos0 = ir f’(0) = ire0i = ir f’(0) = g’(0) g’’(θ) = -rcosθ - rsinθ f’’(θ) = -reθi = -f(θ) = -g(θ)

Roots of Unity 16th roots of unity square each of them and collapse 164 165 163 166 162 167 161 168 160 1615 169 1614 1610 1613 1611 1612

Roots of Unity 16th roots of unity square each of them and collapse 164 Are these special? 166 162 168 160 1614 1610 1612

Roots of Unity 8th roots of unity square each of them and collapse 82 Are these special? 83 81 84 80 87 85 86

Roots of Unity 4th roots of unity square each of them and collapse 41 Are these special? 42 40 43

Roots of Unity 2th roots of unity square each of them and collapse Are these special? 21 20 = 1

Roots of Unity My input: (a0,a1,a2,…,an-1) (nth roots of unity ni) My output: (y0,y1,y2,…,yn-1) yi = f(ni) 1st friend’s input? feven: (a0,a2,a4,…,an-2) (n/2th roots of unity n/2i) 2nd friend’s input? fodd: (a1,a3,a5,…,an-1) (n/2th roots of unity n/2i) 2nd friend’s output: i y<odd,i> = fodd(n/2i) 1st friend’s output: i y<even,i> = feven(n/2i) My output: i f(xi) = y<even,i> + xi y<odd,i> T(n) = 2 T(n/2) + O(n) = O(n log n) Excellent

FFT Code Algorithm FFT(y, , n): Input: y = [a0,a1,a2,…,an-1] (Time Domain) = e2i 1/n (nth root of unity) n = # of samples (2r) Output: Y = [y0,y1,y2,…,yn-1] (Frequency Domain) % Separate even and odd indices aeven = [a0,a2,a4,…,an-2] aodd = [a1,a1,a5,…,an-1] % Recurse yeven =FFT(aeven, 2, n/2) (2 = e2i 2/n ) yodd =FFT(aodd , 2, n/2) %Combining For i = 0 to n/2-1 y[i] = yeven[i] + i ∙yodd[i] y[i+n/2] = yeven[i] + i+n/2 ∙yodd[i] Return(Y)

FFT Code

FFT Butterfly

Inverse FFT A inverse Fourier transform changes the representation of a function x0 x1 x2 x3 x … xn-1 y0 y1 y2 y3 y … yn-1 yi = f(xi) from the value f(xi) at key values xi. to the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 This amounts to interpolating these points. 150

Inverse FFT A Fourier Transform is a change in basis. It changes the representation of a function from the coefficients of the polynomial f(x) = a0+a1x +a2x2 + … + an-1xn-1 (x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (x1)0 (x1)1 (x1)2 (x1)3 … (x1)n-1 (xn-1)0(xn-1)1(xn-1)2 (xn-1)3…(xn-1)n-1 (x2)0 (x2)1 (x2)2 (x2)3 … (x2)n-1 (x3)0 (x3)1 (x3)2 (x3)3 … (x3)n-1 (xi)j yi = f(xi) This amounts to evaluating f at these points. 151

xi = i (x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2
Inverse FFT (x0)0 (x0)1 (x0)2 (x0)3 … (x0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (x1)0 (x1)1 (x1)2 (x1)3 … (x1)n-1 (xn-1)0(xn-1)1(xn-1)2 (xn-1)3…(xn-1)n-1 (x2)0 (x2)1 (x2)2 (x2)3 … (x2)n-1 (x3)0 (x3)1 (x3)2 (x3)3 … (x3)n-1 (xi)j xi = i (0)0 (0)1 (0)2 (0)3 … (0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (1)0 (1)1 (1)2 (1)3 … (1)n-1 (2)0 (2)1 (2)2 (2)3 … (2)n-1 (3)0 (3)1 (3)2 (3)3 … (3)n-1 (n-1)0 (n-1)1 (n-1)2 (n-1)3 … (n-1)n-1 (i)j 152

(i)j = ij Vandermonde matrix (0)0 (0)1 (0)2 (0)3 … (0)n-1 a0 a1
Inverse FFT (0)0 (0)1 (0)2 (0)3 … (0)n-1 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = (1)0 (1)1 (1)2 (1)3 … (1)n-1 (2)0 (2)1 (2)2 (2)3 … (2)n-1 (3)0 (3)1 (3)2 (3)3 … (3)n-1 (n-1)0 (n-1)1 (n-1)2 (n-1)3 … (n-1)n-1 (i)j (i)j = ij 0 0 0 0 … 0 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = 0 1 2 3 … n-1 0 2 4 6 … 2n-2 0 3 6 9 … 3n-3 0 n-1 2n-2 3n-3 … (n-1)(n-1) ij Vandermonde matrix 153

V a = y a = V-1 y V-1 = 1/n V -1 Vandermonde matrix
Inverse FFT 0 0 0 0 … 0 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = 0 1 2 3 … n-1 0 2 4 6 … 2n-2 0 3 6 9 … 3n-3 0 n-1 2n-2 3n-3 … (n-1)(n-1) ij Vandermonde matrix V a = y a = V-1 y V-1 = 1/n V -1 154

1/n Vandermonde matrix V-1 = 1/n V 0 0 0 0 … 0 a0 a1 a2 a3 …
Inverse FFT 0 0 0 0 … 0 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = 0 1 2 3 … n-1 0 2 4 6 … 2n-2 0 3 6 9 … 3n-3 0 n-1 2n-2 3n-3 … (n-1)(n-1) ij Vandermonde matrix V-1 = 1/n V -1 0 0 0 0 … 0 y0 y1 y2 y3 … yn-1 a0 a1 a2 a3 an-1 = 0 -1 -2 -3 … -(n-1) 0 -2 -4 -6 … -(2n-2) 0 -3 -6 -9 … -3(n-3) 0 -(n-1) -(2n-2) -(3n-3) … -(n-1)(n-1) -ij 1/n 155

If w is an nth root of unity and a generator of the field 161 160 162 163 164 165 166 167 168 169 1610 1611 1612 1613 1614 1615 = 1616 = 1 -1 = rr

Inverse FFT The inverse w-1 of w is then w-1=wn-1 And is also an nth root of unity and a generator of the field Proof: ww-1 = wwn-1 = wn=1 161 160 162 163 164 165 166 167 168 169 1610 1611 1612 1613 1614 1615 = 1616 = 1 -1 = rr

Inverse FFT Cancellation Property: 161 160 162 163 164 165 166 167 168 169 1610 1611 1612 1613 1614 1615 = 1616 = 1 -1 = rr

1/n Vandermonde matrix V-1 = 1/n V 0 0 0 0 … 0 a0 a1 a2 a3 …
Inverse FFT Vandermonde matrix 0 0 0 0 … 0 a0 a1 a2 a3 … an-1 y0 y1 y2 y3 yn-1 = 0 1 2 3 … n-1 0 2 4 6 … 2n-2 0 3 6 9 … 3n-3 0 n-1 2n-2 3n-3 … (n-1)(n-1) ij V-1 = 1/n V -1 0 0 0 0 … 0 0 -1 -2 -3 … -(n-1) 0 -2 -4 -6 … -(2n-2) 0 -3 -6 -9 … -3(n-3) 0 -(n-1) -(2n-2) -(3n-3) … -(n-1)(n-1) -ij 1/n 159

Inverse FFT The FFT and inverse FFT really are inverse operations Proof: Let A=D -1D. We want to show that A=I, where If i=j, then If i and j are different, then

Inverse FFT The FFT and inverse FFT can use the same hardware FFT Input: <1, , a0,a1,a2,…,an-1 > Output: <y0,y1,y2,…,yn-1 > Inverse FFT Input: <1/n, -1, y0,y1,y2,…,yn-1 > Output: <a0,a1,a2,…,an-1 >

Polynomial Multiplication
f(x) = a0+a1x +a2x2 + … + an-1xn-1 g(x) = b0+b1x +b2x2 + … + bn-1xn-1 [f×g](x) = c0+c1x +c2x … c2n-2x2n-2 x5 coefficient: c5= a0×b5+a1×b4 + a2×b3 + … + a5×b0 Convolution Too much O(n2) Time =

Polynomial Multiplication
Pad with n 0's [ a , 1 2 ,..., -1 ] b DFT ,0,0,...,0] y z Component Multiply inverse DFT c (Convolution) yi = f(xi) zi = g(xi) yi×zi = [g×f](xi)

Multiplying Big Integers
X = 11… (N bits) Y = 10… X×Y = 10… The high school algorithm takes O(N2) bit operations. Can we do it faster? I hope so See Recursion for one way to do it faster. This is another.

Grade School Revisited: How To Multiply Two Numbers
Multiplying Big Integers Grade School Revisited: How To Multiply Two Numbers

X = 0011 … m Break into m = O(log N) bit blocks

X = 0000 … n=2r blocks m O(log p) an … a a6 a a a3 a a a0 Break into m = O(log N) bit blocks Pad with zero 2N bits to hold product n blocks where n is a power of 2, ie n=2r. Let p be a prime log p ≥ block size = m p-1 is divisible by n, so Z mod p has n nth roots of unity. View each block as a finite field element in Z mod p (no actual work)

X = 0000 … m f(x) = an-1xn-1 + … + a5x5 + a4x4 + a3x3 + a2x2 + a1x + a0 g(x) = bn-1xn-1 + … + b5x5 + b4x4 + b3x3 + b2x2 + b1x + b0 View as coefficients of a polynomial. Note X = f(2m). Same for Y = g(2m). Multiply g×f using FFT in time O(nlogn). Note X×Y = [g×f](2m). Evaluate [g×f](2m) in time O(n) operations, but each op could be on O(n) bit numbers for a total of O(n2) time.

Evaluate [g×f](2m) in time O(n). [g×f](x) = cn-1xn-1 + … + c5x5 + c4x4 + c3x3 + c2x2 + c1x + c0 X×Y = m O(log p) Some texts say the ci can just be shifted and joined. Problem: The field elements may be too big.

Evaluate [g×f](2m) in time O(n). [g×f](x) = cnxn + … + c5x5 + c4x4 + c3x3 + c2x2 + c1x + c0 Shift each ci by im. Add m O(log p) 101011 001101 110001 000111 100100 010011 X×Y = Adding n numbers each n bits long takes O(n2) but here the numbers are sparse.

Evaluate [g×f](2m) in time O(n). [g×f](x) = cnxn + … + c5x5 + c4x4 + c3x3 + c2x2 + c1x + c0 Shift each ci by im. Add m O(log p) 101011 001101 110001 000111 100100 010011 X×Y = At each point, at most two numbers overlap  Carry is at most one  O(N) bit operations.

X = 11… (N bits) Y = 10… X×Y = 10… Suppose N is really really big. How many bit operations are needed? O(N logN) O(N logN loglogN) O(N logN loglogN logloglogN loglogloglogN …) FFT time Time stated in text Time as far as I can see

X = … … N’ Input size = N bits Field element size = N’ = log(N) bits # ai = n = N/N’ # of field ops = O(nlogn) Time for × field op = ? X’ = N’’ Input size = N’ bits Field element size = N’’ = log(N’) bits # ai = n’ = N’/N’’ # of field ops = O(n’logn’) Time for × field op = ? Total time: O(N’ logN’ loglogN’ logloglogN’ …) And so on …

X = … … N’ Input size = N bits Field element size = N’ = log(N) bits # ai = n = N/N’ # of field ops = O(nlogn) Time for × field op = ? Total time: = O( n logn ) × O(N’ logN’ loglogN’ logloglogN’ …) = O(N/N’ logN/N’) × O(N’ loglogN logloglogN loglogloglogN …) = O( N logN loglogN logloglogN loglogloglogN …) O(N’ logN’ loglogN’ logloglogN’ …)

The End

eθi = cosθ + isinθ i -i -1 1 θ 176

B1[2]=2 B1[3]=3 B1[1]=1 =e2i 1/n B1[4]=4 B1[0]=0 B1[7]=7 B1[5]=5 B1[6]=6

The 0th basis vector will be the constant function 1. B0[j]=j0  j 178

The 1st basis vector will be the linear function x. B1[j]=j1  j 179

The 2nd basis vector will be the quadratic function x2. B2[j]=j2  j 180

Bn-1[j]=jn-1  j 181

Fourier Transformations

Similar presentations

Presentation on theme: "Fourier Transformations"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Fourier Transformations

Similar presentations

Presentation on theme: "Fourier Transformations"— Presentation transcript:

Similar presentations

About project

Feedback