1. Fourier Transformations Jeff Edmonds York University COSC 6111 Change from Time to Polynomial Basis Evaluating & Interpolating FFT in nlogn Time Roots of Unity Same FFT Code & Butterfly Inverse FFT Sin and Cos Basis FFT Butterfly Polynomial Multiplication Integer Multiplication

2. Fourier Transformation Fourier Transform are a change of basis from the time basis to sine/cosine basis JPG or polynomial basis Purposes: Signal Processing Compressing data (eg images with.jpg) Multiplying integers in n logn loglogn time. Error correcting codes. …. http://www.dspguide.com/ch8.htm The Scientist and Engineer's Guide to Digital Signal Processing By Steven W. Smith, Ph.D. Amazingly once you include complex numbers, the FFT code for sine/cosines and for polynomials are the SAME.

3. Fourier Transformation Instead of using sine and cosines as the basis, Polynomial Basis

4. Fourier Transformation Instead of using sine and cosines as the basis, We use polynomials. Polynomial Basis

5. Change of Basis: T([a 1,a 2,…,a d ]) = [A 1,A 2,…,A d ] Changes the basis used to describe an object. The Time basis of a vector space: A tuple of basis objects Linearly independent Spans the space uniquely  v  [a 1,a 2,…,a d ], v = a 1 w 1 +a 2 w 2 +… + a d w d The Polynomial basis of a vector space: A tuple of basis objects Linearly independent Spans the space uniquely  v  [A 1,A 2,…,A d ], v = A 1 W 1 +A 2 W 2 +… + A d W d Polynomial Basis Fourier Transformation

6. [3,2] v = [a 1,a 2 ] =[1 1 / 5,3 2 / 5 ] [A 1,A 2 ] = Change of Basis: T([a 1,a 2,…,a d ]) = [A 1,A 2,…,A d ] Changes the basis used to describe an object.  v  [a 1,a 2,…,a d ], v = a 1 w 1 +a 2 w 2 +… + a d w d  v  [A 1,A 2,…,A d ], v = A 1 W 1 +A 2 W 2 +… + A d W d =[w 1,w 2 ] =[, ] Standard Basis v = Polynomial Basis New Basis =[W 1,W 2 ] = [, ] W 1 [1] W 1 [2] W 2 [1] W 2 [2] [ ][ ] = [ ] W 1 [1] W 1 [2] W 2 [1] W 2 [2] a1a1 a2a2 A1A1 A2A2 Fourier Transformation

7.  f = a 0 ∙B 0 +a 1 ∙B 1 +a 2 ∙B 2 +… + a n-1 ∙B n-1 =[B 0,B 1,..] f = a 0 =1 1 / 5 a 1 =3 2 / 5 Polynomial Basis =[, ] Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a 1,a 2,…,a n-1 ] Changes the basis used to describe an object. A discrete function x f(x) f(x) = a 0 +a 1 x +a 2 x 2 + …+ a n-1 x n-1 coefficients of the polynomial [a 1,a 2,…,a n-1 ] = Polynomial Basis

8. Have you seen Taylor Expansions of a Function? They show that functions f(x) can be expressed by specifying the coefficients of the polynomial. F(x) = a 0 +a 1 x +a 2 x 2 +a 3 x 3 + … Eg: f(x) = 1/(1-x) F(x) = 1+x +x 2 +x 3 + … Fourier Transformation Polynomial Basis

9. Have you seen Taylor Expansions of a Function? They show that functions f(x) can be expressed by specifying the coefficients of the polynomial. F(x) = a 0 +a 1 x +a 2 x 2 +a 3 x 3 + … Fourier Transformation Polynomial Basis

10. Have you seen Taylor Expansions of a Function? They show that functions f(x) can be expressed by specifying the coefficients of the polynomial. F(x) = a 0 +a 1 x +a 2 x 2 +a 3 x 3 + … Fourier Transformation Polynomial Basis

11. Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a 1,a 2,…,a n-1 ] Changes the basis used to describe an object. Time Basis =[, ] f = Fourier Transformation y[0]=3 y[1]=2 =[I 0,I 1,…] =[?,?] The time basis x I j [x] zero one j  y  [y[0],y[1],…,y[n-1]], y = y[0]I 0 +y[1]I 1 +… + y[n-1]I n-1 Polynomial Basis A discrete function x f(x) the value f(j) of the function at x=j. y[j] = the value f(j) of the function at x=j.

12. Fourier Transformation Instead of evaluating the polynomials on j = 0, 1, 2,…, n-1 We evaluate them on x 0, x 1, x 2, …, x n-1 These x j are special fixed values. Later we set x j = e 2  i j/n Polynomial Basis the value f(j) of the function at x=j. y[j] = the value f(j) of the function at x=j.

13. Change of Basis: T([y[0],y[1],…, y[n-1]]) = [a 1,a 2,…,a n-1 ] Changes the basis used to describe an object. Time Basis =[, ] f = Fourier Transformation y[0]=3 y[1]=2 =[I 0,I 1,…] =[?,?] The time basis x I j [x] zero one xjxj  y  [y[0],y[1],…,y[n-1]], y = y[0]I 0 +y[1]I 1 +… + y[n-1]I n-1 Polynomial Basis A discrete function x f(x) the value f(x j ) of the function at x=j. y[j] = the value f(j) of the function at x=j. x 0 x 1 x 2 x 3 x 4 … x n-1

14. A Fourier Transform is a change in basis. It changes the representation of a function from the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to evaluating f at these points. Evaluating & Interpolating x to the value f(x i ) at key values x i. x 0 x 1 x 2 x 3 x 4 … x n-1 y 0 y 1 y 2 y 3 y 4 … y n-1 Fourier Transformation y i = f(x i )

15. A Fourier Transform is a change in basis. It changes the representation of a function Fourier Transformation from the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to evaluating f at these points. (x 0 ) 0 (x 0 ) 1 (x 0 ) 2 (x 0 ) 3 … (x 0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (x 1 ) 0 (x 1 ) 1 (x 1 ) 2 (x 1 ) 3 … (x 1 ) n-1 (x n-1 ) 0 (x n-1 ) 1 (x n-1 ) 2 (x n-1 ) 3 …(x n-1 ) n-1 (x 2 ) 0 (x 2 ) 1 (x 2 ) 2 (x 2 ) 3 … (x 2 ) n-1 (x 3 ) 0 (x 3 ) 1 (x 3 ) 2 (x 3 ) 3 … (x 3 ) n-1 Vandermonde matrix Invertible if x i distinct. Evaluating & Interpolating y i = f(x i )

16. Fourier Transformation to the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to interpolating these points. An Inverse Fourier Transform is the reverse. It changes the representation of a function Evaluating & Interpolating x from the value f(x i ) at key values x i. x 0 x 1 x 2 x 3 x 4 … x n-1 y 0 y 1 y 2 y 3 y 4 … y n-1 y i = f(x i )

17. Fourier Transformation to the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to interpolating these points. Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points. An Inverse Fourier Transform is the reverse. It changes the representation of a function Evaluating & Interpolating

18. f(x) = a 0 +x(a 1 +x(a 2 + … + x(a n-1 ) … )) Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) & x My output: f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 1+2+3+..+n-1 multiplications? n multiplications and n additions. Given n distinct values (x 0,x 1,x 2,…,x n-1 ). Can they all be done in less than n×O(n) time? Evaluating & Interpolating

19. Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) & x My output: f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 Given n distinct values (x 0,x 1,x 2,…,x n-1 ) Can they all be done in less than n×O(n) time? (x 0 ) 0 (x 0 ) 1 (x 0 ) 2 (x 0 ) 3 … (x 0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (x 1 ) 0 (x 1 ) 1 (x 1 ) 2 (x 1 ) 3 … (x 1 ) n-1 (x n-1 ) 0 (x n-1 ) 1 (x n-1 ) 2 (x n-1 ) 3 …(x n-1 ) n-1 (x 2 ) 0 (x 2 ) 1 (x 2 ) 2 (x 2 ) 3 … (x 2 ) n-1 (x 3 ) 0 (x 3 ) 1 (x 3 ) 2 (x 3 ) 3 … (x 3 ) n-1 y i = f(x i ) I hope so Evaluating & Interpolating

20. The Fast Fourier Transform (FFT) is a very efficient algorithm for performing a discrete Fourier transform FFT principle first used by Gauss in 18?? (But was not interesting without computers) FFT algorithm published by Cooley & Tukey in 1965 In 1969, the 2048 point analysis of a seismic trace took 13 ½ hours. Using the FFT, the same task on the same machine took 2.4 seconds! Fast Fourier Transformation FFT nlogn Time

21. 16 32 64 128 256 512 1024 70 60 50 40 30 20 10 0 Error (ppm) DFTFFT Not only do you get faster speed + in place memory processing but fewer calculations means less round off errors Maybe I should take CSE6111 after all! Fast Fourier Transformation FFT nlogn Time

22. N DFT (N 2 ) FFT (1.5N log N)faster 321,0242404.3 644,0965767.1 12816,3841,34412.2 25665,5363,07221.3 512262,1446,91237.9 10241,048,57615,36068.2 20484,194,30433,792124.1 409616,777,21673,728227.6 Discrete Fourier Transform is too slow for real time! Fast Fourier Transformation FFT nlogn Time

23. Divide & Conquer - Friends - Recursion. Trust your friends to solve any subinstance: as long as smaller and is an instance to the same problem. Fast Fourier Transformation My instance My friend’s Instance My friend’s Instance My friend’s Instance FFT nlogn Time

24. f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 Fast Fourier Transformation My input: (start with one x) (a 0,a 1,a 2,…,a n-1 ) & x = a 0 +a 2 x 2 +a 4 x 4 + … + a n-2 x n-2 + a 1 x+a 3 x 3 +a 5 x 5 + … + a n-1 x n-1 = a 0 +a 2 x 2 +a 4 x 4 + … + a n-2 x n-2 + x( a 1 +a 3 x 2 +a 5 x 4 + … + a n-1 x n-2 ) = f even (x 2 ) + x( f odd (x 2 ) ) My output: f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) & ? 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) & ? f(x) = f even (x 2 ) + x f odd (x 2 ) f even (z) = a 0 +a 2 z+a 4 z 2 +a 6 z 3 + … + a n-2 z n/2-1 FFT nlogn Time

25. Fast Fourier Transformation My input: (start with one x) (a 0,a 1,a 2,…,a n-1 ) & x My output: f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) & x 2 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) & x 2 1 st friend’s output: y even = f even (x 2 ) 2nd friend’s output: y odd = f odd (x 2 ) f(x) = f even (x 2 ) + x f odd (x 2 ) My output: f(x) = y even + x y odd T(n) = 2 T(n/2) + O(1) = O(n) Ok. So it takes O(n) time to evaluate. FFT nlogn Time

26. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) (x 0,x 1,x 2,…,x n-1 ) My output: (y 0,y 1,y 2,…,y n-1 ) y i = f(x i ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) 1 st friend’s output:  i y = f even (x i 2 ) 2 nd friend’s output:  i y = f odd (x i 2 ) My output:  i f(x i ) = y + x i y T(n) = 2 T(n/2) + O(n) = O(n log n) Wow! That was easy. FFT nlogn Time

27. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) (x 0,x 1,x 2,…,x n-1 ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) Oops n coefficients n values of x n / 2 coefficients n values of x Does not meet precondition! FFT nlogn Time

28. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) (x 0,x 1,x 2,…,x n-1 ) My output: (y 0,y 1,y 2,…,y n-1 ) y i = f(x i ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x 0 2,x 1 2,x 2 2,…,x n/2-1 2 ) 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) (x 0 2,x 1 2,x 2 2,…,x n/2-1 2 ) My output:  i f(x i ) = y + x i y T(n) = 4 T(n/2) + O(n) That’s no good. 3 rd friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x n/2 2,x n/2+1 2,…,x n-1 2 ) 4 th friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) (x n/2 2,x n/2+1 2,…,x n-1 2 ) = O(n 2 ) FFT nlogn Time

29. Fast Fourier Transformation The values (x 0,x 1,x 2,…,x n-1 ) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n / 2 distinct values. Eg: …, -3, -2, -1, 1, 2, 3, … square each of them …, 9, 4, 1, 1, 4, 9, … collapse the set 1, 4, 9, … half as many elements. Roots of Unity

30. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) Special (x 0,x 1,x 2,…,x n-1 ) My output: (y 0,y 1,y 2,…,y n-1 ) y i = f(x i ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) n / 2 distinct values 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) n / 2 distinct values 1 st friend’s output:  i y = f even (x i 2 ) 2 nd friend’s output:  i y = f odd (x i 2 ) My output:  i f(x i ) = y + x i y T(n) = 2 T(n/2) + O(n) = O(n log n) That’s better -3 3 f even (9) f odd (9) 9 9 f even (9)f odd (9) 3 f(3) -3 f(-3) Roots of Unity

31. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) Special (x 0,x 1,x 2,…,x n-1 ) My output: (y 0,y 1,y 2,…,y n-1 ) y i = f(x i ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) n / 2 distinct values 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) (x 0 2,x 1 2,x 2 2,…,x n-1 2 ) n / 2 distinct values To meet precondition these also need to be special Roots of Unity

32. Fast Fourier Transformation The values (x 0,x 1,x 2,…,x n-1 ) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n / 2 distinct values. Which are also special Eg: …, -3, -2, -1, 1, 2, 3, … square each of them …, 9, 4, 1, 1, 4, 9, … collapse the set 1, 4, 9, … But these are not special. square each of them 2, 16, 81, … Roots of Unity

33. Fast Fourier Transformation The values (x 0,x 1,x 2,…,x n-1 ) are said to be special if: There are n distinct values. When you square each of them: The set collapses to n / 2 distinct values. Which are also special Eg: -i, -1, 1, i square each of them -1, 1, 1, -1 collapse the set -1, 1 square each of them 1, 1 collapse the set 1  = -i,-1, 1, i are said to be 4 th roots of unity Because  4 = 1 Roots of Unity

34. Fast Fourier Transformation  is said to be an n th root of unity (in a field) if  n = 1 (There should be n solutions of this polynomial) Fermat’s Little Theorem:  b≠0 b p-1 = mod p 1 says every nonzero element is an n th root of unity when n=p-1. Roots of Unity

35. Fast Fourier Transformation  is said to be an n th root of unity (in a field) if  n = 1  is said to be a generator of the field if the numbers 1, ,  2, …,  n-1 are all distinct 1, ,  2, …,  n-1 are then special (when n is even)  n/2+n/2-1,…,  n/2+3,  n/2+2,  n/2+1,  n/2+0,  0,  1,  2,  3,…,  n/2-1 square each of them  n+n-2, …,  n+6,  n+4,  n+2,  n+0,  0,  2,  4,  6, …,  n-2 use  n = 1  n-2, …,  6,  4,  2,  0,  0,  2,  4,  6, …,  n-2 collapse the set  0,  2,  4,  6, …,  n-2 We need these to be n / 2 special values. 1 st half 2 nd half Roots of Unity

36.  16 1  16 0  16 2  16 3  16 4  16 5  16 6  16 7  16 8  16 9  16 10  16 11  16 12  16 13  16 14  16 15 =  16 16 = 1 Fast Fourier Transformation 16 th roots of unity -1 = i -i (  n/2 ) 2 = 1 (  n/4 ) 2 =  n/2 = -1 (  3n/4 ) 2 =  n/2 = -1 These could be Z mod 17 or complex numbers ××r r θ re θi = rcosθ + irsinθ re θi × se αi = (rs)e (θ+α)i Roots of Unity

37. Fast Fourier Transformation Goal: Proof f(θ) = g(θ) f(0) = g(0) f’(0) = g’(0) f’’(θ) = -f(θ) g’’(θ) = -g(θ) Proof by induction (over the reals) that f(θ) = g(θ) f(θ)g(θ) For this θ, f(θ) = g(θ) and f’(θ) = g’(θ) For next θ+ , f(θ+  ) = g(θ+  ) f’’(θ) = -f(θ) = -g(θ) =g’’(θ) For next θ+ , f’(θ+  ) = g’(θ+  ) Roots of Unity

38. Fast Fourier Transformation f(θ) = re θi g(θ) = rcosθ + irsinθ Goal: Proof f(θ) = g(θ) f(0) = re 0i = r g(0) = rcos0 + irsin0 = r f’(θ) = ire θi g’(θ) = -rsinθ + ircosθ f(0) = g(0) f’(0) = ire 0i = ir g’(0) = -rsin0 + ircos0 = ir f’(0) = g’(0) f’’(θ) = -re θi g’’(θ) = -rcosθ - rsinθ = -f(θ)= -g(θ) Roots of Unity

39.  16 1  16 0  16 2  16 4  16 5  16 6  16 7  16 8  16 9  16 10  16 11  16 12  16 13  16 14  16 15 Fast Fourier Transformation 16 th roots of unity square each of them  16 3 and collapse Roots of Unity

40.  16 0  16 2  16 4  16 6  16 8  16 10  16 12  16 14 Fast Fourier Transformation 16 th roots of unity Are these special? square each of them and collapse Roots of Unity

41. 8080 8181 8282 8383 8484 8585 8686 8787 Fast Fourier Transformation 8 th roots of unity Are these special? square each of them and collapse Roots of Unity

42. 4040 4141 4242 4343 Fast Fourier Transformation 4 th roots of unity Are these special? square each of them and collapse Roots of Unity

43. 2020 2121 Fast Fourier Transformation 2 th roots of unity Are these special? = 1 square each of them and collapse Roots of Unity

44. Fast Fourier Transformation My input: (a 0,a 1,a 2,…,a n-1 ) (n th roots of unity  n i ) My output: (y 0,y 1,y 2,…,y n-1 ) y i = f(  n i ) 1 st friend’s input? f even : (a 0,a 2,a 4,…,a n-2 ) ( n / 2 th roots of unity  n/2 i ) 2 nd friend’s input? f odd : (a 1,a 3,a 5,…,a n-1 ) ( n / 2 th roots of unity  n/2 i ) 1 st friend’s output:  i y = f even (  n/2 i ) 2 nd friend’s output:  i y = f odd (  n/2 i ) My output:  i f(x i ) = y + x i y T(n) = 2 T(n/2) + O(n) = O(n log n) Excellent Roots of Unity

45. Fourier Transformation Algorithm FFT(y, , n): Input: y = [a 0,a 1,a 2,…,a n-1 ] (Time Domain)  = e 2  i 1/n (n th root of unity) n = # of samples (2 r ) Output: Y = [y 0,y 1,y 2,…,y n-1 ] (Frequency Domain) % Separate even and odd indices a even = [a 0,a 2,a 4,…,a n-2 ] a odd = [a 1,a 1,a 5,…,a n-1 ] % Recurse y even =FFT(a even,  2, n / 2 ) (  2 = e 2  i 2/n ) y odd =FFT(a odd,  2, n / 2 ) %Combining For i = 0 to n / 2 -1 y[i] = y even [i] +  i ∙y odd [i] y[i+ n / 2 ] = y even [i] +  i+n/2 ∙y odd [i] Return(Y) FFT Code

46. A inverse Fourier transform changes the representation of a function x 0 x 1 x 2 x 3 x 4 … x n-1 y 0 y 1 y 2 y 3 y 4 … y n-1 y i = f(x i ) from the value f(x i ) at key values x i. to the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to interpolating these points. Inverse FFT Fast Fourier Transformation

47. A Fourier Transform is a change in basis. It changes the representation of a function Fast Fourier Transformation from the coefficients of the polynomial f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 This amounts to evaluating f at these points. (x 0 ) 0 (x 0 ) 1 (x 0 ) 2 (x 0 ) 3 … (x 0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (x 1 ) 0 (x 1 ) 1 (x 1 ) 2 (x 1 ) 3 … (x 1 ) n-1 (x n-1 ) 0 (x n-1 ) 1 (x n-1 ) 2 (x n-1 ) 3 …(x n-1 ) n-1 (x 2 ) 0 (x 2 ) 1 (x 2 ) 2 (x 2 ) 3 … (x 2 ) n-1 (x 3 ) 0 (x 3 ) 1 (x 3 ) 2 (x 3 ) 3 … (x 3 ) n-1 (x i ) j y i = f(x i ) Inverse FFT

48. x i =  i (x 0 ) 0 (x 0 ) 1 (x 0 ) 2 (x 0 ) 3 … (x 0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (x 1 ) 0 (x 1 ) 1 (x 1 ) 2 (x 1 ) 3 … (x 1 ) n-1 (x n-1 ) 0 (x n-1 ) 1 (x n-1 ) 2 (x n-1 ) 3 …(x n-1 ) n-1 (x 2 ) 0 (x 2 ) 1 (x 2 ) 2 (x 2 ) 3 … (x 2 ) n-1 (x 3 ) 0 (x 3 ) 1 (x 3 ) 2 (x 3 ) 3 … (x 3 ) n-1 (x i ) j (  0 ) 0 (  0 ) 1 (  0 ) 2 (  0 ) 3 … (  0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (  1 ) 0 (  1 ) 1 (  1 ) 2 (  1 ) 3 … (  1 ) n-1 (  2 ) 0 (  2 ) 1 (  2 ) 2 (  2 ) 3 … (  2 ) n-1 (  3 ) 0 (  3 ) 1 (  3 ) 2 (  3 ) 3 … (  3 ) n-1 (  n-1 ) 0 (  n-1 ) 1 (  n-1 ) 2 (  n-1 ) 3 … (  n-1 ) n-1 (i)j(i)j Inverse FFT

49. (  i ) j =  ij Vandermonde matrix (  0 ) 0 (  0 ) 1 (  0 ) 2 (  0 ) 3 … (  0 ) n-1 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 = (  1 ) 0 (  1 ) 1 (  1 ) 2 (  1 ) 3 … (  1 ) n-1 (  2 ) 0 (  2 ) 1 (  2 ) 2 (  2 ) 3 … (  2 ) n-1 (  3 ) 0 (  3 ) 1 (  3 ) 2 (  3 ) 3 … (  3 ) n-1 (  n-1 ) 0 (  n-1 ) 1 (  n-1 ) 2 (  n-1 ) 3 … (  n-1 ) n-1 (i)j(i)j  0  0  0  0 …  0 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 =  0  1  2  3 …  n-1  0  2  4  6 …  2n-2  0  3  6  9 …  3n-3  0  n-1  2n-2  3n-3 …  (n-1)(n-1) ijij Inverse FFT

50. V  a = y a = V  -1 y Vandermonde matrix  0  0  0  0 …  0 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 =  0  1  2  3 …  n-1  0  2  4  6 …  2n-2  0  3  6  9 …  3n-3  0  n-1  2n-2  3n-3 …  (n-1)(n-1) ijij V  -1 = 1 / n V  -1 Inverse FFT

51. Vandermonde matrix  0  0  0  0 …  0 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 =  0  1  2  3 …  n-1  0  2  4  6 …  2n-2  0  3  6  9 …  3n-3  0  n-1  2n-2  3n-3 …  (n-1)(n-1) ijij V  -1 = 1 / n V  -1  0  0  0  0 …  0 y 0 y 1 y 2 y 3 … y n-1 a 0 a 1 a 2 a 3 … a n-1 =  0  -1  -2  -3 …  -(n-1)  0  -2  -4  -6 …  -(2n-2)  0  -3  -6  -9 …  -3(n-3)  0  -(n-1)  -(2n-2)  -(3n-3) …  -(n-1)(n-1)  -ij 1/n1/n Inverse FFT

52. If  is an n th root of unity and a generator of the field  16 1  16 0  16 2  16 3  16 4  16 5  16 6  16 7  16 8  16 9  16 10  16 11  16 12  16 13  16 14  16 15 =  16 16 = 1 -1 = r Fast Fourier Transformation Inverse FFT

53. The inverse  -1  of  is –then  -1 =  n-1 –And is also an n th root of unity and a generator of the field –Proof:  -1 =  n-1 =  n =1  16 1  16 0  16 2  16 3  16 4  16 5  16 6  16 7  16 8  16 9  16 10  16 11  16 12  16 13  16 14  16 15 =  16 16 = 1 -1 = r Fast Fourier Transformation Inverse FFT

54. Cancellation Property:  16 1  16 0  16 2  16 3  16 4  16 5  16 6  16 7  16 8  16 9  16 10  16 11  16 12  16 13  16 14  16 15 =  16 16 = 1 -1 = r Fast Fourier Transformation Inverse FFT

55. Vandermonde matrix  0  0  0  0 …  0 a 0 a 1 a 2 a 3 … a n-1 y 0 y 1 y 2 y 3 … y n-1 =  0  1  2  3 …  n-1  0  2  4  6 …  2n-2  0  3  6  9 …  3n-3  0  n-1  2n-2  3n-3 …  (n-1)(n-1) ijij V  -1 = 1 / n V  -1  0  0  0  0 …  0 y 0 y 1 y 2 y 3 … y n-1 a 0 a 1 a 2 a 3 … a n-1 =  0  -1  -2  -3 …  -(n-1)  0  -2  -4  -6 …  -(2n-2)  0  -3  -6  -9 …  -3(n-3)  0  -(n-1)  -(2n-2)  -(3n-3) …  -(n-1)(n-1)  -ij 1/n1/n Inverse FFT

56. The FFT and inverse FFT really are inverse operations Proof: Let A=D -1 D. We want to show that A=I, where If i=j, then If i and j are different, then Fast Fourier Transformation Inverse FFT

57. Fast Fourier Transformation The FFT and inverse FFT can use the same hardware FFT Input: Output: Inverse FFT Input: Output: Inverse FFT

58. Modifies DFT frequency coefficient calculations: ReX[ k ] = x[n] cos(2πkn/N) 0 < k < N/2 x[i] ε Real ImX[ k ] = - x[n] sin(2πkn/N) Uses complex and polar numbers as a shorthand: X k = ReX[ k ] + i ImX[ k ] X k = x n e – i2πkn/N = x n ω kn Ʃ Ʃ n=0 N-1 Complex Ʃ n=0 N-1 r·e iθ = r·cosθ + i r·sinθ = r θ Ʃ n=0 N-1 ω = e – i2π/N N = 2 r Fast Fourier Transformation Sin & Cos basis

59. 1. Convert your N real sampled values to complex numbers by adding 0i to them 2. Feed this as the input to the FFT 3.Remove FFT output’s redundant information (i.e. all frequencies above N/2) x n = x n + 0i 0 < n < N-1 0 N/2 N-1 “Negative” Frequency ReX Even Symmetry About N/2 (f s /2) 0 N/2 ImX Odd Symmetry About N/2 (f s /2) “Negative” Frequency N-1 Fast Fourier Transformation Sin & Cos basis

60. Fast Fourier Transformation FFT Butterfly

61. (  0 ) 0 (  0 ) 1 (  0 ) 2 (  0 ) 3 … (  0 ) n-1 (  1 ) 0 (  1 ) 1 (  1 ) 2 (  1 ) 3 … (  1 ) n-1 (  2 ) 0 (  2 ) 1 (  2 ) 2 (  2 ) 3 … (  2 ) n-1 (  3 ) 0 (  3 ) 1 (  3 ) 2 (  3 ) 3 … (  3 ) n-1 (  n-1 ) 0 (  n-1 ) 1 (  n-1 ) 2 (  n-1 ) 3 …(  n-1 ) n-1... = x 0 x 1 x 2 x 3 … x N-1 X 0 X 1 X 2 X 3 … X N-1 Behold the Vandermonde matrix! But that’s O(N 2 ) !! The Ugly Math for the FFT

62. = x 0 x 1 x 2 x 3 … x N-1 X 0 X 1 X 2 X 3 … X N-1 But if I multiply the exponents...  0  n-1  2n-2  3n-3 …  (n-1)(n-1)  0  0  0  0 …  0  0  1  2  3 …  n-1  0  2  4  6 …  2n-2  0  3  6  9 …  3n-3... But that’s still O(N 2 ) !! The Ugly Math for the FFT

63. Just watch! For example, if N=8 and I use the N roots of unity... 0000000000000000 0123456701234567  0  2  4  6  8  10  12  14  0  3  6  9  12  15  18  21  0  4  8  12  16  20  24  28  0  5  10  15  20  25  30  35  0  6  12  18  24  30  36  42  0  7  14  21  28  35  42  49 = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 44 00 11 22 33 55 66 77  p+4 = -  p  p =  p mod 8  =  8 = e -i2π/8  0 = 1  4 = -1

64. = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 1 1111111111111111 1   2  3 -  -  2 -  3 1  2 -  2 1  2 -  2 1  3 -  2  -  3  2 -  1 1 1 1 1 -   2 -  3  -  2  3 1 -  2  2 1 -  2  2 1 -  3 -  2 -   3  2  Now the 2 nd half of each row either equals the 1 st half or its negative + + + - + + + -

65. = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 1 1111111111111111 1  3 -  2  -  3  2 -  1 1 1 1 1 -  3 -  2 -   3  2  1   2  3 -  -  2 -  3 1 -   2 -  3  -  2  3 1  2 -  2 1  2 -  2 1 -  2  2 1 -  2  2 x4x4 x0x0 + + + - + + + -  x 0 and x 4 have identical coefficients (ignoring sign) as do: x 1 and x 5 x 2 and x 6 x 3 and x 7

66. = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 + + + - + + + - 1 1111111111111111 1  3 -  2  -  3  2 -  1 1 1 1 1 -  3 -  2 -   3  2  1   2  3 -  -  2 -  3 1 -   2 -  3  -  2  3 1  2 -  2 1  2 -  2 1 -  2  2 1 -  2  2 x1x1 x5x5

67. = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7  x 0 and x 4 have identical coefficients (ignoring sign) as do: x 1 and x 5 x 2 and x 6 x 3 and x 71 1111111111111111 1  3 -  2  -  3  2 -  1 1 1 1 1 -  3 -  2 -   3  2  1   2  3 -  -  2 -  3 1 -   2 -  3  -  2  3 1  2 -  2 1  2 -  2 1 -  2  2 1 -  2  2 x2x2 x6x6 + + + - + + + -

68. = x0x1x2x3x4x5x6x7x0x1x2x3x4x5x6x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7  x 0 and x 4 have identical coefficients (ignoring sign) as do: x 1 and x 5 x 2 and x 6 x 3 and x 71 1111111111111111 1  3 -  2  -  3  2 -  1 1 1 1 1 -  3 -  2 -   3  2  1   2  3 -  -  2 -  3 1 -   2 -  3  -  2  3 1  2 -  2 1  2 -  2 1 -  2  2 1 -  2  2 x3x3 x7x7 + + + - + + + -

69. Now rewrite the matrix as equations in terms of: x 0 ± x 4, x 2 ± x 6, x 1 ± x 5, x 3 ± x 7 Oh my! Half the columns are gone. What’s next? (x 0 + x 4 ) + (x 2 + x 6 ) + (x 1 + x 5 ) + (x 3 + x 7 ) = X 0 (x 0 - x 4 ) +  2 (x 2 - x 6 ) +  (x 1 - x 5 ) +  3 (x 3 - x 7 ) = X 1 (x 0 + x 4 ) - (x 2 + x 6 ) +  2 (x 1 + x 5 ) -  2 (x 3 + x 7 ) = X 2 (x 0 - x 4 ) -  2 (x 2 - x 6 ) +  3 (x 1 - x 5 ) +  (x 3 - x 7 ) = X 3 (x 0 + x 4 ) + (x 2 + x 6 ) - (x 1 + x 5 ) - (x 3 + x 7 ) = X 4 (x 0 - x 4 ) +  2 (x 2 - x 6 ) -  (x 1 - x 5 ) -  3 (x 3 - x 7 ) = X 5 (x 0 + x 4 ) - (x 2 + x 6 ) -  2 (x 1 + x 5 ) +  2 (x 3 + x 7 ) = X 6 (x 0 - x 4 ) -  2 (x 2 - x 6 ) -  3 (x 1 - x 5 ) -  (x 3 - x 7 ) = X 7

70. Think signal flow and construct the equations using the butterfly operator: Ex. for (x 0 + x 4 ) + (x 2 + x 6 ) + (x 1 + x 5 ) + (x 3 + x 7 ) = X 0 (x 0 - x 4 ) +  2 (x 2 - x 6 ) +  (x 1 - x 5 ) +  3 (x 3 - x 7 ) = X 1 (x 0 + x 4 ) - (x 2 + x 6 ) +  2 (x 1 + x 5 ) -  2 (x 3 + x 7 ) = X 2 (x 0 - x 4 ) -  2 (x 2 - x 6 ) +  3 (x 1 - x 5 ) +  (x 3 - x 7 ) = X 3 (x 0 + x 4 ) + (x 2 + x 6 ) - (x 1 + x 5 ) - (x 3 + x 7 ) = X 4 (x 0 - x 4 ) +  2 (x 2 - x 6 ) -  (x 1 - x 5 ) -  3 (x 3 - x 7 ) = X 5 (x 0 + x 4 ) - (x 2 + x 6 ) -  2 (x 1 + x 5 ) +  2 (x 3 + x 7 ) = X 6 (x 0 - x 4 ) -  2 (x 2 - x 6 ) -  3 (x 1 - x 5 ) -  (x 3 - x 7 ) = X 7 Ʃ Ʃ ωpωp + + + - xoxo x4x4 xo + ωp x4xo + ωp x4 x o - ω p x 4 x 0 ± x 4 ( ω p = 1)

71. Note - the butterfly has a shorthand notation of: ωk ωk xoxo x4x4 xo + ωp x4xo + ωp x4 xo - ωp x4xo - ωp x4 (x 0 + x 4 ) + (x 2 + x 6 ) + (x 1 + x 5 ) + (x 3 + x 7 ) = X 0 (x 0 - x 4 ) +  2 (x 2 - x 6 ) +  (x 1 - x 5 ) +  3 (x 3 - x 7 ) = X 1 (x 0 + x 4 ) - (x 2 + x 6 ) +  2 (x 1 + x 5 ) -  2 (x 3 + x 7 ) = X 2 (x 0 - x 4 ) -  2 (x 2 - x 6 ) +  3 (x 1 - x 5 ) +  (x 3 - x 7 ) = X 3 (x 0 + x 4 ) + (x 2 + x 6 ) - (x 1 + x 5 ) - (x 3 + x 7 ) = X 4 (x 0 - x 4 ) +  2 (x 2 - x 6 ) -  (x 1 - x 5 ) -  3 (x 3 - x 7 ) = X 5 (x 0 + x 4 ) - (x 2 + x 6 ) -  2 (x 1 + x 5 ) +  2 (x 3 + x 7 ) = X 6 (x 0 - x 4 ) -  2 (x 2 - x 6 ) -  3 (x 1 - x 5 ) -  (x 3 - x 7 ) = X 7

72. Damn you and your re “cursed” friends! Using Bit Reverse Order and a tree of butterflies, my Decimation in Time Algorithm can solve this in O (N log N) No friends this time. They’d just be overhead invading my stack space. If you’re lying, I’ll claim your soul! Decimation in Time & Bit Reverse Order ( (rearranging the order of the N samples) 0000 001 010 011 100 101 110 111 0 00 1 2 3 4 5 6 7 0 2 4 6 1 3 5 7 0 4 2 6 1 5 3 7 0000 100 010 110 001 101 011 111

73. STAGE 1 2 – Point Butterfly 2 combined 2-Point Butterflies 4 combined 2-Point Butterflies STAGE 3STAGE 2 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 FFT Block Diagram

74. X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8080 8282 8080 8080 8080 8282 Stepping Through the FFT  8 0 = 1  8 2 =  4 1

75. X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 8181 8282 8383 8080 8080 8080 8080 8282 8080 8080 8080 8282 Stepping Through the FFT x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7

76. X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8080 8282 8080 8080 8080 8282 Stepping Through the FFT

77. 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8080 8282 Stepping Through the FFT

78. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8282 N/2 Stepping Through the FFT

79. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8282 N/2 Stepping Through the FFT

80. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8282 N/2 Stepping Through the FFT

81. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8282 N/2 Stepping Through the FFT

82. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 8080 8282 N/2 + N/2 Stepping Through the FFT

83. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 N/2 + N/2 Stepping Through the FFT 8080 8282

84. 8080 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8282 8383 8080 8080 8080 8282 N/2 + N/2 Stepping Through the FFT 8080 8282

85. 8080 8080 8080 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8383 8080 8080 8080 8282 N/2 + N/2 8282 Stepping Through the FFT 8080 8282 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7

86. 8080 8080 X0X1X2X3X4X5X6X7X0X1X2X3X4X5X6X7 x0x4x2x6x1x5x3x7x0x4x2x6x1x5x3x7 8181 8383 8080 8080 8080 8080 8282 N/2 + N/2 + N/2  O (N log N) 8282 Stepping Through the FFT 8080 8282

87. Algorithm FFT (ReX, ImX) Input: ReX[ ], ImX[ ] = real, imaginary parts of the time samples Output: ReX[ ], ImX[ ] = cosine, sine coefficients of frequency domain N = SizeOf( ReX ) PutInBitReverseOrder (ReX, ImX) % time domain decomposition % frequency domain synthesis (done in place) for k = 1 to log 2 N % Loop for each stage Wre = 1; Wim = 0; θ = 2π/ 2 k % Initialize stage constants for j = 1 to 2 k-1 % Loop for each sub DFT for i = j-1 to N-1 step 2 k % Loop for each butterfly ip = i + 2 k-1 tmpRe = ReX[ip]·Wre - ImX[ip]·Wim tmpIm = ReX[ip]·Wim + ImX[ip]·Wre ReX[ip] = ReX[ i ] - tmpRe ImX[ip] = ImX[ i ] - tempIm ReX[ i ] = ReX[ i ] + tempRe ImX[ i ] = ImX[ i ] + tempIm next i tempRe = Wre Wre = tmpRe·cos(θ) + Wim·sin(θ) Wim = - tmpRe·sin(θ) + Wim· cos(θ) next j next k return (ReX, ImX) % ReX[ ],ImX[ ] return freq coeffs 0 to N-1

88. Polynomial Multiplication f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 g(x) = b 0 +b 1 x +b 2 x 2 + … + b n-1 x n-1 [f×g](x) = c 0 +c 1 x +c 2 x 2 + … +c 2n-2 x 2n-2 x 5 coefficient: c 5 = a 0 ×b 5 +a 1 ×b 4 + a 2 ×b 3 + … + a 5 ×b 0 Time = O(n 2 ) Too much Convolution

89. Polynomial Multiplication f(x) = a 0 +a 1 x +a 2 x 2 + … + a n-1 x n-1 g(x) = b 0 +b 1 x +b 2 x 2 + … + b n-1 x n-1 [f×g](x) = c 0 +c 1 x +c 2 x 2 + … +c 2n-2 x 2n-2 Coefficient Domain a j Evaluation Domain y i [a 0,a 1,a 2,…,a n-1 ] [b 0,b 1,b 2,…,b n-1 ] Fast Fourier Transform takes O(nlogn) time! y i = f(x i ) z i = g(x i ) y i ×z i = [g×f](x i ) Multipling values pointwise takes O(n) time! [c 0,c 1,c 2,…,c n-1 ]

90. Multiplying Big Integers X = 11…10100011101100010010 (N bits) Y = 10…01001100011001001111 X×Y = 10…1110110101001001010100010100110010011110 The high school algorithm takes O(N 2 ) bit operations. Can we do it faster? I hope so See Recursion for one way to do it faster. This is another.

91. Grade School Revisited: How To Multiply Two Numbers Multiplying Big Integers

92. X = 0011 … 1010 0011 1011 0001 0010 Break into m = O(log N) bit blocks m

93. Multiplying Big Integers X = 0000 … 0000 0000 0011 1010 0011 1011 0001 0010 Break into m = O(log N) bit blocks Pad with zero 2N bits to hold product n blocks where n is a power of 2, ie n=2 r. Let p be a prime log p ≥ block size = m p-1 is divisible by n, so Z mod p has n n th roots of unity. View each block as a finite field element in Z mod p. (no actual work) n=2 r blocks O(log p) a n … a 7 a 6 a 5 a 4 a 3 a 2 a 1 a 0 m

94. Multiplying Big Integers X = 0000 … 0000 0000 0011 1010 0011 1011 0001 0010 View as coefficients of a polynomial. Note X = f(2 m ). Same for Y = g(2 m ). Multiply g×f using FFT in time O(nlogn). Note X×Y = [g×f](2 m ). Evaluate [g×f](2 m ) in time O(n) operations, but each op could be on O(n) bit numbers for a total of O(n 2 ) time. f(x) = a n-1 x n-1 + … + a 5 x 5 + a 4 x 4 + a 3 x 3 + a 2 x 2 + a 1 x + a 0 m g(x) = b n-1 x n-1 + … + b 5 x 5 + b 4 x 4 + b 3 x 3 + b 2 x 2 + b 1 x + b 0

95. Multiplying Big Integers X×Y = 0011 1010 0011 1011 0001 0010 1110 0011 0010 Evaluate [g×f](2 m ) in time O(n). [g×f](x) = c n-1 x n-1 + … + c 5 x 5 + c 4 x 4 + c 3 x 3 + c 2 x 2 + c 1 x + c 0 m O(log p) Some texts say the c i can just be shifted and joined. Problem: The field elements may be too big.

96. Multiplying Big Integers Evaluate [g×f](2 m ) in time O(n). [g×f](x) = c n x n + … + c 5 x 5 + c 4 x 4 + c 3 x 3 + c 2 x 2 + c 1 x + c 0 m O(log p) 101011 001101 110001 000111 101011 100100 010011 Shift each c i by im. Add X×Y = 01 0101 0110 1011 1010 0001 1111 1011 Adding n numbers each n bits long takes O(n 2 ) but here the numbers are sparse.

97. Multiplying Big Integers Evaluate [g×f](2 m ) in time O(n). [g×f](x) = c n x n + … + c 5 x 5 + c 4 x 4 + c 3 x 3 + c 2 x 2 + c 1 x + c 0 m O(log p) 101011 001101 110001 000111 101011 100100 010011 Shift each c i by im. Add X×Y = 01 0101 0110 1011 1010 0001 1111 1011 At each point, at most two numbers overlap  Carry is at most one  O(N) bit operations.

98. Multiplying Big Integers X = 11…10100011101100010010 (N bits) Y = 10…01001100011001001111 X×Y = 10…1110110101001001010100010100110010011110 Suppose N is really really big. How many bit operations are needed? O(N logN) O(N logN loglogN) O(N logN loglogN logloglogN loglogloglogN …) FFT time Time stated in text Time as far as I can see

99. Multiplying Big Integers X = …101000111011000100101010001001010 … Input size = N bits Field element size = N’ = log(N) bits # a i = n = N / N’ # of field ops = O(nlogn) Time for × field op = ? N’ X’ = 1010 0111 0110 0010 0101 0100 0100 1010 Input size = N’ bits Field element size = N’’ = log(N’) bits # a i = n’ = N’ / N’’ # of field ops = O(n’logn’) Time for × field op = ? Total time: N’’ And so on … O(N’ logN’ loglogN’ logloglogN’ …)

100. Multiplying Big Integers X = …101000111011000100101010001001010 … Input size = N bits Field element size = N’ = log(N) bits # a i = n = N / N’ # of field ops = O(nlogn) Time for × field op = ? Total time: = O( n logn ) × O(N’ logN’ loglogN’ logloglogN’ …) = O( N / N’ log N / N’ ) × O(N’ loglogN logloglogN loglogloglogN …) = O( N logN loglogN logloglogN loglogloglogN …) N’ O(N’ logN’ loglogN’ logloglogN’ …)

101. The End