1. Orthogonal Transforms
Fourier

2. Review Introduce the concepts of base functions:
For Reed-Muller, FPRM
For Walsh
Linearly independent matrix
Non-Singular matrix
Examples
Butterflies, Kronecker Products, Matrices
Using matrices to calculate the vector of spectral coefficients from the data vector
Our goal is to discuss the best approximation of a function using orthogonal functions

3. Orthogonal Functions

4. Orthogonal Functions

5. Note that these are arbitrary functions, we do not assume sinusoids

6. Illustrate it for Walsh and RM

10. Mean Square Error

11. Mean Square Error

14. Important result

15. We want to minimize this kinds of errors.
Other error measures are also used.

16. Unitary Transforms

17. Unitary Transforms Unitary Transformation for 1-Dim. Sequence
Series representation of
Basis vectors :
Energy conservation :
Here is the proof

18. Unitary Transformation for 2-Dim. Sequence
Definition :
Basis images :
Orthonormality and completeness properties
Orthonormality :
Completeness :

19. Unitary Transformation for 2-Dim. Sequence
Separable Unitary Transforms
separable transform reduces the number of multiplications and additions from to
Energy conservation

20. Properties of Unitary Transform
Covariance matrix

21. Example of arbitrary basis functions being rectangular waves

22. This determining first function determines next functions

23. 1

25. Small error with just 3 coefficients

26. This slide shows four base functions multiplied by their respective coefficients

27. This slide shows that using only four base functions the approximation is quite good
End of example

28. Orthogonality and separability

29. Orthogonal and separable Image Transforms

30. Extending general transforms to 2-dimensions

31. Forward transform
inverse transform
separable

32. Fourier Transforms in new notations
We emphasize generality
Matrices

33. Fourier Transform
separable

34. Extension of Fourier Transform to two dimensions

38. Discrete Fourier Transform (DFT)
New notation

40. Fast Algorithms for Fourier Transform
Task for students:
Draw the butterfly for these matrices, similarly as we have done it for Walsh and Reed-Muller Transforms 2
Pay attention to regularity of kernels and order of columns corresponding to factorized matrices

41. Fast Factorization Algorithms are general and there is many of them

42. 1-dim. DFT (cont.)
Calculation of DFT : Fast Fourier Transform Algorithm (FFT)
Decimation-in-time algorithm
Derivation of decimation in time

43. Decimation in Time versus Decismation in Frequency

44. 1-dim. DFT (cont.) FFT (cont.) Decimation-in-time algorithm (cont.)
Butterfly for Derivation of decimation in time
Please note recursion

45. 1-dim. DFT (cont.) FFT (cont.)
Decimation-in-frequency algorithm (cont.)
Derivation of Decimation-in-frequency algorithm

46. Decimation in frequency butterfly shows recursion
1-dim. DFT (cont.)
FFT (cont.)
Decimation-in-frequency algorithm (cont.)

47. Conjugate Symmetry of DFT
For a real sequence, the DFT is conjugate symmetry

48. Use of Fourier Transforms for fast convolution

49. Calculations for circular matrix

50. By multiplying

51. W * = Cw*
In matrix form next slide

52. w * = Cw*

53. Here is the formula for linear convolution, we already discussed for 1D and 2D data, images

54. Linear convolution can be presented in matrix form as follows:

55. As we see, circular convolution can be also represented in matrix form

56. Important result

57. Inverse DFT of convolution

58. Thus we derived a fast algorithm for linear convolution which we illustrated earlier and discussed its importance.
This result is very fundamental since it allows to use DFT with inverse DFT to do all kinds of image processing based on convolution, such as edge detection, thinning, filtering, etc.

59. 2-D DFT

60. 2-D DFT

61. Circular convolution works for 2D images

62. 2-Dim. DFT (cont.) example Circular convolution works for 2D images
So we can do all kinds of edge-detection, filtering etc very efficiently
2-Dim. DFT (cont.)
example
(a) Original Image
(b) Magnitude
(c) Phase

63. 2-Dim. DFT (cont.)
Properties of 2D DFT
Separability

64. (d) resulting spectrum
2-Dim. DFT (cont.)
Properties of 2D DFT (cont.)
Rotation
(a) a sample image
(b) its spectrum
(c) rotated image
(d) resulting spectrum

65. 2-Dim. DFT (cont.) Properties of 2D DFT Circular convolution and DFT
Correlation

66. 2-Dim. DFT (cont.) Calculation of 2-dim. DFT
Direct calculation
Complex multiplications & additions :
Using separability
Using 1-dim FFT
Complex multiplications & additions : ???
Three ways of calculating 2-D DFT

67. Questions to Students
You do not have to remember derivations but you have to understand the main concepts.
Much software for all discussed transforms and their uses is available on internet and also in Matlab, OpenCV, and similar packages.
How to create an algorithm for edge detection based on FFT?
How to create a thinning algorithm based on DCT?
How to use DST for convolution – show example.
Low pass filter based on Hadamard.
Texture recognition based on Walsh