1. Transform Techniques
Mark Stamp
Transform Techniques

2. Intro Signal can be viewed in… Many types of transformations
Time domain  usual view, raw signal
Frequency domain  transformed view
Many types of transformations
Fourier transform  most well-known
Wavelet transform  some advantages
Transform Techniques

3. Intro Fourier and wavelet transforms are reversible
From time domain representation to frequency domain, and vice versa
Fourier transform is in terms of functions sin(nx) and cos(nx)
Wavelet can use a wide variety of different “basis” functions
Transform Techniques

4. Fourier Series
Generally, we can write f(x) in terms of series of sin(nx) and cos(nx)
Exact, but generally need infinite series
Finite sum usually just an approximation
Coefficients on sin(nx) and cos(nx) tell us “how much” of that frequency
May not be obvious from function
Can be very useful information
Transform Techniques

5. Fourier Series For example, consider sawtooth function: s(x) = x / π
The graph is…
Transform Techniques

6. Sawtooth Function 1 term of Fourier series 2 terms 3 terms 4 terms
Transform Techniques

7. Fourier Transform
A function f(x) is usually viewed in the “time domain”
Transform allows us to also view it in “frequency domain”
What does this mean?
See next slide…
Why might this be useful?
Again, reveals non-obvious structure
Transform Techniques

8. Time vs Frequency
Function f(x) written as sums of functions ansin(nx) and bncos(nx)
Coefficients (amplitudes) an and bn
Tell us “how much” of each frequency
Transform Techniques

9. Time vs Frequency
Time domain
Frequency domain
Frequency domain view gives us info about the function
More complicated the signal, less obvious the frequency perspective may be
Transform Techniques

10. Time vs Frequency Time domain in red What does blue tell us?
Frequency domain in blue
What does blue tell us?
Dominant low frequency
Some high frequencies
Note that blue tells us nothing about time…
I.e., we do not know where frequencies occur
Transform Techniques

11. Speech Example
Frequency domain info used to extract important characteristics
 Time domain signal
 Sonogram
Transform Techniques

12. Fourier Transform Many different transforms exist
So, why is Fourier so popular?
Fast, efficient algorithms
Fast Fourier Transform (FFT)
Apply transform to entire function?
May not be too informative, since we lose track of where frequencies occur
Usually, want to understand local behavior
Transform Techniques

13. Global vs Local Function can change a lot over time…
Global frequency info not so useful
Local frequency info is much better
Transform Techniques

14. Global vs Local
Use Short Time Fourier Transform (STFT) for each window
Note that windows can overlap
Transform Techniques

15. Window Size How big should the window be?
Small? May not have enough freq info
Big? May not have useful time info
about right
too small
too big
Transform Techniques

16. Window Size
Looks like ideal case would be windows that match frequency
Bigger windows for low frequency areas
Smaller windows for high frequency
The bottom line?
Too big of window gives good frequency resolution, but poor time resolution
Too small of window gives good time resolution, but poor frequency resolution
Transform Techniques

17. Uncertainty Principle
Cannot have accurate frequency and time resolution simultaneously
Form of Heisenberg Uncertainty Principle
So, this is something we must deal with
Since it’s the law! (of physics…)
Is there any alternative to STFT?
Yes, “multiresolution analysis”
What the … ?
Transform Techniques

18. Windowing Revisited Window in STFT is really a function
Selects f(x) within current window
“Window function” is essentially 1 within current window, 0 outside of it
For wavelets, “windows” much fancier
Like Windows 95 vs Windows 7…
Effect is to filter based on frequencies
Can mitigate some of the problems inherent in the uncertainty principle
Transform Techniques

19. Fourier Transform
In Fourier transform, frequency resolution, but no time resolution
frequency
time
Transform Techniques

20. Short Time Fourier Trans.
In STFT, time resolution via windowing
frequency
time
Transform Techniques

21. Wavelet Transform Time resolution based on frequency frequency time
Transform Techniques

22. Wavelet
Recall that Fourier analysis is based on sin(nx) and cos(nx) functions
Wavelet analysis based on wavelets
Duh!
But, what is a wavelet?
A small wave, of course…
“Wave”, so it oscillates (integrates to 0)
“Small”, meaning acts like finite window
Transform Techniques

23. Wavelets Many different wavelet functions to choose from
Select a “mother” wavelet or basis
Form translations and dilations of basis
Examples include
Haar wavelets (piecewise constant)
Daubechies wavelets
…and many others
Transform Techniques

24. Haar: Translation & Dilation1 1 1 1 1 1 -1 -1 -1
Transform Techniques

25. Advantages of Wavelets
Wavelet basis is local
Unlike Fourier basis of sine and cosine
Local, implies better time resolution
Basis functions all mutually orthogonal
Makes computations fast
Fourier basis also orthogonal, but requires “extreme cancellation” outside window
In effect, “windowing” built in to wavelet basis
Wavelets faster to compute than FFT
A recursive paradise…
Transform Techniques

26. Disadvantage of Wavelets
Approximation with Haar functions…
For example, sine function is trivial in Fourier analysis, not so easy with Haar
Transform Techniques

27. Wavelets: Bottom Line Fourier ideal wrt frequency resolution
But sine/cosine bad wrt time resolution
Wavelets excels at time resolution
Since basis functions finite (compact) support, and employ translation/dilation
In effect, filters by frequency and time
Complicated mathematics
But fairly easy to implement and use
Transform Techniques

28. Discrete Transforms
In practice, apply transforms to discrete time series, a0,a1,a2,…
We assume ai = f(xi) for unknown f(x)
Discrete transforms are very fast
FFT is O(n log n)
Fast wavelet transform is O(n)
Discrete transforms based on some fancy linear algebra
Transform Techniques

29. Transform Uses Speech processing Image/video processing Compression
Construct sonogram (spectrogram)
Speech recognition
Image/video processing
Remove noise, sharpen images, etc., etc.
Compression
And many, many more…
Transform Techniques

30. What About Malware Detection?
We measure some characteristic of a exe file to obtain series a0,a1,a2,…
Compute wavelet transform and…
Filter out high frequency “noise” (i.e., insignificant variations)
And segment file based on where the significant changes occur
Ironically, transform used to pinpoint significant changes wrt time
Transform Techniques

31. Malware Detection Example
Compute entropy measurement using ai = entropy(Bi) for i = 0,1,2,…,n
Where Bi is block of i consecutive bytes
Computed on (overlapping) “windows”
“Window” here not same as in transform
Apply discrete transform to a0,a1,…
Find significant changes in entropy
Use resulting sequence for scoring
Transform Techniques

32. References R. Polikar, The wavelet tutorial
A. J. Jerri, Introduction to Wavelets
G. Strang, Wavelet transforms versus Fourier transforms, Bulletin of the American Mathematical Society, 28: , 1993
Transform Techniques