Transform Techniques Mark Stamp Transform Techniques

Intro Signal can be viewed in… Many types of transformations Time domain  usual view, raw signal Frequency domain  transformed view Many types of transformations Fourier transform  most well-known Wavelet transform  some advantages Transform Techniques

Intro Fourier and wavelet transforms are reversible From time domain representation to frequency domain, and vice versa Fourier transform is in terms of functions sin(nx) and cos(nx) Wavelet can use a wide variety of different “basis” functions Transform Techniques

Fourier Series Generally, we can write f(x) in terms of series of sin(nx) and cos(nx) Exact, but generally need infinite series Finite sum usually just an approximation Coefficients on sin(nx) and cos(nx) tell us “how much” of that frequency May not be obvious from function Can be very useful information Transform Techniques

Fourier Series For example, consider sawtooth function: s(x) = x / π The graph is… Transform Techniques

Sawtooth Function 1 term of Fourier series 2 terms 3 terms 4 terms Transform Techniques

Fourier Transform A function f(x) is usually viewed in the “time domain” Transform allows us to also view it in “frequency domain” What does this mean? See next slide… Why might this be useful? Again, reveals non-obvious structure Transform Techniques

Time vs Frequency Function f(x) written as sums of functions ansin(nx) and bncos(nx) Coefficients (amplitudes) an and bn Tell us “how much” of each frequency Transform Techniques

Time vs Frequency Time domain Frequency domain Frequency domain view gives us info about the function More complicated the signal, less obvious the frequency perspective may be Transform Techniques

Time vs Frequency Time domain in red What does blue tell us? Frequency domain in blue What does blue tell us? Dominant low frequency Some high frequencies Note that blue tells us nothing about time… I.e., we do not know where frequencies occur Transform Techniques

Speech Example Frequency domain info used to extract important characteristics  Time domain signal  Sonogram Transform Techniques

Fourier Transform Many different transforms exist So, why is Fourier so popular? Fast, efficient algorithms Fast Fourier Transform (FFT) Apply transform to entire function? May not be too informative, since we lose track of where frequencies occur Usually, want to understand local behavior Transform Techniques

Global vs Local Function can change a lot over time… Global frequency info not so useful Local frequency info is much better Transform Techniques

Global vs Local Use Short Time Fourier Transform (STFT) for each window Note that windows can overlap Transform Techniques

Window Size How big should the window be? Small? May not have enough freq info Big? May not have useful time info about right too small too big Transform Techniques

Window Size Looks like ideal case would be windows that match frequency Bigger windows for low frequency areas Smaller windows for high frequency The bottom line? Too big of window gives good frequency resolution, but poor time resolution Too small of window gives good time resolution, but poor frequency resolution Transform Techniques

Uncertainty Principle Cannot have accurate frequency and time resolution simultaneously Form of Heisenberg Uncertainty Principle So, this is something we must deal with Since it’s the law! (of physics…) Is there any alternative to STFT? Yes, “multiresolution analysis” What the … ? Transform Techniques

Windowing Revisited Window in STFT is really a function Selects f(x) within current window “Window function” is essentially 1 within current window, 0 outside of it For wavelets, “windows” much fancier Like Windows 95 vs Windows 7… Effect is to filter based on frequencies Can mitigate some of the problems inherent in the uncertainty principle Transform Techniques

Fourier Transform In Fourier transform, frequency resolution, but no time resolution frequency time Transform Techniques

Short Time Fourier Trans. In STFT, time resolution via windowing frequency time Transform Techniques

Wavelet Transform Time resolution based on frequency frequency time Transform Techniques

Wavelet Recall that Fourier analysis is based on sin(nx) and cos(nx) functions Wavelet analysis based on wavelets Duh! But, what is a wavelet? A small wave, of course… “Wave”, so it oscillates (integrates to 0) “Small”, meaning acts like finite window Transform Techniques

Wavelets Many different wavelet functions to choose from Select a “mother” wavelet or basis Form translations and dilations of basis Examples include Haar wavelets (piecewise constant) Daubechies wavelets …and many others Transform Techniques

Haar: Translation & Dilation 1 1 1 1 1 1 -1 -1 -1 Transform Techniques

Advantages of Wavelets Wavelet basis is local Unlike Fourier basis of sine and cosine Local, implies better time resolution Basis functions all mutually orthogonal Makes computations fast Fourier basis also orthogonal, but requires “extreme cancellation” outside window In effect, “windowing” built in to wavelet basis Wavelets faster to compute than FFT A recursive paradise… Transform Techniques

Disadvantage of Wavelets Approximation with Haar functions… For example, sine function is trivial in Fourier analysis, not so easy with Haar Transform Techniques

Wavelets: Bottom Line Fourier ideal wrt frequency resolution But sine/cosine bad wrt time resolution Wavelets excels at time resolution Since basis functions finite (compact) support, and employ translation/dilation In effect, filters by frequency and time Complicated mathematics But fairly easy to implement and use Transform Techniques

Discrete Transforms In practice, apply transforms to discrete time series, a0,a1,a2,… We assume ai = f(xi) for unknown f(x) Discrete transforms are very fast FFT is O(n log n) Fast wavelet transform is O(n) Discrete transforms based on some fancy linear algebra Transform Techniques

Transform Uses Speech processing Image/video processing Compression Construct sonogram (spectrogram) Speech recognition Image/video processing Remove noise, sharpen images, etc., etc. Compression And many, many more… Transform Techniques

What About Malware Detection? We measure some characteristic of a exe file to obtain series a0,a1,a2,… Compute wavelet transform and… Filter out high frequency “noise” (i.e., insignificant variations) And segment file based on where the significant changes occur Ironically, transform used to pinpoint significant changes wrt time Transform Techniques

Malware Detection Example Compute entropy measurement using ai = entropy(Bi) for i = 0,1,2,…,n Where Bi is block of i consecutive bytes Computed on (overlapping) “windows” “Window” here not same as in transform Apply discrete transform to a0,a1,… Find significant changes in entropy Use resulting sequence for scoring Transform Techniques

References R. Polikar, The wavelet tutorial A. J. Jerri, Introduction to Wavelets G. Strang, Wavelet transforms versus Fourier transforms, Bulletin of the American Mathematical Society, 28:288-305, 1993 Transform Techniques