1. Digital Signal Processing II Lecture 9: Filter Banks - Special Topics Marc Moonen Dept. E.E./ESAT, K.U.Leuven marc.moonen@esat.kuleuven.be www.esat.kuleuven.be/scd/

2. DSP-II p. 2 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Part-II : Filter Banks : Preliminaries Filter bank set-up and applications `Perfect reconstruction’ problem + 1st example (DFT/IDFT) Multi-rate systems review (10 slides) : Maximally decimated FBs Perfect reconstruction filter banks (PR FBs) Paraunitary PR FBs : Modulated FBs Maximally decimated DFT-modulated FBs Oversampled DFT-modulated FBs : Special Topics Cosine-modulated FBs Non-uniform FBs & Wavelets Frequency domain filtering Lecture-6 Lecture-7 Lecture-8 Lecture-9

3. DSP-II p. 3 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Topic-1: Cosine-Modulated Filter Banks Motivation : Cosine-modulated FBs offer an alternative to DFT-modulated FBs… Similar to DFT-modulated FBs, cosine-modulated FBs offer economy in design- and implementation complexity Unlike DFT-modulated FBs, cosine-modulated FBs can be PR/FIR/paraunitary under maximal decimation (with design flexibility).

4. DSP-II p. 4 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Uniform DFT-modulated filter banks: Ho(z) is prototype lowpass filter, cutoff at for N filters Cosine-modulated filter banks : Po(z) is prototype lowpass filter, cutoff at for N filters Then... etc... H0H3H2H1 P0 H1 Ho

5. DSP-II p. 5 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Cosine-modulated filter banks : - if Po(z) is prototype FIR lowpass filter with real coefficients po[n], n=0,1,…,L then i.e. `cosine modulation’ (with real coefficients) instead of `exponential modulation’ (with complex coeffs, see DFT-modulated FBs Lecture-8) - if Po(z) is `good’ lowpass filter, then Hk(z)’s are `good’ bandpass filters

6. DSP-II p. 6 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Realization based on polyphase decomposition (analysis): - if Po(z) has 2N-fold polyphase expansion (ps: 2N-fold for N filters!!!) then... u[k] : :

7. DSP-II p. 7 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Realization based on polyphase decomposition (continued) : - if Po(z) has L+1=m.2N taps, and m is even (similar formulas for m odd) (i.e. `m’ is the number of taps in each polyphase component) then... With ignore all details here !!!!!!!!!!!!!!!

8. DSP-II p. 8 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Realization based on polyphase decomposition (continued): - Note that C (the only dense matrix here) is NxN DCT-matrix (`Type 4’) hence fast implementation (=fast matrix-vector product) based on fast discrete cosine transform (DCT) procedure, with complexity O(N.logN) Modulated filter bank gives economy in * design (only prototype Po(z) ) * implementation (prototype + modulation (DCT)) Similar structure for synthesis bank u[k] : :

9. DSP-II p. 9 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Maximally decimated cosine modulated (analysis) bank : u[k] : N N N : N N N =

10. DSP-II p. 10 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Question: How do we obtain Maximal Decimation + PR/FIR/Paraunitariness? Theorem : (proof omitted) -If prototype Po(z) is a real-coefficient (L+1)-taps FIR filter, (L+1)=2N.m for integer m and po[n]=po[L-n] (linear phase), with polyphase components Ek(z), k=0,1,…2N-1, -then the (FIR) cosine-modulated analysis bank is PARAUNITARY if and only if (for all k) are power complementary, i.e. form a lossless 1 input/2 output system And then FIR synthesis bank (for PR) can be obtained by paraconjugation !!! = great result…..this is the hard part…

11. DSP-II p. 11 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks Perfect Reconstruction (continued) Design procedure: Parameterize lossless systems for k=0,1..,N-1 Optimize all parameters in this parametrization so that the prototype Po(z) based on these polyphase components is a linear-phase lowpass filter that satisfies the given specifications Example parameterization: Parameterize lossless systems for k=0,1..,N-1, -> lattice structure (see Part-I), where parameters are rotation angles

12. DSP-II p. 12 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks PS: Linear phase property for po[n] implies that only half of the power complementary pairs have to be designed. The other pairs are then defined by symmetry properties. u[k] : N N p.9 = : : lossless 1-in/2-out

13. DSP-II p. 13 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Cosine-Modulated Filter Banks PS: Cosine versus DFT modulation In a maximally decimated cosine-modulated (analysis) filter bank 2 polyphase components of the prototype filter,, actually take the place of only 1 polyphase component in the DFT- modulated case. For paraunitariness (hence FIR-PR) in a cosine-modulated bank, each such pair of polyphase filters should form a power complementary pair, i.e. represent a lossless system. In the DFT-modulated case, imposing paraunitariness is equivalent to imposing losslessness for each polyphase component separately, i.e. each polyphase component should be an `allpass’ transfer function. Allpass functions are always IIR, except for trivial cases (pure delays). Hence all FIR paraunitary DFT-modulated banks (with maximal decimation) are trivial modifications of the DFT bank. no FIR-design flexibility provides flexibility for FIR-design

14. DSP-II p. 14 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Topic-2: Non-Uniform FBs / Wavelets Starting point is discrete-time Fourier transform: = infinitely long sequence u[k] is evaluated at infinitely many frequencies Inversion/reconstruction/synthesis (=filter bank jargon) is.. = sequence u[k] is represented as weighted sum of basis functions Prelude

15. DSP-II p. 15 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets `uncertainty principle’ says that if u[k] has a narrow support (i.e. is localized), then U(.) has a wide support (i.e. is non- localized), and vice versa Hence notion of `frequency that varies with time’ not accommodated (e.g. `short lived sine’ will correspond to non-localized spectrum) Prelude

16. DSP-II p. 16 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Tool to fill this need is `short-time Fourier transform’(STFT) where w[n] is your favorite window function (typically with `compact support’ (=FIR) ) Window slides past the data. For each window position n, compute discrete-time Fourier transform. PS: If w[n]=1 for all n, then result is discrete-time FT for all n In following slides, will provide a filter bank version of STFT, also leading to simple inversion formula Non-Uniform FBs / Wavelets Prelude

17. DSP-II p. 17 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Rewrite STFT formula as… If we forget about the fase factor up front (meaning what?), then this corresponds to performing a convolution with a filter In practice, will compute this for a discrete set of (N) frequencies leading to a set of filters This is a DFT-modulated analysis bank, prototype filter = window function Prelude

18. DSP-II p. 18 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Efficient implementation based on polyphase decomposition of prototype Ho + DFT-modulation Often window length=N, hence 1-tap polyphase components u[k] Prelude freq.resolution N u[k] window length/N

19. DSP-II p. 19 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets If maximally decimated (M=N, decimation=`window shift’), decimated DFT-modulated analysis bank corresponds to xk[n] = decimated subband signals = STFT-coefficients = infinitely long sequence u[k] is evaluated at N frequencies, infinitely many times (i.e. for infinitely many window positions)..to be compared to page 14 Prelude

20. DSP-II p. 20 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets With a corresponding (PR) synthesis filter bank (see Lecture 7) Ex: the reconstruction/synthesis formula (=inverse STFT) is..to be compared to page 14 PS: can also do oversampled versions Prelude H2(z) H3(z) 4 4 4 4 F2(z) F3(z) y[k] H0(z) H1(z) 4 4 u[k] 4 4 F0(z) F1(z) +

21. DSP-II p. 21 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Now, for some applications (e.g. audio) would like to have a non-uniform filter bank, hence also with non-uniform (maximum) decimation, for instance non-uniform filters = low frequency resolution at high frequencies, high frequency resolution at low frequencies (as human hearing) non-uniform decimation = high time resolution at high frequencies, low time resolution at low frequencies H2(z) H3(z) 4 2 H0(z) H1(z) 8 8 u[k] H0H3 H2H1

22. DSP-II p. 22 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets This can be built as a tree-structure, based on a 2-channel filter bank with H0H3 H2 H1 u[k] 2 2 2 2 2 2

23. DSP-II p. 23 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Note that may be viewed as a prototype filter, from which a series of filters is derived The lowpass filters are then needed to turn these multi-band filters into bandpass filters (i.e. remove images)

24. DSP-II p. 24 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Similar synthesis bank can be constructed with If and form a PR FB, then the complete analysis/synthesis structure is PR (why?) 2 2 + 2 2 + 2 2 +

25. DSP-II p. 25 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Analysis bank corresponds to `discrete-time wavelet transform’ (DTWT) With a corresponding (PR) synthesis filter bank, the reconstruction/synthesis formula (inverse DTWT) is..to be compared to page 14 & 20

26. DSP-II p. 26 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Reconstruction formula may be viewed as an expansion of u[n], using a set of basis functions (infinitely many) If the 2-channel filter bank is paraunitary, then this basis is orthonormal (which is a desirable property) : =`orthonormal wavelet basis’

27. DSP-II p. 27 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Example : `Haar’ wavelet (after Alfred Haar) Compare to 2-channel DFT/IDFT bank Derive formulas for Ho, H1, H2, H3, … Derive formulas for Fo, F1, F2, F3, … Paraunitary FB (orthonormal wavelet basis) ?

28. DSP-II p. 28 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Not treated here : `continuous wavelet transform’ (CWT) of a continuous-time function u(t) h(t)=prototype p,q are real-valued continuous variables p introduces `dilation’ of prototype, q introduces `shift’ of prototype `discrete wavelet transform’ (DWT) is CWT with discretized p,q T is sampling interval k,n are real-valued integer variables mostly a=2 ignore details…

29. DSP-II p. 29 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Non-Uniform FBs / Wavelets Not treated here : Theory - multiresolution analysis - wavelet packets - 2D transforms - etc … Applications : - audio: de-noising, … - communications : wavelet modulation - image : image coding

30. DSP-II p. 30 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Topic-3 : Frequency Domain Filtering See DSP-I : cheap FIR filtering based on frequency domain realization (`time domain convolution equivalent to component-wise multiplication in the frequency domain’), cfr. `overlap-add’ & `overlap-save’ procedures This can be cast in the subband processing setting, as a non-critically downsampled (2-fold oversampled) DFT- modulated filter bank operation! Leads to more general approach to performance/delay trade-off PS: formulae given for N=4, for conciseness (but without loss of generality)

31. DSP-II p. 31 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering Have to know a theorem from linear algebra here: A `circulant’ matrix is a matrix where each row is obtained from the previous row using a right-shift (by 1 position), the rightmost element which spills over is circulated back to become the leftmost element The eigenvalue decomposition of a `circulant’ matrix is trivial…. example (4x4): with F the NxN DFT-matrix, this means that the eigenvectors are equal to the column-vectors of the IDFT-matrix, and that then eigenvalues are obtained as the DFT of the first column of the circulant matrix (proof by Matlab)

32. DSP-II p. 32 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering Starting point is this (see Lecture-7) : meaning that a filtering with can be realized in a multirate structure, based on a pseudo- circulant matrix T(z)*u[k-3] u[k] 4 4 4 4 4 4 4 4 +

33. DSP-II p. 33 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering Now some matrix manipulation… :

34. DSP-II p. 34 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering An (8-channel) filter bank representation of this is... Analysis bank: Synthesis bank: Subband processing: …………………………… This is a 2N-channel filter bank, with N-fold downsampling. The analysis FB is a 2N-channel uniform DFT filter bank. The synthesis FB is matched to the analysis bank, for PR under 2-fold oversampling. u[k] 4 4 4 4 4 4 4 4 + y[k]

35. DSP-II p. 35 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering This is known as an `overlap-save’ realization : –Analysis bank: performs 2N-point DFT (FFT) of a block of (N=4) samples, together with the previous block of (N) samples (hence `overlap’) –Synthesis bank: performs 2N-point IDFT (IFFT), throws away the first half of the result, saves the second half (hence `save’) –Subband processing corresponds to `frequency domain’ operation `block’ `previous block’ `save’`throw away’

36. DSP-II p. 36 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering `Overlap-add’ can be similarly derived :

37. DSP-II p. 37 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering This is known as an `overlap-add’ realization : –Analysis bank: performs 2N-point DFT (FFT) of a block of (N=4) samples, padded with N zero samples –Synthesis bank: performs 2N-point IDFT (IFFT), adds second half of the result to first half of previous IDFT (hence `add’) –Subband processing corresponds to `frequency domain’ operation `block’ `zero padding’ `add’`overlap’

38. DSP-II p. 38 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering Standard `Overlap-add’ and `overlap-save’ realizations are derived when 0 th order poly-phase components are used in the above derivation, i.e. each poly-phase component represents 1 tap of an N-tap filter T(z). The corresponding 0 th order subband processing (H) then corresponds to what is usually referred to as the `component-wise multiplication’ in the frequency domain. Note that for an N-tap filter, with large N, this leads to a cheap realization based on FFT/IFFTs instead of DFT/IDFTs. However, for large N, as 2N-point FFT/IFFTs are needed, this may also lead to an unacceptably large processing delay (latency) between filter input and output.

39. DSP-II p. 39 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Frequency Domain Filtering In the more general case, with higher-order polyphase components (hence N smaller than the filter length), a smaller complexity reduction is achieved, but the processing delay is also smaller. This provides an interesting trade-off between complexity reduction and latency !!

40. DSP-II p. 40 Version 2009-2010 Lecture-9: Filter Banks – Special Topics Conclusions Great (=FIR/paraunitary) perfect reconstruction FB designs based on `modulation’: –Oversampled DFT-modulated FBs (Lecture-8) –Maximally decimated (and oversampled (not treated here) ) cosine-modulated FBs `Perfect reconstruction’ concept provides framework for time-frequency analysis of signals Filter bank concept provides framework for frequency domain realization of long FIR filters