# Wavelets and Filter Banks 彭思龙 中国科学院自动化研究所 国家专用集成电路设计工程技术研究中心 2008.2.29.

## Presentation on theme: "Wavelets and Filter Banks 彭思龙 中国科学院自动化研究所 国家专用集成电路设计工程技术研究中心 2008.2.29."— Presentation transcript:

Wavelets and Filter Banks 彭思龙 Silong.peng@ia.ac.cn 中国科学院自动化研究所 国家专用集成电路设计工程技术研究中心 2008.2.29

Background needed Mathematics: –Linear algebra –Polynomial –Mathematical analysis –Functional analysis Signal processing Image processing Matlab programming

Contents Signal processing basic Filter bank Mathematical basic MRA (multiresolution analysis) Wavelet lifting scheme Two dimensional wavelet The application of wavelet –Wavelet domain denoising –Fast object searching

Contents (cont.) –Wavelet domain image deconvolution –Wavelet domain image super-resolution –Wavelet domain image compression and post- processing –Wavelet domain image fusion and mosaicing –Filter approximation –Adaptive wavelet (pyramid)

Contents (cont.) Advances of wavelet now: –Nonlinear signal transform: Empirical Mode Decomposition (Hilbert-Huang transform) Local narrow band signal based decomposition –Geometry wavelet in 2D (optional) –Image decomposition (optional)

Contents (cont.) Some ideas in life and research –How to win before forty –…

Lecture 1 Introduction –Filter banks=a set of filters, filter is widely used in many fields of engineering and science for a long time. –Wavelet, an old and new tool to produce filter banks, have been thoroughly studied in past 20 years. Here we use wavelets to indicate many kinds of wavelets with different properties. –Application: image compression, pattern recognition, image processing, video processing…

Some basic concepts Signal: x(t) or {x(n)} Filter: a vector, h={h(n)}, for a given signal {x(n)}, the process of filtering: y=h*x, where * is the convolution operator: FIR=Finite Impulse Response=finite length IIR=Infinite Impulse Response=infinite length Example of filtering:x=sin(- 4:0.08:4)+0.1*randn(1,101);h=[1 1 1 1]/4;y=x*h

Continuous Fourier Transform Some basic properties: –Linearity –Parseval Identity:

Z transform

Lowpass Filter=moving average=passing low frequency –h={h(n)}, if sum of h is not zero, we call it a lowpass filter, in most time, the sum of h is 1. –H(z), H(1)=1 –Example: Simplest: H={1 1}/2; (average) Spline: {1 2 1}/4 General: {h(n)} Previous figure Highpass Filter=moving difference=passing high frequency

–h={h(n)}, is called highpass filter is the sum of h is zero. –H(z), H(1)=0 –Examples: Simplest: h={1 –1}/2, difference Dual spline: {-1 2 –1}/4; General:{h(n)} sum of h is 0. Figure,x as before, h={-1 2 –1}/4, y=x*h;

Frequency response of {1 1}/2

Frequency response of {1 2 1}/4

Frequency response of {-1 2 -1}/4

Phase – is called the phase of H –If, we say H has linear phase –H has linear phase is equivalent to say H is symmetric or antisymmetric, –Previous filters are symmetric, have linear phase

Invertibility or noninvertibility –Y(z)=H(z)X(z), If we want to reconstruct X, H can not be 0 at any point z. If H does not equal to 0 at any |z|=1, we say H is invertible, that is to say, we can reconstruct X by: X(z)=Y(z)/H(z), which is a inverse filtering, the filter is 1/H(z). But in most cases, H equals to 0 at some points, we can not reconstruct X exactly. –Example: H(z)=1+0.5z is invertible, but H(z)=(1+z)/2 is not invertible. How to reconstruct a signal? We can use filter banks

Filter banks=Lowpass+Highpass(inter- complement), –Simplest idea: H0 and H1, where H0 is a lowpass filter, and H1 is a highpass filter, the lost information in the process of lowpass filtering can be fund in the output of the highpass filter. –Some problems: How to reconstruct the signal? How to find such filter bank? How to reduce the computation and/or storage? Any more properties beside reconstruction?

Inner product –F_1 and F_2 are two functions in L_2, the inner product of these two functions is defined as: Orthogonality –If =0, we say they are orthogonal.

Biorthogonality –Two sets of function {F_j} and {G_j}, if =1 if j=k and 0 otherwise. We call the two set of functions are biorthogonal. Compact support –For a given function f, supp(f)={x|f(x) is not 0} –If measure(supp(f)) is finite, we say f is compactly supported or f has compact support. –Corresponding to FIR

Filters Signal –Sequence of numbers, {…x(-1), x(0), x(1), …} –Unit impulse: x(n)=1 if n=0 and 0 otherwise –Usually use Dirac delta symbol

is called sampling function: – –Continuous signal x(t)  discrete signal x(n) Sampling rules: –For band-limited and energy limited –Nyquist rate –Sharp: Nyquist, –Redundant: fast than Nyquist –Alias: slow than Nyquist

Shannon sampling theorem: –If signal f(t) satisfies: supp( ) is included in the interval [-T, T], and sampling rate is r, then If r>, we can not reconstruct signal f(t); If r<=, we have Where is called Nyquist sampling rate. Simple Proof:

Delay –Sx(n)=x(n-1) Advance –S -1 x(n)=x(n+1) SS -1 =S -1 S=I, where I represent the unit operator. Time-invariant filters: H is a linear filter, if H(Sx)=S(Hx): a shift of the input produces a shift of the output.

Ideal filters, –Ideal lowpass: –Ideal Highpass: –For ideal lowpass fitler: – –But in practice, we must use finite filter for convolution, the first idea is to use finite part of this filter, it leads to Gibbs phenomenon.(see ex)

function h=idealh(N) step=pi/200; o=-pi; h=zeros(400,1); for kk=1:400 h(kk)=1/2; sn=1; for ii=1:N nu=2*ii-1; h(kk)=h(kk)+2*sn*cos(nu*o)/pi/nu; sn=-sn; end o=o+step; end figure;plot(h);

We can verify that We will meet this equation later in constructing filter banks. Traditional filter design methods: Firstly, we note that we only need to construct lowpass filter, and shift in phase by pi we can get correspondent highpass filters.

Window method: h(n)=h I (n)w(n) –Hamming window: –Hanning window: –Kaiser window:

Equiripple method: –The filter with the smallest maximum error in passband and stopband is an equiripple filter. Means the ripples in passband and stopband is equal height. –Remez exchange algorithm Weighted least squares(eigenfilters) –This method is to minimize the function:

In discrete case, we can rewrite the above function into E=h T Ph, where h is the unknown coefficient vector. To minimize the function, h must satisfies Ph=u h, that is to say, h is an eigenvector of the matrix P, such that the optimal problem reduces to find the eigenvector of P. In detail for designing lowpass filter: –Given a unknown symmetric filter h(n)=h(2L-1- n) of length 2L –

in stop band, we need the desired response from w s to PI is zero., then the error function is: In the passband, from 0 to w p, we need the desired response all pass, such that we can use normalized constant h T c(0) to rewrite the error in pass band into :

Where the matrix P is known since we know c. We consider the weighted error: E=aEp+(1-a)Es, find the eigenvectors of P=Pp+Ps which will be the suitable filters we needed. Halfband and Mth band filter design(filter banks). Maximally Flat Filter: (Daubechies wavelets).

Poisson Summation Formula: –We use Dirac function as a sampling function: –By sampling rules, we need x(kr), k is any integer, consideris called Dirac comb which likes a comb in figure. –Poisson summation formula: –Simple Proof: using distribution and Parseval Identity.

Another equivalent form: Heisenberg’s Uncertainty Principle: Define two window width:time and frequency: Then: if ||f||=1, we have If f is the Gaussian function, the minimum value is reached. The inverse is true.

Basis and frames –Basis: unique representation; linear independence and completeness –Frame: linear independence and completeness but stable –Riesz basis: stable basis

The Wigner-Ville Transform—time- frequency analysis: Given a signal f(t), the WV transform is: Analyze signal in time-frequency plane. Some properties:

Shortcomings: –Not positive; –Interactive parts between separated frequency parts. Cohen class: General theory reference: –Time-frequency signal analysis Leon Cohen, – 中译本：时频信号分析， L. 科恩. Related theory: Matching Pursuit ( S. Mallat, Refer to “A Wavelet tour of signal processing”)

Downsampling and Upsamping –DS: {x(n)}  {y(n)}, y(n)=x(2n); –US: {x(n)}  {y(n)}, y(2n)=x(n), y(2n+1)=0; –Examples of DS and US. –Two functions: ds and us –Recoverable for half-band signal by using Shannon sampling theorem. –Otherwise, we can not recover the signal always. –We use D denotes the Downsamping operator, and U denote the Upsampling operator. –DU=I, that is, D after U does not change the signal.

Downsmapling in the frequency domain. Simple proof: Example: Aliasing: (see alias.m) –Extreme aliasing –No aliasing –Typical aliasing Upsamping in the frequency domain:

Imaging: (see imaging.m) …… Upsamping after downsampling Produce both aliasing and imaging In the Z-domain:

To remove the aliasing and imaging, we use filtering before downsampling to remove aliasing and after upsampling to remove imaging. ……

M-channel subsampling: V (n)=x(Mn), and u(Mn)=x(n) and 0 otherwise. In the z domain:

Fractional sampling rate: D L and U M can commute if and only if L and M are relatively prime. Fundamental rule: if L and M are relatively prime, then {Mk, k=0, …., L-1} is the same as {0, 1, …, L-1} besides a integer times of L. Simple proof:…

Filters exchanged with samplers. –G(z)D M =D M G(z M ) Proof: –U M G(z)=G(z M )U M Proof:

Filter Bank Lowpass+Highpass X(n) Ideal Lowpass Ideal Highpass X(n)

Improved Lowpass+Highpass X(n) Ideal LP Ideal HP X(n) 2 2 2 2 Ideal LP Ideal HP

Filter bank General lowpass and highpass X(n) Lowpass Highpass X(n) ?

Filter banks Perfect Reconstruction condition We need T=X to recover the original signal. X(n) H0H0 H1H1 y 0 (n) y 1 (n) 2 2 v0v0 v1v1 2 2 u 0 (n) u 1 (n) F1F1 F0F0 T(n)

Theorem 4.1. –A 2-channel filter bank gives perfect reconstruction when –F 0 (z)H 0 (z)+F 1 (z)H 1 (z)=2z -L –F 0 (z)H 0 (-z)+F 1 (z)H 1 (-z)=0 –F 0 (z)H 0 (-z)=-F 1 (z)H 1 (-z)

Alias Cancellation and the product filter –F 0 (z)=H 1 (-z), and F 1 (z)=-H 0 (-z) –Let P 0 (z)=F 0 (z)H 0 (z) and P 1 (z)=F 1 (z)H 1 (z), then –P 1 (z)=-P 0 (-z), and then –P 0 (z)-P 0 (-z)=2z -L –L must be odd, so let P(z)=z L P 0 (z) –Then we have P(z)+P(-z)=2 –Because all even terms in P(z) are zero, we can conclude that P(z) is a half band filter. –Some examples. –Haar Filter

Ex3: –P(z)=(-z 3 +9z+16+9 -1 -z -3 )/16 – P=H 0 F 0 –The roots of P is: c=2+3 1/2, 2-3 1/2, -1(4) –H 0 or F 0 can be: –The order N can be: N=0, 1 N=1, 1+z -1 N=2, (1+ z -1 ) 2 or (1+z -1 )(c-z -1 ) N=3, (1+ z -1 ) 3 or (1+z -1 ) 2 (c-z -1 )

(1+z -1 ) 3 and (-1+ 3 z -1 +3 z -2 - z -3 ) (1+z -1 ) 2 (c- z -1 ) and (1+z -1 ) 2 (1/c- z -1 ), Daubechies wavelet of length 4

Modulation matrix: Theorem 4.2. –If all filters are symmetric ( or anti-symmetric) around zero, h(k)=h(-k), then the condition of PR becomes a statement about inverse matrices –F m (z)H m (z)=2I

Early choice: –Croisier-Esaban-Galand(1976) –H 1 (z)=H 0 (-z) –H 0 2 (z) - H 0 2 (-z) =2z -L –So called QMF(Quadrature Mirror Filter) just because |H 1 (z)|=|H 0 (-z)|, they are symmetric about PI/2--- quadrature frequency. –No FIR filters (except Haar). –Simple proof: use polyphase expansion Better choice: –Smith and Barnwell (1984-6), Mintzer(1985) –H 1 (z)=-z N H 0 (-z -1 ), –Orthogonal filter banks. –Simple example: db4

General choice: F 0 (z)H 0 (z) is a half band filter. Biorthogonal Theorem 4.3: In a biorthogoanl linear-phase filter bank with two channels, the filter lengths are all odd or all even. The analysis filters can be: –A)both symmetric, of odd length –B)one symmetric, and one antisymmetric of even lenth. –Proof: Perfect reconstruction with M Channels. Modula matrix: –H m (z)=(H jk (z)) jk –Where H jk (z)=H j (zW k ) for j,k=0, …, M-1

Polyphase matrix –Meaning of polyphase –Purpose of polyphase, efficient for computing. xH 2 v

(HX) e =H e X e +z -1 H o X o Polyphase matrix: x 2 2 HoHo HeHe (HX) e z -1

X(n) H0H0 H1H1 y 0 (n) y 1 (n) 2 2 v0v0 v1v1

. Efficient Filter bank by using polyphase

2 2

X(n) z -1 2 2 Hp v 0 (n) v 1 (n) X(n) H0H0 H1H1 y 0 (n) y 1 (n) 2 2 v0v0 v1v1

Relations with Modula matrix

Polyphase for upsampling and reconstruction v(n) 2 u(n) F w(n) v(n) F even F odd 2 2 z -1 w(n)

Synthesis bank: direct and polyphase v 0 (n) 2 F0F0 v 1 (n) 2 F1F1

Polyphase matrix of synthesis filter bank:

v 0 (n) v 1 (n) 2 2 z -1 v 0 (n) 2 F0F0 v 1 (n) 2 F1F1

X(n) H0H0 H1H1 y 0 (n) y 1 (n) 2 2 v0v0 v1v1 2 2 u 0 (n) u 1 (n) F1F1 F0F0 X(n) z -1 2 2 Hp v 0 (n) v 1 (n) 2 2 z -1

Type 2 polyphase: v 0 (n) v 1 (n) Fp-type 2 2 2 z -1

X(n) H0H0 H1H1 y 0 (n) y 1 (n) 2 2 v0v0 v1v1 2 2 u 0 (n) u 1 (n) F1F1 F0F0

v 0 (n) v 1 (n) Fp-type 2 2 2 z -1 X(n) z -1 2 2 Hp v 0 (n) v 1 (n)

Polyphase matrix

Perfect reconstruction –Theorem 4.9: If Ex: QMF: F 0 (z)=H 1 (-z), and F 1 (z)=-H 0 (-z) banks give perfect reconstruction when Fp and Hp are inverse: –Fp(z)Hp(z)=I or z -L I Hp is of type 1, and Fp is type 2, transposed, for synthesis

Lattice structure –How to find a solution of above equation? To find suitable analysis and synthesis bank? –The simplest example: –To ensure that Fp is also a FIR, det(Hp) must be a monomial. –Particularly,

We can get Fp=inverse(Hp) In fact, The condition of linear phase to require: –a=d, and b=c

Another case: a=d, and b=-c: Orthogonal filter when |z|=1, we can have Fp is a unitary matrix, we call it paraunitary. In general, let

X(n) 2 2 z -1

If H 0 =0 at z=-1, then the polyphase matrix has: Which means in orthogonal cases, the angels of the lattices add to PI/4.

Synthesis filter banks: Theorem 4.7 –Every lowpass-highpass orthonormal filter bank has a polyphase matrix of a lattice form as above.

Theorem 4.8: –Every linear phase PR filter bank with equal (even) length filters has a lattice factorization: –We can collect all a’s together to be one constant to reduce the computational complex.

Theorem 4.9: –An FIR analysis bank has an FIR synthesis bank that gives PR if and only if the determinat of Hp is a nonzero monomial. The lattice complexity is approximately half of the polyphase complexity. X(n) 2 2 z -1

Orthogonal Filter banks Paraunitary matrices –Definition 5.1, The matrix H(z) is paraunitary if it is unitary for all |z|=1: – H T (1/z)H(z)=I, for all |z|=1; –The above formula is true for all z. Theorem 5.1, –Det(H(z))=+z -L or -z -L

Condition O in polyphase form: –A filter bank is orthogonal when its polyphase matrix is Paraunitary. Theorem 5.2: –For an orthogonal filter bank the lowpass filter H must satisfy Condition O:

Proof:

Theorem:

Theorem 5.3, A symmetric orthogonal FIR filter can only have two nonzero coefficients. Proof:

Let P(z)=|H(z)| 2, we have P(z)+P(-z)=2, that is to say, P(z) is a halfband filter: –p(2m)=1 if m=0, and 0 otherwise. –P(z) is real, symmetric, nonnegative, halfband

Spectral Factorization –Given P(z), how to find H? –Can any nonnegative P(z) have such factorization? –How to factorize? – The root of P(z) (polynomial), z, 1/z, con(z), 1/con(z), for z is complex z, z for z is real.

For orthogonal factorization, If all z’s are in unit circle, we call it minimum phase spectral factor. We can choose F and H freely to be biorthogonal filter bank. To ensure that H and F are real, relatively conjugate, linear phase needs symmetric about unit circle.

Which one is better?(length=10) Same?

Maxflat (Daubechies) filters –Condition O: P=|H| 2 is a normalized halfband filter: p(0)=1, p(2)=….=p(2p-2)=0 –Condition Ap: H has a zero of order p at π:

Lemma:

Formulas for P: Daubechies method:

–Unit factorization method:

Two formulas:

Meyer’s method:

Transition band for maxflat filters –Theorem 5.6, the maxflat filter has center slope proportional to.The transition rom 0.98 to 0.02 is over an interval of length 4/, where N is the order of the filter.

Download ppt "Wavelets and Filter Banks 彭思龙 中国科学院自动化研究所 国家专用集成电路设计工程技术研究中心 2008.2.29."

Similar presentations