1. EE565 Advanced Image Processing Copyright Xin Li 2008 1 Statistical Modeling of Natural Images in the Wavelet Space Why do we need transform? A 30-min. Tutorial on Wavelet  From filter to filter bank  Complete vs. overcomplete expansion Parametric models of wavelet coefficients  Univariate i.i.d. models  Bivariate probability models Application into texture synthesis  Pyramid-based scheme (Heeger&Bergen’1995)  Projection-based scheme (Portilla&Simoncelli’2000)

2. EE565 Advanced Image Processing Copyright Xin Li 2008 2 Why Transform? x1x1 x2x2 y1y1 y2y2 x 1 and x 2 are highly correlated p(x 1 x 2 )  p(x 1 )p(x 2 ) y 1 and y 2 are less correlated p(y 1 y 2 )  p(y 1 )p(y 2 )

3. EE565 Advanced Image Processing Copyright Xin Li 2008 3 Transform=Change of Coordinates Intuitively speaking, transform plays the role of facilitating the source modeling  Due to the decorrelating property of transform, it is easier to model transform coefficients (Y) instead of pixel values (X) An appropriate choice of transform (transform matrix A) depends on the source statistics P(X)  The thumb rule is to achieve the maximum sparsity (measured by the number of nonzero coefficients)  Let us see some toy examples first

4. EE565 Advanced Image Processing Copyright Xin Li 2008 4 Toy Examples Signal class 1: f(x)=Asin(w 1 x)+Bcos(w 2 x) Signal class 2: f(x)=A  (x-x 1 )+B  (x-x 2 ) Signal class 3: wedge+impulse+step

5. EE565 Advanced Image Processing Copyright Xin Li 2008 5 Two-Channel Filter Bank x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) Analysis Synthesis analysissynthesis

6. EE565 Advanced Image Processing Copyright Xin Li 2008 6 Numerical Example x(n) s(n) d(n)

7. EE565 Advanced Image Processing Copyright Xin Li 2008 7 From One-level to Multi-level x(n) H0H0 H1H1 2 2 H0H0 H1H1 2 2 d 0 (n) s 0 (n) d 1 (n) s 1 (n) w d0d0 s2s2 d1d1 d2d2 0 

8. EE565 Advanced Image Processing Copyright Xin Li 2008 8 Numerical Example xs0s0 d0d0 d0d0 d1d1 s1s1 s2s2 d0d0 d1d1 d2d2

9. EE565 Advanced Image Processing Copyright Xin Li 2008 9 From 1D to 2D x(m,n) H0H0 H1H1 2 2 H0H0 H1H1 2 2 H0H0 H1H1 2 2 ll(m,n) lh(m,n) hl(m,n) hh(m,n) row transform column transform s(m,n) d(m,n)

10. EE565 Advanced Image Processing Copyright Xin Li 2008 10 Graphical Illustration After row transform: each row is decomposed into low-band (approximation) and high-band (detail) s(m,n) d(m,n) LL LH HL HH Note that the order of row/column transform does not matter

11. EE565 Advanced Image Processing Copyright Xin Li 2008 11 Orientation of Subbands LL LH HL HH

12. EE565 Advanced Image Processing Copyright Xin Li 2008 12 From Single-level WT to Multi- level WT single-level WT x(m,n) ll 1 (m,n) single-level WT ll 2 (m,n) lh 1 (m,n) hl 1 (m,n) hh 1 (m,n) lh 2 (m,n) hl 2 (m,n) hh 2 (m,n)

13. EE565 Advanced Image Processing Copyright Xin Li 2008 13 Example

14. EE565 Advanced Image Processing Copyright Xin Li 2008 14 Wavelet Summary Harmonic Analysis Filter Bank Theory Multi-Resolution Analysis wavelets Electrical Engineer Computer Scientist Mathematician

15. EE565 Advanced Image Processing Copyright Xin Li 2008 15 Overcomplete Expansion Encore Recall: Why do we need decimation? In non-coding applications (analysis, detection, recognition etc.), decimation does more harm than good – “aliasing is bad (signal processing) or “loss of translation invariance is bad” (pattern recognition) Without the worry about data expansion, FB design becomes more flexible

16. EE565 Advanced Image Processing Copyright Xin Li 2008 16 Pyramid Decomposition X(m,n) d 1 (m,n) s 1 (m,n) s 2 (m,n) d 2 (m,n)

17. EE565 Advanced Image Processing Copyright Xin Li 2008 17 Gaussian-Laplacian Pyramid d 1 (m,n)x(m,n) d 2 (m,n) s 1 (m,n) s 2 (m,n)

18. EE565 Advanced Image Processing Copyright Xin Li 2008 18 Data Expansion of Pyramid d 1 (m,n) x(m,n) d 2 (m,n) s 1 (m,n) s 2 (m,n) d k (m,n) s k (m,n) output Expansion ratio=1+1/4+1/16+…+1/4 K  4/3 MN MN/4 MN/16 MN/4 K

19. EE565 Advanced Image Processing Copyright Xin Li 2008 19 Steerable Pyramid linear multi-scale, multi-orientation image decomposition http://www.cns.nyu.edu/~eero/STEERPYR/

20. EE565 Advanced Image Processing Copyright Xin Li 2008 20 Basic Idea: directional derivative Why called “steerable”? It consists of a set of oriented bandpass subbands (similar to the Gabor basis functions you have seen in assignment 1)

21. EE565 Advanced Image Processing Copyright Xin Li 2008 21 Image Example overcomplete ratio = 4k/3, where k is the number of orientation bands

22. EE565 Advanced Image Processing Copyright Xin Li 2008 22 Summary of Wavelet Transforms Any linear transform is a change of coordinate – in the transformed space, modeling transform coefficients is often easier than modeling pixel values (when transform is designed properly) Translation invariance (TI) might get lost during the transform (e.g., due to down-sampling), which will affect our choice of transform in different tasks (e.g., coding vs. non-coding) Transform, just like prediction, serves as a basic tool for modeling the correlation of the source – whether use it or not depends on the problem at hand.