1. EE565 Advanced Image Processing Copyright Xin Li 2008 1 Statistical Modeling of Natural Images in the Wavelet Space Parametric models of wavelet coefficients  Univariate i.i.d. models  Spatially adaptive models Application into texture synthesis  Pyramid-based scheme (Heeger&Bergen’1995)  Projection-based scheme (Portilla&Simoncelli’2000)

2. EE565 Advanced Image Processing Copyright Xin Li 2008 2 Recall: Transform Facilitates Modeling x1x1 x2x2 y1y1 y2y2 x 1 and x 2 are highly correlated p(x 1 x 2 )  p(x 1 )p(x 2 ) y 1 and y 2 are less correlated p(y 1 y 2 )  p(y 1 )p(y 2 )

3. EE565 Advanced Image Processing Copyright Xin Li 2008 3 Empirical Observation H1H1 A single peak at zero

4. EE565 Advanced Image Processing Copyright Xin Li 2008 4 Univariate Probability Model Laplacian: Gaussian:

5. EE565 Advanced Image Processing Copyright Xin Li 2008 5 Gaussian Distribution

6. EE565 Advanced Image Processing Copyright Xin Li 2008 6 Laplacian Distribution

7. EE565 Advanced Image Processing Copyright Xin Li 2008 7 Statistical Testing How do we know which parametric model better fits the empirical distribution of wavelet coefficients? In addition to visual inspection (which is often subjective and less accurate), we can use various statistical testing tools to objectively evaluate the closeness of an empirical cumulative distribution function (ECDF) to the hypothesized one One of the most widely used techniques is Kolmogorov-Smirnov Test (MATLAB function: >help kstest).

8. EE565 Advanced Image Processing Copyright Xin Li 2008 8 Kolmogorov-Smirnov Test* The K-S test is based on the maximum distance between empirical CDF (ECDF) and hypothesized CDF (e.g., the normal distribution N(0,1)).

9. EE565 Advanced Image Processing Copyright Xin Li 2008 9 Example Usage: [H,P,KS,CV] = KSTEST(X,CDF) If CDF is omitted, it assumes pdf of N(0,1) x: computer-generated samples (0<P<1, the larger P, the more likely) Accept the hypothesis Reject the hypothesis d: high-band wavelet coefficients of lena image (note the normalization by signal variance)

10. EE565 Advanced Image Processing Copyright Xin Li 2008 10 Generalized Gaussian/Laplacian Distribution where Laplacian Gaussian P: shape parameter : variance parameter

11. EE565 Advanced Image Processing Copyright Xin Li 2008 11 Model Parameter Estimation* Maximum Likelihood Estimation Method of moments Linear regression method [1] Sharifi, K. and Leon-Garcia, A. “ Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video, ” IEEE T-CSVT, No. 1, February 1995, pp. 52-56. [2] www.cimat.mx/reportes/enlinea/I-01-18_eng.pdf Ref.

12. EE565 Advanced Image Processing Copyright Xin Li 2008 12 I.I.D. Assumption Challenged If wavelet coefficients of each subband are indeed i.i.d., then random permutation of pixels should produce another image of the same class (natural images) The fundamental question here: does WT completely decorrelate image signals?

13. EE565 Advanced Image Processing Copyright Xin Li 2008 13 Image Example High-band coefficients permutation You can run the MATLAB demo to check this experiment

14. EE565 Advanced Image Processing Copyright Xin Li 2008 14 Another Experiment Joint pdf of two uncorrelated random variables X and Y X Y

15. EE565 Advanced Image Processing Copyright Xin Li 2008 15 Joint PDF of Wavelet Coefficients Neighborhood I(Q): {Left,Up,cousin and aunt} X= Y= Joint pdf of two correlated random variables X and Y

16. EE565 Advanced Image Processing Copyright Xin Li 2008 16 Heeger&Bergen’1995: Histogram-based Pyramid-based (using steerable pyramids)  Facilitate the statistical modeling Histogram matching  Enforce the first-order statistical constraint Texture matching  Alternate histogram matching in spatial and wavelet domain Boundary handling: use periodic extension Color consistency: use color transformation Basic idea: two visually similar textures will also have similar statistics

17. EE565 Advanced Image Processing Copyright Xin Li 2008 17 Histogram Matching Generalization of histogram equalization (the target is the histogram of a given image instead of uniform distribution)

18. EE565 Advanced Image Processing Copyright Xin Li 2008 18 Histogram Equalization Uniform Quantization Note: L 1 x L y 0 cumulative probability function

19. EE565 Advanced Image Processing Copyright Xin Li 2008 19 MATLAB Implementation function y=hist_eq(x) [M,N]=size(x); for i=1:256 h(i)=sum(sum(x= =i-1)); End y=x;s=sum(h); for i=1:256 I=find(x= =i-1); y(I)=sum(h(1:i))/s*255; end Calculate the histogram of the input image Perform histogram equalization

20. EE565 Advanced Image Processing Copyright Xin Li 2008 20 Histogram Equalization Example

21. EE565 Advanced Image Processing Copyright Xin Li 2008 21 Histogram Specification ST S -1 * T histogram 1 histogram 2 ?

22. EE565 Advanced Image Processing Copyright Xin Li 2008 22 Texture Matching Objective: the histogram of both subbands and synthesized image matches the given template Basic hypothesis: if two texture images visually look similar, then they have similar histograms in both spatial and wavelet domain

23. EE565 Advanced Image Processing Copyright Xin Li 2008 23 Image Examples

24. EE565 Advanced Image Processing Copyright Xin Li 2008 24 Portilla&Simoncelli’2000: Parametric Instead of matching histogram (nonparametric models), we can build parametric models for wavelet coefficients and enforce the synthesized image to inherit the parameters of given image Model parameters: 710 parameters were used in Portilla and Simoncelli ’ s experiment (4 orientations, 4 scales, 7  7 neighborhood) Basic idea: two visually similar textures will also have similar statistics

25. EE565 Advanced Image Processing Copyright Xin Li 2008 25 Statistical Constraints Four types of constraints  Marginal Statistics  Raw coefficient correlation  Coefficient magnitude statistics  Cross-scale phase statistics Alternating Projections onto the four constraint sets  Projection-onto-convex-set (POCS)

26. EE565 Advanced Image Processing Copyright Xin Li 2008 26 Convex Set A set Ω is said to be convex if for any two point We have Convex set examples Non-convex set examples

27. EE565 Advanced Image Processing Copyright Xin Li 2008 27 Projection Operator f g Projection onto convex set C C In simple words, the projection of f onto a convex set C is the element in C that is closest to f in terms of Euclidean distance

28. EE565 Advanced Image Processing Copyright Xin Li 2008 28 Alternating Projection X0X0 X1X1 X2X2 X∞X∞ Projection-Onto-Convex-Set (POCS) Theorem: If C 1, …,C k are convex sets, then alternating projection P 1, …,P k will converge to the intersection of C 1, …,C k if it is not empty Alternating projection does not always converge in the case of non-convex set. Can you think of any counter-example? C1C1 C2C2

29. EE565 Advanced Image Processing Copyright Xin Li 2008 29 Convex Constraint Sets ● Non-negative set ● Bounded-value set ● Bounded-variance set A given signal or

30. EE565 Advanced Image Processing Copyright Xin Li 2008 30 Non-convex Constraint Set Histogram matching used in Heeger&Bergen’1995 Bounded Skewness and Kurtosis skewnesskurtosis The derivation of projection operators onto constraint sets are tedious are referred to the paper and MATLAB codes by Portilla&Simoncelli.

31. EE565 Advanced Image Processing Copyright Xin Li 2008 31 Image Examples original synthesized

32. EE565 Advanced Image Processing Copyright Xin Li 2008 32 Image Examples (Con’d) original synthesized

33. EE565 Advanced Image Processing Copyright Xin Li 2008 33 When Does It Fail? original synthesized

34. EE565 Advanced Image Processing Copyright Xin Li 2008 34 Summary Textures represent an important class of structures in natural images – unlike edges characterizing object boundaries, textures often associate with the homogeneous property of object surfaces Wavelet-domain parametric models provide a parsimonious representation of high-order statistical dependency within textural images