1. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 1 Image Denoising: a Statistical Approach Linear estimation theory summary Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

2. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 2 Denoising Problem Noisy measurements N(0,σ w 2 ) MMSE estimator Wiener’s idea To simplify estimation by computing the best estimator that is a linear scaling of Y Difficulty: we need to know conditional pdf N(0,σ x 2 )

3. Orthogonality Principle EE565 Advanced Image Processing Copyright Xin Li 2009-2012 3 A linear estimator X of a random variable X ^ Minimizes E{(X-X) 2 } if and only if ^ Geometric Interpretation X Y X-X ^ X ^

4. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 4 Linear MMSE Estimation For Gaussian signal The optimal LMMSE estimation is given by And it achieves Note: it can be shown such linear estimator is indeed E[X|Y] for Gaussian signal

5. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 5 What if Signal Variance is Unknown? Maximum-likelihood estimation ofis given by Since variance is nonnegative, we modify it When multiple observations y i ’s are available, we have

6. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 6 Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

7. Conventional Wiener Filtering Basic assumption: image source is modeled by a stationary Gaussian process Signal variance is estimated from the noisy observation data Can be interpreted as a linear frequency weighting EE565 Advanced Image Processing Copyright Xin Li 2009-2012 7

8. 8 Linear Frequency Weighting FT Power spectrum |X| 2

9. Image Example EE565 Advanced Image Processing Copyright Xin Li 2009-2012 9 Noisy,  =50 (MSE=2500) denoised (MSE=1130)

10. Image Example (Con’d) EE565 Advanced Image Processing Copyright Xin Li 2009-2012 10 Noisy,  =10 (MSE=100) denoised (MSE=437)

11. Conclusions from the Experiments Why did it Fail? Nonstationary NonGaussian Poor modeling How to improve? Achieve spatial adaptation Use linear transform Putting them together EE565 Advanced Image Processing Copyright Xin Li 2009-2012 11

12. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 12 Spatially Adaptive Wiener Filtering Basic assumption: image source is modeled by a nonstationary Gaussian process Signal variance is locally estimated from the windowed noisy observation data T T N=T 2 Recall

13. Image Example EE565 Advanced Image Processing Copyright Xin Li 2009-2012 13 Noisy,  =10 (MSE=100) denoised (T=3,MSE=56)

14. Image Example (Con’d) EE565 Advanced Image Processing Copyright Xin Li 2009-2012 14 Noisy,  =50 (MSE=2500) denoised (MSE=354)

15. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 15 Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Experimental Results Why transform helps? Why spatial adaptation helps?

16. From Scalar to Vector Case EE565 Advanced Image Processing Copyright Xin Li 2009-2012 16 Suppose X is a Gaussian process whose covariance matrix is a diagonalized matrix R X =diag{η m }(m=0,…,N-1), the linear MMSE estimator is given by (A) and the minimal MSE is given by

17. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 17 Decorrelating Q: What if X={X[0],…,X[N-1]} is correlated (i.e., R x is not diagonalized)? A: We need to transform X into a set of uncorrelated basis and then apply the above result. The celebrated Karhunen-Loeve Transform does this job! Diagonal matrix Karhunen-Loeve Transform

18. Transform-Domain Denoising EE565 Advanced Image Processing Copyright Xin Li 2009-2012 18 Forward Transform Inverse Transform Denoising operation e.g., KLT DCT WT e.g., Linear Wiener filtering Nonlinear Thresholding Noisy signal denoised signal The performance of such transform-domain denoising is determined by how well the assumed probability model in the transform domain matches the true statistics of source signal (optimality can only be established for the Gaussian source so far).

19. One-Minute Tour of Wavelets EE565 Advanced Image Processing Copyright Xin Li 2009-2012 19 G0G0 G1G1 x(n) H0H0 H1H1 y 0 (n) y 1 (n) x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) complete expansion (with decimation) overcomplete expansion (without decimation) T ce T ce -1 T oe T oe -1

20. Why Wavelet Denoising? We need to distinguish spatially-localized events (edges) from noise components More about noise components EE565 Advanced Image Processing Copyright Xin Li 2009-2012 20 Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain As long as it does not generate exceptions Additive White Gaussian Noise is just one of them

21. Wavelet Thresholding EE565 Advanced Image Processing Copyright Xin Li 2009-2012 21 DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

22. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 22 Choice of Threshold Donoho and Johnstone’1994 Gives denoising performance close to the “ideal weighting” Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp. 435-453)

23. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 23 Soft vs. Hard thresholding ● It can be also viewed as a computationally efficient approximation of ideal weighting soft ideal ● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better. ● Shrinking the amplitude by T guarantees with a high probability that.

24. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 24 Denoising Example noisy image (σ 2 =100) Wiener-filtering (ISNR=2.48dB) Wavelet-thresholding (ISNR=2.98dB) X: original, Y: noisy, X: denoised ~ Improved SNR

25. What is Wrong with Wavelets? EE565 Advanced Image Processing Copyright Xin Li 2009-2012 25 0 1 N-1 … x(n) H1H1 T -T

26. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 26 Translation Invariance (TI) Denoising T oe T oe -1 Thresholding T ce T ce -1 Thresholding T ce T ce -1 Thresholding z + x(n) Implementation based on overcomplete expansion Implementation based on complete expansion z -1

27. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 27 2D Extension Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

28. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 28 Example Wavelet-thresholding (ISNR=2.98dB) Translation-Invariant thresholding (ISNR=3.51dB)

29. Challenges with wavelet thresholding Determination of a global optimal threshold Spatially adjusting threshold based on local statistics How to go beyond thresholding? We need an accurate modeling of wavelet coefficients – nonlinear thresholding is a computationally efficient yet suboptimal solution EE565 Advanced Image Processing Copyright Xin Li 2009-2012 29 Go Beyond Thresholding

30. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 30 Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

31. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 31 Practical Implementation T T N=T 2 Recall Conceptually very similar to its counterpart in the spatial domain In demo3.zip, you can find a C-coded example (de_noise.c) (ML estimation of signal variance)

32. Example EE565 Advanced Image Processing Copyright Xin Li 2009-2012 32 Translation-Invariant thresholding (ISNR=3.51dB) Spatially-adaptive wiener filtering (ISNR=4.53dB)

33. Further Improvements* Gaussian scalar mixture (GSM) based denoising (Portilla et al.’ 2003) Instead of estimating the variance, it explicitly addresses the issue of uncertainty with variance estimation Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band coefficients (refer to the posted paper) EE565 Advanced Image Processing Copyright Xin Li 2009-2012 33

34. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 34 Gaussian Scalar Mixture (GSM)* Model definition:u~N(0,1) Noisy observation model Gaussian pdf scale (variance) parameter

35. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 35 Basic Idea* In spatially adaptive Wiener filtering, we estimate the variance from the data of a local window. The uncertainty with such variance estimation is ignored. In GSM model, such uncertainty is addressed through the scalar z (it decides the variance of GSM). Instead of using a single z (estimated variance), we build a probability model over z, i.e., E{x|y}=E z {E{x|y,z}}

36. EE565 Advanced Image Processing Copyright Xin Li 2009-2012 36 Posterior Distribution* where Due to is so-called Jeffery’s prior Question: What is E{x c |y,z}? Bayesian formula (proof left as exercise)

37. GSM Denoising Algorithm* EE565 Advanced Image Processing Copyright Xin Li 2009-2012 37 http://decsai.ugr.es/~javier/denoise/index.htmlMATLAB codes available at:

38. Image Examples EE565 Advanced Image Processing Copyright Xin Li 2009-2012 38 Noisy,  =50 (MSE=2500) denoised (MSE=201)

39. Image Examples (Con’d) EE565 Advanced Image Processing Copyright Xin Li 2009-2012 39 Noisy,  =10 (MSE=100) denoised (MSE=31.7)