1. EE565 Advanced Image Processing Copyright Xin Li 2008 1 Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering Latest advances Patch-based image denoising Learning-based image denoising

2. EE565 Advanced Image Processing Copyright Xin Li 2008 2 From Scalar to Vector Case Suppose X is a Gaussian process whose covariance matrix is a diagonalized matrix R X =diag{η m }(m=0,…,N-1), the linear MMSE estimator is given by (A) and the minimal MSE is given by

3. EE565 Advanced Image Processing Copyright Xin Li 2008 3 Decorrelating Q: What if X={X[0],…,X[N-1]} is correlated (i.e., R x is not diagonalized)? A: We need to transform X into a set of uncorrelated basis and then apply the above result. The celebrated Karhunen-Loeve Transform does this job! Diagonal matrix Karhunen-Loeve Transform

4. EE565 Advanced Image Processing Copyright Xin Li 2008 4 Transform-Domain Denoising Forward Transform Inverse Transform Denoising operation e.g., KLT DCT WT e.g., Linear Wiener filtering Nonlinear Thresholding Noisy signal denoised signal The performance of such transform-domain denoising is determined by how well the assumed probability model in the transform domain matches the true statistics of source signal (optimality can only be established for the Gaussian source so far).

5. EE565 Advanced Image Processing Copyright Xin Li 2008 5 Why Wavelet Denoising? We need to distinguish spatially- localized events (edges) from noise components More about noise components Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain As long as it does not generate exceptions Additive White Gaussian Noise is just one of them

6. EE565 Advanced Image Processing Copyright Xin Li 2008 6 Tails of Distribution noise signal

7. EE565 Advanced Image Processing Copyright Xin Li 2008 7 Wavelet Thresholding DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

8. EE565 Advanced Image Processing Copyright Xin Li 2008 8 Choice of Threshold Donoho and Johnstone’1994 Gives denoising performance close to the “ideal weighting” Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp. 435-453)

9. EE565 Advanced Image Processing Copyright Xin Li 2008 9 Soft vs. Hard thresholding ● It can be also viewed as a computationally efficient approximation of ideal weighting soft ideal ● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better. ● Shrinking the amplitude by T guarantees with a high probability that.

10. EE565 Advanced Image Processing Copyright Xin Li 2008 10 Denoising Example noisy image (σ 2 =100) Wiener-filtering (ISNR=2.48dB) Wavelet-thresholding (ISNR=2.98dB) X: original, Y: noisy, X: denoised ~ Improved SNR

11. EE565 Advanced Image Processing Copyright Xin Li 2008 11 Duality with Image Coding* DWT IWTThresholding DWT IWT Q Q -1 Channel Image denoising system Image coding system

12. EE565 Advanced Image Processing Copyright Xin Li 2008 12 Difference from Image Coding G0G0 G1G1 x(n) H0H0 H1H1 y 0 (n) y 1 (n) x(n) H0H0 H1H1 2 2 G0G0 2 2 G1G1 s(n) d(n) complete expansion (non-redundant) - suitable for image coding overcomplete expansion (redundant) - suitable for image denoising T ce T ce -1 T oe T oe -1

13. EE565 Advanced Image Processing Copyright Xin Li 2008 13 What do We Buy from Redundancy? 0 1 N-1 … x(n) H1H1 T -T

14. EE565 Advanced Image Processing Copyright Xin Li 2008 14 Translation Invariance (TI) Denoising T oe T oe -1 Thresholding T ce T ce -1 Thresholding T ce T ce -1 Thresholding z + x(n) Implementation based on overcomplete expansion Implementation based on complete expansion z -1

15. EE565 Advanced Image Processing Copyright Xin Li 2008 15 2D Extension Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

16. EE565 Advanced Image Processing Copyright Xin Li 2008 16 Example Wavelet-thresholding (ISNR=2.98dB) Translation-Invariant thresholding (ISNR=3.51dB)

17. EE565 Advanced Image Processing Copyright Xin Li 2008 17 Go Beyond Thresholding Challenges with wavelet thresholding Determination of a global optimal threshold Spatially adjusting threshold based on local statistics How to go beyond thresholding? We need an accurate modeling of wavelet coefficients – nonlinear thresholding is a computationally efficient yet suboptimal solution

18. EE565 Advanced Image Processing Copyright Xin Li 2008 18 Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

19. EE565 Advanced Image Processing Copyright Xin Li 2008 19 Practical Implementation T T N=T 2 Recall Conceptually very similar to its counterpart in the spatial domain In demo3.zip, you can find a C-coded example (de_noise.c) (ML estimation of signal variance)

20. EE565 Advanced Image Processing Copyright Xin Li 2008 20 Example Translation-Invariant thresholding (ISNR=3.51dB) Spatially-adaptive wiener filtering (ISNR=4.53dB)