1. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image smoothing based approach – Wavelet-thresholding based approach Deringing of wavelet coded images – Re-compression approach – PDE-based approach Post-processing by alternating projections: a unified approach

2. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 2 Block Artifacts Smooth areas become blocky JPEG decoded image at 0.23bpp zoom

3. Why block artifacts occur? EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 3 x f(x) original x f(x) ^ B2B3B JPEG decoded at low bit rate Only DC component is preserved

4. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 4 Deblocking as Denoising Standard image denoising algorithm JPEG compressed image postprocessed image th 1000 Manually tune the threshold parameter!

5. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 5 Experiment Results before deblockingafter deblocking

6. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 6 Wavelet-based Deblocking Behavior of block artifacts in wavelet domain

7. Fundamental Issues behind Deblocking Motivation - modeling uncertainty – Location of block artifacts is known (block boundaries) – How to distinguish significant coefficients generated by artifacts from those associated with true edges Strategy – Recall how JPEG2000 is free from block artifacts EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 7

8. 8 Deblocking via Wavelet Thresholding Wavelet Transform Block Wavelet Transform reorder X X Y1Y1 Y2Y2 X: JPEG decoded image

9. Apply both WT and block-based WT to X to get Y 1,Y 2 ; Locate the coefficients at block boundaries; If |Y 1 (i,j)|>T and |Y 2 (i,j)|<T, apply soft thresholding to Y 1 (i,j); Apply IWT to processed Y 1 to obtain deblocked image EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 9 Deblocking Algorithm

10. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 10 Example Before deblocking (PSNR=27.39dB) After deblocking (PSNR=28.07dB)

11. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 11 Ringing Artifacts Sharp edges become unnatural JPEG2000 decoded image at 0.125bpp zoom

12. Why ringing artifacts occur? EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 12 x(n) H1H1 Key observation: wavelet transform lacks translation invariance 2 x(n-1) H1H1 2 origin

13. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 13 Deringing by Re-compression JPEG: JPEG2000 encoder JPEG -1 : JPEG2000 decoder

14. Example EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 14 before processing after processing

15. PDE-based Deringing The power of anisotropic diffusion – Nonlinear diffusion can handle a variety of noise – Which PDE is suitable for deringing? – Implication into wavelet coding EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 15

16. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 16 Perona-Malik Filtering PSNR=30.86dB PSNR=31.09dB

17. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 17 Mean Curvature Filtering PSNR=31.09dB PSNR=30.27dB

18. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 18 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image smoothing based approach – Wavelet-thresholding based approach Deringing of wavelet coded images – Re-compression approach – PDE-based approach Post-processing by alternating projections: a unified approach

19. Recall: Alternating Projection EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 19 X0X0 X1X1 X2X2 X∞X∞ Projection-Onto-Convex-Set (POCS) Theorem: If C 1,…,C k are convex sets, then alternating projection P 1,…,P k will converge to the intersection of C 1,…,C k if it is not empty Alternating projection does not always converge in the case of non-convex set. Can you think of any counter-example? C1C1 C2C2

20. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 20 Projection Operators ● Constraint set y y+T/2 y-T/2 ● Constraint set x B x B fP s (f)

21. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 21 Projection-based Deblocking ● DCT quantization set DCT Quantization ● Smoothness constraint set C s ={f|f is smooth in the block boundaries} at block boundaries Linear edge detection operator Yongyi Yang; Galatsanos, N.P.; Katsaggelos, A.K.;, "Projection-based spatially adaptive reconstruction of block-transform compressed images,“ IEEE Trans. on Image Proc., vol.4, no.7, pp.896-908, Jul 1995

22. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 22 Projection-based Deringing ● WT quantization set WT Quantization ● Smoothness constraint set Perona-Malik diffusion as a nonlinear projection operator Xin Li;, "Improved wavelet decoding via set theoretic estimation," IEEE Trans. on CSVT, vol.15, no.1, pp. 108- 112, Jan. 2005

23. Algorithm Flowchart EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 23 C 1 : observation constraint set C 2 : regularization constraint set

24. Summary Connection with models (PDE-based, wavelet- based, patch-based) – They serve as image prior/regularization constraint set – Jointly work with quantization (observation data) constraint set Convergence is NOT always guaranteed but can be terminated strategically. EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 24