1. Predicting Wavelet Coefficients Over Edges Using Estimates Based on Nonlinear Approximants Onur G. Guleryuz oguleryuz@erd.epson.com Epson Palo Alto Laboratory Palo Alto, CA google: Onur Guleryuz

2. Overview Topic: Wavelet compression of piecewise smooth signals with edges. (piecewise sparse) Benchmark scenario: Piecewise smooth signal Erase all high frequency wavelet coefficients Predict erased datamse? Outline: Background and Problem Statement Formulation Algorithm Results More than what I am doing, it’s how I am doing it.

3. Notes Q: What are edges? (Vague and loose) A: Edges are localized singularities that separate statistically uniform regions of a nonstationary process. Caveats: This method is not: edge/singularity detection, convex (and therefore not POCS), solving inverse problems under additive noise (wavelet-vaguelette), an explicit edge/singularity model. This method is: a systematic way of constructing adaptive linear estimators, an adaptive sparse reconstruction, based on sparse nonlinear approximants (non-convex by design), a model for non-edges (sparsity/predictable detection). No amount of looking at one side helps predict the other side.

4. Wavelet Compression in 1-D and 2-D Wavelets of compact support achieve sparse decompositions A. Cohen, I. Daubechies, O. G. Guleryuz, and M. T. Orchard, ``On the importance of combining wavelet-based nonlinear approximation with coding strategies,'' IEEE Trans. Info. Theory}, vol. 48, no. 7, pp. 1895-1921, July 2002. 1-D M. N. Do, P. L. Dragotti, R. Shukla, and M. Vetterli, ``On the compression of two-dimensional piecewise smooth functions,'‘ Proc. IEEE Int. Conf. on Image Proc. ICIP ’01, Thessaloniki, Greece, 2001. 2-D Too many wavelet coefficients over edges (Need to reduce)

5. Current Approaches “1”: Modeling higher order dependencies over edges in wavelet domain. F. Arandiga, A. Cohen, M. Doblas, and B. Matei, ``Edge Adapted Nonlinear Multiscale Transforms for Compact Image Representation,'‘ Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003. H. F. Ates and M. T. Orchard, ``Nonlinear Modeling of Wavelet Coefficients around Edges,'‘ Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003. J. Starck, E. J. Candes, and D. L. Donoho, ``The Curvelet Transform for Image Denoising,'‘ IEEE Trans. on Image Proc., vol. 11, pp. 670-684, 2002. P.L. Dragotti and M. Vetterli, ``Wavelet footprints: theory, algorithms, and applications,'‘ IEEE Trans. on Sig. Proc., vol. 51, pp. 1306-1323, 2003. M. Wakin, J. Romberg, C. Hyeokho, and R. Baraniuk, ``Rate-distortion optimized image compression using wedgelets,'‘ Proc. IEEE Int. Conf. Image Proc. June 2002. “2”: New Representations. … … Translation/rotation invariance is an issue. Best linear representations are given by overcomplete transforms. (Reduce by prediction) (Don’t create too many)

6. Q: What are Overcomplete Transforms? Spatial DCT tilings of an Image … image-wide, orthonormal transform G 1 G 2 … G M Image arranged in a (Nx1) vector x, are (NxN) G i Example: Translation invariant, overcomplete transforms

7. Sparse Decompositions and Overcomplete Transforms G 1 sparse portionsnonsparse portions No single orthonormal transform in the overcomplete set provides a very sparse decomposition. G 2 G M … …image

8. Issues with Overcomplete Trfs Compression angle: Thresholding based Denoising: sparse portionsnonsparse portions G 1 G M … … image (x) … remove the insignificant coefficients and the noise that they contain

9. DCC’02 Onur G. Guleryuz, "Nonlinear Approximation Based Image Recovery Using Adaptive Sparse Reconstructions and Iterated Denoising: Part I - Theory“, “Part II – Adaptive Algorithms,” IEEE Transactions on Image Processing, in review. http://eeweb.poly.edu/~onur Fill missing information with initial values, T=T. Denoise image with hard-threshold T. Enforce available information. T=T-dT 0

10. Nonlinear Approximation and Nonconvex Image Models available sample missing sample Sample coordinates for a two sample signal Recovery transform coordinates Find the missing data to minimize Assume single transform

11. Underlying Estimation Method There is method to the denoise, denoise, …, denoise madness. No explicit statistical modeling. Systematic way of generating adaptive linear estimators. It doesn’t care about the nonsparse portions of transforms (must identify sparse portions correctly) Sparse predictable. Relationships to harmonic analysis.

12. DCT2=shift(DCT1)DCTM=… Modeling “Non-Edges” (Sparse Regions) smooth edge DCT1 I don’t care how badly the transform I am using does over the edges. I determine non-edges aggressively.

13. Algorithm Onur G. Guleryuz, ``Weighted Overcomplete Denoising,‘’ Proc. Asilomar Conference on Signals and Systems, Pacific Grove, CA, Nov. 2003. Fill missing information (high frequency wavelet coefficients) with initial values (0), T=T. Denoise image with hard-threshold T. Enforce available information (low frequency wavelet coefficients). T=T-dT 0 I use DCTs and a simple but good denoising technique: http://eeweb.poly.edu/~onur

14. Test Images Teapot (960x1280) Lena (512x512) Graphics (512x512) Bubbles (512x512) Pattern (512x512) Cross (512x512) I admit, you can do edge detection on this one

15. Implementation 1: l-level wavelet transform (l=1, l=2) 2: All high frequency coefficients set to zero (l=1 half resolution, l=2 quarter resolution) 3: Predict missing information 4: Report PSNR=10log10(255*255/mse)

16. Results on Graphics Graphics, l=1 Graphics, l=2 30.48dB to 51dB27.15dB to 37.44dB

17. Results on Bubbles Bubbles, l=1 Bubbles, l=2 33.10dB to 35.10dB29.03dB to 30.14dB

18. Bubbles crop, l=1 Unproc.: 30.41dBPredicted: 33.00dB magnitude info. location info

19. Bubbles crop, l=2 Unproc.: 26.92dBPredicted: 28.20dB

20. Pattern crop, l=1 Unproc.: 25.94dBPredicted: 26.63dB still a jump Holder exponent extrapolation, step edge assumption, edge detection, etc., aren’t going to work well here.

21. Cross crop, l=1 Unproc.: 18.52dBPredicted: 18.78dB Holder exponent extrapolation, step edge assumption, edge detection, etc., aren’t going to work well here.

22. PSNR over 3 and 5 pixel neighborhood of edges (l=1) 3 pixel neigh.5 pixel neigh.overall GraphicsUnprocessed18.23 dB20.22 dB30.48 dB Predicted39.00 dB41.00 dB51.00 dB BubblesU24.61 dB26.52 dB33.10 dB P28.56 dB30.29 dB35.10 dB PatternU20.46 dB22.02 dB27.04 dB P22.39 dB23.83 dB27.48 dB CrossU16.88 dB17.44 dB18.72 dB P18.32 dB18.52 dB18.87 dB +21 dB +2 dB+4 dB +0 dB+1.5 dB +0.5 dB+2 dB

23. Comments and Conclusion I will show a few more results. Around edges, magnitude and location distortions. Instead of trying to model many different types of edges, model non-edges as sparse (same algorithm handles all varieties). Early work 1: Interpolation in pixel domain may give misleading PSNR numbers for two reasons. Early work 2: Hemami’s group and Vetterli’s group have wavelet domain results (based on Holder exponents), but not on same scale. You can implement this for your own transform/filter bank (denoise, available info, reduce threshold, …).

24. Results on Teapot Teapot, l=1 Teapot, l=2 36.17dB to 41.81dB32.54dB to 35.93dB

25. Teapot crop, l=1 Unproc.: 28.38dBPredicted: 34.78dB

26. Teapot crop, l=2 Unproc.: 25.10dBPredicted: ??.??dB

27. Results on Lena Lena, l=1 Lena, l=2 35.26dB to 35.65dB29.58dB to 30.04dB

28. Lena crop, l=1 Unproc.: 34.42dBPredicted: 35.03dB

29. Lena crop, l=2 Unproc.: 27.79dBPredicted: 29.83dB