1. Representation and Compression of Multi-Dimensional Piecewise Functions Dror Baron Signal Processing and Systems (SP&S) Seminar June 2009 Joint work with: Venkat Chandrasekaran Michael Wakin Richard Baraniuk

2. The Challenge of Multi-D Horizon Functions Signals have edges –images (2D) –video (3D) –light field imaging (4D, 5D) Horizon class model –multidimensional –discontinuities –smooth areas Main challenge: sparse representation Related applications: approximation, compression, denoising, classification, segmentation… N = 2N = 3

3. Existing tool: 1D Wavelets Advantages for 1D signals: –efficient filter bank implementation –multiresolution framework –sparse representation for smooth signals Success motivates application to 2D, but…

4. 2D Signal Representations Challenge: geometry - discontinuities along 1D contours –separable 2D wavelets (squares) fail to capture geometric structure Response: –tight frames: curvelets [Candés & Donoho], contourlets [Do & Vetterli], bandelets [Mallat] –geometric tilings: wedgelets [Donoho], wedgeprints [Wakin et al.] 1326 52

5. Wedgelet Dictionary [Donoho] wedgelet decomposition Piecewise linear, multiscale representation –supported over a square dyadic block Tree-structured approximation Intended for C 2 discontinuities

6. Non-Separable Representations have Potential to be Sparse Non-separable geometric tiling 1326 52 Separable wavelets

7. Signal Representations in Higher Dimensions Failure of separable wavelets more pronounced in N>2 dimensions Similar problems exist –smooth regions separated by discontinuities –discontinuities often smooth functions in N-1 dimensions Shortcomings of existing work –not yet extended to higher dimensions –intended for efficient (sparse) representations for C 2 discontinuities

8. Goals Develop representation for higher-dimensional data containing discontinuities –smooth N-dimensional function –(N-1)-dimensional smooth discontinuity Optimal rate-distortion (RD) performance –metric entropy – order of RD function Flow of research: –From N=2 dimensions, C 2 -smooth discontinuities –To N ¸ 2 dimensions, arbitrary smoothness

9. Piecewise Constant Horizon Functions [Donoho] f: binary function in N dimensions b: C K smooth (N-1)-dimensional horizon/boundary discontinuity Let x 2 [0,1] N and y = { x 1,…,x N-1 } 2 [0,1] (N-1)

10. Example Horizon Class Functions N = 2N = 3

11. Compression Problem Approximate f with R bits ! Squared L 2 error metric (energy) Need optimal tradeoff between rate and L 2 distortion

12. Compression via Implicit Approximation Edge detection: –estimate horizon discontinuity b –encode using (N-1)-dimensional wavelets [Cohen et al.] Implicitly approximate f from b Theorem [Kolmogorov & Tihomirov; Clements] : Metric entropy for C K smooth (N-1)-D function: L 1 distortion O( ¢ ) lower bound

13. Metric Entropy for Horizon Functions Problems with edge detection: –edge detection often impractical –want to approximate f (not b) require solution that provides estimate in N- dimensions, without explicit knowledge of b Theorem: Metric entropy for N-D horizon function f with C K smooth (N-1)-D discontinuity: Converse result – our algorithms achieve this RD performance

14. Motivation for Solution: Taylor’s Theorem For a C K function b in (N-1) dimensions, Key idea: order (K-1) polynomial approximation on small regions Challenge: organize tractable discrete dictionary for piecewise polynomial approximation derivatives

15. Surflets: Piecewise Polynomial Approximations on Dyadic Hypercubes Surflet at scale j –N-dimensional atom –defined on hypercube X j of size 2 -j £ 2 -j £  £ 2 -j –horizon function with order K-1 polynomial discontinuity (“surface”-let) Tile to form multiscale approximation to f K = 2K = 3K = 4 Wedgelet

16. 3D Surflets K = 2 K = 3

17. Discrete Surflet Dictionary Describe surflet using polynomial coefficients K = 2K = 3K = 4 K = 2K = 3 Wedgelet

18. Quantization Challenge: with naïve quantization of coefficients, dictionary size blows up with K and N Surflet coefficients approximate Taylor coefficients Higher-order coefficients quantized with lesser precision  same order error for all coefficients Response: for order- l coefficient, use step-size ~ O(2 -(K-2)j ) ~ O(2 -2j ) ~ O(2 -Kj )

19. Approximation without Edge Detection “Taylor surflets” –obtained by quantizing derivatives of b –requires knowledge/estimation of b “L 2 -best surflets” –obtained by searching dictionary for best fit –requires no explicit knowledge of b –fast search algorithm via manifolds Theorem: Taylor or L 2 –best surflets have same asymptotic performance

20. Tree-structured Surflet Approximation Arrange surflets on 2 N -tree –each node is either a leaf or has 2 N children –all nodes labeled with surflets –leaf nodes provide approximation –interior nodes useful for predictive coding

21. Tree-structured Surflet Encoder Surflet leaf encoder achieves near-optimal RD performance Top-down predictive encoder –code all nodes in surflet tree –use parent surflets to predict children –constant # bits per surflet regardless of scale –layered coarse-scale approximation in early bits Theorem: Top-down predictive encoder achieves

22. Discretization Signals often acquired discretely (pixels/voxels) Pixelization artifacts at fine scales Approach to discrete data –discretize continuous surflet dictionary –coarse scales: use regular dictionary –smaller dictionary at fine scales Theorem: Predictive encoder achieves same RD performance at low rate with discretized dictionary

23. Numerical Example N=2,K=3 1024 £ 1024 pixels Scale-adaptive dictionaries Wedgelets: 482 bits, 29.9 dB Surflets: 275 bits, 30.2 dB

24. RD Results Dictionary 1: fixed-scale wedgelets Dictionary 2: wedgelets + scale-adaptive Dictionary 3: surflets + scale-adaptive

25. Piecewise Smooth Horizon Functions g 1,g 2 : real-valued smooth functions –N dimensional –C K s smooth b: C K d smooth (N-1)-dimensional horizon/boundary discontinuity Theorem: Metric entropy for C K s smooth N-D horizon function f with C K d smooth discontinuity: b(x 1 ) g 1 ([x 1, x 2 ]) g 2 ([x 1, x 2 ])

26. Surfprints Challenge: –wavelets good in smooth regions –wavelets wasteful near discontinuity Surflets good near edges Response: surfprints project surflets to wavelet subspace 1326 52

27. Tree-structured Surprint Encoder Discontinuity information needed at finer scales Top-down encoder Prediction not used Theorem: Top-down encoder achieves near-optimal w ww wwww ww surfprint w coarse intermediate maximal –coarse: keep wavelet nodes –intermediate: nodes with discontinuity –maximal depth: surfprints

28. Conclusions and Future Work Metric entropy (converse) –piecewise constant/smooth horizon functions –arbitrary dimension & arbitrary smoothness Multiresolution compression framework (achievable) –quantization scheme  tractable dictionary size –predictive top-down coding  optimal performance –scale-adaptive approach to discretization –surfprints at maximal depth  near-optimal Future research: algorithms

29. THE END