1. From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein (Technion), David L. Donoho (Stanford), Michael Elad (Technion) SIAM REVIEW 2009 Presented by: Mingyuan Zhou Duke University, ECE June 11, 2009

2. Outline Introduction The sparsest solution of Ax = b Variations on P 0 Sparsity-seeking methods in signal processing Processing of sparsely generated signals Applications in image processing

3. Under-determined linear system equation L2 norm L0 norm How can uniqueness of a solution be claimed? How to verify a candidate solution? How to efficiently solve the problem (the exhaustive search is a NP-hard problem)? What kind of approximations will work and how accurate can those be? Introduction

4. Conditions under which has a unique solution Conditions under which has the unique solution as Conditions under which the solution can be found by some “pursuit” algorithm Less restrictive notions of sparsity, impact of noise, the behavior of approximate solutions, and the properties of problem instances defined by ensembles of random matrices… Current achievements

5. JPEG, DCT JPEG-2000, DWT The sparsity of representation under given basis is key to many important signal and image processing problems: Image compression, Image denoising, image deblurring, speech compression, audio compression… The signal processing perspective Measuring sparsity

6. The sparsest solution of Ax = b

7. Uniqueness via the Spark Uniqueness via the Mutual Coherence Uniqueness

8. Greedy Algorithms Convex Relaxation Techniques Pursuit Algorithms: Practice

9. Greedy Algorithms Convex Relaxation Techniques Pursuit Algorithms: Performance

10. Uncertainty Principles and Sparsity Variations on P 0

11. From Exact to Approximate Solutions Relaxed constraint: Stability: Pursuit algorithms: OMP Iteratively reweighted least squares (IRLS) Iterative thresholding Stepwise algorithms: LARS and Homotopy

12. Performance of pursuit algorithms

13. Beyond Coherence Arguments Without noise With noise Empirical evidence: The column of A is drawn at random from a Gaussian distribution,,

15. Phase transitions in typical behavior:

16. Phase transitions in typical behavior:

17. Restricted isometry property (RIP):

18. The sparsest solution of Ax = b: A summary Uniqueness Solvability Approximate solutions Beyond coherence

19. Sparsity-Seeking Methods in Signal Processing Non-Gaussian Prior Combined representation

20. Processing of Sparsely Generated Signals Applications Analysis Compression Denoising Inverse problems Compressive sensing Morphological Component Analysis

21. The quest for a dictionary Reconstructed dictionaries Dictionaries learned from training data Dictionaries learned from data under test Learning Methods: MOD, K-SVD, BPFA

22. Applications in Image Processing Compression of Facial Images

24. Denoising of Images

25. Denoising of Images

26. Summary