From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein (Technion), David L. Donoho (Stanford), Michael Elad (Technion) SIAM REVIEW 2009 Presented by: Mingyuan Zhou Duke University, ECE June 11, 2009

Outline Introduction The sparsest solution of Ax = b Variations on P 0 Sparsity-seeking methods in signal processing Processing of sparsely generated signals Applications in image processing

Under-determined linear system equation L2 norm L0 norm How can uniqueness of a solution be claimed? How to verify a candidate solution? How to efficiently solve the problem (the exhaustive search is a NP-hard problem)? What kind of approximations will work and how accurate can those be? Introduction

Conditions under which has a unique solution Conditions under which has the unique solution as Conditions under which the solution can be found by some “pursuit” algorithm Less restrictive notions of sparsity, impact of noise, the behavior of approximate solutions, and the properties of problem instances defined by ensembles of random matrices… Current achievements

JPEG, DCT JPEG-2000, DWT The sparsity of representation under given basis is key to many important signal and image processing problems: Image compression, Image denoising, image deblurring, speech compression, audio compression… The signal processing perspective Measuring sparsity

The sparsest solution of Ax = b

Uniqueness via the Spark Uniqueness via the Mutual Coherence Uniqueness

Greedy Algorithms Convex Relaxation Techniques Pursuit Algorithms: Practice

Greedy Algorithms Convex Relaxation Techniques Pursuit Algorithms: Performance

Uncertainty Principles and Sparsity Variations on P 0

From Exact to Approximate Solutions Relaxed constraint: Stability: Pursuit algorithms: OMP Iteratively reweighted least squares (IRLS) Iterative thresholding Stepwise algorithms: LARS and Homotopy

Performance of pursuit algorithms

Beyond Coherence Arguments Without noise With noise Empirical evidence: The column of A is drawn at random from a Gaussian distribution,,

Phase transitions in typical behavior:

Phase transitions in typical behavior:

Restricted isometry property (RIP):

The sparsest solution of Ax = b: A summary Uniqueness Solvability Approximate solutions Beyond coherence

Sparsity-Seeking Methods in Signal Processing Non-Gaussian Prior Combined representation

Processing of Sparsely Generated Signals Applications Analysis Compression Denoising Inverse problems Compressive sensing Morphological Component Analysis

The quest for a dictionary Reconstructed dictionaries Dictionaries learned from training data Dictionaries learned from data under test Learning Methods: MOD, K-SVD, BPFA

Applications in Image Processing Compression of Facial Images

Denoising of Images

Denoising of Images

Summary