1. Elaine Hale, Wotao Yin, Yin Zhang Computational and Applied Mathematics Rice University 2007-08-13, McMaster University, ICCOPT II TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAA

2. 2 The FPC (fixed-point continuation) algorithm A short introduction to Compressed Sensing An imaging perspective Image compression Scene Picture 10 Mega Pixels Why do we compress images?

3. 3 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing Image are compressible Image compression –Take an input image u –Pick a good dictionary © –Find a sparse representation x of u such that || ©x - u || 2 is small –Save x Because Only certain part of information is important (e.g. objects and their edges) Some information is unwanted (e.g. noise) This is traditional compression.

4. 4 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing An imaging perspective This is traditional compression. 10 Mega Pixels 100 Kilobytes

5. 5 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing An imaging perspective This is traditional compression. n : 10 Mega Pixels k:100 Kilobytes

6. 6 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing If only 100 kilobytes are saved, why do we need a 10- megapixel camera in the first place? Answer: a traditional compression algorithm needs the complete image to compute © and x Can we do better than this?

7. 7 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing Let k =|| x || 0, n =dim( x )=dim( u ). In compressed sensing based on l 1 minimization, the number of measurements is m =O( k log( n / k )) (Donoho, Candés-Tao)

8. 8 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing Input Linear encoding Signal acquisition Signal reconstruction Signal representation

9. 9 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing Input Linear encoding Signal acquisition Signal reconstruction Signal representation

10. 10 The FPC (fixed-point continuation) algorithm Introduction to Compressed Sensing Input: b = Bu = B©x, A = B© Output: x In compressed sensing, m =dim( b )<<dim( u )=dim( x )= n Therefore, Ax = b is an underdetermined system Approaches for recovering x (hence the image u ): –Solve min || x || 0, subject to Ax = b –Solve min || x || 1, subject to Ax = b –Other approaches –Their differences: encoding / decoding method, number of measurements, complexity of decoding algorithm, robustness to noise

11. 11 The FPC (fixed-point continuation) algorithm We solve the following problems

12. 12 The FPC (fixed-point continuation) algorithm Existing approaches Methods exactly solving the l1-problem –GPSR (Figueiredo, Nowak, Wright) –Iterative Shresholding (Daubechies, Defrise, and De Mol) –l1_ls (Kim, Koh, Lustig, Boyd, Gorinvesky) –l1_magic (Candes, Romberg) –Lasso, LARS –…… Methods not solving the l1-problem per se –OMP, StOMP, Random Projections, Sparse Tree Rep., Belief Propogation, Gradient Pursuits, Sparsify, Randomized Algorithms, Chaining Pursuit, HHS Pursuit.

13. 13 The FPC (fixed-point continuation) algorithm Difficulties Large scales Completely dense data: A However Solutions x are expected to be sparse The matrices A are often fast transforms

14. 14 The FPC (fixed-point continuation) algorithm Original problem: Linearize f and solve iteratively: Must keep x close to x k : Linearization Combine

15. 15 The FPC (fixed-point continuation) algorithm Shrinkage (Soft-thresholding)

16. 16 The FPC (fixed-point continuation) algorithm Original problem: First-order Taylor series: Must keep x close to x k : Linearization Combine

17. 17 The FPC (fixed-point continuation) algorithm An (well-known) optimality theorem Theorem: Let ¿ >0. x solves the convex problem if and only if

18. 18 The FPC (fixed-point continuation) algorithm The sketch of the algorithm In our report, we used 1/º instead of º.

19. 19 The FPC (fixed-point continuation) algorithm The sketch of the algorithm Identical to iterative thresholding by Daubechies et al (2004) for convex quadratic f.

20. 20 The FPC (fixed-point continuation) algorithm The sketch of the algorithm

21. 21 The FPC (fixed-point continuation) algorithm Summary of properties Simple –Easy to implement –Free of matrix factorization (Low memory requirement) –Takes advantage of fast transforms Flexible Parallelizable Convergence speed? Practical performance?

22. 22 The FPC (fixed-point continuation) algorithm Convergence Under some conditions on f ( x ) (e.g., bounded max/min Hessian eigenvalues): Strong convergence (Combettes and Vajs) q -linearly convergent in 2-norm Finite convergent on certain quantities –Zeros in x * –The signs of non-zero components in x *

23. 23 The FPC (fixed-point continuation) algorithm Computations At each iteration Shrinkage (soft thresholding): Min TV: non-trivial in 2 and higher dimensional spaces

24. 24 The FPC (fixed-point continuation) algorithm

25. 25 The FPC (fixed-point continuation) algorithm Computations At each iteration Shrinkage: TV minimization: parametric max-flow (see our Poster)

26. 26 The FPC (fixed-point continuation) algorithm Continuation Convergence requires Support and signs can be obtained earlier than convergence Faster convergence is achieved by All of the above suggests using an increasing sequence of

27. 27 The FPC (fixed-point continuation) algorithm Some References I.Daubechies, M.Defrise, C.De Mol. Iterative Thresholding, CPAM, 2004. D.Donoho, Y.Tsaig, I.Drori, J.-C.Starck. StOMP, preprint, 2006 M.Figueiredo, R.Nowak, S.Wright. GPSR, preprint, 2007. S.Kim, K.Koh, M.Lustig, S.Boyd, D.Gorinevsky. l1_ls, preprint, 2007. E.Candes, J.Romberg. l1-magic. 2005 A report on FPC is available on my homepage.