1. IPIM, IST, José Bioucas, 2007 1 Shrinkage/Thresholding Iterative Methods Nonquadratic regularizers Total Variation lp- norm Wavelet orthogonal/redundant representations sparse regression Majorization Minimization revisietd IST- Iterative Shrinkage Thresolding Methods TwIST-Two step IST

2. IPIM, IST, José Bioucas, 2007 2 Linear Inverse Problems -LIPs

3. IPIM, IST, José Bioucas, 2007 3 References J. Bioucas-Dias and M. Figueiredo, "A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration“ Submitted to IEEE Transactions on Image processing, 2007. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization-Minimization Algorithms for Wavelet-Based Image Deconvolution'', Submitted to IEEE Transactions on Image processing, 2006.

4. IPIM, IST, José Bioucas, 2007 4 More References M. Figueiredo and R. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. on Image Processing, vol. 12, no. 8, pp. 906–916, 2003. J. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors,” IEEE Trans. on Image Processing, vol. 15, pp. 937–951, 2006. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, pp. 89-97, 2004. P. Combettes and V. Wajs, “Signal recovery by proximal forwardbackward splitting,” SIAM Journal on Multiscale Modeling & Simulation vol. 4, pp. 1168–1200, 2005 I. Daubechies, M. Defriese, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint”, Communications on Pure and Applied Mathematics, vol. LVII, pp. 1413-1457, 2004

5. IPIM, IST, José Bioucas, 2007 5 Majorization Minorization (MM) Framework Let EM is an algorithm of this type. Majorization Minorization algorithm:....with equality if and only if Easy to prove monotonicity: Notes: should be easy to maximize

6. IPIM, IST, José Bioucas, 2007 6 MM Algorithms for LIPs IST Class: Majorize IRS Class: Majorize IST/IRS: Majorize and

7. IPIM, IST, José Bioucas, 2007 7 MM Algorithms: IST class Assume that

8. IPIM, IST, José Bioucas, 2007 8 MM Algorithms: IST class Majorizer: Let: IST Algorithm

9. IPIM, IST, José Bioucas, 2007 9 MM Algorithms: IST class Overrelaxed IST Algorithm Convergence: [Combettes and V. Wajs, 2004] is convex the set of minimizers, G, of is non-empty  2 ]0,1] Then converges to a point in G

10. IPIM, IST, José Bioucas, 2007 10 Denoising with convex regularizers Denoising function also known as the Moreou proximal mapping Classes of convex regularizers: 1- homogeneous (TV, lp-norm (p>1)) 2- p power of an lp norm

11. IPIM, IST, José Bioucas, 2007 11 1-Homogeneous regularizers Then where is a closed convex set and denotes the orthogonal projection on the convex set

12. IPIM, IST, José Bioucas, 2007 12 Total variation regularization Total variation [S. Osher, L. Rudin, and E. Fatemi, 1992]  is convex (although not strightly) and 1-homogeneous Total variation is a discontinuity-preserving regularizer have the same TV

13. IPIM, IST, José Bioucas, 2007 13 Then Total variation regularization [Chambolle, 2004]

14. IPIM, IST, José Bioucas, 2007 14 Total variation denoising

15. IPIM, IST, José Bioucas, 2007 15 Total variation deconvolution 2000 IST iterations !!!

16. IPIM, IST, José Bioucas, 2007 16 Weighted lp-norms  is convex (although not strightly) and 1-homogeneous There is no closed form expression for excepts for some particular cases Thus

17. IPIM, IST, José Bioucas, 2007 17 Soft thresholding: p=1 Thus

18. IPIM, IST, José Bioucas, 2007 18 Soft thresholding: p=1

19. IPIM, IST, José Bioucas, 2007 19 Soft thresholding: p=1

20. IPIM, IST, José Bioucas, 2007 20 Another way to look at it: Since L is convex: the point is a global minimum of L iif where is the subdifferential of L at f’

21. IPIM, IST, José Bioucas, 2007 21 Example: Wavelet-based restoration Wavelet basis Wavelet coefficients Detail coefficients (h – high pass filter) Approximation coefficients (g-low pass filter) g,h – quadrature mirror filters DWT, Harr, J=2

22. IPIM, IST, José Bioucas, 2007 22 Example: Wavelet-based restoration Histogram of coefficients - h Histogram of coefficients – log h