1. A Weighted Average of Sparse Representations is Better than the Sparsest One Alone Michael Elad and Irad Yavneh SIAM Conference on Imaging Science ’08 Presented by Dehong Liu ECE, Duke University July 24, 2009

2. Outline Motivation A mixture of sparse representations Experiments and results Analysis Conclusion

3. Motivation Noise removal problem y=x+v, in which y is a measurement signal, x is the clean signal, v is assumed to be zero mean iid Gaussian. Sparse representation x=D , in which D  R n  m, n<m,  is a sparse vector. Compressive sensing problem Orthogonal Matching Pursuit (OMP) Sparsest representation Question: “Does this mean that other competitive and slightly inferior sparse representations are meaningless?”

4. A mixture of sparse representations How to generate a set of sparse representations? –Randomized OMP How to fuse these sparse representations? –A plain averaging

5. OMP algorithm

6. Randomized OMP

7. Experiments and results Model: y=x+v=D  +v D: 100x200 random dictionary with entries drawn from N(0,1), and then with columns normalized;  : a random representations with k=10 non- zeros chosen at random and with values drawn from N(0,1); v: white Gaussian noise with entries drawn from N(0,1); Noise threshold in OMP algorithm T=100(??); Run the OMP once, and the RandOMP 1000 times.

8. Observations

9. Sparse vector reconstruction The average representation over 1000 RandOMP representations is not sparse at all.

10. Denoising factor based on 1000 experiments Denoising factor= Run RandOMP 100 times for each experiment.

11. Performance with different parameters

12. Analysis The RandOMP is an approximation of the Minimum-Mean-Squared-Error (MMSE) estimate. “ ”

13. The above results correspond to a 20x30 dictionary. Parameters: True support=3,  x =1, Averaged over 1000 experiments. 0.511.5  0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Relative Mean-Squared-Error 2 0 1. Emp. Oracle 2. Theor. Oracle 3. Emp. MMSE 4. Theor. MMSE 5. Emp. MAP 6. Theor. MAP 7. OMP 8. RandOMP Comparison

14. Conclusion The paper shows that averaging several sparse representations for a signal lead to better denoising, as it approximates the MMSE estimator.