Introduction to Compressive Sensing Presenter: 黃乃珊 Advisor: 吳安宇 教授 Date: 2014/03/11

Schedule 19:30 @ EEII-225 日期 內容 Lab & HW Speaker 3/11 Introduction to Compressive Sensing System Nhuang 3/25 Reconstruction Algorithm 4/8 Lab1 4/15  Break; 決定期末題目方向 4/22 Sampling Algorithm:   Yumin 4/29 Midterm Presentation (Tutorial, Survey) 5/6 Application: Single Pixel Camera Lab2 5/13 ~ 6/10 期末報告討論 6/24 Final Presentation

Outline Review Digital Signal Processing Motivation of Compressive Sensing Introduction to Compressive Sensing Application of Compressive Sensing Reference

Digital Signal Processing Channel C/D Compress Processing Transmit Channel Receive Recovery D/C

Sampling in Digital Signal Processing Normal sampling and quantization Fixed sampling period Fixed point precision Nyquist-Shannon sampling theorem Nyquist sampling rate = bandwidth * 2

Compression in Digital Signal Processing Original Lossless compression No losing information Data compression Ex. Lempel-Ziv (LZ) compression, Zip with LZR Lossy compression Some loss of information Image compression, audio compression Ex. JPEG, MPEG Compressed

Motivation of Compressive Sensing Measure as much as we can in sampling and discard as much as we can in compressing ADC with high sampling rate is expensive Tradeoff between precision and price In certain application, such as MRI, long sampling time may cause problem But shorten sampling time may reduce image quality Sample Compress Recovery Channel 𝒙= 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 ⋯ 𝑵 𝒚= 𝟏 𝟏 𝟎⋯ 𝑴 𝒙 𝑵

Compressive Sensing Now, “measure what can be measure” Compressing after sampling wastes time and memory Compressive sensing (CS) is to “measure what should be measure” Nowadays Sample Compress Recovery Channel 𝒙= 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 ⋯ 𝑵 𝒚= 𝟏 𝟏 𝟎⋯ 𝑴 𝒙 𝑵 Future Compressive Sensing Recovery Channel 𝒚= 𝟏 𝟏 𝟎⋯ 𝑴 𝒙 𝑵

Compressive Sensing in Mathematics Sampling matrices should satisfy restricted isometry property (RIP) Ex. Gaussian matrices Reconstruction solves an underdetermined question Linear Programming (ex. Basis Pursuit) Greedy Algorithm (ex. Orthogonal Matching Pursuit) Iterative Thresholding (1−𝛿)∙ 𝑀 𝑁 ∙ 𝑥 2 2 ≤ Φ𝑥 2 2 ≤(1+𝛿)∙ 𝑀 𝑁 ∙ 𝑥 2 2 Sampling Reconstruction Channel 𝒚 𝑴 = 𝚽 𝑴×𝑵 𝒙 𝑵 𝒙 𝑵 𝒙 𝑵 𝒚 𝑴 +𝒏𝒐𝒊𝒔𝒆 min 𝑥 𝑥 0 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝒚=Φ 𝒙

Measurement Acquisition (1/2) signal , basis Big pixel, ex. camera Line integrals, ex. CT Sinusoids, ex. MRI

Measurement Acquisition (2/2) To minimize the number of samples Why not take wavelets as basis? Can we get adaptive approximation from a fixed set of basis? Yes! Compressive Sensing! But, which one?

Random Sampling in Compressive Sensing Take random Gaussian signal as basis Signal structure is local and measurement is global Each measurement picks up a little information about each component

Sparse Recovery Reconstruction problem Solve an underdetermined question Linear Programming min 𝑥 𝑥 0 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝒚=Φ 𝒙

Recovery Algorithms of Compressive Sensing Linear Programming Basis Pursuit (BP) [8] Greedy Algorithm Matching Pursuit Orthogonal Matching Pursuit (OMP) [9][10] Stagewise Orthogonal Matching Pursuit (StOMP) [11] Compressive Sampling Matching Pursuit (CoSaMP) [12][13] Subspace Pursuit (SP) [14] Iterative Thresholding Iterative Hard Thresholding (IHT) [15]

Outline Review Digital Signal Processing Motivation of Compressive Sensing Introduction to Compressive Sensing Application of Compressive Sensing Reference

Application: Magnetic Resonance Imaging Magnetic Resonance Imaging (MRI) is susceptible to motion Reduce sampling to avoid motion artifacts CS improve the resolution in under-sampling M. Lustig, D. Donoho, J. Santos, and J. Pauly, “Compressed Sensing MRI,” IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 72–82, Mar. 2008. Conventional MRI CS MRI

Application: Edges

Application: Classification with M. Davenport, M. Duarte, R. Baraniuk

Application: Single Pixel Camera (1/2)

Application: Single Pixel Camera (2/2)

Application: Manifold Lifting

Reference [1] E. J. Candes, and M. B. Wakin, "An Introduction To Compressive Sampling," Signal Processing Magazine, IEEE , vol.25, no.2, pp.21-30, March 2008 [2] D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006. [3] R. Baraniuk, J. Romberg, and M. Wakin, “Tutorial on Compressive Sensing” [4] J. Romberg, and M. Wakin, “Compressed Sensing: A Tutorial,” Statistical Signal Processing Workshop, IEEE, Aug. 2007 [5] M. Mishali, and Y. Eldar, “Xampling: From Theory to Hardware of Sub-Nyquist Sampling,” ICSSP Tutorial, May 2011 [6] M. Lusting, D. L. Donoho, J. M. Santos, and J. M. Paulty, “Compressed Sensing MRI”, Signal Processing Magazine, IEEE , vol.25, no.2, pp.72,82, March 2008 [7] Willett RM, Marcia RF, and Nichols JM, “Compressed Sening for Practical Optical Imaging Systems: A Tutorial,” Optical Engineering (2011) [8] http://videolectures.net/mlss09us_candes_ocsssrl1m/