1. Introduction to Compressive Sensing
Presenter: 黃乃珊
Advisor: 吳安宇 教授
Date: 2014/03/11

2. Schedule 19:30 @ EEII-225 日期 內容 Lab & HW Speaker 3/11
Introduction to Compressive Sensing System
Nhuang
3/25
Reconstruction Algorithm
4/8
Lab1
4/15 
Break; 決定期末題目方向
4/22
Sampling Algorithm:
Yumin
4/29
Midterm Presentation (Tutorial, Survey)
5/6
Application: Single Pixel Camera
Lab2
5/13 ~ 6/10
期末報告討論
6/24
Final Presentation

3. Outline Review Digital Signal Processing
Motivation of Compressive Sensing
Introduction to Compressive Sensing
Application of Compressive Sensing
Reference

4. Digital Signal Processing
Channel
C/D
Compress
Processing
Transmit
Channel
Receive
Recovery
D/C

5. Sampling in Digital Signal Processing
Normal sampling and quantization
Fixed sampling period
Fixed point precision
Nyquist-Shannon sampling theorem
Nyquist sampling rate = bandwidth * 2

6. Compression in Digital Signal Processing
Original
Lossless compression
No losing information
Data compression
Ex. Lempel-Ziv (LZ) compression, Zip with LZR
Lossy compression
Some loss of information
Image compression, audio compression
Ex. JPEG, MPEG
Compressed

7. Motivation of Compressive Sensing
Measure as much as we can in sampling and discard as much as we can in compressing
ADC with high sampling rate is expensive
Tradeoff between precision and price
In certain application, such as MRI, long sampling time may cause problem
But shorten sampling time may reduce image quality
Sample
Compress
Recovery
Channel
𝒙= 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 ⋯ 𝑵
𝒚= 𝟏 𝟏 𝟎⋯ 𝑴
𝒙 𝑵

8. Compressive Sensing Now, “measure what can be measure”
Compressing after sampling wastes time and memory
Compressive sensing (CS) is to “measure what should be measure”
Nowadays
Sample
Compress
Recovery
Channel
𝒙= 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 ⋯ 𝑵
𝒚= 𝟏 𝟏 𝟎⋯ 𝑴
𝒙 𝑵
Future
Compressive
Sensing
Recovery
Channel
𝒚= 𝟏 𝟏 𝟎⋯ 𝑴
𝒙 𝑵

9. Compressive Sensing in Mathematics
Sampling matrices should satisfy restricted isometry property (RIP)
Ex. Gaussian matrices
Reconstruction solves an underdetermined question
Linear Programming (ex. Basis Pursuit)
Greedy Algorithm (ex. Orthogonal Matching Pursuit)
Iterative Thresholding
(1−𝛿)∙ 𝑀 𝑁 ∙ 𝑥 2 2 ≤ Φ𝑥 2 2 ≤(1+𝛿)∙ 𝑀 𝑁 ∙ 𝑥 2 2
Sampling
Reconstruction
Channel
𝒚 𝑴 = 𝚽 𝑴×𝑵 𝒙 𝑵
𝒙 𝑵
𝒙 𝑵
𝒚 𝑴 +𝒏𝒐𝒊𝒔𝒆
min 𝑥 𝑥 0 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝒚=Φ 𝒙

10. Measurement Acquisition (1/2)
signal , basis
Big pixel, ex. camera
Line integrals, ex. CT
Sinusoids, ex. MRI

11. Measurement Acquisition (2/2)
To minimize the number of samples
Why not take wavelets as basis?
Can we get adaptive approximation from a fixed set of basis?
Yes! Compressive Sensing!
But, which one?

12. Random Sampling in Compressive Sensing
Take random Gaussian signal as basis
Signal structure is local and measurement is global
Each measurement picks up a little information about each component

13. Sparse Recovery Reconstruction problem
Solve an underdetermined question
Linear Programming
min 𝑥 𝑥 0 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝒚=Φ 𝒙

14. Recovery Algorithms of Compressive Sensing
Linear Programming
Basis Pursuit (BP) [8]
Greedy Algorithm
Matching Pursuit
Orthogonal Matching Pursuit (OMP) [9][10]
Stagewise Orthogonal Matching Pursuit (StOMP) [11]
Compressive Sampling Matching Pursuit (CoSaMP) [12][13]
Subspace Pursuit (SP) [14]
Iterative Thresholding
Iterative Hard Thresholding (IHT) [15]

15. Outline Review Digital Signal Processing
Motivation of Compressive Sensing
Introduction to Compressive Sensing
Application of Compressive Sensing
Reference

16. Application: Magnetic Resonance Imaging
Magnetic Resonance Imaging (MRI) is susceptible to motion
Reduce sampling to avoid motion artifacts
CS improve the resolution in under-sampling
M. Lustig, D. Donoho, J. Santos, and J. Pauly, “Compressed Sensing MRI,” IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 72–82, Mar
Conventional MRI CS MRI

17. Application: Edges

18. Application: Classification
with M. Davenport, M. Duarte, R. Baraniuk

19. Application: Single Pixel Camera (1/2)

20. Application: Single Pixel Camera (2/2)

21. Application: Manifold Lifting

22. Reference
[1] E. J. Candes, and M. B. Wakin, "An Introduction To Compressive Sampling," Signal Processing Magazine, IEEE , vol.25, no.2, pp.21-30, March 2008 [2] D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr [3] R. Baraniuk, J. Romberg, and M. Wakin, “Tutorial on Compressive Sensing” [4] J. Romberg, and M. Wakin, “Compressed Sensing: A Tutorial,” Statistical Signal Processing Workshop, IEEE, Aug [5] M. Mishali, and Y. Eldar, “Xampling: From Theory to Hardware of Sub-Nyquist Sampling,” ICSSP Tutorial, May 2011 [6] M. Lusting, D. L. Donoho, J. M. Santos, and J. M. Paulty, “Compressed Sensing MRI”, Signal Processing Magazine, IEEE , vol.25, no.2, pp.72,82, March 2008 [7] Willett RM, Marcia RF, and Nichols JM, “Compressed Sening for Practical Optical Imaging Systems: A Tutorial,” Optical Engineering (2011) [8]