Compressed sensing Carlos Becker, Guillaume Lemaître & Peter Rennert 3D Digitization Course Carlos Becker, Guillaume Lemaître & Peter Rennert

Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert Outline Introduction and motivation Compressed sensing and reconstruction workflow Applications: MRI and single-pixel camera 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Signal sparsity 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Why do we care about sparsity? Original 1 Megapixel image Non-sparse values 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Why do we care about sparsity? But, in the wavelet domain we get these coefficients: 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Why do we care about sparsity? And the histogram of those coefficients is: The image is a nearly sparse in the wavelet domain… 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What happens if we only keep the 50000 highest coefficients in the wavelet domain, set the rest to zero and reconstruct the image ? Reconstructed image (only 50k highest wavelet coefficients) Original image 95% of the original image data was discarded 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Real world signal x(t), sampled into a discrete version x[n] Conservative approach: Nyquist-Shanon sampling theorem: 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? So, why would we sample 1 million pixels if we are going to throw away 95% of image data when compressing the image in JPEG? Candès et al. showed that it is possible to subsample a signal if it is sparse in some domain, being able to obtain a perfect reconstruction if certain conditions are met. 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? The Candès-Romberg-Tao Framework Consider the case that we want to estimate a signal x Suppose we know that x, with dimension N, is sparse in some domain, with at most K non-zero coefficients in that domain (ie: the wavelet domain) Then, we need to measure least K log(N/K) random samples of x to be able to reconstruct the signal x perfectly. In the previous case, this means that we could reconstruct an approximation to the original image with only 66,000 measurements (15x subsampling factor) Candès, E.J.; Romberg, J.; Tao, T.: “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information” (2004) 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

What is compressed sensing? Classic approach to compression: Measure everything (ie: all pixels) Apply some compression algorithm (ie: JPEG2000) But, why would we sample 1 million pixels if we are going to throw away 90% of image data when compressing the image in JPEG? Compressed sensing approach: if signal is sparse in some domain Sample M << N random measurements Reconstruct original signal by L1 minimization Full-resolution image (N pixels/measurements) Lossy compression Candès et al. showed that it is possible to subsample a signal if it is sparse in some domain, being able to obtain a perfect reconstruction if certain conditions are met. Random sampling (M << N measurements) Image reconstruction 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed Sensing Motivation from mri 2004, Candes came to results that people of his time could not believe For a simple phantom (a) its possible to sample at only 22 radial lines (b) (equal to a sampling rate of π / 22, about 50 times below the Nyquist rate of 2 π) to retrieve a perfect reconstruction (d) What does the trick? Simply setting the unknown Fourier coefficients to 0 leads to a very bad result (c) Candès, E.J.; Romberg, J.; Tao, T.: “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information” (2004) 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed sensing Reconstruction Workflow Sparse signal gets randomly sampled in another non-sparse domain (k-space) Reconstruction leads to noisy non-sparse signal with significant peaks where original signal was high After thresholding of significant peaks the strongest components of the original signal are detected Using the noisy reconstruction of the newly sampled strongest components in k-space, the impact of this strongest components on the first reconstruction are determined and subtracted, leaving peaks of weaker components With this iterative strategy weaker and weaker components can be retrieved Michael Lustig, David Donoho, John M. Pauly: “Sparse MRI: The application of compressed sensing for rapid MR imaging” (2007) DL Donoho, I Drori, Y Tsaig : Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit” (2006) 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed sensing Rapid MRI – non-sparse Signal (Signal here means the underlying image that is sensed in the Fourier space) Non- sparse signal sampled in sparse domain That means: reconstruction of samples will produce no significant peaks (since there are no outstanding peaks in the signal domain) Solution: use other sparse domain of signal for “reconstruction” and filtering of significant peaks Michael Lustig, David Donoho, John M. Pauly: “Sparse MRI: The application of compressed sensing for rapid MR imaging” 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera General principle Object Photodiode Σ Memory Several Measurements Reconstruction DMD M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera Reconstruction For M measurements: Y: Measurements X: Image input Φ: Concatenation of test functions M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera Reconstruction Implication of sparsity: X: Input Image Ψ: Dictionnary or Basis Matrix α : Wavelet Coefficients M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera Reconstruction Thus: X: Input Image Φ: Concatenation of test functions Ψ: Dictionnary or Basis Matrix α : Wavelet Coefficients M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera Reconstruction Reconstruction using: such as Number theoric of measurements: M: Number theoric of measurements K: Number of non zeros in α N: Number of pixels M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Single pixel camera Results 2 % measurements 5 % measurements Original: 16384 pixels 10 % measurements 20 % measurements M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, Vol. 25, No. 2, pp. 83-91, March 2008 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert

Compressed sensing conclusion Compressed sensing lets us sub-sample a signal w.r.t. Nyquist rate and reconstruct it perfectly, if this signal is known to be sparse in some domain and some conditions are met Compressed sensing is promising for a wide range of future technologies, specially for high-frequency signals Speeds up acquisition process: specially interesting for MRI Cheaper hardware (ie: IR cameras with only a few sensors) Sparsity can also be exploited for classification and image processing tasks[Huang, K. and Aviyente, S., Sparse representation for signal classification] 8/11/2010 Compressed Sensing - Carlos Becker, Guillaume Lemaître, Peter Rennert