Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell Came out of my personal experience with 301 – fourier analysis and linear systems The work has been supported by the Israel Science Foundation and National Science Foundation.
CS Measurement Replace samples by more general encoder based on a few linear projections (inner products) measurements sparse signal # non-zeros
Caveats Input x strictly sparse w/ real values Noiseless measurements noise can be addressed (later) Assumptions relevant to content distribution (later)
Why is Decoding Expensive? Culprit: dense, unstructured sparse signal measurements nonzero entries
Sparse Measurement Matrices (dense later!) LDPC measurement matrix (sparse) Only {-1,0,+1} in Each row of contains L randomly placed nonzeros Fast matrix-vector multiplication fast encoding & decoding sparse signal measurements nonzero entries
Example 1 4 0 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 0 0 0 0 1 1 ?
Example What does zero measurement imply? Hint: x strictly sparse 1 4 0 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 0 0 0 0 1 1 ?
Example Graph reduction! 0 1 1 0 0 0 ? 1 0 0 0 1 1 0 1 1 0 0 1 0 4 1 4 0 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 0 0 0 0 1 1 ?
Example What do matching measurements imply? Hint: non-zeros in x are real numbers 1 4 0 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 0 0 0 0 1 1 ?
Example What is the last entry of x? 0 1 1 0 0 0 1 0 0 0 1 1 0 1 4 0 1 1 0 0 0 0 0 0 1 1 0 1 1 0 0 1 0 0 0 0 0 1 1 1 ?
Noiseless Algorithm [Luby & Mitzenmacher 2005] [Sarvotham, Baron, & Baraniuk 2006] [Zhang & Pfister 2008] Phase1: zero measurements Initialize Phase2: matching measurements typically iterate 2-3 times Phase3: singleton measurements Arrange output Done? yes no
Numbers (4 seconds) N=40,000 5% non-zeros M=0.22N L=20 ones per row Iteration, Phase Zeros Non-zeros Total 1,1 30615 1,2 35224 977 36201 1,3 1500 36724 2,1 36800 38300 2,2 37180 1833 39013 2,3 2063 39243 3,1 37268 39331 3,2 37289 2074 39363 3,3 2083 39372 4,1 4,2 37291 2084 39375 4,3 iteration #1 N=40,000 5% non-zeros M=0.22N L=20 ones per row Only 2-3 iterations
Challenge With measurements parts of signal still not reconstructed How do we recover the rest of the signal?
Solution: Hybrid Dense/Sparse Matrix With measurements parts of signal still not reconstructed Add extra dense measurements Residual of signal w/ residual dense columns residual columns
Sudocodes with Two-Part Decoding [Sarvotham, Baron, & Baraniuk 2006] Sudocodes (related to sudoku) Graph reduction solves most of CS problem Residual solved via matrix inversion Residual via matrix inversion sudo decoder residual columns
Contribution 1: Two-Part Reconstruction Many CS algorithms for sparse matrices [Gilbert et al., Berinde & Indyk, Sarvotham et al.] Many CS algorithms for dense matrices [Cormode & Muthukrishnan, Candes et al., Donoho et al., Gilbert et al., Milenkovic et al., Berinde & Indyk, Zhang & Pfister, Hale et al.,…] Solve each part with appropriate algorithm sparse solver residual via dense solver residual columns
Runtimes (K=0.05N, M=0.22N)
Theoretical Results [Sarvotham, Baron, & Baraniuk 2006] Fast encoder and decoder sub-linear decoding complexity caveat: constructing data structure Distributed content distribution sparsified data measurements stored on different servers any M measurements suffice Strictly sparse signals, noiseless measurements
Contribution 2: Noisy Measurements Results can be extended to noisy measurements Part 1 (zero measurements): measurement |ym|< Part 2 (matching): |yi-yj|< Part 3 (singleton): unchanged
Problems with Noisy Measurements Multiple iterations alias noise into next iteration! Use one iteration Requires small threshold (large SNR) Contribution 3: Provable reconstruction deterministic & random variants
Summary Hybrid Dense/Sparse Matrix Simple (cute?) algorithm Fast Two-part reconstruction Simple (cute?) algorithm Fast Applicable to content distribution Expandable to measurement noise
THE END