Shriram Sarvotham Dror Baron Richard Baraniuk ECE Department Rice University dsp.rice.edu/cs Sudocodes Fast measurement and reconstruction of sparse signals

Motivation: coding of sparse data Distributed delivery of data with sparse representation –Content delivery networks –Peer to peer networks –Distributed file storage systems E.g. thresholded DCT/wavelet coefficients used in JPEG/JPEG2000

Motivation: coding of sparse data Distributed coding of sparse data –Can we exploit sparsity? –Efficient? –Low complexity?

Sparse signal processing Signal has non-zero coefficients Efficient ways to measure and recover ? Traditional DSP approach: –Acquisition: first obtain measurements –Then exploit sparsity is in the processing stage

Sparse signal processing Signal has non-zero coefficients Efficient ways to measure and recover ? Traditional DSP approach: –Acquisition: first obtain measurements –Then exploit sparsity is in the processing stage New compressive sampling (CS) approach: –Acquisition: obtain just measurements –Sparsity is exploited during signal acquisition [Candes et al; Donoho]

Compressive sampling Signal is -sparse Measure signal via few linear projections Enough to encode the signal measurements sparse signal nonzero entries

Compressive sampling Signal is -sparse Measure signal via few linear projections Random Gaussian measurements will work! measurements sparse signal nonzero entries

CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries Find the explanation with smallest L 1 norm [Candes et al; Donoho] If then perfect reconstruction w/ high probability

CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries Performance – Polynomial complexity reconstruction – Efficient encoding

CS Miracle: L 1 reconstruction measurements sparse signal nonzero entries But… is still impractical for many applications Reconstruction times:  N=1,000t=10 seconds  N=10,000t=3 hours  N=100,000t=~months

Why is reconstruction expensive? measurements sparse signal nonzero entries

Why is reconstruction expensive? measurements sparse signal nonzero entries Culprit: dense, unstructured

Fast CS reconstruction measurements sparse signal nonzero entries Sudocode matrix (sparse) Only 0/1 in Each row of contains randomly placed 1’s

Sudocodes measurements sparse signal nonzero entries Sudocode performance –Efficient encoding –Sub-linear complexity reconstruction Encouraging numerical results N=100,000 K=1,000  t=5.47 seconds M=5,132

Sudocode reconstruction measurements sparse signal nonzero entries Process each in succession Each can recover some ‘’s

Case 1: Zero measurement

Resolves all coefficients in the support Can resolve up to coefficients

Case 1: Zero measurement Resolves all coefficients in the support Can resolve up to coefficients Reduces size of problem

Case 2: #(support set)=1

Trivially resolves

Case 2: #(support set)=1 Trivially resolves

Case 3: Matching measurements

Common support Matches originate from same support Disjoint support  coefficients = 0 Common support  contain nonzeros

Case 3: Matching measurements Matches originate from same support Disjoint support  coefficients = 0 Common support  contain nonzeros

Trigger of revelations Recovery of can trigger more revelations

Trigger of revelations Recovery of can trigger more revelations An avalanche of coefficient revelations

Trigger of revelations Recovery of can trigger more revelations An avalanche of coefficient revelations

Sudocode reconstruction measurements sparse signal nonzero entries Like sudoku puzzles

Practical considerations Bottleneck: search for matches –With Binary Search Tree, matches ~ Re-explain measurements: more data structures Search for matches

Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed

Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed

Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Intuition: so that

Design of Sudo measurement matrix Choice of Small : Most measurements reveal Many measurements needed Large : Most measurements uninformative Many measurements needed Intuition: so that

Related work [Cormode, Muthukrishnan] –CS scheme based on group testing – –Complexity [Gilbert et. al.] Chaining Pursuit –CS scheme based on group testing and iterating – –Complexity –Works best for super-sparse signals

Performance comparison L 1 reconstruction Chaining Pursuit Sudocodes N=10,000 K=10 M=99 T3 hours M=5,915 t=0.16 sec M=461 t=0.14 sec N=10,000 K=100 M=664 T3 hours M=90,013 t=2.43 sec M=803 t=0.37 sec N=100,000 K=10 M=1,329 Tmonths M=17,398 t=1.13 sec M=931 t=1.09 sec N=10,000 K=1000 M=3,321 T3 hours M>10 6 t>30 sec M=5,132 t=5.47 sec measurements sparse signal nonzero entries

Utility in CDNs Measurements come from different sources Needs enough measurements

Ongoing work Statistical dependencies between non-zero coefficients Irregular degree distributions Adaptive linear projections Noisy measurements

Conclusions Sudocodes for CS –highly efficient –low complexity Key idea: use sparse Applications to content distribution

THE END Compressed sensing webpage: dsp.rice.edu/cs

Number of measurements Theorem: With, phase 1 requires to exactly reconstruct coefficients Proof sketch:

Two phase decoding Phase 1: decode coefficients Phase 2: decode remaining coefficients Why? –When most coefficients are decoded, Phase 2 saves a factor of measurements is not measured

Phase 2 measurements and decoding is non-sparse of dimension

Phase 2 measurements and decoding is non-sparse of dimension Resolve remaining coefficients by inverting the sub-matrix of

Phase 2 measurements and decoding is non-sparse of dimension Resolve remaining coefficients by inverting the sub-matrix of Phase 2 complexity = Key: choose Phase 2 complexity is

Compressive Sampling Signal is -sparse in basis/dictionary –WLOG assume sparsity in space domain Measure signal via few linear projections Random sparse measurements will work! measurements sparse signal nonzero entries

Signal model measurements sparse signal nonzero entries Signal is strictly sparse Every nonzero ~ continuous distribution  each nonzero coefficient is unique almost surely