Object Specific Compressed Sensing by minimizing a weighted L2-norm A. Mahalanobis

Background Lockheed Martin has been working on the DARPA ISP program  Team includes Duke, JHU, Yale and NAVAIR An adaptive sensing scheme has been developed that allocates sensor resources (spectral and spatial) based on relevant information content  Algorithms are currently working in a coded aperture hyperspectral imager hardware Compressed Sensing is a natural extension of this ISP concept

Motivation Can we create an efficient sensing process where objects of interest are well resolved, but other parts of the scene are heavily compressed?  Economize on number of data measurements required and the computations needed to reconstruct the image Currently, Compressed Sensing is focused on the general reconstruction problem  We are not interested in the perfect reconstruction of the whole scene Our approach embeds pattern recognition objectives (detection, discrimination) and compression in the sensing process, while producing visually meaningful images.

Approach It has been shown that under certain conditions, minimizing the L-1 Norm yields the optimum solution for perfect reconstruction, but the optimization requires iterative (potentially cumbersome) techniques L-2 norm techniques are well known, analytical closed form solutions that are easy to implement  Computationally attractive for the formation of large images  However, the minimum L2 norm solution does not yeild good reconstruction Can a weighted L2-norm arrive “close” to the optimum solution when we are interested in specific objects ?  How can we incorporate prior knowledge about the objects ?

The general solution Assume that the image vector y can be represented as linear combination of basis vectors (columns of the matrix A) such that  h is the coefficient vector we seek to estimate from a small number of measurements, and hence re-construct y In compressed sensing, we measure a smaller vector u, (i.e. the projection of the image y through a “random” mask W) The most general family of solution for the estimate h that satisfies the above linear constraints is  All solutions (including those which minimize the L-0, L-1 or L-2 norm) belong to this family  The particular solution is the “minimum L-2 norm” solution  The homogeneous solution can be viewed as a correction to the L-2 norm that results in other solutions with different properties Particular solutionHomogeneous Solution A random vector

Weighted L-2 norm Minimizing the L-2 norm does not relate to a well-defined “information” metric for reconstruction  It minimizes the variation in the estimate when white noise is present in the measurement Rather, we seek a weighting that minimizes the L-2 norm of the coefficient vector while maximizing information about the objects of interest  This results in attenuation of those weights which do not bear useful information for reconstruction  Or maximize  This implies that the best choice for the weights is We envision that can be calculated “apriori” from a set of representative images of the class of objects of interest, or a suitable statistical model may be used.

Solution using the methods of Lagrange multipliers Problem is stated as  Minimize the quadraticsubject to the linear constraints D is a diagonal matrix whose diagonal elements are calculated apriori from a set of representative images or a statistical model  The well known solution for the estimate of the coefficient vector is now h is estimate of the coefficients based on the measurements u A is a matrix that can be used as a basis to represent the image W is a random matrix on which the image is projected to obtain u D is a weight vector that maximizes information for the objects of interest

Reconstruction Equation The Reconstructed Image is given by where depends on the basis functions and the weights Without weights, R = I, and the solution does not depends on the underlying basis set  Minimum L-2 norm solution is then simply We will use i) DCT and ii) KL basis sets to demonstrate performance  For the KL basis set, D is the same as the eigen-values

Example using ideal weights Original image is 32 x 32 (1024 elements) DCT is used as a basis set  Any other basis set that allows compact representation can be used  ideal coefficients are used as a “place-holder” for weights In practice, these will be estimated representative images of the class of objects of interest, or statistically modeled. Weighted L2 norm produces recognizable results using 1/4 th the data (256 measurements)  Conventional L2 norm does not perform well K=256 mse=0.19 K=192 mse=0.25 K=64 mse=0.5 K=256 mse=0.86 Conventional L2 norm WEIGTED L2 norm Original 32 x 32 image

DCT Basis Set and Weights The DCT of the image shows good compaction properties.  Indicates it should be possible to achieve nearly zero mse with only 50% of the coefficients  Other basis sets should yield much greater compactness A (as a 2D image)

Example 2: weights estimated for a “class” The goal is to sense all objects that belong to a “class”  Exact weights for any one image is not known, but an average estimate for the class is used The average DCT is estimated using 1600 representative views and the inverse of the DCT coefficients is used weights in the reconstruction process Object DCT of Object Average DCT

Weighted vs. Conventional approach using DCT basis Comparison of conventional and weighted minimum L2 norm reconstruction using the DCT basis functions. Weighting the reconstruction process makes a significant difference in the reconstruction error

Reconstruction based on DCT with and without weights Reconstructions using 512 projections and the DCT basis set with weighting estimated over the class shows better performance than without weighting, i.e. the conventional minimum L2 norm solution (a) Weighted (b) Unweighted

Using the K-L Basis set The weights are the reciprocal of the square-root of the eigen-values of the auto- correlation matrix estimated using representative images of the class of vehicles of interest.  Only M=450 basis functions are necessary for accurately representing the images, which reduces the size of the matrix R and hence the overall computations

Weighted vs. Conventional approach using KL basis Reconstruction using the KL basis far out-performs DCT when weights are used  Performance of unweighted scheme is comparable to the unweighted DCT (not surprising)

Other Computational advantages of the KL set KL transform offers computational advantages in Two ways:  Fewer measurements are necessary (reduces the number of rows of R)  Fewer basis functions as required to represent the image (reduces the number of columns of R) Image on the left was reconstructed using the first 450 eigen-vectors of the KL decomposition, whereas all 1024 were used on the right. The two images are almost identical, although the image in (a) requires considerably less computations. (a) Esimated using 256 measurements and 450 eigen-vectors (b) Esimated using 256 measurements and All 1024 eigen-vectors

Example of full scene reconstruction (back to DCT) L2-norm approach easily reconstructs large scene  Computationally straightforward Weighted optimization clearly demonstrates ability to heavily compress uninteresting regions of the scene, while achieving reasonable reconstruction where true objects are present Original Image

Summary Minimizing the L-2 norm is a viable way of reconstructing objects of interest in a compressed sensing scheme  Requires prior knowledge of the weights that are representative of the class of objects Embeds attributes of pattern recognition in the sensing process to preserve visual detail for the human user, while effectively achieving detection, discrimination and compression Selection of basis set is important  Good basis sets require fewer measurements and fewer terms in the representation which speeds up the computations. The selection of basis sets and criterion for choosing weights both require further research