1. Reconstruction with Adaptive Feature-specific Imaging Jun Ke 1 and Mark A. Neifeld 1,2 1 Department of Electrical and Computer Engineering, 2 College of Optical Sciences University of Arizona Frontiers in Optics 2007

2. Outline Frontiers in Optics 2007  Motivation for FSI and adaptation.  Adaptive FSI using PCA/Hadamard features.  Adaptive FSI in noise.  Conclusion.

3. Motivation - FSI Reconstruction with Feature-specific Imaging (FSI) : Frontiers in Optics 2007 FSI benefits:  Lower hardware complexity  Smaller equipment size/weight  Higher measurement SNR  High data acquisition rate  Lower operation bandwidth  Less power consumption Reconstruction matrix M (nxm) object object estimate DMD Imaging optics single detector feature Sequential architecture: Parallel architecture: LCD M (nxm)

4. Motivation - Adaptation Frontiers in Optics 2007  Acquire feature measurements sequentially  Use acquired feature measurements and training data to adapt the next projection vector  The design of projection vector effects reconstruction quality. Estimation Poorly designed projection vectors Testing sample Training samples Projection axis 2 Static PCA Projection axis 1  Using PCA projection as example Well designed projection vectors Adaptive PCA Estimation Projection axis 2 Projection value Training samples for 2 nd projection vector Projection axis 1

5. Frontiers in Optics 2007 Object estimate y i = f i T x Calculate f i+1 Reconstruction Object x Update A i to A i+1 according to y i Computational Optics Calculate f 1 R i+1 Calculate R 1 from A 1 Adaptive FSI (AFSI) – PCA: i: adaptive step index A i : i th training set R i : autocorrelation matrix of A i f i : dominate eigenvector of A i y i : feature value measured by f i  High diversity of training data helps adaptation PCA-Based AFSI Testing sample K (1) nearest samples Projection axis Testing sample K (1) nearest samples Selected samples According to 1 st feature According to 2 nd feature K (2) nearest samples Projection axis 2 Projection axis 1

6. Object examples (32x32):  Reconstructed object:  RMSE:  Feature measurements: where,  is the total # of features PCA-Based AFSI Frontiers in Optics 2007  Number of training objects: 100,000  Number of testing objects: 60

7.  RMSE reduces using more features  RMSE reduces using AFSI compare to static FSI  Improvement is larger for high diversity data  RMSE improvement is 33% and 16% for high and low diversity training data, when M = 250. Frontiers in Optics 2007 AFSI – PCA: PCA-Based AFSI Reconstruction from static FSI (M = 100) Reconstruction from AFSI (M = 100) K increases

8.  Projection vector’s implementation order is adapted. Frontiers in Optics 2007 x i (mean) : average vector of A i f i : dominant Hadamard vector for A i AFSI – Hadamard: Hadamard-Based AFSI Object estimate y iL+j = f iL+j T x (j=1,…,L) Choose f iL+1 ~ f (i+1)L Reconstruction Object x Update A i to A i+1 according to y iL+j Computational Optics Choose f 1 ~f L x i+1 (mean) x 1 (mean) Sort Hadamard bases Selected samples K (1) nearest samples testing sample projection axis 1 K (1) nearest samples testing sample projection axis 2 K(2) nearest samples sample mean First 5 Hadamard basis ←Static FSI AFSI→ according to 1st feature according to 2nd feature sample mean projection axis 1

9.  RMSE reduces in AFSI compared with static FSI  RMSE improvement is 32% and 18% for high and low diversity training data, when M = 500 and L = 10.  AFSI has smaller RMSE using small L when M is also small  AFSI has smaller RMSE using large L when M is also large Hadamard-Based AFSI Frontiers in Optics 2007 AFSI – Hadamard: Reconstruction from adaptive FSI Reconstruction from static FSI K increases L decreases L increases

10. Hadamard-Based AFSI – Noise Frontiers in Optics 2007 AFSI – Hadamard:  Hadmard projection is used because of its good reconstruction performance  Feature measurements are de-noised before used in adaptation  Auto-correlation matrix is updated in each adaptation step  Wiener operator is used for object reconstruction Object estimate y iL+j = f iL+j T x+n iL+j (j = 1,2,…L) Choose f iL+1 ~f (i+1)L Reconstruction Object x Update A i to A i+1 according to Computational Optics Choose f 1 ~f L x i+1 (mean) Calculate x 1 (mean) from de- noising y iL+j Calculate R i for A i  T : integration time  σ 0 2 = 1 Sort Hadamard bases Sort

11. Frontiers in Optics 2007  RMSE in AFSI is smaller than in static FSI  RMSE is reduced further by modifying R x in each adaptation step  RMSE improvement is larger using small L when M is also small  RMSE is small using large L when M is also large Hadamard-Based AFSI – Noise High diversity training data; σ 0 2 = 1  T : integration time/per feature  σ 0 2 = 1  detector noise variance σ 2 = σ 0 2 /T High diversity training data; σ 0 2 = 1 K increases L decreases L increases

12. T : integration time/per feature; M 0 : the number of features Total feature collection time = T × M 0  RMSE reduces as T increases  High reconstruction quality requirement needs longer total feature collection time  To achieve each RMSE requirement, there is a minimum total feature collection time. Hadamard-Based AFSI – Noise Frontiers in Optics 2007 High diversity training data; σ 0 2 = 1

13. Conclusion Frontiers in Optics 2007 Noise free measurements:  PCA-based and Hadmard-based AFSI system are presented  AFSI system presents lower RMSE than static FSI system Noisy measurements:  Hadamard-based AFSI system in noise is presented  AFSI system presents smaller RMSE than static FSI system  There is a minimum total feature collection time to achieve a reconstruction quality requirement