1. Tutorial on Computational Optical Imaging University of Minnesota 19-23 September David J. Brady Duke University www.disp.duke.edu

2. Lectures 1.Computational Imaging 2.Geometric Optics and Tomography 3.Fresnel Diffraction 4.Holography 5.Lenses, Imaging and MTF 6.Wavefront Coding 7.Interferometry and the van Cittert Zernike Theorem 8.Optical coherence tomography and modal analysis 9.Spectra, coherence and polarization 10.Computational spectroscopy and imaging

3. Lecture 10. Computational Spectroscopy and Imaging Compressive Optical Montage Photography Coded Aperture Spectroscopy

4. 2. Multiaperture systems for imaging system miniaturization Prototype COMP-I visible Imager

5. Image reconstruction from subpixel shifts 4 Images taken by 2 by 2 array

6. Measurement Model

7. Inference Strategy

8. Reconstruction Best raw image Synthesized image

9. Reconstruction Zooms

10. Shot of COSI meeting One of 4 low res images Reconstructed image Zoom on speaker

11. 3. Compressive Sampling Conventional imaging COMP-I imaging

12. Compressive Sampling Conventional imaging COMP-I imaging

13. Pixel Block Coding

14. Quantized Cosine transform Block-based transform maintains compressive property of DCT Arithmetic complexity O(n log n) Values from {-1,0,1} amenable to mask implementations

15. 8x8 DCT

16. 8x8 meanQCT

17. 8x8 signQCT

18. MTF and Registration Image shift coding produces fundamentally bandlimited images Focal plane coding can recover high resolution images Registration may be addressed by PSF coding

19. Characterization of sub-pixel optical PSF Approximately 4x undersampled Green: MTF from sampling grid Blue: FT of opt psf + grid Red: fit to MTF based on 3  m opt psf 1.8mm focal length lens, 3  m psf width, 5.6  m pixels Scan showing pixel-pixel progression of impulse Pixel number time

20. Demonstration of sub-pixel PSF in IR System Approximately 5x undersampled Green: MTF from sampling grid Blue: FT of ir psf + grid 25mm focal length lens, 13  m psf width, 30  m pixels

21. imagesc(horiz,[-1000 6000])imagesc(zero,[-1000 5000]) imagesc(vert,[-1000 6000])imagesc(quad,[-1000 2000]) Diffractive PSF coding for nondegenerate measurement on IR system

22. 80100120 30 40 50 60 70 1.3 1.4 1.5 x 10 4 80100120 30 40 50 60 70 1.5 2 2.5 3 3.5 x 10 4 80100 120 30 40 50 60 70 1.4 1.6 1.8 x 10 4 80100120 30 40 50 60 70 1.4 1.6 1.8 x 10 4 Performance of the Fabricated Device

23. 8.9  m Bandpass Filter, 650ºC Blackbody Testing of Delware diffractive coder

24. Focal Plane Coding Sony CCD: Pixel size: 5.6 micron by 5.6 micron Pixel number is: 480 by 600 Pixel Mask on CCD/CMOS Sensors

25. Experiment Setup 15 micron pin hole at ~ 2 meter away Wavelength: 0.6328 micron Lens: NA= 0.5

26. Aperture Pattern One Pitch is matched to the pixel size of 5.6 micron 4 micron2 micron3 micron1 micron

27. Scan Crossing Apertures

28. Characterization of Lumina CMOS Sensor Pixel Size: 5.2 X 5.2 micron square Number of Pixels: 1024 X 1280 Focus light on a Single Pixel

29. Pixel Scan (without mask) in X and Y direction

30. Sub-pixel Response of the CMOS Sensors (with the mask)

31. Coded aperture spectroscopy Focal plane coding microspectrometer using Hadamard codes, 0.5 nm resolution in a 5 cubic centimeter volume Coding technology and algorithms very similar to COMP-I imaging algorithms Advantages as spectrometer include very compact volume, high throughput

32. Data Recorded with Coded Aperture Mask For a single grating, system records spectrum convolved with coded mask on CCD K-fold multiplex grating records sum of K spectra convolved with coded mask on CCD

33. System Details 640x480 Firewire Webcam Relay Lenses Multiplex Hologram Coded Aperture Mask 1000 Spectral Channels between 400 – 700nm

34. Spectral reconstruction Ocean Optics Spectrum DMS Spectrum

35. DISP Tissue Spectrometer

36. Raman Spectroscopy Raman spectra uniquely identify molecules by revealing information about ro-vibrational level spacing IR Raman spectroscopy is very useful for biomedical applications because of the “diagnostic window”, an IR region in which many biological tissues have low absorption Raman signals are very weak and spatially incoherent Shown to be effective in analyzing blood components in ex vivo samples

37. Light Scattering in Tissue Satisfies diffusion equation, in the scattering- dominated limit ( κ d =1 / diffusion length) Spot sizes become very large (2-4 mm) in tissue creating difficulty in coupling light to conventional instruments T. Vo-Dinh, Biomedical photonics handbook, CRC Press, Boca Raton, Fla., 2003.

38. Simulation of Photon Distribution for 800-850 nm Light in Tissue and Blood

39. Throughput- Resolution Tradeoff Narrow slit to achieve high spectral resolution on detector Detector Plane Dispersion Element Detector Plane Dispersion Element Increasing slit width creates ambiguity on detector plane

40. Coded Aperture Multiplexing Use coded aperture to measure combinations of spectral channels on each detector, preserving resolution and increasing throughput Detector Plane Dispersion Element Use multiple rows of detector in order to measure a family of spectral combinations, leading to a well conditioned matrix inversion

41. Reasons for Coded Aperture Fully incoherent sources can not be locally increased in intensity (constant radiance theorem) D. J. Brady, “Multiplex sensors and the constant radiance theorem," Optics Letters 27(1), pp. 16-18, 2002. Turbid samples typical in biological systems Spatial filtering typically done through a slit to eliminate spatio-spectral ambiguity Source Optic “Focused” Source

42. Common Solution Structured fiber bundle to go from circular aperture to rectangular aperture Our prototype system- 36 um apertures, 4x2 mm area with 50% throughput Source can be magnified to increase angular acceptance and reduce interrogation area For 36 um slit, 4 mm 2 becomes 36 um x 111 mm High performance scientific cameras ~8 x 8 mm sensor, would require 14 cameras!!!

43. Excitation source: 8 ~808nm laser diodes providing multi-wavelength excitation for fluorescence rejection and noise reduction Mask: 64 row aperture mask based on N=32 Hadamard matrix Optical system: High-throughput, low- distortion optical system Sensor: Andor CCD camera with high QE, low noise, 16-bit digitization Software: Custom algorithms for spectral reconstruction and fluorescence rejection DISP Raman Spectrometer

44. Optical Design

45. Data Inversion Process

46. Molecular spectral model 1.a sufficient “dictionary” S of analytes spectra is calibrated 2.any mixture with concentration vector c has a Raman spectrum r

47. Challenge Estimation of ethanol concentration in the presence of multiple analytes of unknown concentrations

48. Methodology Linear multivariate calibration enables extraction of concentration from observed spectra Calculate a regression vector w, such that a weighted linear combination of the measured spectral intensities estimates the concentration Each element of w reveals the contribution of that spectral region Different w vectors may estimate the concentration of different analytes from the same spectrum

49. The problem statement Given a set of training spectra A of known ethanol concentrations c Find weights vector w such that

50. Singular Value Decomposition

51. Different algorithms Least squares Principal component regression Partial least squares Hybrid linear analysis

52. Cross validation calibration of training data Leave-one-out cross validation determines the rank of the approximation used for the LS solution

53. Ethanol Tests Sample solution –tissue-like lipid (20%) based in soybean oil –3 trials @ 4 min each –30 mW laser power (ANSI maximal permissible exposure for tissue at 808 nm) Eleven concentrations –20%, 10%, 5%, 2%, 1%, – 0.5%, 0.2%, 0.1% – 0.05%, 0.02%, 0.01% Two different sets of testing Ten single and dual laser excitations

54. LD2 cross validation

55. LD6 cross validation

56. LD8 cross validation

57. LD7 cross validation

58. 2 Lasers

60. 4 Lasers

62. Blood Analysis Tests –3 trials @ 4 min each –30 mW laser power (ANSI maximal permissible exposure for tissue at 808 nm) Analytes –Hemoglobin –Hematocrit –Glucose –Urea Ten single and dual laser excitations

63. Hemoglobin

64. Hematocrit

65. Glucose

66. Urea

67. New algorithmic approaches Bypass spectrum reconstruction altogether, using “economy” SVD decomposition –Raw CCD data –Random linear combinations and projections Pure ethanol spectra in HLA Non-linear estimators Classification algorithms

68. Hybrid linear analysis Let T be the dictionary with all spectra but the ethanol spectrum We would like to write where

69. Current work Data collection and manipulation –database software for access and validation –large volume of data –data must span over a period of time Consistency –correct for laser wavelength drift power variation Preprocessing strategies –scaling and normalization

70. Conclusions Coded aperture spectroscopy shows to be an effective method to measure the Raman spectra of ethanol in diffuse, fluorescent, media Such systems show promise for making systems with a combination of smaller excitation powers, shorter exposure times, and/or less costly components

71. Acknowledgements This work is supported by DARPA, AFOSR and the National Institute on Alcoholism and Alcohol Abuse Thanks to the members of DISP

73. optical imaging is a field that n also is ripe for mathematical attention