0. Practical Spectral Photography
Ralf Habel1
Michael Kudenov2
Michael Wimmer1
Institute of Computer Graphics and Algorithms Vienna University of Technology1
Optical Detection Lab University of Arizona2

1. Motivation
Spectroscopy is most important analysis tool in all natural sciences
Astrophysics, chemical/material sciences, biomedicine, geophysics,…
Industry applications:
Mining, airborne sensing, QA,…
In computer graphics:
Colors
Material reflectance
Spectral/predictive rendering …
Ralf Habel

2. Spectral Imaging Records image at narrow wavelength bands
In visible range not only RGB (3 channels) but many more (6-400 channels)
Result: 3D data cube
2 spatial image axis
1 wavelength axis
Ralf Habel

3. Spectral Imaging Usually done with highly specialized devices
Many methods to build devices
Scanning slits, rotating mirrors, special sensor, filters, prisms, …
Usually scan along one of the data cube axis
All very costly due to opto-mechanical components
“Simplest” spectral imager:
Camera + band filters
Requires switching of filters
Limited in number of bands
Ralf Habel

4. Motivation
Why not use consumer cameras and equipment for spectral imaging?
High quality, very sensitive
Highly accurate lenses
Practical Constraints:
No camera modification
No lab/desktop/optical bench setup
No expensive components
Ralf Habel

5. CTIS Principle Computed Tomography Image Spectrometer
Diffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):
Ralf Habel

6. CTIS Principle Computed Tomography Image Spectrometer
Diffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):
Ralf Habel

7. CTIS Principle Sensor records projections of 3D data cube
All information needed is recorded in one image
“Snapshot” spectrometry
Challenge is to reconstruct 3D data cube from projections
Tomographic rec. with Expectation Maximization
More details in paper
Ralf Habel

8. CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Ralf Habel

9. CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Ralf Habel

10. CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Ralf Habel

11. CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Re-imaging lens focuses on sensor
Ralf Habel

12. CTIS Optical Path
Imaging lens + square/slit aperture creates virtual image
Collimating lens makes light parallel
Diffraction grating creates projections
Re-imaging lens focuses on sensor
Ralf Habel

13. CTIS Optical Path Built with: Drain pipe & duct tape
50mm, 17-40mm and macro lens
Diffraction gel ($2 per sheet) in gel holder
Ralf Habel

14. CTIS Camera Objective
Ralf Habel

15. CTIS Camera Objective
Ralf Habel

16. HDR Image Acquisition No overexposed pixels allowed
Projections (diffractions) weaker than center image
Avoids noisy signal where camera response is weak
Ralf Habel

17. Spatial Wavelength Calibration
Mapping from 3D data cube into projections
Laser pointers (red, green and blue) with known wavelengths shot through a diffusor and pinhole
Monochromatic point light source
Pictures of pinhole give mapping of one voxel in 3D data cube
All other projections values interpolated/extrapolated
Ralf Habel

18. CTIS Principle
Ralf Habel

19. Spatial Wavelength Calibration
Ralf Habel

20. Spectral Response Calibration
Spectral response of the diffraction grating + RGB sensor for red, green and blue
Picture of light source with continuous known spectrum
We use calibrated halogen lamp
Ralf Habel

21. Spectral Photography Results
Take HDR picture with CTIS camera objective
Reconstruct 3D data cube for red, green and blue image color channels
Mapping from spatial calibration
Combine RGB spectral response of each pixel to true spectrum with spectral de-mosaicking
Mapping from spectral response calibration
Ralf Habel

22. Spectral Photography Results
Protoype data cube resolutions: 120x120 pixels 4.59 nm (54 channels)
Accuracy reduced in high blue and low reds due to color filters
Slight Expectation Maximization reconstruction artifacts
Nowhere near possible optimum!
Ralf Habel

23. Spectral Photography Results
Ralf Habel

24. Spectral Photography Results
Ralf Habel

25. Future Better CTIS objective Increase data cube resolution/accuracy:
Drain pipes and duct tape have their limits…
Optimized optical path and components
More compact/integrated device
Increase data cube resolution/accuracy:
Structured aperture
Digital holography – form diffraction/projections in any way
Better solutions to tomographic reconstruction
Is active research in optics
No vision based approach yet!
Ralf Habel

26. Future
Turning mobile devices into spectrometers - consumer spectroscopy?
8 MP high sensitivity sensors
HDR capabilities
Very low cost!
“Snapshot” capability: Spectral movies with consumer cameras?
Not only good for computer graphics:
Blood sample analysis
Water contamination analysis
As part of a TricorderTM
Ralf Habel

27. Practical Spectral Photography
Thank You!
Ralf Habel