1. PIBM Sept 05 Andy Harvey 1 Spectral Imaging of the Retina Andy R.Harvey, Ied Abboud, Alistair Gorman, Andy I.McNaught * School of Engineering & Physical Sciences, Heriot Watt University, Edinburgh, UK *Eye Unit, Cheltenham General Hospital, Cheltenham, UK

2. PIBM Sept 05 Andy Harvey 2 Outline What is spectral imaging? Spectral retinal imaging Why? Spectral time-sequential spectral imaging For flexibility and research 2D snapshot For real-time, high throughput screening Conclusions

3. PIBM Sept 05 Andy Harvey 3 Conventional Spectral Imaging RGB Image Spectrally classified image Dysplastic cell Superficial squamous cell Intermediate cell Lymphocyte PMN Courtesy CRI Classification spectra

4. PIBM Sept 05 Andy Harvey 4 Imaging the eye Sclera Cornea Iris Lens Retina Macula Vitreous humour Light Anterior chamber (full of aqueous humour) Optic nerve Posterior chamber Choroid Optic disc

5. PIBM Sept 05 Andy Harvey 5 The Role of Spectral Retinal Imaging By 2020 there will be 200 million visually- impaired people world wide Glaucoma, diabetic retinopathy, ARMD 80% of those cases are preventable or treatable Screening and early detection are crucial Can spectral imaging offer enhancements to current screening techniques ? Spectral imaging is non-invasive and safe cf. fluorescein angiogram Spectral imaging can enable imaging of Retinal biochemistry Blood oximetry Diabetic retinopathy, glaucoma Lipofuscin etc Age-related macula degeneration

6. PIBM Sept 05 Andy Harvey 6 Spectral Imaging: Traditional approaches And Fourier-transform equivalents N (t) NxNx NyNy N NxNx N y (t) Time-sequential spectral multiplex Time-sequential spatial multiplex Limitations Optically inefficient 2D time-varying scenes 2D snapshot required for: Retinal imaging in vitro, in vivo imaging

7. PIBM Sept 05 Andy Harvey 7 Spectral Fundus Camera Source filtering by LCTF incorporated into COTS fundus camera 10 nm spectral width 20 msec random access Images captured using a cooled, low-noise CCD camera

8. PIBM Sept 05 Andy Harvey 8 Isobestic point Coregistered Spectral Images of a Healthy Retina Images translationally and rotationally coregistered

9. PIBM Sept 05 Andy Harvey 9 Spectral angle map of healthy and diabetic retina Shading indicates similarity of each pixel spectrum with artery and vein spectra Qualitative oxymetry Normal Retina Diabetic Retina

10. PIBM Sept 05 Andy Harvey 10 Supervised spectral classifiaction Implicit calibration based on spectral signatures within the eye Classification possible without absolute calibration Clear distinction between veins/arteries, on/off optic disc Spectra depends on local environment Inversion of data to calculate biochemical concentrations (eg oxygenation) requires a model of light propagation and scattering in the retina to remove environmental effects Monte Carlo, Kubelka Monk, Transfer equation

11. PIBM Sept 05 Andy Harvey 11 Requirements for a snapshot technique: retinal imaging Improved calibration Patient patience Remove imperfect coregistration due to Variations in imaging distortion between images Similar issues with other in vivo applications Imaging internal epithelial cancers Eg gastrointestinal PC15

12. PIBM Sept 05 Andy Harvey 12 Image Replication Imaging Spectrometer: IRIS Snapshot image zero temporal misregistration ‘100%’ optical efficiency World’s only snapshot, 2D spectral imager (almost !) Conceptually related to Lyot filter Large format detector Spectral Demultiplexor

13. PIBM Sept 05 Andy Harvey 13 Lyot filter: principle of operation Polariser Waveplate

14. PIBM Sept 05 Andy Harvey 14 Wollaston prism polarisers replicate images Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses All possible products of spectral responses are formed at detector IRIS snapshot spectral imager: principle of operation

15. PIBM Sept 05 Andy Harvey 15 Spectral transmission cos 2  sin 2  cos 2 (  cos 2 (2  ) sin 2 (  cos 2 (2  ) cos 2 (  sin 2 (2  ) sin 2 (  sin 2 (2  ) cos 2 (  cos 2 (2  )cos 2 (4  ) cos 2 (  sin 2 (2  )cos 2 (4  ) cos 2 (  cos 2 (2  )sin 2 (4  )cos 2 (  sin 2 (  )sin 2 (4  ) sin 2 (  cos 2 (2  )cos 2 (4  )sin 2 (  sin 2 (2  )cos 2 (4  ) sin 2 (  cos 2 (2  )sin 2 (4  )sin 2 (  sin 2 (  )sin 2 (4  ) Wollaston/waveplate assembly

16. PIBM Sept 05 Andy Harvey 16 Spectral responses Bands are overlapping bell shapes Choose cost function to minimise sidelobes Small (~5%) reduction in spectral separation Cut-off filters used to define spectral range 8 channel visible-band system 520nm  820m 3 Quartz retarders 32 channel, visible-band system 520nm  720nm 5 Quartz retarders

17. PIBM Sept 05 Andy Harvey 17 Optical scaling laws Hamamatsu ORCA-ER Inputs: FoV Sub image size on CCD CCD pixel size Primary lens magnification & F# Collimating lens back focal distance, focal length & front element diameter Prism birefringence Outputs: Field stop size Collimating lens rear element diameter Splitting angles, apertures & depths of prisms Apertures of retarders, polarisers and filters Imaging lens focal length & front element diameter Field stop Collimating lens Bandpass filter Imaging lens Camera Polariser, retarders & Wollaston prisms (index matched) Primary lens

18. PIBM Sept 05 Andy Harvey 18 Components & Assembly 8 channel system 520nm to 820nm 3 Quartz retarders 3 Calcite Wollaston prisms

19. PIBM Sept 05 Andy Harvey 19 Spectral Retinal Imaging Difficult imaging conditions render application of traditional HSI techniques problematic IRIS enables real-time and snapshot spectral imaging Canon

20. PIBM Sept 05 Andy Harvey 20 Blood oximetry Optimal spectral band for retinal oximetry Vessel thickness ~ optical depth 570-615 nm Eight bands approximately equally spaced 40 20 80

21. PIBM Sept 05 Andy Harvey 21 Video sequence recorded with bandpass filtered inspection lamp

22. PIBM Sept 05 Andy Harvey 22 Retinal image recorded with flash illumination

23. PIBM Sept 05 Andy Harvey 23 574581 585 592595 603 607 613 Coregistered and PCA images PC1PC2PC1 & PC2

24. PIBM Sept 05 Andy Harvey 24 Summary Spectral imaging of the retina shows promise for non-invasive detection of retinal disease Clinical trials on-going LCTF-based, time-sequential spectral filtering enables rapid and flexible 2D spectral retinal imaging Flexible data acquisition Pulse and other motion artefacts limit accuracy Snapshot spectral imaging in 2D (IRIS) promises high-performance real-time multi-spectral imaging Ideal for in vivo imaging No temporal misregistration Absolute, quantitative data requires a model of light interaction within the retina

25. PIBM Sept 05 Andy Harvey 25 Wollaston prism polarisers replicate images Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses All possible products of spectral responses are formed at detector IRIS snapshot spectral imager

26. PIBM Sept 05 Andy Harvey 26 Measured & predicted spectral responses

27. PIBM Sept 05 Andy Harvey 27 Absolute total transmission Bandpass filter & polariser dominate losses Improved system: T>80% Theoretical throughput is 2 n times higher than for spatial/spectral multiplexed techniques! 0 25 50 Response (%) Absolute response curves in polarised light

28. PIBM Sept 05 Andy Harvey 28 Application to microscopy: Imaging of multiple fluorophors IRIS fitted to conventional epi-fluorescence microscope Germinating spores of Neurospora crassa stained with GFP – nucleii fluoresce at 510 nm FM4-64 – membranes fluoresce at >580 nm 0 2525 5050 Response (%)

29. PIBM Sept 05 Andy Harvey 29 Principle component decomposition PC1 PC15 Artery structure is a pulse artefact Very difficult to co- register by image processing means Snapshot technique desirable PC3

30. PIBM Sept 05 Andy Harvey 30 Conclusions IRIS is a new spectral imaging technique that enables snapshot spectral imaging in 2D No rejection of light No data inversion Highest-possible signal-to-noise ratios Simple logistics Inherently compact and robust Simply fitted to conventional imaging systems Birefringent materials exist for applications from 0.2  m to 12  m Applications In vivo, in vitro imaging Retinal imaging Microscopy Multiple fluorophors Quantum dots Surveillance Remote sensing Etc.

31. PIBM Sept 05 Andy Harvey 31 Optical depth of Hb & HbO 2 dominates variation of penetration with Tissues vary between highly turbid and transparent Blue light images retinal surface Light at ~600 nm enables spectral oximetry within retinal blood vessels optical depth of HbO 2 > vessel thickness so vessels translucent optical depth of Hb < vessel thickness so vessels are opaque Light > 640 nm penetrates to coroid BlueGreenRed Isobestic point Spectral Characteristics of the Retina

32. PIBM Sept 05 Andy Harvey 32 Issues for Spectral Retinal Imaging Calibration Components of interest within a complex turbid medium Patient tolerance Using current technology, time-sequential spectral bandpass offers Optimal SNR Reduced light intensity at the retina Agile selection of spectral bands (data efficient) Issues Coregistration Calibration ± 100 pixels ±2º±2º Spectral imaging of static scenes is relatively ‘easy’ Spectral imaging of the retina encounters Imaging through an erratically moving, low-quality f/6 eye-lens system Solutions: 2D snapshot spectral imaging

33. PIBM Sept 05 Andy Harvey 33 The End

34. PIBM Sept 05 Andy Harvey 34 1D image x path difference  Fixed mirror Scanning mirror Detector array NN NxNx N y (t) NN NxNx FT N (t) NxNx NyNy N NxNx N y (t) Direct Imaging Spectrometry(Fourier) Transform Imaging Spectrometry Temporally scanned Snapshot/fully staring N  (t) NxNx NyNy FT NN NxNx NyNy

35. PIBM Sept 05 Andy Harvey 35 Why another spectral imaging technique? Traditional approaches Time sequential spectral multiplex Monochromatic two-dimensional image in snapshot Time sequential spatial multiplex One-dimensional spectral image in a snapshot (and Fourier-transform equivalents) Problems Cannot record two-dimensional spectral images of time-varying scenes Optically inefficient Time-resolved (snapshot) spectral imaging is required for Dynamic scenes In vitro, in vivo imaging and microsocopy Combustion dynamics, surveillance… Irregular motion between scene and imager In vivo imaging Ophthalmology Remote sensing, airborne surveillance, industrial inspection…