1. GENIE: Automated Feature Extraction for Pathology Applications Neal R. Harvey Kim Edlund Los Alamos National Laboratory harve/kedlund@lanl.gov

2. Acknowledgements: We should like to thank the following for providing their medical expertise, data and some results shown during this presentation: Dr. Richard Levenson, CRI inc. Dr. David Rimm, Yale University Dr. Carola Zalles, Yale University Dr. Cesar Angeletti (formerly of Yale University)

3. So much Data, So Little Information  Satellite-based and other instrumentation today produces unprecedented quantities of raw image and signal data.  Hidden in this data is information of interest to analysts and scientists.  How can this information be extracted: Easily Rapidly Reliably

4. So much Data, So Little Information  Microscope cameras, slide scanners and other instrumentation today produces unprecedented quantities of raw image data.  Hidden in this data is information of interest to pathologists, other medics and scientists.  How can this information be extracted: Easily Rapidly Reliably

5. Traditional Approach Physical Modeling

6. GENIE: Machine Learning Easier to show a machine what to find…...than to tell a machine how to find it GENIE automatically generates an algorithm for future use Train Exploit

7. Evolving Solutions GENIE is an Adaptive System : –It derives a general purpose image classifier from a limited set of user-supplied examples. –It uses a hybrid genetic algorithm, combining evolutionary exploration with statistical machine learning.

8. Issues in Pixel Classification Spectral information often inadequate. Need to make use of textural and spatial context cues. Many, many ways of describing/encoding such spatial context information. Best techniques are task-specific. How do we do learn to map pixels to categories in general?

9. The GENIE Approach Give GENIE a large and flexible “toolbox” of image processing algorithms. Use an evolutionary algorithm to explore which tools are most appropriate for the current task. Use statistical machine learning to learn how to combine those tools together to give an accurate classification.

10. GENIE Development  1999: Initial funding from two NRO DII’s  Continued research funding from LANL, DOE and others  2002: R&D 100 Award Genie Pro  2003: Transition to NGA funding for operational version: Genie Pro.  2004: Genie Pro wins NGA Feature Extraction Evaluation (“bake-off”)

11. GENIE and Pathology? Initial experiments in applying GENIE to bio- medical data –Apply GENIE “as is” on multi-spectral pathology data –i.e. make no modifications to/customization of GENIE for the pathology field

13. GENIE and Colon Cancer Detection H & E Stained Colon Tissue (Cancer & Normal) GENIE Classification (cancer vs normal)

14. GENIE and colon cancer detection (Training) True color image Colon: containing cancer and normal tissue Training data Green: cancerous nuclei Red: everything else (i.e. not cancerous nuclei) True color image Colon: containing only normal tissue Training data Green: none because no cancerous nuclei Red: everything else (i.e. not cancerous nuclei)

15. GENIE and colon cancer detection (Exploitation) GENIE Result: Cancer (Training Data) GENIE Result: Normal (Training Data) GENIE Result: Cancer (Testing Data) GENIE Result: Normal (Testing Data)

16. GENIE: Breast Cancer Detection (cancerous nuclei) - Training Data Training Data: Cancer Training Data: Normal

17. GENIE: Breast Cancer Detection (Cancerous Nuclei) – Results for training Data Classification Results: Cancer Classification Results: Normal

18. GENIE: Breast Cancer Detection (Cancerous Nuclei) – Results for testing data (cancer)

19. GENIE: Breast Cancer Detection (Cancerous Nuclei) – Results for testing data (normal)

20. GENIE and endometrial gland detection (training data) True color image Training data Green: gland boundary Red: everything else True color image Training data Green:gland boundary Red: everything else

21. GENIE endometrium gland detection: exploitation over training data

22. GENIE endometrium gland detection: exploitation over testing data

23. GENIE and kidney inflammation detection (training) True color image Training data Green: inflammation Red: everything else Training result Green:inflammation Red: everything else

24. GENIE and kidney inflammation detection (testing) True color image Testing result Green:inflammation Red: everything else

25. GENIE and Other Bio-Medical Applications Vibrational Hyperspectral Imaging –Fluorescence imaging –FTIR (Fourier Transform Infra Red) imaging –Raman spectroscopy –CARS (Coherent Anti-Stokes Raman Scattering) Can exploit specific molecular signatures in vibrational spectrum

26. GENIE application to VHI Hyperspectral fluorescence image of bacteria (E. Coli) bio-engineered to express GFP (green fluorescent protein), added to sample of macrophages stained to reveal ROS (reactive oxygen species). Task set GENIE – find E. Coli that had been taken up (engulfed) by the macrophages. Training data provided to GENIE GENIE classification result

27. GENIE: Urine Cytology Classification

28. GENIE Results: Cover of Laboratory Investigation “When tested on urothelial cytology specimens collected at two separate institutions over a span of 4 years, GENIE showed a combined sensitivity and specificity of 85 and 95%, respectively. Of particular note is that when ‘training’ was performed on cases initially diagnosed as ‘equivocal’ on cytology but with follow-up biopsy, surgical specimen or cytology, which was unequivocally benign or malignant, GENIE was superior to the cytopathologist interpreting the initial ‘equivocal’ cytology specimen.”

29. Genie Pro Commercialization Genie Pro has been exclusively licensed to Aperio For all digital pathology applications