1. Automated lymphocyte counting in tissue microarrays using the Nuance/Vectra/inForm imaging system Ian Hagemann, MD, PhD Cliff Hoyt, MS Mike Feldman, MD, PhD

2. Tumor-infiltrating lymphocytes (TILs) in ovarian cancer Ovarian cancer may be recognized and attacked by the immune system Tumor may contain a lymphocytic infiltrate TILs exhibit oligoclonal expansion, recognize tumor antigens, circulate in vivo, and display tumor-specific cytolytic activity in vitro Clinical results have been seen with interferon or adoptive T cell immunotherapy Zhang L, NEJM 2003

4. The question How many lymphocytes are present in this tumor? – Intraepithelial – Stromal Alternate phrasing: how densely is this tumor infiltrated by lymphocytes? The problem Ambiguous histology – Limited tissue – Hematoxylin only Variable surface area of core, tumor, and stroma Human factors – Difficult to count numerous events – Boredom

5. Vectra system (CRI, Inc.) Multispectral brightfield and fluorescent slide imaging (Nuance) Pattern recognition- based, partially automated scanning (Vectra) Automated tissue and cell segmentation (inForm)

6. Imaging a TMA using Vectra Input: Stained TMA slide Output: Hundreds of multispectral image files indexed by grid location.

7. Input Training regions for tissue segmenter Output of tissue segmenterOutput of tissue and cell segmenter

8. Review classified images Some histospots will have been classified incorrectly – Core fell off or folded over – Unsuitable tissue – Tumor interpreted as stroma, or vice versa – Lymphocyte over- or undercounting Task: visually review each core for appropriate segmentation – Despite sophisticated segmentation algorithms, this step (performed by a human) appears to be essential

9. Tales of woe

10. Segmentation algorithms fail on some fraction of histospots Total histospots evaluated618 Pre-algorithmic failures Spot fell off Unsuitable tissue (e.g., colon or fat only) 37 77 Tissue segmentation failures Tumor interpreted as stroma Stroma interpreted as tumor 26 49 Cell segmentation failures Overdetection of lymphocytes Underdetection of lymphocytes 9393 Spots successfully segmented436

11. Manual and automated TIL scores are significantly correlated r=0.54 (95% CI, 0.47–0.61)r=0.68 (95% CI, 0.61–0.74) p<0.0001

12. Simulated perfect concordance between manual and automated TIL counts

13. Observations and conclusions Automated event scoring provides a consistent approach to tedious, poorly reproducible tasks. Histology scoring tasks can probably never be completely automated. Automated lymphocyte counts are significantly correlated with manual counts. Gold-standard performance for this task is undefined (and probably impossible to define)

14. Future directions Improved machine learning and classification algorithms will shrink the group of segmentation failures (never to zero) Greater leveraging of multispectral technology may allow a qualitative leap forward in the depth of tissue annotation (e.g., “tumor mask” staining by cytokeratin) An integrated TMA-aware workflow would reduce manual steps (cut and paste) and increase throughput Quantitative direct feature counting can inform semi- quantitative analyses (e.g., where to set cutoffs?)

15. Acknowledgments UPENN Mike Feldman, MD, PhD Tim Baradet, PhD George Coukos, MD, PhD Andrea Hagemann, MD CRi, Inc. Cliff Hoyt, MS Craig Lassy, PhD ian.hagemann@uphs.upenn.edu