Automated lymphocyte counting in tissue microarrays using the Nuance/Vectra/inForm

imaging system

Ian Hagemann, MD, PhDCliff Hoyt, MS

Mike Feldman, MD, PhD

Tumor-infiltrating lymphocytes (TILs) in ovarian cancer

• Ovarian cancer may be recognized and attacked by the immune system

• Tumor may contain a lymphocytic infiltrate

• TILs exhibit oligoclonal expansion, recognize tumor antigens, circulate in vivo, and display tumor-specific cytolytic activity in vitro

• Clinical results have been seen with interferon or adoptive T cell immunotherapy

Zhang L , NEJM 2003

The question

• How many lymphocytes are present in this tumor?– Intraepithelial– Stromal

• Alternate phrasing: how densely is this tumor infiltrated by lymphocytes?

The problem

• Ambiguous histology– Limited tissue– Hematoxylin only

• Variable surface area of core, tumor, and stroma

• Human factors– Difficult to count numerous

events– Boredom

Vectra system (CRI, Inc.)

• Multispectral brightfield and fluorescent slide imaging (Nuance)

• Pattern recognition-based, partially automated scanning (Vectra)

• Automated tissue and cell segmentation (inForm)

Imaging a TMA using VectraInput: Stained TMA slideOutput: Hundreds of multispectral image files indexed by grid location.

Input

Training regionsfor tissue segmenter

Output of tissue segmenter Output of tissue and cell segmenter

Review classified images

• Some histospots will have been classified incorrectly– Core fell off or folded over– Unsuitable tissue– Tumor interpreted as stroma, or vice versa– Lymphocyte over- or undercounting

• Task: visually review each core for appropriate segmentation– Despite sophisticated segmentation algorithms,

this step (performed by a human) appears to be essential

Tales of woe

Segmentation algorithms fail

on some fraction of histospots

Total histospots evaluated 618Pre-algorithmic failures

Spot fell offUnsuitable tissue (e.g., colon or fat

only)

3777

Tissue segmentation failuresTumor interpreted as stroma

Stroma interpreted as tumor2649

Cell segmentation failuresOverdetection of lymphocytesUnderdetection of lymphocytes

93

Spots successfully segmented 436

Manual and automated TIL scores are significantly correlated

r=0.54 (95% CI, 0.47–0.61) r=0.68 (95% CI, 0.61–0.74) p<0.0001 p<0.0001

Simulated perfect concordance between manual and automated TIL counts

Observations and conclusions

• Automated event scoring provides a consistent approach to tedious, poorly reproducible tasks.

• Histology scoring tasks can probably never be completely automated.

• Automated lymphocyte counts are significantly correlated with manual counts.

• Gold-standard performance for this task is undefined (and probably impossible to define)

Future directions

• Improved machine learning and classification algorithms will shrink the group of segmentation failures (never to zero)

• Greater leveraging of multispectral technology may allow a qualitative leap forward in the depth of tissue annotation (e.g., “tumor mask” staining by cytokeratin)

• An integrated TMA-aware workflow would reduce manual steps (cut and paste) and increase throughput

• Quantitative direct feature counting can inform semi-quantitative analyses (e.g., where to set cutoffs?)

Acknowledgments

UPENNMike Feldman, MD, PhD

Tim Baradet, PhDGeorge Coukos, MD, PhDAndrea Hagemann, MD

CRi, Inc.Cliff Hoyt, MS

Craig Lassy, PhD

ian.hagemann@uphs.upenn.edu