Deep learning automated pathology in ex vivo microscopy

Marc Combalia, Sergio Garcia, Josep Malvehy, Susana Puig, Alba Guembe Mulberger, James Browning, Sandra Garcet, James G. Krueger, Samantha R. Lish, Rivka Lax, Jeannie Ren, Mary Stevenson, Nicole Doudican, John A. Carucci, Manu Jain, Kevin White, Jaroslav Rakos, Daniel S. Gareau

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Standard histopathology is currently the gold standard for assessment of margin status in Mohs surgical removal of skin cancer. Ex vivo confocal microscopy (XVM) is potentially faster, less costly and inherently 3D/digital compared to standard histopathology. Despite these advantages, XVM use is not widespread due, in part, to the need for pathologists to retrain to interpret XVM images. We developed artificial intelligence (AI)-driven XVM pathology by implementing algorithms that render intuitive XVM pathology images identical to standard histopathology and produce automated tumor positivity maps. XVM images have fluorescence labeling of cellular and nuclear biology on the background of endogenous (unstained) reflectance contrast as a grounding counter-contrast. XVM images of 26 surgical excision specimens discarded after Mohs micrographic surgery were used to develop an XVM data pipeline with 4 stages: flattening, colorizing, enhancement and automated diagnosis. The first two stages were novel, deterministic image processing algorithms, and the second two were AI algorithms. Diagnostic sensitivity and specificity were calculated for basal cell carcinoma detection as proof of principal for the XVM image processing pipeline. The resulting diagnostic readouts mimicked the appearance of histopathology and found tumor positivity that required first collapsing the confocal stack to a 2D image optimized for cellular fluorescence contrast, then a dark field-to-bright field colorizing transformation, then either an AI image transformation for visual inspection or an AI diagnostic binary image segmentation of tumor obtaining a diagnostic sensitivity and specificity of 88% and 91% respectively. These results show that video-assisted micrographic XVM pathology could feasibly aid margin status determination in micrographic surgery of skin cancer.

Original languageEnglish (US)
Pages (from-to)3103-3116
Number of pages14
JournalBiomedical Optics Express
Issue number6
StatePublished - Jun 1 2021

ASJC Scopus subject areas

  • Biotechnology
  • Atomic and Molecular Physics, and Optics


Dive into the research topics of 'Deep learning automated pathology in ex vivo microscopy'. Together they form a unique fingerprint.

Cite this