SHIFT: Speedy histopathological-To-immunofluorescent translation of whole slide images using conditional generative adversarial networks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Multiplexed imaging such as multicolor immunofluorescence staining, multiplexed immunohistochemistry (mIHC) or cyclic immunofluorescence (cycIF) enables deep assessment of cellular complexity in situ and, in conjunction with standard histology stains like hematoxylin and eosin (H and E), can help to unravel the complex molecular relationships and spatial interdependencies that undergird disease states. However, these multiplexed imaging methods are costly and can degrade both tissue quality and antigenicity with each successive cycle of staining. In addition, computationally intensive image processing such as image registration across multiple channels is required. We have developed a novel method, speedy histopathological-To-immunofluorescent translation (SHIFT) of whole slide images (WSIs) using conditional generative adversarial networks (cGANs). This approach is rooted in the assumption that specific patterns captured in IF images by stains like DAPI, pan-cytokeratin (panCK), or α-smooth muscle actin (α-SMA) are encoded in H and E images, such that a SHIFT model can learn useful feature representations or architectural patterns in the H and E stain that help generate relevant IF stain patterns. We demonstrate that the proposed method is capable of generating realistic tumor marker IF WSIs conditioned on corresponding H and E-stained WSIs with up to 94.5% accuracy in a matter of seconds. Thus, this method has the potential to not only improve our understanding of the mapping of histological and morphological profiles into protein expression profiles, but also greatly increase the efficiency of diagnostic and prognostic decision-making.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2018
Subtitle of host publicationDigital Pathology
PublisherSPIE
Volume10581
ISBN (Electronic)9781510616516
DOIs
StatePublished - Jan 1 2018
EventMedical Imaging 2018: Digital Pathology - Houston, United States
Duration: Feb 11 2018Feb 12 2018

Other

OtherMedical Imaging 2018: Digital Pathology
CountryUnited States
CityHouston
Period2/11/182/12/18

Fingerprint

Hematoxylin
Eosine Yellowish-(YS)
chutes
Coloring Agents
Imaging techniques
Histology
Image registration
Fluorescent Antibody Technique
Muscle
Image processing
Decision making
staining
Tissue
Staining and Labeling
Proteins
Tumor Biomarkers
Keratins
Smooth Muscle
smooth muscle
Actins

Keywords

  • Conditional generative adversarial network
  • Deep learning
  • Digital pathology
  • Image translation

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging

Cite this

SHIFT : Speedy histopathological-To-immunofluorescent translation of whole slide images using conditional generative adversarial networks. / Burlingame, Erik A.; Margolin, Adam; Gray, Joe; Chang, Young Hwan.

Medical Imaging 2018: Digital Pathology. Vol. 10581 SPIE, 2018. 1058105.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Burlingame, EA, Margolin, A, Gray, J & Chang, YH 2018, SHIFT: Speedy histopathological-To-immunofluorescent translation of whole slide images using conditional generative adversarial networks. in Medical Imaging 2018: Digital Pathology. vol. 10581, 1058105, SPIE, Medical Imaging 2018: Digital Pathology, Houston, United States, 2/11/18. https://doi.org/10.1117/12.2293249
@inproceedings{e2d120e36ac84a029d6b5f056f6f6a11,
title = "SHIFT: Speedy histopathological-To-immunofluorescent translation of whole slide images using conditional generative adversarial networks",
abstract = "Multiplexed imaging such as multicolor immunofluorescence staining, multiplexed immunohistochemistry (mIHC) or cyclic immunofluorescence (cycIF) enables deep assessment of cellular complexity in situ and, in conjunction with standard histology stains like hematoxylin and eosin (H and E), can help to unravel the complex molecular relationships and spatial interdependencies that undergird disease states. However, these multiplexed imaging methods are costly and can degrade both tissue quality and antigenicity with each successive cycle of staining. In addition, computationally intensive image processing such as image registration across multiple channels is required. We have developed a novel method, speedy histopathological-To-immunofluorescent translation (SHIFT) of whole slide images (WSIs) using conditional generative adversarial networks (cGANs). This approach is rooted in the assumption that specific patterns captured in IF images by stains like DAPI, pan-cytokeratin (panCK), or α-smooth muscle actin (α-SMA) are encoded in H and E images, such that a SHIFT model can learn useful feature representations or architectural patterns in the H and E stain that help generate relevant IF stain patterns. We demonstrate that the proposed method is capable of generating realistic tumor marker IF WSIs conditioned on corresponding H and E-stained WSIs with up to 94.5{\%} accuracy in a matter of seconds. Thus, this method has the potential to not only improve our understanding of the mapping of histological and morphological profiles into protein expression profiles, but also greatly increase the efficiency of diagnostic and prognostic decision-making.",
keywords = "Conditional generative adversarial network, Deep learning, Digital pathology, Image translation",
author = "Burlingame, {Erik A.} and Adam Margolin and Joe Gray and Chang, {Young Hwan}",
year = "2018",
month = "1",
day = "1",
doi = "10.1117/12.2293249",
language = "English (US)",
volume = "10581",
booktitle = "Medical Imaging 2018",
publisher = "SPIE",

}

TY - GEN

T1 - SHIFT

T2 - Speedy histopathological-To-immunofluorescent translation of whole slide images using conditional generative adversarial networks

AU - Burlingame, Erik A.

AU - Margolin, Adam

AU - Gray, Joe

AU - Chang, Young Hwan

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Multiplexed imaging such as multicolor immunofluorescence staining, multiplexed immunohistochemistry (mIHC) or cyclic immunofluorescence (cycIF) enables deep assessment of cellular complexity in situ and, in conjunction with standard histology stains like hematoxylin and eosin (H and E), can help to unravel the complex molecular relationships and spatial interdependencies that undergird disease states. However, these multiplexed imaging methods are costly and can degrade both tissue quality and antigenicity with each successive cycle of staining. In addition, computationally intensive image processing such as image registration across multiple channels is required. We have developed a novel method, speedy histopathological-To-immunofluorescent translation (SHIFT) of whole slide images (WSIs) using conditional generative adversarial networks (cGANs). This approach is rooted in the assumption that specific patterns captured in IF images by stains like DAPI, pan-cytokeratin (panCK), or α-smooth muscle actin (α-SMA) are encoded in H and E images, such that a SHIFT model can learn useful feature representations or architectural patterns in the H and E stain that help generate relevant IF stain patterns. We demonstrate that the proposed method is capable of generating realistic tumor marker IF WSIs conditioned on corresponding H and E-stained WSIs with up to 94.5% accuracy in a matter of seconds. Thus, this method has the potential to not only improve our understanding of the mapping of histological and morphological profiles into protein expression profiles, but also greatly increase the efficiency of diagnostic and prognostic decision-making.

AB - Multiplexed imaging such as multicolor immunofluorescence staining, multiplexed immunohistochemistry (mIHC) or cyclic immunofluorescence (cycIF) enables deep assessment of cellular complexity in situ and, in conjunction with standard histology stains like hematoxylin and eosin (H and E), can help to unravel the complex molecular relationships and spatial interdependencies that undergird disease states. However, these multiplexed imaging methods are costly and can degrade both tissue quality and antigenicity with each successive cycle of staining. In addition, computationally intensive image processing such as image registration across multiple channels is required. We have developed a novel method, speedy histopathological-To-immunofluorescent translation (SHIFT) of whole slide images (WSIs) using conditional generative adversarial networks (cGANs). This approach is rooted in the assumption that specific patterns captured in IF images by stains like DAPI, pan-cytokeratin (panCK), or α-smooth muscle actin (α-SMA) are encoded in H and E images, such that a SHIFT model can learn useful feature representations or architectural patterns in the H and E stain that help generate relevant IF stain patterns. We demonstrate that the proposed method is capable of generating realistic tumor marker IF WSIs conditioned on corresponding H and E-stained WSIs with up to 94.5% accuracy in a matter of seconds. Thus, this method has the potential to not only improve our understanding of the mapping of histological and morphological profiles into protein expression profiles, but also greatly increase the efficiency of diagnostic and prognostic decision-making.

KW - Conditional generative adversarial network

KW - Deep learning

KW - Digital pathology

KW - Image translation

UR - http://www.scopus.com/inward/record.url?scp=85049222353&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049222353&partnerID=8YFLogxK

U2 - 10.1117/12.2293249

DO - 10.1117/12.2293249

M3 - Conference contribution

AN - SCOPUS:85049222353

VL - 10581

BT - Medical Imaging 2018

PB - SPIE

ER -