Accurate determination of imaging modality using an ensemble of text-and image-based classifiers

Charles E. Kahn, Jayashree Kalpathy-Cramer, Cesar A. Lam, Christina E. Eldredge

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Imaging modality can aid retrieval of medical images for clinical practice, research, and education. We evaluated whether an ensemble classifier could outperform its constituent individual classifiers in determining the modality of figures from radiology journals. Seventeen automated classifiers analyzed 77,495 images from two radiology journals. Each classifier assigned one of eight imaging modalities - computed tomography, graphic, magnetic resonance imaging, nuclear medicine, positron emission tomography, photograph, ultrasound, or radiograph - to each image based on visual and/or textual information. Three physicians determined the modality of 5,000 randomly selected images as a reference standard. A "Simple Vote" ensemble classifier assigned each image to the modality that received the greatest number of individual classifiers' votes. A "Weighted Vote" classifier weighted each individual classifier's vote based on performance over a training set. For each image, this classifier's output was the imaging modality that received the greatest weighted vote score. We measured precision, recall, and F score (the harmonic mean of precision and recall) for each classifier. Individual classifiers' F scores ranged from 0.184 to 0.892. The simple vote and weighted vote classifiers correctly assigned 4,565 images (F score, 0.913; 95% confidence interval, 0.905-0.921) and 4,672 images (F score, 0.934; 95% confidence interval, 0.927-0.941), respectively. The weighted vote classifier performed significantly better than all individual classifiers. An ensemble classifier correctly determined the imaging modality of 93% of figures in our sample. The imaging modality of figures published in radiology journals can be determined with high accuracy, which will improve systems for image retrieval.

Original languageEnglish (US)
Pages (from-to)37-42
Number of pages6
JournalJournal of Digital Imaging
Volume25
Issue number1
DOIs
StatePublished - Feb 2012

Fingerprint

Radiology
Classifiers
Imaging techniques
Confidence Intervals
Nuclear Medicine
Positron-Emission Tomography
Tomography
Magnetic Resonance Imaging
Physicians
Education
Research
Nuclear medicine
Positron emission tomography
Image retrieval
Magnetic resonance
Ultrasonics

Keywords

  • Classification
  • Computer vision
  • Content-based image retrieval
  • Data mining
  • Digital libraries
  • Image analysis
  • Image retrieval

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Medicine(all)

Cite this

Accurate determination of imaging modality using an ensemble of text-and image-based classifiers. / Kahn, Charles E.; Kalpathy-Cramer, Jayashree; Lam, Cesar A.; Eldredge, Christina E.

In: Journal of Digital Imaging, Vol. 25, No. 1, 02.2012, p. 37-42.

Research output: Contribution to journalArticle

Kahn, Charles E. ; Kalpathy-Cramer, Jayashree ; Lam, Cesar A. ; Eldredge, Christina E. / Accurate determination of imaging modality using an ensemble of text-and image-based classifiers. In: Journal of Digital Imaging. 2012 ; Vol. 25, No. 1. pp. 37-42.
@article{ede4208fedab4162b5a48bb979186cec,
title = "Accurate determination of imaging modality using an ensemble of text-and image-based classifiers",
abstract = "Imaging modality can aid retrieval of medical images for clinical practice, research, and education. We evaluated whether an ensemble classifier could outperform its constituent individual classifiers in determining the modality of figures from radiology journals. Seventeen automated classifiers analyzed 77,495 images from two radiology journals. Each classifier assigned one of eight imaging modalities - computed tomography, graphic, magnetic resonance imaging, nuclear medicine, positron emission tomography, photograph, ultrasound, or radiograph - to each image based on visual and/or textual information. Three physicians determined the modality of 5,000 randomly selected images as a reference standard. A {"}Simple Vote{"} ensemble classifier assigned each image to the modality that received the greatest number of individual classifiers' votes. A {"}Weighted Vote{"} classifier weighted each individual classifier's vote based on performance over a training set. For each image, this classifier's output was the imaging modality that received the greatest weighted vote score. We measured precision, recall, and F score (the harmonic mean of precision and recall) for each classifier. Individual classifiers' F scores ranged from 0.184 to 0.892. The simple vote and weighted vote classifiers correctly assigned 4,565 images (F score, 0.913; 95{\%} confidence interval, 0.905-0.921) and 4,672 images (F score, 0.934; 95{\%} confidence interval, 0.927-0.941), respectively. The weighted vote classifier performed significantly better than all individual classifiers. An ensemble classifier correctly determined the imaging modality of 93{\%} of figures in our sample. The imaging modality of figures published in radiology journals can be determined with high accuracy, which will improve systems for image retrieval.",
keywords = "Classification, Computer vision, Content-based image retrieval, Data mining, Digital libraries, Image analysis, Image retrieval",
author = "Kahn, {Charles E.} and Jayashree Kalpathy-Cramer and Lam, {Cesar A.} and Eldredge, {Christina E.}",
year = "2012",
month = "2",
doi = "10.1007/s10278-011-9399-5",
language = "English (US)",
volume = "25",
pages = "37--42",
journal = "Journal of Digital Imaging",
issn = "0897-1889",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - Accurate determination of imaging modality using an ensemble of text-and image-based classifiers

AU - Kahn, Charles E.

AU - Kalpathy-Cramer, Jayashree

AU - Lam, Cesar A.

AU - Eldredge, Christina E.

PY - 2012/2

Y1 - 2012/2

N2 - Imaging modality can aid retrieval of medical images for clinical practice, research, and education. We evaluated whether an ensemble classifier could outperform its constituent individual classifiers in determining the modality of figures from radiology journals. Seventeen automated classifiers analyzed 77,495 images from two radiology journals. Each classifier assigned one of eight imaging modalities - computed tomography, graphic, magnetic resonance imaging, nuclear medicine, positron emission tomography, photograph, ultrasound, or radiograph - to each image based on visual and/or textual information. Three physicians determined the modality of 5,000 randomly selected images as a reference standard. A "Simple Vote" ensemble classifier assigned each image to the modality that received the greatest number of individual classifiers' votes. A "Weighted Vote" classifier weighted each individual classifier's vote based on performance over a training set. For each image, this classifier's output was the imaging modality that received the greatest weighted vote score. We measured precision, recall, and F score (the harmonic mean of precision and recall) for each classifier. Individual classifiers' F scores ranged from 0.184 to 0.892. The simple vote and weighted vote classifiers correctly assigned 4,565 images (F score, 0.913; 95% confidence interval, 0.905-0.921) and 4,672 images (F score, 0.934; 95% confidence interval, 0.927-0.941), respectively. The weighted vote classifier performed significantly better than all individual classifiers. An ensemble classifier correctly determined the imaging modality of 93% of figures in our sample. The imaging modality of figures published in radiology journals can be determined with high accuracy, which will improve systems for image retrieval.

AB - Imaging modality can aid retrieval of medical images for clinical practice, research, and education. We evaluated whether an ensemble classifier could outperform its constituent individual classifiers in determining the modality of figures from radiology journals. Seventeen automated classifiers analyzed 77,495 images from two radiology journals. Each classifier assigned one of eight imaging modalities - computed tomography, graphic, magnetic resonance imaging, nuclear medicine, positron emission tomography, photograph, ultrasound, or radiograph - to each image based on visual and/or textual information. Three physicians determined the modality of 5,000 randomly selected images as a reference standard. A "Simple Vote" ensemble classifier assigned each image to the modality that received the greatest number of individual classifiers' votes. A "Weighted Vote" classifier weighted each individual classifier's vote based on performance over a training set. For each image, this classifier's output was the imaging modality that received the greatest weighted vote score. We measured precision, recall, and F score (the harmonic mean of precision and recall) for each classifier. Individual classifiers' F scores ranged from 0.184 to 0.892. The simple vote and weighted vote classifiers correctly assigned 4,565 images (F score, 0.913; 95% confidence interval, 0.905-0.921) and 4,672 images (F score, 0.934; 95% confidence interval, 0.927-0.941), respectively. The weighted vote classifier performed significantly better than all individual classifiers. An ensemble classifier correctly determined the imaging modality of 93% of figures in our sample. The imaging modality of figures published in radiology journals can be determined with high accuracy, which will improve systems for image retrieval.

KW - Classification

KW - Computer vision

KW - Content-based image retrieval

KW - Data mining

KW - Digital libraries

KW - Image analysis

KW - Image retrieval

UR - http://www.scopus.com/inward/record.url?scp=84861328860&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84861328860&partnerID=8YFLogxK

U2 - 10.1007/s10278-011-9399-5

DO - 10.1007/s10278-011-9399-5

M3 - Article

C2 - 21748413

AN - SCOPUS:84861328860

VL - 25

SP - 37

EP - 42

JO - Journal of Digital Imaging

JF - Journal of Digital Imaging

SN - 0897-1889

IS - 1

ER -