PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images

Samuel G. Armato, Henkjan Huisman, Karen Drukker, Lubomir Hadjiiski, Justin S. Kirby, Nicholas Petrick, George Redmond, Maryellen L. Giger, Kenny Cha, Artem Mamonov, Jayashree Kalpathy-Cramer, Keyvan Farahani

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from -0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.

Original languageEnglish (US)
Article number044501
JournalJournal of Medical Imaging
Volume5
Issue number4
DOIs
StatePublished - Oct 1 2018
Externally publishedYes

Fingerprint

Prostate
Magnetic Resonance Spectroscopy
Process Assessment (Health Care)
Diagnostic Imaging
ROC Curve
Biomedical Research
Prostatic Neoplasms

Keywords

  • Gleason Grade Group
  • grand challenge
  • imaging biomarker
  • lesion classification
  • multiparametric magnetic resonance images
  • prostate cancer

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Armato, S. G., Huisman, H., Drukker, K., Hadjiiski, L., Kirby, J. S., Petrick, N., ... Farahani, K. (2018). PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. Journal of Medical Imaging, 5(4), [044501]. https://doi.org/10.1117/1.JMI.5.4.044501

PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. / Armato, Samuel G.; Huisman, Henkjan; Drukker, Karen; Hadjiiski, Lubomir; Kirby, Justin S.; Petrick, Nicholas; Redmond, George; Giger, Maryellen L.; Cha, Kenny; Mamonov, Artem; Kalpathy-Cramer, Jayashree; Farahani, Keyvan.

In: Journal of Medical Imaging, Vol. 5, No. 4, 044501, 01.10.2018.

Research output: Contribution to journalArticle

Armato, SG, Huisman, H, Drukker, K, Hadjiiski, L, Kirby, JS, Petrick, N, Redmond, G, Giger, ML, Cha, K, Mamonov, A, Kalpathy-Cramer, J & Farahani, K 2018, 'PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images', Journal of Medical Imaging, vol. 5, no. 4, 044501. https://doi.org/10.1117/1.JMI.5.4.044501
Armato, Samuel G. ; Huisman, Henkjan ; Drukker, Karen ; Hadjiiski, Lubomir ; Kirby, Justin S. ; Petrick, Nicholas ; Redmond, George ; Giger, Maryellen L. ; Cha, Kenny ; Mamonov, Artem ; Kalpathy-Cramer, Jayashree ; Farahani, Keyvan. / PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. In: Journal of Medical Imaging. 2018 ; Vol. 5, No. 4.
@article{80eafc2270004d23b08709a5919db2b8,
title = "PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images",
abstract = "Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from -0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.",
keywords = "Gleason Grade Group, grand challenge, imaging biomarker, lesion classification, multiparametric magnetic resonance images, prostate cancer",
author = "Armato, {Samuel G.} and Henkjan Huisman and Karen Drukker and Lubomir Hadjiiski and Kirby, {Justin S.} and Nicholas Petrick and George Redmond and Giger, {Maryellen L.} and Kenny Cha and Artem Mamonov and Jayashree Kalpathy-Cramer and Keyvan Farahani",
year = "2018",
month = "10",
day = "1",
doi = "10.1117/1.JMI.5.4.044501",
language = "English (US)",
volume = "5",
journal = "Journal of Medical Imaging",
issn = "2329-4302",
publisher = "SPIE",
number = "4",

}

TY - JOUR

T1 - PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images

AU - Armato, Samuel G.

AU - Huisman, Henkjan

AU - Drukker, Karen

AU - Hadjiiski, Lubomir

AU - Kirby, Justin S.

AU - Petrick, Nicholas

AU - Redmond, George

AU - Giger, Maryellen L.

AU - Cha, Kenny

AU - Mamonov, Artem

AU - Kalpathy-Cramer, Jayashree

AU - Farahani, Keyvan

PY - 2018/10/1

Y1 - 2018/10/1

N2 - Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from -0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.

AB - Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from -0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.

KW - Gleason Grade Group

KW - grand challenge

KW - imaging biomarker

KW - lesion classification

KW - multiparametric magnetic resonance images

KW - prostate cancer

UR - http://www.scopus.com/inward/record.url?scp=85056752803&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056752803&partnerID=8YFLogxK

U2 - 10.1117/1.JMI.5.4.044501

DO - 10.1117/1.JMI.5.4.044501

M3 - Article

VL - 5

JO - Journal of Medical Imaging

JF - Journal of Medical Imaging

SN - 2329-4302

IS - 4

M1 - 044501

ER -