Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis

the Imaging and Informatics in ROP (i-ROP) Research Consortium

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

IMPORTANCE: Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions. OBJECTIVE: To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis. DESIGN, SETTING, AND PARTICIPANTS: A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts. MAINOUTCOMES ANDMEASURES: The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD. RESULTS: Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable tothat of11 expert physicians (mean 87%, range 79%-99%). CONCLUSIONS AND RELEVANCE: Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

Original languageEnglish (US)
Pages (from-to)651-657
Number of pages7
JournalJAMA Ophthalmology
Volume134
Issue number6
DOIs
StatePublished - Jun 1 2016

Fingerprint

Retinopathy of Prematurity
Informatics
Veins
Arteries
Blood Vessels
Outcome Assessment (Health Care)
Neonatal Intensive Care Units
Computer Systems
Retina
Dilatation
Physicians

ASJC Scopus subject areas

  • Ophthalmology

Cite this

Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis. / the Imaging and Informatics in ROP (i-ROP) Research Consortium.

In: JAMA Ophthalmology, Vol. 134, No. 6, 01.06.2016, p. 651-657.

Research output: Contribution to journalArticle

the Imaging and Informatics in ROP (i-ROP) Research Consortium. / Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis. In: JAMA Ophthalmology. 2016 ; Vol. 134, No. 6. pp. 651-657.
@article{90b2d7aafd5546cfadb5e4b4a6da144b,
title = "Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis",
abstract = "IMPORTANCE: Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions. OBJECTIVE: To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis. DESIGN, SETTING, AND PARTICIPANTS: A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts. MAINOUTCOMES ANDMEASURES: The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD. RESULTS: Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95{\%}; 95{\%} CI, 94{\%}-95{\%}) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90{\%} or less when using only arterial tortuosity and 85{\%} or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95{\%}) was comparable tothat of11 expert physicians (mean 87{\%}, range 79{\%}-99{\%}). CONCLUSIONS AND RELEVANCE: Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.",
author = "{the Imaging and Informatics in ROP (i-ROP) Research Consortium} and John Campbell and Esra Ataer-Cansizoglu and Veronica Bolon-Canedo and Alican Bozkurt and Deniz Erdogmus and Jayashree Kalpathy-Cramer and Patel, {Samir N.} and Reynolds, {James D.} and Jason Horowitz and Kelly Hutcheson and Michael Shapiro and Repka, {Michael X.} and Phillip Ferrone and Kimberly Drenser and Martinez-Castellanos, {Maria Ana} and Susan Ostmo and Karyn Jonas and {Paul Chan}, {R. V.} and Michael Chiang and Sonmez, {Mustafa (Kemal)} and Osode Coki and Eccles, {Cheryl Ann} and Leora Sarna and Audina Berrocal and Catherin Negron and Kristi Cumming and Tammy Osentoski and Tammy Check and Thomas Lee and Evan Kruger and Kathryn McGovern and Charles Simmons and Raghu Murthy and Sharon Galvis and Jerome Rotter and Ida Chen and Xiaohui Li and Kaye Roll and Samantha Salinas-Longoria and Rafael Romero and Andrea Arriola",
year = "2016",
month = "6",
day = "1",
doi = "10.1001/jamaophthalmol.2016.0611",
language = "English (US)",
volume = "134",
pages = "651--657",
journal = "JAMA Ophthalmology",
issn = "2168-6165",
publisher = "American Medical Association",
number = "6",

}

TY - JOUR

T1 - Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis

AU - the Imaging and Informatics in ROP (i-ROP) Research Consortium

AU - Campbell, John

AU - Ataer-Cansizoglu, Esra

AU - Bolon-Canedo, Veronica

AU - Bozkurt, Alican

AU - Erdogmus, Deniz

AU - Kalpathy-Cramer, Jayashree

AU - Patel, Samir N.

AU - Reynolds, James D.

AU - Horowitz, Jason

AU - Hutcheson, Kelly

AU - Shapiro, Michael

AU - Repka, Michael X.

AU - Ferrone, Phillip

AU - Drenser, Kimberly

AU - Martinez-Castellanos, Maria Ana

AU - Ostmo, Susan

AU - Jonas, Karyn

AU - Paul Chan, R. V.

AU - Chiang, Michael

AU - Sonmez, Mustafa (Kemal)

AU - Coki, Osode

AU - Eccles, Cheryl Ann

AU - Sarna, Leora

AU - Berrocal, Audina

AU - Negron, Catherin

AU - Cumming, Kristi

AU - Osentoski, Tammy

AU - Check, Tammy

AU - Lee, Thomas

AU - Kruger, Evan

AU - McGovern, Kathryn

AU - Simmons, Charles

AU - Murthy, Raghu

AU - Galvis, Sharon

AU - Rotter, Jerome

AU - Chen, Ida

AU - Li, Xiaohui

AU - Roll, Kaye

AU - Salinas-Longoria, Samantha

AU - Romero, Rafael

AU - Arriola, Andrea

PY - 2016/6/1

Y1 - 2016/6/1

N2 - IMPORTANCE: Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions. OBJECTIVE: To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis. DESIGN, SETTING, AND PARTICIPANTS: A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts. MAINOUTCOMES ANDMEASURES: The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD. RESULTS: Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable tothat of11 expert physicians (mean 87%, range 79%-99%). CONCLUSIONS AND RELEVANCE: Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

AB - IMPORTANCE: Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions. OBJECTIVE: To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis. DESIGN, SETTING, AND PARTICIPANTS: A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts. MAINOUTCOMES ANDMEASURES: The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD. RESULTS: Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable tothat of11 expert physicians (mean 87%, range 79%-99%). CONCLUSIONS AND RELEVANCE: Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

UR - http://www.scopus.com/inward/record.url?scp=84973909618&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973909618&partnerID=8YFLogxK

U2 - 10.1001/jamaophthalmol.2016.0611

DO - 10.1001/jamaophthalmol.2016.0611

M3 - Article

C2 - 27077667

AN - SCOPUS:84973909618

VL - 134

SP - 651

EP - 657

JO - JAMA Ophthalmology

JF - JAMA Ophthalmology

SN - 2168-6165

IS - 6

ER -