Plus disease in retinopathy of prematurity: Pilot study of computer-based and expert diagnosis

Rony Gelman, Lei Jiang, Yunling E. Du, M. Elena Martinez-Perez, John T. Flynn, Michael Chiang

Research output: Contribution to journalArticle

54 Citations (Scopus)

Abstract

Purpose: To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). Methods: Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. Results: Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8%) of 22 experts. Conclusions: Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.

Original languageEnglish (US)
Pages (from-to)532-540
Number of pages9
JournalJournal of AAPOS
Volume11
Issue number6
DOIs
StatePublished - Dec 2007
Externally publishedYes

Fingerprint

Retinopathy of Prematurity
Computer Systems
Area Under Curve
Venules
Arterioles
ROC Curve
Sensitivity and Specificity
Research

ASJC Scopus subject areas

  • Ophthalmology

Cite this

Plus disease in retinopathy of prematurity : Pilot study of computer-based and expert diagnosis. / Gelman, Rony; Jiang, Lei; Du, Yunling E.; Martinez-Perez, M. Elena; Flynn, John T.; Chiang, Michael.

In: Journal of AAPOS, Vol. 11, No. 6, 12.2007, p. 532-540.

Research output: Contribution to journalArticle

Gelman, Rony ; Jiang, Lei ; Du, Yunling E. ; Martinez-Perez, M. Elena ; Flynn, John T. ; Chiang, Michael. / Plus disease in retinopathy of prematurity : Pilot study of computer-based and expert diagnosis. In: Journal of AAPOS. 2007 ; Vol. 11, No. 6. pp. 532-540.
@article{961ec51872f64fa6bb28eda6a1ff2d1f,
title = "Plus disease in retinopathy of prematurity: Pilot study of computer-based and expert diagnosis",
abstract = "Purpose: To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). Methods: Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. Results: Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8{\%}) of 22 experts. Conclusions: Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.",
author = "Rony Gelman and Lei Jiang and Du, {Yunling E.} and Martinez-Perez, {M. Elena} and Flynn, {John T.} and Michael Chiang",
year = "2007",
month = "12",
doi = "10.1016/j.jaapos.2007.09.005",
language = "English (US)",
volume = "11",
pages = "532--540",
journal = "Journal of AAPOS",
issn = "1091-8531",
publisher = "Mosby Inc.",
number = "6",

}

TY - JOUR

T1 - Plus disease in retinopathy of prematurity

T2 - Pilot study of computer-based and expert diagnosis

AU - Gelman, Rony

AU - Jiang, Lei

AU - Du, Yunling E.

AU - Martinez-Perez, M. Elena

AU - Flynn, John T.

AU - Chiang, Michael

PY - 2007/12

Y1 - 2007/12

N2 - Purpose: To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). Methods: Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. Results: Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8%) of 22 experts. Conclusions: Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.

AB - Purpose: To measure accuracy of plus disease diagnosis by recognized experts in retinopathy of prematurity (ROP), and to conduct a pilot study examining performance of a computer-based image analysis system, Retinal Image multiScale Analysis (RISA). Methods: Twenty-two ROP experts independently interpreted a set of 34 wide-angle retinal images for presence of plus disease. A reference standard diagnosis based on expert consensus was defined for each image. Images were analyzed by the computer-based system using individual and linear combinations of system parameters for arterioles and venules: integrated curvature (IC), diameter, and tortuosity index (TI). Sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) for plus disease diagnosis compared with the reference standard were determined for each expert, as well as for the computer-based system. Results: Expert sensitivity ranged from 0.308 to 1.000, specificity ranged from 0.571 to 1.000, and AUC ranged from 0.784 to 1.000. Among individual computer system parameters, venular IC had highest AUC (0.853). Among all computer system parameters, the linear combination of arteriolar IC, arteriolar TI, venular IC, venular diameter, and venular TI had highest AUC (0.967), which was greater than that of 18 (81.8%) of 22 experts. Conclusions: Accuracy of ROP experts for plus disease diagnosis is imperfect. A computer-based image analysis system has potential to diagnose plus disease with high accuracy. Further research involving RISA system parameter cut-off values from this study are required to fully validate performance of this computer-based system compared with that of human experts.

UR - http://www.scopus.com/inward/record.url?scp=37049011831&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=37049011831&partnerID=8YFLogxK

U2 - 10.1016/j.jaapos.2007.09.005

DO - 10.1016/j.jaapos.2007.09.005

M3 - Article

C2 - 18029210

AN - SCOPUS:37049011831

VL - 11

SP - 532

EP - 540

JO - Journal of AAPOS

JF - Journal of AAPOS

SN - 1091-8531

IS - 6

ER -