Evaluation of a Computer-Based System for Plus Disease Diagnosis in Retinopathy of Prematurity

Susan Koreen, Rony Gelman, M. Elena Martinez-Perez, Lei Jiang, Audina M. Berrocal, Ditte J. Hess, John T. Flynn, Michael F. Chiang

Research output: Contribution to journalArticle

62 Scopus citations


Objective: To measure accuracy and reliability of the computer-based Retinal Image Multiscale Analysis (RISA) system compared with those of recognized retinopathy of prematurity (ROP) experts, for plus disease diagnosis. Design: Evaluation of diagnostic test or technology. Participants: Eleven recognized ROP experts and the RISA image analysis system interpreted a set of 20 wide-angle retinal photographs for presence of plus disease. Methods: All experts used a secure Web site to review independently 20 images for presence of plus disease. Images were also analyzed by measuring individual computer-based system parameters (integrated curvature [IC], diameter, and tortuosity index) for arterioles and venules and by computing linear combinations and logical combinations of those parameters. Performance was compared with a reference standard, defined as the majority vote of experts. Main Outcome Measures: Diagnostic accuracy was measured by calculating sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for plus disease diagnosis by each expert, and by each computer-based system parameter, compared with the reference standard. Diagnostic agreement was measured by calculating the mean κ value of each expert compared with all other experts and the mean κ value of each computer-based system parameter compared with all experts. Results: Among the 11 experts, sensitivity ranged from 0.167 to 1.000, specificity ranged from 0.714 to 1.000, AUC ranged from 0.798 to 1.000, and mean κ compared with all other experts ranged from 0.288 to 0.689. Among individual computer system parameters, arteriolar IC had the highest diagnostic accuracy, with sensitivity of 1.000; specificity, 0.846; and AUC, 0.962. Arteriolar IC had the highest diagnostic agreement with experts, with a mean κ value of 0.578. Conclusions: A computer-based image analysis system has the potential to perform comparably to recognized ROP experts for plus disease diagnosis.

Original languageEnglish (US)
Pages (from-to)e59-e67
Issue number12
StatePublished - Dec 1 2007


ASJC Scopus subject areas

  • Ophthalmology

Cite this

Koreen, S., Gelman, R., Martinez-Perez, M. E., Jiang, L., Berrocal, A. M., Hess, D. J., Flynn, J. T., & Chiang, M. F. (2007). Evaluation of a Computer-Based System for Plus Disease Diagnosis in Retinopathy of Prematurity. Ophthalmology, 114(12), e59-e67. https://doi.org/10.1016/j.ophtha.2007.10.006