TY - JOUR
T1 - Evaluation of a Computer-Based System for Plus Disease Diagnosis in Retinopathy of Prematurity
AU - Koreen, Susan
AU - Gelman, Rony
AU - Martinez-Perez, M. Elena
AU - Jiang, Lei
AU - Berrocal, Audina M.
AU - Hess, Ditte J.
AU - Flynn, John T.
AU - Chiang, Michael F.
N1 - Funding Information:
Supported by a Career Development Award from Research to Prevent Blindness, New York, New York, and the National Institutes of Health, Bethesda, Maryland (grant no. EY13972 [MFC]).
PY - 2007/12
Y1 - 2007/12
N2 - Objective: To measure accuracy and reliability of the computer-based Retinal Image Multiscale Analysis (RISA) system compared with those of recognized retinopathy of prematurity (ROP) experts, for plus disease diagnosis. Design: Evaluation of diagnostic test or technology. Participants: Eleven recognized ROP experts and the RISA image analysis system interpreted a set of 20 wide-angle retinal photographs for presence of plus disease. Methods: All experts used a secure Web site to review independently 20 images for presence of plus disease. Images were also analyzed by measuring individual computer-based system parameters (integrated curvature [IC], diameter, and tortuosity index) for arterioles and venules and by computing linear combinations and logical combinations of those parameters. Performance was compared with a reference standard, defined as the majority vote of experts. Main Outcome Measures: Diagnostic accuracy was measured by calculating sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for plus disease diagnosis by each expert, and by each computer-based system parameter, compared with the reference standard. Diagnostic agreement was measured by calculating the mean κ value of each expert compared with all other experts and the mean κ value of each computer-based system parameter compared with all experts. Results: Among the 11 experts, sensitivity ranged from 0.167 to 1.000, specificity ranged from 0.714 to 1.000, AUC ranged from 0.798 to 1.000, and mean κ compared with all other experts ranged from 0.288 to 0.689. Among individual computer system parameters, arteriolar IC had the highest diagnostic accuracy, with sensitivity of 1.000; specificity, 0.846; and AUC, 0.962. Arteriolar IC had the highest diagnostic agreement with experts, with a mean κ value of 0.578. Conclusions: A computer-based image analysis system has the potential to perform comparably to recognized ROP experts for plus disease diagnosis.
AB - Objective: To measure accuracy and reliability of the computer-based Retinal Image Multiscale Analysis (RISA) system compared with those of recognized retinopathy of prematurity (ROP) experts, for plus disease diagnosis. Design: Evaluation of diagnostic test or technology. Participants: Eleven recognized ROP experts and the RISA image analysis system interpreted a set of 20 wide-angle retinal photographs for presence of plus disease. Methods: All experts used a secure Web site to review independently 20 images for presence of plus disease. Images were also analyzed by measuring individual computer-based system parameters (integrated curvature [IC], diameter, and tortuosity index) for arterioles and venules and by computing linear combinations and logical combinations of those parameters. Performance was compared with a reference standard, defined as the majority vote of experts. Main Outcome Measures: Diagnostic accuracy was measured by calculating sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for plus disease diagnosis by each expert, and by each computer-based system parameter, compared with the reference standard. Diagnostic agreement was measured by calculating the mean κ value of each expert compared with all other experts and the mean κ value of each computer-based system parameter compared with all experts. Results: Among the 11 experts, sensitivity ranged from 0.167 to 1.000, specificity ranged from 0.714 to 1.000, AUC ranged from 0.798 to 1.000, and mean κ compared with all other experts ranged from 0.288 to 0.689. Among individual computer system parameters, arteriolar IC had the highest diagnostic accuracy, with sensitivity of 1.000; specificity, 0.846; and AUC, 0.962. Arteriolar IC had the highest diagnostic agreement with experts, with a mean κ value of 0.578. Conclusions: A computer-based image analysis system has the potential to perform comparably to recognized ROP experts for plus disease diagnosis.
UR - http://www.scopus.com/inward/record.url?scp=36549002810&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=36549002810&partnerID=8YFLogxK
U2 - 10.1016/j.ophtha.2007.10.006
DO - 10.1016/j.ophtha.2007.10.006
M3 - Article
C2 - 18054630
AN - SCOPUS:36549002810
SN - 0161-6420
VL - 114
SP - e59-e67
JO - Ophthalmology
JF - Ophthalmology
IS - 12
ER -