Telemedical Diagnosis of Retinopathy of Prematurity. Intraphysician Agreement between Ophthalmoscopic Examination and Image-Based Interpretation

Objective: To evaluate the intraphysician agreement between ophthalmoscopic examination and image-based telemedical interpretation for retinopathy of prematurity (ROP) diagnosis, when performed by the same expert physician grader. Design: Prospective, nonrandomized, comparative study. Participants: Sixty-seven consecutive premature infants who underwent ROP examination at a major university medical center whose parents consented for participation. Methods: Infants underwent standard dilated ophthalmoscopy by one of two pediatric ophthalmologists, followed by retinal imaging with a commercially available wide-angle fundus camera by a trained neonatal nurse. Study examinations were performed at 31 to 33 weeks postmenstrual age (PMA) and/or 35 to 37 weeks PMA. Images were uploaded to a Web-based telemedicine system developed by the authors. After a 4- to 12-month period, telemedical interpretations were performed in which each physician graded images from infants upon whom he had initially performed ophthalmoscopic examinations. Diagnoses were classified using an ordinal scale: no ROP, mild ROP, type 2 prethreshold ROP, and treatment-requiring ROP. Main Outcome Measures: Absolute intraphysician agreement and κ statistic between ophthalmoscopic examination and telemedical interpretation were calculated by eye. All intraphysician discrepancies were reviewed, and underlying causes were classified by eye as no ROP identified by ophthalmoscopic examination, no ROP identified by telemedical interpretation, discrepancy about presence of zone 1 ROP, discrepancy about presence of plus disease, or other discrepancy in classification of ROP stage. Results: Absolute intraphysician agreement between ophthalmoscopic examination and telemedical interpretation was 86.3%. The κ statistic for intraphysician agreement between examinations ranged from 0.657 (substantial agreement) for diagnosis of treatment-requiring ROP to 0.854 (near-perfect agreement) for diagnosis of mild or worse ROP. Among 206 eye examinations (103 infant examinations), there were 28 (13.6%) intraphysician discrepancies in diagnosis, 8 of which resulted from uncertainty about presence of zone 1 disease and 4 from uncertainty about presence of plus disease. Conclusions: Intraphysician agreement between ophthalmoscopic examination and telemedical interpretation for ROP was very high. Neither examination modality appeared to have a systematic tendency to overdiagnose or underdiagnose ROP. Diagnosis of zone 1 disease and plus disease were major sources of clinically significant discrepancies.