Remote image based retinopathy of prematurity diagnosis: A receiver operating characteristic analysis of accuracy

Michael Chiang, J. Starren, Y. E. Du, J. D. Keenan, W. M. Schiff, G. R. Barile, J. Li, R. A. Johnson, D. J. Hess, J. T. Flynn

Research output: Contribution to journalArticle

52 Citations (Scopus)

Abstract

Background/aims: Telemedicine offers potential to improve the accessibility and quality ot diagnosis ot retinopathy of prematurity (ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic (ROC) analysis. Methods: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP (that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP (that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. Results: Areas under ROC curves ranged from 0.747-0.896 for detection of mild or worse ROP, 0.905-0.946 for detection of type 2 prethreshold or worse ROP, and 0.941-0.968 for detection of ROP requiring treatment. Conclusions: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.

Original languageEnglish (US)
Pages (from-to)1292-1296
Number of pages5
JournalBritish Journal of Ophthalmology
Volume90
Issue number10
DOIs
StatePublished - Oct 2006
Externally publishedYes

Fingerprint

Retinopathy of Prematurity
ROC Curve
Therapeutics
Ophthalmoscopy

ASJC Scopus subject areas

  • Ophthalmology

Cite this

Remote image based retinopathy of prematurity diagnosis : A receiver operating characteristic analysis of accuracy. / Chiang, Michael; Starren, J.; Du, Y. E.; Keenan, J. D.; Schiff, W. M.; Barile, G. R.; Li, J.; Johnson, R. A.; Hess, D. J.; Flynn, J. T.

In: British Journal of Ophthalmology, Vol. 90, No. 10, 10.2006, p. 1292-1296.

Research output: Contribution to journalArticle

Chiang, M, Starren, J, Du, YE, Keenan, JD, Schiff, WM, Barile, GR, Li, J, Johnson, RA, Hess, DJ & Flynn, JT 2006, 'Remote image based retinopathy of prematurity diagnosis: A receiver operating characteristic analysis of accuracy', British Journal of Ophthalmology, vol. 90, no. 10, pp. 1292-1296. https://doi.org/10.1136/bjo.2006.091900
Chiang, Michael ; Starren, J. ; Du, Y. E. ; Keenan, J. D. ; Schiff, W. M. ; Barile, G. R. ; Li, J. ; Johnson, R. A. ; Hess, D. J. ; Flynn, J. T. / Remote image based retinopathy of prematurity diagnosis : A receiver operating characteristic analysis of accuracy. In: British Journal of Ophthalmology. 2006 ; Vol. 90, No. 10. pp. 1292-1296.
@article{1bec662bc0dc404098914041eea5975a,
title = "Remote image based retinopathy of prematurity diagnosis: A receiver operating characteristic analysis of accuracy",
abstract = "Background/aims: Telemedicine offers potential to improve the accessibility and quality ot diagnosis ot retinopathy of prematurity (ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic (ROC) analysis. Methods: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP (that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP (that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. Results: Areas under ROC curves ranged from 0.747-0.896 for detection of mild or worse ROP, 0.905-0.946 for detection of type 2 prethreshold or worse ROP, and 0.941-0.968 for detection of ROP requiring treatment. Conclusions: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.",
author = "Michael Chiang and J. Starren and Du, {Y. E.} and Keenan, {J. D.} and Schiff, {W. M.} and Barile, {G. R.} and J. Li and Johnson, {R. A.} and Hess, {D. J.} and Flynn, {J. T.}",
year = "2006",
month = "10",
doi = "10.1136/bjo.2006.091900",
language = "English (US)",
volume = "90",
pages = "1292--1296",
journal = "British Journal of Ophthalmology",
issn = "0007-1161",
publisher = "BMJ Publishing Group",
number = "10",

}

TY - JOUR

T1 - Remote image based retinopathy of prematurity diagnosis

T2 - A receiver operating characteristic analysis of accuracy

AU - Chiang, Michael

AU - Starren, J.

AU - Du, Y. E.

AU - Keenan, J. D.

AU - Schiff, W. M.

AU - Barile, G. R.

AU - Li, J.

AU - Johnson, R. A.

AU - Hess, D. J.

AU - Flynn, J. T.

PY - 2006/10

Y1 - 2006/10

N2 - Background/aims: Telemedicine offers potential to improve the accessibility and quality ot diagnosis ot retinopathy of prematurity (ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic (ROC) analysis. Methods: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP (that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP (that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. Results: Areas under ROC curves ranged from 0.747-0.896 for detection of mild or worse ROP, 0.905-0.946 for detection of type 2 prethreshold or worse ROP, and 0.941-0.968 for detection of ROP requiring treatment. Conclusions: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.

AB - Background/aims: Telemedicine offers potential to improve the accessibility and quality ot diagnosis ot retinopathy of prematurity (ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic (ROC) analysis. Methods: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP (that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP (that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. Results: Areas under ROC curves ranged from 0.747-0.896 for detection of mild or worse ROP, 0.905-0.946 for detection of type 2 prethreshold or worse ROP, and 0.941-0.968 for detection of ROP requiring treatment. Conclusions: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.

UR - http://www.scopus.com/inward/record.url?scp=33750284621&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33750284621&partnerID=8YFLogxK

U2 - 10.1136/bjo.2006.091900

DO - 10.1136/bjo.2006.091900

M3 - Article

C2 - 16613919

AN - SCOPUS:33750284621

VL - 90

SP - 1292

EP - 1296

JO - British Journal of Ophthalmology

JF - British Journal of Ophthalmology

SN - 0007-1161

IS - 10

ER -