Plus disease in retinopathy of prematurity qualitative analysis of diagnostic process by experts

Nina J. Hewing, David R. Kaufman, R. V Paul Chan, Michael Chiang

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

IMPORTANCE Plus disease is the most important parameter that characterizes severe treatment-requiring retinopathy of prematurity, yet diagnostic agreement among experts is imperfect and the precise factors involved in clinical diagnosis are unclear. This study is designed to address these gaps in knowledge by analyzing cognitive aspects of the plus disease diagnostic process by experts. OBJECTIVE To examine the diagnostic reasoning process of experts for plus disease in retinopathy of prematurity using qualitative research techniques. DESIGN Cognitive walk-through, with qualitative analysis of videotaped expert responses and quantitative analysis of expert diagnoses. SETTING Experimental setting in which experts were videotaped while reviewing study data. PARTICIPANTS A panel of international retinopathy of prematurity experts who had the experience of using qualitative retinal features as their primary basis for clinical diagnosis. INTERVENTION Six experts were video recorded while independently reviewing 7 wide-angle retinal images from infants with retinopathy of prematurity. Experts were asked to explain their diagnostic process in detail (think-aloud protocol), mark findings relevant to their reasoning, and diagnose each image (plus vs pre-plus vs neither). Subsequently, each expert viewed the images again while being asked to examine arteries and veins in isolation and answer specific questions. Video recordings were transcribed and reviewed. Diagnostic process of experts was analyzed using a published cognitive model. MAIN OUTCOME AND MEASURES Interexpert and intraexpert agreement. RESULTS Based on the think-aloud protocol, 5 of 6 experts agreed on the same diagnosis in 3 study images and 3 of 6 experts agreed in 3 images. When experts were asked to rank images in order of severity, the mean correlation coefficient between pairs of experts was 0.33 (range, -0.04 to 0.75). All experts considered arterial tortuosity and venous dilation while reviewing each image. Some considered venous tortuosity, arterial dilation, peripheral retinal features, and other factors. When experts were asked to rereview images to diagnose plus disease based strictly on definitions of sufficient arterial tortuosity and venous dilation, all but 1 expert changed their diagnosis compared with the think-aloud protocol. CONCLUSIONS AND RELEVANCE Diagnostic consistency in plus disease is imperfect. Experts differ in their reasoning process, retinal features that they focus on, and interpretations of the same features. Understanding these factors may improve diagnosis and education. Future research defining more precise diagnostic criteria may be warranted.

Original languageEnglish (US)
Pages (from-to)1026-1032
Number of pages7
JournalJAMA Ophthalmology
Volume131
Issue number8
DOIs
StatePublished - Aug 2013

Fingerprint

Retinopathy of Prematurity
Dilatation
Video Recording
Qualitative Research
Veins
Research Design
Arteries
Education

ASJC Scopus subject areas

  • Ophthalmology

Cite this

Plus disease in retinopathy of prematurity qualitative analysis of diagnostic process by experts. / Hewing, Nina J.; Kaufman, David R.; Chan, R. V Paul; Chiang, Michael.

In: JAMA Ophthalmology, Vol. 131, No. 8, 08.2013, p. 1026-1032.

Research output: Contribution to journalArticle

Hewing, Nina J. ; Kaufman, David R. ; Chan, R. V Paul ; Chiang, Michael. / Plus disease in retinopathy of prematurity qualitative analysis of diagnostic process by experts. In: JAMA Ophthalmology. 2013 ; Vol. 131, No. 8. pp. 1026-1032.
@article{dfa4e58970e84a9cb6a97bbe8a283cc9,
title = "Plus disease in retinopathy of prematurity qualitative analysis of diagnostic process by experts",
abstract = "IMPORTANCE Plus disease is the most important parameter that characterizes severe treatment-requiring retinopathy of prematurity, yet diagnostic agreement among experts is imperfect and the precise factors involved in clinical diagnosis are unclear. This study is designed to address these gaps in knowledge by analyzing cognitive aspects of the plus disease diagnostic process by experts. OBJECTIVE To examine the diagnostic reasoning process of experts for plus disease in retinopathy of prematurity using qualitative research techniques. DESIGN Cognitive walk-through, with qualitative analysis of videotaped expert responses and quantitative analysis of expert diagnoses. SETTING Experimental setting in which experts were videotaped while reviewing study data. PARTICIPANTS A panel of international retinopathy of prematurity experts who had the experience of using qualitative retinal features as their primary basis for clinical diagnosis. INTERVENTION Six experts were video recorded while independently reviewing 7 wide-angle retinal images from infants with retinopathy of prematurity. Experts were asked to explain their diagnostic process in detail (think-aloud protocol), mark findings relevant to their reasoning, and diagnose each image (plus vs pre-plus vs neither). Subsequently, each expert viewed the images again while being asked to examine arteries and veins in isolation and answer specific questions. Video recordings were transcribed and reviewed. Diagnostic process of experts was analyzed using a published cognitive model. MAIN OUTCOME AND MEASURES Interexpert and intraexpert agreement. RESULTS Based on the think-aloud protocol, 5 of 6 experts agreed on the same diagnosis in 3 study images and 3 of 6 experts agreed in 3 images. When experts were asked to rank images in order of severity, the mean correlation coefficient between pairs of experts was 0.33 (range, -0.04 to 0.75). All experts considered arterial tortuosity and venous dilation while reviewing each image. Some considered venous tortuosity, arterial dilation, peripheral retinal features, and other factors. When experts were asked to rereview images to diagnose plus disease based strictly on definitions of sufficient arterial tortuosity and venous dilation, all but 1 expert changed their diagnosis compared with the think-aloud protocol. CONCLUSIONS AND RELEVANCE Diagnostic consistency in plus disease is imperfect. Experts differ in their reasoning process, retinal features that they focus on, and interpretations of the same features. Understanding these factors may improve diagnosis and education. Future research defining more precise diagnostic criteria may be warranted.",
author = "Hewing, {Nina J.} and Kaufman, {David R.} and Chan, {R. V Paul} and Michael Chiang",
year = "2013",
month = "8",
doi = "10.1001/jamaophthalmol.2013.135",
language = "English (US)",
volume = "131",
pages = "1026--1032",
journal = "JAMA Ophthalmology",
issn = "2168-6165",
publisher = "American Medical Association",
number = "8",

}

TY - JOUR

T1 - Plus disease in retinopathy of prematurity qualitative analysis of diagnostic process by experts

AU - Hewing, Nina J.

AU - Kaufman, David R.

AU - Chan, R. V Paul

AU - Chiang, Michael

PY - 2013/8

Y1 - 2013/8

N2 - IMPORTANCE Plus disease is the most important parameter that characterizes severe treatment-requiring retinopathy of prematurity, yet diagnostic agreement among experts is imperfect and the precise factors involved in clinical diagnosis are unclear. This study is designed to address these gaps in knowledge by analyzing cognitive aspects of the plus disease diagnostic process by experts. OBJECTIVE To examine the diagnostic reasoning process of experts for plus disease in retinopathy of prematurity using qualitative research techniques. DESIGN Cognitive walk-through, with qualitative analysis of videotaped expert responses and quantitative analysis of expert diagnoses. SETTING Experimental setting in which experts were videotaped while reviewing study data. PARTICIPANTS A panel of international retinopathy of prematurity experts who had the experience of using qualitative retinal features as their primary basis for clinical diagnosis. INTERVENTION Six experts were video recorded while independently reviewing 7 wide-angle retinal images from infants with retinopathy of prematurity. Experts were asked to explain their diagnostic process in detail (think-aloud protocol), mark findings relevant to their reasoning, and diagnose each image (plus vs pre-plus vs neither). Subsequently, each expert viewed the images again while being asked to examine arteries and veins in isolation and answer specific questions. Video recordings were transcribed and reviewed. Diagnostic process of experts was analyzed using a published cognitive model. MAIN OUTCOME AND MEASURES Interexpert and intraexpert agreement. RESULTS Based on the think-aloud protocol, 5 of 6 experts agreed on the same diagnosis in 3 study images and 3 of 6 experts agreed in 3 images. When experts were asked to rank images in order of severity, the mean correlation coefficient between pairs of experts was 0.33 (range, -0.04 to 0.75). All experts considered arterial tortuosity and venous dilation while reviewing each image. Some considered venous tortuosity, arterial dilation, peripheral retinal features, and other factors. When experts were asked to rereview images to diagnose plus disease based strictly on definitions of sufficient arterial tortuosity and venous dilation, all but 1 expert changed their diagnosis compared with the think-aloud protocol. CONCLUSIONS AND RELEVANCE Diagnostic consistency in plus disease is imperfect. Experts differ in their reasoning process, retinal features that they focus on, and interpretations of the same features. Understanding these factors may improve diagnosis and education. Future research defining more precise diagnostic criteria may be warranted.

AB - IMPORTANCE Plus disease is the most important parameter that characterizes severe treatment-requiring retinopathy of prematurity, yet diagnostic agreement among experts is imperfect and the precise factors involved in clinical diagnosis are unclear. This study is designed to address these gaps in knowledge by analyzing cognitive aspects of the plus disease diagnostic process by experts. OBJECTIVE To examine the diagnostic reasoning process of experts for plus disease in retinopathy of prematurity using qualitative research techniques. DESIGN Cognitive walk-through, with qualitative analysis of videotaped expert responses and quantitative analysis of expert diagnoses. SETTING Experimental setting in which experts were videotaped while reviewing study data. PARTICIPANTS A panel of international retinopathy of prematurity experts who had the experience of using qualitative retinal features as their primary basis for clinical diagnosis. INTERVENTION Six experts were video recorded while independently reviewing 7 wide-angle retinal images from infants with retinopathy of prematurity. Experts were asked to explain their diagnostic process in detail (think-aloud protocol), mark findings relevant to their reasoning, and diagnose each image (plus vs pre-plus vs neither). Subsequently, each expert viewed the images again while being asked to examine arteries and veins in isolation and answer specific questions. Video recordings were transcribed and reviewed. Diagnostic process of experts was analyzed using a published cognitive model. MAIN OUTCOME AND MEASURES Interexpert and intraexpert agreement. RESULTS Based on the think-aloud protocol, 5 of 6 experts agreed on the same diagnosis in 3 study images and 3 of 6 experts agreed in 3 images. When experts were asked to rank images in order of severity, the mean correlation coefficient between pairs of experts was 0.33 (range, -0.04 to 0.75). All experts considered arterial tortuosity and venous dilation while reviewing each image. Some considered venous tortuosity, arterial dilation, peripheral retinal features, and other factors. When experts were asked to rereview images to diagnose plus disease based strictly on definitions of sufficient arterial tortuosity and venous dilation, all but 1 expert changed their diagnosis compared with the think-aloud protocol. CONCLUSIONS AND RELEVANCE Diagnostic consistency in plus disease is imperfect. Experts differ in their reasoning process, retinal features that they focus on, and interpretations of the same features. Understanding these factors may improve diagnosis and education. Future research defining more precise diagnostic criteria may be warranted.

UR - http://www.scopus.com/inward/record.url?scp=84881640446&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84881640446&partnerID=8YFLogxK

U2 - 10.1001/jamaophthalmol.2013.135

DO - 10.1001/jamaophthalmol.2013.135

M3 - Article

C2 - 23702696

AN - SCOPUS:84881640446

VL - 131

SP - 1026

EP - 1032

JO - JAMA Ophthalmology

JF - JAMA Ophthalmology

SN - 2168-6165

IS - 8

ER -