TY - JOUR
T1 - Analysis of underlying causes of inter-expert disagreement in reti-nopathy of prematurity diagnosis
T2 - Application of machine learning principles
AU - Ataer-Cansizoglu, E.
AU - Kalpathy-Cramer, J.
AU - You, S.
AU - Keck, K.
AU - Erdogmus, D.
AU - Chiang, M. F.
N1 - Publisher Copyright:
© Schattauer 2015.
PY - 2015
Y1 - 2015
N2 - Objective: Inter-expert variability in imagebased clinical diagnosis has been demonstrated in many diseases including retinopa -thy of prematurity (ROP), which is a disease affecting low birth weight infants and is a major cause of childhood blindness. In order to better understand the underlying causes of variability among experts, we propose a method to quantify the variability of expert decisions and analyze the relationship between expert diagnoses and features computed from the images. Identification of these features is relevant for development of computer-based decision support systems and educational systems in ROP, and these methods may be applicable to other diseases where inter-expert variability is observed.Methods: The experiments were carried out on a dataset of 34 retinal images, each with diagnoses provided independently by 22 experts. Analysis was performed using concepts of Mutual Information (MI) and Kernel Density Estimation. A large set of structural features (a total of 66) were extracted from retinal images. Feature selection was utilized to identify the most important features that correlated to actual clinical decisions by the 22 study experts.Results: The results demonstrate that a group of observers (17 among 22) decide consistently with each other. Mean and second central moment of arteriolar tortuosity is among the reasons of disagreement between this group and the rest of the observers, meaning that the group of experts consider amount of tortuosity as well as the variation of tortuosity in the image.Conclusion: Given a set of image-based features, the proposed analysis method can identify critical image-based features that lead to expert agreement and disagreement in diagnosis of ROP. Although tree-based features and various statistics such as central moment are not popular in the literature, our results suggest that they are important for diagnosis.
AB - Objective: Inter-expert variability in imagebased clinical diagnosis has been demonstrated in many diseases including retinopa -thy of prematurity (ROP), which is a disease affecting low birth weight infants and is a major cause of childhood blindness. In order to better understand the underlying causes of variability among experts, we propose a method to quantify the variability of expert decisions and analyze the relationship between expert diagnoses and features computed from the images. Identification of these features is relevant for development of computer-based decision support systems and educational systems in ROP, and these methods may be applicable to other diseases where inter-expert variability is observed.Methods: The experiments were carried out on a dataset of 34 retinal images, each with diagnoses provided independently by 22 experts. Analysis was performed using concepts of Mutual Information (MI) and Kernel Density Estimation. A large set of structural features (a total of 66) were extracted from retinal images. Feature selection was utilized to identify the most important features that correlated to actual clinical decisions by the 22 study experts.Results: The results demonstrate that a group of observers (17 among 22) decide consistently with each other. Mean and second central moment of arteriolar tortuosity is among the reasons of disagreement between this group and the rest of the observers, meaning that the group of experts consider amount of tortuosity as well as the variation of tortuosity in the image.Conclusion: Given a set of image-based features, the proposed analysis method can identify critical image-based features that lead to expert agreement and disagreement in diagnosis of ROP. Although tree-based features and various statistics such as central moment are not popular in the literature, our results suggest that they are important for diagnosis.
KW - Feature selection
KW - Inter-expert disagreement
KW - Kernel density estimation
KW - Retinopathy of prematurity
UR - http://www.scopus.com/inward/record.url?scp=84921286003&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84921286003&partnerID=8YFLogxK
U2 - 10.3414/ME13-01-0081
DO - 10.3414/ME13-01-0081
M3 - Article
C2 - 25434784
AN - SCOPUS:84921286003
SN - 0026-1270
VL - 54
SP - 93
EP - 102
JO - Methods of Information in Medicine
JF - Methods of Information in Medicine
IS - 1
ER -