Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation

Patricia (Patty) Carney, Kimberly H. Allison, Natalia V. Oster, Paul D. Frederick, Thomas R. Morgan, Berta M. Geller, Donald L. Weaver, Joann G. Elmore

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We examined how pathologists’ process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.Modern Pathology advance online publication, 8 April 2016; doi:10.1038/modpathol.2016.62.

Original languageEnglish (US)
JournalModern Pathology
DOIs
StateAccepted/In press - Apr 8 2016

Fingerprint

Breast
Pathology
Continuing Medical Education
Pathologists
Learning
Carcinoma, Intraductal, Noninfiltrating
Publications
Consensus
Breast Neoplasms

ASJC Scopus subject areas

  • Pathology and Forensic Medicine

Cite this

Carney, P. P., Allison, K. H., Oster, N. V., Frederick, P. D., Morgan, T. R., Geller, B. M., ... Elmore, J. G. (Accepted/In press). Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation. Modern Pathology. https://doi.org/10.1038/modpathol.2016.62

Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation. / Carney, Patricia (Patty); Allison, Kimberly H.; Oster, Natalia V.; Frederick, Paul D.; Morgan, Thomas R.; Geller, Berta M.; Weaver, Donald L.; Elmore, Joann G.

In: Modern Pathology, 08.04.2016.

Research output: Contribution to journalArticle

Carney, Patricia (Patty) ; Allison, Kimberly H. ; Oster, Natalia V. ; Frederick, Paul D. ; Morgan, Thomas R. ; Geller, Berta M. ; Weaver, Donald L. ; Elmore, Joann G. / Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation. In: Modern Pathology. 2016.
@article{34cfba07801c4e38ad337d8169c2f8dc,
title = "Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation",
abstract = "We examined how pathologists’ process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5{\%} (75.9{\%} actual agreement; 81.4{\%} estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48{\%}; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6{\%}). Before the continuing medical education program, nearly 55{\%} (54.9{\%}) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80{\%} (79.8{\%}) reported learning new information from this individualized web-based continuing medical education program, and 23.9{\%} of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.Modern Pathology advance online publication, 8 April 2016; doi:10.1038/modpathol.2016.62.",
author = "Carney, {Patricia (Patty)} and Allison, {Kimberly H.} and Oster, {Natalia V.} and Frederick, {Paul D.} and Morgan, {Thomas R.} and Geller, {Berta M.} and Weaver, {Donald L.} and Elmore, {Joann G.}",
year = "2016",
month = "4",
day = "8",
doi = "10.1038/modpathol.2016.62",
language = "English (US)",
journal = "Modern Pathology",
issn = "0893-3952",
publisher = "Nature Publishing Group",

}

TY - JOUR

T1 - Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation

AU - Carney, Patricia (Patty)

AU - Allison, Kimberly H.

AU - Oster, Natalia V.

AU - Frederick, Paul D.

AU - Morgan, Thomas R.

AU - Geller, Berta M.

AU - Weaver, Donald L.

AU - Elmore, Joann G.

PY - 2016/4/8

Y1 - 2016/4/8

N2 - We examined how pathologists’ process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.Modern Pathology advance online publication, 8 April 2016; doi:10.1038/modpathol.2016.62.

AB - We examined how pathologists’ process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.Modern Pathology advance online publication, 8 April 2016; doi:10.1038/modpathol.2016.62.

UR - http://www.scopus.com/inward/record.url?scp=84964048558&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964048558&partnerID=8YFLogxK

U2 - 10.1038/modpathol.2016.62

DO - 10.1038/modpathol.2016.62

M3 - Article

C2 - 27056072

AN - SCOPUS:84964048558

JO - Modern Pathology

JF - Modern Pathology

SN - 0893-3952

ER -