Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures

Diana L. Miglioretti, Laura Ichikawa, Robert A. Smith, Lawrence W. Bassett, Stephen A. Feig, Barbara Monsees, Jay R. Parikh, Robert D. Rosenberg, Edward A. Sickles, Patricia (Patty) Carney

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Objective. Using a combination of performance measures, we updated previously proposed criteria for identifying physicians whose performance interpreting screening mammography may indicate suboptimal interpretation skills. Materials and Methods. In this study, six expert breast imagers used a method based on the Angoff approach to update criteria for acceptable mammography performance on the basis of two sets of combined performance measures: set 1, sensitivity and specificity for facilities with complete capture of false-negative cancers; and set 2, cancer detection rate (CDR), recall rate, and positive predictive value of a recall (PPV1) for facilities that cannot capture false-negative cancers but have reliable cancer follow-up information for positive mammography Results. Decisions were informed by normative data from the Breast Cancer Surveillance Consortium (BCSC). Results. Updated combined ranges for acceptable sensitivity and specificity of screening mammography are sensitivity. 80% and specificity. 85% or sensitivity 75.79% and specificity 88.97%. Updated ranges for CDR, recall rate, and PPV1 are: CDR. 6 per 1000, recall rate 3.20%, and any PPV1; CDR 4.6 per 1000, recall rate 3.15%, and PPV1. 3%; or CDR 2.5.4.0 per 1000, recall rate 5.12%, and PPV1 3.8%. Using the original criteria, 51% of BCSC radiologists had acceptable sensitivity and specificity; 40% had acceptable CDR, recall rate, and PPV1. Using the combined criteria, 69% had acceptable sensitivity and specificity and 62% had acceptable CDR, recall rate, and PPV1. Conclusion. The combined criteria improve previous criteria by considering the interrelationships of multiple performance measures and broaden the acceptable performance ranges compared with previous criteria based on individual measures.

Original languageEnglish (US)
Pages (from-to)W486-W491
JournalAmerican Journal of Roentgenology
Volume204
Issue number4
DOIs
StatePublished - Apr 1 2015

Fingerprint

Mammography
Neoplasms
Sensitivity and Specificity
Radiologists
Breast Neoplasms
Breast
Physicians

Keywords

  • Audit
  • Breast Cancer Surveillance Consortium (BCSC)
  • Mammography
  • Performance
  • Screening

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures. / Miglioretti, Diana L.; Ichikawa, Laura; Smith, Robert A.; Bassett, Lawrence W.; Feig, Stephen A.; Monsees, Barbara; Parikh, Jay R.; Rosenberg, Robert D.; Sickles, Edward A.; Carney, Patricia (Patty).

In: American Journal of Roentgenology, Vol. 204, No. 4, 01.04.2015, p. W486-W491.

Research output: Contribution to journalArticle

Miglioretti, DL, Ichikawa, L, Smith, RA, Bassett, LW, Feig, SA, Monsees, B, Parikh, JR, Rosenberg, RD, Sickles, EA & Carney, PP 2015, 'Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures', American Journal of Roentgenology, vol. 204, no. 4, pp. W486-W491. https://doi.org/10.2214/AJR.13.12313
Miglioretti, Diana L. ; Ichikawa, Laura ; Smith, Robert A. ; Bassett, Lawrence W. ; Feig, Stephen A. ; Monsees, Barbara ; Parikh, Jay R. ; Rosenberg, Robert D. ; Sickles, Edward A. ; Carney, Patricia (Patty). / Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures. In: American Journal of Roentgenology. 2015 ; Vol. 204, No. 4. pp. W486-W491.
@article{e2e9f6227a1142e38da449ad47704c92,
title = "Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures",
abstract = "Objective. Using a combination of performance measures, we updated previously proposed criteria for identifying physicians whose performance interpreting screening mammography may indicate suboptimal interpretation skills. Materials and Methods. In this study, six expert breast imagers used a method based on the Angoff approach to update criteria for acceptable mammography performance on the basis of two sets of combined performance measures: set 1, sensitivity and specificity for facilities with complete capture of false-negative cancers; and set 2, cancer detection rate (CDR), recall rate, and positive predictive value of a recall (PPV1) for facilities that cannot capture false-negative cancers but have reliable cancer follow-up information for positive mammography Results. Decisions were informed by normative data from the Breast Cancer Surveillance Consortium (BCSC). Results. Updated combined ranges for acceptable sensitivity and specificity of screening mammography are sensitivity. 80{\%} and specificity. 85{\%} or sensitivity 75.79{\%} and specificity 88.97{\%}. Updated ranges for CDR, recall rate, and PPV1 are: CDR. 6 per 1000, recall rate 3.20{\%}, and any PPV1; CDR 4.6 per 1000, recall rate 3.15{\%}, and PPV1. 3{\%}; or CDR 2.5.4.0 per 1000, recall rate 5.12{\%}, and PPV1 3.8{\%}. Using the original criteria, 51{\%} of BCSC radiologists had acceptable sensitivity and specificity; 40{\%} had acceptable CDR, recall rate, and PPV1. Using the combined criteria, 69{\%} had acceptable sensitivity and specificity and 62{\%} had acceptable CDR, recall rate, and PPV1. Conclusion. The combined criteria improve previous criteria by considering the interrelationships of multiple performance measures and broaden the acceptable performance ranges compared with previous criteria based on individual measures.",
keywords = "Audit, Breast Cancer Surveillance Consortium (BCSC), Mammography, Performance, Screening",
author = "Miglioretti, {Diana L.} and Laura Ichikawa and Smith, {Robert A.} and Bassett, {Lawrence W.} and Feig, {Stephen A.} and Barbara Monsees and Parikh, {Jay R.} and Rosenberg, {Robert D.} and Sickles, {Edward A.} and Carney, {Patricia (Patty)}",
year = "2015",
month = "4",
day = "1",
doi = "10.2214/AJR.13.12313",
language = "English (US)",
volume = "204",
pages = "W486--W491",
journal = "AJR. American journal of roentgenology",
issn = "0361-803X",
publisher = "American Roentgen Ray Society",
number = "4",

}

TY - JOUR

T1 - Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures

AU - Miglioretti, Diana L.

AU - Ichikawa, Laura

AU - Smith, Robert A.

AU - Bassett, Lawrence W.

AU - Feig, Stephen A.

AU - Monsees, Barbara

AU - Parikh, Jay R.

AU - Rosenberg, Robert D.

AU - Sickles, Edward A.

AU - Carney, Patricia (Patty)

PY - 2015/4/1

Y1 - 2015/4/1

N2 - Objective. Using a combination of performance measures, we updated previously proposed criteria for identifying physicians whose performance interpreting screening mammography may indicate suboptimal interpretation skills. Materials and Methods. In this study, six expert breast imagers used a method based on the Angoff approach to update criteria for acceptable mammography performance on the basis of two sets of combined performance measures: set 1, sensitivity and specificity for facilities with complete capture of false-negative cancers; and set 2, cancer detection rate (CDR), recall rate, and positive predictive value of a recall (PPV1) for facilities that cannot capture false-negative cancers but have reliable cancer follow-up information for positive mammography Results. Decisions were informed by normative data from the Breast Cancer Surveillance Consortium (BCSC). Results. Updated combined ranges for acceptable sensitivity and specificity of screening mammography are sensitivity. 80% and specificity. 85% or sensitivity 75.79% and specificity 88.97%. Updated ranges for CDR, recall rate, and PPV1 are: CDR. 6 per 1000, recall rate 3.20%, and any PPV1; CDR 4.6 per 1000, recall rate 3.15%, and PPV1. 3%; or CDR 2.5.4.0 per 1000, recall rate 5.12%, and PPV1 3.8%. Using the original criteria, 51% of BCSC radiologists had acceptable sensitivity and specificity; 40% had acceptable CDR, recall rate, and PPV1. Using the combined criteria, 69% had acceptable sensitivity and specificity and 62% had acceptable CDR, recall rate, and PPV1. Conclusion. The combined criteria improve previous criteria by considering the interrelationships of multiple performance measures and broaden the acceptable performance ranges compared with previous criteria based on individual measures.

AB - Objective. Using a combination of performance measures, we updated previously proposed criteria for identifying physicians whose performance interpreting screening mammography may indicate suboptimal interpretation skills. Materials and Methods. In this study, six expert breast imagers used a method based on the Angoff approach to update criteria for acceptable mammography performance on the basis of two sets of combined performance measures: set 1, sensitivity and specificity for facilities with complete capture of false-negative cancers; and set 2, cancer detection rate (CDR), recall rate, and positive predictive value of a recall (PPV1) for facilities that cannot capture false-negative cancers but have reliable cancer follow-up information for positive mammography Results. Decisions were informed by normative data from the Breast Cancer Surveillance Consortium (BCSC). Results. Updated combined ranges for acceptable sensitivity and specificity of screening mammography are sensitivity. 80% and specificity. 85% or sensitivity 75.79% and specificity 88.97%. Updated ranges for CDR, recall rate, and PPV1 are: CDR. 6 per 1000, recall rate 3.20%, and any PPV1; CDR 4.6 per 1000, recall rate 3.15%, and PPV1. 3%; or CDR 2.5.4.0 per 1000, recall rate 5.12%, and PPV1 3.8%. Using the original criteria, 51% of BCSC radiologists had acceptable sensitivity and specificity; 40% had acceptable CDR, recall rate, and PPV1. Using the combined criteria, 69% had acceptable sensitivity and specificity and 62% had acceptable CDR, recall rate, and PPV1. Conclusion. The combined criteria improve previous criteria by considering the interrelationships of multiple performance measures and broaden the acceptable performance ranges compared with previous criteria based on individual measures.

KW - Audit

KW - Breast Cancer Surveillance Consortium (BCSC)

KW - Mammography

KW - Performance

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=84932605606&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84932605606&partnerID=8YFLogxK

U2 - 10.2214/AJR.13.12313

DO - 10.2214/AJR.13.12313

M3 - Article

C2 - 25794100

AN - SCOPUS:84932605606

VL - 204

SP - W486-W491

JO - AJR. American journal of roentgenology

JF - AJR. American journal of roentgenology

SN - 0361-803X

IS - 4

ER -