TY - JOUR
T1 - A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies
AU - Smith, C. Scott
AU - Morris, Magdalena
AU - Francovich, Chris
AU - Tivis, Rick
AU - Bush, Roger
AU - Sanders, Shelley Schoepflin
AU - Graham, Jeremy
AU - Niven, Alex
AU - Kai, Mari
AU - Knight, Christopher
AU - Hardman, Joseph
AU - Caverzagie, Kelly
AU - Iobst, William
PY - 2013/7
Y1 - 2013/7
N2 - PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.
AB - PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.
UR - http://www.scopus.com/inward/record.url?scp=84879985356&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84879985356&partnerID=8YFLogxK
U2 - 10.1097/ACM.0b013e3182951efc
DO - 10.1097/ACM.0b013e3182951efc
M3 - Article
C2 - 23702529
AN - SCOPUS:84879985356
SN - 1040-2446
VL - 88
SP - 997
EP - 1001
JO - Academic Medicine
JF - Academic Medicine
IS - 7
ER -