Promoting Value Through Patient-Centered Communication: A Multisite Validity Study of Third-Year Medical Students

Purpose To examine validity evidence for a standardized patient scenario assessing medical students' ability to promote value using patient-centered communication (in response to a patient requesting an unnecessary test) and to explore the potential effect of various implementation and curricular factors on student scores. Method Third-year medical students (N = 516) from 5 U.S. MD-granting medical schools completed the communication scenario between 2014 and 2017 as part of a larger objective structured clinical examination (OSCE). Centralized raters assessed performance using an 11-item checklist. The authors collected multiple sources of validity evidence. Results The mean checklist score was 0.85 (standard deviation 0.09). Interrater reliability for checklist scores was excellent (0.87, 95% confidence interval = 0.78-0.93). Generalizability and Phi-coefficients were, respectively, 0.65 and 0.57. Scores decreased as the number of OSCE stations increased (r = -0.15, P =.001) and increased when they were used for summative purposes (r = 0.26, P <.001). Scores were not associated with curricular time devoted to high-value care (r = 0.02, P =.67) and decreased when more clerkships were completed before the assessment (r = -0.12, P =.006). Conclusions This multisite study provides validity evidence supporting the use of scenario scores to assess the ability of medical students to promote value in clinical encounters using patient-centered communication. Findings illuminate the potential effect of OSCE structure and purpose on student performance and suggest clerkship learning experiences may not reinforce what students are taught in the formal curriculum regarding high-value care. Devoting more time to the topic appears insufficient to counteract this erosion.