The Effect and Use of Milestones in the Assessment of Neurological Surgery Residents and Residency Programs

Objectives: The purpose of this study was to determine the effect of the Accreditation Council for Graduate Medical Education Milestones on the assessment of neurological surgery residents. The authors sought to determine the feasibility, acceptability, and utility of this new framework in making judgments of progressive competence, its implementation within programs, and the influence on curricula. Residents were also surveyed to elicit the effect of Milestones on their educational experience and professional development. Design, Setting, and Participants: In 2015, program leadership and residents from 21 neurological surgery residency programs participated in an online survey and telephone interview in which they reflected on their experiences with the Milestones. Survey data were analyzed using descriptive statistics. Interview transcripts were analyzed using grounded theory. Results: Response themes were categorized into 2 groups: outcomes of the Milestones implementation process, and facilitators and barriers. Because of Milestones implementation, participants reported changes to the quality of the assessment process, including the ability to identify struggling residents earlier and design individualized improvement plans. Some programs revised their curricula based on training gaps identified using the Milestones. Barriers to implementation included limitations to the adoption of a developmental progression model in the context of rotation block schedules and misalignment between progression targets and clinical experience. The shift from time-based to competency-based evaluation presented an ongoing adjustment for many programs. Organized preparation before clinical competency committee meetings and diverse clinical competency committee composition led to more productive meetings and perceived improvement in promotion decisions. Conclusions: The results of this study can be used by program leadership to help guide further implementation of the Milestones and program improvement. These results also help to guide the evolution of Milestones language and their implementation across specialties.