Predicting performance on the American Board of Surgery qualifying and certifying examinations: A multi-institutional study

Background: We sought to determine whether US Medical Licensing Examination (USMLE) Step 1 score, American Board of Surgery (ABS) In-Training Examination (ABSITE) score, and other variables are associated with failing the ABS qualifying and certifying examinations. Identifying such factors may assist in the early implementation of an academic intervention for at-risk residents. Design: Retrospective review. Setting: Seventeen general surgery training programs in the western United States. Participants: Six hundred seven residents who graduated in 2000-2007. Main Outcome Measures: First-time pass rates on the qualifying and certifying examinations, US vs non-US medical school graduation, USMLE Steps 1 and 2 scores, ABSITE scores, operative case volume, fellowship training, residency program type, and mandatory research. Results: The first-time qualifying and certifying examination pass rates for the 607 graduating residents were 78% and 74%, respectively. On multivariable analysis, scoring below the 35th percentile on the ABSITE at any time during residency was associated with an increased risk of failing both examinations (odds ratio, 0.23 [95% confidence interval, 0.08-0.68] for the qualifying examination and 0.35 [0.20-0.61] for the certifying examination), as was scoring less than 200 on the USMLE Step 1 (0.36 [0.21-0.62] for the qualifying examination and 0.62 [0.42-0.93] for the certifying examination). A mandatory research year was associated with an increased likelihood of passing the certifying examination (odds ratio, 3.3 [95% confidence interval, 1.6-6.8]). Conclusions: Residents who are more likely to fail the ABS qualifying and certifying examinations can be identified by a low USMLE Step 1 score and by poor performance on the ABSITE at any time during residency. These findings support the use of the USMLE Step 1 score in the surgical residency selection process and a formal academic intervention for residents who perform poorly on the ABSITE.