Adapting SQUIRE 2.0 to Create a Quality Improvement Evidence-Based Medicine Critical Appraisal Tool (QI-EBM-CAT) for Graduate Medical Education Trainees

Background Evidence-based medicine (EBM) has long been taught to physician trainees for critical appraisal of research manuscripts. There is no parallel or similar framework to guide trainees in the appraisal of quality improvement (QI) literature. Objective To adapt existing guidelines of QI manuscript reporting into an educational QI-EBM appraisal tool to help residents distinguish research and QI manuscripts, assess QI designs and methodologies, and evaluate QI manuscripts’ strengths and weaknesses. Methods Between 2018 and 2021, we developed a QI-EBM critical appraisal tool (QI-EBM-CAT) and performed 3 plan-do-study-act cycles to refine the tool based on JAMA and SQUIRE 2.0 guidelines. We then surveyed residents regarding the usefulness of the tool and their confidence in evaluating QI manuscripts before and after completing a QI-EBM workshop using the QI appraisal tool. Results Sixty-six of 74 internal medicine postgraduate year (PGY)-1 to PGY-3 residents (89.2%) completed the workshop and assessment surveys in 2021. The workshop was found to be moderately to very useful by 85.1% (63 of 74) of residents as a framework for QI manuscript critical analysis. The summary confidence score in QI manuscript critical appraisal improved from a 64% rating of moderately to very confident in the pre-period to 94.6% in the post-period (P,.001) with statistical improvements in all 5 confidence areas assessed (P,.001). Conclusions The QI-EBM-CAT, designed to teach residents how to critically assess QI manuscripts using EBM principles, resulted in subjective improvements in confidence of QI manuscript analysis.