The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain

Background: Propensity scores are typically applied in retrospective cohort studies. We describe the feasibility of matching on a propensity score derived from a retrospective cohort and subsequently applied in a prospective cohort study of patients with chronic musculoskeletal pain before the start of acupuncture or usual care treatment and enrollment in a comparative effectiveness study that required patient reported pain outcomes. Methods: We assembled a retrospective cohort study using data from 2010 to develop a propensity score for acupuncture versus usual care based on electronic healthcare record and administrative data (e.g., pharmacy) from an integrated health plan, Kaiser Permanente Northwest. The propensity score’s probabilities allowed us to match acupuncture-referred and non-referred patients prospectively in 2013-14 after a routine outpatient visit for pain. Among the matched patients, we collected patient-reported pain before treatment and during follow-up to assess the comparative effectiveness of acupuncture. We assessed balance in patient characteristics with the post-matching c-statistic and standardized differences. Results: Based on the propensity score and other characteristics (e.g., patient-reported pain), we were able to match all 173 acupuncture-referred patients to 350 non-referred (usual care) patients. We observed a residual imbalance (based on the standardized differences) for some characteristics that contributed to the score; for example, age, -0.283, and the Charlson comorbidity score, -0.264, had the largest standardized differences. The overall balance of the propensity score appeared more favorable according to the post-matching c-statistic, 0.503. Conclusion: The propensity score matching was feasible statistically and logistically and allowed approximate balance on patient characteristics, some of which will require adjustment in the comparative effectiveness regression model. By transporting propensity scores to new patients, healthcare systems with electronic health records can conduct comparative effectiveness cohort studies that require prospective data collection, such as patient-reported outcomes, while approximately balancing numerous patient characteristics that might confound the benefit of an intervention. The approach offers a new study design option.