The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain

Eric S. Johnson, John F. Dickerson, William M. Vollmer, Alee M. Rowley, Cheryl Ritenbaugh, Richard (Rick) Deyo, Lynn DeBar

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Background: Propensity scores are typically applied in retrospective cohort studies. We describe the feasibility of matching on a propensity score derived from a retrospective cohort and subsequently applied in a prospective cohort study of patients with chronic musculoskeletal pain before the start of acupuncture or usual care treatment and enrollment in a comparative effectiveness study that required patient reported pain outcomes. Methods: We assembled a retrospective cohort study using data from 2010 to develop a propensity score for acupuncture versus usual care based on electronic healthcare record and administrative data (e.g., pharmacy) from an integrated health plan, Kaiser Permanente Northwest. The propensity score’s probabilities allowed us to match acupuncture-referred and non-referred patients prospectively in 2013-14 after a routine outpatient visit for pain. Among the matched patients, we collected patient-reported pain before treatment and during follow-up to assess the comparative effectiveness of acupuncture. We assessed balance in patient characteristics with the post-matching c-statistic and standardized differences. Results: Based on the propensity score and other characteristics (e.g., patient-reported pain), we were able to match all 173 acupuncture-referred patients to 350 non-referred (usual care) patients. We observed a residual imbalance (based on the standardized differences) for some characteristics that contributed to the score; for example, age, -0.283, and the Charlson comorbidity score, -0.264, had the largest standardized differences. The overall balance of the propensity score appeared more favorable according to the post-matching c-statistic, 0.503. Conclusion: The propensity score matching was feasible statistically and logistically and allowed approximate balance on patient characteristics, some of which will require adjustment in the comparative effectiveness regression model. By transporting propensity scores to new patients, healthcare systems with electronic health records can conduct comparative effectiveness cohort studies that require prospective data collection, such as patient-reported outcomes, while approximately balancing numerous patient characteristics that might confound the benefit of an intervention. The approach offers a new study design option.

Original languageEnglish (US)
Article number42
JournalBMC Medical Research Methodology
Volume17
Issue number1
DOIs
StatePublished - Mar 16 2017

Fingerprint

Propensity Score
Acupuncture
Chronic Pain
Cohort Studies
Prospective Studies
Pain
Retrospective Studies
Delivery of Health Care
Social Adjustment
Musculoskeletal Pain
Electronic Health Records
Comorbidity
Patient Care
Outpatients

Keywords

  • Acupuncture
  • Comparative effectiveness
  • Pain
  • Propensity score
  • Prospective cohort

ASJC Scopus subject areas

  • Epidemiology
  • Health Informatics

Cite this

The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain. / Johnson, Eric S.; Dickerson, John F.; Vollmer, William M.; Rowley, Alee M.; Ritenbaugh, Cheryl; Deyo, Richard (Rick); DeBar, Lynn.

In: BMC Medical Research Methodology, Vol. 17, No. 1, 42, 16.03.2017.

Research output: Contribution to journalArticle

Johnson, Eric S. ; Dickerson, John F. ; Vollmer, William M. ; Rowley, Alee M. ; Ritenbaugh, Cheryl ; Deyo, Richard (Rick) ; DeBar, Lynn. / The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain. In: BMC Medical Research Methodology. 2017 ; Vol. 17, No. 1.
@article{18ed71243fc048cd900899d40a8e13f7,
title = "The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain",
abstract = "Background: Propensity scores are typically applied in retrospective cohort studies. We describe the feasibility of matching on a propensity score derived from a retrospective cohort and subsequently applied in a prospective cohort study of patients with chronic musculoskeletal pain before the start of acupuncture or usual care treatment and enrollment in a comparative effectiveness study that required patient reported pain outcomes. Methods: We assembled a retrospective cohort study using data from 2010 to develop a propensity score for acupuncture versus usual care based on electronic healthcare record and administrative data (e.g., pharmacy) from an integrated health plan, Kaiser Permanente Northwest. The propensity score’s probabilities allowed us to match acupuncture-referred and non-referred patients prospectively in 2013-14 after a routine outpatient visit for pain. Among the matched patients, we collected patient-reported pain before treatment and during follow-up to assess the comparative effectiveness of acupuncture. We assessed balance in patient characteristics with the post-matching c-statistic and standardized differences. Results: Based on the propensity score and other characteristics (e.g., patient-reported pain), we were able to match all 173 acupuncture-referred patients to 350 non-referred (usual care) patients. We observed a residual imbalance (based on the standardized differences) for some characteristics that contributed to the score; for example, age, -0.283, and the Charlson comorbidity score, -0.264, had the largest standardized differences. The overall balance of the propensity score appeared more favorable according to the post-matching c-statistic, 0.503. Conclusion: The propensity score matching was feasible statistically and logistically and allowed approximate balance on patient characteristics, some of which will require adjustment in the comparative effectiveness regression model. By transporting propensity scores to new patients, healthcare systems with electronic health records can conduct comparative effectiveness cohort studies that require prospective data collection, such as patient-reported outcomes, while approximately balancing numerous patient characteristics that might confound the benefit of an intervention. The approach offers a new study design option.",
keywords = "Acupuncture, Comparative effectiveness, Pain, Propensity score, Prospective cohort",
author = "Johnson, {Eric S.} and Dickerson, {John F.} and Vollmer, {William M.} and Rowley, {Alee M.} and Cheryl Ritenbaugh and Deyo, {Richard (Rick)} and Lynn DeBar",
year = "2017",
month = "3",
day = "16",
doi = "10.1186/s12874-017-0318-4",
language = "English (US)",
volume = "17",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - The feasibility of matching on a propensity score for acupuncture in a prospective cohort study of patients with chronic pain

AU - Johnson, Eric S.

AU - Dickerson, John F.

AU - Vollmer, William M.

AU - Rowley, Alee M.

AU - Ritenbaugh, Cheryl

AU - Deyo, Richard (Rick)

AU - DeBar, Lynn

PY - 2017/3/16

Y1 - 2017/3/16

N2 - Background: Propensity scores are typically applied in retrospective cohort studies. We describe the feasibility of matching on a propensity score derived from a retrospective cohort and subsequently applied in a prospective cohort study of patients with chronic musculoskeletal pain before the start of acupuncture or usual care treatment and enrollment in a comparative effectiveness study that required patient reported pain outcomes. Methods: We assembled a retrospective cohort study using data from 2010 to develop a propensity score for acupuncture versus usual care based on electronic healthcare record and administrative data (e.g., pharmacy) from an integrated health plan, Kaiser Permanente Northwest. The propensity score’s probabilities allowed us to match acupuncture-referred and non-referred patients prospectively in 2013-14 after a routine outpatient visit for pain. Among the matched patients, we collected patient-reported pain before treatment and during follow-up to assess the comparative effectiveness of acupuncture. We assessed balance in patient characteristics with the post-matching c-statistic and standardized differences. Results: Based on the propensity score and other characteristics (e.g., patient-reported pain), we were able to match all 173 acupuncture-referred patients to 350 non-referred (usual care) patients. We observed a residual imbalance (based on the standardized differences) for some characteristics that contributed to the score; for example, age, -0.283, and the Charlson comorbidity score, -0.264, had the largest standardized differences. The overall balance of the propensity score appeared more favorable according to the post-matching c-statistic, 0.503. Conclusion: The propensity score matching was feasible statistically and logistically and allowed approximate balance on patient characteristics, some of which will require adjustment in the comparative effectiveness regression model. By transporting propensity scores to new patients, healthcare systems with electronic health records can conduct comparative effectiveness cohort studies that require prospective data collection, such as patient-reported outcomes, while approximately balancing numerous patient characteristics that might confound the benefit of an intervention. The approach offers a new study design option.

AB - Background: Propensity scores are typically applied in retrospective cohort studies. We describe the feasibility of matching on a propensity score derived from a retrospective cohort and subsequently applied in a prospective cohort study of patients with chronic musculoskeletal pain before the start of acupuncture or usual care treatment and enrollment in a comparative effectiveness study that required patient reported pain outcomes. Methods: We assembled a retrospective cohort study using data from 2010 to develop a propensity score for acupuncture versus usual care based on electronic healthcare record and administrative data (e.g., pharmacy) from an integrated health plan, Kaiser Permanente Northwest. The propensity score’s probabilities allowed us to match acupuncture-referred and non-referred patients prospectively in 2013-14 after a routine outpatient visit for pain. Among the matched patients, we collected patient-reported pain before treatment and during follow-up to assess the comparative effectiveness of acupuncture. We assessed balance in patient characteristics with the post-matching c-statistic and standardized differences. Results: Based on the propensity score and other characteristics (e.g., patient-reported pain), we were able to match all 173 acupuncture-referred patients to 350 non-referred (usual care) patients. We observed a residual imbalance (based on the standardized differences) for some characteristics that contributed to the score; for example, age, -0.283, and the Charlson comorbidity score, -0.264, had the largest standardized differences. The overall balance of the propensity score appeared more favorable according to the post-matching c-statistic, 0.503. Conclusion: The propensity score matching was feasible statistically and logistically and allowed approximate balance on patient characteristics, some of which will require adjustment in the comparative effectiveness regression model. By transporting propensity scores to new patients, healthcare systems with electronic health records can conduct comparative effectiveness cohort studies that require prospective data collection, such as patient-reported outcomes, while approximately balancing numerous patient characteristics that might confound the benefit of an intervention. The approach offers a new study design option.

KW - Acupuncture

KW - Comparative effectiveness

KW - Pain

KW - Propensity score

KW - Prospective cohort

UR - http://www.scopus.com/inward/record.url?scp=85015168260&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015168260&partnerID=8YFLogxK

U2 - 10.1186/s12874-017-0318-4

DO - 10.1186/s12874-017-0318-4

M3 - Article

VL - 17

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

M1 - 42

ER -