Measuring Preventive Care Delivery: Comparing Rates Across Three Data Sources

Steffani Bailey, John Heintzman, Miguel Marino, Megan J. Hoopes, Brigit A. Hatch, Rachel Gold, Stuart C. Cowburn, Christine A. Nelson, Heather E. Angier, Jennifer Devoe

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Introduction Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness. Methods This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015. Results Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96–1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83–0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67–0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services. Conclusions Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

Original languageEnglish (US)
Pages (from-to)752-761
Number of pages10
JournalAmerican Journal of Preventive Medicine
Volume51
Issue number5
DOIs
StatePublished - Nov 1 2016

Fingerprint

Preventive Medicine
Electronic Health Records
Information Storage and Retrieval
Medicaid
Early Detection of Cancer
Documentation
Colorectal Neoplasms
Breast Neoplasms
Community Health Centers
Chlamydia
Hyperlipidemias
Uterine Cervical Neoplasms
Human Influenza
Primary Health Care
Vaccination
Referral and Consultation
Cross-Sectional Studies
Blood Pressure

ASJC Scopus subject areas

  • Epidemiology
  • Public Health, Environmental and Occupational Health

Cite this

Measuring Preventive Care Delivery : Comparing Rates Across Three Data Sources. / Bailey, Steffani; Heintzman, John; Marino, Miguel; Hoopes, Megan J.; Hatch, Brigit A.; Gold, Rachel; Cowburn, Stuart C.; Nelson, Christine A.; Angier, Heather E.; Devoe, Jennifer.

In: American Journal of Preventive Medicine, Vol. 51, No. 5, 01.11.2016, p. 752-761.

Research output: Contribution to journalArticle

Bailey, Steffani ; Heintzman, John ; Marino, Miguel ; Hoopes, Megan J. ; Hatch, Brigit A. ; Gold, Rachel ; Cowburn, Stuart C. ; Nelson, Christine A. ; Angier, Heather E. ; Devoe, Jennifer. / Measuring Preventive Care Delivery : Comparing Rates Across Three Data Sources. In: American Journal of Preventive Medicine. 2016 ; Vol. 51, No. 5. pp. 752-761.
@article{1d7619897fda4156a699608610c01097,
title = "Measuring Preventive Care Delivery: Comparing Rates Across Three Data Sources",
abstract = "Introduction Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness. Methods This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015. Results Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96–1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83–0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67–0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services. Conclusions Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.",
author = "Steffani Bailey and John Heintzman and Miguel Marino and Hoopes, {Megan J.} and Hatch, {Brigit A.} and Rachel Gold and Cowburn, {Stuart C.} and Nelson, {Christine A.} and Angier, {Heather E.} and Jennifer Devoe",
year = "2016",
month = "11",
day = "1",
doi = "10.1016/j.amepre.2016.07.004",
language = "English (US)",
volume = "51",
pages = "752--761",
journal = "American Journal of Preventive Medicine",
issn = "0749-3797",
publisher = "Elsevier Inc.",
number = "5",

}

TY - JOUR

T1 - Measuring Preventive Care Delivery

T2 - Comparing Rates Across Three Data Sources

AU - Bailey, Steffani

AU - Heintzman, John

AU - Marino, Miguel

AU - Hoopes, Megan J.

AU - Hatch, Brigit A.

AU - Gold, Rachel

AU - Cowburn, Stuart C.

AU - Nelson, Christine A.

AU - Angier, Heather E.

AU - Devoe, Jennifer

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Introduction Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness. Methods This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015. Results Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96–1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83–0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67–0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services. Conclusions Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

AB - Introduction Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness. Methods This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015. Results Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96–1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83–0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67–0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services. Conclusions Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

UR - http://www.scopus.com/inward/record.url?scp=84994174602&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994174602&partnerID=8YFLogxK

U2 - 10.1016/j.amepre.2016.07.004

DO - 10.1016/j.amepre.2016.07.004

M3 - Article

C2 - 27522472

AN - SCOPUS:84994174602

VL - 51

SP - 752

EP - 761

JO - American Journal of Preventive Medicine

JF - American Journal of Preventive Medicine

SN - 0749-3797

IS - 5

ER -