Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program

Susan L. Norris, David Atkins, Wendy Bruening, Steven Fox, Eric Johnson, Robert Kane, Sally C. Morton, Mark Oremus, Maria Ospina, Gurvaneet Randhawa, Karen Schoelles, Paul Shekelle, Meera Viswanathan

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

Objective: Systematic reviewers disagree about the ability of observational studies to answer questions about the benefits or intended effects of pharmacotherapeutic, device, or procedural interventions. This study provides a framework for decision making on the inclusion of observational studies to assess benefits and intended effects in comparative effectiveness reviews (CERs). Study Design and Setting: The conceptual model and recommendations were developed using a consensus process by members of the methods workgroup of the Effective Health Care Program of the Agency for Healthcare Research and Quality. Results: In considering whether to use observational studies in CERs for addressing beneficial effects, reviewers should answer two questions: (1) Are there gaps in the evidence from randomized controlled trials (RCTs)? (2) Will observational studies provide valid and useful information? The latter question involves the following: (a) refocusing the study questions on gaps in the evidence from RCTs, (b) assessing the risk of bias of the body of evidence of observational studies, and (c) assessing whether available observational studies address the gap review questions. Conclusions: Because it is unusual to find sufficient evidence from RCTs to answer all key questions concerning benefit or the balance of benefits and harms, comparative effectiveness reviewers should routinely assess the appropriateness of inclusion of observational studies for questions of benefit. Furthermore, reviewers should explicitly state the rationale for inclusion or exclusion of observational studies when conducting CERs.

Original languageEnglish (US)
Pages (from-to)1178-1186
Number of pages9
JournalJournal of Clinical Epidemiology
Volume64
Issue number11
DOIs
StatePublished - Nov 2011

Fingerprint

Observational Studies
Delivery of Health Care
Randomized Controlled Trials
Aptitude
Health Services Research
Consensus
Decision Making
Equipment and Supplies

Keywords

  • Evidence-based medicine, Research design
  • Observational studies
  • Systematic reviews, Meta-analyses, Treatment outcome

ASJC Scopus subject areas

  • Epidemiology

Cite this

Observational studies in systemic reviews of comparative effectiveness : AHRQ and the Effective Health Care Program. / Norris, Susan L.; Atkins, David; Bruening, Wendy; Fox, Steven; Johnson, Eric; Kane, Robert; Morton, Sally C.; Oremus, Mark; Ospina, Maria; Randhawa, Gurvaneet; Schoelles, Karen; Shekelle, Paul; Viswanathan, Meera.

In: Journal of Clinical Epidemiology, Vol. 64, No. 11, 11.2011, p. 1178-1186.

Research output: Contribution to journalArticle

Norris, SL, Atkins, D, Bruening, W, Fox, S, Johnson, E, Kane, R, Morton, SC, Oremus, M, Ospina, M, Randhawa, G, Schoelles, K, Shekelle, P & Viswanathan, M 2011, 'Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program', Journal of Clinical Epidemiology, vol. 64, no. 11, pp. 1178-1186. https://doi.org/10.1016/j.jclinepi.2010.04.027
Norris, Susan L. ; Atkins, David ; Bruening, Wendy ; Fox, Steven ; Johnson, Eric ; Kane, Robert ; Morton, Sally C. ; Oremus, Mark ; Ospina, Maria ; Randhawa, Gurvaneet ; Schoelles, Karen ; Shekelle, Paul ; Viswanathan, Meera. / Observational studies in systemic reviews of comparative effectiveness : AHRQ and the Effective Health Care Program. In: Journal of Clinical Epidemiology. 2011 ; Vol. 64, No. 11. pp. 1178-1186.
@article{541d5020ec124bc6bca03f19152eac94,
title = "Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program",
abstract = "Objective: Systematic reviewers disagree about the ability of observational studies to answer questions about the benefits or intended effects of pharmacotherapeutic, device, or procedural interventions. This study provides a framework for decision making on the inclusion of observational studies to assess benefits and intended effects in comparative effectiveness reviews (CERs). Study Design and Setting: The conceptual model and recommendations were developed using a consensus process by members of the methods workgroup of the Effective Health Care Program of the Agency for Healthcare Research and Quality. Results: In considering whether to use observational studies in CERs for addressing beneficial effects, reviewers should answer two questions: (1) Are there gaps in the evidence from randomized controlled trials (RCTs)? (2) Will observational studies provide valid and useful information? The latter question involves the following: (a) refocusing the study questions on gaps in the evidence from RCTs, (b) assessing the risk of bias of the body of evidence of observational studies, and (c) assessing whether available observational studies address the gap review questions. Conclusions: Because it is unusual to find sufficient evidence from RCTs to answer all key questions concerning benefit or the balance of benefits and harms, comparative effectiveness reviewers should routinely assess the appropriateness of inclusion of observational studies for questions of benefit. Furthermore, reviewers should explicitly state the rationale for inclusion or exclusion of observational studies when conducting CERs.",
keywords = "Evidence-based medicine, Research design, Observational studies, Systematic reviews, Meta-analyses, Treatment outcome",
author = "Norris, {Susan L.} and David Atkins and Wendy Bruening and Steven Fox and Eric Johnson and Robert Kane and Morton, {Sally C.} and Mark Oremus and Maria Ospina and Gurvaneet Randhawa and Karen Schoelles and Paul Shekelle and Meera Viswanathan",
year = "2011",
month = "11",
doi = "10.1016/j.jclinepi.2010.04.027",
language = "English (US)",
volume = "64",
pages = "1178--1186",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "11",

}

TY - JOUR

T1 - Observational studies in systemic reviews of comparative effectiveness

T2 - AHRQ and the Effective Health Care Program

AU - Norris, Susan L.

AU - Atkins, David

AU - Bruening, Wendy

AU - Fox, Steven

AU - Johnson, Eric

AU - Kane, Robert

AU - Morton, Sally C.

AU - Oremus, Mark

AU - Ospina, Maria

AU - Randhawa, Gurvaneet

AU - Schoelles, Karen

AU - Shekelle, Paul

AU - Viswanathan, Meera

PY - 2011/11

Y1 - 2011/11

N2 - Objective: Systematic reviewers disagree about the ability of observational studies to answer questions about the benefits or intended effects of pharmacotherapeutic, device, or procedural interventions. This study provides a framework for decision making on the inclusion of observational studies to assess benefits and intended effects in comparative effectiveness reviews (CERs). Study Design and Setting: The conceptual model and recommendations were developed using a consensus process by members of the methods workgroup of the Effective Health Care Program of the Agency for Healthcare Research and Quality. Results: In considering whether to use observational studies in CERs for addressing beneficial effects, reviewers should answer two questions: (1) Are there gaps in the evidence from randomized controlled trials (RCTs)? (2) Will observational studies provide valid and useful information? The latter question involves the following: (a) refocusing the study questions on gaps in the evidence from RCTs, (b) assessing the risk of bias of the body of evidence of observational studies, and (c) assessing whether available observational studies address the gap review questions. Conclusions: Because it is unusual to find sufficient evidence from RCTs to answer all key questions concerning benefit or the balance of benefits and harms, comparative effectiveness reviewers should routinely assess the appropriateness of inclusion of observational studies for questions of benefit. Furthermore, reviewers should explicitly state the rationale for inclusion or exclusion of observational studies when conducting CERs.

AB - Objective: Systematic reviewers disagree about the ability of observational studies to answer questions about the benefits or intended effects of pharmacotherapeutic, device, or procedural interventions. This study provides a framework for decision making on the inclusion of observational studies to assess benefits and intended effects in comparative effectiveness reviews (CERs). Study Design and Setting: The conceptual model and recommendations were developed using a consensus process by members of the methods workgroup of the Effective Health Care Program of the Agency for Healthcare Research and Quality. Results: In considering whether to use observational studies in CERs for addressing beneficial effects, reviewers should answer two questions: (1) Are there gaps in the evidence from randomized controlled trials (RCTs)? (2) Will observational studies provide valid and useful information? The latter question involves the following: (a) refocusing the study questions on gaps in the evidence from RCTs, (b) assessing the risk of bias of the body of evidence of observational studies, and (c) assessing whether available observational studies address the gap review questions. Conclusions: Because it is unusual to find sufficient evidence from RCTs to answer all key questions concerning benefit or the balance of benefits and harms, comparative effectiveness reviewers should routinely assess the appropriateness of inclusion of observational studies for questions of benefit. Furthermore, reviewers should explicitly state the rationale for inclusion or exclusion of observational studies when conducting CERs.

KW - Evidence-based medicine, Research design

KW - Observational studies

KW - Systematic reviews, Meta-analyses, Treatment outcome

UR - http://www.scopus.com/inward/record.url?scp=80053346464&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053346464&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2010.04.027

DO - 10.1016/j.jclinepi.2010.04.027

M3 - Article

C2 - 21636246

AN - SCOPUS:80053346464

VL - 64

SP - 1178

EP - 1186

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 11

ER -