Observational evidence and strength of evidence domains: Case examples

Maya O'Neil, Nancy Berkman, Lisa Hartling, Stephanie Chang, Johanna Anderson, Makalapua Motu'apuaka, Jeanne-Marie Guise, Marian McDonagh

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Background: Systematic reviews of healthcare interventions most often focus on randomized controlled trials (RCTs). However, certain circumstances warrant consideration of observational evidence, and such studies are increasingly being included as evidence in systematic reviews. Methods: To illustrate the use of observational evidence, we present case examples of systematic reviews in which observational evidence was considered as well as case examples of individual observational studies, and how they demonstrate various strength of evidence domains in accordance with current Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) methods guidance. Results: In the presented examples, observational evidence is used when RCTs are infeasible or raise ethical concerns, lack generalizability, or provide insufficient data. Individual study case examples highlight how observational evidence may fulfill required strength of evidence domains, such as study limitations (reduced risk of selection, detection, performance, and attrition); directness; consistency; precision; and reporting bias (publication, selective outcome reporting, and selective analysis reporting), as well as additional domains of dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect).Conclusions: The cases highlighted in this paper demonstrate how observational studies may provide moderate to (rarely) high strength evidence in systematic reviews.

Original languageEnglish (US)
Article number35
JournalSystematic Reviews
Volume3
Issue number1
DOIs
StatePublished - Apr 23 2014

Fingerprint

Observational Studies
Randomized Controlled Trials
Publication Bias
Evidence-Based Practice
Health Services Research
Delivery of Health Care

Keywords

  • AHRQ Effective Health Care Program
  • Case series
  • Case-control studies
  • Cross-sectional studies
  • Integrative reviews
  • Mixed methods reviews
  • Non-randomized studies
  • Observational studies
  • Strength of evidence
  • Systematic reviews

ASJC Scopus subject areas

  • Medicine (miscellaneous)

Cite this

Observational evidence and strength of evidence domains : Case examples. / O'Neil, Maya; Berkman, Nancy; Hartling, Lisa; Chang, Stephanie; Anderson, Johanna; Motu'apuaka, Makalapua; Guise, Jeanne-Marie; McDonagh, Marian.

In: Systematic Reviews, Vol. 3, No. 1, 35, 23.04.2014.

Research output: Contribution to journalArticle

O'Neil M, Berkman N, Hartling L, Chang S, Anderson J, Motu'apuaka M et al. Observational evidence and strength of evidence domains: Case examples. Systematic Reviews. 2014 Apr 23;3(1). 35. https://doi.org/10.1186/2046-4053-3-35
O'Neil, Maya ; Berkman, Nancy ; Hartling, Lisa ; Chang, Stephanie ; Anderson, Johanna ; Motu'apuaka, Makalapua ; Guise, Jeanne-Marie ; McDonagh, Marian. / Observational evidence and strength of evidence domains : Case examples. In: Systematic Reviews. 2014 ; Vol. 3, No. 1.
@article{a903f520c95445769bb5c3840f7124f7,
title = "Observational evidence and strength of evidence domains: Case examples",
abstract = "Background: Systematic reviews of healthcare interventions most often focus on randomized controlled trials (RCTs). However, certain circumstances warrant consideration of observational evidence, and such studies are increasingly being included as evidence in systematic reviews. Methods: To illustrate the use of observational evidence, we present case examples of systematic reviews in which observational evidence was considered as well as case examples of individual observational studies, and how they demonstrate various strength of evidence domains in accordance with current Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) methods guidance. Results: In the presented examples, observational evidence is used when RCTs are infeasible or raise ethical concerns, lack generalizability, or provide insufficient data. Individual study case examples highlight how observational evidence may fulfill required strength of evidence domains, such as study limitations (reduced risk of selection, detection, performance, and attrition); directness; consistency; precision; and reporting bias (publication, selective outcome reporting, and selective analysis reporting), as well as additional domains of dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect).Conclusions: The cases highlighted in this paper demonstrate how observational studies may provide moderate to (rarely) high strength evidence in systematic reviews.",
keywords = "AHRQ Effective Health Care Program, Case series, Case-control studies, Cross-sectional studies, Integrative reviews, Mixed methods reviews, Non-randomized studies, Observational studies, Strength of evidence, Systematic reviews",
author = "Maya O'Neil and Nancy Berkman and Lisa Hartling and Stephanie Chang and Johanna Anderson and Makalapua Motu'apuaka and Jeanne-Marie Guise and Marian McDonagh",
year = "2014",
month = "4",
day = "23",
doi = "10.1186/2046-4053-3-35",
language = "English (US)",
volume = "3",
journal = "Systematic Reviews",
issn = "2046-4053",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Observational evidence and strength of evidence domains

T2 - Case examples

AU - O'Neil, Maya

AU - Berkman, Nancy

AU - Hartling, Lisa

AU - Chang, Stephanie

AU - Anderson, Johanna

AU - Motu'apuaka, Makalapua

AU - Guise, Jeanne-Marie

AU - McDonagh, Marian

PY - 2014/4/23

Y1 - 2014/4/23

N2 - Background: Systematic reviews of healthcare interventions most often focus on randomized controlled trials (RCTs). However, certain circumstances warrant consideration of observational evidence, and such studies are increasingly being included as evidence in systematic reviews. Methods: To illustrate the use of observational evidence, we present case examples of systematic reviews in which observational evidence was considered as well as case examples of individual observational studies, and how they demonstrate various strength of evidence domains in accordance with current Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) methods guidance. Results: In the presented examples, observational evidence is used when RCTs are infeasible or raise ethical concerns, lack generalizability, or provide insufficient data. Individual study case examples highlight how observational evidence may fulfill required strength of evidence domains, such as study limitations (reduced risk of selection, detection, performance, and attrition); directness; consistency; precision; and reporting bias (publication, selective outcome reporting, and selective analysis reporting), as well as additional domains of dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect).Conclusions: The cases highlighted in this paper demonstrate how observational studies may provide moderate to (rarely) high strength evidence in systematic reviews.

AB - Background: Systematic reviews of healthcare interventions most often focus on randomized controlled trials (RCTs). However, certain circumstances warrant consideration of observational evidence, and such studies are increasingly being included as evidence in systematic reviews. Methods: To illustrate the use of observational evidence, we present case examples of systematic reviews in which observational evidence was considered as well as case examples of individual observational studies, and how they demonstrate various strength of evidence domains in accordance with current Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) methods guidance. Results: In the presented examples, observational evidence is used when RCTs are infeasible or raise ethical concerns, lack generalizability, or provide insufficient data. Individual study case examples highlight how observational evidence may fulfill required strength of evidence domains, such as study limitations (reduced risk of selection, detection, performance, and attrition); directness; consistency; precision; and reporting bias (publication, selective outcome reporting, and selective analysis reporting), as well as additional domains of dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect).Conclusions: The cases highlighted in this paper demonstrate how observational studies may provide moderate to (rarely) high strength evidence in systematic reviews.

KW - AHRQ Effective Health Care Program

KW - Case series

KW - Case-control studies

KW - Cross-sectional studies

KW - Integrative reviews

KW - Mixed methods reviews

KW - Non-randomized studies

KW - Observational studies

KW - Strength of evidence

KW - Systematic reviews

UR - http://www.scopus.com/inward/record.url?scp=84907273489&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907273489&partnerID=8YFLogxK

U2 - 10.1186/2046-4053-3-35

DO - 10.1186/2046-4053-3-35

M3 - Article

C2 - 24758494

AN - SCOPUS:84907273489

VL - 3

JO - Systematic Reviews

JF - Systematic Reviews

SN - 2046-4053

IS - 1

M1 - 35

ER -