Grading the strength of a body of evidence when assessing health care interventions: An EPC update

Nancy D. Berkman, Kathleen N. Lohr, Mohammed T. Ansari, Ethan M. Balk, Robert Kane, Marian McDonagh, Sally C. Morton, Meera Viswanathan, Eric B. Bass, Mary Butler, Gerald Gartlehner, Lisa Hartling, Melissa McPheeters, Laura C. Morgan, James Reston, Priyanka Sista, Evelyn Whitlock, Stephanie Chang

Research output: Contribution to journalArticle

60 Citations (Scopus)

Abstract

Objectives To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). Study Design and Setting A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. Results Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. Conclusion No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.

Original languageEnglish (US)
Pages (from-to)1312-1324
Number of pages13
JournalJournal of Clinical Epidemiology
Volume68
Issue number11
DOIs
StatePublished - Nov 1 2015

Fingerprint

Evidence-Based Practice
Delivery of Health Care
Observational Studies
United States Agency for Healthcare Research and Quality
Randomized Controlled Trials
Equipment and Supplies
Pharmaceutical Preparations
Therapeutics

Keywords

  • Clinical practice guidelines
  • Evidence-based practice
  • Health care delivery
  • Health policy
  • Methods
  • Minimally important differences
  • Optimal information size
  • Strength of evidence
  • Systematic reviews

ASJC Scopus subject areas

  • Epidemiology

Cite this

Grading the strength of a body of evidence when assessing health care interventions : An EPC update. / Berkman, Nancy D.; Lohr, Kathleen N.; Ansari, Mohammed T.; Balk, Ethan M.; Kane, Robert; McDonagh, Marian; Morton, Sally C.; Viswanathan, Meera; Bass, Eric B.; Butler, Mary; Gartlehner, Gerald; Hartling, Lisa; McPheeters, Melissa; Morgan, Laura C.; Reston, James; Sista, Priyanka; Whitlock, Evelyn; Chang, Stephanie.

In: Journal of Clinical Epidemiology, Vol. 68, No. 11, 01.11.2015, p. 1312-1324.

Research output: Contribution to journalArticle

Berkman, ND, Lohr, KN, Ansari, MT, Balk, EM, Kane, R, McDonagh, M, Morton, SC, Viswanathan, M, Bass, EB, Butler, M, Gartlehner, G, Hartling, L, McPheeters, M, Morgan, LC, Reston, J, Sista, P, Whitlock, E & Chang, S 2015, 'Grading the strength of a body of evidence when assessing health care interventions: An EPC update', Journal of Clinical Epidemiology, vol. 68, no. 11, pp. 1312-1324. https://doi.org/10.1016/j.jclinepi.2014.11.023
Berkman, Nancy D. ; Lohr, Kathleen N. ; Ansari, Mohammed T. ; Balk, Ethan M. ; Kane, Robert ; McDonagh, Marian ; Morton, Sally C. ; Viswanathan, Meera ; Bass, Eric B. ; Butler, Mary ; Gartlehner, Gerald ; Hartling, Lisa ; McPheeters, Melissa ; Morgan, Laura C. ; Reston, James ; Sista, Priyanka ; Whitlock, Evelyn ; Chang, Stephanie. / Grading the strength of a body of evidence when assessing health care interventions : An EPC update. In: Journal of Clinical Epidemiology. 2015 ; Vol. 68, No. 11. pp. 1312-1324.
@article{42f06c7da78748bda08a51e58eb8e214,
title = "Grading the strength of a body of evidence when assessing health care interventions: An EPC update",
abstract = "Objectives To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). Study Design and Setting A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. Results Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. Conclusion No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.",
keywords = "Clinical practice guidelines, Evidence-based practice, Health care delivery, Health policy, Methods, Minimally important differences, Optimal information size, Strength of evidence, Systematic reviews",
author = "Berkman, {Nancy D.} and Lohr, {Kathleen N.} and Ansari, {Mohammed T.} and Balk, {Ethan M.} and Robert Kane and Marian McDonagh and Morton, {Sally C.} and Meera Viswanathan and Bass, {Eric B.} and Mary Butler and Gerald Gartlehner and Lisa Hartling and Melissa McPheeters and Morgan, {Laura C.} and James Reston and Priyanka Sista and Evelyn Whitlock and Stephanie Chang",
year = "2015",
month = "11",
day = "1",
doi = "10.1016/j.jclinepi.2014.11.023",
language = "English (US)",
volume = "68",
pages = "1312--1324",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "11",

}

TY - JOUR

T1 - Grading the strength of a body of evidence when assessing health care interventions

T2 - An EPC update

AU - Berkman, Nancy D.

AU - Lohr, Kathleen N.

AU - Ansari, Mohammed T.

AU - Balk, Ethan M.

AU - Kane, Robert

AU - McDonagh, Marian

AU - Morton, Sally C.

AU - Viswanathan, Meera

AU - Bass, Eric B.

AU - Butler, Mary

AU - Gartlehner, Gerald

AU - Hartling, Lisa

AU - McPheeters, Melissa

AU - Morgan, Laura C.

AU - Reston, James

AU - Sista, Priyanka

AU - Whitlock, Evelyn

AU - Chang, Stephanie

PY - 2015/11/1

Y1 - 2015/11/1

N2 - Objectives To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). Study Design and Setting A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. Results Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. Conclusion No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.

AB - Objectives To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). Study Design and Setting A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. Results Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. Conclusion No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.

KW - Clinical practice guidelines

KW - Evidence-based practice

KW - Health care delivery

KW - Health policy

KW - Methods

KW - Minimally important differences

KW - Optimal information size

KW - Strength of evidence

KW - Systematic reviews

UR - http://www.scopus.com/inward/record.url?scp=84945490686&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84945490686&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2014.11.023

DO - 10.1016/j.jclinepi.2014.11.023

M3 - Article

C2 - 25721570

AN - SCOPUS:84945490686

VL - 68

SP - 1312

EP - 1324

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 11

ER -