Living systematic reviews

3. Statistical methods for updating meta-analyses

Living Systematic Review Network

Research output: Contribution to journalReview article

18 Citations (Scopus)

Abstract

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Original languageEnglish (US)
Pages (from-to)38-46
Number of pages9
JournalJournal of Clinical Epidemiology
Volume91
DOIs
StatePublished - Nov 1 2017

Fingerprint

Meta-Analysis
Economic Inflation
Decision Making
Research

Keywords

  • Heterogeneity
  • Living systematic review
  • Meta-analysis
  • Type I error
  • Type II error

ASJC Scopus subject areas

  • Epidemiology

Cite this

Living systematic reviews : 3. Statistical methods for updating meta-analyses. / Living Systematic Review Network.

In: Journal of Clinical Epidemiology, Vol. 91, 01.11.2017, p. 38-46.

Research output: Contribution to journalReview article

@article{a1446a32f31941858f8414459c9a6e13,
title = "Living systematic reviews: 3. Statistical methods for updating meta-analyses",
abstract = "A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.",
keywords = "Heterogeneity, Living systematic review, Meta-analysis, Type I error, Type II error",
author = "{Living Systematic Review Network} and Mark Simmonds and Georgia Salanti and Joanne McKenzie and Julian Elliott and Thomas Agoritsas and John Hilton and Caroline Perron and Elie Akl and Rebecca Hodder and Charlotte Pestridge and Lauren Albrecht and Tanya Horsley and Joanne Platt and Rebecca Armstrong and Nguyen, {Phi Hung} and Robert Plovnick and Anneliese Arno and Noah Ivers and Gail Quinn and Agnes Au and Renea Johnston and Gabriel Rada and Matthew Bagg and Arwel Jones and Philippe Ravaud and Catherine Boden and Lara Kahale and Bernt Richter and Isabelle Boisvert and Homa Keshavarz and Rebecca Ryan and Linn Brandt and Kolakowsky-Hayner, {Stephanie A.} and Dina Salama and Alexandra Brazinova and Nagraj, {Sumanth Kumbargere} and Georgia Salanti and Rachelle Buchbinder and Toby Lasserson and Lina Santaguida and Chris Champion and Rebecca Lawrence and Nancy Santesso and Jackie Chandler and Zbigniew Les and Sch{\"u}nemann, {Holger J.} and Andreas Charidimou and Stefan Leucht and Roger Chou and Mark Helfand",
year = "2017",
month = "11",
day = "1",
doi = "10.1016/j.jclinepi.2017.08.008",
language = "English (US)",
volume = "91",
pages = "38--46",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Living systematic reviews

T2 - 3. Statistical methods for updating meta-analyses

AU - Living Systematic Review Network

AU - Simmonds, Mark

AU - Salanti, Georgia

AU - McKenzie, Joanne

AU - Elliott, Julian

AU - Agoritsas, Thomas

AU - Hilton, John

AU - Perron, Caroline

AU - Akl, Elie

AU - Hodder, Rebecca

AU - Pestridge, Charlotte

AU - Albrecht, Lauren

AU - Horsley, Tanya

AU - Platt, Joanne

AU - Armstrong, Rebecca

AU - Nguyen, Phi Hung

AU - Plovnick, Robert

AU - Arno, Anneliese

AU - Ivers, Noah

AU - Quinn, Gail

AU - Au, Agnes

AU - Johnston, Renea

AU - Rada, Gabriel

AU - Bagg, Matthew

AU - Jones, Arwel

AU - Ravaud, Philippe

AU - Boden, Catherine

AU - Kahale, Lara

AU - Richter, Bernt

AU - Boisvert, Isabelle

AU - Keshavarz, Homa

AU - Ryan, Rebecca

AU - Brandt, Linn

AU - Kolakowsky-Hayner, Stephanie A.

AU - Salama, Dina

AU - Brazinova, Alexandra

AU - Nagraj, Sumanth Kumbargere

AU - Salanti, Georgia

AU - Buchbinder, Rachelle

AU - Lasserson, Toby

AU - Santaguida, Lina

AU - Champion, Chris

AU - Lawrence, Rebecca

AU - Santesso, Nancy

AU - Chandler, Jackie

AU - Les, Zbigniew

AU - Schünemann, Holger J.

AU - Charidimou, Andreas

AU - Leucht, Stefan

AU - Chou, Roger

AU - Helfand, Mark

PY - 2017/11/1

Y1 - 2017/11/1

N2 - A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

AB - A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

KW - Heterogeneity

KW - Living systematic review

KW - Meta-analysis

KW - Type I error

KW - Type II error

UR - http://www.scopus.com/inward/record.url?scp=85028983900&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028983900&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.08.008

DO - 10.1016/j.jclinepi.2017.08.008

M3 - Review article

VL - 91

SP - 38

EP - 46

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -