Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs

S. C. Hay, H. Kirpalani, C. Viner, R. Soll, Dmitry Dukhovny, W. Y. Mao, J. Profit, S. B. DeMauro, J. A.F. Zupancic

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Objective:To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.Study Design:We selected meta-Analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-Analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.Results:For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ≥20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.Conclusion:Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.

Original languageEnglish (US)
Pages (from-to)1215-1219
Number of pages5
JournalJournal of Perinatology
Volume37
Issue number11
DOIs
StatePublished - Nov 1 2017

Fingerprint

Uncertainty
Meta-Analysis
Confidence Intervals
Research Peer Review
Randomized Controlled Trials
Odds Ratio
Aptitude
Mortality
Advisory Committees
Logistic Models
Research Personnel
Outcome Assessment (Health Care)

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Obstetrics and Gynecology

Cite this

Hay, S. C., Kirpalani, H., Viner, C., Soll, R., Dukhovny, D., Mao, W. Y., ... Zupancic, J. A. F. (2017). Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs. Journal of Perinatology, 37(11), 1215-1219. https://doi.org/10.1038/jp.2017.126

Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs. / Hay, S. C.; Kirpalani, H.; Viner, C.; Soll, R.; Dukhovny, Dmitry; Mao, W. Y.; Profit, J.; DeMauro, S. B.; Zupancic, J. A.F.

In: Journal of Perinatology, Vol. 37, No. 11, 01.11.2017, p. 1215-1219.

Research output: Contribution to journalArticle

Hay, SC, Kirpalani, H, Viner, C, Soll, R, Dukhovny, D, Mao, WY, Profit, J, DeMauro, SB & Zupancic, JAF 2017, 'Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs', Journal of Perinatology, vol. 37, no. 11, pp. 1215-1219. https://doi.org/10.1038/jp.2017.126
Hay, S. C. ; Kirpalani, H. ; Viner, C. ; Soll, R. ; Dukhovny, Dmitry ; Mao, W. Y. ; Profit, J. ; DeMauro, S. B. ; Zupancic, J. A.F. / Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs. In: Journal of Perinatology. 2017 ; Vol. 37, No. 11. pp. 1215-1219.
@article{434630b26d724a74abf8e0a71dfdc753,
title = "Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs",
abstract = "Objective:To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.Study Design:We selected meta-Analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-Analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.Results:For the stated primary outcome, 18{\%} of latest RCTs failed to narrow the confidence interval (CI), and 55{\%} failed to decrease the CI by ≥20{\%}. Only 8{\%} changed the typical estimate directionality, and 11{\%} caused a change to or from significance. Latest RCTs did not change the typical estimate in 18{\%} of cases, and only 41{\%} changed the typical estimate by at least 10{\%}. The ability to narrow the CI by >20{\%} was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.Conclusion:Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.",
author = "Hay, {S. C.} and H. Kirpalani and C. Viner and R. Soll and Dmitry Dukhovny and Mao, {W. Y.} and J. Profit and DeMauro, {S. B.} and Zupancic, {J. A.F.}",
year = "2017",
month = "11",
day = "1",
doi = "10.1038/jp.2017.126",
language = "English (US)",
volume = "37",
pages = "1215--1219",
journal = "Journal of Perinatology",
issn = "0743-8346",
publisher = "Nature Publishing Group",
number = "11",

}

TY - JOUR

T1 - Do trials reduce uncertainty? Assessing impact through cumulative meta-Analysis of neonatal RCTs

AU - Hay, S. C.

AU - Kirpalani, H.

AU - Viner, C.

AU - Soll, R.

AU - Dukhovny, Dmitry

AU - Mao, W. Y.

AU - Profit, J.

AU - DeMauro, S. B.

AU - Zupancic, J. A.F.

PY - 2017/11/1

Y1 - 2017/11/1

N2 - Objective:To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.Study Design:We selected meta-Analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-Analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.Results:For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ≥20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.Conclusion:Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.

AB - Objective:To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.Study Design:We selected meta-Analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-Analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.Results:For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ≥20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.Conclusion:Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.

UR - http://www.scopus.com/inward/record.url?scp=85034058606&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034058606&partnerID=8YFLogxK

U2 - 10.1038/jp.2017.126

DO - 10.1038/jp.2017.126

M3 - Article

C2 - 28880258

AN - SCOPUS:85034058606

VL - 37

SP - 1215

EP - 1219

JO - Journal of Perinatology

JF - Journal of Perinatology

SN - 0743-8346

IS - 11

ER -