The scientific impact of positive and negative phase 3 cancer clinical trials

Joseph M. Unger, William E. Barlow, Scott D. Ramsey, Michael LeBlanc, Charles Blanke, Dawn L. Hershman

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Importance: Positive phase 3 cancer clinical trials are widely hailed, while trials with negative results are often interpreted as scientific failures.We hypothesized that these interpretations would be reflected in the scientific literature. Objective: To compare the scientific impact of positive vs negative phase 3 cancer clinical treatment trials. Design, Setting, and Participants: We examined the phase 3 trial history of SWOG, a national cancer clinical trials consortium, over a 30-year period (1985-2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar. Citation counts were compared using generalized estimating equations for Poisson regression. Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Trials were categorized as positive if they achieved a statistically significant result in favor of the new experimental treatment for the protocol-specified primary end point. Trials were categorized as negative if they achieved a statistically significant result in favor of standard therapy or a null result with no statistically significant benefit for either the experimental or standard therapy. Main Outcomes and Measures: Impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial articles and all secondary articles associated with the trials. Results: Ninety-four studies enrolling n = 46 424 patients were analyzed. Twenty-eight percent of trials were positive (26 of 94). The primary publications from positive trials were published in journals with higher mean (SD) 2-year impact factors (28 [19] vs 18 [13]; P = .007) and were cited twice as often as negative trials (mean per year, 43 vs 21; relative risk, 2.0; 95%CI, 1.1-3.9; P = .03). However, the number of citations from all primary and secondary articles did not significantly differ between positive and negative trials (mean per year, 55 vs 45; relative risk, 1.2; 95%CI, 0.7-2.3; P = .53). Conclusions and Relevance: The scientific impact of the primary articles from positive phase 3 randomized cancer clinical trials was twice as great as for negative trials. But when all of the articles associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.

Original languageEnglish (US)
Pages (from-to)875-881
Number of pages7
JournalJAMA oncology
Volume2
Issue number7
DOIs
StatePublished - Jul 1 2016

Fingerprint

Phase III Clinical Trials
Publications
Clinical Trials
Neoplasms
Journal Impact Factor
Literature
Therapeutics
Clinical Protocols
Randomized Controlled Trials
Outcome Assessment (Health Care)

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Unger, J. M., Barlow, W. E., Ramsey, S. D., LeBlanc, M., Blanke, C., & Hershman, D. L. (2016). The scientific impact of positive and negative phase 3 cancer clinical trials. JAMA oncology, 2(7), 875-881. https://doi.org/10.1001/jamaoncol.2015.6487

The scientific impact of positive and negative phase 3 cancer clinical trials. / Unger, Joseph M.; Barlow, William E.; Ramsey, Scott D.; LeBlanc, Michael; Blanke, Charles; Hershman, Dawn L.

In: JAMA oncology, Vol. 2, No. 7, 01.07.2016, p. 875-881.

Research output: Contribution to journalArticle

Unger, JM, Barlow, WE, Ramsey, SD, LeBlanc, M, Blanke, C & Hershman, DL 2016, 'The scientific impact of positive and negative phase 3 cancer clinical trials', JAMA oncology, vol. 2, no. 7, pp. 875-881. https://doi.org/10.1001/jamaoncol.2015.6487
Unger, Joseph M. ; Barlow, William E. ; Ramsey, Scott D. ; LeBlanc, Michael ; Blanke, Charles ; Hershman, Dawn L. / The scientific impact of positive and negative phase 3 cancer clinical trials. In: JAMA oncology. 2016 ; Vol. 2, No. 7. pp. 875-881.
@article{754c82e989b244908228b6dac444f93e,
title = "The scientific impact of positive and negative phase 3 cancer clinical trials",
abstract = "Importance: Positive phase 3 cancer clinical trials are widely hailed, while trials with negative results are often interpreted as scientific failures.We hypothesized that these interpretations would be reflected in the scientific literature. Objective: To compare the scientific impact of positive vs negative phase 3 cancer clinical treatment trials. Design, Setting, and Participants: We examined the phase 3 trial history of SWOG, a national cancer clinical trials consortium, over a 30-year period (1985-2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar. Citation counts were compared using generalized estimating equations for Poisson regression. Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Trials were categorized as positive if they achieved a statistically significant result in favor of the new experimental treatment for the protocol-specified primary end point. Trials were categorized as negative if they achieved a statistically significant result in favor of standard therapy or a null result with no statistically significant benefit for either the experimental or standard therapy. Main Outcomes and Measures: Impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial articles and all secondary articles associated with the trials. Results: Ninety-four studies enrolling n = 46 424 patients were analyzed. Twenty-eight percent of trials were positive (26 of 94). The primary publications from positive trials were published in journals with higher mean (SD) 2-year impact factors (28 [19] vs 18 [13]; P = .007) and were cited twice as often as negative trials (mean per year, 43 vs 21; relative risk, 2.0; 95{\%}CI, 1.1-3.9; P = .03). However, the number of citations from all primary and secondary articles did not significantly differ between positive and negative trials (mean per year, 55 vs 45; relative risk, 1.2; 95{\%}CI, 0.7-2.3; P = .53). Conclusions and Relevance: The scientific impact of the primary articles from positive phase 3 randomized cancer clinical trials was twice as great as for negative trials. But when all of the articles associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.",
author = "Unger, {Joseph M.} and Barlow, {William E.} and Ramsey, {Scott D.} and Michael LeBlanc and Charles Blanke and Hershman, {Dawn L.}",
year = "2016",
month = "7",
day = "1",
doi = "10.1001/jamaoncol.2015.6487",
language = "English (US)",
volume = "2",
pages = "875--881",
journal = "JAMA oncology",
issn = "2374-2437",
publisher = "American Medical Association",
number = "7",

}

TY - JOUR

T1 - The scientific impact of positive and negative phase 3 cancer clinical trials

AU - Unger, Joseph M.

AU - Barlow, William E.

AU - Ramsey, Scott D.

AU - LeBlanc, Michael

AU - Blanke, Charles

AU - Hershman, Dawn L.

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Importance: Positive phase 3 cancer clinical trials are widely hailed, while trials with negative results are often interpreted as scientific failures.We hypothesized that these interpretations would be reflected in the scientific literature. Objective: To compare the scientific impact of positive vs negative phase 3 cancer clinical treatment trials. Design, Setting, and Participants: We examined the phase 3 trial history of SWOG, a national cancer clinical trials consortium, over a 30-year period (1985-2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar. Citation counts were compared using generalized estimating equations for Poisson regression. Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Trials were categorized as positive if they achieved a statistically significant result in favor of the new experimental treatment for the protocol-specified primary end point. Trials were categorized as negative if they achieved a statistically significant result in favor of standard therapy or a null result with no statistically significant benefit for either the experimental or standard therapy. Main Outcomes and Measures: Impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial articles and all secondary articles associated with the trials. Results: Ninety-four studies enrolling n = 46 424 patients were analyzed. Twenty-eight percent of trials were positive (26 of 94). The primary publications from positive trials were published in journals with higher mean (SD) 2-year impact factors (28 [19] vs 18 [13]; P = .007) and were cited twice as often as negative trials (mean per year, 43 vs 21; relative risk, 2.0; 95%CI, 1.1-3.9; P = .03). However, the number of citations from all primary and secondary articles did not significantly differ between positive and negative trials (mean per year, 55 vs 45; relative risk, 1.2; 95%CI, 0.7-2.3; P = .53). Conclusions and Relevance: The scientific impact of the primary articles from positive phase 3 randomized cancer clinical trials was twice as great as for negative trials. But when all of the articles associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.

AB - Importance: Positive phase 3 cancer clinical trials are widely hailed, while trials with negative results are often interpreted as scientific failures.We hypothesized that these interpretations would be reflected in the scientific literature. Objective: To compare the scientific impact of positive vs negative phase 3 cancer clinical treatment trials. Design, Setting, and Participants: We examined the phase 3 trial history of SWOG, a national cancer clinical trials consortium, over a 30-year period (1985-2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar. Citation counts were compared using generalized estimating equations for Poisson regression. Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Trials were categorized as positive if they achieved a statistically significant result in favor of the new experimental treatment for the protocol-specified primary end point. Trials were categorized as negative if they achieved a statistically significant result in favor of standard therapy or a null result with no statistically significant benefit for either the experimental or standard therapy. Main Outcomes and Measures: Impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial articles and all secondary articles associated with the trials. Results: Ninety-four studies enrolling n = 46 424 patients were analyzed. Twenty-eight percent of trials were positive (26 of 94). The primary publications from positive trials were published in journals with higher mean (SD) 2-year impact factors (28 [19] vs 18 [13]; P = .007) and were cited twice as often as negative trials (mean per year, 43 vs 21; relative risk, 2.0; 95%CI, 1.1-3.9; P = .03). However, the number of citations from all primary and secondary articles did not significantly differ between positive and negative trials (mean per year, 55 vs 45; relative risk, 1.2; 95%CI, 0.7-2.3; P = .53). Conclusions and Relevance: The scientific impact of the primary articles from positive phase 3 randomized cancer clinical trials was twice as great as for negative trials. But when all of the articles associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.

UR - http://www.scopus.com/inward/record.url?scp=85010680111&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010680111&partnerID=8YFLogxK

U2 - 10.1001/jamaoncol.2015.6487

DO - 10.1001/jamaoncol.2015.6487

M3 - Article

C2 - 26967260

AN - SCOPUS:85010680111

VL - 2

SP - 875

EP - 881

JO - JAMA oncology

JF - JAMA oncology

SN - 2374-2437

IS - 7

ER -