Publication bias in antipsychotic trials: An analysis of efficacy comparing the published literature to the us food and drug administration database

Erick Turner, Daniel Knoepflmacher, Lee Shapley

Research output: Contribution to journalArticle

68 Citations (Scopus)

Abstract

Background: Publication bias compromises the validity of evidence-based medicine, yet a growing body of research shows that this problem is widespread. Efficacy data from drug regulatory agencies, e.g., the US Food and Drug Administration (FDA), can serve as a benchmark or control against which data in journal articles can be checked. Thus one may determine whether publication bias is present and quantify the extent to which it inflates apparent drug efficacy. Methods and Findings: FDA Drug Approval Packages for eight second-generation antipsychotics-aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection (risperidone LAI), and ziprasidone-were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four (17%) were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest (8%) and not statistically significant. On the other hand, the effect size for unpublished trials (0.23, 95% confidence interval 0.07 to 0.39) was less than half that for the published trials (0.47, 95% confidence interval 0.40 to 0.54), a difference that was significant. Conclusions: The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently. Without increased access to regulatory agency data, publication bias will continue to blur distinctions between effective and ineffective drugs. Please see later in the article for the Editors' Summary.

Original languageEnglish (US)
Article numbere1001189
JournalPLoS Medicine
Volume9
Issue number3
DOIs
StatePublished - Mar 2012

Fingerprint

Pharmaceutical Databases
Publication Bias
United States Food and Drug Administration
Antipsychotic Agents
Risperidone
Pharmaceutical Preparations
olanzapine
Publications
Placebos
Confidence Intervals
Drug Approval
Benchmarking
Information Storage and Retrieval
Evidence-Based Medicine
Antidepressive Agents
Medicine
Outcome Assessment (Health Care)
Injections
Research

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Publication bias in antipsychotic trials : An analysis of efficacy comparing the published literature to the us food and drug administration database. / Turner, Erick; Knoepflmacher, Daniel; Shapley, Lee.

In: PLoS Medicine, Vol. 9, No. 3, e1001189, 03.2012.

Research output: Contribution to journalArticle

@article{4f98fb9110ab43378a5c7fbdb98f8363,
title = "Publication bias in antipsychotic trials: An analysis of efficacy comparing the published literature to the us food and drug administration database",
abstract = "Background: Publication bias compromises the validity of evidence-based medicine, yet a growing body of research shows that this problem is widespread. Efficacy data from drug regulatory agencies, e.g., the US Food and Drug Administration (FDA), can serve as a benchmark or control against which data in journal articles can be checked. Thus one may determine whether publication bias is present and quantify the extent to which it inflates apparent drug efficacy. Methods and Findings: FDA Drug Approval Packages for eight second-generation antipsychotics-aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection (risperidone LAI), and ziprasidone-were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four (17{\%}) were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest (8{\%}) and not statistically significant. On the other hand, the effect size for unpublished trials (0.23, 95{\%} confidence interval 0.07 to 0.39) was less than half that for the published trials (0.47, 95{\%} confidence interval 0.40 to 0.54), a difference that was significant. Conclusions: The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently. Without increased access to regulatory agency data, publication bias will continue to blur distinctions between effective and ineffective drugs. Please see later in the article for the Editors' Summary.",
author = "Erick Turner and Daniel Knoepflmacher and Lee Shapley",
year = "2012",
month = "3",
doi = "10.1371/journal.pmed.1001189",
language = "English (US)",
volume = "9",
journal = "PLoS Medicine",
issn = "1549-1277",
publisher = "Nature Publishing Group",
number = "3",

}

TY - JOUR

T1 - Publication bias in antipsychotic trials

T2 - An analysis of efficacy comparing the published literature to the us food and drug administration database

AU - Turner, Erick

AU - Knoepflmacher, Daniel

AU - Shapley, Lee

PY - 2012/3

Y1 - 2012/3

N2 - Background: Publication bias compromises the validity of evidence-based medicine, yet a growing body of research shows that this problem is widespread. Efficacy data from drug regulatory agencies, e.g., the US Food and Drug Administration (FDA), can serve as a benchmark or control against which data in journal articles can be checked. Thus one may determine whether publication bias is present and quantify the extent to which it inflates apparent drug efficacy. Methods and Findings: FDA Drug Approval Packages for eight second-generation antipsychotics-aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection (risperidone LAI), and ziprasidone-were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four (17%) were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest (8%) and not statistically significant. On the other hand, the effect size for unpublished trials (0.23, 95% confidence interval 0.07 to 0.39) was less than half that for the published trials (0.47, 95% confidence interval 0.40 to 0.54), a difference that was significant. Conclusions: The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently. Without increased access to regulatory agency data, publication bias will continue to blur distinctions between effective and ineffective drugs. Please see later in the article for the Editors' Summary.

AB - Background: Publication bias compromises the validity of evidence-based medicine, yet a growing body of research shows that this problem is widespread. Efficacy data from drug regulatory agencies, e.g., the US Food and Drug Administration (FDA), can serve as a benchmark or control against which data in journal articles can be checked. Thus one may determine whether publication bias is present and quantify the extent to which it inflates apparent drug efficacy. Methods and Findings: FDA Drug Approval Packages for eight second-generation antipsychotics-aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection (risperidone LAI), and ziprasidone-were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four (17%) were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest (8%) and not statistically significant. On the other hand, the effect size for unpublished trials (0.23, 95% confidence interval 0.07 to 0.39) was less than half that for the published trials (0.47, 95% confidence interval 0.40 to 0.54), a difference that was significant. Conclusions: The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently. Without increased access to regulatory agency data, publication bias will continue to blur distinctions between effective and ineffective drugs. Please see later in the article for the Editors' Summary.

UR - http://www.scopus.com/inward/record.url?scp=84858957743&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858957743&partnerID=8YFLogxK

U2 - 10.1371/journal.pmed.1001189

DO - 10.1371/journal.pmed.1001189

M3 - Article

C2 - 22448149

AN - SCOPUS:84858957743

VL - 9

JO - PLoS Medicine

JF - PLoS Medicine

SN - 1549-1277

IS - 3

M1 - e1001189

ER -