DREAMTools: A Python package for scoring collaborative challenges

Thomas Cokelaer, Mukesh Bansal, Christopher Bare, Erhan Bilal, Brian M. Bot, Elias Chaibub Neto, Federica Eduati, Alberto de la Fuente, Mehmet Gonen, Steven M. Hill, Bruce Hoff, Jonathan R. Karr, Robert Küffner, Michael P. Menden, Pablo Meyer, Raquel Norel, Abhishek Pratap, Robert J. Prill, Matthew T. Weirauch, James C. CostelloGustavo Stolovitzky, Julio Saez-Rodriguez

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.

Original languageEnglish (US)
Article number1030
JournalF1000Research
Volume4
DOIs
StatePublished - 2016

Fingerprint

Boidae
Computational methods
Metadata
Medicine
Learning systems
Websites
Synapses
Availability
Translational Medical Research
Systems Biology
Systems Analysis
Documentation
Software
Research Personnel

ASJC Scopus subject areas

  • Medicine(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Immunology and Microbiology(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)

Cite this

Cokelaer, T., Bansal, M., Bare, C., Bilal, E., Bot, B. M., Chaibub Neto, E., ... Saez-Rodriguez, J. (2016). DREAMTools: A Python package for scoring collaborative challenges. F1000Research, 4, [1030]. https://doi.org/10.12688/f1000research.7118.2

DREAMTools : A Python package for scoring collaborative challenges. / Cokelaer, Thomas; Bansal, Mukesh; Bare, Christopher; Bilal, Erhan; Bot, Brian M.; Chaibub Neto, Elias; Eduati, Federica; de la Fuente, Alberto; Gonen, Mehmet; Hill, Steven M.; Hoff, Bruce; Karr, Jonathan R.; Küffner, Robert; Menden, Michael P.; Meyer, Pablo; Norel, Raquel; Pratap, Abhishek; Prill, Robert J.; Weirauch, Matthew T.; Costello, James C.; Stolovitzky, Gustavo; Saez-Rodriguez, Julio.

In: F1000Research, Vol. 4, 1030, 2016.

Research output: Contribution to journalArticle

Cokelaer, T, Bansal, M, Bare, C, Bilal, E, Bot, BM, Chaibub Neto, E, Eduati, F, de la Fuente, A, Gonen, M, Hill, SM, Hoff, B, Karr, JR, Küffner, R, Menden, MP, Meyer, P, Norel, R, Pratap, A, Prill, RJ, Weirauch, MT, Costello, JC, Stolovitzky, G & Saez-Rodriguez, J 2016, 'DREAMTools: A Python package for scoring collaborative challenges', F1000Research, vol. 4, 1030. https://doi.org/10.12688/f1000research.7118.2
Cokelaer T, Bansal M, Bare C, Bilal E, Bot BM, Chaibub Neto E et al. DREAMTools: A Python package for scoring collaborative challenges. F1000Research. 2016;4. 1030. https://doi.org/10.12688/f1000research.7118.2
Cokelaer, Thomas ; Bansal, Mukesh ; Bare, Christopher ; Bilal, Erhan ; Bot, Brian M. ; Chaibub Neto, Elias ; Eduati, Federica ; de la Fuente, Alberto ; Gonen, Mehmet ; Hill, Steven M. ; Hoff, Bruce ; Karr, Jonathan R. ; Küffner, Robert ; Menden, Michael P. ; Meyer, Pablo ; Norel, Raquel ; Pratap, Abhishek ; Prill, Robert J. ; Weirauch, Matthew T. ; Costello, James C. ; Stolovitzky, Gustavo ; Saez-Rodriguez, Julio. / DREAMTools : A Python package for scoring collaborative challenges. In: F1000Research. 2016 ; Vol. 4.
@article{2aa492599fc547dfbd90afdc744ff7bb,
title = "DREAMTools: A Python package for scoring collaborative challenges",
abstract = "DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80{\%} of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.",
author = "Thomas Cokelaer and Mukesh Bansal and Christopher Bare and Erhan Bilal and Bot, {Brian M.} and {Chaibub Neto}, Elias and Federica Eduati and {de la Fuente}, Alberto and Mehmet Gonen and Hill, {Steven M.} and Bruce Hoff and Karr, {Jonathan R.} and Robert K{\"u}ffner and Menden, {Michael P.} and Pablo Meyer and Raquel Norel and Abhishek Pratap and Prill, {Robert J.} and Weirauch, {Matthew T.} and Costello, {James C.} and Gustavo Stolovitzky and Julio Saez-Rodriguez",
year = "2016",
doi = "10.12688/f1000research.7118.2",
language = "English (US)",
volume = "4",
journal = "F1000Research",
issn = "2046-1402",
publisher = "F1000 Research Ltd.",

}

TY - JOUR

T1 - DREAMTools

T2 - A Python package for scoring collaborative challenges

AU - Cokelaer, Thomas

AU - Bansal, Mukesh

AU - Bare, Christopher

AU - Bilal, Erhan

AU - Bot, Brian M.

AU - Chaibub Neto, Elias

AU - Eduati, Federica

AU - de la Fuente, Alberto

AU - Gonen, Mehmet

AU - Hill, Steven M.

AU - Hoff, Bruce

AU - Karr, Jonathan R.

AU - Küffner, Robert

AU - Menden, Michael P.

AU - Meyer, Pablo

AU - Norel, Raquel

AU - Pratap, Abhishek

AU - Prill, Robert J.

AU - Weirauch, Matthew T.

AU - Costello, James C.

AU - Stolovitzky, Gustavo

AU - Saez-Rodriguez, Julio

PY - 2016

Y1 - 2016

N2 - DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.

AB - DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.

UR - http://www.scopus.com/inward/record.url?scp=84969591048&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84969591048&partnerID=8YFLogxK

U2 - 10.12688/f1000research.7118.2

DO - 10.12688/f1000research.7118.2

M3 - Article

AN - SCOPUS:84969591048

VL - 4

JO - F1000Research

JF - F1000Research

SN - 2046-1402

M1 - 1030

ER -