DREAMTools: A Python package for scoring collaborative challenges

Thomas Cokelaer, Mukesh Bansal, Christopher Bare, Erhan Bilal, Brian M. Bot, Elias Chaibub Neto, Federica Eduati, Alberto de la Fuente, Mehmet Gonen, Steven M. Hill, Bruce Hoff, Jonathan R. Karr, Robert Küffner, Michael P. Menden, Pablo Meyer, Raquel Norel, Abhishek Pratap, Robert J. Prill, Matthew T. Weirauch, James C. CostelloGustavo Stolovitzky, Julio Saez-Rodriguez

Research output: Contribution to journalArticle

5 Scopus citations

Abstract

DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.

Original languageEnglish (US)
Article number1030
JournalF1000Research
Volume4
DOIs
Publication statusPublished - 2016

    Fingerprint

ASJC Scopus subject areas

  • Medicine(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Immunology and Microbiology(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)

Cite this

Cokelaer, T., Bansal, M., Bare, C., Bilal, E., Bot, B. M., Chaibub Neto, E., ... Saez-Rodriguez, J. (2016). DREAMTools: A Python package for scoring collaborative challenges. F1000Research, 4, [1030]. https://doi.org/10.12688/f1000research.7118.2