DREAMTools: A Python package for scoring collaborative challenges

Thomas Cokelaer, Mukesh Bansal, Christopher Bare, Erhan Bilal, Brian M. Bot, Elias Chaibub Neto, Federica Eduati, Alberto de la Fuente, Mehmet Gönen, Steven M. Hill, Bruce Hoff, Jonathan R. Karr, Robert Küffner, Michael P. Menden, Pablo Meyer, Raquel Norel, Abhishek Pratap, Robert J. Prill, Matthew T. Weirauch, James C. CostelloGustavo Stolovitzky, Julio Saez-Rodriguez

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability: DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools.

Original languageEnglish (US)
Article number1030
StatePublished - 2016

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Immunology and Microbiology(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)


Dive into the research topics of 'DREAMTools: A Python package for scoring collaborative challenges'. Together they form a unique fingerprint.

Cite this