TY - JOUR
T1 - Systematic analysis of challenge-driven improvements in molecular prognostic models for breast cancer
AU - Margolin, Adam A.
AU - Bilal, Erhan
AU - Huang, Erich
AU - Norman, Thea C.
AU - Ottestad, Lars
AU - Mecham, Brigham H.
AU - Sauerwine, Ben
AU - Kellen, Michael R.
AU - Mangravite, Lara M.
AU - Furia, Matthew D.
AU - Vollan, Hans Kristian Moen
AU - Rueda, Oscar M.
AU - Guinney, Justin
AU - Deflaux, Nicole A.
AU - Hoff, Bruce
AU - Schildwachter, Xavier
AU - Russnes, Hege G.
AU - Park, Daehoon
AU - Vang, Veronica O.
AU - Pirtle, Tyler
AU - Youseff, Lamia
AU - Citro, Craig
AU - Curtis, Christina
AU - Kristensen, Vessela N.
AU - Hellerstein, Joseph
AU - Friend, Stephen H.
AU - Stolovitzky, Gustavo
AU - Aparicio, Samuel
AU - Caldas, Carlos
AU - Børresen-Dale, Anne Lise
PY - 2013/4/17
Y1 - 2013/4/17
N2 - Although molecular prognostics in breast cancer are among the most successful examples of translating genomic analysis to clinical applications, optimal approaches to breast cancer clinical risk prediction remain controversial. The Sage Bionetworks-DREAM Breast Cancer Prognosis Challenge (BCC) is a crowdsourced research study for breast cancer prognostic modeling using genome-scale data. The BCC provided a community of data analysts with a common platform for data access and blinded evaluation of model accuracy in predicting breast cancer survival on the basis of gene expression data, copy number data, and clinical covariates. This approach offered the opportunity to assess whether a crowdsourced community Challenge would generate models of breast cancer prognosis commensurate with or exceeding current best-in-class approaches. The BCC comprised multiple rounds of blinded evaluations on held-out portions of data on 1981 patients, resulting in more than 1400 models submitted as open source code. Participants then retrained their models on the full data set of 1981 samples and submitted up to five models for validation in a newly generated data set of 184 breast cancer patients. Analysis of the BCC results suggests that the best-performing modeling strategy outperformed previously reported methods in blinded evaluations; model performance was consistent across several independent evaluations; and aggregating community-developed models achieved performance on par with the best-performing individual models.
AB - Although molecular prognostics in breast cancer are among the most successful examples of translating genomic analysis to clinical applications, optimal approaches to breast cancer clinical risk prediction remain controversial. The Sage Bionetworks-DREAM Breast Cancer Prognosis Challenge (BCC) is a crowdsourced research study for breast cancer prognostic modeling using genome-scale data. The BCC provided a community of data analysts with a common platform for data access and blinded evaluation of model accuracy in predicting breast cancer survival on the basis of gene expression data, copy number data, and clinical covariates. This approach offered the opportunity to assess whether a crowdsourced community Challenge would generate models of breast cancer prognosis commensurate with or exceeding current best-in-class approaches. The BCC comprised multiple rounds of blinded evaluations on held-out portions of data on 1981 patients, resulting in more than 1400 models submitted as open source code. Participants then retrained their models on the full data set of 1981 samples and submitted up to five models for validation in a newly generated data set of 184 breast cancer patients. Analysis of the BCC results suggests that the best-performing modeling strategy outperformed previously reported methods in blinded evaluations; model performance was consistent across several independent evaluations; and aggregating community-developed models achieved performance on par with the best-performing individual models.
UR - http://www.scopus.com/inward/record.url?scp=84877765675&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84877765675&partnerID=8YFLogxK
U2 - 10.1126/scitranslmed.3006112
DO - 10.1126/scitranslmed.3006112
M3 - Article
C2 - 23596205
AN - SCOPUS:84877765675
SN - 1946-6234
VL - 5
JO - Science translational medicine
JF - Science translational medicine
IS - 181
M1 - 181re1
ER -