A Prospective Evaluation of an Automated Classification System to Support Evidence-based Medicine and Systematic Review

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Systematic reviews (SR) are an important and labor-intensive part of the Evidence-based Medicine process that could benefit from automated literature classification tools. We conducted a prospective study of a support vector machine-based classifier for supporting the SR literature triage process. Over 50,000 training data samples were collected for 18 topics prior to March 2008, and used to make predictions on 11,000 test data samples collected during the subsequent two years. Test performance (AUC) was comparable to that estimated by cross-validation on the training set, and ranging from 0.75 - 0.99. Mean AUC macro-averaged across all topics was 0.89, demonstrating that these methods can achieve accurate results in near-real world conditions and are promising tools for deployment to groups conducting SRs.

Original languageEnglish (US)
Pages (from-to)121-125
Number of pages5
JournalAMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium
Volume2010
StatePublished - 2010

Fingerprint

Evidence-Based Medicine
Area Under Curve
Triage
Prospective Studies
Support Vector Machine

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{63be24380d2d48ceaad6a0ea297a33f7,
title = "A Prospective Evaluation of an Automated Classification System to Support Evidence-based Medicine and Systematic Review",
abstract = "Systematic reviews (SR) are an important and labor-intensive part of the Evidence-based Medicine process that could benefit from automated literature classification tools. We conducted a prospective study of a support vector machine-based classifier for supporting the SR literature triage process. Over 50,000 training data samples were collected for 18 topics prior to March 2008, and used to make predictions on 11,000 test data samples collected during the subsequent two years. Test performance (AUC) was comparable to that estimated by cross-validation on the training set, and ranging from 0.75 - 0.99. Mean AUC macro-averaged across all topics was 0.89, demonstrating that these methods can achieve accurate results in near-real world conditions and are promising tools for deployment to groups conducting SRs.",
author = "Aaron Cohen and Kyle Ambert and Marian McDonagh",
year = "2010",
language = "English (US)",
volume = "2010",
pages = "121--125",
journal = "AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium",
issn = "1559-4076",
publisher = "American Medical Informatics Association",

}

TY - JOUR

T1 - A Prospective Evaluation of an Automated Classification System to Support Evidence-based Medicine and Systematic Review

AU - Cohen, Aaron

AU - Ambert, Kyle

AU - McDonagh, Marian

PY - 2010

Y1 - 2010

N2 - Systematic reviews (SR) are an important and labor-intensive part of the Evidence-based Medicine process that could benefit from automated literature classification tools. We conducted a prospective study of a support vector machine-based classifier for supporting the SR literature triage process. Over 50,000 training data samples were collected for 18 topics prior to March 2008, and used to make predictions on 11,000 test data samples collected during the subsequent two years. Test performance (AUC) was comparable to that estimated by cross-validation on the training set, and ranging from 0.75 - 0.99. Mean AUC macro-averaged across all topics was 0.89, demonstrating that these methods can achieve accurate results in near-real world conditions and are promising tools for deployment to groups conducting SRs.

AB - Systematic reviews (SR) are an important and labor-intensive part of the Evidence-based Medicine process that could benefit from automated literature classification tools. We conducted a prospective study of a support vector machine-based classifier for supporting the SR literature triage process. Over 50,000 training data samples were collected for 18 topics prior to March 2008, and used to make predictions on 11,000 test data samples collected during the subsequent two years. Test performance (AUC) was comparable to that estimated by cross-validation on the training set, and ranging from 0.75 - 0.99. Mean AUC macro-averaged across all topics was 0.89, demonstrating that these methods can achieve accurate results in near-real world conditions and are promising tools for deployment to groups conducting SRs.

UR - http://www.scopus.com/inward/record.url?scp=84964986938&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964986938&partnerID=8YFLogxK

M3 - Article

C2 - 21346953

AN - SCOPUS:84964986938

VL - 2010

SP - 121

EP - 125

JO - AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium

JF - AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium

SN - 1559-4076

ER -