Collective intelligence meets medical decision-making: The collective outperforms the best radiologist

Max Wolf, Jens Krause, Patricia (Patty) Carney, Andy Bogart, Ralf H J M Kurvers

Research output: Contribution to journalArticle

39 Citations (Scopus)

Abstract

While collective intelligence (CI) is a powerful approach to increase decision accuracy, few attempts have been made to unlock its potential in medical decision-making. Here we investigated the performance of three well-known collective intelligence rules ("majority", "quorum", and "weighted quorum") when applied to mammography screening. For any particular mammogram, these rules aggregate the independent assessments of multiple radiologists into a single decision (recall the patient for additional workup or not). We found that, compared to single radiologists, any of these CI-rules both increases true positives (i.e., recalls of patients with cancer) and decreases false positives (i.e., recalls of patients without cancer), thereby overcoming one of the fundamental limitations to decision accuracy that individual radiologists face. Importantly, we find that all CI-rules systematically outperform even the best-performing individual radiologist in the respective group. Our findings demonstrate that CI can be employed to improve mammography screening; similarly, CI may have the potential to improve medical decision-making in a much wider range of contexts, including many areas of diagnostic imaging and, more generally, diagnostic decisions that are based on the subjective interpretation of evidence.

Original languageEnglish (US)
Article numbere0134269
JournalPLoS One
Volume10
Issue number8
DOIs
StatePublished - Aug 12 2015

Fingerprint

Mammography
Intelligence
decision making
Screening
Decision making
screening
neoplasms
Imaging techniques
image analysis
Diagnostic Imaging
Radiologists
Clinical Decision-Making
Neoplasms

ASJC Scopus subject areas

  • Agricultural and Biological Sciences(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Medicine(all)

Cite this

Collective intelligence meets medical decision-making : The collective outperforms the best radiologist. / Wolf, Max; Krause, Jens; Carney, Patricia (Patty); Bogart, Andy; Kurvers, Ralf H J M.

In: PLoS One, Vol. 10, No. 8, e0134269, 12.08.2015.

Research output: Contribution to journalArticle

Wolf, Max ; Krause, Jens ; Carney, Patricia (Patty) ; Bogart, Andy ; Kurvers, Ralf H J M. / Collective intelligence meets medical decision-making : The collective outperforms the best radiologist. In: PLoS One. 2015 ; Vol. 10, No. 8.
@article{132679c1b86e48cdbb2d95c2204681ca,
title = "Collective intelligence meets medical decision-making: The collective outperforms the best radiologist",
abstract = "While collective intelligence (CI) is a powerful approach to increase decision accuracy, few attempts have been made to unlock its potential in medical decision-making. Here we investigated the performance of three well-known collective intelligence rules ({"}majority{"}, {"}quorum{"}, and {"}weighted quorum{"}) when applied to mammography screening. For any particular mammogram, these rules aggregate the independent assessments of multiple radiologists into a single decision (recall the patient for additional workup or not). We found that, compared to single radiologists, any of these CI-rules both increases true positives (i.e., recalls of patients with cancer) and decreases false positives (i.e., recalls of patients without cancer), thereby overcoming one of the fundamental limitations to decision accuracy that individual radiologists face. Importantly, we find that all CI-rules systematically outperform even the best-performing individual radiologist in the respective group. Our findings demonstrate that CI can be employed to improve mammography screening; similarly, CI may have the potential to improve medical decision-making in a much wider range of contexts, including many areas of diagnostic imaging and, more generally, diagnostic decisions that are based on the subjective interpretation of evidence.",
author = "Max Wolf and Jens Krause and Carney, {Patricia (Patty)} and Andy Bogart and Kurvers, {Ralf H J M}",
year = "2015",
month = "8",
day = "12",
doi = "10.1371/journal.pone.0134269",
language = "English (US)",
volume = "10",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "8",

}

TY - JOUR

T1 - Collective intelligence meets medical decision-making

T2 - The collective outperforms the best radiologist

AU - Wolf, Max

AU - Krause, Jens

AU - Carney, Patricia (Patty)

AU - Bogart, Andy

AU - Kurvers, Ralf H J M

PY - 2015/8/12

Y1 - 2015/8/12

N2 - While collective intelligence (CI) is a powerful approach to increase decision accuracy, few attempts have been made to unlock its potential in medical decision-making. Here we investigated the performance of three well-known collective intelligence rules ("majority", "quorum", and "weighted quorum") when applied to mammography screening. For any particular mammogram, these rules aggregate the independent assessments of multiple radiologists into a single decision (recall the patient for additional workup or not). We found that, compared to single radiologists, any of these CI-rules both increases true positives (i.e., recalls of patients with cancer) and decreases false positives (i.e., recalls of patients without cancer), thereby overcoming one of the fundamental limitations to decision accuracy that individual radiologists face. Importantly, we find that all CI-rules systematically outperform even the best-performing individual radiologist in the respective group. Our findings demonstrate that CI can be employed to improve mammography screening; similarly, CI may have the potential to improve medical decision-making in a much wider range of contexts, including many areas of diagnostic imaging and, more generally, diagnostic decisions that are based on the subjective interpretation of evidence.

AB - While collective intelligence (CI) is a powerful approach to increase decision accuracy, few attempts have been made to unlock its potential in medical decision-making. Here we investigated the performance of three well-known collective intelligence rules ("majority", "quorum", and "weighted quorum") when applied to mammography screening. For any particular mammogram, these rules aggregate the independent assessments of multiple radiologists into a single decision (recall the patient for additional workup or not). We found that, compared to single radiologists, any of these CI-rules both increases true positives (i.e., recalls of patients with cancer) and decreases false positives (i.e., recalls of patients without cancer), thereby overcoming one of the fundamental limitations to decision accuracy that individual radiologists face. Importantly, we find that all CI-rules systematically outperform even the best-performing individual radiologist in the respective group. Our findings demonstrate that CI can be employed to improve mammography screening; similarly, CI may have the potential to improve medical decision-making in a much wider range of contexts, including many areas of diagnostic imaging and, more generally, diagnostic decisions that are based on the subjective interpretation of evidence.

UR - http://www.scopus.com/inward/record.url?scp=84942849529&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942849529&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0134269

DO - 10.1371/journal.pone.0134269

M3 - Article

C2 - 26267331

AN - SCOPUS:84942849529

VL - 10

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 8

M1 - e0134269

ER -