Living systematic reviews

2. Combining human and machine effort

on behalf of the, Living Systematic Review Network, Living Systematic Review Network

Research output: Contribution to journalReview article

34 Citations (Scopus)

Abstract

New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential—and limitations—of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine “technologies” are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.

Original languageEnglish (US)
Pages (from-to)31-37
Number of pages7
JournalJournal of Clinical Epidemiology
Volume91
DOIs
StatePublished - Nov 1 2017

Fingerprint

Automation
Technology
Information Services
Workflow
Research

Keywords

  • Automation
  • Citizen science
  • Crowdsourcing
  • Machine learning
  • Systematic review
  • Text mining

ASJC Scopus subject areas

  • Epidemiology

Cite this

on behalf of the, Living Systematic Review Network, & Living Systematic Review Network (2017). Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology, 91, 31-37. https://doi.org/10.1016/j.jclinepi.2017.08.011

Living systematic reviews : 2. Combining human and machine effort. / on behalf of the; Living Systematic Review Network; Living Systematic Review Network.

In: Journal of Clinical Epidemiology, Vol. 91, 01.11.2017, p. 31-37.

Research output: Contribution to journalReview article

on behalf of the, Living Systematic Review Network & Living Systematic Review Network 2017, 'Living systematic reviews: 2. Combining human and machine effort', Journal of Clinical Epidemiology, vol. 91, pp. 31-37. https://doi.org/10.1016/j.jclinepi.2017.08.011
on behalf of the, Living Systematic Review Network, Living Systematic Review Network. Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology. 2017 Nov 1;91:31-37. https://doi.org/10.1016/j.jclinepi.2017.08.011
on behalf of the ; Living Systematic Review Network ; Living Systematic Review Network. / Living systematic reviews : 2. Combining human and machine effort. In: Journal of Clinical Epidemiology. 2017 ; Vol. 91. pp. 31-37.
@article{36f3108360474954a35abb54cf00d1e0,
title = "Living systematic reviews: 2. Combining human and machine effort",
abstract = "New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential—and limitations—of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine “technologies” are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.",
keywords = "Automation, Citizen science, Crowdsourcing, Machine learning, Systematic review, Text mining",
author = "{on behalf of the} and {Living Systematic Review Network} and {Living Systematic Review Network} and James Thomas and Anna Noel-Storr and Iain Marshall and Byron Wallace and Steven McDonald and Chris Mavergames and Paul Glasziou and Ian Shemilt and Anneliese Synnot and Tari Turner and Julian Elliott and Thomas Agoritsas and John Hilton and Caroline Perron and Elie Akl and Rebecca Hodder and Charlotte Pestridge and Lauren Albrecht and Tanya Horsley and Joanne Platt and Rebecca Armstrong and Nguyen, {Phi Hung} and Robert Plovnick and Anneliese Arno and Noah Ivers and Gail Quinn and Agnes Au and Renea Johnston and Gabriel Rada and Matthew Bagg and Arwel Jones and Philippe Ravaud and Catherine Boden and Lara Kahale and Bernt Richter and Isabelle Boisvert and Homa Keshavarz and Rebecca Ryan and Linn Brandt and Kolakowsky-Hayner, {Stephanie A.} and Dina Salama and Alexandra Brazinova and Nagraj, {Sumanth Kumbargere} and Georgia Salanti and Rachelle Buchbinder and Toby Lasserson and Lina Santaguida and Chris Champion and Roger Chou and Mark Helfand",
year = "2017",
month = "11",
day = "1",
doi = "10.1016/j.jclinepi.2017.08.011",
language = "English (US)",
volume = "91",
pages = "31--37",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Living systematic reviews

T2 - 2. Combining human and machine effort

AU - on behalf of the

AU - Living Systematic Review Network

AU - Living Systematic Review Network

AU - Thomas, James

AU - Noel-Storr, Anna

AU - Marshall, Iain

AU - Wallace, Byron

AU - McDonald, Steven

AU - Mavergames, Chris

AU - Glasziou, Paul

AU - Shemilt, Ian

AU - Synnot, Anneliese

AU - Turner, Tari

AU - Elliott, Julian

AU - Agoritsas, Thomas

AU - Hilton, John

AU - Perron, Caroline

AU - Akl, Elie

AU - Hodder, Rebecca

AU - Pestridge, Charlotte

AU - Albrecht, Lauren

AU - Horsley, Tanya

AU - Platt, Joanne

AU - Armstrong, Rebecca

AU - Nguyen, Phi Hung

AU - Plovnick, Robert

AU - Arno, Anneliese

AU - Ivers, Noah

AU - Quinn, Gail

AU - Au, Agnes

AU - Johnston, Renea

AU - Rada, Gabriel

AU - Bagg, Matthew

AU - Jones, Arwel

AU - Ravaud, Philippe

AU - Boden, Catherine

AU - Kahale, Lara

AU - Richter, Bernt

AU - Boisvert, Isabelle

AU - Keshavarz, Homa

AU - Ryan, Rebecca

AU - Brandt, Linn

AU - Kolakowsky-Hayner, Stephanie A.

AU - Salama, Dina

AU - Brazinova, Alexandra

AU - Nagraj, Sumanth Kumbargere

AU - Salanti, Georgia

AU - Buchbinder, Rachelle

AU - Lasserson, Toby

AU - Santaguida, Lina

AU - Champion, Chris

AU - Chou, Roger

AU - Helfand, Mark

PY - 2017/11/1

Y1 - 2017/11/1

N2 - New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential—and limitations—of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine “technologies” are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.

AB - New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities (“crowds”) as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential—and limitations—of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine “technologies” are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.

KW - Automation

KW - Citizen science

KW - Crowdsourcing

KW - Machine learning

KW - Systematic review

KW - Text mining

UR - http://www.scopus.com/inward/record.url?scp=85028965305&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028965305&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.08.011

DO - 10.1016/j.jclinepi.2017.08.011

M3 - Review article

VL - 91

SP - 31

EP - 37

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -