Ventral striatum’s role in learning from gains and losses

Craig A. Taswell, Vincent Costa, Elisabeth A. Murray, Bruno B. Averbeck

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Adaptive behavior requires animals to learn from experience. Ideally, learning should both promote choices that lead to rewards and reduce choices that lead to losses. Because the ventral striatum (VS) contains neurons that respond to aversive stimuli and aversive stimuli can drive dopamine release in the VS, it is possible that the VS contributes to learning about aversive outcomes, including losses. However, other work suggests that the VS may play a specific role in learning to choose among rewards, with other systems mediating learning from aversive outcomes. To examine the role of the VS in learning from gains and losses, we compared the performance of macaque monkeys with VS lesions and unoperated controls on a reinforcement learning task. In the task, the monkeys gained or lost tokens, which were periodically cashed out for juice, as outcomes for choices. They learned over trials to choose cues associated with gains, and not choose cues associated with losses. We found that monkeys with VS lesions had a deficit in learning to choose between cues that differed in reward magnitude. By contrast, monkeys with VS lesions performed as well as controls when choices involved a potential loss. We also fit reinforcement learning models to the behavior and compared learning rates between groups. Relative to controls, the monkeys with VS lesions had reduced learning rates for gain cues. Therefore, in this task, the VS plays a specific role in learning to choose between rewarding options.

Original languageEnglish (US)
Pages (from-to)E12398-E12406
JournalProceedings of the National Academy of Sciences of the United States of America
Volume115
Issue number52
DOIs
StatePublished - Dec 26 2018
Externally publishedYes

Fingerprint

Learning
Haplorhini
Cues
Reward
Ventral Striatum
Psychological Adaptation
Macaca
Dopamine
Neurons

Keywords

  • Appetitive
  • Aversive
  • Neuroeconomics
  • Reinforcement learning
  • Ventral striatum

ASJC Scopus subject areas

  • General

Cite this

Ventral striatum’s role in learning from gains and losses. / Taswell, Craig A.; Costa, Vincent; Murray, Elisabeth A.; Averbeck, Bruno B.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 115, No. 52, 26.12.2018, p. E12398-E12406.

Research output: Contribution to journalArticle

Taswell, Craig A. ; Costa, Vincent ; Murray, Elisabeth A. ; Averbeck, Bruno B. / Ventral striatum’s role in learning from gains and losses. In: Proceedings of the National Academy of Sciences of the United States of America. 2018 ; Vol. 115, No. 52. pp. E12398-E12406.
@article{13324f48ccd44497b699a7befe555812,
title = "Ventral striatum’s role in learning from gains and losses",
abstract = "Adaptive behavior requires animals to learn from experience. Ideally, learning should both promote choices that lead to rewards and reduce choices that lead to losses. Because the ventral striatum (VS) contains neurons that respond to aversive stimuli and aversive stimuli can drive dopamine release in the VS, it is possible that the VS contributes to learning about aversive outcomes, including losses. However, other work suggests that the VS may play a specific role in learning to choose among rewards, with other systems mediating learning from aversive outcomes. To examine the role of the VS in learning from gains and losses, we compared the performance of macaque monkeys with VS lesions and unoperated controls on a reinforcement learning task. In the task, the monkeys gained or lost tokens, which were periodically cashed out for juice, as outcomes for choices. They learned over trials to choose cues associated with gains, and not choose cues associated with losses. We found that monkeys with VS lesions had a deficit in learning to choose between cues that differed in reward magnitude. By contrast, monkeys with VS lesions performed as well as controls when choices involved a potential loss. We also fit reinforcement learning models to the behavior and compared learning rates between groups. Relative to controls, the monkeys with VS lesions had reduced learning rates for gain cues. Therefore, in this task, the VS plays a specific role in learning to choose between rewarding options.",
keywords = "Appetitive, Aversive, Neuroeconomics, Reinforcement learning, Ventral striatum",
author = "Taswell, {Craig A.} and Vincent Costa and Murray, {Elisabeth A.} and Averbeck, {Bruno B.}",
year = "2018",
month = "12",
day = "26",
doi = "10.1073/pnas.1809833115",
language = "English (US)",
volume = "115",
pages = "E12398--E12406",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "52",

}

TY - JOUR

T1 - Ventral striatum’s role in learning from gains and losses

AU - Taswell, Craig A.

AU - Costa, Vincent

AU - Murray, Elisabeth A.

AU - Averbeck, Bruno B.

PY - 2018/12/26

Y1 - 2018/12/26

N2 - Adaptive behavior requires animals to learn from experience. Ideally, learning should both promote choices that lead to rewards and reduce choices that lead to losses. Because the ventral striatum (VS) contains neurons that respond to aversive stimuli and aversive stimuli can drive dopamine release in the VS, it is possible that the VS contributes to learning about aversive outcomes, including losses. However, other work suggests that the VS may play a specific role in learning to choose among rewards, with other systems mediating learning from aversive outcomes. To examine the role of the VS in learning from gains and losses, we compared the performance of macaque monkeys with VS lesions and unoperated controls on a reinforcement learning task. In the task, the monkeys gained or lost tokens, which were periodically cashed out for juice, as outcomes for choices. They learned over trials to choose cues associated with gains, and not choose cues associated with losses. We found that monkeys with VS lesions had a deficit in learning to choose between cues that differed in reward magnitude. By contrast, monkeys with VS lesions performed as well as controls when choices involved a potential loss. We also fit reinforcement learning models to the behavior and compared learning rates between groups. Relative to controls, the monkeys with VS lesions had reduced learning rates for gain cues. Therefore, in this task, the VS plays a specific role in learning to choose between rewarding options.

AB - Adaptive behavior requires animals to learn from experience. Ideally, learning should both promote choices that lead to rewards and reduce choices that lead to losses. Because the ventral striatum (VS) contains neurons that respond to aversive stimuli and aversive stimuli can drive dopamine release in the VS, it is possible that the VS contributes to learning about aversive outcomes, including losses. However, other work suggests that the VS may play a specific role in learning to choose among rewards, with other systems mediating learning from aversive outcomes. To examine the role of the VS in learning from gains and losses, we compared the performance of macaque monkeys with VS lesions and unoperated controls on a reinforcement learning task. In the task, the monkeys gained or lost tokens, which were periodically cashed out for juice, as outcomes for choices. They learned over trials to choose cues associated with gains, and not choose cues associated with losses. We found that monkeys with VS lesions had a deficit in learning to choose between cues that differed in reward magnitude. By contrast, monkeys with VS lesions performed as well as controls when choices involved a potential loss. We also fit reinforcement learning models to the behavior and compared learning rates between groups. Relative to controls, the monkeys with VS lesions had reduced learning rates for gain cues. Therefore, in this task, the VS plays a specific role in learning to choose between rewarding options.

KW - Appetitive

KW - Aversive

KW - Neuroeconomics

KW - Reinforcement learning

KW - Ventral striatum

UR - http://www.scopus.com/inward/record.url?scp=85059215917&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059215917&partnerID=8YFLogxK

U2 - 10.1073/pnas.1809833115

DO - 10.1073/pnas.1809833115

M3 - Article

C2 - 30545910

AN - SCOPUS:85059215917

VL - 115

SP - E12398-E12406

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 52

ER -