Reversal learning and dopamine

A Bayesian perspective

Vincent Costa, Valery L. Tran, Janita Turchi, Bruno B. Averbeck

Research output: Contribution to journalArticle

39 Citations (Scopus)

Abstract

Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.

Original languageEnglish (US)
Pages (from-to)2407-2416
Number of pages10
JournalJournal of Neuroscience
Volume35
Issue number6
DOIs
StatePublished - Jan 1 2015
Externally publishedYes

Fingerprint

Reversal Learning
Dopamine
Haloperidol
Bayes Theorem
Haplorhini
Learning
Choice Behavior
Levodopa
Prefrontal Cortex
Dissection
Placebos

Keywords

  • Bayesian
  • Dopamine
  • Haloperidol
  • L-DOPA
  • Reinforcement learning
  • Reversal learning

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Reversal learning and dopamine : A Bayesian perspective. / Costa, Vincent; Tran, Valery L.; Turchi, Janita; Averbeck, Bruno B.

In: Journal of Neuroscience, Vol. 35, No. 6, 01.01.2015, p. 2407-2416.

Research output: Contribution to journalArticle

Costa, Vincent ; Tran, Valery L. ; Turchi, Janita ; Averbeck, Bruno B. / Reversal learning and dopamine : A Bayesian perspective. In: Journal of Neuroscience. 2015 ; Vol. 35, No. 6. pp. 2407-2416.
@article{88a0c18fd9b549d6858797024552c68e,
title = "Reversal learning and dopamine: A Bayesian perspective",
abstract = "Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.",
keywords = "Bayesian, Dopamine, Haloperidol, L-DOPA, Reinforcement learning, Reversal learning",
author = "Vincent Costa and Tran, {Valery L.} and Janita Turchi and Averbeck, {Bruno B.}",
year = "2015",
month = "1",
day = "1",
doi = "10.1523/JNEUROSCI.1989-14.2015",
language = "English (US)",
volume = "35",
pages = "2407--2416",
journal = "Journal of Neuroscience",
issn = "0270-6474",
publisher = "Society for Neuroscience",
number = "6",

}

TY - JOUR

T1 - Reversal learning and dopamine

T2 - A Bayesian perspective

AU - Costa, Vincent

AU - Tran, Valery L.

AU - Turchi, Janita

AU - Averbeck, Bruno B.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.

AB - Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.

KW - Bayesian

KW - Dopamine

KW - Haloperidol

KW - L-DOPA

KW - Reinforcement learning

KW - Reversal learning

UR - http://www.scopus.com/inward/record.url?scp=84922572232&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84922572232&partnerID=8YFLogxK

U2 - 10.1523/JNEUROSCI.1989-14.2015

DO - 10.1523/JNEUROSCI.1989-14.2015

M3 - Article

VL - 35

SP - 2407

EP - 2416

JO - Journal of Neuroscience

JF - Journal of Neuroscience

SN - 0270-6474

IS - 6

ER -