Entropy-based metrics for predicting choice behavior based on local response to reward

Ethan Trepka, Mehran Spitmaan, Bilal A. Bari, Vincent D. Costa, Jeremiah Y. Cohen, Alireza Soltani

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

Original languageEnglish (US)
Article number6567
JournalNature communications
Volume12
Issue number1
DOIs
StatePublished - Dec 2021
Externally publishedYes

ASJC Scopus subject areas

  • Chemistry(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Physics and Astronomy(all)

Fingerprint

Dive into the research topics of 'Entropy-based metrics for predicting choice behavior based on local response to reward'. Together they form a unique fingerprint.

Cite this