Representing the reinforcement learning state in a negotiation dialogue

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Citations (Scopus)

Abstract

Most applications of Reinforcement Learning (RL) for dialogue have focused on slot-filling tasks. In this paper, we explore a task that requires negotiation, in which conversants need to exchange information in order to decide on a good solution. We investigate what information should be included in the system's RL state so that an optimal policy can be learned and so that the state space stays reasonable in size. We propose keeping track of the decisions that the system has made, and using them to constrain the system's future behavior in the dialogue. In this way, we can compositionally represent the strategy that the system is employing. We show that this approach is able to learn a good policy for the task. This work is a first step to a more general exploration of applying RL to negotiation dialogues.

Original languageEnglish (US)
Title of host publicationProceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009
Pages450-455
Number of pages6
DOIs
StatePublished - 2009
Event2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009 - Merano, Italy
Duration: Dec 13 2009Dec 17 2009

Other

Other2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009
CountryItaly
CityMerano
Period12/13/0912/17/09

Fingerprint

Reinforcement learning

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Signal Processing

Cite this

Heeman, P. (2009). Representing the reinforcement learning state in a negotiation dialogue. In Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009 (pp. 450-455). [5373413] https://doi.org/10.1109/ASRU.2009.5373413

Representing the reinforcement learning state in a negotiation dialogue. / Heeman, Peter.

Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009. 2009. p. 450-455 5373413.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Heeman, P 2009, Representing the reinforcement learning state in a negotiation dialogue. in Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009., 5373413, pp. 450-455, 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009, Merano, Italy, 12/13/09. https://doi.org/10.1109/ASRU.2009.5373413
Heeman P. Representing the reinforcement learning state in a negotiation dialogue. In Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009. 2009. p. 450-455. 5373413 https://doi.org/10.1109/ASRU.2009.5373413
Heeman, Peter. / Representing the reinforcement learning state in a negotiation dialogue. Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009. 2009. pp. 450-455
@inproceedings{a649b7f79beb4e2babfa583a52048f38,
title = "Representing the reinforcement learning state in a negotiation dialogue",
abstract = "Most applications of Reinforcement Learning (RL) for dialogue have focused on slot-filling tasks. In this paper, we explore a task that requires negotiation, in which conversants need to exchange information in order to decide on a good solution. We investigate what information should be included in the system's RL state so that an optimal policy can be learned and so that the state space stays reasonable in size. We propose keeping track of the decisions that the system has made, and using them to constrain the system's future behavior in the dialogue. In this way, we can compositionally represent the strategy that the system is employing. We show that this approach is able to learn a good policy for the task. This work is a first step to a more general exploration of applying RL to negotiation dialogues.",
author = "Peter Heeman",
year = "2009",
doi = "10.1109/ASRU.2009.5373413",
language = "English (US)",
isbn = "9781424454792",
pages = "450--455",
booktitle = "Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009",

}

TY - GEN

T1 - Representing the reinforcement learning state in a negotiation dialogue

AU - Heeman, Peter

PY - 2009

Y1 - 2009

N2 - Most applications of Reinforcement Learning (RL) for dialogue have focused on slot-filling tasks. In this paper, we explore a task that requires negotiation, in which conversants need to exchange information in order to decide on a good solution. We investigate what information should be included in the system's RL state so that an optimal policy can be learned and so that the state space stays reasonable in size. We propose keeping track of the decisions that the system has made, and using them to constrain the system's future behavior in the dialogue. In this way, we can compositionally represent the strategy that the system is employing. We show that this approach is able to learn a good policy for the task. This work is a first step to a more general exploration of applying RL to negotiation dialogues.

AB - Most applications of Reinforcement Learning (RL) for dialogue have focused on slot-filling tasks. In this paper, we explore a task that requires negotiation, in which conversants need to exchange information in order to decide on a good solution. We investigate what information should be included in the system's RL state so that an optimal policy can be learned and so that the state space stays reasonable in size. We propose keeping track of the decisions that the system has made, and using them to constrain the system's future behavior in the dialogue. In this way, we can compositionally represent the strategy that the system is employing. We show that this approach is able to learn a good policy for the task. This work is a first step to a more general exploration of applying RL to negotiation dialogues.

UR - http://www.scopus.com/inward/record.url?scp=77949374322&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77949374322&partnerID=8YFLogxK

U2 - 10.1109/ASRU.2009.5373413

DO - 10.1109/ASRU.2009.5373413

M3 - Conference contribution

SN - 9781424454792

SP - 450

EP - 455

BT - Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2009

ER -