An investigation of interruptions and resumptions in multi-tasking dialogues

Fan Yang, Peter Heeman, Andrew L. Kun

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

In this article we focus on human-human multi-tasking dialogues, in which pairs of conversants, using speech, work on an ongoing task while occasionally completing real-time tasks. The ongoing task is a poker game in which conversants need to assemble a poker hand, and the real-time task is a picture game in which conversants need to find out whether they have a certain picture on their displays. We employ empirical corpus studies and machine learning experiments to understand the mechanisms that people use in managing these complex interactions. First, we examine task interruptions: switching from the ongoing task to a real-time task. We find that generally conversants tend to interrupt at a less disruptive context in the ongoing task when possible. We also find that the discourse markers oh and wait occur in initiating a task interruption twice as often as in the conversation of the ongoing task. Pitch is also found to be statistically correlated with task interruptions; in fact, the more disruptive the task interruption, the higher the pitch. Second, we examine task resumptions: returning to the ongoing task after completing an interrupting real-time task. We find that conversants might simply resume the conversation where they left off, but sometimes they repeat the last utterance or summarize the critical information that was exchanged before the interruption. Third, we apply machine learning to determine how well task interruptions can be recognized automatically and to investigate the usefulness of the cues that we find in the corpus studies. We find that discourse context, pitch, and the discourse markers oh and wait are important features to reliably recognize task interruptions; and with non-lexical features one can improve the performance of recognizing task interruptions with more than a 50% relative error reduction over a baseline. Finally, we discuss the implication of our findings for building a speech interface that supports multi-tasking dialogue.

Original languageEnglish (US)
Pages (from-to)75-104
Number of pages30
JournalComputational Linguistics
Volume37
Issue number1
DOIs
StatePublished - Mar 2011

Fingerprint

multiple stress
Multitasking
Learning systems
dialogue
discourse
conversation
Display devices
learning
Experiments
time
Interruption
Resumption
experiment
interaction
performance

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence
  • Linguistics and Language
  • Language and Linguistics

Cite this

An investigation of interruptions and resumptions in multi-tasking dialogues. / Yang, Fan; Heeman, Peter; Kun, Andrew L.

In: Computational Linguistics, Vol. 37, No. 1, 03.2011, p. 75-104.

Research output: Contribution to journalArticle

@article{c981d53a78f044439b58314a807304c0,
title = "An investigation of interruptions and resumptions in multi-tasking dialogues",
abstract = "In this article we focus on human-human multi-tasking dialogues, in which pairs of conversants, using speech, work on an ongoing task while occasionally completing real-time tasks. The ongoing task is a poker game in which conversants need to assemble a poker hand, and the real-time task is a picture game in which conversants need to find out whether they have a certain picture on their displays. We employ empirical corpus studies and machine learning experiments to understand the mechanisms that people use in managing these complex interactions. First, we examine task interruptions: switching from the ongoing task to a real-time task. We find that generally conversants tend to interrupt at a less disruptive context in the ongoing task when possible. We also find that the discourse markers oh and wait occur in initiating a task interruption twice as often as in the conversation of the ongoing task. Pitch is also found to be statistically correlated with task interruptions; in fact, the more disruptive the task interruption, the higher the pitch. Second, we examine task resumptions: returning to the ongoing task after completing an interrupting real-time task. We find that conversants might simply resume the conversation where they left off, but sometimes they repeat the last utterance or summarize the critical information that was exchanged before the interruption. Third, we apply machine learning to determine how well task interruptions can be recognized automatically and to investigate the usefulness of the cues that we find in the corpus studies. We find that discourse context, pitch, and the discourse markers oh and wait are important features to reliably recognize task interruptions; and with non-lexical features one can improve the performance of recognizing task interruptions with more than a 50{\%} relative error reduction over a baseline. Finally, we discuss the implication of our findings for building a speech interface that supports multi-tasking dialogue.",
author = "Fan Yang and Peter Heeman and Kun, {Andrew L.}",
year = "2011",
month = "3",
doi = "10.1162/coli_a_00036",
language = "English (US)",
volume = "37",
pages = "75--104",
journal = "Computational Linguistics",
issn = "0891-2017",
publisher = "MIT Press Journals",
number = "1",

}

TY - JOUR

T1 - An investigation of interruptions and resumptions in multi-tasking dialogues

AU - Yang, Fan

AU - Heeman, Peter

AU - Kun, Andrew L.

PY - 2011/3

Y1 - 2011/3

N2 - In this article we focus on human-human multi-tasking dialogues, in which pairs of conversants, using speech, work on an ongoing task while occasionally completing real-time tasks. The ongoing task is a poker game in which conversants need to assemble a poker hand, and the real-time task is a picture game in which conversants need to find out whether they have a certain picture on their displays. We employ empirical corpus studies and machine learning experiments to understand the mechanisms that people use in managing these complex interactions. First, we examine task interruptions: switching from the ongoing task to a real-time task. We find that generally conversants tend to interrupt at a less disruptive context in the ongoing task when possible. We also find that the discourse markers oh and wait occur in initiating a task interruption twice as often as in the conversation of the ongoing task. Pitch is also found to be statistically correlated with task interruptions; in fact, the more disruptive the task interruption, the higher the pitch. Second, we examine task resumptions: returning to the ongoing task after completing an interrupting real-time task. We find that conversants might simply resume the conversation where they left off, but sometimes they repeat the last utterance or summarize the critical information that was exchanged before the interruption. Third, we apply machine learning to determine how well task interruptions can be recognized automatically and to investigate the usefulness of the cues that we find in the corpus studies. We find that discourse context, pitch, and the discourse markers oh and wait are important features to reliably recognize task interruptions; and with non-lexical features one can improve the performance of recognizing task interruptions with more than a 50% relative error reduction over a baseline. Finally, we discuss the implication of our findings for building a speech interface that supports multi-tasking dialogue.

AB - In this article we focus on human-human multi-tasking dialogues, in which pairs of conversants, using speech, work on an ongoing task while occasionally completing real-time tasks. The ongoing task is a poker game in which conversants need to assemble a poker hand, and the real-time task is a picture game in which conversants need to find out whether they have a certain picture on their displays. We employ empirical corpus studies and machine learning experiments to understand the mechanisms that people use in managing these complex interactions. First, we examine task interruptions: switching from the ongoing task to a real-time task. We find that generally conversants tend to interrupt at a less disruptive context in the ongoing task when possible. We also find that the discourse markers oh and wait occur in initiating a task interruption twice as often as in the conversation of the ongoing task. Pitch is also found to be statistically correlated with task interruptions; in fact, the more disruptive the task interruption, the higher the pitch. Second, we examine task resumptions: returning to the ongoing task after completing an interrupting real-time task. We find that conversants might simply resume the conversation where they left off, but sometimes they repeat the last utterance or summarize the critical information that was exchanged before the interruption. Third, we apply machine learning to determine how well task interruptions can be recognized automatically and to investigate the usefulness of the cues that we find in the corpus studies. We find that discourse context, pitch, and the discourse markers oh and wait are important features to reliably recognize task interruptions; and with non-lexical features one can improve the performance of recognizing task interruptions with more than a 50% relative error reduction over a baseline. Finally, we discuss the implication of our findings for building a speech interface that supports multi-tasking dialogue.

UR - http://www.scopus.com/inward/record.url?scp=79952791985&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79952791985&partnerID=8YFLogxK

U2 - 10.1162/coli_a_00036

DO - 10.1162/coli_a_00036

M3 - Article

AN - SCOPUS:79952791985

VL - 37

SP - 75

EP - 104

JO - Computational Linguistics

JF - Computational Linguistics

SN - 0891-2017

IS - 1

ER -