Intercoder reliability in annotating complex disfluencies

Peter Heeman, Andy McMillin, J. Scott Yaruss

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.

Original languageEnglish (US)
Title of host publicationInternational Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007
Pages1957-1960
Number of pages4
Volume3
StatePublished - 2007
Event8th Annual Conference of the International Speech Communication Association, Interspeech 2007 - Antwerp, Belgium
Duration: Aug 27 2007Aug 31 2007

Other

Other8th Annual Conference of the International Speech Communication Association, Interspeech 2007
CountryBelgium
CityAntwerp
Period8/27/078/31/07

Fingerprint

Annotation
Speech
Model
Training

Keywords

  • Annotation scheme
  • Disfluencies
  • Stuttering

ASJC Scopus subject areas

  • Computer Science Applications
  • Software
  • Modeling and Simulation
  • Linguistics and Language
  • Communication

Cite this

Heeman, P., McMillin, A., & Yaruss, J. S. (2007). Intercoder reliability in annotating complex disfluencies. In International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007 (Vol. 3, pp. 1957-1960)

Intercoder reliability in annotating complex disfluencies. / Heeman, Peter; McMillin, Andy; Yaruss, J. Scott.

International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007. Vol. 3 2007. p. 1957-1960.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Heeman, P, McMillin, A & Yaruss, JS 2007, Intercoder reliability in annotating complex disfluencies. in International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007. vol. 3, pp. 1957-1960, 8th Annual Conference of the International Speech Communication Association, Interspeech 2007, Antwerp, Belgium, 8/27/07.
Heeman P, McMillin A, Yaruss JS. Intercoder reliability in annotating complex disfluencies. In International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007. Vol. 3. 2007. p. 1957-1960
Heeman, Peter ; McMillin, Andy ; Yaruss, J. Scott. / Intercoder reliability in annotating complex disfluencies. International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007. Vol. 3 2007. pp. 1957-1960
@inproceedings{8404bea5b6724b6ba5ee503641b5b8c0,
title = "Intercoder reliability in annotating complex disfluencies",
abstract = "In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.",
keywords = "Annotation scheme, Disfluencies, Stuttering",
author = "Peter Heeman and Andy McMillin and Yaruss, {J. Scott}",
year = "2007",
language = "English (US)",
isbn = "9781605603162",
volume = "3",
pages = "1957--1960",
booktitle = "International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007",

}

TY - GEN

T1 - Intercoder reliability in annotating complex disfluencies

AU - Heeman, Peter

AU - McMillin, Andy

AU - Yaruss, J. Scott

PY - 2007

Y1 - 2007

N2 - In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.

AB - In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.

KW - Annotation scheme

KW - Disfluencies

KW - Stuttering

UR - http://www.scopus.com/inward/record.url?scp=56149094457&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=56149094457&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781605603162

VL - 3

SP - 1957

EP - 1960

BT - International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007

ER -