Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency

Kathleen C Y Sie, Jacqueline R. Starr, David C. Bloom, Michael Cunningham, Lianne M. De Serres, Amelia F. Drake, Ravindhra G. Elluru, Joseph Haddad, Christopher Hartnick, Carol Macarthur, Henry Milczuk, Harlan R. Muntz, Jonathan A. Perkins, Craig Senders, Marshall E. Smith, Travis Tottefson, Jay Paul Willging, Carlton J. Zdanski

Research output: Contribution to journalArticle

24 Citations (Scopus)

Abstract

Objective: To explore interrater and intrarater reliability (R inter and Rintra, respectively) of a standardized scale applied to nasoendoscopic assessment of velopharyngeal (VP) function, across multiple centers. Design: Multicenter blinded Rinter and R intra study. Setting: Eight academic tertiary care centers. Participants: Sixteen otolaryngologists from 8 centers. Main Outcome Measures: Raters estimated lateral pharyngeal and palatal movement on nasoendoscopic tapes from 50 different patients. Raters were asked to (1) estimate gap size during phonation and (2) note the presence of the Passavant ridge, a midline palatal notch on the nasal surface of the soft palate, and aberrant pulsations. Primary outcome measures were Rinter and Rintra coefficients for estimated gap size, lateral wall, and palatal movement; κ coefficients for the Passavant ridge, a mid- line palatal notch on the nasal soft palate, and aberrant pulsations were also calculated. Results: The Rinter coefficients were 0.63 for estimated gap size, 0.41 for lateral wall movement, and 0.43 for palate movement; corresponding Rintra coefficients were 0.86, 0.79, and 0.83, respectively. Interrater k values for qualitative features were 0.10 for the Passavant ridge; 0.48 for a notch on the nasal surface of the soft palate, 0.56 for aberrant pulsations, and 0.39 for estimation of gap size. Conclusions: In these data, there was good Rintra and fair R inter when using the Golding-Kushner scale for rating VP function based on nasoendoscopy. Estimates of VP gap size demonstrate higher reliability coefficients than total lateral wall, mean palate estimates, and categorical estimate of gap size. The reliability of rating qualitative characteristics (ie, the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate) is variable.

Original languageEnglish (US)
Pages (from-to)757-763
Number of pages7
JournalArchives of Otolaryngology - Head and Neck Surgery
Volume134
Issue number7
DOIs
StatePublished - Jul 2008

Fingerprint

Velopharyngeal Insufficiency
Soft Palate
Nose
Palate
Outcome Assessment (Health Care)
Phonation
Tertiary Care Centers

ASJC Scopus subject areas

  • Otorhinolaryngology

Cite this

Sie, K. C. Y., Starr, J. R., Bloom, D. C., Cunningham, M., De Serres, L. M., Drake, A. F., ... Zdanski, C. J. (2008). Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency. Archives of Otolaryngology - Head and Neck Surgery, 134(7), 757-763. https://doi.org/10.1001/archotol.134.7.757

Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency. / Sie, Kathleen C Y; Starr, Jacqueline R.; Bloom, David C.; Cunningham, Michael; De Serres, Lianne M.; Drake, Amelia F.; Elluru, Ravindhra G.; Haddad, Joseph; Hartnick, Christopher; Macarthur, Carol; Milczuk, Henry; Muntz, Harlan R.; Perkins, Jonathan A.; Senders, Craig; Smith, Marshall E.; Tottefson, Travis; Willging, Jay Paul; Zdanski, Carlton J.

In: Archives of Otolaryngology - Head and Neck Surgery, Vol. 134, No. 7, 07.2008, p. 757-763.

Research output: Contribution to journalArticle

Sie, KCY, Starr, JR, Bloom, DC, Cunningham, M, De Serres, LM, Drake, AF, Elluru, RG, Haddad, J, Hartnick, C, Macarthur, C, Milczuk, H, Muntz, HR, Perkins, JA, Senders, C, Smith, ME, Tottefson, T, Willging, JP & Zdanski, CJ 2008, 'Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency', Archives of Otolaryngology - Head and Neck Surgery, vol. 134, no. 7, pp. 757-763. https://doi.org/10.1001/archotol.134.7.757
Sie, Kathleen C Y ; Starr, Jacqueline R. ; Bloom, David C. ; Cunningham, Michael ; De Serres, Lianne M. ; Drake, Amelia F. ; Elluru, Ravindhra G. ; Haddad, Joseph ; Hartnick, Christopher ; Macarthur, Carol ; Milczuk, Henry ; Muntz, Harlan R. ; Perkins, Jonathan A. ; Senders, Craig ; Smith, Marshall E. ; Tottefson, Travis ; Willging, Jay Paul ; Zdanski, Carlton J. / Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency. In: Archives of Otolaryngology - Head and Neck Surgery. 2008 ; Vol. 134, No. 7. pp. 757-763.
@article{5ec2c7f2218e41dd8810a272cc0fc229,
title = "Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency",
abstract = "Objective: To explore interrater and intrarater reliability (R inter and Rintra, respectively) of a standardized scale applied to nasoendoscopic assessment of velopharyngeal (VP) function, across multiple centers. Design: Multicenter blinded Rinter and R intra study. Setting: Eight academic tertiary care centers. Participants: Sixteen otolaryngologists from 8 centers. Main Outcome Measures: Raters estimated lateral pharyngeal and palatal movement on nasoendoscopic tapes from 50 different patients. Raters were asked to (1) estimate gap size during phonation and (2) note the presence of the Passavant ridge, a midline palatal notch on the nasal surface of the soft palate, and aberrant pulsations. Primary outcome measures were Rinter and Rintra coefficients for estimated gap size, lateral wall, and palatal movement; κ coefficients for the Passavant ridge, a mid- line palatal notch on the nasal soft palate, and aberrant pulsations were also calculated. Results: The Rinter coefficients were 0.63 for estimated gap size, 0.41 for lateral wall movement, and 0.43 for palate movement; corresponding Rintra coefficients were 0.86, 0.79, and 0.83, respectively. Interrater k values for qualitative features were 0.10 for the Passavant ridge; 0.48 for a notch on the nasal surface of the soft palate, 0.56 for aberrant pulsations, and 0.39 for estimation of gap size. Conclusions: In these data, there was good Rintra and fair R inter when using the Golding-Kushner scale for rating VP function based on nasoendoscopy. Estimates of VP gap size demonstrate higher reliability coefficients than total lateral wall, mean palate estimates, and categorical estimate of gap size. The reliability of rating qualitative characteristics (ie, the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate) is variable.",
author = "Sie, {Kathleen C Y} and Starr, {Jacqueline R.} and Bloom, {David C.} and Michael Cunningham and {De Serres}, {Lianne M.} and Drake, {Amelia F.} and Elluru, {Ravindhra G.} and Joseph Haddad and Christopher Hartnick and Carol Macarthur and Henry Milczuk and Muntz, {Harlan R.} and Perkins, {Jonathan A.} and Craig Senders and Smith, {Marshall E.} and Travis Tottefson and Willging, {Jay Paul} and Zdanski, {Carlton J.}",
year = "2008",
month = "7",
doi = "10.1001/archotol.134.7.757",
language = "English (US)",
volume = "134",
pages = "757--763",
journal = "JAMA Otolaryngology - Head and Neck Surgery",
issn = "2168-6181",
publisher = "American Medical Association",
number = "7",

}

TY - JOUR

T1 - Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency

AU - Sie, Kathleen C Y

AU - Starr, Jacqueline R.

AU - Bloom, David C.

AU - Cunningham, Michael

AU - De Serres, Lianne M.

AU - Drake, Amelia F.

AU - Elluru, Ravindhra G.

AU - Haddad, Joseph

AU - Hartnick, Christopher

AU - Macarthur, Carol

AU - Milczuk, Henry

AU - Muntz, Harlan R.

AU - Perkins, Jonathan A.

AU - Senders, Craig

AU - Smith, Marshall E.

AU - Tottefson, Travis

AU - Willging, Jay Paul

AU - Zdanski, Carlton J.

PY - 2008/7

Y1 - 2008/7

N2 - Objective: To explore interrater and intrarater reliability (R inter and Rintra, respectively) of a standardized scale applied to nasoendoscopic assessment of velopharyngeal (VP) function, across multiple centers. Design: Multicenter blinded Rinter and R intra study. Setting: Eight academic tertiary care centers. Participants: Sixteen otolaryngologists from 8 centers. Main Outcome Measures: Raters estimated lateral pharyngeal and palatal movement on nasoendoscopic tapes from 50 different patients. Raters were asked to (1) estimate gap size during phonation and (2) note the presence of the Passavant ridge, a midline palatal notch on the nasal surface of the soft palate, and aberrant pulsations. Primary outcome measures were Rinter and Rintra coefficients for estimated gap size, lateral wall, and palatal movement; κ coefficients for the Passavant ridge, a mid- line palatal notch on the nasal soft palate, and aberrant pulsations were also calculated. Results: The Rinter coefficients were 0.63 for estimated gap size, 0.41 for lateral wall movement, and 0.43 for palate movement; corresponding Rintra coefficients were 0.86, 0.79, and 0.83, respectively. Interrater k values for qualitative features were 0.10 for the Passavant ridge; 0.48 for a notch on the nasal surface of the soft palate, 0.56 for aberrant pulsations, and 0.39 for estimation of gap size. Conclusions: In these data, there was good Rintra and fair R inter when using the Golding-Kushner scale for rating VP function based on nasoendoscopy. Estimates of VP gap size demonstrate higher reliability coefficients than total lateral wall, mean palate estimates, and categorical estimate of gap size. The reliability of rating qualitative characteristics (ie, the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate) is variable.

AB - Objective: To explore interrater and intrarater reliability (R inter and Rintra, respectively) of a standardized scale applied to nasoendoscopic assessment of velopharyngeal (VP) function, across multiple centers. Design: Multicenter blinded Rinter and R intra study. Setting: Eight academic tertiary care centers. Participants: Sixteen otolaryngologists from 8 centers. Main Outcome Measures: Raters estimated lateral pharyngeal and palatal movement on nasoendoscopic tapes from 50 different patients. Raters were asked to (1) estimate gap size during phonation and (2) note the presence of the Passavant ridge, a midline palatal notch on the nasal surface of the soft palate, and aberrant pulsations. Primary outcome measures were Rinter and Rintra coefficients for estimated gap size, lateral wall, and palatal movement; κ coefficients for the Passavant ridge, a mid- line palatal notch on the nasal soft palate, and aberrant pulsations were also calculated. Results: The Rinter coefficients were 0.63 for estimated gap size, 0.41 for lateral wall movement, and 0.43 for palate movement; corresponding Rintra coefficients were 0.86, 0.79, and 0.83, respectively. Interrater k values for qualitative features were 0.10 for the Passavant ridge; 0.48 for a notch on the nasal surface of the soft palate, 0.56 for aberrant pulsations, and 0.39 for estimation of gap size. Conclusions: In these data, there was good Rintra and fair R inter when using the Golding-Kushner scale for rating VP function based on nasoendoscopy. Estimates of VP gap size demonstrate higher reliability coefficients than total lateral wall, mean palate estimates, and categorical estimate of gap size. The reliability of rating qualitative characteristics (ie, the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate) is variable.

UR - http://www.scopus.com/inward/record.url?scp=47949107816&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=47949107816&partnerID=8YFLogxK

U2 - 10.1001/archotol.134.7.757

DO - 10.1001/archotol.134.7.757

M3 - Article

C2 - 18645127

AN - SCOPUS:47949107816

VL - 134

SP - 757

EP - 763

JO - JAMA Otolaryngology - Head and Neck Surgery

JF - JAMA Otolaryngology - Head and Neck Surgery

SN - 2168-6181

IS - 7

ER -