Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation

Patrick Colley, Jess C. Mace, Madeleine R. Schaberg, Timothy Smith, Abtin Tabaee

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P <0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.

Original languageEnglish (US)
Pages (from-to)2259-2265
Number of pages7
JournalLaryngoscope
Volume125
Issue number10
DOIs
StatePublished - Oct 1 2015

Fingerprint

Nose
Endoscopy
Otolaryngology
Mucous Membrane
Nasal Septum
Turbinates
Education

Keywords

  • interrater agreement
  • Nasal endoscopy
  • otolaryngology
  • residency
  • sinusitis

ASJC Scopus subject areas

  • Otorhinolaryngology

Cite this

Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation. / Colley, Patrick; Mace, Jess C.; Schaberg, Madeleine R.; Smith, Timothy; Tabaee, Abtin.

In: Laryngoscope, Vol. 125, No. 10, 01.10.2015, p. 2259-2265.

Research output: Contribution to journalArticle

Colley, Patrick ; Mace, Jess C. ; Schaberg, Madeleine R. ; Smith, Timothy ; Tabaee, Abtin. / Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation. In: Laryngoscope. 2015 ; Vol. 125, No. 10. pp. 2259-2265.
@article{8dc2ffc983c245c2b36e03211dd21938,
title = "Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation",
abstract = "Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P <0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.",
keywords = "interrater agreement, Nasal endoscopy, otolaryngology, residency, sinusitis",
author = "Patrick Colley and Mace, {Jess C.} and Schaberg, {Madeleine R.} and Timothy Smith and Abtin Tabaee",
year = "2015",
month = "10",
day = "1",
doi = "10.1002/lary.25240",
language = "English (US)",
volume = "125",
pages = "2259--2265",
journal = "Laryngoscope",
issn = "0023-852X",
publisher = "John Wiley and Sons Inc.",
number = "10",

}

TY - JOUR

T1 - Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation

AU - Colley, Patrick

AU - Mace, Jess C.

AU - Schaberg, Madeleine R.

AU - Smith, Timothy

AU - Tabaee, Abtin

PY - 2015/10/1

Y1 - 2015/10/1

N2 - Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P <0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.

AB - Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P <0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.

KW - interrater agreement

KW - Nasal endoscopy

KW - otolaryngology

KW - residency

KW - sinusitis

UR - http://www.scopus.com/inward/record.url?scp=84942694168&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942694168&partnerID=8YFLogxK

U2 - 10.1002/lary.25240

DO - 10.1002/lary.25240

M3 - Article

VL - 125

SP - 2259

EP - 2265

JO - Laryngoscope

JF - Laryngoscope

SN - 0023-852X

IS - 10

ER -