Interrater reliability of endoscopic parameters following sinus surgery

Timothy Smith, Peter H. Hwang, Andrew H. Murr, Francois Lavigne, Andrea Koreck

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Objectives/Hypothesis: To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Study Design: Prospective cohort with retrospective review. Methods: One hundred twenty video-endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout-Fleiss test for continuous data. The level of agreement between the panel and the real-time clinical investigator was also assessed. Results: For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa = 0.499, prevalence index = 0.925) and adhesions (kappa = 0.364, prevalence index = 0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient = 0.554) and crusting (reliability coefficient = 0.620). Real-time investigator scoring and panel scoring showed strong agreement. Conclusions: These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.

Original languageEnglish (US)
Pages (from-to)230-236
Number of pages7
JournalLaryngoscope
Volume122
Issue number1
DOIs
StatePublished - Jan 2012

Fingerprint

Turbinates
Research Personnel
Inflammation
Visual Analog Scale
Endoscopy
Prospective Studies

Keywords

  • Chronic rhinosinusitis
  • endoscopic grading
  • interrater reliability
  • sinus surgery
  • visual analog scale

ASJC Scopus subject areas

  • Otorhinolaryngology

Cite this

Smith, T., Hwang, P. H., Murr, A. H., Lavigne, F., & Koreck, A. (2012). Interrater reliability of endoscopic parameters following sinus surgery. Laryngoscope, 122(1), 230-236. https://doi.org/10.1002/lary.22440

Interrater reliability of endoscopic parameters following sinus surgery. / Smith, Timothy; Hwang, Peter H.; Murr, Andrew H.; Lavigne, Francois; Koreck, Andrea.

In: Laryngoscope, Vol. 122, No. 1, 01.2012, p. 230-236.

Research output: Contribution to journalArticle

Smith, T, Hwang, PH, Murr, AH, Lavigne, F & Koreck, A 2012, 'Interrater reliability of endoscopic parameters following sinus surgery', Laryngoscope, vol. 122, no. 1, pp. 230-236. https://doi.org/10.1002/lary.22440
Smith, Timothy ; Hwang, Peter H. ; Murr, Andrew H. ; Lavigne, Francois ; Koreck, Andrea. / Interrater reliability of endoscopic parameters following sinus surgery. In: Laryngoscope. 2012 ; Vol. 122, No. 1. pp. 230-236.
@article{74657927ebf14aafb24e9855181fe31d,
title = "Interrater reliability of endoscopic parameters following sinus surgery",
abstract = "Objectives/Hypothesis: To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Study Design: Prospective cohort with retrospective review. Methods: One hundred twenty video-endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout-Fleiss test for continuous data. The level of agreement between the panel and the real-time clinical investigator was also assessed. Results: For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa = 0.499, prevalence index = 0.925) and adhesions (kappa = 0.364, prevalence index = 0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient = 0.554) and crusting (reliability coefficient = 0.620). Real-time investigator scoring and panel scoring showed strong agreement. Conclusions: These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.",
keywords = "Chronic rhinosinusitis, endoscopic grading, interrater reliability, sinus surgery, visual analog scale",
author = "Timothy Smith and Hwang, {Peter H.} and Murr, {Andrew H.} and Francois Lavigne and Andrea Koreck",
year = "2012",
month = "1",
doi = "10.1002/lary.22440",
language = "English (US)",
volume = "122",
pages = "230--236",
journal = "Laryngoscope",
issn = "0023-852X",
publisher = "John Wiley and Sons Inc.",
number = "1",

}

TY - JOUR

T1 - Interrater reliability of endoscopic parameters following sinus surgery

AU - Smith, Timothy

AU - Hwang, Peter H.

AU - Murr, Andrew H.

AU - Lavigne, Francois

AU - Koreck, Andrea

PY - 2012/1

Y1 - 2012/1

N2 - Objectives/Hypothesis: To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Study Design: Prospective cohort with retrospective review. Methods: One hundred twenty video-endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout-Fleiss test for continuous data. The level of agreement between the panel and the real-time clinical investigator was also assessed. Results: For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa = 0.499, prevalence index = 0.925) and adhesions (kappa = 0.364, prevalence index = 0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient = 0.554) and crusting (reliability coefficient = 0.620). Real-time investigator scoring and panel scoring showed strong agreement. Conclusions: These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.

AB - Objectives/Hypothesis: To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Study Design: Prospective cohort with retrospective review. Methods: One hundred twenty video-endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout-Fleiss test for continuous data. The level of agreement between the panel and the real-time clinical investigator was also assessed. Results: For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa = 0.499, prevalence index = 0.925) and adhesions (kappa = 0.364, prevalence index = 0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient = 0.554) and crusting (reliability coefficient = 0.620). Real-time investigator scoring and panel scoring showed strong agreement. Conclusions: These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.

KW - Chronic rhinosinusitis

KW - endoscopic grading

KW - interrater reliability

KW - sinus surgery

KW - visual analog scale

UR - http://www.scopus.com/inward/record.url?scp=84355162799&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84355162799&partnerID=8YFLogxK

U2 - 10.1002/lary.22440

DO - 10.1002/lary.22440

M3 - Article

C2 - 22095409

AN - SCOPUS:84355162799

VL - 122

SP - 230

EP - 236

JO - Laryngoscope

JF - Laryngoscope

SN - 0023-852X

IS - 1

ER -