Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale

Andrew R. Pines, Mohammed S. Alghoul, Youssef J. Hamade, Mithun G. Sattur, Rami James N. Aoun, Tariq K. Halasa, Chandan Krishna, Samer G. Zammar, Najib El Tecle, Tarek Y. El Ahmadieh, Salah G. Aoun, Richard W. Byrne, James S. Harrop, Brian T. Ragel, Daniel K. Resnick, Russell R. Lonser, Nathan Selden, Bernard R. Bendok

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

BACKGROUND: The potential for simulation-based learning in neurosurgical training has led the Congress of Neurosurgical Surgeons to develop a series of simulationmodules. The Northwestern Objective Microanastomosis Assessment Tool (NOMAT) was created as the corresponding assessment tool for the Congress of Neurosurgical Surgeons Microanastomosis Module. The face and construct validity of the NOMAT have been previously established. OBJECTIVE: To further validate the NOMAT by determining its interrater reliability (IRR) between raters of varying levels of microsurgical expertise. METHODS: The NOMAT was used to assess residents' performance in a microanastomosis simulation module in 2 settings: Northwestern University and the Society of Neurological Surgeons 2014 Boot Camp at the University of Indiana. At Northwestern University, participants were scored by 2 experienced microsurgeons. At the University of Indiana, participants were scored by 2 postdoctoral fellows and an experienced microsurgeon. The IRR of NOMATwas estimated by computing the intraclass correlation coefficient using SPSS v22.0 (IBM, Armonk, New York). RESULTS: A total of 75 residents were assessed. At Northwestern University, 21 residents each performed microanastomosis on 2 model vessels of different sizes, one 3 mm and one 1 mm. At the University of Indiana, 54 residents performed a single microanastomosis procedure on3-mm vessels. The intraclass correlation coefficient of the totalNOMAT scores was 0.88 at Northwestern University and 0.78 at the University of Indiana. CONCLUSION: This study indicates high IRR for the NOMAT. These results suggest that the use of raters with varying levels of expertise does not compromise the precision or validity of the scale. This allows for a wider adoption of the scale and, hence, a greater potential educational impact.

Original languageEnglish (US)
Pages (from-to)108-112
Number of pages5
JournalOperative Neurosurgery
Volume13
Issue number1
DOIs
StatePublished - 2017

Fingerprint

Reproducibility of Results
Learning
Neurosurgeons
Surgeons

Keywords

  • Microanastomosis
  • Northwestern Objective Microanastomosis Assessment Tool
  • Objective structured assessment of technical skill
  • Resident training
  • Simulation education
  • Simulation in neurosurgery
  • Validity

ASJC Scopus subject areas

  • Clinical Neurology
  • Surgery

Cite this

Pines, A. R., Alghoul, M. S., Hamade, Y. J., Sattur, M. G., Aoun, R. J. N., Halasa, T. K., ... Bendok, B. R. (2017). Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale. Operative Neurosurgery, 13(1), 108-112. https://doi.org/10.1227/NEU.0000000000001403

Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale. / Pines, Andrew R.; Alghoul, Mohammed S.; Hamade, Youssef J.; Sattur, Mithun G.; Aoun, Rami James N.; Halasa, Tariq K.; Krishna, Chandan; Zammar, Samer G.; El Tecle, Najib; El Ahmadieh, Tarek Y.; Aoun, Salah G.; Byrne, Richard W.; Harrop, James S.; Ragel, Brian T.; Resnick, Daniel K.; Lonser, Russell R.; Selden, Nathan; Bendok, Bernard R.

In: Operative Neurosurgery, Vol. 13, No. 1, 2017, p. 108-112.

Research output: Contribution to journalArticle

Pines, AR, Alghoul, MS, Hamade, YJ, Sattur, MG, Aoun, RJN, Halasa, TK, Krishna, C, Zammar, SG, El Tecle, N, El Ahmadieh, TY, Aoun, SG, Byrne, RW, Harrop, JS, Ragel, BT, Resnick, DK, Lonser, RR, Selden, N & Bendok, BR 2017, 'Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale', Operative Neurosurgery, vol. 13, no. 1, pp. 108-112. https://doi.org/10.1227/NEU.0000000000001403
Pines, Andrew R. ; Alghoul, Mohammed S. ; Hamade, Youssef J. ; Sattur, Mithun G. ; Aoun, Rami James N. ; Halasa, Tariq K. ; Krishna, Chandan ; Zammar, Samer G. ; El Tecle, Najib ; El Ahmadieh, Tarek Y. ; Aoun, Salah G. ; Byrne, Richard W. ; Harrop, James S. ; Ragel, Brian T. ; Resnick, Daniel K. ; Lonser, Russell R. ; Selden, Nathan ; Bendok, Bernard R. / Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale. In: Operative Neurosurgery. 2017 ; Vol. 13, No. 1. pp. 108-112.
@article{ecc9c1d010f5463fb1f118c93628a2a9,
title = "Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale",
abstract = "BACKGROUND: The potential for simulation-based learning in neurosurgical training has led the Congress of Neurosurgical Surgeons to develop a series of simulationmodules. The Northwestern Objective Microanastomosis Assessment Tool (NOMAT) was created as the corresponding assessment tool for the Congress of Neurosurgical Surgeons Microanastomosis Module. The face and construct validity of the NOMAT have been previously established. OBJECTIVE: To further validate the NOMAT by determining its interrater reliability (IRR) between raters of varying levels of microsurgical expertise. METHODS: The NOMAT was used to assess residents' performance in a microanastomosis simulation module in 2 settings: Northwestern University and the Society of Neurological Surgeons 2014 Boot Camp at the University of Indiana. At Northwestern University, participants were scored by 2 experienced microsurgeons. At the University of Indiana, participants were scored by 2 postdoctoral fellows and an experienced microsurgeon. The IRR of NOMATwas estimated by computing the intraclass correlation coefficient using SPSS v22.0 (IBM, Armonk, New York). RESULTS: A total of 75 residents were assessed. At Northwestern University, 21 residents each performed microanastomosis on 2 model vessels of different sizes, one 3 mm and one 1 mm. At the University of Indiana, 54 residents performed a single microanastomosis procedure on3-mm vessels. The intraclass correlation coefficient of the totalNOMAT scores was 0.88 at Northwestern University and 0.78 at the University of Indiana. CONCLUSION: This study indicates high IRR for the NOMAT. These results suggest that the use of raters with varying levels of expertise does not compromise the precision or validity of the scale. This allows for a wider adoption of the scale and, hence, a greater potential educational impact.",
keywords = "Microanastomosis, Northwestern Objective Microanastomosis Assessment Tool, Objective structured assessment of technical skill, Resident training, Simulation education, Simulation in neurosurgery, Validity",
author = "Pines, {Andrew R.} and Alghoul, {Mohammed S.} and Hamade, {Youssef J.} and Sattur, {Mithun G.} and Aoun, {Rami James N.} and Halasa, {Tariq K.} and Chandan Krishna and Zammar, {Samer G.} and {El Tecle}, Najib and {El Ahmadieh}, {Tarek Y.} and Aoun, {Salah G.} and Byrne, {Richard W.} and Harrop, {James S.} and Ragel, {Brian T.} and Resnick, {Daniel K.} and Lonser, {Russell R.} and Nathan Selden and Bendok, {Bernard R.}",
year = "2017",
doi = "10.1227/NEU.0000000000001403",
language = "English (US)",
volume = "13",
pages = "108--112",
journal = "Operative Neurosurgery",
issn = "2332-4252",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Assessment of the interrater reliability of the congress of neurological surgeons microanastomosis assessment scale

AU - Pines, Andrew R.

AU - Alghoul, Mohammed S.

AU - Hamade, Youssef J.

AU - Sattur, Mithun G.

AU - Aoun, Rami James N.

AU - Halasa, Tariq K.

AU - Krishna, Chandan

AU - Zammar, Samer G.

AU - El Tecle, Najib

AU - El Ahmadieh, Tarek Y.

AU - Aoun, Salah G.

AU - Byrne, Richard W.

AU - Harrop, James S.

AU - Ragel, Brian T.

AU - Resnick, Daniel K.

AU - Lonser, Russell R.

AU - Selden, Nathan

AU - Bendok, Bernard R.

PY - 2017

Y1 - 2017

N2 - BACKGROUND: The potential for simulation-based learning in neurosurgical training has led the Congress of Neurosurgical Surgeons to develop a series of simulationmodules. The Northwestern Objective Microanastomosis Assessment Tool (NOMAT) was created as the corresponding assessment tool for the Congress of Neurosurgical Surgeons Microanastomosis Module. The face and construct validity of the NOMAT have been previously established. OBJECTIVE: To further validate the NOMAT by determining its interrater reliability (IRR) between raters of varying levels of microsurgical expertise. METHODS: The NOMAT was used to assess residents' performance in a microanastomosis simulation module in 2 settings: Northwestern University and the Society of Neurological Surgeons 2014 Boot Camp at the University of Indiana. At Northwestern University, participants were scored by 2 experienced microsurgeons. At the University of Indiana, participants were scored by 2 postdoctoral fellows and an experienced microsurgeon. The IRR of NOMATwas estimated by computing the intraclass correlation coefficient using SPSS v22.0 (IBM, Armonk, New York). RESULTS: A total of 75 residents were assessed. At Northwestern University, 21 residents each performed microanastomosis on 2 model vessels of different sizes, one 3 mm and one 1 mm. At the University of Indiana, 54 residents performed a single microanastomosis procedure on3-mm vessels. The intraclass correlation coefficient of the totalNOMAT scores was 0.88 at Northwestern University and 0.78 at the University of Indiana. CONCLUSION: This study indicates high IRR for the NOMAT. These results suggest that the use of raters with varying levels of expertise does not compromise the precision or validity of the scale. This allows for a wider adoption of the scale and, hence, a greater potential educational impact.

AB - BACKGROUND: The potential for simulation-based learning in neurosurgical training has led the Congress of Neurosurgical Surgeons to develop a series of simulationmodules. The Northwestern Objective Microanastomosis Assessment Tool (NOMAT) was created as the corresponding assessment tool for the Congress of Neurosurgical Surgeons Microanastomosis Module. The face and construct validity of the NOMAT have been previously established. OBJECTIVE: To further validate the NOMAT by determining its interrater reliability (IRR) between raters of varying levels of microsurgical expertise. METHODS: The NOMAT was used to assess residents' performance in a microanastomosis simulation module in 2 settings: Northwestern University and the Society of Neurological Surgeons 2014 Boot Camp at the University of Indiana. At Northwestern University, participants were scored by 2 experienced microsurgeons. At the University of Indiana, participants were scored by 2 postdoctoral fellows and an experienced microsurgeon. The IRR of NOMATwas estimated by computing the intraclass correlation coefficient using SPSS v22.0 (IBM, Armonk, New York). RESULTS: A total of 75 residents were assessed. At Northwestern University, 21 residents each performed microanastomosis on 2 model vessels of different sizes, one 3 mm and one 1 mm. At the University of Indiana, 54 residents performed a single microanastomosis procedure on3-mm vessels. The intraclass correlation coefficient of the totalNOMAT scores was 0.88 at Northwestern University and 0.78 at the University of Indiana. CONCLUSION: This study indicates high IRR for the NOMAT. These results suggest that the use of raters with varying levels of expertise does not compromise the precision or validity of the scale. This allows for a wider adoption of the scale and, hence, a greater potential educational impact.

KW - Microanastomosis

KW - Northwestern Objective Microanastomosis Assessment Tool

KW - Objective structured assessment of technical skill

KW - Resident training

KW - Simulation education

KW - Simulation in neurosurgery

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=85019750092&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019750092&partnerID=8YFLogxK

U2 - 10.1227/NEU.0000000000001403

DO - 10.1227/NEU.0000000000001403

M3 - Article

VL - 13

SP - 108

EP - 112

JO - Operative Neurosurgery

JF - Operative Neurosurgery

SN - 2332-4252

IS - 1

ER -