Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia

M. J. Watson, D. M. Wong, R. Kluger, A. Chuan, M. D. Herrick, I. Ng, D. J. Castanelli, L. Lin, A. K. Lansdown, M. J. Barrington

Research output: Contribution to journalArticlepeer-review

24 Scopus citations

Abstract

Assessment tools must be investigated for reliability, validity and feasibility before being implemented. In 2013, the Australian and New Zealand College of Anaesthetists introduced workplace-based assessments, including a direct observation of a procedural skills assessment tool. The objective of this study was to evaluate the psychometric properties of this assessment tool for ultrasound-guided regional anaesthesia. Six experts assessed 30 video-recorded trainee performances of ultrasound-guided regional anaesthesia. Inter-rater reliability, assessed using absolute agreement intraclass correlation coefficients, varied from 0.10 to 0.49 for the nine individual nine-point scale items, and was 0.25 for a 'total score' of all items. Internal consistency was measured by correlation between 'total score' and 'overall performance' scale item (r = 0.68, p < 0.001). Construct validity was demonstrated by the 'total score' correlating with trainee experience (r = 0.51, p = 0.004). The mean time taken to complete assessments was 6 min 35 s.

Original languageEnglish (US)
Pages (from-to)604-612
Number of pages9
JournalAnaesthesia
Volume69
Issue number6
DOIs
StatePublished - Jun 2014
Externally publishedYes

ASJC Scopus subject areas

  • Anesthesiology and Pain Medicine

Fingerprint Dive into the research topics of 'Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia'. Together they form a unique fingerprint.

Cite this