Development of a bench station objective structured assessment of technical skills

Barbara A. Goff, Gretchen M. Lentz, David Lee, Dee Fenner, Jamie Morris, L. S. Mandel

Research output: Contribution to journalArticle

70 Citations (Scopus)

Abstract

OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach's ∝ were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.

Original languageEnglish (US)
Pages (from-to)412-416
Number of pages5
JournalObstetrics and Gynecology
Volume98
Issue number3
DOIs
StatePublished - 2001

Fingerprint

Checklist
Salpingostomy
Anatomic Models
Aptitude
Abdominal Wall
Internship and Residency
Gynecology
Reproducibility of Results
Mental Competency
Suspensions
Urinary Bladder
Research Design
Animal Models
Costs and Cost Analysis

ASJC Scopus subject areas

  • Obstetrics and Gynecology

Cite this

Development of a bench station objective structured assessment of technical skills. / Goff, Barbara A.; Lentz, Gretchen M.; Lee, David; Fenner, Dee; Morris, Jamie; Mandel, L. S.

In: Obstetrics and Gynecology, Vol. 98, No. 3, 2001, p. 412-416.

Research output: Contribution to journalArticle

Goff, Barbara A. ; Lentz, Gretchen M. ; Lee, David ; Fenner, Dee ; Morris, Jamie ; Mandel, L. S. / Development of a bench station objective structured assessment of technical skills. In: Obstetrics and Gynecology. 2001 ; Vol. 98, No. 3. pp. 412-416.
@article{15e7afe8cb4f4c18bb1dd6db4a9e3ad8,
title = "Development of a bench station objective structured assessment of technical skills",
abstract = "OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach's ∝ were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.",
author = "Goff, {Barbara A.} and Lentz, {Gretchen M.} and David Lee and Dee Fenner and Jamie Morris and Mandel, {L. S.}",
year = "2001",
doi = "10.1016/S0029-7844(01)01473-9",
language = "English (US)",
volume = "98",
pages = "412--416",
journal = "Obstetrics and Gynecology",
issn = "0029-7844",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - Development of a bench station objective structured assessment of technical skills

AU - Goff, Barbara A.

AU - Lentz, Gretchen M.

AU - Lee, David

AU - Fenner, Dee

AU - Morris, Jamie

AU - Mandel, L. S.

PY - 2001

Y1 - 2001

N2 - OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach's ∝ were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.

AB - OBJECTIVE: We have previously shown that objective structured assessment of technical skills performed in an animal model was an innovative, reliable, and valid method of assessing surgical skills. Our goal was to develop a less costly bench station objective structured assessment of technical skills and to evaluate the feasibility, reliability, and validity of this exam. METHODS: A seven-station examination was administered to 24 residents. The tests included laparoscopic procedures (salpingostomy, intracorporeal knot tying, closure of port sites) and open abdominal procedures (subcuticular closure, bladder neck suspension, repair of enterotomy, abdominal wall closure). All tasks were performed using life-like surgical models. Residents were timed and assessed at each station using three methods of scoring: a task-specific checklist, a global rating scale, and a pass/fail grade. RESULTS: Assessment of construct validity, the ability of the test to discriminate among residency levels, found significant differences on the checklist, global rating scale, time for procedures, and pass/fail grade by level of training. Reliability indices calculated with Cronbach's ∝ were 0.77 for the checklists and 0.94 for the global rating scale. Overall interrater reliability indices were 0.91 for the global rating scale and 0.92 for the checklists. Total cost for replaceable parts and facilities was $1900. CONCLUSION: The less costly and more portable bench station objective structured assessment of technical skills can reliably and validly assess the surgical skills of gynecology residents. This type of examination can be a useful tool to identify residents who need additional surgical instruction, provide remediation, and may become a mechanism to certify surgical skill competence.

UR - http://www.scopus.com/inward/record.url?scp=0034826151&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034826151&partnerID=8YFLogxK

U2 - 10.1016/S0029-7844(01)01473-9

DO - 10.1016/S0029-7844(01)01473-9

M3 - Article

VL - 98

SP - 412

EP - 416

JO - Obstetrics and Gynecology

JF - Obstetrics and Gynecology

SN - 0029-7844

IS - 3

ER -