Measuring Coaching in Undergraduate Medical Education

the Development and Psychometric Validation of New Instruments

Patricia (Patty) Carney, Erin M. Bonura, Jeff Kraakevik, Amy Miller Juve, Leslie Kahl, Nicole Deiorio

Research output: Contribution to journalArticle

Abstract

Background: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. Objective: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. Design: Observational psychometric. Participants: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches’ response rate was 71%. Main Measures: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. Key Results: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students’ assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. Conclusions: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

Original languageEnglish (US)
Pages (from-to)677-683
Number of pages7
JournalJournal of general internal medicine
Volume34
Issue number5
DOIs
StatePublished - May 15 2019

Fingerprint

Undergraduate Medical Education
Psychometrics
Medical Students
Mentoring
Educational Measurement
Educational Models

Keywords

  • academic coaching
  • undergraduate medical education

ASJC Scopus subject areas

  • Internal Medicine

Cite this

Measuring Coaching in Undergraduate Medical Education : the Development and Psychometric Validation of New Instruments. / Carney, Patricia (Patty); Bonura, Erin M.; Kraakevik, Jeff; Juve, Amy Miller; Kahl, Leslie; Deiorio, Nicole.

In: Journal of general internal medicine, Vol. 34, No. 5, 15.05.2019, p. 677-683.

Research output: Contribution to journalArticle

@article{4bd0715cdd954ef7a3cf3eeadb60baa5,
title = "Measuring Coaching in Undergraduate Medical Education: the Development and Psychometric Validation of New Instruments",
abstract = "Background: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. Objective: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. Design: Observational psychometric. Participants: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7{\%} from 2014 and 75.5{\%} from 2015. Overall, the coaches’ response rate was 71{\%}. Main Measures: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. Key Results: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students’ assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. Conclusions: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.",
keywords = "academic coaching, undergraduate medical education",
author = "Carney, {Patricia (Patty)} and Bonura, {Erin M.} and Jeff Kraakevik and Juve, {Amy Miller} and Leslie Kahl and Nicole Deiorio",
year = "2019",
month = "5",
day = "15",
doi = "10.1007/s11606-019-04888-w",
language = "English (US)",
volume = "34",
pages = "677--683",
journal = "Journal of General Internal Medicine",
issn = "0884-8734",
publisher = "Springer New York",
number = "5",

}

TY - JOUR

T1 - Measuring Coaching in Undergraduate Medical Education

T2 - the Development and Psychometric Validation of New Instruments

AU - Carney, Patricia (Patty)

AU - Bonura, Erin M.

AU - Kraakevik, Jeff

AU - Juve, Amy Miller

AU - Kahl, Leslie

AU - Deiorio, Nicole

PY - 2019/5/15

Y1 - 2019/5/15

N2 - Background: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. Objective: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. Design: Observational psychometric. Participants: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches’ response rate was 71%. Main Measures: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. Key Results: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students’ assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. Conclusions: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

AB - Background: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. Objective: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. Design: Observational psychometric. Participants: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches’ response rate was 71%. Main Measures: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. Key Results: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students’ assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. Conclusions: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

KW - academic coaching

KW - undergraduate medical education

UR - http://www.scopus.com/inward/record.url?scp=85064633205&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064633205&partnerID=8YFLogxK

U2 - 10.1007/s11606-019-04888-w

DO - 10.1007/s11606-019-04888-w

M3 - Article

VL - 34

SP - 677

EP - 683

JO - Journal of General Internal Medicine

JF - Journal of General Internal Medicine

SN - 0884-8734

IS - 5

ER -