Mock Oral Examination as an Opportunity to Generate Entrustment Data: A Pilot Study.

OBJECTIVE

Entrustable professional activities (EPAs) provide a competency-based framework for assessing readiness for independent practice. The American Board of Surgery Certifying Exam similarly evaluates readiness for independent practice and is simulated by mock oral examinations (MOEs). This study piloted an EPA-aligned MOE performance scale, evaluated performance outcomes, and assessed faculty perspectives on MOE-based entrustment assessments.

DESIGN

The MOE included 12 standardized cases featuring 6 to 7 questions each. For each case, 2 examiners independently scored all questions (correct/incorrect), assigned a case score (pass, borderline, fail), and provided an entrustment rating (practice ready, indirect supervision, direct supervision, limited participation) based on overall impression of performance. Case-specific percentage correct was defined as the percentage of questions answered correctly during each case. Mixed-effects models identified predictors of passing scores and practice-ready ratings. A postexam survey assessed faculty perspectives on MOEs as a complement to EPA assessment.

SETTING

The MOE was administered over Zoom by the (redacted institution name) General Surgery Residency Program.

PARTICIPANTS

A total of 67 residents (39% female) and 74 faculty surgeons (34% female) from 7 general surgery residency programs.

RESULTS

Of the 853 passing scores with an accompanying entrustment rating, only 36% were paired with a practice-ready rating. Higher case-specific percentage correct predicted passing scores (odds ratio 1.09, 95% confidence interval [1.08, 1.11], p < 0.001) and practice-ready ratings (odds ratio 1.02, 95% confidence interval [1.01, 1.03], p < 0.001). Most faculty perceived that MOEs allow for assessment of knowledge and judgment relevant to EPAs (88%) and would support incorporating MOEs into their EPA assessment process (74%).

CONCLUSIONS

Higher case-specific percentage correct predicted passing scores and practice-ready ratings; however, practice-ready ratings were significantly rarer, indicating a higher threshold for full entrustment. MOEs represent a potential pragmatic tool for generating supplemental entrustment data with preliminary construct validity, and faculty support their use in this capacity.