Background: Mock oral examinations (MOE) are used to prepare residents and assess their readiness for the American Board of Surgery Certifying Exam (ABSCE). Numerous prior studies have demonstrated a positive correlation between trainee participation in MOEs and first-time pass rates on the oral boards. Delivery of MOEs varies by institution and previous studies have demonstrated significant implementation barriers such as availability of faculty examiners and exam scenarios.

Objective: To assess the value and participant satisfaction of a standardized multi-institutional MOE for general surgery residents.

Methods: 33 general surgery residents and 37 faculty members from 3 institutions participated in a regional MOE. Residents were examined in three 20-minute sessions. Faculty examiners were given a wide selection of pre-scripted exam scenarios and instructed to use standardized grading rubrics during a brief orientation on the day of the exam. All participants were surveyed on their overall experience and results were analyzed by descriptive and kappa statistics, two-tailed pair t-test, and percent agreement.

Results: Of the 33 participating residents, 26 (79%) passed the MOE (92% of R5, 91% R4, and 50% of R3). Response rates were 91% for residents and 57% for faculty members. Most respondents were satisfied with the overall exam experience (88%), standardized question quality (86%), and question variety (82%). A total of 92% of respondents agreed that the time, effort, and cost of the MOE was justified by its educational value to residents. Higher medical knowledge ratings assigned by faculty examiners correlated with stronger trainee performance (b = 0.48; 95% confidence interval [CI] 0.29-0.66), while patient care and interpersonal communication skill ratings were not associated with trainee performance. The standardized grading rubric achieved moderate inter-rater reliability among examiner pairs with 70.6% agreement (Kappa 0.47).

Conclusions: General Surgery residents and faculty perceived the standardized multi-institutional MOE to be a highly satisfactory educational experience and valuable assessment tool. Developing a repertoire of pre-scripted exam scenarios made it feasible to recruit sufficient faculty participants, and standardizing grading rubrics allowed for a consistent exam experience with moderate inter-rater reliability.