Value of Standardized Testing in Surgical Training

Characteristics Associated With Outstanding General Surgery Residency Graduate Performance, as Rated by Surgical Educators

Importance

Characteristics of outstanding graduating surgical residents are currently undefined. Identifying these qualities may be important in guiding resident selection and resident education.

Objective

To determine characteristics that are most strongly associated with being rated as an outstanding graduating surgical resident.

Design, Setting, and Participants

The multi-institutional study had 3 phases. First, an expert panel developed a list of characteristics embodied by top graduating surgical residents. Second, groups of faculty from 14 US general-surgery residency programs ranked 2017 through 2020 graduates into quartiles of overall performance. Third, faculty evaluated their graduates on each characteristic using a 5-point Likert scale. Data were analyzed using Spearman rank-order correlation to identify which individual characteristics were associated with overall graduate performance. A least absolute shrinkage and selection operator (LASSO) ordinal regression was performed to select a parsimonious model to predict the outcome of overall performance rating from individual characteristic scores.

Main Outcome and Measures

Surgical educators' rankings of general surgery residency graduates' overall performance.

Results

Fifty faculty from 14 US residency programs with a median of 13 (range, 5-30) years of surgical education experience evaluated 297 general surgery residency graduates. Surgical educators identified 21 characteristics that they believed outstanding graduating surgical residents possessed. Two hundred ninety-seven surgical residency graduates were evaluated. Higher scores in every characteristic correlated with better overall performance. Characteristics most strongly associated with higher overall performance scores were surgical judgment (r = 0.728; P < .001), leadership (r = 0.726; P < .001), postoperative clinical skills (r = 0.715; P < .001), and preoperative clinical skills (r = 0.707; P < .001). The remainder of the characteristics were moderately associated with overall performance. The LASSO regression model identified 3 characteristics from which overall resident performance could be accurately predicted without measuring other qualities: surgical judgment (odds ratio [OR] per 1 level of 5-level Likert scale OR, 1.27; 95% CI, 1.03-1.51), leadership (OR, 1.27; 95% CI, 1.06-1.48), and medical knowledge (OR, 1.16; 95% CI, 1.01-1.33).

Conclusions and Relevance

All individual characteristics identified by surgical educators as being qualities of outstanding graduating surgical residents were positively associated with overall graduate performance. Surgical judgment and leadership skills had the strongest individual associations. Assessment of only 3 qualities (surgical judgment, leadership, and medical knowledge) were required to predict overall resident performance ratings. These findings highlight the importance of developing specific surgical judgment and leadership skills curricula and assessments during surgical residency.

An Online, Modular Curriculum Enhances Surgical Education and Improves Learning Outcomes in East, Central, and Southern Africa: A Mixed-Methods Study

Objective

We aimed to determine the impact of a standardized curriculum on learning outcomes for surgical trainees in East, Central, and Southern Africa (ECSA).

Background

As surgical education expands throughout ECSA, there is a recognized need for a standardized curriculum. We previously described the design of a novel, large-scale, flipped-classroom, surgical curriculum for trainees in ECSA.

Methods

In January 2020, the first year of curricular content for trainees of the College of Surgeons of ECSA was released, containing 11 monthly thematic topics, each with 2 to 5 weekly modular subtopics. We aimed to evaluate 3 outcomes utilizing data sources incorporated into the curriculum structure. Learner engagement was assessed by the number of trainees completing curriculum topics. User experience was evaluated using quantitative and qualitative feedback responses to embedded surveys for each content week. Curriculum impact on trainee examination performance was assessed by comparing certification examination scores stratified by the number of curricular topics each trainee completed.

Results

Two hundred seventy-one trainees (96%) in 17 countries accessed at least 1 weekly module. Trainees completed a median of 9 topics (interquartile range: 6-10). The feedback survey response rate was 92% (5742/6233). Quantitative and qualitative responses were positive in overall module value (93.7% + 2.6%), amount of learning experienced (97.9% + 1.4%), confidence in achieving learning objectives (97.1% + 2.4%), and ease of use of the module (77.6% + 5.98%). Topic-related certification examination performance improved significantly with increased completion of thematic topics.

Conclusions

A standardized surgical curriculum in ECSA demonstrated excellent trainee usage, positive feedback, and improved examination scores.