Association Between Resident Physician Training Experience and Program-Level Performance on Board Examinations

Development and testing of a post competency scale for traditional Chinese medicine physicians undergoing standardized training

Objective

To develop and test a post competency scale for traditional Chinese medicine (TCM) physicians undergoing standardized training to provide an applicable tool for scientific evaluation.

Methods

Based on literature analysis, behavioral event interviews, and expert consultations, measurement questions were formulated and the initial scale was designed. A questionnaire survey was conducted from July 2022 to May 2023 among TCM physicians undergoing standardized training in China. The rationality of the scale was confirmed through item purification, factor analysis, and tests of reliability and validity.

Results

The post competency scale consisted of three dimensions (TCM fundamentals and research abilities, TCM thinking and skill abilities, and personal traits and communication abilities) with 21 items. Exploratory factor analysis identified three common factors, accounting for a cumulative variance contribution rate of 62.165%. Confirmatory factor analysis demonstrated that the fit indices of the three-factor model fell within a relatively ideal level. The Cronbach's alpha coefficient of the scale was 0.885. Through convergent validity analysis, the standardized loading coefficients of the 21 items were >0.5, and the average extracted variance (AVE) of the three factors was also >0.5. Moreover, the square roots of the AVE values for each dimension exceeded the correlation coefficients between it and the other dimensions.

Conclusions

The findings suggest that the post competency scale of TCM physicians undergoing standardized training can provide a reliable scientific basis for training and assessment within China.

Factors Influencing Exam Performance of Surgical Trainees in Sub-Saharan Africa: A Retrospective Analysis of the College of Surgeons in East, Central, and Southern Africa Membership Examination

INTRODUCTION

The College of Surgeons of East, Central, and Southern Africa (COSECSA) has been expanding surgical training in sub-Saharan Africa to respond to the shortage in the region. However, acquiring surgical skills requires rigorous training, and these skills are repeatedly assessed throughout training. Therefore, understanding the factors influencing these assessments is crucial. Previous research has identified individual characteristics, educational background, curriculum structure and previous exam outcomes to influence performance. However, COSECSA's Membership of the College of Surgeons (MCS) exam has not been investigated for factors influencing performance, which this study aims to investigate.

METHODS

Data from MCS trainees who took the exam between 2015 and 2021 were analyzed. Trainee demographics, institutional affiliation, operative experience, and exam performance were considered. Linear regression models were used to analyze the factors related to written and clinical exam performance.

RESULTS

Out of 354 trainees, 228 were included in the study. Factors such as training duration, the ratio of emergency surgeries, institutional funding source, and country language were associated with written exam performance. Training duration, funding source, exposure to major surgeries, and the ratio of performing operations were significant factors for the clinical exam.

DISCUSSION

Operative experience, institutional affiliation, training duration, and language proficiency influence exam performance. Hospitals funded by faith-based organizations or nongovernmental organizations had trainees with higher scores. Prolonged training did not guarantee improved performance. Lastly, having English as an official language improved written exam scores. Gender and country of training did not significantly impact performance.

CONCLUSION

This study highlights the importance of operative experience, institutional affiliation, and language proficiency in the exam performance of surgical trainees in COSECSA. Interventions to enhance surgical training and improve exam outcomes in sub-Saharan Africa should consider these factors. Further research is needed to explore additional outcome measures and gather comprehensive data on trainee and hospital characteristics.

A scoping review of successful strategies for passing the American Board of Surgery certifying examination

Background

This scoping review identifies existing literature that investigates what factors contribute to success on the American Board of Surgery (ABS) Certifying Exam (CE) to provide practical, evidence-based recommendations.

Methods

A Pubmed search was completed utilizing the preferred reporting items for systematic reviews and meta-analysis extension for scoping review (PRISMA-ScR) method.

Results

Of 4368 articles identified, 45 articles met criteria for review. Manuscripts were placed into one of five categories: predictors from medical school, program interventions, modifiable candidate factors, the effect of mock oral exams, and those factors shown not to provide benefit for CE preparation.

Conclusions

A variety of factors have either been shown to provide benefit for or be predictive of CE performance. Acknowledgement of these factors can provide benefit to both surgery residents as well as surgery programs. Despite these findings, research into these factors is generally of low quality, prompting the need for ongoing, high-quality investigations.

Quality Metrics and Indicators for Surgical Training: A Scoping Review

BACKGROUND

Surgical training quality is critical to ensure that trainees receive adequate preparation to perform surgical procedures independently and that patients receive safe, effective, and high-quality care. Numerous surgical training quality indicators have been proposed, investigated and implemented. However, the existing evidence base for these indicators is limited, with most studies originating from English-speaking, high-income countries.

OBJECTIVES

This scoping review aimed to identify the range of quality indicators that have been proposed and evaluated in the literature, and to critically evaluate the existing evidence base for these indicators.

METHODS

A systematic literature search was conducted using MEDLINE and Embase databases to identify studies reporting on surgical training quality indicators. A total of 68 articles were included in the review.

RESULTS

Operative volume is the most commonly cited indicator and has been investigated for its effects on trainee exam performance and career progression. Other indicators include operative diversity, workplace-based assessments, regular evaluation and feedback, academic achievements, formal teaching, and learning agreements, and direct observation of procedural skills. However, these indicators are largely based on qualitative analyses and expert opinions and have not been validated quantitatively using clear outcome measures for trainees and patients.

CONCLUSIONS

Future research is necessary to establish evidence-based indicators of high-quality surgical training, including in low-resource settings. Quantitative and qualitative studies are required to validate existing indicators and to identify new indicators that are relevant to diverse surgical training environments. Lastly, any approach to surgical training quality must prioritize the benefit to both trainees and patients, ensuring training success, career progression, and patient safety.

Predictive Factors of First Time Pass Rate on the American Board of Surgery Certification in General Surgery Exams: A Systematic Review

OBJECTIVE

General Surgery residency programs are evaluated on their American Board of Surgery (ABS) Qualifying examination (QE) and Certifying examination (CE) pass rates. This systematic review aims to evaluate predictive factors of ABS QE and CE first time pass rates.

DESIGN

Using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines, the following electronic databases were searched: PubMed, Embase, JAMA Network, and Google Scholar. Studies available in the English language published between January 2000 and July 2020 were deemed eligible. Articles that did not assess either of the ABS board examinations performance and pass-rates as outcomes were excluded. The Oxford Centre for Evidence-Based Medicine was used to determine the quality and risk of bias of each study.

RESULTS

A total of 31 publications were included for analysis. Undergraduate medical education variables associated with first-time pass rates on the QE and CE include USMLE score, AOA membership, and class rank. Program factors affecting pass rates include program size, geographic location, and Program Director turnover. There is strong correlation between ABSITE and QE. Evidence supports the utility of mock oral examinations (MOEs) in predicting CE success.

CONCLUSIONS

ABSITE scores demonstrated higher correlation with QE pass rate than CE pass rate. MOEs have a positive association with first-time CE pass rates. Nonmodifiable factors such as race/ethnicity, marital status, and geographic location were also found to be predictors. Delaying board certification examination beyond 1 year after residency graduation significantly reduces first-time pass rate.

Constructing Learning Curves to Benchmark Operative Performance of General Surgery Residents Against a National Cohort of Peers

OBJECTIVE

No method or data exist to allow surgical trainees or their programs to contextualize their technical progress. The objective of this study was to create peer benchmarks for Cumulative Sum (CUSUM) charts based upon operative evaluations from a national cohort of general surgery residents.

DESIGN, SETTING, PARTICIPANTS

In 2016-2018, faculty from 26 general surgery residency programs nationwide rated 328 residents' operative performance on a case-by-case basis using a validated 5-point Likert scale. An individual case was considered a "misstep" if scoring below the national median score for that procedure in that postgraduate year (PGY). We constructed 2-sided observed-expected CUSUM charts to capture each resident's cumulative performance over time relative to the national medians. Upper (failure) and lower (positive outlier) benchmarks were established based on the PGY-specific 75th percentile and median misstep rates; consistent/repeated missteps are reflected by crossing of the upper boundary. Procedures with ≤10 observations and residents who were evaluated <10 times for each PGY were excluded.

RESULTS

Around 8,161 evaluations on 76 procedure types were analyzed. The individual misstep rate was lowest among PGY-3s at 13.3% and highest among PGY-4s at 28.6%. No interns had curves that crossed the failure boundary. 8.7% of PGY-2s and 8.9% of PGY-3s finished the year past the failure boundary. PGY-2s had the most positive outliers, with 28.3% of them demonstrating an outlying success performance beyond the lower boundary for at least once. PGY-5s most frequently failed, with 16.7% ever crossing the upper boundary and 11.1% remaining above it at graduation.

CONCLUSIONS

CUSUM is a valid statistical approach for benchmarking individual residents' operative performance against national peers as they progress through the year in real-time. With further validation, CUSUM could be used to set progression and/or graduation standards and objectively identify residents who might benefit from remediation.