Predicting Quality of Clinical Performance From Cardiology Fellowship Applications

Factors associated with infectious diseases fellowship academic success

BACKGROUND

A multitude of factors are considered in an infectious disease (ID) training program's meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes.

METHODS

In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (1 July 2013- 30 June 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation.

RESULTS

Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. The USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores.

CONCLUSIONS

Multiple aspects of a prospective fellow's application must be considered as part of a holistic review process for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

Factors Associated with Infectious Diseases Fellowship Academic Success

Background

A multitude of factors are considered in an infectious diseases (ID) training program's meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes.

Methods

In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (July 1, 2013- June 30, 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation.

Results

Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores.

Conclusions

Multiple aspects of a prospective fellow's application must be considered as part of a holistic reviewprocess for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

Predictive factors of first-time failure on the American Board of Colorectal Surgery certifying and qualifying examinations

Objective

To discover if first-attempt failure of the American Board of Colon and Rectal Surgery (ABCRS) board examination is associated with surgical training or personal demographic characteristics.

Methods

Current colon and rectal surgery program directors in the United States were contacted via email. Deidentified records of trainees from 2011 to 2019 were requested. Analysis was performed to identify associations between individual risk factors and failure on the ABCRS board examination on the first attempt.

Results

Seven programs contributed data, totaling 67 trainees. The overall first-time pass rate was 88% (n = 59). Several variables demonstrated potential for association, including Colon and Rectal Surgery In-Training Examination (CARSITE) percentile (74.5 vs 68.0, P = 0.09), number of major cases in colorectal residency (245.0 vs 219.2, P = 0.16), >5 publications during colorectal residency (75.0% vs 25.0%, P = 0.19), and first-time passage of the American Board of Surgery certifying examination (92.5% vs 7.5%, P = 0.18).

Conclusion

The ABCRS board examination is a high-stakes test, and training program factors may be predictive of failure. Although several factors showed potential for association, none reached statistical significance. Our hope is that by increasing our data set, we will identify statistically significant associations that can potentially benefit future trainees in colon and rectal surgery.

Validity of a cardiology fellow performance assessment: reliability and associations with standardized examinations and awards

BACKGROUND

Most work on the validity of clinical assessments for measuring learner performance in graduate medical education has occurred at the residency level. Minimal research exists on the validity of clinical assessments for measuring learner performance in advanced subspecialties. We sought to determine validity characteristics of cardiology fellows' assessment scores during subspecialty training, which represents the largest subspecialty of internal medicine. Validity evidence included item content, internal consistency reliability, and associations between faculty-of-fellow clinical assessments and other pertinent variables.

METHODS

This was a retrospective validation study exploring the domains of content, internal structure, and relations to other variables validity evidence for scores on faculty-of-fellow clinical assessments that include the 10-item Mayo Cardiology Fellows Assessment (MCFA-10). Participants included 7 cardiology fellowship classes. The MCFA-10 item content included questions previously validated in the assessment of internal medicine residents. Internal structure evidence was assessed through Cronbach's α. The outcome for relations to other variables evidence was overall mean of faculty-of-fellow assessment score (scale 1-5). Independent variables included common measures of fellow performance.

FINDINGS

Participants included 65 cardiology fellows. The overall mean ± standard deviation faculty-of-fellow assessment score was 4.07 ± 0.18. Content evidence for the MCFA-10 scores was based on published literature and core competencies. Cronbach's α was 0.98, suggesting high internal consistency reliability and offering evidence for internal structure validity. In multivariable analysis to provide relations to other variables evidence, mean assessment scores were independently associated with in-training examination scores (beta = 0.088 per 10-point increase; p = 0.05) and receiving a departmental or institutional award (beta = 0.152; p = 0.001). Assessment scores were not associated with educational conference attendance, compliance with completion of required evaluations, faculty appointment upon completion of training, or performance on the board certification exam. R² for the multivariable model was 0.25.

CONCLUSIONS

These findings provide sound validity evidence establishing item content, internal consistency reliability, and associations with other variables for faculty-of-fellow clinical assessment scores that include MCFA-10 items during cardiology fellowship. Relations to other variables evidence included associations of assessment scores with performance on the in-training examination and receipt of competitive awards. These data support the utility of the MCFA-10 as a measure of performance during cardiology training and could serve as the foundation for future research on the assessment of subspecialty learners.