What Does Honors Mean? National Analysis of Medical School Clinical Clerkship Grading

Analysis of variability and trends in medical school clerkship grades

Background

Medical school clerkship grades are used to evaluate orthopedic surgery residency applicants, however, high interinstitutional variability in grade distribution calls into question the utility of clerkship grades when evaluating applicants from different medical schools. This study aims to evaluate the variability in grade distribution among medical schools and look for trends in grade distribution over recent years.

Methods

Applications submitted to Rush University's orthopedic surgery residency program from 2015, 2019, and 2022 were collected from the Electronic Residency Application Service. Applications from the top 100 schools according to the 2023-2024 U.S. News and World Report Research Rankings were reviewed. The percentage of "honors" grades awarded by medical schools for the surgery and internal medicine clerkships were extracted from applicants' Medical Student Performance Evaluation letters.

Results

The median percentage of honors given in 2022 was 36.0 % (range 10.0-82.0) for the surgery clerkship and 33.0 % (range 6.7-80.0) for the internal medicine clerkship. Honors were given 6.6 % more in the surgery clerkship in 2022 compared to 2015. There was a negative correlation between a higher (worse) U.S. News and World Report research ranking and the percentage of honors awarded in 2022 for the surgery and internal medicine clerkships.

Conclusion

There is substantial interinstitutional variability in the rate that medical schools award an "honors" grade with evidence of grade inflation in the surgery clerkship. Residency programs using clerkship grades to compare applicants should do so cautiously provided the variability demonstrated in this study.

The importance of USMLE step 2 on the screening and selection of applicants for general surgery residency positions

Background

As announced by the Federation of State Medical Boards (FSMB) and National Board of Medical Examiners (NBME), the United States Medical Licensing Examination (USMLE) Step 1 score reporting has transitioned to pass/fail outcomes instead of the traditional numeric score after January 26, 2022. USMLE Step 1 scores have been used widely as a crucial tool in screening and selecting applicants for residency programs. This study aims to determine the role of USMLE Step 2 in the selection of applicants for general surgery residency.

Methods

A retrospective study was conducted over six recruiting cycles from 2016 to 2021. The data from 334 interviewed applicants from one general surgery residency program were assessed. Data analyzed included USMLE Step 1 and Step 2 scores, applicant gender, Alpha Omega Alpha (AOA) status, letters of recommendation (LOR), and research/publications (RS).

Results

Of the 334 interviewed applicants, 209 (62.6%) were male. The mean [SD] USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores were 239.6 [±10.4] and 249.2 [±11.4], respectively. The mean (SD) LOR and RS scores were 4.24 [±0.4] and 3.9 [±0.7], respectively. A positive correlation was observed between USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores (r = 0.60, p < .001), LOR scores (r = 0.24, p = .008), and AOA status (r = 0.19, p = .038). There was a negligible correlation between USMLE scores and applicant gender.

Conclusion

Transitioning USMLE Step 1 to pass/fail will make the initial screening and selection process of applications challenging for residency programs. In the short term, USMLE Step 2 scores, LOR, and AOA status are important as screening assessments. Valid measures to ensure appropriate, equitable, and fair assessments are needed.

Fixing a Broken Clerkship Assessment Process: Reflections on Objectivity and Equity Following the USMLE Step 1 Change to Pass/Fail

Clerkship grading is a core feature of evaluation for medical students' skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.

Does Objective Scoring of Applications for Radiology Residency Affect Diversity?

RATIONALE AND OBJECTIVES

It is important to recognize if reliance on certain factors in applications affects selection of trainees from under-represented groups. Our purpose is to determine if objective scoring of radiology residency applications based on quantifiable data regarding academic performance, United States Medical Licensing Examination (USMLE) scores and research productivity affects selection of female and under-represented minority (URM) candidates.

MATERIALS AND METHODS

We reviewed 502 applications from three successive match cycles from United States allopathic medical students. Each application was scored for academic performance, USMLE results and research productivity determining an overall score. The scores of males were compared to females and URM were compared to non-URM candidates. USMLE cutoff scores were evaluated for disparate effects.

RESULTS

There were 348 male, 154 female, 73 URM and 429 non-URM candidates. For male versus female applicants, there was no significant difference in mean academic performance, USMLE Step 2 Clinical Knowledge and research productivity scores. Males had higher mean USMLE Step 1 (p = 0.005) and overall candidate scores (p = 0.02). Between URM and non-URM candidates there was no significant difference in academic performance. Non-URM applicants had higher mean USMLE Step 1 (p = 0.008), USMLE Step 2 (p = 0.002), research productivity (p = 0.001) and overall scores (p = 0.02). Use of USMLE cutoff scores demonstrated disparate effects on female and URM candidates.

CONCLUSION

Objective scoring of applications and use of USMLE cutoff scores can disadvantage candidates from underrepresented groups. Screening filters can affect the diversity of candidate pools for radiology residency.