The association of USMLE Step 1 and Step 2 CK scores with residency match specialty and location

A Linguistic Analysis of United States Navy Orthopaedic Surgery Applicant Personal Statements

INTRODUCTION

Despite the importance of linguistic analysis, no systematic research has been explored in the form of linguistic analysis on personal statements for military orthopedic surgery residency programs. This study was conducted to analyze U.S. Navy (USN) orthopedic surgery applicants' personal statements using an automated textual analysis program to assess personal statements for linguistic styles.

METHODS

A retrospective analysis of USN orthopedic applicant personal statements from application years 2016 to 2019 was performed utilizing the Linguistic Inquiry and Word Count (LIWC) software. LIWC analyzed the text for summary variables: analytical thinking, clout, authenticity, and emotional tone. We compared this analysis with Step 1 and Step 2 scores and determined whether an applicant matched.

RESULTS

A total of 94 personal statements (60,230 words) were analyzed using LIWC. The average word count was 640.7, with an average of 23 words per sentence. The average-matched applicant USMLE Step 1 and Step 2 scores were 240 and 250, respectively. When examining summary traits utilizing multiple logistic regression analysis, only analytical thinking demonstrated a statistically significant difference in matched versus unmatched applicants with a P = .011 (OR = 1.10).

CONCLUSION

As the USMLE Step 1 exam transitions from a scoring system to Pass/Fail grading, programs will look at other characteristics to determine who would likely succeed in residency. From a linguistic analysis standpoint, matched applicants' personal statements demonstrated higher analytical thinking, clout, affiliation, power, and risk focus than unmatched applicants. Unmatched applicants demonstrated higher authenticity than matched applicants.

Trends in the Main Residency Match From 2007 to 2020

Background The United States Medical Licensing Exam (USMLE) Step 1 was recently changed from a numerically scored grading system to a pass/fail grading system. Until late 2024, there will be no formal studies about the impact that the grading change will have on the match process. To thoroughly assess the impact that this change will have on the overall match process, it is important to look at what the trends in applicants' objective measures have been in the years before the change. We aim to systematically evaluate the rates of change and mean trend of objective metrics found in residency applications in the main residency match.  Methods Objective medical student data of matched and unmatched applicants were queried from the National Matching Program's Charting Outcomes in the Match Reports for the 2007 to 2020 application cycles. Data were used to create linear regression analyses and statistical tests were performed to evaluate trends over time. Results For matched applicants, there were statistically significant positive trends for the mean number of contiguous ranks (m=0.33, p<0.01), having another non-doctoral graduate degree (m=0.67, p<0.01), membership to Alpha Omega Alpha (AOA) honor society (m=0.22, p<0.01), mean USMLE Step 1 score (m=1.01, p<0.01), mean USMLE Step 2 score (m=1.68, p<0.01), mean number of research experiences (m=0.12, p<0.01), and mean number of abstracts, presentations, and publications (m=0.34, p<0.01). Additionally, there was a statistically significant negative trend for the percentage who graduated from a top 40 National Institutes of Health-funded medical school (m=-0.41, p<0.01). For unmatched applicants, there were statistically significant positive trends for having another non-doctoral graduate degree (m=0.83, p<0.01), mean USMLE Step 1 score (m=1.26, p<0.01), mean USMLE Step 2 score (m=2.27, p<0.01), mean number of research experiences (m=0.13, p<0.01), and mean number of abstracts, presentations, and publications (m=0.33, p<0.01). Conclusion Our study shows that there have been statistically significant increases in almost all objective measures in the residency application. Recent changes to the abstracts, presentations, and publications on the Step 1 scoring system will force almost all residency programs to overhaul their application process and potentially increase reliance on Step 2, research, and other nonobjective factors. For students early in their medical education, emphasis on Step 2 and research will yield increased chances of matching into residency in the future.

Pure Wisdom or Potemkin Villages? A Comparison of ChatGPT 3.5 and ChatGPT 4 on USMLE Step 3 Style Questions: Quantitative Analysis

BACKGROUND

The United States Medical Licensing Examination (USMLE) has been critical in medical education since 1992, testing various aspects of a medical student's knowledge and skills through different steps, based on their training level. Artificial intelligence (AI) tools, including chatbots like ChatGPT, are emerging technologies with potential applications in medicine. However, comprehensive studies analyzing ChatGPT's performance on USMLE Step 3 in large-scale scenarios and comparing different versions of ChatGPT are limited.

OBJECTIVE

This paper aimed to analyze ChatGPT's performance on USMLE Step 3 practice test questions to better elucidate the strengths and weaknesses of AI use in medical education and deduce evidence-based strategies to counteract AI cheating.

METHODS

A total of 2069 USMLE Step 3 practice questions were extracted from the AMBOSS study platform. After including 229 image-based questions, a total of 1840 text-based questions were further categorized and entered into ChatGPT 3.5, while a subset of 229 questions were entered into ChatGPT 4. Responses were recorded, and the accuracy of ChatGPT answers as well as its performance in different test question categories and for different difficulty levels were compared between both versions.

RESULTS

Overall, ChatGPT 4 demonstrated a statistically significant superior performance compared to ChatGPT 3.5, achieving an accuracy of 84.7% (194/229) and 56.9% (1047/1840), respectively. A noteworthy correlation was observed between the length of test questions and the performance of ChatGPT 3.5 (ρ=-0.069; P=.003), which was absent in ChatGPT 4 (P=.87). Additionally, the difficulty of test questions, as categorized by AMBOSS hammer ratings, showed a statistically significant correlation with performance for both ChatGPT versions, with ρ=-0.289 for ChatGPT 3.5 and ρ=-0.344 for ChatGPT 4. ChatGPT 4 surpassed ChatGPT 3.5 in all levels of test question difficulty, except for the 2 highest difficulty tiers (4 and 5 hammers), where statistical significance was not reached.

CONCLUSIONS

In this study, ChatGPT 4 demonstrated remarkable proficiency in taking the USMLE Step 3, with an accuracy rate of 84.7% (194/229), outshining ChatGPT 3.5 with an accuracy rate of 56.9% (1047/1840). Although ChatGPT 4 performed exceptionally, it encountered difficulties in questions requiring the application of theoretical concepts, particularly in cardiology and neurology. These insights are pivotal for the development of examination strategies that are resilient to AI and underline the promising role of AI in the realm of medical education and diagnostics.

Beyond Accreditation Council for Graduate Medical Education Required Scholarship: The Academic Journey of Plastic Surgery Residents

INTRODUCTION

Research productivity is an important part of required Accreditation Council for Graduate Medical Education scholarship during residency training and critical to trainees who intend to pursue careers in academia. This study aims to determine plastic surgery residents' experiences with and attitudes toward research.

METHODS

Accredited independent (52) and integrated (86) plastic surgery program websites were manually searched for currently active residents' names and email addresses. Identified residents were emailed a survey consisting of 25 questions through Research Electronic Data Capture.

RESULTS

A total of 45 plastic surgery residents responded to the survey request (14.6% response rate). Respondents were 57% female and 43% male, with an average age of 30.7 y. At the time of survey participation, 95% of surgery residents were involved in research endeavors, voluntarily or as part of their residency training. Of the respondents, 13 (32%) previously participated in a research fellowship compared to 28 (68%) respondents who did not. Interestingly, respondents who completed fellowships were 2.84 times (95% confidence interval: 0.52-15.38, P = 0.2269) more likely to intend continuing research endeavors after residency. Participants were most in agreement with statements suggesting their research fellowship benefitted their application in the plastic surgery match process (4 [interquartile range (IQR): 4, 4]), improved their ability to conduct research (4 [IQR: 4, 4]), and helped to better understand medical literature (4 [IQR: 3, 4]).

CONCLUSIONS

Plastic surgery programs' robust research emphasis has a favorable translation into residents' self-perceived understanding of medical literature and clinical knowledge.

Matching into competitive surgical residencies: predictors of success

Evidence-informed data may help students matching into competitive residency programs guide curricular activities, extracurricular activities, and residency career choices. We aimed to examine the characteristics of students applying to competitive surgical residencies and identify predictors of matching success. We identified the five lowest match rates for the surgical subspecialities listed in the 2020 National Resident Matching Program report to define a surgical residency as competitive. We analyzed a database from 115 United States medical schools regarding application data from 2017 to 2020. Multilevel logistic regression was used to determine predictors of matching. Statistical significance was set at p < 0.05.A total of 1,448 medical students submitted 25,549 applications. The five most competitive specialties included were plastic surgery (N = 172), otolaryngology (N = 342), neurological surgery (N = 163), vascular surgery (N = 52), orthopedic surgery (N = 679), and thoracic surgery (N = 40). We found that medical students with a geographical connection (adjusted OR, 1.65 [95% CI, 1.41 to 1.93]), and students who did an away rotation at the applied program (adjusted OR, 3.22 [95% CI, 2.75 to 3.78]) had statistically significantly increased odds of matching into a competitive surgical specialty. Furthermore, we found that students with a United States Medical Licensing Examination (USMLE) Step 1 score below 230 and Step 2 Clinical Knowledge (CK) score below 240 had increased odds of matching if they completed an away rotation at the applied program. Completing an away rotation and geographical connection to the institution may contribute more than academic criteria for selection into a competitive surgical residency after an interview. This finding may be due to less variation in academic criteria among this pool of high-performing medical students. Students with limited resources who apply to a competitive surgical specialty may be at a disadvantage given the financial burden of an away rotation.

The Timing, Factors, and Impact of USMLE Step 1 Becoming Pass/Fail on the Process of Choosing a Specialty

Background  Choosing a medical specialty is an important decision. A combination of factors influenced this decision. Student characteristics and examination performances can influence this decision. With the transition of the United States Medical Licensing Examination (USMLE) Step 1 becoming pass/fail, it is important to analyze the specialty decision process. Objective The purpose of this multimethod study is to assess when in the curriculum students choose a specialty, what factors influence their decision, and the impact of USMLE Step 1 scores on a student's assessment of competitiveness. Methods In February 2022, a survey was prepared and approved by the University of Nevada, Las Vegas (UNLV) Institutional Review Board (IRB). The survey contained multiple-choice questions and a free-response section. The survey was sent to the Class of 2022 and 2023 students at Kirk Kerkorian School of Medicine who follow a Longitudinal Integrated Clerkship. Descriptive statistics and one-sample t-tests were calculated. Results A total of 89 students completed the survey: 42 out of 60 students (70%) from the Class of 2022 and 47 out of 61 students (77%) from the Class of 2023. This study found that 78.8% of longitudinal interleaved clerkship (LInC) students committed to their specialty during the second half of the clinical year. The effects of positive and negative experiences during clerkships were most significantly different (p < 0.001). Conclusion The majority of LInC students arrive at their decision by the latter half of the clinical year. A variety of factors help students arrive at their decision. Our findings suggest that the pass/fail grading system will make it more difficult for students to assess their personal competitiveness.

Medical resident's pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study

IMPORTANCE

Standardized examinations assess both learners and training programs within the medical training system in Japan. However, it is unknown if there is an association between clinical proficiency as assessed by the General Medicine In-Training Examination (GM-ITE) and pursuing specialty.

OBJECTIVE

To determine the relative achievement of fundamental skills as assessed by the standardized GM-ITE based on pursuing career specialty among residents in the Japanese training system.

DESIGN

Nationwide cross-sectional study.

SETTING

Medical residents in Japan who attempted the GM-ITE in their first or second year were surveyed.

PARTICIPANTS

A total of 4,363 postgraduate years 1 and 2 residents who completed the GM-ITE were surveyed between January 18 and March 31, 2021.

MAIN MEASURES

GM-ITE total scores and individual scores in each of four domains assessing clinical knowledge: 1) medical interview and professionalism, 2) symptomatology and clinical reasoning, 3) physical examination and treatment, and 4) detailed disease knowledge.

RESULTS

When compared to the most pursued specialty, internal medicine, only those residents who chose general medicine achieved higher GM-ITE scores (coefficient 1.38, 95% CI 0.08 to 2.68, p = 0.038). Conversely, the nine specialties and "Other/Not decided" groups scored significantly lower. Higher scores were noted among residents entering general medicine, emergency medicine, and internal medicine and among those who trained in community hospitals with higher numbers of beds, were more advanced in their training, spent more time working and studying, and cared for a moderate but not an extreme number of patients at a time.

CONCLUSIONS

Levels of basic skill achievement differed depending on respective chosen future specialties among residents in Japan. Scores were higher among those pursuing careers in general medical fields and lower among those pursuing highly specialized careers. Residents in training programs devoid of specialty-specific competition may not possess the same motivations as those in competitive systems.

The Impact on Medical Student Stress in Relation to a Change in USMLE Step 1 Examination Score Reporting to Pass/Fail

The United States Medical Licensing Examination (USMLE) Step 1 was designed to be a benchmark measure of knowledge and has been used heavily in the residency application process. Step 1 has moved from 3-digit scoring to a pass/fail scoring system, in part to decrease the stress associated with the exam. Emerging literature suggests that this transition has led to other stresses for students. Our study compared student stress levels, both overall and in relation to Step 1, leading up to the exam between a scored cohort and pass/fail cohort. We administered to each cohort a 14-item survey that included demographics, the PSS-4 stress scale, and 6 other potential stressors. Data was analyzed using two-tailed t test for independent means and analysis of variance. We found that while there was no difference in general overall stress between the students who took Step 1 for a score and students who took Step 1 pass/fail, we did see differences in stress related to the Step 1 exam. Step 1 stress was significantly lower for the pass/fail cohort than the score cohort during the second year of medical education leading up to the exam. However, this difference in Step 1 stress between the cohorts disappeared by the dedicated study period immediately before the exam. The change in scoring appears to have decreased stress specifically related to Step 1, but this reduction was not sustained as students entered their study period to prepare for Step 1.

Educational Videos Versus Question Banks: Maximizing Medical Student Performance on the United States Medical Licensing Examination Step 1 Exam

Objective The aim of this research was to determine if medical students' use of the active study strategy of working practice questions is associated with improved performance on the United States Medical Licensing Examination (USMLE) Step 1 exam when compared to students who used the passive study strategy of watching educational videos. Methods The study used a correlational design. Participants were students from two cohorts in a United States medical school (N=164 and N=163) who completed their first two years and took the USMLE Step 1 exam. Data collected retrospectively included the number of practice questions completed, educational videos watched, Step 1 exam scores, average scores on in-class exams, and scores on the Medical College Admission Test (MCAT). Results The number of videos watched was negatively and significantly correlated with the Step 1 score for cohort 2022 (r= -0.294, α=0.01) and cohort 2023 (r= -0.175, α=0.05). The number of practice questions worked was positively and significantly correlated with the Step 1 score for cohort 2022 (r=0.176, α=0.05) and cohort 2023 (r=0.143 though not significant). The number of practice questions was a significant positive predictor of Step 1 score for cohort 2022 (β=0.141, p=0.017) and cohort 2023 (β=0.133, p=0.015). Videos were significant negative predictors for cohort 2023 (β= -0.118, p=0.034). Conclusions Answering practice questions appears to be a more effective study method than passively watching videos. Though other studies have supported the use of active learning methods, this study is unique in finding a negative correlation between test scores and the number of educational videos watched. Medical students should be urged to make the most effective use of study time by incorporating working practice questions and limiting watching educational videos.

Selection of Neurosurgical Applicants in the High-Income Developing Country Lacking Local Residency Program: A Cross-Sectional Study

Background  Neurosurgery residency became one of the most competitive specialties in the medical field worldwide, which is increasing with time in contrast to the limited positions. Therefore, the requirements for the program have increased. There are different criteria for each program, which are determined by specific factors. It has become increasingly important for medical students to be aware of the factors that affect their acceptance into the program. There was a lack of data regarding the factors that contribute to the selection of neurosurgery residents in Oman Methods  A questionnaire composed of 14 questions was conducted, using the SurveyMonkey Web site, among neurosurgeons in Oman which was distributed to the five hospitals that have neurosurgery departments in Oman. SPSS software was used in the analysis of the collected data. Results  Forty-four participants responded to the survey. Ninety-five percent of them answered all the questions. Out of all participants, only two were female participants. Standardized international exam scores, such as the United State Medical Licensing Examination and Medical Council of Canada Qualifying Examination, ranked as the most important factor with a percentage of 44, followed by interview performance with a percentage of 33. While the least important factor was the age of applicants, which 46% of the participants ranked 8. Conclusion  Most of the participants agreed that standardized exams are the most important factor in the selection of neurosurgery residents followed by interview performance, although there was no significant statistical difference between the two.

Interventions Associated With Racial and Ethnic Diversity in US Graduate Medical Education: A Scoping Review

IMPORTANCE

Racially and ethnically minoritized individuals remain underrepresented in graduate medical education relative to their proportion in the population. While many programs and initiatives have been developed to address this problem, there is little consensus regarding strategies that work to improve representation across specialties.

OBJECTIVE

To examine and synthesize evidence-based practices that have been used to increase the proportions of underrepresented in medicine (URiM) trainees at US residency and fellowship programs.

EVIDENCE REVIEW

The authors searched PubMed, Google Scholar, Embase, PsycInfo, ERIC, Cochrane Reviews, Cochrane Trials, CINAHL, Scopus, and PROSPERO electronic databases to identify relevant studies published through January 2022. They screened all titles and abstracts for relevance and read full-text articles to identify articles reporting reliable data describing the outcomes of interventions to improve racial and ethnic diversity among trainees.

FINDINGS

Twenty-seven articles were included in this review. Two studies reported on fellowship programs. The most common interventions included holistic review (48%), decreased emphasis on United States Medical Licensing Examination Step 1 scores (48%), and explicit institutional messaging regarding the importance of diversity (37%). A combination of interventions was associated with an increased number of URiM applicants, interviewees, and matriculants across various medical and surgical specialties.

CONCLUSIONS AND RELEVANCE

In this scoping review, approaches and interventions associated with increased diversity in residency and fellowship programs were identified. Continued efforts are necessary to sustain such efforts and assess long-term outcomes.

Pre-clerkship National Board of Medical Examiners Subject Examinations Versus End-of-Semester Final Examinations: How Well Do They Assess Preparedness for the United States Medical Licensing Examination Step 1?

Background The ability to provide performance insights of various United States Medical Licensing Examination (USMLE) Step 1 assessments is of great importance to medical educators. Two custom pre-clerkship assessments used at the Kirk Kerkorian School of Medicine at the University of Nevada, Las Vegas (KSOM) are National Board of Medical Examiners (NBME)-derived end-of-semester final examinations and subject examinations. The authors sought to determine if performance on these custom assessments can provide feedback on a medical student's readiness to undertake the USMLE Step 1 examination. Methodology Deidentified student performance data were provided by institutional databases for the KSOM graduating class of 2023 (N = 60). Pearson correlation analyses were utilized to evaluate the strength of the correlation between USMLE Step 1 performance and NBME subject examinations versus NBME end-of-semester final examinations. Results The results indicated that the NBME end-of-semester final examinations have a statistically higher correlation to the USMLE Step 1 score than the majority of the individual NBME subject examinations. However, the mean NBME subject examination score (Semester 1: r = 0.53, p < 0.05; Semester 2: r = 0.58, p < 0.05) demonstrated significantly higher correlation to the USMLE Step 1 performance than the NBME end-of-semester final examination score for both Semesters 1 and 2 (Semester 1: r = 0.50, p < 0.05; Semester 2: r = 0.48, p < 0.05). Conclusions These results showed that the mean of the NBME subject examination score was a better metric to assess readiness for the USMLE Step 1 than the NBME end-of-semester final examinations. However, each NBME end-of-semester final examination score showed a better correlation than the majority of the NBME subject examinations.

To Wait or Two Weeks: The Relationship Between Step 2 CK Scores and the Length of Dedicated Study Time Within a Longitudinal Interleaved Clerkship Curriculum

Introduction: Graduate medical education program directors report that United States Medical Licensing Examination (USMLE) Step 2 CK exam scores will likely have greater importance in the future selection of residents due to USMLE Step 1 transitioning to a pass/fail score as early as January 2022. With emphasis moving to the Step 2 exam, it is important to examine factors that maximize student Step 2 CK performance, such as third-year curriculum models and exam timing. This study analyzes whether or not Step 2 performance is affected by a specific length of dedicated study time within a Longitudinal Interleaved Clerkship (LInC) curriculum.

Methods: A regression model was used to predict Step 2 scores for 102 students using previous performance measures. Actual and predicted scores were compared to indicate which students overperformed or underperformed on Step 2. A t-test was used to compare the mean difference between predicted and actual performance of students who had two weeks or less of dedicated study time for Step 2 CK versus students who had a longer dedicated study period. 

Results: Students who completed Step 2 with two weeks or less of dedicated study significantly overperformed (t(100)=2.06, p=0.042) on the exam (Mean=1.61, SD=9.21) compared to students who had more than two weeks of dedicated study (Mean=-1.67, SD=6.44) in a LInC curriculum.

Conclusion: Although studies of Step 2 preparation time have shown the importance of taking Step 2 soon after completion of clinical rotations, this study adds a specific timeframe. Our findings show that a dedicated study period of two weeks or less for Step 2 CK within a LInC curriculum is associated with better performance. This study was limited to a LInC curriculum and may not apply to other clinical year curricula.

Differential Item Functioning Analysis of United States Medical Licensing Examination Step 1 Items

PURPOSE

Previous studies have examined and identified demographic group score differences on United States Medical Licensing Examination (USMLE) Step examinations. It is necessary to explore potential etiologies of such differences to ensure fairness of examination use. Although score differences are largely explained by preceding academic variables, one potential concern is that item-level bias may be associated with remaining group score differences. The purpose of this 2019-2020 study was to statistically identify and qualitatively review USMLE Step 1 exam questions (items) using differential item functioning (DIF) methodology.

METHOD

Logistic regression DIF was used to identify and classify the effect size of DIF on Step 1 items meeting minimum sample size criteria. After using DIF to flag items statistically, subject matter expert (SME) review was used to identify potential reasons why items may have performed differently between racial and gender groups, including characteristics such as content, format, wording, context, or stimulus materials. USMLE SMEs reviewed items to identify the group difference they believed was present, if any; articulate a rationale behind the group difference; and determine whether that rationale would be considered construct relevant or construct irrelevant.

RESULTS

All identified DIF rationales were relevant to the constructs being assessed and therefore did not reflect item bias. Where SME-generated rationales aligned with statistical differences (flags), they favored self-identified women on items tagged to women's health content categories and were judged to be construct relevant.

CONCLUSIONS

This study did not find evidence to support the hypothesis that group-level performance differences beyond those explained by prior academic performance variables are driven by item-level bias. Health professions examination programs have an obligation to assess for group differences, and when present, investigate to what extent, if any, measurement bias plays a role.

Race, Ethnicity, and Immigration Status in a Medical Licensing Educational Resource: a Systematic, Mixed-Methods Analysis

BACKGROUND

Medical students preparing for the United States Medical Licensing Exam (USMLE) Step 2 Clinical Knowledge (CK) Exam frequently use the UWorld Step 2 CK Question Bank (QBank). Over 90% of medical students use UWorld QBanks to prepare for at least one USMLE. Although several questions in the QBank mention race, ethnicity, or immigration status, their contributions to the QBank remain underexamined.

OBJECTIVE

We conducted a systematic, mixed-methods content analysis to assess whether and how disease conditions might be racialized throughout this popular medical education resource.

DESIGN

We screened 3537 questions in the QBank between May 28 and August 11, 2020, for mentions of race, ethnicity, or immigration status. We performed multinomial logistic regression to assess the likelihood of each racial/ethnic category occurring in either the question stem, answer explanation, or both. We used an inductive technique for codebook development and determined code frequencies.

MAIN MEASURES

We reviewed the frequency and distribution of race or ethnicity in question stems, answer choices, and answer explanations; assessed associations between disease conditions and racial and ethnic categories; and identified whether and how these associations correspond to race-, ethnicity-, or migration-based care.

RESULTS

References to Black race occurred most frequently, followed by Asian, White, and Latinx groups. Mentions of race/ethnicity varied significantly by location in the question: Asian race had 6.40 times greater odds of occurring in the answer explanation only (95% CI 1.19-34.49; p < 0.031) and White race had 9.88 times greater odds of occurring only in the question stem (95% CI 2.56-38.08; p < 0.001). Qualitative analyses suggest frequent associations between disease conditions and racial, ethnic, and immigration categories, which often carry implicit or explicit biological and genetic explanations.

CONCLUSIONS

Our analysis reveals patterns of race-based disease associations that have potential for systematic harm, including promoting incorrect race-based associations and upholding cultural conventions of White bodies as normative.

The importance of personality traits for predicting clinical clerkship grades and USMLE scores

Background: Important aspects of medical school success are thought to rely on non-cognitive attributes such as personality traits. Methods: In a sample of medical students who completed their 3rd-year clerkships, the current study examined the impact of personality traits on performance on four measures of 3rd-year clerkship grades and United States Medical Licensing Examination (USMLE) Step 2CK (Clinical Knowledge) scores. Results: We found that specific personality traits, including conscientiousness, achievement-striving, morality, and self-efficacy predicted multiple outcome measures, including Step 2CK performance. However, while we observed a clear link between personality and success during the 3rd year, we also found that Step 1 scores, not specific or grouped personality traits, best predicted all measures of clerkship performance except clerkship evaluations and, as expected, best predicted Step 2CK performance. Additionally, we created a composite personality profile for our sample by averaging the high and low scores of the Big Five personality trait scales and their subscales. However, this personality composite, which included the traits agreeableness, cooperation, Conscientiousness, and cautiousness, did not predict any of our outcome measures. Conclusions: These findings indicate that personality traits individually and in combination predict 3rd year medical school success, especially when performance may rely less on intellectual prowess, and more on personality factors. Importantly, however, Step 1 scores were usually much better predictors not only of Step 2CK and shelf exam performance, but even overall clerkship grades and Objective Structured Clinical Examination (OSCE) performance. We hope these results can be used to help clerkship directors and faculty understand that personality factors impact medical school performance and that they should be considered when evaluating the more non-cognitive aspects of student achievement.

The role research gap years play in a successful dermatology match

BACKGROUND

A new trend includes taking a dedicated year away from medical school to complete a research fellowship. There is minimal data on the benefit of a gap year. We aimed to identify if a gap year makes a dermatology applicant more successful in The Match.

METHODS

Dermatology applicants who applied to Mayo Clinic Arizona for the 2018-2019 application cycle and Mayo Clinic Rochester, Arizona, and Florida for the 2019-2020 application cycle were surveyed.

RESULTS

In total, 291 dermatology applicants completed the initial survey, and 236 completed the follow-up survey. Ninety applicants took a gap year, 198 applicants did not. There was no significant difference in match rates. When comparing match rates at top dermatology residency programs, 40.6% of gap-year applicants matched to these residencies versus 19.0% of no gap-year applicants (P < 0.01).

CONCLUSION

Applicants should weigh the opportunity costs before pursuing research gap years as they may not be universally helpful. Applicants who want to match at a top dermatology program may benefit from a research gap year. This data may have limited generalizability outside of the United States.

Prospective investigation of factors associated with success on the American College of Veterinary Emergency and Critical Care certification examination (2016-2018)

OBJECTIVE

To assess the association of candidate attributes and residency training factors with success on the American College of Veterinary Emergency and Critical Care (ACVECC) board certification examination and to develop multivariable models of first-attempt success.

DESIGN

Prospective survey-based study.

SETTING

Post-assessment ACVECC examination candidates.

ANIMALS

None.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Comprehensive surveys were distributed to ACVECC examination candidates in 2016 to 2018 after completion of their assessments, but prior to publication of examination results. Unique anonymous candidate identification numbers were used to match survey responses to outcome data from the office of the ACVECC Executive Secretary. After curation to retain only the first response from each candidate, there were 97 unique candidate responses available for analysis. Univariate analyses identified multiple factors associated with first-attempt success and multiple differences between academic and private practice residency programs. Multivariable logistic regression modeling suggested that 5 factors were independently associated with first-attempt success on the ACVECC examination, specifically younger age, more weeks of study prior to the examination, training at a facility with more ACVECC Diplomates, training at a facility with more ACVECC residents, and having no requirement to manage both Emergency Room (ER) and Critical Care (CC) cases simultaneously.

CONCLUSIONS

Numerous resident and training center factors are associated with success in the ACVECC board certification examination. Residents and training centers might be able to use these data to enhance training, but caution must be exercised because these data are associative only.

Predictors of getting a residency interview: Differences by medical specialty

OBJECTIVES

Gaining medical residency interviews has become more competitive and costly for medical students. Although limited evidence from residency programme directors indicates predictors for successfully matching into a programme, past research has not sufficiently explored the application components necessary to receive an interview offer. The present study will identify which application components are most helpful in obtaining interview offers for different medical specialties.

METHODS

Data were sourced from the Texas Seeking Transparency in Application to Residency (STAR), a survey of recently matched fourth-year American medical students who self-reported information on their residency application components and interview offers. Multi-level logistic regression analyses were employed to predict the odds of interview offer according to applicants' academic, research and extracurricular characteristics. Sub-analyses were conducted for each medical specialty.

RESULTS

Nearly 10 000 students reported information on over 419 010 applications submitted, which resulted in 164 696 interview offers. Across the sample, applicants had greater odds of receiving an interview offer if they had a geographic connection to the programme (odds ratio [OR] = 4.10, 95% confidence interval [CI] 4.00-4.20), had completed an away rotation at the programme (OR = 16.00, 95% CI 14.92-17.15), were Alpha Omega Alpha Honor Medical Society members (OR = 1.49, 95% CI 1.36-1.64), or had been inducted into the Gold Humanism Honor Society (OR = 1.50, 95% CI 1.39-1.62). Applicants had reduced odds of getting an interview if they had been required to remediate a course in medical school (OR = 0.73, 95% CI 0.64-0.83) or had failed the US Medical Licensing Examination Step 1 or Step 2 examination on their first attempt (OR = 0.40, 95% CI 0.33-0.47). Predictors of obtaining an interview varied by specialty.

CONCLUSIONS

The present findings can assist senior medical students as they prepare residency applications and identify programmes to which they will apply. Knowledge of the significant factors can help applicants more efficiently use resources to maximise their number of interview offers. Completing away rotations and selecting programmes with which applicants have geographic connections may increase their odds of receiving interview offers.

Objective predictors of intern performance

BACKGROUND

Residency programs select medical students for interviews and employment using metrics such as the United States Medical Licensing Examination (USMLE) scores, grade-point average (GPA), and class rank/quartile. It is unclear whether these metrics predict performance as an intern. This study tested the hypothesis that performance on these metrics would predict intern performance.

METHODS

This single institution, retrospective cohort analysis included 244 graduates from four classes (2015-2018) who completed an Accreditation Council for Graduate Medical Education (ACGME) certified internship and were evaluated by program directors (PDs) at the end of the year. PDs provided a global assessment rating and ratings addressing ACGME competencies (response rate = 47%) with five response options: excellent = 5, very good = 4, acceptable = 3, marginal = 2, unacceptable = 1. PDs also classified interns as outstanding = 4, above average = 3, average = 2, and below average = 1 relative to other interns from the same residency program. Mean USMLE scores (Step 1 and Step 2CK), third-year GPA, class rank, and core competency ratings were compared using Welch's ANOVA and follow-up pairwise t-tests.

RESULTS

Better performance on PD evaluations at the end of intern year was associated with higher USMLE Step 1 (p = 0.006), Step 2CK (p = 0.030), medical school GPA (p = 0.020) and class rank (p = 0.016). Interns rated as average had lower USMLE scores, GPA, and class rank than those rated as above average or outstanding; there were no significant differences between above average and outstanding interns. Higher rating in each of the ACGME core competencies was associated with better intern performance (p < 0.01).

CONCLUSIONS

Better performance as an intern was associated with higher USMLE scores, medical school GPA and class rank. When USMLE Step 1 reporting changes from numeric scores to pass/fail, residency programs can use other metrics to select medical students for interviews and employment.

Current Applicant Perceptions of the Ophthalmology Residency Match

Importance

The ophthalmology residency application process is critical for applicants and residency programs, and knowledge about the preferences of applicants would assist both groups in improving the process.

Objective

To evaluate the experiences and preferences of ophthalmology residency applicants.

Design, Setting, and Participants

This cross-sectional, nonvalidated survey was conducted online. All applicants to the Bascom Palmer Eye Institute ophthalmology residency program during the 2018-2019 application cycle were invited to complete the survey. Data collection occurred from April 1, 2019, to April 30, 2019.

Main Outcomes and Measures

Applicant demographics, application submissions, interview experiences, financial considerations, match results, and suggestions for improvement of the application process.

Results

Responses were received from 185 applicants (36.4%), including 77 women (41.6%). A successful match into an ophthalmology residency was achieved by 172 respondents (93.0%). There was a mean (SD) US Medical Licensing Examination Step 1 score of 245.8 (13.3) points. Respondents applied to a mean (SD) of 76.4 (23.5) ophthalmology residency programs, received 14.0 (9.0) invitations to interview, and attended 10.3 (4.4) interviews. Choices regarding applications and interviews were based mostly on program reputation, location, and advisor recommendation. A usual lead time of at least 3 weeks between the invitation and interview was reported by 126 respondents (69.2%), which was reduced to 14 respondents (15.1%) when a wait-list was involved. The ophthalmology residency application process cost a mean (SD) of $5704 ($2831) per applicant. Respondents reported that they were most able to reduce costs through housing choices (hotel stays or similar arrangements) and least able to reduce costs by limiting the number of programs to which they applied or at which they interviewed.

Conclusions and Relevance

The ophthalmology residency application process is complex and poses substantial challenges to applicants and residency programs. These findings suggest that many current applicants have difficulty selecting programs to apply to, and most respondents desired changes to the current system of interview invitations and scheduling.

The Residency Selection Process in the New World; Time for a Rethink

The transition of the USMLE to a pass fail system provides a unique opportunity to re-evaluate the residency selection process.

Trends in the Academic Credentials of Matched Dermatology Residency Applicants

Introduction Research can be used to enhance the competitiveness of an application and is associated with a successful match. However, current reports regarding the publication record among prospective dermatology residents may be inaccurate. We sought to accurately assess the research credentials of matched dermatology residency candidates at the time of application. Methods We performed a bibliometric analysis to identify published articles of 1152 matched dermatology candidates and calculated the h-index of each applicant at the time of application. Details on article type, first authorship, and dermatology-relatedness of articles were collected. Results The median number of publications was two and the median h-index was 0. At the time of residency application, one-quarter of matched dermatology candidates (24%, n=278) possessed no publications. Over time, the median number of publications (R 0.10, p<0.001) and h-index (R 0.07, p=0.014) of matched applicants increased. The proportion of first-authored articles, dermatology-related papers, and each article type remained constant across application cycles (p>0.0500). An additional graduate degree, completion of a research fellowship, and graduation from a non-US medical school were independently associated with greater research credentials (p<0.0500). Conclusions Each year, applicants are publishing more articles and have a greater scholarly impact than in previous application cycles. However, the verified publication volume of matched dermatology applicants is strikingly lower than the values reported in national statistics.

Implications of the United States Medical Licensing Examination Step 1 Examination Transition to Pass/Fail on Medical Students Education and Future Career Opportunities

BACKGROUND

United States Medical Licensing Examination (USMLE) Step 1 will transition to pass/fail score by 2022. We aim to investigate US medical students' perspectives on the potential implications this transition would have on their education and career opportunities.

METHODS

A cross-sectional study investigating US medical students' perspectives on the implications of transition of the USMLE Step 1 exam to pass/fail. Students were asked their preferences regarding various aspects of the USMLE Step 1 examination, including activities, educational opportunities, expenses regarding preparation for the examination, and future career opportunities.

RESULTS

215 medical students responded to the survey, 59.1% were women, 80.9% were allopathic vs. 19.1% osteopathic students. 34.0% preferred the USMLE Step 1 to be graded on a pass/fail score, whereas 53.5% preferred a numeric scale. Osteopathic vs. allopathic students were more likely to report that the pass/fail transition will negatively impact their residency match (aOR = 1.454, 95% CI: 0.515, 4.106) and specialty of choice (aOR = 3.187, 95% CI: 0.980, 10.359). 57.7% of respondents reported that the transition to a pass/fail grading system will change their study habits.

CONCLUSIONS

The transition of the USMLE Step 1 to a pass/fail system has massive implications on medical students and residency programs alike. Though the majority of medical students did not prefer the USMLE Step 1 to have a pass/fail score, they must adapt their strategies to remain competitive for residency applications. Residency programs should create a composite score based off all aspects of medical students' applications in order to create a holistic and fair evaluation and ranking system.

Medical Simulation-Based Learning Outcomes in Pre-Clinical Medical Education

Introduction Medical simulation is widely used in the United States medical curriculum. However, learning outcomes based on simulation have yet to be reported. In this study, we aim to characterize the objective performance of first- and second-year medical students following eight weeks of medical simulation-based learning. Methods First- (n=25) and second-year (n=15) medical students were recruited for this study. We designed and administered a novel pre-experience examination to collect participant demography and assess simulation and non-simulation knowledge. Following 14 high-fidelity simulation scenarios over the course of eight weeks, we administered an identical post-experience examination and compared performance, primarily using a within-subjects analytic design. Results Student performance improved by an average of 18% following the medical simulation experience, and first-year students demonstrated greater benefit (22%) as compared to second-years (12%). Relative to first-years, second-year students showed higher overall performance on both pre- and post-examination. Demographic factors and prior medical experience were not significantly associated with assessment performance and score improvement. Conclusions Our data supported the efficacy of simulation-based learning as evidenced by the significant improvement in objective performance on a standardized examination. That is, both first- and second-year medical students demonstrated test-score improvement following an eight-week medical simulation program. Of note, the first-year students exhibited greater benefit (at the group level). Importantly, these findings were statistically unrelated to participant demographic and background variables. Collectively, this study provides preliminary evidence that medical simulation in the pre-clinical phase of undergraduate medical education is an effective tool for student learning.

A Comparison of Selected Items Found in Graduation Survey Instruments from MD and DO Schools. What It Reveals About Satisfaction with Career Choice

Selected items in graduation survey instruments from MD and DO schools were compared using a novel combined approach which revealed meaningful information about career choice. Although the student satisfaction with medical education had remained steady during the past decade for both MD and DO programs, the dissatisfaction with medical programs at time of graduation was different (p < 0.001). The level of unhappiness with career choice was also different (p < 0.001). An analysis of the Year Two Questionnaire, introduced in 2015 by the American Association of Medical Colleges, showed dissatisfaction with career choice in MD programs increased after graduation.

Do Year-Out Programs Make Medical Students More Competitive Candidates for Orthopedic Surgery Residencies?

OBJECTIVE

As orthopedic surgery residency programs are becoming more competitive, medical students interested in orthopedics are increasingly completing "year-out" programs. This study sought to evaluate student and faculty perceptions of these programs.

DESIGN

A survey evaluating baseline characteristics was disseminated to directors of year-out programs identified through postings on the orthopedics forum, Orthogate.org. A second survey was sent to all program directors (PDs) of accredited US orthopedic residencies, while a final survey was distributed to participants identified by year-out PDs.

SETTING

Ninety-six orthopedic year-out programs at 56 institutions were contacted.

PARTICIPANTS

Twenty-six year-out programs, 72 PDs of ACGME-accredited orthopedic residencies, and 34 year-out participants from 6 programs completed our questionnaires.

RESULTS

73.1% (19) year-out program provided funding to participants, averaging $30,368. 84.6% (22) reported >75% match rates into orthopedics for participants. 65.4% (17) of programs selected students between their MS3/MS4 school years. 4.2% (3) of residency PDs agreed or strongly agreed that year-out programs were important factors for consideration in residency programs, compared with 82.4% (28) of year-out participants and 69.2% (18) of year-out PDs (p < 0.001). 58.8% (2) of year-out participants cited completion of a year-out for improving the chance of matching into any orthopedic residency, while 85.3% (29) wanted to be more competitive for top programs. The average Step 1 score was 248, which was insignificantly different from the national average for matched orthopedic applicants.

CONCLUSIONS

Orthopedic year-out programs have dramatically increased in number over the last 20 years. Most of these programs are funded, 1-year clinical research fellowships with relative match success for participants pursuing orthopedic residencies. While year-out PDs and students consider participation in such programs to be an important factor for residency applications, and often participate in them in order to improve their competitiveness for matching at desired programs, residency PDs overall hold different views.

Assessment of burnout in medical students using the Maslach Burnout Inventory-Student Survey: a cross-sectional data analysis

BACKGROUND

Medical student burnout can cause emotional and physical exhaustion and detachment. The objectives of this study were to evaluate burnout using the Maslach Burnout Inventory-Student Survey (MBI-SS), identify factors that may predict burnout, and assess wellness initiatives effectiveness at reducing burnout.

METHODS

The MBI-SS was administered to all medical students (Classes 2019 to 2022) at the University of Illinois College of Medicine (UICOM) from February to May 2019. Factor analysis and internal consistency of the MBI-SS were assessed. Mean MSBI-SS subscale scores for burnout were calculated for cynicism (CY), emotional exhaustion (EE), and academic efficacy (AE). Multiple regression analysis was used to identify student factors that may predict burnout.

RESULTS

A total of 273 (21.6%) UICOM students completed the survey and 110 (40.3%) respondents reported self-perceived burnout. MBI-SS subscale scores were significantly higher for CY and EE, and significantly lower for AE in students who reported suffering from burnout versus students who did not report burnout. Mean ± SD subscale scores for CY, EE, and AE in burnout students were 14.44 ± 5.59, 23.23 ± 4.74, and 24.81 ± 5.35, respectively. In comparison, mean ± SD subscale scores for CY, EE, and AE in non-burnout students were 7.59 ± 5.16, 14.96 ± 5.71, and 28.74 ± 3.21, respectively. Regression analysis denoted significant associations between burnout and being out-of-phase in the curriculum, the effectiveness of wellness initiatives, and strength of motivation for medical school (SMMS) in both the two- and three-dimensional MBI-SS models. Gender was significantly associated with burnout in only the two-dimensional model.

CONCLUSIONS

Self-reported burnout in medical students at UICOM was validated using the MBI-SS. Being out-of-phase in the curriculum, being female, rating wellness initiatives as less effective, and demonstrating lower motivation for continued medical school education may be used as predictors of medical student burnout. This investigation may act as a guide for measuring burnout in medical student populations and how the implementation of wellness initiatives may ameliorate burnout.

The Evolving Impact of COVID-19 on Medical Student Orthopedic Education: Perspectives From Medical Students in Different Phases of the Curriculum

Introduction:

The world-wide lockdown caused by Coronavirus Disease 2019 (COVID-19) has upended the trajectories of lives everywhere. The medical profession has been on the front lines of this rapidly developing situation, which in turn has called for unprecedented changes in the medical school curriculum. These changes have severe implications for medical students interested in applying to competitive surgical specialties like orthopedics.

Methods:

As medical students in 3 different class years pursuing orthopedic surgery, we provide our perspectives on the impact that COVID-19 has had on medical student orthopedic education.

Results:

With the removal of away rotations and a shift to virtual interviews, rising fourth year medical students are arguably the most impacted as they prepare for the orthopedic residency application process. Third year students, who are in the exploratory phase of choosing a specialty, also face uncertainties in the shift to a “new” clerkship experience that may (1) be of shorter duration, (2) implement shifts to limit overcrowding of clinical space, and (3) increase the use of telehealth over direct patient contact.

Discussion:

The COVID-19 pandemic has altered the course of medical students’ orthopedic education in unprecedented ways. We believe the following suggestions may be helpful for students seeking alternative, supplemental ways of learning: (1) read up on major orthopedic journals, (2) reach out to orthopedic surgeons in areas of interest, (3) reach out to program directors/medical clerkship directors/program coordinators for opportunities to attend their educational curriculum virtually, (4) attend online lectures and hospital grand rounds, and (5) practice suturing technique with a practice kit.

Conclusions:

While the medical education landscape remains uncertain amid the evolving conditions of COVID-19, as medical students we strive to learn from this pandemic and respond to future unforeseen challenges with resilience, dedication, and compassion: all qualities we admire in orthopedic surgeons.

On Step 1 Mania, USMLE Score Reporting, and Financial Conflict of Interest at the National Board of Medical Examiners

Though intended to inform a binary decision on initial medical licensure, the United States Medical Licensing Examination (USMLE) is frequently used for screening candidates for residency positions. Some have argued that reporting results as pass/fail would honor the test's purpose while preventing inappropriate use. To date, the USMLE's sponsor organizations have declined to make such a change. In this Perspective, the authors examine the history and mission of the National Board of Medical Examiners (NBME), trace the rise of "Step 1 mania," and consider the current financial incentives for the NBME in implementing a pass/fail score-reporting policy.The NBME was founded in 1915 to address the lack of interstate reciprocity in medical licensure examination. With the creation of the USMLE in 1992, a single pathway for licensure was established, and the organization's original mission was achieved. Yet even after fulfilling its primary purpose, the NBME-classified as a nonprofit organization-has seen its revenues rise dramatically over the past 2 decades. Much of the increased revenue is derived from test products and services not required for medical licensure, with sales driven by the increasing importance of Step 1 scores in residency selection. Revenue from these products and services would likely decline if the NBME reported Step 1 results as pass/fail.A financial conflict of interest occurs when a judgment concerning a primary interest may be influenced by a secondary interest, such as financial gain. The data presented here demonstrate that the NBME has a conflict of interest in its current score-reporting policy. Possible remedies, such as disclosure, recusal, divestiture, and restructuring, are considered.

Preparation for the United States Medical Licensing Examinations in the Face of COVID-19

The COVID-19 pandemic has led to the canceling and rescheduling of the United States Medical Licensing Examination (USMLE) examinations due to the nationwide closure of the Prometric testing centers, which poses a significant challenge to medical students. The rescheduling of a high-stakes board examination leads to significant stress and potential burnout. Students may need guidance to decrease anxiety and reframe their study plan while maintaining their knowledge. Here, we combined board examination coaching tips with specific worked examples to describe how to prevent burnout, give recommendations for scheduling, and suggest practical approaches to USMLE and other high-stakes examinations.

Strategies to Encourage Medical Student Interest in Urology

PURPOSE OF REVIEW

Urology residency positions have steadily increased but applications have remained stagnant. This is an alarming trend given the aging general population and thus increased need for urologists. The purpose of this review is to describe barriers and suggest strategies to encourage medical students to pursue urology.

RECENT FINDINGS

Barriers to interest in urology include educational factors, such as timing of exposure to urology in medical school, USMLE scores, research experience, and deciding in time for an early match, as well as socioeconomic barriers, such as cost, being underrepresented in medicine, and gender. Steps the urological community can take include increasing involvement in medical school curricula, increasing faculty mentor availability, and broadening students' range of urological experiences. Strategies to encourage interest in urology fall into three categories: creating interest, supporting interest, and removing barriers for students considering urology. Ultimately, the goal is to garner excellent residents in a field that must expand to meet the needs of a growing and aging population.

Separate but Equal? The Sorting of USMDs and Non-USMDs in Internal Medicine Residency Programs

BACKGROUND

The US internal medicine workforce relies on international and osteopathic medical graduates to fill gaps in residency. Little is known about the distribution and impact of IMGs, DOs, and USMDs concentrating in different types of IM programs.

OBJECTIVE

Determining the extent to which USMDs, DOs, and IMGs concentrate in different types of IM programs and comparing Board pass rates by program concentration.

DESIGN, SETTINGS, AND PARTICIPANTS

This survey study used data from the AMA's FREIDA database for 476 non-military IM programs in 2017-2018, and 2016-2018 ABIM exam pass rates for 388 accredited programs.

MEASUREMENTS

Outcomes were (1) program concentration based on percentage of residents who were USMDs, IMGs, and DOs in 2017-2018 and (2) 2016-2018 program ABIM pass rates as proxies for program quality. Key independent variables were hospital type (community-based, community-based university-affiliated, or university-based) when program concentration was the outcome, and program concentration when Board pass rates were the outcome.

RESULTS

Twenty-five percent of programs were "USMD-dominated," 17% were "DO-dominated," 42% were "IMG dominated," and 16% were "integrated." The chances that a university hospital was USMD-dominated were 32 percentage points higher than that for a community hospital (AME = 0.32, baseline probability = 0.11, 95% CI, 0.17-0.46, P < .001). USMD-dominated programs also had significantly higher pass rates by 4.0 percentage points (AME = 0.04, baseline proportion = 0.90, 95% CI, 0.02-0.06, P < .001) than integrated programs, while DO-dominated programs had significantly lower pass rates (AME = - 0.1, baseline proportion = 0.90, 95% CI, - 0.15 to - 0.04, P < .001).

CONCLUSION

USMDs and non-USMDs systematically cluster in certain types of residency programs and their training may not be equal, as measured by board pass rates.

Correlation of Medical College Admission Test Scores and Self-assessment Materials with the United States Medical Licensing Examination Step 1 Performance

Purpose

Candidates' performance on the United States Medical Licensing Examination (USMLE) Step 1 examination had been correlated with the Medical College Admission Test (MCAT). However, in 2015, a new MCAT format was released and its correlation with Step 1 remains to be fully analyzed. Preparation for Step 1 typically involves purchasing and perusing practice tests from the National Board of Medical Examiners (NBME) and UWorld; however, their predictive value to performance on Step 1 remains to be ascertained, especially with the release of five new NBME practice tests. Additionally, there is a need for accurately predicting Step 1 scores to self-evaluate study progress and reduce student anxiety.

Rationale

Program directors rank USMLE Step 1 scores as the number one criterion in selecting interviewees for residency. Step 1 scores are more important than Step 2 scores, Dean’s letter, or other letters of recommendation in determining the overall ranking of a candidate after interviews.

Hypotheses

The authors hypothesized that the new MCAT scores correlated positively with Step 1 scores and that the new NBME practice tests were more predictive of performance on Step 1 as compared to old NBME tests. 

Methods

Linear regression analysis followed by either analysis of variance (ANOVA) or Student's t-tests were used to analyze 399 responses. Data obtained was used to update an existing Step 1 score predictor, which was then validated.

Results

A positive correlation between the MCAT (average score: 510.1 ± 6.3) and Step 1 scores (average score: 246.1 ± 14.2) was observed. While new NBME practice tests were more predictive of Step 1 scores than old NBME tests, UWorld test scores were the most predictive. Students who practiced with the new NBME practice tests scored significantly higher than students who did not use them. However, students using any of the UWorld practice tests did significantly better than students who practiced using only NBME practice tests but not UWorld practice tests. Ironically, NBME16,the second-most correlativetest to Step 1 performance, is no longer available for purchase. Overall, taking six or more practice tests significantly enhanced Step 1 scores; the optimal number of tests was found to be between six and nine. The predicted score by an updated Step 1 score predictor was within 3.8 points or 1.6% of the actual Step 1 score.

Conclusions

We believe this study will aid in the selection and purchase of appropriate self-assessment tests as preparatory material for the USMLE Step 1 examination. It will also introduce them to an existing Step 1 score predictor that will help determine their readiness for Step 1.

Patterns of medical student engagement in a second-year pathophysiology course: relationship to USMLE Step 1 performance

Historically, attendance has been a marker of academic performance, but the current medical education literature has had mixed results. In addition, attendance is dropping in the preclinical curricula, whereas, at the same time, the focus on United States Medical Licensing Examination Step 1 performance is increasing. This present study is a mixed-method approach correlating student attendance and access to the formal curriculum in a second-year pathophysiology course to performance on Step 1. Additionally, survey and focus group data evaluated the usage and importance of both the formal curriculum and third-party resources. Out of 112 eligible students, 77 participated in the study. There was no correlation between attendance or access to the learning materials and Step 1 performance. There was a strong correlation between the performance on the final examination and that of Step 1 (r = 0.813; P < 0.001) and a moderate correlation between formative quiz (r = 0.321; P = 0.005) and individual readiness assessment test performance (r = 0.351; P = 0.002) and Step 1 performance. Survey and focus group data show that students place high importance on faculty-developed materials that they can use on their own, but not attendance. The third-party resources are highly used as an adjunct to the formal curriculum and to focus on Step 1 study. Attendance and access to the formal curriculum do not predict Step 1 performance, whereas performance on high- and low-stakes internal assessments do. Further study on how the lack of social interaction gained from attendance affects development of other competencies and the learning climate are warranted.

Underrepresented Minorities are Underrepresented Among General Surgery Applicants Selected to Interview

OBJECTIVE

Diversity is an ill-defined entity in general surgery training. The Accreditation Council for Graduate Medical Education recently proposed new common program requirements including verbiage requiring diversity in residency. "Recruiting" for diversity can be challenging within the constraints of geographic preference, type of program, and applicant qualifications. In addition, the Match process adds further uncertainty. We sought to study the self-identified racial/ethnic distribution of general surgery applicants to better ascertain the characteristics of underrepresented minorities (URM) within the general surgery applicant pool.

DESIGN

Program-specific data from the Electronic Residency Application Service was collated for the 2018 medical student application cycle. Data were abstracted for all participating programs' applicants and those selected to interview. Applicants who did not enter a self-identified race/ethnicity were excluded from analysis. URM were defined as those identifying as Black/African American, Hispanic/Latino/of Spanish origin, American Indian/Alaskan Native, or Native Hawaiian/Pacific Islander-Samoan. Appropriate statistical analyses were accomplished.

SETTING

Ten general surgery residency programs-5 independent programs and 5 university programs.

PARTICIPANTS

Residency applicants to the participating general surgery residency programs.

RESULTS

Ten surgery residency programs received 10,312 applications from 3192 unique applicants. Seven hundred and seventy-eight applications did not include a self-identified race/ethnicity and were excluded from analysis. The racial/ethnic makeup of applicants in this study cohort was similar to that from 2017 to 2018 Electronic Residency Application Service data of 4262 total applicants to categorical general surgery. Programs received a median of 1085 (range: 485-1264) applications each and altogether selected 617 unique applicants for interviews. Overall, 2148 applicants graduated from US medical schools, and of those, 595 (28%) were offered interviews. The mean age of applicants was 28.8 ± 3.8 years and 1316 (41%) were female. Hispanic/Latino/of Spanish origin, Black, and American Indian/Alaskan Native/Hawaiian/Pacific Islander-Samoan applicants constituted 12%, 8%, and 1% of total applicants, but only 8%, 6%, and 1% of those selected for interview. Overall, 29% of applicants had United States Medical Licensing Examination (USMLE) Step 1 scores ≤220; 37 (6%) of those selected for interviews had a USMLE Step 1 score of ≤220. A higher proportion of URM applicants had USMLE scores ≤220 compared to White and Asian applicants. Non-white self-identification was a significant independent predictor of a lower likelihood of interview selection. Female gender, USMLE Step 1 score >220, and graduating from a US medical school were associated with an increased likelihood of being selected to interview.

CONCLUSIONS

URM applicants represented a disproportionately smaller percentage of applicants selected for interview. USMLE Step 1 scores were lower among the URM applicants. Training programs that use discreet USMLE cutoffs are likely excluding URM at a higher rate than their non-URM applicants. Attempts to recruit racially/ethnically diverse trainees should include program-level analysis to determine disparities and a focused strategy to interview applicants who might be overlooked by conventional screening tools.

Building a Medical Education Outcomes Center: Development Study

BACKGROUND

Medical education outcomes and clinical data exist in multiple unconnected databases, resulting in 3 problems: (1) it is difficult to connect learner outcomes with patient outcomes, (2) learners cannot be easily tracked over time through the education-training-practice continuum, and (3) no standard methodology ensures quality and privacy of the data.

OBJECTIVE

The purpose of this study was to develop a Medical Education Outcomes Center (MEOC) to integrate education data and to build a framework to standardize the intake and processing of requests for using these data.

METHODS

An inventory of over 100 data sources owned or utilized by the medical school was conducted, and nearly 2 dozen of these data sources have been vetted and integrated into the MEOC. In addition, the American Medical Association (AMA) Physician Masterfile data of the University of Minnesota Medical School (UMMS) graduates were linked to the data from the National Provider Identifier (NPI) registry to develop a mechanism to connect alumni practice data to education data.

RESULTS

Over 160 data requests have been fulfilled, culminating in a range of outcomes analyses, including support of accreditation efforts. The MEOC received data on 13,092 UMMS graduates in the AMA Physician Masterfile and could link 10,443 with NPI numbers and began to explore their practice demographics. The technical and operational work to expand the MEOC continues. Next steps are to link the educational data to the clinical practice data through NPI numbers to assess the effectiveness of our medical education programs by the clinical outcomes of our graduates.

CONCLUSIONS

The MEOC provides a replicable framework to allow other schools to more effectively operate their programs and drive innovation.

Factors Predictive of Performance on USMLE Step 1: Do Commercial Study Aids Improve Scores?

BACKGROUND

The United States Medical Licensing Examination (USMLE) Step 1 was the most cited factor in granting medical student applicants' residency interviews in a 2016 NRMP survey. Medical students utilize numerous commercial resources to achieve the highest score possible. This study sought to examine preparatory resources and their association with USMLE Step 1 performance.

METHODS

In 2016 and 2017, 170 medical students from a public, Midwestern medical school completed a voluntary survey after completing USMLE Step 1. The survey asked about study aids, anticipated performance, test preparation time, and practice exam performance. Actual scores and academic performance (overall percent for years 1 and 2 of medical school) were matched to survey responses. Data was analyzed using descriptive and inferential statistical analyses.

RESULTS

USMLE Step 1 performance was associated with academic performance (r = 0.719, p < 0.01). Post hoc tests indicated a positive difference with the UWorld Qbank and exam performance (F _4,134 = 10.29, p < 0.05). Amount of days dedicated to test preparation was negatively correlated with performance in the 2017 student cohort (r = - 0.304, p < 0.01). The use of other commercially available products and the number of practice tests completed were not correlated with performance.

DISCUSSION

Cumulative pre-clinical grades had the strongest association with performance on the USMLE Step 1. Completing UWorld Qbank at least once was positively related with Step 1 scores. However, the use of other study aids, completing multiple practice exams, and delaying exam dates were not statistically significant. This study indicates performance in the pre-clinical curriculum and increased usage of UWorld are strongly associated with USMLE Step 1 performance.

Performing Under Pressure: Varsity Athletes Excel in Medical School

PURPOSE

The admission process for medical school relies on objective and subjective measures of personal achievement, and selecting successful medical students is a daunting task for admission committees. While there is a significant body of literature examining MCAT scores and undergraduate grade point average (GPA) with medical school performance, there is a paucity of research on impact of specific student accomplishments on future performance. We hypothesize participation in a varsity collegiate sport will correlate with higher performance during medical school.

METHODS

A retrospective review of four medical school classes at a single institution was completed. Each student's admission application was examined for at least 1 year of participation in a varsity-level sport at their respective undergraduate institution. A total of 62 athletes (16.36%) were identified out of 441 total students. Multiple medical school performance metrics were obtained for each student.

RESULTS

There was no difference in MCAT scores between athletes and non-athlete medical students. There was a significant difference in step 1, step 2 CK, NBME shelf exams, cumulative year 3 performance, and AOA status with the athletes outperforming their peers.

CONCLUSION

Students who participate in collegiate varsity athletics excel in medical school. One explanation for this finding may be participation in high-stakes athletic training and competition results in development of specific attributes beneficial in medical school. These attributes may include receptiveness to criticism, time management, resiliency, team participation, and performing under pressure. Additional research is needed to elucidate the attributes that determine improved medical school performance, such that medical educators can utilize this knowledge to better prepare all students for the rigors of medical school.