Looking Beyond the Numbers: Increasing Diversity and Inclusion Through Holistic Review in General Surgery Recruitment

General surgery applicant perspectives: Two years of virtual interview experiences and supplemental application impressions

BACKGROUND

We sought to evaluate the unique benefits and challenges the virtual recruitment and interviewing platform had on general surgery residency applicants.

METHODS

Applicants who interviewed for a categorical position at our institution during the 2021 and 2022 Match season were contacted to participate in the anonymous online survey focused on applicant behavior related to the virtual interview format. Data were analyzed using chi-square and paired t-tests.

RESULTS

A response rate of 56.7 ​% (n ​= ​135) was achieved. Applicants accepted a median of 17 (IQR 13-20) interviews in 2021 and 15 (IQR 11-19) interviews in 2022. More than half (54 ​%) of applicants indicated they applied to more programs, and 53 ​% accepted more interviews, because of the virtual format. The greatest advantages of the virtual interviews as cited by applicants were saving money (96.3 ​%), saving time (49.6 ​%), and avoiding travel risks (43.7 ​%). The top limitations of virtual interviews were less exposure to current residents and faculty (61.5 ​%), to the city or location of the program (58.5 ​%), and difficultly comparing programs (57.8 ​%). The 2022 Match cycle included use of the supplemental application; however, 85 ​% of applicants did not feel that the supplemental improved their overall application. Some applicants (20 ​%) who "signaled" programs did not receive an interview offer from any of the programs they signaled.

CONCLUSION

The transition to virtual interviews saved applicants time and money but limited their exposure. Future efforts to maintain virtual interviews will need to be balanced against the intangible benefit of human interaction and observing a program's culture.

Dissecting Diversity: A Comprehensive Look at the Present Landscape and Future Challenges in Surgical Specialties

BACKGROUND

As the US demographic evolves, surgical fields must adapt to ensure equitable healthcare. Healthcare disparities notably affect minority populations, with communities of color often facing physician shortages and higher rates of diseases such as coronary disease, stroke, and cancer. Research shows that minority physicians significantly improve patient satisfaction and outcomes in underserved communities, highlighting the need for increased physician diversity to enhance cultural competency and patient centered care. Data from the Association of American Medical Colleges (AAMC) reveals minimal increases in underrepresented minorities (URM) in surgical residency and academic careers over the past thirty-six years, with little change URM applicants and matriculants in the nine surgical specialties recognized by the American College of Surgeons from 2010 to 2018.

OBJECTIVE

This review aims to critically evaluate the current landscape of racial and gender diversity in six out of the nine defined surgical specialties (general surgery, plastic surgery, neurosurgery, orthopedic surgery, cardiothoracic surgery, and vascular surgery) in the US.

DESIGN

We conducted a comprehensive literature review to assess of the state of diversity within surgical specialties in the United States. By analyzing the benefits of diversity in surgical fields, evaluating the effectiveness of various diversity programs and initiatives, examining the comparative diversity between surgical subspecialties, and assessing the impact of diversity on patient outcomes, our aim is to highlight the critical importance of enhancing diversity in surgical fields.

RESULTS

While nuances in representation and diversity vary across surgical specialties, all fields persistently exhibit underrepresentation of certain racial/ethnic groups and persistent gender disparities. These disparities manifest throughout various phases, including in residency, and in the recruitment and retention of URM individuals in surgery and surgical subspecialties. While interventions over the past decade have contributed to improving diversity in surgical fields, significant disparities persist. Limitations include the time required for recent interventions to show significant impacts and the inability of established interventions to eliminate disparities.

CONCLUSIONS

Despite the clear benefits, diversity within surgical specialties remains an uphill battle. Addressing the diversity gap in surgical fields is crucial for improving patient outcomes, healthcare access, and workplace environments, requiring strategies such as targeted recruitment, mentorship programs, and addressing systemic biases. This review highlights the undeniable imperative for change and serve a call to action.

Early Lessons From Implementation of Holistic Review for Pediatric Surgery Fellowship Applicants

PURPOSE

Holistic review (HR) considers applicants' unique identities and experiences rather than focusing on academic metrics. Though several residency programs have demonstrated increases in women and those underrepresented in medicine (URiM), this is the first study to examine HR in pediatric surgery (PS).

METHODS

Using a retrospective review of applicants, demographic, academic, and non-academic metrics of traditional review (TR) [2015-2017] were compared to HR [2018-2022]. HR initiatives include expansion of faculty reviewers, implementation of a pre-screening rubric, and greater prioritization of non-academic factors. Chi-squared/Fisher's exact tests, Wilcoxon rank-sum tests, and two sample z-test for proportions were used where appropriate.

RESULTS

For 635 applicants (TR: 268, HR: 367), the proportion offered interviews in the TR and HR cohorts were similar (31.7 vs 36%, p = 0.30). Candidates selected for interview pre- and post-HR most commonly graduated from residency programs affiliated with PS fellowships (56.5 vs 50%, p = 0.65). After HR implementation, no change in proportion of women interviewees (TR: 52.9 vs HR: 54.5%, p = 0.93) was observed. Though URiM residents applying to PS remained consistently low (TR: 14.6 vs HR: 10.9%, p = 0.21), significantly more received interviews with HR (30.8 vs 42.5%, p = 0.001). The median number of peer-review publications per interviewee increased (17 vs 22, p = 0.02) as did non-academic achievements (leadership, service, athletic awards, etc.) per applicant (1.0 vs 1.5, p = 0.104), though the latter did not reach significance, demonstrating similar qualification of interviewees in HR and TR.

CONCLUSION

Holistic review of PS fellowship applications increased the proportion of URiM interviewees, despite a persistently low URiM proportion in the applicant pool. Furthermore, implementing HR did not sacrifice the caliber of interviewees, as publications and non-academic achievements increased by over 25% in the HR cohort.

LEVEL OF EVIDENCE

IV.

How Did Black and Hispanic Orthopaedic Applicants and Residents Compare to General Surgery Between 2015 and 2022?

BACKGROUND

Despite the heavy demand for and knowledge of the benefits of diversity, there is a persistent lack of racial, ethnic, and gender diversity in orthopaedic surgery. Since the implementation of diversity initiatives, data have shown that general surgery has been one of the top competitive surgical fields and has demonstrated growth in racial, ethnic, and gender diversity, making general surgery a good point of reference and comparison when analyzing racial and ethnic growth in orthopaedic surgery.

QUESTIONS/PURPOSES

(1) What were the growth rates for Black and Hispanic orthopaedic residency applicants and residents between 2015 and 2022? (2) How did the growth rates of Black and Hispanic individuals in orthopaedic surgery compare with those of general surgery? (3) How did applicant recruitment and resident acceptance differ between Black and Hispanic people in orthopaedic surgery?

METHODS

Applicant data were obtained from historical specialty-specific data from the Association of American Medical Colleges Electronic Residency Application Service Statistics database between 2018 and 2022, and resident data were obtained from the Accreditation Council of Graduate Medical Education Data Resource Book between 2015 and 2021. Between 2018 and 2022, the number of residency applicants totaled 216,677, with 17,912 Black residency applicants and 20,413 Hispanic residency applicants. Between 2015 and 2021, the number of active residents totaled 977,877, with 48,600 Black residents and 62,605 Hispanic residents. Because the applicant and resident data do not overlap throughout all years of observation, a sensitivity analysis of overlapping years (between 2018 and 2021) was conducted to ensure observed trends were consistent and valid throughout the study. All datasets obtained were used to establish the different racial and ethnic proportions of Black and Hispanic residency applicants and residents in four nonsurgical primary care specialties and four surgical subspecialties. A reference slope was created using data from the Association of American Medical Colleges and Accreditation Council of Graduate Medical Education to represent the growth rate for total residency applicants and residents, independently, across all residency specialties reported in each database. This slope was used for comparison among the resident and applicant growth rates for all eight selected specialties. Datapoints were placed into a scatterplot with regression lines, using slope equations to depict rate of growth and R2 values to depict linear fit. Applicant growth corresponded to applicant recruitment and resident growth corresponded to resident acceptance. Chi-square tests were used to compare residents and residency applicants for the Black and Hispanic populations, separately. Two-way analysis of variance with a time-by-specialty interaction term (F-test) was conducted to determine differences between growth slopes.

RESULTS

There was no difference in the growth rate of Black orthopaedic surgery applicants between 2018 and 2022, and there was no difference in the growth rate of Hispanic orthopaedic surgery applicants (R2 = 0.43; p = 0.23 and R2 = 0.63; p = 0.11, respectively). However, there was a very slight increase in the growth rate of Black orthopaedic surgery residents between 2015 and 2021, and a very slight increase in the growth rate of Hispanic orthopaedic surgery residents (R2 = 0.73; p = 0.02 and R2 = 0.79; p = 0.01, respectively). There were no differences in orthopaedic and general surgery rates of growth for Black applicants between 2018 and 2022 (0.004 applicants/year versus -0.001 applicants/year; p = 0.22), and no differences were found in orthopaedic and general surgery rates of growth for Black residents between 2015 and 2021 (0.003 residents/year versus 0.002 residents/year; p = 0.59). Likewise, Hispanic orthopaedic applicant growth rates did not differ between 2018 and 2022 from the rates of general surgery (0.004 applicants/year versus 0.005 applicants/year; p = 0.68), and there were no differences in orthopaedic and general surgery rates of growth for Hispanic residents (0.007 residents/year versus 0.01 residents/year; p = 0.35). Furthermore, growth rate comparisons between Black orthopaedic applicants and residents between 2018 and 2021 showed applicant growth was larger than resident growth, illustrating that the recruitment of Black applicants increased slightly more rapidly than resident acceptance. Growth rate comparisons between Hispanic applicants and residents showed a larger rate of resident growth, illustrating Hispanic resident acceptance increased slightly faster than applicant recruitment during that time.

CONCLUSION

We found low acceptance of Black residents compared with the higher recruitment of Black applicants, as well as overall low proportions of Black and Hispanic applicants and residents. Future studies might explore the factors contributing to the higher acceptances of Hispanic orthopaedic residents than Black orthopaedic residents.

CLINICAL RELEVANCE

We recommend that more emphasis should be placed on increasing Black and Hispanic representation at the department level to ensure cultural considerations remain at the forefront of applicant recruitment. Internal or external reviews of residency selection processes should be considered, and more immersive, longitudinal orthopaedic surgery clerkships and research mentorship experiences should be targeted toward Black and Hispanic students. Holistic reviews of applications and selection processes should be implemented to produce an increased racially and ethnically diverse applicant pool and a diverse residency work force, and implicit bias training should be implemented to address potential biases and diversity barriers that are present in residency programs and leadership.

Are We Speaking the Same "Language" Regarding Underrepresented Groups in Plastic Surgery and Increasing Diversity Within Our Field?

BACKGROUND

Plastic Surgery is one of the fields that lags behind the rest when it comes to surgeons from backgrounds underrepresented in medicine (URiM). Extensive research has shown that diversity in health care not only fosters inclusivity but also saves lives. The study aim is to quantify how many integrated plastic surgery residency programs have outlined criteria defining diversity goals and/or groups of people they consider to be URiM.

METHODS

All American Council for Graduate Medical Education-accredited integrated plastic surgery program Web sites were reviewed for diversity missions/statements and explicit mentions of the racial and ethnic groups. Web sites were deemed "up-to-date" if they were last updated within 6 months before the initial data collection period. The data collection period was from November 20 to 29, 2022.

RESULTS

A total of 86 program were reviewed. Only 8 programs (9%) had clear URiM criteria listed on their Web sites, whereas 26 (30%) relied on institution/department-wide criteria, 1 (1%) listed that they were adhering to American Association of Medical Colleges definition of URiM, and 51 programs (60%) had no form of definition for what is considered URiM. When looking at the programs that have some form of criteria for URiM (n = 35 [40%]), all programs (100%) considered African American/Black, Native American/Alaskan Native, Hispanic/Latinx, and Pacific Islander/Native Hawaiian as groups URiM. Assessing the same subset of programs that have a form of criteria listed (n = 35 [40%]), 19 (58%) had listed other groups outside of race/ethnicity considered to be URiM for their program, and 14 (42%) programs did not. Fourteen programs (74%) considered LGBTQIA+ as a URiM group.

CONCLUSION AND SIGNIFICANCE

There still is a great deal of heterogeneity among residency programs when it comes to identifying which medical students are URiM. Numerous plastic surgery organizations have placed diversity and inclusive excellence at the forefront of their agendas; however, it is critical that residency programs also actively align their efforts in an equitable and intentional way. This study serves to encourage residency programs to evaluate their mission toward diversity, equity, and inclusion and to spark discussion toward creating a clearer URiM definition to be consistent among all programs.

Impact of Mentoring for Underrepresented Groups in Undergraduate Medical Education: A Systematic Review

OBJECTIVES

Despite efforts to mitigate challenges to advance underrepresented groups (URG) groups' representation in medical education, diversity remains underwhelming. In response to this several mentoring programs to increase diversification within medical education have been implemented. However, the impact of these programs on URG representation across disciplines is unknown. The aim of this review is to evaluate the impact of structured mentoring programs on URGs in undergraduate medical education.

DESIGN

Systematic review.

METHODS

A comprehensive search strategy was performed of electronic databases including PubMed, Ovid Medline, and EMBASE between January and September 2022 in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Both qualitative and quantitative outcomes including characteristics of the mentoring structure explored within each study were collected, and differences in study outcomes analyzed.

RESULTS

In total 17 studies were included for analysis. Mentoring within URGs resulted in an increase in research opportunities, exam performance, medical specialty applications, and residency matching. Structured feedback yielded positive experiences by both mentors and mentees participating in programs, with both emotional and cultural competence issues explored across studies included for analysis.

CONCLUSIONS

Mentoring for medical students underrepresented in medicine across mentoring relationships including peer, senior, formal, and informal structures yield positive outcomes within research, academic modules, and career pathways. Future applications of mentoring programs should consider the use of tandem mentoring from both senior and peer mentors to optimize benefits URG students derive from each mentoring relationship.

A Learning Assessment to Increase Diversity in Academic Health Sciences

Importance

Strategies and innovations to advance racial and ethnic equity in recruitment, promotion, and retention at academic health science institutions are needed.

Objective

This learning assessment aims to isolate evidence-based strategies to advance racial equity in the academic health sciences, which have implications for policy and institution-level interventions.

Evidence Review

This learning assessment used a mixed-methods approach, including a quantitative survey, qualitative in-depth interviews, and a scoping literature review. Survey respondents were recruited from outreach lists that included researchers working with racial and ethnic minoritized populations. In-depth interviews were conducted among 60 university administrators, faculty/staff, scholars, students, and individuals affiliated with governmental, nongovernmental, and identity-based professional associations. A search of the literature in PsycINFO, MEDLINE, ERIC, Education Source, Academic Search Ultimate, and CINAHL was conducted for the scoping review. The scoping review included 366 primary articles of studies evaluating strategies to advance racial and ethnic equity at academic health science institutions.

Findings

The survey yielded analyzable results from 328 individuals, including faculty, students, administrators, or staff, and individuals not currently employed at or enrolled full time at a university or college. The interviews included 60 participants with a mean (SD) age of 49.3 (16.5) years, and 39 (65%) were female. The scoping review included 366 primary research articles that met inclusion criteria for analysis. Data were analyzed individually across the survey, interviews, and scoping review, and findings were triangulated. While each of the 3 assessments yielded unique findings, 13 common themes emerged across all project components. Results revealed strategies implemented and evaluated successfully, as well as challenges and barriers to advancing equity in the academic health sciences.

Conclusions and Relevance

In this study, 13 meaningful strategies emerged across the survey, in-depth interviews, and scoping review. Through triangulation of findings, recommendations of actionable steps were made.

An analysis of osteopathic medical students applying to surgical residencies following transition to a single graduate medical education accreditation system

CONTEXT

Upon requests from osteopathic medical schools, the National Resident Matching Program (NRMP) Charting Outcomes were redesigned to include osteopathic medical school seniors beginning in 2018 and one joint graduate medical education (GME) accreditation system, the Accreditation Council for Graduate Medical Education (ACGME), formed in 2020.

OBJECTIVES

The goal of this study is to analyze the match outcomes and characteristics of osteopathic applicants applying to surgical specialties following the ACGME transition.

METHODS

A retrospective analysis of osteopathic senior match outcomes in surgical specialties from the NRMP Main Residency Match data from 2020 to 2022 and the NRMP Charting Outcomes data from 2020 to 2022 was performed.

RESULTS

For surgical specialties, results show matching increased as United States Medical Licensing Examination (USMLE) Step 2 CK (clinical knowledge) and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2 CE (cognitive evaluation) scores increased along with the number of contiguous rankings (p<0.001). The greatest indication for matching looking at scores alone were those who scored greater than 230 on Step 2 CK compared to below (p<0.001) and above 650 on Level 2 CE (p<0.001). However, those who scored 240 (p=0.025) on Step 2 CK were just as likely to match as those who scored 250 (p=0.022) when compared to those who scored below those scores. Increasing research involvement had little to no significance with the likelihood of matching across most surgical subspecialties.

CONCLUSIONS

Our study demonstrates that there are unique thresholds for Step 2 CK scores, Level 2 CE scores, and the number of contiguous ranks for each surgical specialty that, when reached, are significantly associated with match success. Although certain board score delineations are linked with higher match success rates, the rates level off after this point for most surgical specialties and do not significantly increase further with higher scores. In addition, thresholds within contiguous ranks for increasing match likelihood exist and vary across surgical specialties. Overall, this study highlights that the quantitative metrics utilized to assess applicants lack the correlation reported historically, and the data presently available need to be more substantiated.

Improving Graduate Medical Education by Aggregating Data Across the Medical Education Continuum

ABSTRACT

Meaningful improvements to graduate medical education (GME) have been achieved in recent decades, yet many GME improvement pilots have been small trials without rigorous outcome measures and with limited generalizability. Thus, lack of access to large-scale data is a key barrier to generating empiric evidence to improve GME. In this article, the authors examine the potential of a national GME data infrastructure to improve GME, review the output of 2 national workshops on this topic, and propose a path toward achieving this goal.The authors envision a future where medical education is shaped by evidence from rigorous research powered by comprehensive, multi-institutional data. To achieve this goal, premedical education, undergraduate medical education, GME, and practicing physician data must be collected using a common data dictionary and standards and longitudinally linked using unique individual identifiers. The envisioned data infrastructure could provide a foundation for evidence-based decisions across all aspects of GME and help optimize the education of individual residents.Two workshops hosted by the National Academies of Sciences, Engineering, and Medicine Board on Health Care Services explored the prospect of better using GME data to improve education and its outcomes. There was broad consensus about the potential value of a longitudinal data infrastructure to improve GME. Significant obstacles were also noted.Suggested next steps outlined by the authors include producing a more complete inventory of data already being collected and managed by key medical education leadership organizations, pursuing a grass-roots data sharing pilot among GME-sponsoring institutions, and formulating the technical and governance frameworks needed to aggregate data across organizations.The power and potential of big data is evident across many disciplines, and the authors believe that harnessing the power of big data in GME is the best next step toward advancing evidence-based physician education.

Socioeconomic Disparities in Research Participation: Bias in Plastic Surgery Residency Match

Background

Integrated plastic surgery residency applicants have increased at a rate disproportionate to available positions. Research productivity has become a surrogate marker for competitiveness, and many applicants pursue it to distinguish themselves. To date, no study has investigated socioeconomic disparities in extended research experience (ERE) participation.

Methods

A 35-question cross-sectional survey was distributed to applicants to United States-based integrated plastic surgery residency programs during the 2019-2022 application cycles. Summary tables, student t test, and chi-square tests were used for statistical analysis.

Results

A total of 161 responses (response rate: 20.9%) were recorded. Fifty-nine (40.7%) respondents participated in an ERE. The most common reason for ERE participation was strengthening one's application. The most common reason against participation was avoiding delays in career progression. A greater percentage of respondents from Northeastern medical schools participated in EREs (P = 0.019). There were no significant differences in debt burden between those who did or did not participate in an ERE. A greater percentage of applicants whose parents had advanced degrees participated in EREs (P = 0.053).

Conclusions

There may be geographic and socioeconomic biases present in access to ERE for students interested in plastic surgery. The growing popularity of EREs may have unintended consequences for applicant diversity. As most plastic surgeons ultimately practice in nonacademic settings, applicants and plastic surgeons may consider the financial hardships and possible socioeconomic disparities in research opportunities before participating in or recommending them.

Implementing Holistic Review Practices in a Pulmonary and Critical Care Fellowship

Background

Medical schools have used holistic review in admissions to increase mission-aligned enrollment of students from backgrounds underrepresented in medicine. Graduate medical education programs have increasingly followed suit. However, there is a paucity of literature regarding holistic review at the fellowship level.

Objective

Here, we share our experience implementing the Association of American Medical Colleges core principles of holistic review during the 2021 recruitment cycle.

Methods

We used a partially asynchronous and online learning strategy to train division members on the principles of holistic review. Following the match, we conducted a survey of faculty members and fellows to understand their opinions on our holistic review training and implementation.

Results

Although few of our colleagues clearly understood holistic review before the training, they were able to identify broad-based criteria that aligned with our division's mission and balanced applicants' experiences, attributes, competencies, and metrics. These were viewed as better selection criteria than traditional measures and were incorporated into the individualized consideration of applicants. Our survey had a 41.5% response rate, with 10 of 22 fellows and 24 of 60 faculty members responding. Most faculty members and fellows agreed that holistic review decreases socioeconomic disparities in fellowship recruitment (79.2% and 80.0%, respectively) and promotes inclusion and diversity (83.3% and 90.0%, respectively). Faculty members appeared more confident than fellows that our training efforts had influenced recruitment. All respondents agreed that it would be critical for such training to be repeated yearly.

Conclusion

Although this was a single-institution experience, implementing holistic review was feasible and well received by faculty and fellows.

A Practical Guide to Implementing Holistic Review during Surgery Resident Selection

Provider workforce diversity is a key component of improving healthcare quality and addressing healthcare disparities. Furthermore, the traditional approach of "score-centered" application metrics do not consistently correlate with meeting milestones in surgery, nor do they adequately predict a surgical resident's clinical strength and operative abilities. We present here an adaptable process by which surgical residency programs can identify their values and incorporate holistic review into their resident selection process to improve resident selection and physician workforce diversity.

Patient Preferences and Perceptions of Provider Diversity in Orthopaedic Surgery

BACKGROUND

Orthopaedic surgery in the U.S. historically has been among the least demographically diverse specialties in medicine. Currently, limited data exist on how patients perceive diversity within the field and what patients look for when choosing an orthopaedic surgeon. The purpose of this study was to identify specific patient preferences for surgeon demographics and understand patient perceptions of racial and gender diversity in orthopaedic surgery.

METHODS

Nonconsecutive patients from orthopaedic clinics affiliated with a U.S. academic health system voluntarily completed a 39-item questionnaire that surveyed basic demographic information, perception of diversity, racial and gender preferences during surgeon selection, and perception of health-care inequalities. Bivariate analyses were used to test the association between patient-surgeon demographic variables and ratings of diversity. Multiple regression models were used to identify independent predictors of overall perceived diversity ratings.

RESULTS

A total of 349 patients (80.6% White, 17.9% Black, and 1.5% other) were analyzed. Black patients were more likely to experience difficulty relating to their surgeon than White patients (11.48% versus 2.29%; odds ratio [OR], 5.62; 95% confidence interval [CI], 1.55 to 21.1; p = 0.004). Moreover, Black patients were more likely to perceive racial bias from their surgeon than White patients (5.17% versus 0.37%; OR, 14.44; 95% CI, 1.14 to 766.29; p = 0.02). While the level of racial diversity perceived by White patients (2.57 of 10) was significantly higher than that perceived by Black patients (2.10 of 10) (p = 0.001), the absolute difference between these 2 figures was small, suggesting that both groups perceived racial diversity in orthopaedics to be low. White and Black patients differed in their importance ranking of a surgeon's race (p < 0.0001): Black patients ranked a surgeon's race with higher importance (mean, 3.49 of 10) when selecting a surgeon compared with White patients (1.45 of 10). Both male and female patients gave relatively low importance rankings for a surgeon's gender (mean, 1.58 of 10 and 2.15 of 10, respectively, p = 0.02).

CONCLUSIONS

Patients in this study did not perceive orthopaedic surgery as a diverse field (overall diversity rating, <3 of 10). There were significant racial and gender differences in patients' preferences for specific physician characteristics when choosing an orthopaedic surgeon, which may help explain some instances of perceived racial bias and difficulty relating to their orthopaedic surgeon.

Using Natural Language Processing and Machine Learning to Identify Internal Medicine-Pediatrics Residency Values in Applications

PROBLEM

Although holistic review has been used successfully in some residency programs to decrease bias, such review is time-consuming and unsustainable for many programs without initial prescreening. The unstructured qualitative data in residency applications, including notable experiences, letters of recommendation, personal statement, and medical student performance evaluations, require extensive time, resources, and metrics to evaluate; therefore, previous applicant screening relied heavily on quantitative metrics, which can be socioeconomically and racially biased.

APPROACH

Using residency applications to the University of Utah internal medicine-pediatrics program from 2015 to 2019, the authors extracted relevant snippets of text from the narrative sections of applications. Expert reviewers annotated these snippets into specific values (academic strength; intellectual curiosity; compassion; communication; work ethic; teamwork; leadership; self-awareness; diversity, equity, and inclusion; professionalism; and adaptability) previously identified as associated with resident success. The authors prospectively applied a machine learning model (MLM) to snippets from applications from 2023, and output was compared with a manual holistic review performed without knowledge of MLM results.

OUTCOMES

Overall, the MLM had a sensitivity of 0.64, specificity of 0.97, positive predictive value of 0.62, negative predictive value of 0.97, and F1 score of 0.63. The mean (SD) total number of annotations per application was significantly correlated with invited for interview status (invited: 208.6 [59.1]; not invited: 145.2 [57.2]; P < .001). In addition, 8 of the 10 individual values were significantly predictive of an applicant's invited for interview status.

NEXT STEPS

The authors created an MLM that can identify several values important for resident success in internal medicine-pediatrics programs with moderate sensitivity and high specificity. The authors will continue to refine the MLM by increasing the number of annotations, exploring parameter tuning and feature engineering options, and identifying which application sections have the highest correlation with invited for interview status.

Our Efforts to Create a Welcoming Surgical Residency for LGBTQ Trainees

OBJECTIVE

To provide a more welcoming environment for LGBTQ residents.

DESIGN

This paper is based upon the authors experience creating a training program welcoming to LGBTQ residents.

SETTING

University General Surgery Training Program.

PARTICIPANTS

Surgical Residents and Surgical Faculty.

RESULTS

Over a 9 year period, our residency has made intentional efforts to welcome LGBTQ applicants and support LGBTQ residents. We now have a residency in which over 25% of our residents identify as LGBTQ.

CONCLUSIONS

We see opportunities for other residencies to signal to applicants and trainees that they are welcome and celebrated for who they are.

Unintended Consequence: Diversity as a Casualty of Eliminating United States Medical Licensing Examination Step 1 Scores

PURPOSE

The purpose of this study was to use a discrete-choice experiment to model the trade-offs evaluators make between academic attributes and demographics when the United States Medical Licensing Examination (USMLE) Step 1 switches to pass/fail.

METHODS

A discrete-choice experiment was administered to faculty members from a geographically diverse mix of 14 academic and community radiology departments in the United States from August through November 2020. Reviewers reviewed 10 applicant pairs with numeric Step 1 scores (part 1) and 10 applicant pairs with a pass Step 1 result (part 2). Applicant attributes included medical school rank, gender, race/ethnicity, USMLE Step 1 score, USMLE Step 2 score, class rank, clerkship honors, and publications. Conditional logistic regression modeled the influence of attribute levels.

RESULTS

Two hundred twelve evaluators completed the study (response rate 59%). The most influential attribute was Step 1 score in part 1 and medical school rank in part 2. The relative importance of race/ethnicity and gender decreased by 25% and 29%, respectively, when Step 1 switches to pass/fail. Evaluators weigh race/ethnicity the strongest when applicants have the same Step 1 score (preference weights of 0.85 for African American, 1.42 for Hispanic, and 0 for White and Asian applicants). Race/ethnicity is relatively more important when Step 1 scores are higher (preference weights of 1.58 for African American, 0.90 for Hispanic, and 0 for White and Asian applicants).

CONCLUSIONS

The loss of numeric Step 1 scores reduced the residency evaluator preference for diversity. Reviewers prioritize underrepresented-in-medicine applicants when Step 1 scores are higher and comparable with White and Asian applicants.

Implementation of a Holistic Review Process of US Allopathic Medical Students Eliminates Non-Comparable Metrics and Bias in General Surgery Residency Interview Invitations

OBJECTIVE

Use of traditional scoring metrics for residency recruitment creates racial and gender bias. In addition, widespread use of pass/fail grading has led to noncomparable data. To adjust to these challenges, we developed a holistic review (HR) rubric for scoring residency applicants for interview selection.

DESIGN

Single-center observational study comparing the proportion of underrepresented in medicine (URM) students and their United States Medical Licensing Exam (USMLE) scores who were invited for interview before (2015-2020) and after (2022) implementation of a holistic review process.

SETTING

General surgery residency program at a tertiary academic center.

PARTICIPANTS

US allopathic medical students applying for general surgery residency.

RESULTS

After initial screening, a total of 1514 allopathic applicants were narrowed down to 586 (38.7%) for HR. A total of 52% were female and 17% identified as URM. Based on HR score, 20% (118/586) of applicants were invited for an interview. The median HR score was 11 (range 4-19). There was a fourfold higher coefficient of variation of HR scores (22.3; 95% CI 21.0-23.7) compared to USMLE scores (5.1; 95% Cl 4.8-5.3), resulting in greater spread and distinction among applicants. There were no significant differences in HR scores between genders (p = 0.60) or URM vs non-URM (p = 0.08). There were no significant differences in Step 1 (p = 0.60) and 2CK (p = 0.30) scores between those who were invited to interview or not. On multivariable analysis, USMLE scores (OR 1.01; 95% CI 0.98-1.03), URM status (OR 1.71 95% CI 0.98-2.92), and gender (OR 0.94, 95% CI 0.60-1.45) did not predict interview selection (all p > 0.05). There was a meaningful increase in the percentage of URM interviewed after HR implementation (12.9% vs 23.1%, p = 0.016).

CONCLUSION

The holistic review process is feasible and eliminates the use of noncomparable metrics for surgical applicant interview invitations and increases the percentage of URM applicants invited to interview.

Impact of holistic review on diversity of interviewed and matriculating residents in graduate medical education: a systematic review protocol

INTRODUCTION

Diversity in the physician workforce improves patient-centred outcomes. Patients are more likely to trust in and comply with care when seeing gender/racially concordant providers. A current emphasis on standardised metrics in academic achievement often serves as a barrier to the recruitment and retention of gender and racial minorities in medicine. Holistic review of residency applicants has been supported as a means of encouraging diversification but is not yet standardised. The current body of evidence examining the effects of holistic review on the recruitment of racial and gender minorities in surgical residencies is small. We therefore propose a systematic review to summarise the state of holistic review in graduate medical education in the USA and its impact on diversification.

METHODS AND ANALYSIS

Our systematic review protocol has been designed with plans to report our review findings in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines. PubMed and Embase will be searched with the assistance of a health sciences librarian with expertise in systematic review. We will include studies of graduate medical education programmes that describe the implementation of holistic review, outline the components of their holistic review process and compare proportions of under-represented minorities (URM) and women interviewed and matriculating before and after holistic review implementation. We will first report a summary of the findings regarding the operationalisation of holistic review as described by studies included. We will then pool the percentages of URM and women for interviewee and matriculant populations from each study and report the collective odds ratios of each for holistic review compared with traditional review as our primary outcome.

ETHICS AND DISSEMINATION

This study is a protocol for systematic review, and therefore does not involve any human subjects. Findings will be published in the form of a manuscript submitted to a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42023401389.

Racial and Ethnic Bias in Medical School Clinical Grading: A Review

OBJECTIVE

Improving diversity in healthcare is a widely recognized national goal. The diversity of medical student matriculants has increased, yet this trend is not seen in the composition of competitive residency programs. In this review, we examine racial and ethnic disparities in medical student grading during clinical years and explore the consequences of how this may exclude minority students from accessing competitive residency positions.

DESIGN

Following PRISMA guidelines, we searched PubMed, Embase, Scopus, and ERIC databases using variations of the terms "race," "ethnicity," "clerkship," "rotation," "grade," "evaluation", or "shelf exam." Of 391 references found using the criteria, 29 were related to clinical grading and race/ethnicity and included in the review. The GRADE criteria were used to determine the quality of evidence.

SETTING

Johns Hopkins School of Medicine, Baltimore MD.

RESULTS

Five studies examining a total of 107,687 students from up to 113 different schools found racial minority students receive significantly fewer Honors grades in core clerkships compared to White students. Three studies examining 94,814 medical student evaluations from up to 130 different schools found significant disparities in the wording of written clerkship evaluations based on race and/or ethnicity.

CONCLUSIONS

A large body of evidence suggests the presence of racial bias in subjective clinical grading and written clerkship evaluations of medical students. Grading disparities can disadvantage minority students when applying to competitive residency programs and may contribute to a lack of diversity in these fields. As low minority representation has a negative impact on patient care and research advancement, strategies to resolve this issue must be further explored.

The Use of Facial Recognition Software and Published Manuscripts to Examine Trends in Surgical Editorial Board Diversity

INTRODUCTION

Recent social justice movements have highlighted the need for improved diversity and inclusion. These movements have emphasized the need for inclusivity of all genders and races in all sectors including surgical editorial boards. There is currently not an established, standardized method to assess the gender, racial, and ethnic makeup of surgical editorial board rosters, yet artificial intelligence is a method that can be utilized to determine gender and race in an unbiased manner. The aim of the present study is to determine if recent social justice movements correlate with an increase in diversity-themed articles published and if there is an increase in the gender and racial makeup of surgical editorial boards determined by artificial intelligence software.

METHODS

Impact factor was used to assess and rank highly regarded general surgery journals. The website of each of these journals was examined for pledges of diversity in their mission statements and core beliefs of conduct. To determine the number of diversity-themed articles that were published during 2016 and 2021, each surgical journal was analyzed for diversity-themed articles using 10 specific keywords in PubMed. To determine the racial and gender makeup of editorial boards in 2016 and 2021, we obtained the current and the 2016 editorial board roster. Roster member images were collected from academic institutional websites. Betaface facial recognition software was used to assess the images. The software assigned the gender, race and ethnicity of the image supplied. Betaface results were analyzed using a Chi Square Test of Independence.

RESULTS

We analyzed 17 surgical journals. Only four of 17 journals were found to have diversity pledges on their website. For diversity themed publications, 1% of articles in 2016 and 2.7% in 2021 were published specifically about diversity. There was a significant increase in the amount of diversity articles/journal published per year in 2016 (6.59) compared to 2021 (25.94, P < 0.001). There was no correlation between impact factor and articles that publish diversity keywords. 1968 editorial board member images were analyzed using Betaface software to determine gender and race in both time periods. There was no significant increase in diversity of editorial board members regarding gender, race, and ethnicity temporally from 2016 to 2021.

CONCLUSIONS

In the present study, we found that although the number of diversity-themed articles has increased over the last 5 y, however the gender and racial makeup of surgical editorial boards has not changed. Further initiatives are needed to better track and diversify the gender and racial composition of surgical editorial boards.

Values-Based Resident Selection in an Internal Medicine-Pediatrics Residency Program

BACKGROUND

There is no consensus regarding values important for medical resident success, and current methods for selecting residents correlate poorly with success in residency.

OBJECTIVE

We developed and validated a set of values demonstrated by exemplary residents in the Internal Medicine-Pediatrics program at the University of Utah and used them to inform our resident selection process.

DESIGN

We utilized a modified Delphi method to identify and internally validate values of successful residents. We implemented these values into the interview evaluation rubric.

PARTICIPANTS

Four members of the Internal Medicine-Pediatrics residency program leadership and eleven current residents aided in value generation. Nine faculty from leadership positions in the residency programs of Internal Medicine-Pediatrics, Internal Medicine, and Pediatrics formed a local expert panel for validation.

APPROACH

We performed a literature review and engaged local stakeholders in a semi-structured group interview to generate 107 values. After consolidation based on redundancy, two iterative cycles of expert review using a modified Delphi approach, and alignment with the Accreditation Council for Graduate Medical Education core competencies, eleven values achieved expert agreement and were integrated into an interview rubric to aid in resident selection.

KEY RESULTS

We identified eleven values important for resident success: academic strength, intellectual curiosity, compassion, communication, work ethic, teamwork, leadership, self-awareness, DEI (diversity, equity, and inclusion), professionalism, and adaptability. The rank list from 2021 was found to correlate with a score based on values, but not Step 2 score, as it did in 2017.

CONCLUSIONS

We applied a modified Delphi method to generate eleven observable values present in the ideal Internal Medicine-Pediatric resident at one academic health center in the Intermountain West. Higher Step 2 scores no longer correlated with higher ranking when we used these values to inform our rank list.

The Impact of Socioeconomic Factors on the 2022 Plastic Surgery Match

BACKGROUND

In 2022, the plastic and reconstructive surgery (PRS) match faced unprecedented system-wide transitions that have redefined conventional measures of applicant success. This challenges the equitable assessment of student competitiveness and diversity in the field.

METHODS

A survey of demography, application content, and 2022 match outcomes was distributed to applicants to a single PRS residency program. Comparative statistics and regression models were performed to assess the predictive value of factors in match success and quality.

RESULTS

A total of 151 respondents (response rate 49.7%) were analyzed. Although step 1 and step 2 CK scores were significantly higher among matched applicants, neither examination predicted match success. Most respondents (52.3%) were women, although gender was also not significantly associated with match success. Underrepresented in medicine applicants made up 19.2% of responses and 16.7% of matches, and the plurality of respondents (22.5%) were raised with a household income ≥$300,000. Both Black race and household income ≤$100,000 were associated with lower odds of scoring above a 240 on either step 1 or step 2 CK (Black: OR, 0.03 and 0.06; P < 0.05 and P < 0.001; income: OR, 0.07-0.47 and 0.1 to 0.8, among income subgroups), receiving interview offers (OR, -9.4; P < 0.05; OR, -11.0 to -5.4), and matching into PRS (OR, 0.2; P < 0.05; OR, 0.2 to 0.5), compared with White and high-income applicants, respectively.

CONCLUSIONS

Systemic inequities in the match process disadvantage underrepresented in medicine candidates and those from lower household incomes. As the residency match continues to evolve, programs must understand and mitigate the impacts of bias in various application components.

Diversity among endovascular neurointerventionalists in Canada results of a national survey 2022

Aim

Neurointervention (NIR) is a relatively new developing filed of medicine. Diversity and inclusion in various medical fields has made a significant progress. However, many surgical and interventional fields are still lagging in this respect. The aim of this study was to evaluate the degree of diversity and inclusion amongst neurointerventionalists in Canada.

Materials and methods

A survey was completed in June 2022 by each neurointerventional division in Canada. The survey included questions regarding demographics, inclusivity, diversity, social and personal parameters. The collected data was analysed using semi-quantitative analysis.

Results

As of 2022, 85 physicians were actively practicing NIR in Canada. 52% were neuroradiologists, 38% neurosurgeons and 9% neurologists. 41% were immigrants to Canada (from 19 countries), for 35% English or French were not first language, 35% were visible minority. Women comprised only 21% of the practitioners, with comparable proportion of women in leadership positions. Most practitioners were in the 30–49 age group. 2.4% practitioners identified as LGBTQ. There was no gender difference in terms of life to work balance, with majority of practitioners being engaged in long term relationships and having children.

Conclusions

Our study shows encouraging results in terms of diversity and inclusion amongst Canadian neurointerventionalists regarding the representation of various specialty backgrounds, immigrants, and visible minorities. NIR centers are distributed according to population density and better coverage is needed in smaller communities and remote/isolated areas. Both women and men Canadian neurointerventionalists seem to have a favourable life-work balance. Gaps still exist regarding inclusion of first nations and women which are under-represented among Canadian Neurointerventionalists. Women however are proportionally serving in leadership positions.

Interventions Associated With Racial and Ethnic Diversity in US Graduate Medical Education: A Scoping Review

IMPORTANCE

Racially and ethnically minoritized individuals remain underrepresented in graduate medical education relative to their proportion in the population. While many programs and initiatives have been developed to address this problem, there is little consensus regarding strategies that work to improve representation across specialties.

OBJECTIVE

To examine and synthesize evidence-based practices that have been used to increase the proportions of underrepresented in medicine (URiM) trainees at US residency and fellowship programs.

EVIDENCE REVIEW

The authors searched PubMed, Google Scholar, Embase, PsycInfo, ERIC, Cochrane Reviews, Cochrane Trials, CINAHL, Scopus, and PROSPERO electronic databases to identify relevant studies published through January 2022. They screened all titles and abstracts for relevance and read full-text articles to identify articles reporting reliable data describing the outcomes of interventions to improve racial and ethnic diversity among trainees.

FINDINGS

Twenty-seven articles were included in this review. Two studies reported on fellowship programs. The most common interventions included holistic review (48%), decreased emphasis on United States Medical Licensing Examination Step 1 scores (48%), and explicit institutional messaging regarding the importance of diversity (37%). A combination of interventions was associated with an increased number of URiM applicants, interviewees, and matriculants across various medical and surgical specialties.

CONCLUSIONS AND RELEVANCE

In this scoping review, approaches and interventions associated with increased diversity in residency and fellowship programs were identified. Continued efforts are necessary to sustain such efforts and assess long-term outcomes.

Analysis of Diversity, Equity, and Inclusion Initiatives Presented Across US Academic Department of Surgery Websites in 2021

Despite increased attention devoted to diversity, equity, and inclusion (DEI) within academic medicine, representation, lack of workforce and leadership diversity, and bias within medicine remain persistent problems. The purpose of the current study was to understand the current efforts and attention to DEI within academic departments of surgery in the United States. 251 department of surgery websites were reviewed, using a standardized data collection form and scoring procedure, accompanied by a 10 percent fidelity check by an independent reviewer. Only 16% of departments of surgery included DEI-specific information, such as a DEI mission statement or initiatives on their departmental sites, with less than seven percent of departments reporting a DEI committee. Such public information may have implications for recruitment and retention of diverse faculty and trainees, downstream effects for patient care, and could be critical to public accountability to improve diversity and create a culture of equity and inclusion.

The Impact of Holistic Review of Urology Residency Applications on Selection for Interview During the COVID-19 Pandemic

OBJECTIVE

To investigate the impact of a holistic review of urology residency applications on interview selection at our institution during the COVID-19 pandemic.

METHODS

In the 2019-2020 cycle, applicants were filtered by a Step 1 score of 230 and whether they applied from selected east coast medical schools. For the 2020-2021 and 2021-2022 cycles, we implemented a scoring system which focused on desirable attributes based on our program training needs and resources. We compared applicant and interviewee demographics and United States Medical Licensing Examination (USMLE) scores using descriptive statistics and 1-way analysis of variance tests.

RESULTS

A total of 282, 300, and 367 students applied to our residency program with 50, 45, and 52 selected for interviews during the 2019-2020, 2020-2021, and 2021-2022 cycles, respectively. Compared to 2019-2020, the 2020-2021 and 2021-2022 interviewee cohorts comprised of more non-tri-state applicants (36%, 55.6%, and 46.2%, respectively). Underrepresented minority representation increased for the 2020-2021 interviewee cohort; however, this was not observed in 2021-2022 (16%, 24.4%, 15.4%, respectively). Additionally, USMLE Step 1 and 2 scores were similar between interviewee cohorts in 2019-2020, 2020-2021 and 2021-2022, respectively (Step 1: 244.2 ± 8.8, 242 ± 12.1, 242.8 ± 12.4, P = .624) (Step 2: 249.1 ± 11.5, 251.5 ± 10.5, 254.4 ± 10.8, P = .143).

CONCLUSION

Utilizing a comprehensive review resulted in a geographically diverse interview pool and no significant difference in academic performance among interviewees. Holistic review provides an alternative, balanced evaluation of residency applicants which may increase diversity in urology.

A Path Toward Equity and Inclusion: Establishing a DEI Committee in a Department of Pediatrics

The Diversity, Equity, and Inclusion (DEI) committee was established in 2017 within the Department of Pediatrics at Rush University Medical Center (RUMC), an academic medical health center located on the near west side of Chicago, IL. Results from climate surveys highlighted the need for increased DEI initiatives within the department, and a renewed national reckoning on racial tensions sparked an additional sense of urgency for system-level change. This paper outlines the initial creation and ongoing efforts of the DEI committee. Information related to the structure of our committee, aims of our work, progress toward identified goals, as well as ongoing barriers is provided. Academic medical health centers are tasked not only with working and training together, but also to care for a diverse group of patients within a larger community. As such, academic medical health centers represent a unique backdrop and opportunity for individual and system-level change.

Designing the "match of the future": challenges and proposed solutions in the interview and match phase of the UME-GME transition

This review focuses on the interview and match process with the purpose of broadly reviewing challenges in the current surgical residency selection process, detailing potential solutions, and identifying future avenues of investigation.

Deconstructing the roadmap to surgical residency: a national survey of residents illuminates factors associated with recruitment success as well as applicants' needs and beliefs

Purpose

As applications increase and residency becomes more competitive, applicants and programs will be challenged by increased demands on recruitment, metric assessment, and rank determination. Studies have investigated program opinions; however, this survey sought to illuminate the process from an applicant's perspective.

Methods

An anonymous survey was distributed to past or current surgery residents nationwide using social media and program director emails. Regression analyses were performed to assess factors correlating with percentage of programs which offered the applicant an interview.

Results

There were 223 respondents who applied to an average of 61 programs (± 40) with 16 (± 11) interviews offered. Applicants believed that programs were most interested in (1) personality, (2) letter of recommendation (LOR) writers, and (3) medical school reputation. Top factors considered by applicants in ranking were resident culture, location, program reputation, and autonomy. Bivariate analysis found factors that decreased percent of interview invites to be Asian race, whereas factors that increased interview invites included age, year of match, surgery clerkship grade, medicine clerkship grade, AOA status, honor surgery rotation, gold humanism (GHHS) status, phone call for interview made, and step scores (all p < 0.05). AOA status, step scores, honor surgery rotation, year of match, and Asian race remained significant after multivariate analysis.

Conclusions

National surveys illuminate how applicants approach the application process and what programs and applicants appear to value. This information provides insight and guidance to candidates and programs as the process of matching becomes more challenging with surging application numbers, changes in testing parameters and virtual interviews.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44186-022-00070-9.

National Institutes of Health diversity supplements: Perspectives from administrative insiders

BACKGROUND

The NIH Diversity Administrative Supplement is a funding mechanism that provides support for diverse early-stage researchers. There is limited guidance on how to apply for these awards.

PURPOSE

We describe perspectives of NIH program/diversity officers and university research administrators offering recommendations for diversity supplement submission.

METHODS

This article is the product of a working group exploring diversity in research. Nursing faculty from an R2 Historically Black College and University and an R1 research intensive university conducted stakeholder interviews with NIH program/diversity officers and university research administrators. We used content analysis to categorize respondents' recommendations.

FINDINGS

Recommendations centered on harmonizing the applicant with the program announcement, communication with program/diversity officers, mentor/mentee relationship, scientific plan, and systematic institutional approaches to the diversity supplement.

DISCUSSION

Successful strategies in submitting diversity supplements will facilitate inclusion of diverse researchers in NIH-sponsored programs. Systematic approaches are needed to support development of diverse voices to enhance the scientific community.

Creating a Welcoming Surgery Residency Training Program for LGBTQ Trainees

General Surgery residencies have been perceived as unwelcoming to lesbian, gay, bisexual, transgender, and queer (LGBTQ) applicants. Historically, applicants have been reluctant to reveal their LGBTQ status when interviewing for residency positions and LGBTQ surgery residents are more likely to consider leaving their residency. Despite the increased acceptance in society and calls for diversity by governing bodies, there is a perception that the field of surgery is lagging behind. In this article, we summarize the recent literature concerning LGBTQ status in surgery residencies and the impact it has on students and trainees. We then review the recent calls to action to remedy the identified shortcomings. We then share what has been done by our surgery residency, which has been successful in recruiting LGBTQ-identifying residents and summarize recommendation for moving forward.

Identifying US Plastic Surgery Training Programs that Effectively Establish Gender and Ethnically Diverse Faculty

Background:

Successful strategies to improve the representation of female and ethnically underrepresented in medicine (UIM) physicians among US plastic and reconstructive surgery (PRS) faculty have not been adequately explored. Accordingly, we aimed to identify programs that have had success, and in parallel gather PRS program directors’ and chiefs/chairs’ perspectives on diversity recruitment intentionality and strategies.

Methods:

We conducted a cross-sectional analysis of the demographic composition of female and UIM faculty of PRS residency training programs. Separate lists of programs in the top quartile for female and UIM faculty representation were collated. Additionally, a 14-question survey was administered to program directors and chiefs/chairs of all 99 Accreditation Council for Graduate Medical Education-accredited PRS residency programs. The questions comprised three domains: (1) demographic information; (2) perceptions about diversity; and (3) recruitment strategies utilized to diversify faculty.

Results:

Female and UIM faculty representation ranged from 0% to 63% and 0% to 50%, respectively. Survey responses were received from program directors and chiefs/chairs of 55 institutions (55% response rate). Twenty-five (43%) respondents felt their program was diverse. Fifty-one (80%) respondents felt diversity was important to the composition of PRS faculty. Active recruitment of diverse faculty and the implementation of a diversity, equity, and inclusion committee were among the most frequently cited strategies to establish a culturally sensitive and inclusive environment.

Conclusions:

These findings reveal that female and UIM representation among US PRS faculty remains insufficient; however, some programs have had success through deliberate and intentional implementation of diversity, equity, and inclusion strategies.

Assessing barriers to the career ladder and professional development for ethnic minority genetic counselors in the United States

Ethnic diversity is not reflected within healthcare professions, including genetic counseling, where lack of growth and membership among minority colleagues extends to upper-level and executive roles. While diversity and inclusion-based topics have been emphasized, studies on potential barriers to career advancement in the field of genetic counseling have not received the same attention. Our study examined the current state of mentorship and sponsorship programs, the presence of diversity and inclusion initiatives, and opportunities for career advancement through the lens of a minority genetic counselor. Practicing genetic counselors in the United States identifying as part of any racial group, other than non-Hispanic White alone, were recruited through the Minority Genetics Professionals Network for survey participation. A 31-item survey was fully completed by 19 practicing genetic counselors from a variety of ethnic backgrounds. Data were analyzed using descriptive statistics and thematic analysis, allowing for individual stories and accounts to be amplified. Results showed 16 of 19 participants had never been promoted in their current employment setting. Additionally, 7 out of 19 respondents disagreed or strongly disagreed that their company had a commitment to an ethnically diverse workforce within upper-level positions. Prominent themes identified from open-ended responses included lack of social connection with supervisors and the cross-race effect, a term referencing a tendency for individuals to better recognize members of their own race or ethnicity than others. Additional themes revealed feelings of isolation, need for support from White colleagues, as well as desired emphasis on sponsorship tailored toward professional growth. These findings demonstrate a need for proactive involvement in reaching ethnic and racial minority genetic counselors through companywide policy efforts, support and advocacy from White colleagues, and modification of cultural perception frameworks. Further focus and emphasis on these distinct but critical topics may be important in promoting increased diversity in upper-level positions in the field of genetic counseling.

Reshaping Residency Recruitment: Achieving Alignment Between Applicants and Programs in Surgery

OBJECTIVE

The residency recruitment process has become increasingly challenging for both applicants and program directors, in part, due to the inflation in the number of applications per student. As a result, it has become more daunting for programs to design processes that evaluate applicants holistically. Furthermore, the existing methods used to evaluate and select applicants do not necessarily predict success in residency and may inadvertently lend to gender, racial, and ethnic bias. This narrative review aims to identify innovative tools used in residency recruitment that will allow programs and applicants to better determine concordance of interests and achieve value alignment while supporting improved, objective evaluation of an applicant's unique attributes and experiences.

DESIGN

PubMed was used to conduct a narrative review of recruitment strategies in admission processes of undergraduate and graduate medical education between 1975 and June 2021, using the designated Medical Subject Heading (MeSH0 terms. Inclusion criteria were established surrounding innovative tools to better objectively screen, evaluate, or select applicants. Strategies relying primarily on traditional metrics (United States Medical Licensing Examination (USMLE) scores, Alpha Omega Alpha status, and clerkship grades) were excluded.

RESULTS

Forty-two articles met specific inclusion criteria. Using these articles, a framework was created with two specific aims: (1) to allow applicants and programs to express or assess interest and (2) to foster objective review of unique applicant attributes, skills, experiences, and competencies that align with program mission and values. The following five innovative tools for recruitment were identified: preference signaling, secondary applications, standardized letters of recommendation, situational judgment testing, and surgical simulation.

CONCLUSIONS

As the number of applications continues to rise, strategies must be implemented to allow applicants and institutions to achieve better alignment or "fit," while also giving balanced consideration to all of an applicant's unique characteristics. A more holistic approach to applicant selection is a necessary tool in order to increase diversity and inclusion within the field of surgery.

Strategies to Increase Racial and Ethnic Diversity in the Surgical Workforce: A State of the Art Review

OBJECTIVE

To evaluate strategies to increase racial and ethnic diversity in the surgical workforce among trainees and faculty across surgical specialties.

DATA SOURCES

Embase, OVID/Medline, and Web of Science Core Collection.

REVIEW METHODS

A review of US-based, peer-reviewed articles examining the effect of targeted strategies on racial and ethnic diversity in the surgical workforce was conducted from 2000 to 2020 with the PRISMA checklist and STROBE tool. Studies without an outlined strategy and associated outcomes were excluded. Eleven studies met inclusion criteria and were completed in general surgery, orthopaedic surgery, and otolaryngology-head and neck surgery.

CONCLUSIONS

Efforts to increase exposure to surgery through internship programs and required clerkships with efforts to improve mentorship were common (6 of 11 [54.5%] and 3 of 11 [27.3%] studies, respectively). Three (27.3%) studies aimed to diversify the recruitment and selection process for the residency match and faculty hiring, and 2 (18.2%) aimed to increase representation among trainees, faculty, and leadership through holistic review processes paired with departmental commitment. Outcome metrics included surgical residency applications for individuals underrepresented in medicine, interview and match rates, faculty hiring, measures of a successful academic surgical career, and leadership representation. All strategies were successful in increasing diversity in the surgical workforce.

IMPLICATIONS FOR PRACTICE

A convincing yet limited body of literature exists to describe strategies and outcomes that address racial and ethnic diversity in the surgical workforce. While future inquiry is needed to move this field of interest forward, the evidence presented provides a framework for surgical residency programs/departments to develop approaches to increase racial and ethnic diversity.

Diversity, Equity, and Inclusion: A Current Analysis of General Surgery Residency Programs

BACKGROUND

Local, regional, and national diversity, equity, and inclusion (DEI) initiatives have been established to combat barriers to entry and promote retention in surgery residency programs. Our study evaluates changes in diversity in general surgery residency programs. We hypothesize that diversity trends have remained stable nationally and regionally.

MATERIALS AND METHODS

General surgery residents in all postgraduate years were queried regarding their self-reported sex, race, and ethnicity following the 2020 ABSITE. Residents were then grouped into geographic regions. Data were analyzed utilizing descriptive statistics, Kruskal-Wallis test, and chi-square analyses.

RESULTS

A total of 9276 residents responded. Nationally, increases in female residents were noted from 38.0 to 46.0% (P < .001) and in Hispanic or Latinx residents from 7.3 to 8.3% (P = .031). Across geographic regions, a significant increase in female residents was noted in the Northwest (51.9 to 58.3%, P = .039), Midwest (36.9 to 43.3%, P = .006), and Southwest (35.8 to 47.5%, P = .027). A significant increase in black residents was only noted in the Northwest (0 to 15.8%, P = .031). The proportion of white residents decreased nationally by 8.9% and in the Mid-Atlantic, Southeast, and Southwest between 5.5 and 15.9% (P < .05).

DISCUSSION

In an increasingly diverse society, expanding the numbers of underrepresented surgeons in training, and ultimately in practice, is a necessity. This study shows that there are region-specific increases in diversity, despite minimal change on a national level. This finding may suggest the need for region-specific DEI strategies and initiatives. Future studies will seek to evaluate individual programs with DEI plans and determine if there is a correlation to changing demographics.

New Heuristics to Stratify Applicants: Predictors of General Surgery Residency Applicant Step 1 Scores

OBJECTIVE

In 2022, United States Medical Licensing Examination (USMLE) Step 1 scores will become pass/fail. This may be problematic, as residency programs heavily rely on USMLE Step 1 scores as a metric when determining interview invitations. This study aimed to assess candidate application metrics associated with USMLE Step 1 scores to offer programs new cues for stratifying applicants.

DESIGN

Retrospective cohort study analyzing interviewed applicants to one general surgery residency program in 2019 and 2020. Applicant data analyzed included USMLE Step 1 scores, number of publications, clerkship scores, letter of recommendation scores (out of 2, scored by 0.25 interval), interview overall score (out of 5, scored by integer level), and standardized question score (out of 10). Each year, applicant's answers to one standardized behavioral question during their interview were scored by interviewers.

SETTING

Tertiary medical center, academic general surgery residency program.

PARTICIPANTS

Interviewed applicants at one general surgery residency program whose applications contained complete demographic information (203 out of 247).

RESULTS

Multiple Linear Regression revealed that higher surgical clerkship (β = 0.19, p = 0.006) and higher standardized interview question (β = 0.32, p < 0.001) scores were positively associated with applicant USMLE Step 1 score (F[7, 195] = 6.61, p < 0.001, R2 = 0.19). Letter of recommendation score, number of peer reviewed publications, gender, race, and applicant type (preliminary/categorical) were not associated with USMLE Step 1 scores.

CONCLUSIONS

With USMLE Step 1 scores transitioning to pass/fail, surgical residency programs need new selection heuristics. Surgery clerkship scores and standardized behavioral questions answered by applicants prior to the interview could provide a holistic view of applicants and help programs better stratify candidates without USMLE Step 1 scores.

Mission-Based Filters in the Electronic Residency Application Service: Saving Time and Promoting Diversity

BACKGROUND

Holistic review promotes diversity, but widespread implementation remains limited.

OBJECTIVE

We aimed to develop a practical approach to incorporate holistic review principles in screening applicants in the Electronic Residency Application Service (ERAS) and to assess the impact on diversity.

METHODS

Three residency programs (internal medicine [IM], pediatrics, and surgery) at McGovern Medical School developed filters to identify applicants with experiences/attributes aligned with the institutional mission. These filters were retroactively applied to each program's 2019-2020 applicant pool using built-in ERAS capabilities to group applicants by user-defined features. We compared the demographics of applicants reviewed during the cycle with those identified retrospectively through experiences/attributes filters.

RESULTS

The IM, pediatrics, and surgery programs received 3527, 1341, and 1313 applications, respectively, in 2019-2020. Retrospective use of experiences/attributes filters, without scores, narrowed the IM applicant pool for review to 1301 compared to 1323 applicants reviewed during actual recruitment, while the pediatrics filters identified 514 applicants compared to 384 at baseline. The surgery filters resulted in 582 applicants, but data were missing for baseline comparison. Compared to the baseline screening approach utilizing scores, mission-based filters increased the proportions of underrepresented in medicine applicants selected for review in IM (54.8% [95% CI 52.1-57.5] vs 22.7% [20.4-24.9], P < .0001) and pediatrics (63.2% [95% CI 59.1-67.4] vs 25.3% [20.9-29.6], P < .0001).

CONCLUSIONS

Program directors can leverage existing ERAS features to conduct application screening in alignment with holistic review principles. Widespread implementation could have important repercussions for enhancing physician workforce diversity.

What is the Trend in Representation of Women and Under-represented Minorities in Orthopaedic Surgery Residency?

BACKGROUND

Orthopaedic surgery training programs have lagged behind other surgical specialties in increasing their representation of women and people from under-represented minority (URM) groups. Comparative data between orthopaedic surgery and other specialties are needed to help identify solutions to closing the diversity gap.

QUESTIONS/PURPOSES

(1) Which surgical specialties have the greatest representation of women residents and residents from URM groups? (2) How have the proportions of women residents and residents from URM groups changed across the surgical specialties during the past decade?

METHODS

This was a retrospective evaluation of a large, longitudinally maintained survey database. Resident data by gender and ethnicity were retrieved from the Accreditation Council for Graduate Medical Education Data Resource Books for the 2011 to 2012 through 2019 to 2020 academic years. The Accreditation Council for Graduate Medical Education database is updated annually; thus, it is the most up-to-date and complete database available for gender and ethnicity data for all surgical residents. Data were obtained and analyzed for seven different surgical specialties: orthopaedic surgery, neurosurgery, ophthalmology, otolaryngology, plastic surgery, general surgery, and urology. No sampling was necessary, and thus descriptive statistics of the data were completed. Because the entire population of residents was included for the period of time in question, no statistical comparisons were made, and the reported differences represent absolute differences between the groups for these periods. Linear regression analyses were performed to estimate the annual growth rates of women residents and residents from URM groups in each specialty.

RESULTS

Among the seven surgical specialties, representation of women residents increased from 28% (4640 of 16,854) of residents in 2012 to 33% (6879 of 20,788) in 2020. Orthopaedic surgery had the lowest representation of women residents every year, with women residents comprising 16% of residents (700 of 4342) in 2020. Among the seven surgical specialties, representation of residents from URM groups increased from 8.1% (1362 of 16,854) in 2012 to 9.7% (2013 of 20,788) in 2020. In 2020, the representation of residents from URM groups in orthopaedic surgery was 7.7% (333 of 4342). In 2020, general surgery had the highest representation of women residents (42%; 3696 of 8809) as well as residents from URM groups (12%; 1065 of 8809). Plastic surgery (1.46% per year) and general surgery (0.95% per year) had larger annual growth rates of women residents than the other specialties did. In each surgical specialty, the annual growth rate of residents from URM groups was insignificant.

CONCLUSION

During the past decade, there was only a small increase in the representation of women in orthopaedic surgery, while the representation of people from URM groups did not change. In contrast, by 2020, general surgery had become the most diverse among the seven surgical specialties. To increase diversity in our field, we need to evaluate and implement some of the effective interventions that have helped general surgery become the diverse surgical specialty that it is today.

CLINICAL RELEVANCE

General surgery has substantially reduced gender and ethnic disparities that existed in the past, while those in orthopaedic surgery still persist. General surgery residencies have implemented a holistic review of resident applications and longitudinal mentoring programs to successfully address these disparities. Orthopaedic surgery programs should consider placing less emphasis on United States Medical Licensing Examination score thresholds and more weight on applicants' non-academic attributes, and put more efforts into targeted longitudinal mentorship programs, some of which should be led by non-minority faculty.

Ethnic/Racial Bias in Medical School Performance Evaluation of General Surgery Residency Applicants

OBJECTIVE

Differential use of communal terms (caring/unselfish traits) versus agentic terms (goal-oriented/leadership/achievement traits) may reveal bias and has been extensively reported in letters of recommendation for residency. We evaluated bias in medical student performance evaluations (MSPE) of general surgery residency applicants.

DESIGN

This is a retrospective study evaluating ethnic/race bias, as measured by differential use of agentic and communal terms, in MSPEs of residency applicants. 50% of MSPEs were randomly selected. An ethnic bias calculator derived from an open-source online gender bias calculator was populated with a list of validated agentic and communal terms. Relative frequency of communal and agentic terms was used to estimate bias. Multivariable regression was used to assess the association between the terms and ethnicity/race.

PARTICIPANTS

US medical students applying for a categorical surgery residency position at a single academic institution for a single Match cycle.

RESULTS

A total of 339 MSPEs were reviewed from 119 US medical schools. Genders were equally represented (women, 51.6%); most participants were white and Asian applicants (79.1%). Overall, MSPEs were more agency biased (65.2%) than communal biased (16.2%) or neutral (18.6%). MSPEs for Black and Hispanic/Latinx applicants were more likely to contain communal rather than agentic terms (adjusted OR: 3.02, 95% CI: 1.52-6.02) when compared to white and Asian applicants. This finding was independent of MSPE writer's gender or rank.

CONCLUSIONS

Surgery residency applicants self-identifying as Black and Hispanic/Latinx were more likely to be described using communal traits compared to white and Asian applicants, suggesting ethnic/racial bias. Such differences in language utilized in MSPEs may impact residency opportunities for applicants who are under-represented in medicine. Educational efforts aimed at MSPE writers may help to reduce bias.

Engage me: Will residency program directors listen?

•

The implementation of multi-level modifications, including but not limited to outreach initiatives and the development of a holistic application review approach, are potential strategies to improve diversity and inclusivity in general surgery residency recruitment.

•

We encourage program directors to invest in trainees' success and implement human capital changes that produce sustinable and effective changes, and help in building the pipeline.

•

Program directors are encouraged to work with their institutions to develop a holistic review approach through the following initiatives: recruitment of an inclusive and diverse interview selection committee, implementation of implicit bias training among all faculty and residents participating in the selection process, and promotion of structured interviews that focus on applicant characteristics that resonate with the program's mission statement.

Selecting the Next Generation of Surgeons: General Surgery Program Directors and Coordinators Perspective on USMLE Changes and Holistic Approach

Introduction

The United States Medical Licensing Examination (USMLE) was designed as a universal assessment tool for states to determine physician’s medical licensure's candidacy. Recent changes in the USMLE exam have changed the way future surgical residency candidate applications will be reviewed. The survey aimed to assess the effect of changes in USMLE exams—USMLE Step 1 pass/fail, complete dissolution of USMLE clinical skills exam, and the role of holistic review in future surgical residency candidacy selection.

Methods

An anonymous online survey was created and distributed to general surgery program directors and coordinators across the USA. The survey aimed to assess attitudes toward changes to USMLE exams and the potential changes with a holistic review of candidate applications.

Results

The response rate was 63.7%. Most program directors and coordinators disagree with changing USMLE Step 1 to a pass/fail scoring system. The majority felt that contacts, the medical school's name, and performance in clinical electives and sub-internships would hold more significance. They also believe that a holistic review of application will decrease socioeconomic discrepancies and promote a more diverse and inclusive resident cohort.

Conclusion

Step 2 clinical knowledge (CK) will gain more importance in future residency matches because of the change in the scoring system of Step 1. The medical school's name, personal contacts, and clinical performance in rotations will hold more significance.