Gender Bias in Surgical Oncology Fellowship Recommendation Letters: Gaining Progress

Assessment of Gender Differences in Letters of Recommendation for Physical Therapy Residency Applications

INTRODUCTION

Letters of recommendation (LOR) are an integral component of physical therapy residency applications. Identifying the influence of applicant and writer gender in LOR will help identify whether potential implicit gender bias exists in physical therapy residency application processes.

REVIEW OF LITERATURE

Several medical and surgical residency education programs have reported positive, neutral, or negative LOR female gender bias among applicants and writers. Little research exists on gender differences in LOR to physical therapy education programs or physical therapy residency programs.

SUBJECTS

Seven hundred sixty-eight LOR were analyzed from 256 applications to 3 physical therapy residency programs (neurologic, orthopaedic, sports) at one institution from 2014 to 2020.

METHODS

Thematic categories were developed to identify themes in a sample of LOR. Associations between writer and applicant gender were analyzed using summary statistics, word counts, thematic and psycholinguistic extraction, and rule-based and deep learning Natural Language Processing .

RESULTS

No significant difference in LOR word counts were found based on writer or applicant gender. Increased word counts were seen in sports residency LOR compared with the orthopaedic residency. Thematic analysis showed LOR gender differences with male applicants receiving more positive generalized recommendations and female applicants receiving more comments regarding interpersonal relationship skills. No thematic or psycholinguistic gender differences were seen by LOR writer. Male applicants were 1.9 times more likely to select all male LOR writers, whereas female applicants were 2.1 times more likely to choose all female LOR writers.

DISCUSSION AND CONCLUSION

Gender differences in LORs for physical therapy residencies were found using a comprehensive Natural Language Processing approach that identified both a positive recommendation male applicant gender bias and a positive interpersonal relationship skill female applicant gender bias. Applicants were not harmed nor helped by selecting LOR writers of the opposite gender. Admissions committees and LOR writers should be mindful of potential implicit gender biases in LOR submitted to physical therapy residency programs.

Interventional Pulmonary Fellowship Training: End of the Beginning

Interventional pulmonology (IP) fellowship training has undergone increased popularity and growth. The Accreditation Council of Graduate Medical Education recently recognized IP medicine as a new subspecialty, which leads to new challenges and opportunities for a young subspecialty. Although the specialty-specific requirements are in progress, IP fellowship programs must plan ahead for the known common program requirements and anticipated accreditation process. The educational leadership in IP must identify and execute solutions to sustain continued excellence. This includes transitioning to a new regulatory environment with issues of funding new fellowships, keeping up to date with training/assessment of new procedures, and shaping the future through recruitment of talent to lead the young subspecialty.

Checking Our Blind Spots: Examining Characteristics of Interviewees Versus Matriculants to a Hepatopancreatobiliary Surgical Fellowship Program

OBJECTIVE

Racial and gender biases exist within academic surgery; bias negatively impacts patient care, reimbursement, student training, and staff retention. Few studies have investigated the potential for bias in surgical fellowship recruitment. We aimed to compare the racial and gender diversity at our hepatopancreatobiliary (HPB) surgery fellowship program to nationwide standards. We further aimed to investigate differences in the demographics of resident interviewees versus matriculants to our HPB fellowship.

DESIGN

Retrospective review.

SETTING

North American HPB fellowship training programs.

PARTICIPANTS

Mayo Clinic's HPB surgery fellowship interviewees and North American HPB surgery fellowship graduates from 2013 to 2020.

RESULTS

When compared to general surgery residency graduates during the study period (in 2019), a lower proportion of North American HPB surgery fellowship graduates were female (26% HPB fellowship graduates vs. 43.1% residents, p = 0.005), with no difference in proportion of racially under-represented in medicine (rURM) HPB fellowship graduates (10.7%) compared to rURM proportion of general surgery residents nationally (14.5%). There was an upward trend in female representation among North American HPB fellowship graduates from 11% in 2013 to 32% in 2020, but proportions of rURM HPB fellows remained steadily low. When comparing HPB interviewees at our institution to national general surgery residents, no differences were observed in proportions of female (34.4% interviewees vs. 43.1% residents, p = 0.17) or rURM (interviewees = 6.8%, residents = 14.5%, p = 0.09) applicants. Additionally, there was no significant difference between the proportion of female or rURM interviewees and matriculants to our HPB program.

CONCLUSIONS

While fewer female graduating surgeons are pursuing HPB fellowship training than male graduates, this gender gap has narrowed over time. In contrast, the national percentage of rURM HPB fellowship graduates has remained low, mirroring stagnant proportions of rURM surgical residency graduates. When comparing HPB fellowship interviewees at our own institution to North American fellowship graduates, we observed similar proportions of female interviewees but lower proportions of rURM interviewees. Locally, these data will drive process change toward more intentional examination of our interview selection process. Nationally, more work is needed to increase the racial diversity of surgical residency and fellowship trainees to best reflect and serve our diverse patient populations.

Gender differences in cardiothoracic surgery letters of recommendation

OBJECTIVE

To investigate whether or not gender influences letters of recommendation for cardiothoracic surgery fellowship.

METHODS

From applications to an Accreditation Council Graduate Medical Education cardiothoracic surgery fellowship program between 2016 and 2021, applicant and author characteristics were examined with descriptive statistics, analysis of variance, and Pearson χ2 tests. Linguistic software was used to assess communication differences in letters of recommendation, stratified by author and applicant gender. An additional higher-level analysis was then performed using a generalized estimating equations model to examine linguistic differences among author-applicant gender pairs.

RESULTS

Seven hundred thirty-nine recommendation letters extracted from 196 individual applications were analyzed; 90% (n = 665) of authors were men and 55.8% (n = 412) of authors were cardiothoracic surgeons. Compared with women authors, authors who are men wrote more authentic (P = .01) and informal (P = .03) recommendation letters. When writing for women applicants, authors who are men were more likely to display their own leadership and status (P = .03) and discuss women applicants' social affiliations (P = .01), like occupation of applicant's father or husband. Women authors wrote longer letters (P = .03) and discussed applicants' work (P = .01) more often than authors who are men. They also mentioned leisure activities (P = .03) more often when writing for women applicants.

CONCLUSIONS

Our work identifies gender-specific differences in letters of recommendation. Women applicants may be disadvantaged because their recommendation letters are significantly more likely to focus on their social ties, leisure activities, and the status of the letter writer. Author and reviewer awareness of gender-biased use of language will aid in improvements to the candidate selection process.

Male applicants are more likely to be awarded fellowships than female applicants: A case study of a Japanese national funding agency

Scientific grant applications are subjected to scholarly peer review. Studies show that the success rates of grant applications are often higher for male than for female applicants, suggesting that gender bias is common in peer review. However, these findings mostly come from studies in Europe, North America and Australia. Here we report the analyses of gender-specific success rates of applications to the fellowships offered by Japan Society for the Promotion of Science (JSPS). Because we analyze the observational data (i.e., not experimental), our aim here is to describe the possible gender gaps in the success rates, rather than the examination of gender bias per se. Results show that the success rates are consistently higher for male applicants than for female applicants among five different fellowship categories. The gender gaps in the success rates varied significantly between research fields in some Fellowship categories. Furthermore, in some fellowship categories, the gender gaps were significantly associated with the representation of female applicants (both positive and negative correlations were found). Though the causes of the gender gaps are unknown, unintentional gender bias during the review process is suggested. Pre-application gender gaps may also be contributing to the gender gaps in success rates. At least some of the observed gender gaps were relatively small, which may be partly explicable by the designs of the review process. However, gender gaps or biases acting prior to the application, such as self-selection bias, may have reduced the superficial gender gaps in the success rates. Further investigations that control for the effects of covariates (e.g., scientific merits of each applicant, which were not accessible to us) and those of other funding agencies, especially of non-Western countries, are warranted.

Gender and culture bias in letters of recommendation for computer science and data science masters programs

Letters of Recommendation (LORs) are widely utilized for admission to both undergraduate and graduate programs, and are becoming even more important with the decreasing role that standardized tests play in the admissions process. However, LORs are highly subjective and thus can inject recommender bias into the process, leading to an inequitable evaluation of the candidates' competitiveness and competence. Our study utilizes natural language processing methods and manually determined ratings to investigate gender and cultural differences and biases in LORs written for STEM Master's program applicants. We generate features to measure important characteristics of the LORs and then compare these characteristics across groups based on recommender gender, applicant gender, and applicant country of origin. One set of features, which measure the underlying sentiment, tone, and emotions associated with each LOR, is automatically generated using IBM Watson's Natural Language Understanding (NLU) service. The second set of features is measured manually by our research team and quantifies the relevance, specificity, and positivity of each LOR. We identify and discuss features that exhibit statistically significant differences across gender and culture study groups. Our analysis is based on approximately 4000 applications for the MS in Data Science and MS in Computer Science programs at Fordham University. To our knowledge, no similar study has been performed on these graduate programs.

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

The Presence of Implicit Gender Bias in Colon and Rectal Surgery Residency Letters of Recommendation

BACKGROUND

Selection for colorectal surgery residency relies on letters of recommendation for assessment of candidates' strengths and weaknesses. It is unclear whether this process incorporates implicit gender bias.

OBJECTIVE

This study aimed to assess the presence of gender bias in letters of recommendation for colorectal surgery residency.

DESIGN

Mixed methods assessment of the characteristics described within the blinded letters of the 2019 application cycle to a single academic residency.

SETTINGS

Academic medical center.

PATIENTS

Blinded letters from the 2019 colorectal surgery residency application cycle.

INTERVENTIONS

Characteristics of the letters were qualitatively and quantitatively analyzed.

MAIN OUTCOME MEASURES

Association of gender with the presence of descriptors within the letters.

RESULTS

A total of 111 applicants, 409 letter writers, and 658 letters were analyzed. Forty-three percent of applicants were female. Female and male applicants had an equal mean number of positive (5.4 vs 5.8; p = 0.10) and negative (0.5 vs 0.4; p = 0.07) attributes represented. Female applicants were more likely to be described as having poor academic skills (6.0 vs 3.4%; p = 0.04) and possessing negative leadership qualities (5.2% vs 1.4%; p < 0.01) than male applicants. Male applicants were more likely to be described as kind (36.6% vs 28.3%; p = 0.03), curious (16.4% vs 9.2%; p = 0.01), possessing positive academic skills (33.7% vs 20.0%; p < 0.01), and possessing positive teaching skills (23.5% vs 17.0%; p = 0.04).

LIMITATIONS

This study analyzed a single year of applications to an academic center and may not be generalizable.

CONCLUSIONS

There are differences in the qualities used to describe female versus male applicants in colorectal surgery residency application letters of recommendation. Female applicants were more often described in negative academic terms and possessing negative leadership qualities. Males were more likely to be described as kind, curious, academically impressive, and possessing good teaching skills. The field may benefit from educational initiatives to reduce implicit gender bias in letters of recommendation. See Video Abstract at http://links.lww.com/DCR/C191 .

LA PRESENCIA DE SESGO DE GNERO IMPLCITO EN LAS CARTAS DE RECOMENDACIN DE RESIDENCIA EN CIRUGA DE COLON Y RECTO

ANTECEDENTES:La selección para la residencia en cirugía colorrectal se basa en cartas de recomendación para la evaluación subjetiva de las fortalezas y debilidades de los candidatos. No está claro si este proceso incorpora un sesgo de género implícito.OBJETIVO:Evaluar la presencia de sesgo de género en las cartas de recomendación para la residencia en cirugía colorrectal.DISEÑO:Evaluación de métodos mixtos de las características descritas dentro de las cartas selladas del ciclo de solicitud de 2019 a una sola residencia académica.ENTORNO CLÍNICO:Centro médico académico.PACIENTES:Cartas selladas del ciclo de solicitud de residencia en cirugía colorrectal de 2019.INTERVENCIONES:Las características de las cartas se determinaron utilizando medidas cualitativas y cuantitativas.PRINCIPALES MEDIDAS DE VALORACIÓN:Asociación del género con la presencia de descriptores dentro de las cartas.RESULTADOS:Hubo 111 solicitantes, 409 escritores de cartas y se analizaron 658 cartas. El 43% de los solicitantes eran mujeres. Los solicitantes masculinos y femeninos tenían el mismo promedio de atributos positivos (5,4 frente a 5,8; p = 0,10) y negativos (0,5 frente a 0,4; p = 0,07) representados. Las solicitantes femeninas tenían más probabilidades de ser descritas como con deficientes habilidades académicas (6,0 frente a 3,4%, p = 0,04) y poseían cualidades de liderazgo negativas (5,2% frente a 1,4%; p < 0,01) en comparacion con los solicitantes masculinos. Los solicitantes masculinos tenían más probabilidades de ser descritos como amables (36,6 % frente a 28,3%; p = 0,03), curiosos (16,4% frente a 9,2%; p = 0,01), que poseían habilidades académicas positivas (33,7 % frente a 20,0%; p < 0,01), y habilidades docentes positivas (23,5% vs 17,0%; p = 0,04).LIMITACIONES:Este estudio analizó un solo año de solicitudes a un centro académico y puede no ser generalizable.CONCLUSIÓN:Existen diferencias en las cualidades utilizadas para describir a los solicitantes femeninos versus masculinos en las cartas de recomendación de solicitud de residencia en cirugía colorrectal. Las candidatas femeninas se describieron con mayor frecuencia en términos académicos negativos y poseían cualidades de liderazgo negativas. Los hombres eran más propensos a ser descritos como amables, curiosos, académicamente impresionantes y con buenas habilidades docentes. El campo puede beneficiarse de iniciativas educativas para reducir el sesgo de género implícito en las cartas de recomendación. Consulte Video Resumen en http://links.lww.com/DCR/C191 . (Traducción-Dr. Ingrid Melo ).

Gender Differences in Letters of Recommendation for Gynecology Surgical Fellowship Applicants: Analysis of One Program's Recruitment Cycle

STUDY OBJECTIVE

To examine whether there are gender differences in letters of recommendation (LORs) written for residents applying to gynecology surgical fellowships.

DESIGN

Retrospective study.

SETTING

Single, academic institution.

PATIENTS

LORs for applicants to gynecology oncology, urogynecology, and minimally invasive gynecology fellowships during the 2019-2020 application cycle.

INTERVENTIONS

Not applicable.

MEASUREMENTS AND MAIN RESULTS

We analyzed the linguistic content of the letters for the presence of 4 summary variables and 21 word categories based on previous studies using validated computerized text analysis software. We used multivariable analysis using linear mixed models to compare linguistic characteristics of letters by applicant gender. We performed qualitative content analysis on letters and compared the frequency of code themes by gender. The mixed-method design was planned to allow for analysis of domains not captured in text analysis. Among 680 letters written for 186 applicants, 124 (18.2%) were written for men, and 556 (81.8%) were written for women. There were no differences in the least square mean (standard error) word counts for LORs written for men and women applicants, 465 (20.0) vs 458(9.4) words, p = .74. In multivariable analysis, LORs written for men were found to have higher authentic tone and more risk words (p = .005 and p = .03, respectively). LORs written for women contained more communal (relationship-oriented) words (p = .006). The qualitative analysis demonstrated that ability, interpersonal traits, surgical skills, and research were the most often mentioned themes. Comments about compassion/empathy, leadership potential, teaching, interpersonal skills, and patient rapport were found more often in letters for men. More doubt raisers (words that raise doubt or concern) were present in letters for men, but letters for both genders had similar levels of negative criticism. In contrast, comments on ability, being "drama-free," and self-awareness were found more often in letters for women.

CONCLUSION

There were gender differences in LORs written for obstetrics and gynecology surgical subspecialty fellowship applicants indicating the presence of gender bias.

Descriptors in Letters of Recommendation for Otolaryngology Residency Across Gender, Race, and Time

OBJECTIVE

To determine whether differences exist in the descriptors used in letters of recommendations for Otolaryngology - Head and Neck Surgery (OHNS) residency candidates, comparing race and gender as depicted in visual letters of recommendation (VLORs) from the 2014 and 2019 application cycles.

DESIGN

Four hundred thirty-three LORS (284 narrative LORs and 149 standardized LORs) and 63 medical student performance evaluations for 104 candidates who interviewed at the University of Cincinnati OHNS residency program in 2014 and 2019 were analyzed. Descriptors from LORs and medical student performance evaluations were collected by two reviewers and QSR NVivo 12 was used to generate a word cloud that grouped words by synonym and weighted them by frequency. Reviewers coded these synonyms into one of eight descriptor categories. Race and gender were self-reported from residency applications. The average of each category frequency for each race and gender were compared using student t-tests.

SETTING

University of Cincinnati OHNS Residency Program.

PARTICIPANTS

104 OHNS applicants that interviewed at the University of Cincinnati in 2014 and 2019.

RESULTS

Of the 104 candidates reviewed, 39 were female (37.5%). 66/104 (63%) of applicants identified as white, 31/104 (30%) as non-white, and 7/104 (7%) preferred not to say. No significant differences were found between male vs female descriptors. However, white applicants had more "leadership" descriptors (1.3% vs 0.5%, p = 0.01) and fewer "intelligence" words (6.6% vs 4.8%, p = 0.02) than non-white applicants. Applicants in 2019 were described with more team player (14.2% vs 9.6%, p < 0.0001), leadership (1.6% vs 0.7%, p = 0.047), and reserved (1.7% vs 0.7%, p = 0.02) words, but fewer grindstone (25.2% vs 32.1%, p < 0.0001) and ability/agentic (27.9% vs 32.6%, p < 0.0001) words than applicants in 2014.

CONCLUSIONS

In VLORs for OHNS residency, male and female applicants are described similarly, but white applicants are described as leaders more frequently and as intelligent less frequently than non-white counterparts. Regardless, it is encouraging to see applicants described with a more humanistic vocabulary in 2019.

Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

Purpose Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS).

Design Retrospective analysis of application characteristics versus residency performance was conducted.

Participants Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study.

Methods Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS.

Main Outcome Measures Characteristics from residency applications that correlate with resident performance were the primary outcome of this study.

Results Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance.

Conclusions Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.