Gendered Expectations: the Impact of Gender, Evaluation Language, and Clinical Setting on Resident Trainee Assessment of Faculty Performance

The Impact of Faculty Gender on Resident Evaluations of Faculty Performance in Emergency Medicine

INTRODUCTION

Gender bias impacts the promotion and tenure of female emergency medicine (EM) physicians and limits their ability to advance in academic rank. Many factors influence the promotion and tenure process including research, evaluations, opportunities for leadership, sponsorship, and mentorship. The goal of this study is to determine if resident evaluations of EM faculty differ by faculty gender.

METHODS

A quantitative analysis was used to examine 14,613 teaching evaluations of faculty by residents at a single academic center (The Ohio State University Wexner Medical Center, Columbus) in the years 2017-2019. Anonymized ratings of male and female faculty on a five-point Likert scale were compared using Fischer's exact test and adjusting for multiple comparisons.

RESULTS

Male faculty were more likely to hold the rank of Associate Professor or Professor. When taking both faculty gender and rank into account, male Clinical Instructors and Assistant Professors had significantly higher evaluation scores by residents in the domain of resident autonomy than their female counterparts. Regardless of gender or faculty rank, the majority of faculty received scores greater than four.

CONCLUSION

A significant gender difference was found in resident evaluation scores of faculty in the domain of resident autonomy at the level of Clinical Instructor and Assistant Professor. Resident autonomy refers to the degree of supervision by faculty which evolves over time and is primarily based on level of training. This is important as it demonstrates a gender difference in scores that could be used to determine faculty compensation and promotion. Evaluation tools used for promotion and tenure of academic faculty should be evaluated for implicit bias and appropriate statistical analysis.

Gender and the Receipt of the Association of Residents in Radiation Oncology Educator of the Year Award

Purpose

We hypothesized that there may be a gender disparity in the receipt of the Association of Residents in Radiation Oncology (ARRO) Educator of the Year Award and sought to elucidate factors that contribute to differences in award receipt.

Methods and Materials

Using a database provided by the American Society for Radiation Oncology, award recipients were identified from 2010 to 2022. Publicly available websites were accessed to obtain data regarding gender, years since residency graduation, percentage of female faculty, size of residency program, and program director designation. A 1-sample Z-test was used to assess whether the proportion of female ARRO award winners, defined as the proportion of female radiation oncology faculty members in the nominating universities that year, was significantly less than the population average. Secondary analyses used univariable binary logistic regression to identify global associations between gender, year since gradation, or program size.

Results

The lowest proportion of female awardees occurred in 2013 (14.3%) and the greatest proportion in 2022 (30.6%). Compared with the proportion of female faculty members in nominating programs for the respective year, there were significantly fewer female awardees in 2010 (18% female awardees vs 32% female faculty members; P = .02) and 2013 (14% female awardees vs 31% female faculty members; P = .01). There was a statistically significant increase in female awardees during the study period (P < .01). On logistic regression analysis, large program size (≥10 residents) (odds ratio [OR], 6.86; 95% CI, 2.71-23.1; P < .001) and medium program size (5-9 residents) (OR, 4.05; 95% CI, 1.60-13.7; P < .001) were associated with a greater proportion of female awardees compared with small program size (1-4 residents). There was no association between awardee gender and years since graduation.

Conclusions

A gender disparity was present in the receipt of ARRO Educator Awards. Residency chiefs, program directors, and chairs should work to ensure that a diverse slate of faculty is considered annually for the ARRO Educator Award.

Do Surgery Residents Prepare Enough for Surgical Cases?

IMPORTANCE

This study aimed to identify both modifiable and nonmodifiable factors that affect intraoperative-specific surgical education and performance, with an overall goal of increasing cognizance of such factors to improve surgical training.

OBJECTIVE

To determine whether surgery residents prepare adequately for participation in surgical cases and to examine specific variables that affect resident preparation.

DESIGN

This study is a retrospective survey-based study that included data from 1945 postoperative case evaluations completed by 59 different general surgery residents over a period of 8 years (2014-2022).

SETTING

A Midwestern medical school's general surgery residency program.

PARTICIPANTS

Fifty-nine general surgery residents at Western Michigan University's medical school; 50 attending surgeons and faculty with whom residents regularly operate. The sample was comprised of residents and attendings who voluntarily filled out postoperative performance surveys after elective cases.

RESULTS

This retrospective survey-based study included postoperative evaluation data from 1945 procedures performed by 59 different residents and 50 attendings. Participants included 36 male residents, 23 female residents, 39 male attendings, and 11 female attendings. All included data were for elective cases. Self-reported preoperative communication was worst at the PGY1 level with positive correlation of improvement yearly (r = 0.30, p < 0.001). Positive correlation was seen between overall preparedness and case complexity (r = 0.25, p < 0.001). Positive correlation was seen between case complexity and resident perception of intraoperative teaching quality (r = 0.53, p < 0.001). Preoperative communication initiated by residents was significantly worse when the attending surgeon was female, regardless of resident gender (p < 0.001); this effect was particularly profound with male residents. Male residents overall rated themselves as more prepared compared to their female counterparts (11.13 ± 1.96 vs. 10.84 ± 2.03, p = 0.003).

CONCLUSIONS AND RELEVANCE

There is a need to identify and address quantifiable gaps in communication between residents and faculty to optimize surgical education; one of the first steps is characterizing nonmodifiable factors that correlate with differences in pre-operative communication and case preparation.

Gender Disparity in Teaching Evaluations of Pediatric Faculty by Residents

OBJECTIVES

Despite their overrepresentation, female physicians continue to have lower rates of promotion compared with male physicians. Teaching evaluations play a role in physician advancement. Few studies have investigated gender disparity in resident evaluations of pediatric faculty. We hypothesized that gender disparities in resident evaluations of faculty exist and vary across subspecialties and primary work environments.

METHODS

Pediatric faculty institution-specific evaluations completed by residents from January 1, 2015, to March 9, 2020, were obtained from a single academic center. Mean ratings of faculty performance were compared by gender using a Wilcoxon 2-sample test.

RESULTS

Fifteen-thousand one-hundred and forty-two evaluations (5091 of male faculty and 10 051 of female faculty) were included. Female faculty were rated higher in overall teaching ability (female = 4.67 versus male = 4.65; P = .004). There was no statistical difference in the mean ratings of male and female faculty in the inpatient setting, whereas outpatient female faculty were rated higher in overall teaching ability (female = 4.79 versus male = 4.73; P = .005). For general pediatric faculty, females received higher ratings for overall teaching ability (female = 4.75 versus male = 4.70; P < .001). By contrast, there was no difference in ratings of subspecialty pediatric faculty.

CONCLUSIONS

Pediatric female faculty were statistically rated higher than male faculty in overall teaching ability, although these findings may not be educationally significant. The difference was driven by evaluations in the outpatient setting and for general pediatricians. This study is one of the first in pediatrics adding to the continued investigation of gender disparities in academic medicine.

Differences in Trainee Evaluations of Faculty by Rater and Ratee Gender

PURPOSE

To examine whether gender differences exist in medical trainees' (residents' and fellows') evaluations of faculty at a number of clinical departments.

METHOD

The authors conducted a single-institution (University of Minnesota Medical School) retrospective cohort analysis of 5,071 trainee evaluations of 447 faculty (for which trainee and faculty gender information was available) completed between July 1, 2019, and June 30, 2022. The authors developed and employed a 17-item measure of clinical teaching effectiveness, with 4 dimensions: overall teaching effectiveness, role modeling, facilitating knowledge acquisition, and teaching procedures. Using both between- and within-subject samples, they conducted analyses to examine gender differences among the trainees making ratings (rater effects), the faculty receiving ratings (ratee effects), and whether faculty ratings differed by trainee gender (interaction effects).

RESULTS

There was a statistically significant rater effect for the overall teaching effectiveness and facilitating knowledge acquisition dimensions (B = -0.28 and -0.14, 95% CI: [-0.35, -0.21] and [-0.20, -0.09], respectively, P < .001, medium corrected effect sizes between -0.34 and -0.54); female trainees rated male and female faculty lower than male trainees on both dimensions. There also was a statistically significant ratee effect for the overall teaching effectiveness and role modeling dimensions (B = -0.09 and -0.08, 95% CI: [-0.16, -0.02] and [-0.13, -0.04], P = .01 and < .001, respectively, small to medium corrected effect sizes between -0.16 and -0.44); female faculty were rated lower than male faculty on both dimensions. There was not a statistically significant interaction effect.

CONCLUSIONS

Female trainees rated faculty lower than male trainees and female faculty were rated lower than male faculty on 2 teaching dimensions each. The authors encourage researchers to continue to examine the reasons for the evaluation differences observed and how implicit bias interventions might help to address them.

Can AI Mitigate Bias in Writing Letters of Recommendation?

Letters of recommendation play a significant role in higher education and career progression, particularly for women and underrepresented groups in medicine and science. Already, there is evidence to suggest that written letters of recommendation contain language that expresses implicit biases, or unconscious biases, and that these biases occur for all recommenders regardless of the recommender's sex. Given that all individuals have implicit biases that may influence language use, there may be opportunities to apply contemporary technologies, such as large language models or other forms of generative artificial intelligence (AI), to augment and potentially reduce implicit biases in the written language of letters of recommendation. In this editorial, we provide a brief overview of existing literature on the manifestations of implicit bias in letters of recommendation, with a focus on academia and medical education. We then highlight potential opportunities and drawbacks of applying this emerging technology in augmenting the focused, professional task of writing letters of recommendation. We also offer best practices for integrating their use into the routine writing of letters of recommendation and conclude with our outlook for the future of generative AI applications in supporting this task.

Representation of Women and Minority Faculty and Fellows in Academic Pediatric Cardiology Training Programs

BACKGROUND

Studies have shown that diverse care teams optimize patient outcomes. Describing the current representation of women and minorities has been a critical step in improving diversity across several fields.

OBJECTIVES

To address the lack of data specific to pediatric cardiology, the authors conducted a national survey.

METHODS

U.S. academic pediatric cardiology programs with fellowship training programs were surveyed. Division directors were invited (July 2021 to September 2021) to complete an e-survey of program composition. Underrepresented minorities in medicine (URMM) were characterized using standard definitions. Descriptive analyses at the hospital, faculty, and fellow level were performed.

RESULTS

Altogether, 52 of 61 programs (85%) completed the survey, representing 1,570 total faculty and 438 fellows, with a wide range in program size (7-109 faculty, 1-32 fellows). Although women comprise approximately 60% of faculty in pediatrics overall, they made up 55% of fellows and 45% of faculty in pediatric cardiology. Representation of women in leadership roles was notably less, including 39% of clinical subspecialty directors, 25% of endowed chairs, and 16% of division directors. URMM comprise approximately 35% of the U.S. population; however, they made up only 14% of pediatric cardiology fellows and 10% of faculty, with very few in leadership roles.

CONCLUSIONS

These national data suggest a "leaky pipeline" for women in pediatric cardiology and very limited presence of URRM overall. Our findings can inform efforts to elucidate underlying mechanisms for persistent disparity and reduce barriers to improving diversity in the field.

Resident Operative Autonomy and Attending Verbal Feedback Differ by Resident and Attending Gender

Objectives

This study tests the null hypotheses that overall sentiment and gendered words in verbal feedback and resident operative autonomy relative to performance are similar for female and male residents.

Background

Female and male surgical residents may experience training differently, affecting the quality of learning and graduated autonomy.

Methods

A longitudinal, observational study using a Society for Improving Medical Professional Learning collaborative dataset describing resident and attending evaluations of resident operative performance and autonomy and recordings of verbal feedback from attendings from surgical procedures performed at 54 US general surgery residency training programs from 2016 to 2021. Overall sentiment, adjectives, and gendered words in verbal feedback were quantified by natural language processing. Resident operative autonomy and performance, as evaluated by attendings, were reported on 5-point ordinal scales. Performance-adjusted autonomy was calculated as autonomy minus performance.

Results

The final dataset included objective assessments and dictated feedback for 2683 surgical procedures. Sentiment scores were higher for female residents (95 [interquartile range (IQR), 4-100] vs 86 [IQR 2-100]; P < 0.001). Gendered words were present in a greater proportion of dictations for female residents (29% vs 25%; P = 0.04) due to male attendings disproportionately using male-associated words in feedback for female residents (28% vs 23%; P = 0.01). Overall, attendings reported that male residents received greater performance-adjusted autonomy compared with female residents (P < 0.001).

Conclusions

Sentiment and gendered words in verbal feedback and performance-adjusted operative autonomy differed for female and male general surgery residents. These findings suggest a need to ensure that trainees are given appropriate and equitable operative autonomy and feedback.

Gender Difference in Teaching Evaluation Scores of Pediatric Faculty

OBJECTIVE

To evaluate associations between faculty gender and milestone-based teaching assessment scores assigned by residents.

METHODS

We performed a retrospective cohort study of milestone-based clinical teaching assessments of pediatric faculty completed by pediatric residents at a mid-sized residency program from July 2016 to June 2019. Assessments included 3 domains (Clinical Interactions, Teaching Skills, Role Modeling/Professionalism) comprised of a total of 11 sub-competency items. We used multilevel logistic regression accounting for repeat measures and clustering to evaluate associations between faculty gender and assessment scores in the 1) top quartile, 2) bottom quartile, or 3) top-box (highest score). Findings were adjusted for faculty rank and academic track, and resident year and gender.

RESULTS

Over 3 years, 2889 assessments of 104 faculty were performed by 91 residents. Between assessments of women and men faculty, there were no significant differences in the odds of receiving a score in the top quartile for the 3 domains (Clinical aOR 0.99, P = .86; Teaching aOR 0.99, P = .93; Role Modeling aOR 0.87, P = .089). However, assessments of women were more likely to receive a score in the bottom quartile in both Teaching (aOR 1.23, P = .019) and Role Modeling (aOR 1.26, P = .008). Assessments of women also had lower odds of receiving the highest score in 6 of 11 sub-competencies.

CONCLUSION

Results suggest that gender bias may play a role in resident assessments of pediatric faculty. Future studies are needed to determine if findings are replicated in other settings and to identify opportunities to reduce the gender gap in pediatric academic medicine.