In Pursuit of Honors: A Multi-Institutional Study of Students' Perceptions of Clerkship Evaluation and Grading

Perception of Residency Program Diversity Is Associated With Vulnerability to Race and Gender Stereotype Threat Among Minority and Female Orthopaedic Trainees

Introduction

Stereotype threat (ST) is a psychological phenomenon in which perceived fear of confirming negative stereotypes about one's identity group leads to impaired performance. Gender and racial ST has been described in various academic settings. However, it is prevalence in orthopaedic surgery, where women and minorities are underrepresented, has not been examined. This study analyzes the prevalence of ST among orthopaedic surgery residents and fellows.

Methods

US orthopaedic trainees completed a voluntary anonymous survey, which included demographics, perceived program diversity based on percentage of racial-ethnic and gender-diverse faculty and trainees, and a validated, modified version of the Stereotype Vulnerability Scale (SVS). Higher scores indicate greater ST vulnerability. ST prevalence was analyzed with descriptive statistics, and associations between program diversity, resident demographics, and ST vulnerability were compared using nonparametric tests.

Results

Of 1,127 orthopaedic trainees at 40 programs, 322 responded (response rate 28.6%). Twenty-five percent identified as female, and 26% identified as an underrepresented minority in medicine (i.e., Asian, Black, or Hispanic). Asian (12 points), Black (12.5 points), and Hispanic (13.5 points) trainees had significantly higher SVS scores than White trainees (9 points) (p = 0.0003; p < 0.0001; p = 0.0028, respectively). Black trainees at perceived racially nondiverse residencies had the highest mean SVS scores (16.4 ± 1.03 points), while White trainees at perceived racially nondiverse residencies had the lowest SVS scores (9.3 ± 0.3 points), p = 0.011. Women had significantly higher gender stereotype vulnerability than men (p < 0.0001) in both gender-diverse (17.9 ± 0.2 vs. 9.0 ± 0.3 points) and gender nondiverse residencies (16.4 ± 0.4 vs. 9.6 ± 0.2 points).

Conclusion

Minority and female orthopaedic trainees had higher ST vulnerability, especially in programs perceived as lacking racial or gender diversity. While perceived program diversity may offer some protection for minority and women trainees, women trainees still met the threshold for high vulnerability regardless of program gender diversity. Future strategies to mitigate ST should be explored in orthopaedic training.

Disparities in Medical School Clerkship Grades Associated With Sex, Race, and Ethnicity: A Person-Centered Approach

PURPOSE

To determine whether students' self-reported race/ethnicity and sex were associated with grades earned in 7 core clerkships. A person-centered approach was used to group students based on observed clerkship grade patterns. Predictors of group membership and predictive bias by race/ethnicity and sex were investigated.

METHOD

Using data from 6 medical student cohorts at Johns Hopkins University School of Medicine (JHUSOM), latent class analysis was used to classify students based on clerkship grades. Multinomial logistic regression was employed to investigate if preclerkship measures and student demographic characteristics predicted clerkship performance-level groups. Marginal effects for United States Medical Licensing Exam (USMLE) Step 1 scores were obtained to assess the predictive validity of the test on group membership by race/ethnicity and sex. Predictive bias was examined by comparing multinomial logistic regression prediction errors across racial/ethnic groups.

RESULTS

Three clerkship performance-level groups emerged from the data: low, middle, and high. Significant predictors of group membership were race/ethnicity, sex, and USMLE Step 1 scores. Black or African American students were more likely (odds ratio [OR] = 4.26) to be low performers than White students. Black or African American (OR = 0.08) and Asian students (OR = 0.41) were less likely to be high performers than White students. Female students (OR = 2.51) were more likely to be high performers than male students. Patterns of prediction errors observed across racial/ethnic groups showed predictive bias when using USMLE Step 1 scores to predict clerkship performance-level groups.

CONCLUSIONS

Disparities in clerkship grades associated with race/ethnicity were found among JHUSOM students, which persisted after controlling for USMLE Step 1 scores, sex, and other preclerkship performance measures. Differential predictive validity of USMLE Step 1 exam scores and systematic error predictions by race/ethnicity show predictive bias when using USMLE Step 1 scores to predict clerkship performance across racial/ethnic groups.

Advancing Equity: Understanding, Addressing, and Mitigating Bias in Assessment in Medical Education

Despite the growing presence of women and historically underrepresented groups in academic medicine, significant disparities remain. This article examines a key aspect of these disparities: biases in assessment and learning environments. Reviewing current literature, including in OBGYN, reveals persistent gender and racial biases in subjective clinical narrative assessments. The paper then outlines a 2-pronged approach for change: first, enhancing the learning environment, and subsequently, providing targeted recommendations for individuals, academic leaders, and healthcare institutions to effectively address these biases.

Exploring Preclinical Medical Students' Experience Facilitating Group Dialectical Behavior Therapy (DBT) for a Student-Run Mental Health Clinic: A Qualitative Study

OBJECTIVE

This report explores the experiences of preclinical medical students who led group dialectical behavior therapy (DBT) for a student-run LGBTQ + mental health clinic.

METHODS

In the clinic, experienced clinicians trained and supervised preclinical medical students to facilitate DBT groups. The authors conducted a qualitative study to understand the impact of the DBT groups on the student facilitators via semi-structured interviews, which were evaluated using thematic analysis.

RESULTS

The clinic hosted nine iterations of group DBT facilitated by preclinical medical students, involving 18 student leaders and 30 patients. Twelve student facilitators were interviewed. Participants had a diverse array of specialty interests and were primarily motivated by the opportunity for early clinical experience. They reported improved clinical skills, increased appreciation of psychotherapy as a treatment modality, and increased interest in incorporating psychotherapy in their future practice. Furthermore, participants reported using DBT skills to cultivate wellbeing during clerkship year and in their personal lives.

CONCLUSIONS

Offering preclinical medical students the opportunity to lead group DBT therapy is a novel educational model providing early training in psychotherapy techniques. This opportunity for early direct patient experience in a supervised group setting attracted medical students with a diverse range of specialty interests. This model provided medical students specific DBT skills to implement in future patient care interactions and to maintain their personal wellbeing throughout medical training. The broad appeal and lasting effects of this program may prove beneficial at other institutions.

Frontline assessors' opinions about grading committees in a medicine clerkship

BACKGROUND

Collective decision-making by grading committees has been proposed as a strategy to improve the fairness and consistency of grading and summative assessment compared to individual evaluations. In the 2020-2021 academic year, Washington University School of Medicine in St. Louis (WUSM) instituted grading committees in the assessment of third-year medical students on core clerkships, including the Internal Medicine clerkship. We explored how frontline assessors perceive the role of grading committees in the Internal Medicine core clerkship at WUSM and sought to identify challenges that could be addressed in assessor development initiatives.

METHODS

We conducted four semi-structured focus group interviews with resident (n = 6) and faculty (n = 17) volunteers from inpatient and outpatient Internal Medicine clerkship rotations. Transcripts were analyzed using thematic analysis.

RESULTS

Participants felt that the transition to a grading committee had benefits and drawbacks for both assessors and students. Grading committees were thought to improve grading fairness and reduce pressure on assessors. However, some participants perceived a loss of responsibility in students' grading. Furthermore, assessors recognized persistent challenges in communicating students' performance via assessment forms and misunderstandings about the new grading process. Interviewees identified a need for more training in formal assessment; however, there was no universally preferred training modality.

CONCLUSIONS

Frontline assessors view the switch from individual graders to a grading committee as beneficial due to a perceived reduction of bias and improvement in grading fairness; however, they report ongoing challenges in the utilization of assessment tools and incomplete understanding of the grading and assessment process.

'Yourself in all your forms': A grounded theory exploration of identity safety in medical students

INTRODUCTION

Identity threats, such as stereotype threat and microaggressions, impair learning and erode well-being. In contrast to identity threat, less is known about how learners experience feelings of safety regarding their identity. This exploratory study aims to develop a theory of identity safety in the clinical learning environment.

METHODS

This multi-institutional, qualitative interview study was informed by constructivist grounded theory and critical pedagogy. Participants were clinical students at three public medical schools in the United States in 2022. Investigators purposively sampled participants for interviews based on their responses to an 11-item survey with an open-ended question soliciting students' personal identities and responses to both the racial/ethnic and gender Stereotype Vulnerability Scales. The investigators interviewed, coded, constantly compared and continued sampling until the codes could be developed into categories, then concepts and finally into a theory. The team engaged in critical reflexivity throughout the analytic process to enrich data interpretations.

RESULTS

Sixteen diverse students were interviewed. We organised their identity-salient experiences into identity threat, threat mitigation and identity safety. Participants experienced identity threat through unwelcoming learning environments, feeling compelled to change their behaviour in inauthentic ways or sociopolitical threat. Threat mitigation occurred when a participant or supervisor intervened against an identity threat, dampening but not eliminating the threat impact. Participants characterised identity safety as the ability to exist as their authentic selves without feeling the need to monitor how others perceive their identities. Identity safety manifested when participants demonstrated agency to leverage their identities for patient care, when others upheld their personhood and saw them as unique individuals and when they felt they belonged in the learning environment.

DISCUSSION

Attending to identity safety may lead to educational practices that sustain and leverage team members' diverse identities. Identity safety and threat mitigation may work together to combat identity threats in the learning environment.

Importance of Diversity, Equity, and Inclusion in the Hepatopancreatobiliary Workforce

Diversity is a catalyst for progress that prevents institutional stagnation and, by extension, averts descent to mediocrity. This review focuses on the available data concerning hepatopancreatobiliary (HPB) surgical workforce demographics and identifies evidence-based strategies that may enhance justice, equity, diversity, and inclusion for HPB surgeons and their patients. We report that the current United States HPB surgical workforce does not reflect the population it serves. We review data describing disparity-perpetuating hurdles confronting physicians from minority groups underrepresented in medicine at each stage of training. We further examine evidence showing widespread racial and socioeconomic disparities in HPB surgical care and review the effects of workforce diversity and physician-patient demographic concordance on healthcare outcomes. Evidence-based mitigators of structural racism and segregation are reviewed, including tailored interventions that can address social determinants of health toward the achievement of true excellence in HPB surgical care. Lastly, select evidence-based data driving surgical workforce solutions are reviewed, including intentional compensation plans, mentorship, and sponsorship.

Current practices and perspectives on clerkship grading in obstetrics and gynecology

BACKGROUND

Clerkship grades in obstetrics and gynecology play an increasingly important role in the competitive application process to residency programs. An analysis of clerkship grading practices has not been queried in the past 2 decades in our specialty.

OBJECTIVE

This study aimed to investigate obstetrics and gynecology clerkship directors' practices and perspectives in grading.

STUDY DESIGN

A 12-item electronic survey was developed and distributed to clerkship directors with active memberships in the Association of Professors of Gynecology and Obstetrics.

RESULTS

A total of 174 of 236 clerkship directors responded to the survey (a response rate of 73.7%). Respondents reported various grading systems with the fewest (20/173 [11.6%]) using a 2-tiered or pass or fail system and the most (72/173 [41.6%]) using a 4-tiered system. Nearly one-third of clerkship directors (57/163 [35.0%]) used a National Board of Medical Examiners subject examination score threshold to achieve the highest grade. Approximately 45 of 151 clerkship directors (30.0%) had grading committees. Exactly half of the clerkship directors (87/174 [50.0%]) reported requiring unconscious bias training for faculty who assess students. In addition, some responded that students from groups underrepresented in medicine (50/173 [28.9%]) and introverted students (105/173 [60.7%]) received lower evaluations. Finally, 65 of 173 clerkship directors (37.6%) agreed that grades should be pass or fail.

CONCLUSION

Considerable heterogeneity exists in obstetrics and gynecology clerkship directors' practices and perspectives in grading. Strategies to mitigate inequities and improve the reliability of grading include the elimination of a subject examination score threshold to achieve the highest grade and the implementation of both unconscious bias training and grading committees.

Medical competence as a multilayered construct

BACKGROUND

The conceptualisation of medical competence is central to its use in competency-based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment.

THE MODEL

Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context-dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self-determination theory and Maslow's 'self-actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different.

IMPLICATIONS

The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context-dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded.

Modeling the social determinants of resilience in health professions students: impact on psychological adjustment

Stressors inherent to training and stemming from the learning environment are associated with high rates of burnout, depression, and mental health problems in health professions students (HPS). There is evidence that disadvantaged or stigmatized groups are particularly affected. These problems not only impact students after graduation but may also have detrimental effects on patient outcomes. Resilience, conceptualized as the process of adapting well in the face of adversity, has inspired an increasing number of interventions aimed at addressing those problems in HPS. These interventions have mostly targeted individual students and their psychological traits while ignoring social and structural factors that may enhance or undermine individual resilience. To address this gap in the literature, the authors reviewed the evidence for psychosocial determinants of resilience and proposed a model inspired by the social determinants of health literature and the "upstream-downstream" metaphor. In this theoretical paper, the authors propose that upstream determinants such adverse childhood experiences and socioeconomic and sociodemographic markers of disadvantage have a direct effect on psychological adjustment and an indirect effect mediated by resilience. Additionally, the authors propose that the institutional downstream drivers of learning environment, social support, and sense of belonging moderate the direct and indirect effects of the upstream determinants on psychological adjustment. Future research should test these hypotheses and gather evidence that may guide the development of interventions. The authors present their model as part of a comprehensive response to recent calls to action to address diversity, equity and inclusion in health professions education.

"Rater training" re-imagined for work-based assessment in medical education

In this perspective, the authors critically examine "rater training" as it has been conceptualized and used in medical education. By "rater training," they mean the educational events intended to improve rater performance and contributions during assessment events. Historically, rater training programs have focused on modifying faculty behaviours to achieve psychometric ideals (e.g., reliability, inter-rater reliability, accuracy). The authors argue these ideals may now be poorly aligned with contemporary research informing work-based assessment, introducing a compatibility threat, with no clear direction on how to proceed. To address this issue, the authors provide a brief historical review of "rater training" and provide an analysis of the literature examining the effectiveness of rater training programs. They focus mainly on what has served to define effectiveness or improvements. They then draw on philosophical and conceptual shifts in assessment to demonstrate why the function, effectiveness aims, and structure of rater training requires reimagining. These include shifting competencies for assessors, viewing assessment as a complex cognitive task enacted in a social context, evolving views on biases, and reprioritizing which validity evidence should be most sought in medical education. The authors aim to advance the discussion on rater training by challenging implicit incompatibility issues and stimulating ways to overcome them. They propose that "rater training" (a moniker they suggest be reserved for strong psychometric aims) be augmented with "assessor readiness" programs that link to contemporary assessment science and enact the principle of compatibility between that science and ways of engaging with advances in real-world faculty-learner contexts.

Psychological safety in medical education: A scoping review and synthesis of the literature

PURPOSE

Psychological safety (PS) is the belief that the environment is safe for risk taking. Available data point to a lack of PS in medical education. Based on literature in other fields, PS in clinical learning environments (CLEs) could support trainee well-being, belonging, and learning. However, the literature on PS in medical education has not been broadly assessed.

MATERIALS AND METHODS

In 2020, authors searched PubMed, Web of Science, CINAHL, Scopus, ERIC, PsycInfo, and JSTOR for articles published prior to January 2020. Authors screened all search results for eligibility using specific criteria. Data were extracted and thematic analysis performed.

RESULTS

Fifty-two articles met criteria. The majority focused on graduate medical education (45%), and 42% of studies took place within a CLE. Articles addressed organizational and team level constructs (58%), with fewer descriptions of specific behaviors of team members that promote or hinder safety. The impacts of safe environments for trainees and patients are areas in need of more exploration.

DISCUSSION

Future research should focus on defining specific organizational and interpersonal leader behaviors that promote PS, seek to understand how PS is determined by individual trainees, and measure the impact of PS on learners, learning, and patient care outcomes.

"My Assessments Are Biased!" Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education

Assessing learners is foundational to their training and developmental growth throughout the medical education continuum. However, growing evidence shows the prevalence and impact of harmful bias in assessments in medical education, accelerating the urgency to identify solutions. Assessment bias presents a critical problem for all stages of learning and the broader educational system. Bias poses significant challenges to learners, disrupts the learning environment, and threatens the pathway and transition of learners into health professionals. While the topic of assessment bias has been examined within the context of measurement literature, limited guidance and solutions exist for learners in medical education, particularly in the clinical environment. This article presents an overview of assessment bias, focusing on clinical learners. A definition of bias and its manifestations in assessments are presented. Consequences of assessment bias are discussed within the contexts of validity and fairness and their impact on learners, patients/caregivers, and the broader field of medicine. Messick's unified validity framework is used to contextualize assessment bias; in addition, perspectives from sociocultural contexts are incorporated into the discussion to elaborate the nuanced implications in the clinical training environment. Discussions of these topics are conceptualized within the literature and the interventions used to date. The article concludes with practical recommendations to overcome bias and to develop an ideal assessment system. Recommendations address articulating values to guide assessment, designing assessment to foster learning and outcomes, attending to assessment procedures, promoting continuous quality improvement of assessment, and fostering equitable learning and assessment environments.

The Urgency of Now: Rethinking and Improving Assessment Practices in Medical Education Programs

Assessment is essential to professional development. Assessment provides the information needed to give feedback, support coaching and the creation of individualized learning plans, inform progress decisions, determine appropriate supervision levels, and, most importantly, help ensure patients and families receive high-quality, safe care in the training environment. While the introduction of competency-based medical education has catalyzed advances in assessment, much work remains to be done. First, becoming a physician (or other health professional) is primarily a developmental process, and assessment programs must be designed using a developmental and growth mindset. Second, medical education programs must have integrated programs of assessment that address the interconnected domains of implicit, explicit and structural bias. Third, improving programs of assessment will require a systems-thinking approach. In this paper, the authors first address these overarching issues as key principles that must be embraced so that training programs may optimize assessment to ensure all learners achieve desired medical education outcomes. The authors then explore specific needs in assessment and provide suggestions to improve assessment practices. This paper is by no means inclusive of all medical education assessment challenges or possible solutions. However, there is a wealth of current assessment research and practice that medical education programs can use to improve educational outcomes and help reduce the harmful effects of bias. The authors' goal is to help improve and guide innovation in assessment by catalyzing further conversations.

The Problematic Persistence of Tiered Grading in Medical School

Issue: The evaluation of medical students is a critical, complex, and controversial process. It is tightly woven into the medical school curriculum, beginning at the inception of the medical student's professional journey. In this respect, medical student evaluation is among the first in a series of ongoing, lifelong assessments that influence the interpersonal, ethical, and socioeconomic dimensions necessary for an effective physician workforce. Yet, tiered grading has a questionable historic pedagogic basis in American medical education, and evidence suggests that tiered grading itself is a source of student burnout, anxiety, depression, increased competitiveness, reduced group cohesion, and racial biases. Evidence: In its most basic form, medical student evaluation is an assessment of the initial cognitive and technical competencies ultimately needed for the safe and effective practice of contemporary medicine. At many American medical schools, such evaluation relies largely on norm-based comparisons, such as tiered grading. Yet, tiered grading can cause student distress, is considered unfair by most students, is associated with biases against under-represented minorities, and demonstrates inconsistent correlation with residency performance. While arguments that tiered grading motivates student performance have enjoyed historic precedence in academia, such arguments are not supported by robust data or theories of motivation. Implications: Given the evolving recognition of the deleterious effects on medical student mental health, cohesiveness, and diversity, the use of tiered grading in medical schools to measure or stimulate academic performance, or by residency program directors to distinguish residency applicants, remains questionable. Examination of tiered grading in its historical, psychometric, psychosocial, and moral dimensions and the various arguments used to maintain it reveals a need for investigation of, if not transition to, alternative and non-tiered assessments of our medical students.

Fixing a Broken Clerkship Assessment Process: Reflections on Objectivity and Equity Following the USMLE Step 1 Change to Pass/Fail

Clerkship grading is a core feature of evaluation for medical students' skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.

Racial and Ethnic Bias in Medical School Clinical Grading: A Review

OBJECTIVE

Improving diversity in healthcare is a widely recognized national goal. The diversity of medical student matriculants has increased, yet this trend is not seen in the composition of competitive residency programs. In this review, we examine racial and ethnic disparities in medical student grading during clinical years and explore the consequences of how this may exclude minority students from accessing competitive residency positions.

DESIGN

Following PRISMA guidelines, we searched PubMed, Embase, Scopus, and ERIC databases using variations of the terms "race," "ethnicity," "clerkship," "rotation," "grade," "evaluation", or "shelf exam." Of 391 references found using the criteria, 29 were related to clinical grading and race/ethnicity and included in the review. The GRADE criteria were used to determine the quality of evidence.

SETTING

Johns Hopkins School of Medicine, Baltimore MD.

RESULTS

Five studies examining a total of 107,687 students from up to 113 different schools found racial minority students receive significantly fewer Honors grades in core clerkships compared to White students. Three studies examining 94,814 medical student evaluations from up to 130 different schools found significant disparities in the wording of written clerkship evaluations based on race and/or ethnicity.

CONCLUSIONS

A large body of evidence suggests the presence of racial bias in subjective clinical grading and written clerkship evaluations of medical students. Grading disparities can disadvantage minority students when applying to competitive residency programs and may contribute to a lack of diversity in these fields. As low minority representation has a negative impact on patient care and research advancement, strategies to resolve this issue must be further explored.

The Undergraduate to Graduate Medical Education Transition as a Systems Problem: A Root Cause Analysis

The transition from undergraduate medical education (UME) to graduate medical education (GME) constitutes a complex system with important implications for learner progression and patient safety. The transition is currently dysfunctional, requiring students and residency programs to spend significant time, money, and energy on the process. Applications and interviews continue to increase despite stable match rates. Although many in the medical community acknowledge the problems with the UME-GME transition and learners have called for prompt action to address these concerns, the underlying causes are complex and have defied easy fixes. This article describes the work of the Coalition for Physician Accountability's Undergraduate Medical Education to Graduate Medical Education Review Committee (UGRC) to apply a quality improvement approach and systems thinking to explore the underlying causes of dysfunction in the UME-GME transition. The UGRC performed a root cause analysis using the 5 whys and an Ishikawa (or fishbone) diagram to deeply explore problems in the UME-GME transition. The root causes of problems identified include culture, costs and limited resources, bias, systems, lack of standards, and lack of alignment. Using the principles of systems thinking (components, connections, and purpose), the UGRC considered interactions among the root causes and developed recommendations to improve the UME-GME transition. Several of the UGRC's recommendations stemming from this work are explained. Sustained monitoring will be necessary to ensure interventions move the process forward to better serve applicants, programs, and the public good.

The Effect of Pass/Fail Exam Grading on Exam Performance in a Pediatric Clerkship

OBJECTIVE

Clerkship grades are a component of determining a residency candidate's competitiveness. In 2017, the University of Minnesota Medical School's pediatric clerkship transitioned its standardized multiple-choice exam, the Aquifer Pediatrics Examination, to pass/fail with eligibility for honors being determined by clinical performance, not exam performance. We assessed the effect this change had on Aquifer exam performance and evaluated for correlation between Aquifer exam performance and clinical evaluation scores in order to gather insight into the validity of each type of assessment with respect to one another.

METHODS

We analyzed de-identified data from 750 medical students between the academic years of 2016 to 2017 and 2019 to 2020. Individual Aquifer exam scores were compared to individual clinical performance scores. Differences in exam performance before and after the transition to pass/fail were investigated with a two-sample t-test and Cohen's d for effect size.

RESULTS

No correlation was found between Aquifer exam scores and clinical performance scores. The mean Aquifer exam score prior to the transition to pass/fail was 80.02 ± 7.51 while the mean after the exam was made pass/fail was 77.8 ± 7.42. This difference was statistically significant (P < .001) with a Cohen's d (effect size) of 0.297.

CONCLUSIONS

A lack of correlation between the Aquifer exam scores and clinical performance scores was found. There was a small yet statistically significant decrease in Aquifer exam scores after the change to pass/fail; it is not clear if this represents a meaningful decrease in learning by students.

Hawks and Doves: Perceptions and Reality of Faculty Evaluations

OBJECTIVES

Internal medicine clerkship grades are important for residency selection, but inconsistencies between evaluator ratings threaten their ability to accurately represent student performance and perceived fairness. Clerkship grading committees are recommended as best practice, but the mechanisms by which they promote accuracy and fairness are not certain. The ability of a committee to reliably assess and account for grading stringency of individual evaluators has not been previously studied.

METHODS

This is a retrospective analysis of evaluations completed by faculty considered to be stringent, lenient, or neutral graders by members of a grading committee of a single medical college. Faculty evaluations were assessed for differences in ratings on individual skills and recommendations for final grade between perceived stringency categories. Logistic regression was used to determine if actual assigned ratings varied based on perceived faculty's grading stringency category.

RESULTS

"Easy graders" consistently had the highest probability of awarding an above-average rating, and "hard graders" consistently had the lowest probability of awarding an above-average rating, though this finding only reached statistical significance only for 2 of 8 questions on the evaluation form (P = .033 and P = .001). Odds ratios of assigning a higher final suggested grade followed the expected pattern (higher for "easy" and "neutral" compared to "hard," higher for "easy" compared to "neutral") but did not reach statistical significance.

CONCLUSIONS

Perceived differences in faculty grading stringency have basis in reality for clerkship evaluation elements. However, final grades recommended by faculty perceived as "stringent" or "lenient" did not differ. Perceptions of "hawks" and "doves" are not just lore but may not have implications for students' final grades. Continued research to describe the "hawk and dove effect" will be crucial to enable assessment of local grading variation and empower local educational leadership to correct, but not overcorrect, for this effect to maintain fairness in student evaluations.

The time is now: addressing implicit bias in obstetrics and gynecology education

Obstetrician-gynecologists can improve the learning environment and patient care by addressing implicit bias. Accumulating evidence demonstrates that racial and gender-based discrimination is woven into medical education, formal curricula, patient-provider-trainee interactions in the clinical workspace, and all aspects of learner assessment. Implicit bias negatively affects learners in every space. Strategies to address implicit bias at the individual, interpersonal, institutional, and structural level to improve the well-being of learners and patients are needed. The authors review an approach to addressing implicit bias in obstetrics and gynecology education, which includes: (1) curricular design using an educational framework of antiracism and social justice theories, (2) bias awareness and management pedagogy throughout the curriculum, (3) elimination of stereotypical patient descriptions from syllabi and examination questions, and (4) critical review of epidemiology and evidence-based medicine for underlying assumptions based on discriminatory practices or structural racism that unintentionally reinforce stereotypes and bias. The movement toward competency-based medical education and holistic evaluations may result in decreased bias in learner assessment. Educators may wish to monitor grades and narratives for bias as a form of continuous educational equity improvement. Given that practicing physicians may have little training in this area, faculty development efforts in bias awareness and mitigation strategies may have significant impact on learner well-being.

Correlation of Narrative Evaluations to Clerkship Grades Using Statistical Sentiment Analysis

Introduction

Narrative evaluations are essential components of medical student assessment. This study evaluated how well narrative clerkship evaluation word choice correlated with an assigned letter grade.

Methods

One hundred clerkship evaluations, 50 from family medicine (FM) and 50 from internal medicine (IM), with even distribution of "Honors" and "Near-Honors" among medical students that graduated in 2020 from the Oregon Health and Science University (OHSU) were examined. A textual sentiment analysis, which evaluates positive and negative word choice, was used to determine each evaluation's collective sentiment. An average sentiment score and character count were calculated for Honors and Near-Honors evaluations from both clerkship disciplines. Sentiment word totals were used to form "word clouds" that highlight the most frequent word selections.

Results

While sentiment scores positively correlated with the assigned grade, there was no statistically significant difference between the average sentiment scores among Honors and Near Honors graded evaluations within the FM or IM clerkship evaluation sets. There was no significant difference in evaluation character length among the assigned grades. Among FM evaluations, "outstanding" and "excellent" were the two most common sentiment words used in both Honors and Near-Honors. Among IM evaluations, outstanding and excellent were most commonly used in Honors evaluations, while "excellent" and "good" were most common in Near-Honors.

Conclusion

This study outlines a novel text analysis method for analyzing narrative evaluation association with assigned grade that other institutions can utilize. Sentiment word choices are not significantly different among Honors and Near Honors clerkship narrative evaluations.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-022-01654-2.

Using entrustable professional activities to better prepare students for their postgraduate medical training: A medical student's perspective

THE PROBLEM

Medical students graduate underprepared for postgraduate medical training despite years of classroom and clinical training. In this article, a medical student shares her personal perspectives on three factors contributing to this problem in undergraduate medical education: students' peripheral roles in the clinical environment impede learning, students receive inadequate feedback, and assessments do not measure desired learning outcomes.

A SOLUTION

The authors describe how using entrustable professional activities (EPAs) could address these issues and promote students' clinical engagement by clarifying their roles, providing them with frequent and actionable feedback, and aligning their assessments with authentic work. These factors combined with grading schemes rewarding improvement could contribute to a growth mindset that reprioritizes clinical skill acquisition. The authors explore how medical schools have begun implementing the EPA framework, highlight insights from these efforts, and describe barriers that must be addressed.

THE FUTURE

Incorporating EPAs into medical school curricula could better prepare students for postgraduate training while also alleviating issues that contribute to student burnout by defining students' roles, improving feedback, and aligning assessments with desired learning outcomes.

Beyond Competency: A Student Perspective on Growth Through Clerkship Feedback

This article proposes a paradigm shift from the competency-based model of clerkship feedback using checklists to a coaching-based, action plan-oriented process that centers on individualized student-oriented goals. Using a student perspective, the authors examine the feedback literature and put forward a proposal to use an impact model whose emphasis is to improve the learning climate for students. Several techniques are reviewed which include goal generation and creation of dynamic action plans. By intentionally focusing on coaching relationships as a platform for feedback, the learners and mentors share goals and the result of feedback becomes action-based behaviors which may help negate personal attribution and bias in the feedback process.

Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are widely used, including in licensure examinations. OSCEs assess diverse skills in a structured manner, but can be stressful for students and have a significant organisational burden for faculty. Case-based video OSCE preparation resources were developed for students preparing for a licensure OSCE. The study aimed to examine student engagement, compare scores awarded by students to performances at specific competence standards with faculty scores, and examine usability, usefulness, and acceptability of the video cases.

EDUCATIONAL ACTIVITY AND SETTING

Final year pharmacy students (n = 149) enrolled in an integrated master of pharmacy programme in Ireland were invited to participate. Six sets of recorded OSCE-based video cases were developed, comprising multiple recordings of the same case scenario, with each pitched at a different level of performance. Students watched and scored the video cases. Usability, usefulness, and acceptability were evaluated via questionnaire.

FINDINGS

One or more video cases were accessed by 70.5% of students. Score ratings awarded by the students, when compared to faculty ratings, showed an overall trend towards inter-rater agreement between students and faculty. Students felt positively in terms of the usability, usefulness, and acceptability of the videos.

SUMMARY

Video cases designed to support OSCE preparation for a high-stakes national pharmacy licensure examination were widely used by students and were perceived to be usable, useful, and acceptable. Such video cases may be a feasible alternative to additional mock OSCEs to support student preparation.

Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships

PURPOSE

To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation.

METHOD

This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system.

RESULTS

Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning.

CONCLUSIONS

Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.

Association Between Resident Race and Ethnicity and Clinical Performance Assessment Scores in Graduate Medical Education

PURPOSE

To assess the association between internal medicine (IM) residents' race/ethnicity and clinical performance assessments.

METHOD

The authors conducted a cross-sectional analysis of clinical performance assessment scores at 6 U.S. IM residency programs from 2016 to 2017. Residents underrepresented in medicine (URiM) were identified using self-reported race/ethnicity. Standardized scores were calculated for Accreditation Council for Graduate Medical Education core competencies. Cross-classified mixed-effects regression assessed the association between race/ethnicity and competency scores, adjusting for rotation time of year and setting; resident gender, postgraduate year, and IM In-Training Examination percentile rank; and faculty gender, rank, and specialty.

RESULTS

Data included 3,600 evaluations by 605 faculty of 703 residents, including 94 (13.4%) URiM residents. Resident race/ethnicity was associated with competency scores, with lower scores for URiM residents (difference in adjusted standardized scores between URiM and non-URiM residents, mean [standard error]) in medical knowledge (-0.123 [0.05], P = .021), systems-based practice (-0.179 [0.05], P = .005), practice-based learning and improvement (-0.112 [0.05], P = .032), professionalism (-0.116 [0.06], P = .036), and interpersonal and communication skills (-0.113 [0.06], P = .044). Translating this to a 1 to 5 scale in 0.5 increments, URiM resident ratings were 0.07 to 0.12 points lower than non-URiM resident ratings in these 5 competencies. The interaction with faculty gender was notable in professionalism (difference between URiM and non-URiM for men faculty -0.199 [0.06] vs women faculty -0.014 [0.07], P = .01) with men more than women faculty rating URiM residents lower than non-URiM residents. Using the 1 to 5 scale, men faculty rated URiM residents 0.13 points lower than non-URiM residents in professionalism.

CONCLUSIONS

Resident race/ethnicity was associated with assessment scores to the disadvantage of URiM residents. This may reflect bias in faculty assessment, effects of a noninclusive learning environment, or structural inequities in assessment.

Relative Importance of Applicant Characteristics in Ophthalmology Residency Interview Selection: A Survey of Program Directors

Introduction  Ophthalmology residency positions remain competitive. A lack of clarity regarding which residency selection criteria are prioritized by program directors can heighten the stress associated with the match process. While surveys of program directors in several other medical specialties have been conducted to identify the most important residency selection criteria, there is limited data on selection criteria used by ophthalmology residency program directors. The purpose of our study was to survey ophthalmology residency program directors to identify the current state of interview selection decisions-the factors currently considered most important in determining whether to extend an interview invitation to residency applicants. Methods  We developed and distributed a Web-based questionnaire to all U.S. ophthalmology residency program directors. Questions evaluated program demographics and the relative importance of 23 different selection criteria used by ophthalmology residency program directors when evaluating applicants for residency interviews (Likert scale 1-5, with 1 being "not important" and 5 being "very important"). Program directors were also asked to identify the one factor they felt was most important. Results  The overall residency program director response rate was 56.5% (70/124). The selection criteria with the highest average importance scores were core clinical clerkship grades (4.26/5) followed by letters of recommendation (4.06/5), and United States Medical Licensing Examination (USMLE) Step 1 score (4.03/5). The most frequently cited single most important factor for interview selection was core clinical clerkship grades (18/70, 25.7%), with USMLE Step 1 score (9/70, 12.9%) and rotations at the program director's department (6/70, 8.6%) also commonly reported. Conclusion  Our results suggest that core clinical clerkship grades, letters of recommendation, and USMLE Step 1 scores are deemed the most important selection criteria by ophthalmology residency program directors as of a 2021 survey. With changes in clerkship grading for many medical schools and changes in national USMLE Step 1 score reporting, programs will face challenges in evaluating applicants and the relative importance of other selection criteria will likely increase.

Innovation and Missed Opportunities in Internal Medicine Undergraduate Education During COVID-19: Results from a National Survey

BACKGROUND

COVID-19 disrupted undergraduate clinical education when medical schools removed students from clinical rotations following AAMC recommendations. Clerkship directors (CDs) had to adapt rapidly and modify clerkship curricula. However, the scope and effects of these modifications are unknown.

OBJECTIVE

To examine the effects of the initial phase of COVID-19 on the internal medicine (IM) undergraduate clinical education.

DESIGN

A nationally representative web survey.

PARTICIPANTS

IM CDs from 137 LCME-accredited US medical schools in 2020.

MAIN MEASURES

Items (80) assessed clerkship structure and curriculum, assessment in clerkships, post-clerkship IM clinical experiences, and CD roles and support. The framework of Understanding Crisis Response (Royal Society for Encouragement of Arts, Manufactures, and Commerce) was used to determine whether curricular modifications were "amplified," "restarted," "let go," or "ended."

KEY RESULTS

Response rate was 74%. In response to COVID-19, 32% (32/101) of clerkships suspended all clinical activities and 66% (67/101) only in-person. Prior to clinical disruption, students spent a median of 8.0 weeks (IQR: 2) on inpatient and 2.0 weeks (IQR: 4) on ambulatory rotations; during clinical re-entry, students were spending 5.0 (IQR: 3) and 1.0 (IQR: 2) weeks, respectively. Bedside teaching and physical exam instruction were "let go" during the early phase. Students were removed from direct patient care for a median of 85.5 days. The sub-internship curriculum remained largely unaffected. Before the pandemic, 11% of schools were using a pass/fail grading system; at clinical re-entry 47% and during the survey period 23% were using it. Due to the pandemic, 78.2% of CDs assumed new roles or had expanded responsibilities; 51% reported decreased scholarly productivity.

CONCLUSIONS

Curricular adaptations occurred in IM clerkships across US medical schools as a result of COVID-19. More research is needed to explore the long-term implications of these changes on medical student education and clinical learning environments.

Growth, Engagement, and Belonging in the Clinical Learning Environment: the Role of Psychological Safety and the Work Ahead

Psychological safety is the perception that an environment is safe for interpersonal risk taking, exposing vulnerability, and contributing perspectives without fear of being shamed, blamed, or ignored. The presence of psychological safety has been associated with improved team learning and innovation, leader inclusivity, and team members' sense of belonging. In medical education, psychological safety has additional benefits: it allows learners to be present in the moment and to focus on the tasks at hand, and reduces trainee focus on image. Several key features of psychologically safe environments have already been described, including the presence of high-quality relationships, the absence of social positioning, a learner-driven and flexible learning agenda, the lack of formal assessment, and time for debriefing. However, many of the structures and cultural traditions in medical education are in clear opposition to these features. This paper describes the current barriers to psychological safety in medical education, and sets out an agenda for change. In accordance with benefits seen in other sectors, we anticipate that an emphasis on relationships and psychological safety will support the learning, inclusion, and success of medical trainees.

Profiles for Success: Examining the Relationship between Student Profiles and Clerkship Performance Using Latent Profile Analysis

PHENOMENON

Learning is a complex phenomenon in which learners can vary in their learning orientation, learning approaches, and perception of the learning environment. Learners may be characterized as mastery oriented or performance oriented, and this learning orientation can influence their learning approaches, such as whether to seek feedback or ask for help. The learning environment includes institutional assessment policies and informal interactions that emphasize outcomes and differentiation among students. When learners perceive the learning environment as performance oriented, they feel competition from other learners and fear negative feedback. This complex interplay of learning orientation, its influence on feedback-seeking behavior, and the environmental influence and reinforcement of both can be captured through learner profiles.

APPROACH

In this multi-institutional cross-sectional survey study of students from six U.S. medical schools, we investigated learner profiles that characterize how these domains manifest together across individual learners. We then determined how these profiles are associated with students' clerkship grades. Measures included the Patterns of Adaptive Learning for learning orientation, self-reports for learning approaches, and perceptions of the learning environment. We used latent profile analysis to cluster students who share common characteristics around orientation, behavior, and environment. The relationship between these profiles and the percentage of honors earned was examined using multiple regression analysis.

FINDINGS

We found four distinct learner profiles within 666 of 974 (68%) analyzable responses: (1) performance oriented with negative perceptions of environment, (2) mastery oriented with desirable learning approaches, (3) "average" group, and (4) mastery orientation only. Profile 1 (M = 39%) and Profile 4 (M = 38%) students received fewer clerkship honors compared with other profiles. Profile 2 students earned a significantly higher percentage of honors grades (M = 54%) compared with other students. Profile 3 students (average group) earned the second highest percentage of honors (M = 46%). Effect sizes comparing the percentage of honors varied from 0.32 to 0.65, indicating medium to large differences.

INSIGHTS

The highest performers, as measured by clerkship honors grades, were mastery oriented, perceived a more positive learning environment, and were comfortable asking questions and seeking feedback for their learning. The lowest performing students were performance oriented and had negative perceptions of their learning environments. Contrary to previous findings, we found that the relationship between mastery orientation and learning approaches is not unidirectional but rather seems to be mediated by perceptions of the learning environment.

Moving toward Mastery: Changes in Student Perceptions of Clerkship Assessment with Pass/Fail Grading and Enhanced Feedback

ProblemClerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. Intervention: In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. Context: In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Impact: Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. Lessons Learned: After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.

We Have No Choice but to Transform: The Future of Medical Education After the COVID-19 Pandemic

Medical education exists to prepare the physician workforce that our nation needs, but the COVID-19 pandemic threatened to disrupt that mission. Likewise, the national increase in awareness of social justice gaps in our country pointed out significant gaps in health care, medicine, and our medical education ecosystem. Crises in all industries often present leaders with no choice but to transform-or to fail. In this perspective, the authors suggest that medical education is at such an inflection point and propose a transformational vision of the medical education ecosystem, followed by a 10-year, 10-point plan that focuses on building the workforce that will achieve that vision. Broad themes include adopting a national vision; enhancing medicine's role in social justice through broadened curricula and a focus on communities; establishing equity in learning and processes related to learning, including wellness in learners, as a baseline; and realizing the promise of competency-based, time-variable training. Ultimately, 2020 can be viewed as a strategic inflection point in medical education if those who lead and regulate it analyze and apply lessons learned from the pandemic and its associated syndemics.

What Does Honors Mean? National Analysis of Medical School Clinical Clerkship Grading

OBJECTIVE

Medical Student Performance Evaluation (MSPE) letters provide critical comparative information about clerkship performance, and are a crucial part of the surgical residency application. The elimination of USMLE Step 1 numeric reporting increases the importance of transparency, standardization, and accessibility of comparative information reported on the MSPE. The objective of our study was to measure the variability in clerkship grade reporting on the MSPE from US medical schools, particularly focusing on the highest (honors) grades.

DESIGN, SETTING, AND PARTICIPANTS

We identified representative MSPE letters from US medical schools and recorded the percentage of honors for 5 core clerkships. We grouped medical schools according to medical school rankings, geographic region, and number of grading categories RESULTS: Of 122 medical schools, 106 schools (87%) reported their grading scheme and percent honors. The most commonly used grading scheme was a 4-tier system (51/122; 42%). The percentage of honors was highly variable (from 1-91%) and did not vary by region. However, schools in the top 20 research ranking were less likely to report grade comparisons (30% vs. 10%), and more likely to award more students honors in 4 of the 5 clerkships. Schools in the top 20 primary care ranking were more likely to award more honors in the medicine clerkship.

CONCLUSIONS

There is significant variability in the number of grading tiers used and the percentage of students awarded honors across US medical schools. Factors that correlated to higher grades included schools with higher rankings, and higher ranked schools were less likely to report comparative information at all.

Towards equitable learning environments for medical education: Bias and the intersection of social identities

CONTEXT

Medical educators are increasingly paying attention to how bias creates inequities that affect learners across the medical education continuum. Such bias arises from learners' social identities. However, studies examining bias and social identities in medical education tend to focus on one identity at a time, even though multiple identities often interact to shape individuals' experiences.

METHODS

This article examines prior studies on bias and social identity in medical education, focusing on three social identities that commonly elicit bias: race, gender and profession. By applying the lens of intersectionality, we aimed to generate new insights into intergroup relations and identify strategies that may be employed to mitigate bias and inequities across all social identities.

RESULTS

Although different social identities can be more or less salient at different stages of medical training, they intersect and impact learners' experiences. Bias towards racial and gender identities affect learners' ability to reach different stages of medical education and influence the specialties they train in. Bias also makes it difficult for learners to develop their professional identities as they are not perceived as legitimate members of their professional groups, which influences interprofessional relations. To mitigate bias across all identities, three main sets of strategies can be adopted. These strategies include equipping individuals with skills to reflect upon their own and others' social identities; fostering in-group cohesion in ways that recognise intersecting social identities and challenges stereotypes through mentorship; and addressing intergroup boundaries through promotion of allyship, team reflexivity and conflict management.

CONCLUSIONS

Examining how different social identities intersect and lead to bias and inequities in medical education provides insights into ways to address these problems. This article proposes a vision for how existing strategies to mitigate bias towards different social identities may be combined to embrace intersectionality and develop equitable learning environments for all.

While You Don’t See Color, I See Bias: Identifying Barriers in Access to Graduate Medical Education Training

There is a need to acknowledge and address issues of implicit and explicit bias within medical education. These biases can impact standardized test questions and scores, evaluations of clinical performance, and subsequent letters of recommendation, all of which can affect the selection of diverse candidates advancing through medical training. Biased behavior toward trainees can negatively impact their learning environment and career trajectory. This article outlines key definitions related to bias and discusses the ways in which bias potentially impacts selection and entry into Pulmonary and Critical Care Medicine fellowship training. Finally, we will describe some ways to mitigate bias within the fellowship selection process and training programs.

The best mirror of the students' longitudinal performance: Portfolio or structured oral exam assessment at clerkship?

OBJECTIVE

This study aimed to compare the strength of association (i.e., explained variability) of the cumulative grade point average (GPA) with the grades obtained in the clerkship portfolio and the final structured oral exam by dental students.

METHODS

A prospective longitudinal study was designed to analyze quantitative data from three cohorts of dental school students. Univariate and multivariate linear regression models were built to evaluate the association between the students' cumulative GPA with the grades obtained in their clerkship portfolio and the final structured oral exam.

RESULTS

In total, 171 students in the last year of the undergraduate program were considered (76% women, age average 24.8 ± 1.6 years). The dental students' grades of both portfolio and structured oral exam were significantly associated with the GPA score but with different strengths of association. The clerkship portfolio was more strongly associated with cumulative GPA than the structured oral exam (R²  = 19.6% versus R²  = 7.6%). On the opposite, the association between the structured oral exam and GPA can be interpreted as a lower precision in its practical significance and thus reflecting different concurrent validity.

CONCLUSIONS

Considering the results of this study, it could probably incline the balance toward the portfolio because it may be closer to a programmatic assessment model, with timely feedback, development of metacognition, and the achievement of formative process measurement rather than evidence of a single instance of examination.

No One Size Fits All: A Qualitative Study of Clerkship Medical Students' Perceptions of Ideal Supervisor Responses to Microaggressions

PURPOSE

This study explores medical students' perspectives on the key features of ideal supervisor responses to microaggressions targeting clerkship medical students.

METHOD

This single-institution, qualitative focus group study, based in an interpretivist paradigm, explored clerkship medical students' perceptions in the United States, 2020. During semistructured focus groups, participants discussed 4 microaggression scenarios. The authors employed the framework method of thematic analysis to identify considerations and characteristics of ideal supervisor responses and explored differences in ideal response across microaggression types.

RESULTS

Thirty-nine students participated in 7 focus groups, lasting 80 to 92 minutes per group. Overall, students felt that supervisors' responsibility began before a microaggression occurred, through anticipatory discussions ("pre-brief") with all students to identify preferences. Students felt that effective bystander responses should acknowledge student preferences, patient context, interpersonal dynamics in the room, and the microaggression itself. Microassaults necessitated an immediate response. After a microaggression, students preferred a brief one-on-one check-in with the supervisor to discuss the most supportive next steps including whether further group discussion would be helpful.

CONCLUSIONS

Students described that an ideal supervisor bystander response incorporates both student preferences and the microaggression context, which are best revealed through advanced discussion. The authors created the Bystander Microaggression Intervention Guide as a visual representation of the preferred bystander microaggression response based on students' discussions. Effective interventions promote educational safety and shift power dynamics to empower the student target.

Diversity Is Not Enough: Advancing a Framework for Antiracism in Medical Education

Medical students, residents, and faculty have begun to examine and grapple with the legacy and persistence of structural racism in academic medicine in the United States. Until recently, the discourse and solutions have largely focused on augmenting diversity across the medical education continuum through increased numbers of learners from groups underrepresented in medicine (UIM). Despite deliberate measures implemented by medical schools, residency programs, academic institutions, and national organizations, meaningful growth in diversity has not been attained. To the contrary, the UIM representation among medical trainees has declined or remained below the representation in the general population. Inequities continue to be observed in multiple domains of medical education, including grading, admission to honor societies, and extracurricular obligations. These inequities, alongside learners' experiences and calls for action, led the authors to conclude that augmenting diversity is necessary but insufficient to achieve equity in the learning environment. In this article, the authors advance a 4-step framework, built on established principles and practices of antiracism, to dismantle structural racism in medical education. They ground each step of the framework in the concepts and skills familiar to medical educators. By drawing parallels with clinical reasoning, medical error, continuous quality improvement, the growth mindset, and adaptive expertise, the authors show how learners, faculty, and academic leaders can implement the framework's 4 steps-see, name, understand, and act-to shift the paradigm from a goal of diversity to a stance of antiracism in medical education.

The use of EPA assessments in decision-making: Do supervision ratings correlate with other measures of clinical performance?

BACKGROUND

Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression.

METHODS

Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX).

RESULTS

Both overall clinical evaluation items score (rho 0.40; n = 166) and CPX patient encounter domain score (rho 0.31; n = 149) showed significant correlation with students' overall mean EPA supervision rating during the clerkship year. There was significant correlation between mean supervision rating for EPA assessments of history, exam, note, and oral presentation skills with scores for these skills on clerkship evaluations; less so on the CPX.

CONCLUSIONS

Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

Making it fair: Learners' and assessors' perspectives of the attributes of fair judgement

INTRODUCTION

Optimising the use of subjective human judgement in assessment requires understanding what makes judgement fair. Whilst fairness cannot be simplistically defined, the underpinnings of fair judgement within the literature have been previously combined to create a theoretically-constructed conceptual model. However understanding assessors' and learners' perceptions of what is fair human judgement is also necessary. The aim of this study is to explore assessors' and learners' perceptions of fair human judgement, and to compare these to the conceptual model.

METHODS

A thematic analysis approach was used. A purposive sample of twelve assessors and eight post-graduate trainees undertook semi-structured interviews using vignettes. Themes were identified using the process of constant comparison. Collection, analysis and coding of the data occurred simultaneously in an iterative manner until saturation was reached.

RESULTS

This study supported the literature-derived conceptual model suggesting fairness is a multi-dimensional construct with components at individual, system and environmental levels. At an individual level, contextual, longitudinally-collected evidence, which is supported by narrative, and falls within ill-defined boundaries is essential for fair judgement. Assessor agility and expertise are needed to interpret and interrogate evidence, identify boundaries and provide narrative feedback to allow for improvement. At a system level, factors such as multiple opportunities to demonstrate competence and improvement, multiple assessors to allow for different perspectives to be triangulated, and documentation are needed for fair judgement. These system features can be optimized through procedural fairness. Finally, appropriate learning and working environments which considers patient needs and learners personal circumstances are needed for fair judgments.

DISCUSSION

This study builds on the theory-derived conceptual model demonstrating the components of fair judgement can be explicitly articulated whilst embracing the complexity and contextual nature of health-professions assessment. Thus it provides a narrative to support dialogue between learner, assessor and institutions about ensuring fair judgements in assessment.

Growth mindset in competency-based medical education

The ongoing adoption of competency-based medical education (CBME) across health professions training draws focus to learner-centred educational design and the importance of fostering a growth mindset in learners, teachers, and educational programs. An emerging body of literature addresses the instructional practices and features of learning environments that foster the skills and strategies necessary for trainees to be partners in their own learning and progression to competence and to develop skills for lifelong learning. Aligned with this emerging area is an interest in Dweck's self theory and the concept of the growth mindset. The growth mindset is an implicit belief held by an individual that intelligence and abilities are changeable, rather than fixed and immutable. In this paper, we present an overview of the growth mindset and how it aligns with the goals of CBME. We describe the challenges associated with shifting away from the fixed mindset of most traditional medical education assumptions and practices and discuss potential solutions and strategies at the individual, relational, and systems levels. Finally, we present future directions for research to better understand the growth mindset in the context of CBME.