In Pursuit of Honors: A Multi-Institutional Study of Students' Perceptions of Clerkship Evaluation and Grading

Hopes, fears, and rumors: Medical students and the general surgery clerkship

BACKGROUND

Surgical educators have worked to manage the hopes and fears as well as the recurring rumors that plague the surgical clerkship. It is not known if this has effected change over time.

METHODS

We gathered information on hopes, fears, and rumors during our clerkship orientations from 2017 to 2019 using anonymous polling software with real-time feedback. We analyzed 468 responses using qualitative content analysis.

RESULTS

Students hoped for practical skills acquisition, self-improvement, and understanding the surgical profession. They feared lack of time and knowledge, burnout, mistreatment, and subjective evaluation. Rumors included negative perceptions of surgical culture work environment, and fear of mistreatment despite clerkship changes intended to allay these fears.

CONCLUSION

Students starting surgery clerkships hope to gain surgical and clinical skills but concerns about surgical culture and mistreatment appear to remain unchanged despite structural improvements in the clerkship experience. Surgeons should look beyond the clerkship itself to change these perceptions.

From Grading to Assessment for Learning: A Qualitative Study of Student Perceptions Surrounding Elimination of Core Clerkship Grades and Enhanced Formative Feedback

Problem: Medical students perceive honors grading during core clerkships as unfair and inequitable, and negatively impacting their learning and wellbeing. Eliminating honors grading, a powerful extrinsic motivator, and emphasizing formative feedback may address these problems and promote intrinsic motivation and learning. However, it is unknown how transitioning from honors to pass/fail grading with enhanced formative feedback in the core clerkship year may affect student learning experiences, wellbeing, and perceptions of the learning environment. Intervention: Core clerkship grading was transitioned from honors/pass/fail to pass/fail at one US medical school. In addition, the requirement for students to obtain formative supervisor feedback was formalized to twice per week. Context: This qualitative study utilized semi-structured interviews to explore the perceptions among core clerkship students of learning and assessment. Interview questions addressed motivation, wellbeing, learning behaviors, team dynamics, feedback, and student and supervisor attitudes regarding assessment changes. The authors analyzed data inductively using thematic analysis informed by sensitizing concepts related to theories of motivation (goal orientation theory and self-determination theory). Impact: Eighteen students participated, including five with experience in both honors-eligible and pass/fail clerkships. The authors identified three major themes in students' descriptions of the change in approach to assessment: student engagement in clerkships, wellbeing, and recognition of learning context. Student engagement subthemes included intrinsic motivation for patient care rather than performing; sense of agency over learning, including ability to set learning priorities, seek and receive feedback, take learning risks, and disagree with supervisors, and collaborative relationships with peers and team members. Positive wellbeing was characterized by low stress, sense of authenticity with team members, prioritized physical health, and attention to personal life. Learning context subthemes included recognition of variability of clerkship contexts with pass/fail grading mitigating fairness and equity concerns, support of the grading change from residents and some attendings, and implications surrounding future stress and residency selection. Lessons Learned: Students perceive a transition from honors grading to pass/fail with increased feedback as supporting their engagement in learning, intrinsic motivation, and wellbeing. Drivers of wellbeing appear to include students' feelings of control, achieved through the ability to seek learning opportunities, teaching, and constructive feedback without the perceived need to focus on impressing others. Ongoing evaluation of the consequences of this shift in assessment is needed.

Student-Led Efforts to Advance Anti-Racist Medical Education

Over the past decade, medical schools across the United States have increasingly dedicated resources to advancing racial and social justice, such as by supporting diversity and inclusion efforts and by incorporating social medicine into the traditional medical curricula. While these changes are promising, the academic medicine community must apply an anti-racist lens to every aspect of medical education to equip trainees to recognize and address structural inequities. Notably, organizing and scholarly work led by medical students has been critical in advancing anti-racist curricula. In this article, the authors illustrate how student activism has reshaped medical education by highlighting examples of student-led efforts to advance anti-racist curricula at Harvard Medical School (HMS) and at the University of California, San Francisco (UCSF) School of Medicine. HMS students collaborated with faculty to address aspects of existing clinical practice that perpetuate racism, such as the racial correction factor in determining kidney function. They also responded to the existing curricula by noting missed opportunities to discuss structural racism, and they planned supplemental sessions to address these gaps. At UCSF, students identified specific avenues to improve the rigor of social medicine courses and developed new curricula to equip students with skills to confront and work to dismantle racism. The authors describe how HMS students, in an effort to improve the learning environment, developed a workshop to assist students in navigating microaggressions and discrimination in the clinical setting. At UCSF, students partnered with faculty and administration to advocate pass/fail grading for clerkships after university data revealed racial disparities in students' clerkship assessments. In reviewing these examples of students' advocacy to improve their own curricula and learning environments, the authors aim to provide support for students and faculty pursuing anti-racist curricular changes at their own institutions.

Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a 'problem' to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back and changing perspectives to focus on the fundamental underlying value of fairness in assessment may help re-set the traditional objective approach and provide a more relevant way to determine the appropriateness of subjective human judgements. Changing focus to look at what is 'fair' human judgement in assessment, rather than what is 'objective' human judgement in assessment allows for the embracing of many different perspectives, and the legitimising of human judgement in assessment. However, this requires addressing the question: what makes human judgements fair in health professions assessment? This is not a straightforward question with a single unambiguously 'correct' answer. In this hermeneutic literature review we aimed to produce a scholarly knowledge synthesis and understanding of the factors, definitions and key questions associated with fairness in human judgement in assessment and a resulting conceptual framework, with a view to informing ongoing further research. The complex construct of fair human judgement could be conceptualised through values (credibility, fitness for purpose, transparency and defensibility) which are upheld at an individual level by characteristics of fair human judgement (narrative, boundaries, expertise, agility and evidence) and at a systems level by procedures (procedural fairness, documentation, multiple opportunities, multiple assessors, validity evidence) which help translate fairness in human judgement from concepts into practical components.

A Qualitative Exploration of the Intern Experience in Assessing Medical Student Performance

BACKGROUND

Interns play a key role in medical student education, often observing behaviors that others do not. Their role in assessment, however, is less clear. Despite accreditation standards pertaining to residents' assessment skills, they receive little guidance or formal training in it. In order to better prepare residents for their role in medical student assessment, we need to understand their current experience.

OBJECTIVE

We aimed to describe the first-year resident experience assessing students' performance and providing input to faculty for student clinical performance assessments and grades in the inpatient setting.

METHODS

Pediatric interns at Children's National Hospital (CN) from February 2018 to February 2019 were invited to participate in semistructured interviews about their experience assessing students. Constant comparative methodology was used to develop themes. Ten interviews were conducted, at which point thematic saturation was reached.

RESULTS

We identified 4 major themes: 1) Interns feel as though they assess students in meaningful, unique ways. 2) Interns encounter multiple barriers and facilitators to assessing students. 3) Interns voice varying levels of comfort and motivation assessing different areas of student work. 4) Interns see their role in assessment limited to formative rather than summative assessment.

CONCLUSIONS

These findings depict the intern experience with assessment of medical students at a large pediatric residency program and can help inform ways to develop and utilize the assessment skills of interns.

A Model for Exploring Compatibility Between Applicants and Residency Programs: Right Resident, Right Program

Holistic review of residency applications is touted as the gold standard for selection, yet vast application numbers leave programs reliant on screening using filters such as United States Medical Licensing Examination scores that do not reliably predict resident performance and may threaten diversity. Applicants struggle to identify which programs to apply to, and devote attention to these processes throughout most of the fourth year, distracting from their clinical education. In this perspective, educators across the undergraduate and graduate medical education continuum propose new models for student-program compatibility based on design thinking sessions with stakeholders in obstetrics and gynecology education from a broad range of training environments. First, we describe a framework for applicant-program compatibility based on applicant priorities and program offerings, including clinical training, academic training, practice setting, residency culture, personal life, and professional goals. Second, a conceptual model for applicant screening based on metrics, experiences, attributes, and alignment with program priorities is presented that might facilitate holistic review. We call for design and validation of novel metrics, such as situational judgment tests for professionalism. Together, these steps could improve the transparency, efficiency and fidelity of the residency application process. The models presented can be adapted to the priorities and values of other specialties.

Medical Students' Views on Implementing the Core EPAs: Recommendations From Student Leaders at the Core EPAs Pilot Institutions

In 2014, the Association of American Medical Colleges recruited 10 institutions across the United States to pilot the 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs). The goal was to establish a competency-based framework to prepare graduating medical students for the transition to residency. Within the Core EPAs pilot, medical students play an influential role in the development and implementation of EPA-related curricula. Student engagement was a priority for the Core EPAs institutions given students' roles as the end users of the curriculum, thus they may offer valuable insight into its design and implementation. Here, the authors provide the perspective of medical students who serve as leaders in the Core EPAs pilot at their respective institutions. They describe student leadership models across the pilot institutions as well as 6 key challenges to implementation of the Core EPAs: (1) How and when should the Core EPAs be introduced? (2) Who is responsible for driving the assessment process? (3) What feedback mechanisms are required? (4) What systems are required for advising, mentoring, or coaching students? (5) Should EPA performance contribute to students' grades? and (6) Should entrustment decisions be tied to graduation requirements? Using a polarity management framework to address each challenge, the authors describe inherent tensions, approaches used by the Core EPAs pilot institutions, and student-centered recommendations for resolving each tension. By sharing the experiences and perspectives of students engaged in the Core EPAs pilot, the authors hope to inform implementation of EPA-oriented assessment practices and feedback across institutions in the United States.

Determining Grades in the Internal Medicine Clerkship: Results of a National Survey of Clerkship Directors

PURPOSE

Trust in and comparability of assessments are essential in clerkships in undergraduate medical education for many reasons, including ensuring competency in clinical skills and application of knowledge important for the transition to residency and throughout students' careers. The authors examined how assessments are used to determine internal medicine (IM) core clerkship grades across U.S. medical schools.

METHODS

A multisection web-based survey of core IM clerkship directors at 134 U.S. medical schools with membership in the Clerkship Directors in Internal Medicine was conducted in October through November 2018. The survey included a section on assessment practices to characterize current grading scales used, who determines students' final clerkship grades, the nature/type of summative assessments, and how assessments are weighted. Respondents were asked about perceptions of the influence of the National Board of Medical Examiners (NBME) Medicine Subject Examination (MSE) on students' priorities during the clerkship.

RESULTS

The response rate was 82.1% (110/134). There was considerable variability in the summative assessments and their weighting in determining final grades. The NBME MSE (91.8%), clinical performance (90.9%), professionalism (70.9%), and written notes (60.0%) were the most commonly used assessments. Clinical performance assessments and the NBME MSE accounted for the largest percentage of the total grade (on average 52.8% and 23.5%, respectively). Eighty-seven percent of respondents were concerned that students' focus on the NBME MSE performance detracted from patient care learning.

CONCLUSIONS

There was considerable variability in what IM clerkships assessed and how those assessments were translated into grades. The NBME MSE was a major contributor to the final grade despite concerns about the impact on patient care learning. These findings underscore the difficulty in comparing learners across institutions and serve to advance discussions for how to improve accuracy and comparability of grading in the clinical environment.

Clerkship Grading and the U.S. Economy: What Medical Education Can Learn From America's Economic History

Clerkship grades (like money) are a social construct that function as the currency through which value exchanges in medical education are negotiated between the system's various stakeholders. They provide a widely recognizable and efficient medium through which learner development can be assessed, tracked, compared, and demonstrated and are commonly used to make decisions regarding progression, distinction, and selection for residency. However, substantial literature has demonstrated how grades imprecisely and unreliably reflect the value of learners. In this article, the authors suggest that challenges with clerkship grades are fundamentally tied to their role as currency in the medical education system. Associations are drawn between clerkship grades and the history of the U.S. economy; 2 major concepts are highlighted: regulation and stock prices. The authors describe the history of these economic concepts and how they relate to challenges in clerkship grading. Using lessons learned from the history of the U.S. economy, the authors then propose a 2-step solution to improve upon grading for future generations of medical students: (1) transition from grades to a federally regulated competency-based assessment model and (2) development of a departmental competency letter that incorporates competency-based assessments rather than letter grades and meets the needs of program directors.

"There Is a Lot of Change Afoot": A Qualitative Study of Faculty Adaptation to Elimination of Tiered Grades With Increased Emphasis on Feedback in Core Clerkships

PURPOSE

Medical schools responding to challenges with fairness, equity, learning environments, and student wellness have reconsidered clerkship grades. This study used the Concerns-Based Adoption Model (CBAM) to explore how faculty responded to a change in the approach to assessment from focus on grades toward focus on feedback.

METHOD

This qualitative study used an inductive approach to analyze data from semistructured interviews with teaching faculty and education leaders at University of California, San Francisco, School of Medicine 6 months following the elimination of traditional tiered clerkship grades. Participants were recruited in 2019 using purposive sampling. Interview questions addressed participants' perceptions of the new approach to assessment and grading that emphasizes assessment for learning, including advantages and concerns. The authors analyzed data using thematic analysis informed by sensitizing concepts from CBAM.

RESULTS

Nineteen faculty participated (11 medicine, 8 surgery). Faculty expressed optimism about the impact of the change on their clinical educator roles and sought faculty development to enhance feedback and assessment skills. Perceived benefits to students addressed learning and wellness, with concern for students' motivation and professional development. Faculty shared uncertainty about their roles and larger systemic consequences involving the residency match and institutional reputation. Participants acknowledged that traditional grading is imperfect, yet felt a departure from this system represented significant culture change.

CONCLUSIONS

Faculty perceptions of the assessment and grade change in 2 large departments demonstrated tensions around grading. Their descriptions of change as an ongoing process aligned with CBAM and required them to consider new approaches and perspectives. While discourse about assessment and grading frequently focuses on learners, these findings support institutional consideration of the impact of changes in assessment on teaching faculty.

Standardization in the MSPE: Key Tensions for Learners, Schools, and Residency Programs

The Medical Student Performance Evaluation (MSPE), which summarizes a medical student's academic and professional undergraduate medical education performance and provides salient information during the residency selection process, faces persistent criticisms regarding heterogeneity and obscurity. Specifically, MSPEs do not always provide the same type or amount of information about students, especially from diverse schools, and important information is not always easy to find or interpret. To address these concerns, a key guiding principle from the Recommendations for Revising the MSPE Task Force of the Association of American Medical Colleges (AAMC) was to achieve "a level of standardization and transparency that facilitates the residency selection process." Benefits of standardizing the MSPE format include clarification of performance benchmarks or metrics, consistency across schools to enhance readability, and improved quality. In medical education, standardization may be an important mechanism to ensure accountability of the system for all learners, including those with varied backgrounds and socioeconomic resources. In this article, members of the aforementioned AAMC MSPE task force explore 5 tensions inherent in the pursuit of standardizing the MSPE: (1) presenting each student's individual characteristics and strengths in a way that is relevant, while also working with a standard format and providing standard content; (2) showcasing school-specific curricular strengths while also demonstrating standard evidence of readiness for internship; (3) defining and achieving the right amount of standardization so that the MSPE provides useful information, adds value to the residency selection process, and is efficient to read and understand; (4) balancing reporting with advocacy; and (5) maintaining standardization over time, especially given the tendency for the MSPE format and content to drift. Ongoing efforts to promote collaboration and trust across the undergraduate to graduate medical education continuum offer promise to reconcile these tensions and promote successful educational outcomes.

Acting Wisely: Eliminating Negative Bias in Medical Education-Part 1: The Fundamentals

Bias is a ubiquitous problem in human functioning. It has plagued medical decision making, making physicians prone to errors of perception and judgment. Racial, gender, ethnic, and religious negative biases infest physicians' perception and cognition, causing errors of judgment and behavior that are damaging. In Part 1 of this series of 2 papers, the authors address the problem of harmful bias, the science of cognition, and what is known about how bias functions in human perception and information processing. They lay the groundwork for an approach to reducing negative bias through awareness, reflection, and bias mitigation, an approach in which negative biases can be transformed-by education, experience, practice, and relationships-into positive biases toward one another. The authors propose wisdom as a conceptual framework for imagining a different way of educating medical students. They discuss fundamental cognitive, affective, and reflective components of wisdom-based education. They also review the skills of awareness, using debiasing strategies, compassion, fostering positive emotion, and reflection that are inherent to a wisdom-based approach to eliminating the negative effects of bias in medical education. In Part 2, the authors answer a key question: How can medical educators do better? They describe the interpersonal, structural, and cultural elements supportive of a wisdom-based learning environment, a culture of respect and inclusion in medical education.

They Don't See a Lot of People My Color: A Mixed Methods Study of Racial/Ethnic Stereotype Threat Among Medical Students on Core Clerkships

PURPOSE

Stereotype threat is an important psychological phenomenon in which fear of fulfilling negative stereotypes about one's group impairs performance. The effects of stereotype threat in medical education are poorly characterized. This study examined the prevalence of racial/ethnic stereotype threat amongst fourth-year medical students and explored its impact on students' clinical experience.

METHOD

This was an explanatory sequential mixed methods study at 2 institutions in 2019. First, the authors administered the quantitative Stereotype Vulnerability Scale (SVS) to fourth-year medical students. The authors then conducted semistructured interviews among a purposive sample of students with high SVS scores, using a qualitative phenomenographic approach to analyze experiences of stereotype threat. The research team considered reflexivity through group discussion and journaling.

RESULTS

Overall, 52% (184/353) of students responded to the survey. Collectively, 28% of students had high vulnerability to stereotype threat: 82% of Black, 45% of Asian, 43% of Latinx, and 4% of White students. Eighteen students participated in interviews. Stereotype threat was a dynamic, 3-stage process triggered when students experienced the workplace through the colored lens of race/ethnicity by standing out, reliving past experiences, and witnessing microaggressions. Next, students engaged in internal dialogue to navigate racially charged events and workplace power dynamics. These efforts depleted cognitive resources and interfered with learning. Finally, students responded and coped to withstand threats. Immediate and deferred interventions from allies reduced stereotype threat.

CONCLUSIONS

Stereotype threat is common, particularly among non-White students, and interferes with learning. Increased minority representation and developing evidence-based strategies for allyship around microaggressions could mitigate effects of stereotype threat.

Guidelines: The dos, don'ts and don't knows of remediation in medical education

INTRODUCTION

Two developing forces have achieved prominence in medical education: the advent of competency-based assessments and a growing commitment to expand access to medicine for a broader range of learners with a wider array of preparation. Remediation is intended to support all learners to achieve sufficient competence. Therefore, it is timely to provide practical guidelines for remediation in medical education that clarify best practices, practices to avoid, and areas requiring further research, in order to guide work with both individual struggling learners and development of training program policies.

METHODS

Collectively, we generated an initial list of Do's, Don'ts, and Don't Knows for remediation in medical education, which was then iteratively refined through discussions and additional evidence-gathering. The final guidelines were then graded for the strength of the evidence by consensus.

RESULTS

We present 26 guidelines: two groupings of Do's (systems-level interventions and recommendations for individual learners), along with short lists of Don'ts and Don't Knows, and our interpretation of the strength of current evidence for each guideline.

CONCLUSIONS

Remediation is a high-stakes, highly complex process involving learners, faculty, systems, and societal factors. Our synthesis resulted in a list of guidelines that summarize the current state of educational theory and empirical evidence that can improve remediation processes at individual and institutional levels. Important unanswered questions remain; ongoing research can further improve remediation practices to ensure the appropriate support for learners, institutions, and society.

Focal Length Fluidity: Research Questions in Medical Education Research and Scholarship

Research and scholarship in health professions education has been shaped by intended audience (i.e., producers vs users) and the purpose of research questions (i.e., curiosity driven or service oriented), but these archetypal dichotomies do not represent the breadth of scholarship in the field. Akin to an array of lenses required by scientists to capture images of a black hole, the authors propose the analogy of lenses with different focal lengths to consider how different kinds of research questions can offer insight into health professions research-a microscope, a magnifying glass, binoculars, and telescopes allow us to ask and answer different kinds of research questions. They argue for the relevance of all of the different kinds of research questions (or focal lengths); each provides important insight into a particular phenomenon and contributes to understanding that phenomenon in a different way. The authors propose that research questions can move fluidly across focal lengths. For example, a theoretical question can be made more pragmatic through asking "how" questions ("How can we observe and measure a phenomenon?"), whereas a pragmatic question can be made more theoretic by asking a series of "why" questions ("Why are these findings relevant to larger issues?"). In summary, only through the combination of lenses with different focal lengths, brought to bear through interdisciplinary work, can we fully comprehend important phenomena in health professions education and scholarship-the same way scientists managed to image a black hole.