Josiah Macy Jr. Foundation Conference on Ensuring Fairness in Medical Education Assessment: Conference Recommendations Report

Factors associated with performance on the emergency medicine qualifying examination

Background

Board certification standards exceed the baseline requirements for state licensure, assuring the public that specialists demonstrate additional clinical skills, knowledge, and professional behavior to provide safe and high-quality specialty care. The objective of this study was to determine what physician factors (e.g., gender, age, race/ethnicity, medical school training, and other factors) were associated with the American Board of Emergency Medicine qualifying examination (QE) performance and pass rates.

Methods

This was a national retrospective, observational, cross-sectional study exploring factors associated with the QE. Subjects were physicians who graduated from accredited emergency medicine (EM) categorical residency programs who took the QE for the first time in 2017, 2018, or 2019. A series of multilevel models was used to examine if physician characteristics were related to score and passing the QE. Relative risks were calculated.

Results

There were 6174 EM graduates, 2118 of whom were women and 588 who were underrepresented in medicine (URiM) physicians. Controlling for other variables, in-training examination (ITE) scores and medical doctor (MD) degrees were positively related to mean QE scores, whereas age, 3-year programs, URiM, and male gender were negatively related to mean QE scores. The QE pass rate was 94%, 95% for non-URiM and 86% for URiM. Results of the full model indicated ITE score, age, URiM, gender, MD degree, and residency program format were significantly related to performance on the QE. In the final model, passing the QE was positively related to ITE scores and negatively related to age and URiM. After other variables were controlled for, the adjusted risk ratio for URiM was 0.94.

Conclusions

Several factors were associated with decreased pass rates on the ABEM QE including ITE scores, older age, and URiM, although the risk ratios were small.

Advancing Equity in Assessment in the Surgical Learning Environment

IMPORTANCE

Assessment is a fundamental part of teaching and learning that provides the basis for making inferences about the development of learners. Inequity in assessment disproportionately impacts underrepresented in medicine learners and can limit their opportunities for achievement.

OBSERVATIONS

Unfortunately, inequity in assessment is prevalent in the surgical learning environment due to systemic and individual factors. The Antideficit Achievement framework can be effectively applied to address sources of inequity.

CONCLUSIONS AND RELEVANCE

This paper explores sources of inequity in assessment in the surgical learning environment and illuminates them with a descriptive case study. Recommendations created using the Antideficit Achievement framework provide effective, practical ways to begin to advance equity in assessment in the surgical learning environment.

How Might we Build an Equitable Future? Design Justice, a Counternarrative to Dominant Approaches in Medical Education

PHENOMENON

Marginalized individuals in medicine face many structural inequities which can have enduring consequences on their progress. Therefore, inequity must be addressed by dismantling underlying unjust policies, environments, and curricula. However, once these injustices have been taken apart, how do we build more just systems from the rubble? Many current strategies to address this question have foundational values of urgency, solutionism, and top-down leadership.

APPROACH

This paper explores a counternarrative: Design Justice. As a set of guiding principles, Design Justice centers the experiences and perspectives of marginalized individuals and communities. These principles include mutual accountability and transparency, co-ownership, and community-led outcomes, and honoring local, traditional, Indigenous knowledge.

FINDINGS

Rooted in critical scholarship and critical design, Design Justice recognizes the interconnectedness of various forms of marginalization and works to critically examine power dynamics that exist in every design process. These co-created principles act as practical guardrails, directing progress toward justice.

INSIGHTS

This paper begins with an overview of Design Justice's history in critical scholarship and critical design, providing foundational background knowledge for medical educators, scholars, and leaders in key concepts of justice and design. We explore how the Design Justice principles were developed and have been applied across sectors, highlighting its applications, including education applications. Finally, we raise critical questions about medical education prompted by Design Justice.

Validity in the Next Era of Assessment: Consequences, Social Impact, and Equity

Validity has long held a venerated place in education, leading some authors to refer to it as the "sine qua non" or "cardinal virtue" of assessment. And yet, validity has not held a fixed meaning; rather it has shifted in its definition and scope over time. In this Eye Opener, the authors explore if and how current conceptualizations of validity fit a next era of assessment that prioritizes patient care and learner equity. They posit that health profession education's conceptualization of validity will change in three related but distinct ways. First, consequences of assessment decisions will play a central role in validity arguments. Second, validity evidence regarding impacts of assessment on patients and society will be prioritized. Third, equity will be seen as part of validity rather than an unrelated concept. The authors argue that health professions education has the agency to change its ideology around validity, and to align with values that predominate the next era of assessment such as high-quality care and equity for learners and patients.