Thinking of selection and widening access as complex and wicked problems

Equity with equality? Contextualising everyone can widen participation in medical school admissions

Widening participation outcomes in admissions to UK medical schools have not changed 2007-2018, partly due to inequity in selection. This study models the effects of changing selection, using a novel method of contextualising applicants, on widening participation. We studied 1084 English school leaver applicants to a single medical school over two years, using data from their public exams taken 2 years pre-application (GCSE) and recent admissions test (UCAT). Widening participation was defined by postcode. We modelled two shortlists for a pre-determined number of 500; one ranked on UCAT total score, and the other on a metric that contextualised applicants' GCSE grades against their schools' average GCSE performance. There was a significant difference in the postcode-defined widening participation characteristics of the two shortlists; 46% by contextualisation and 32.2% by UCAT (Chisquare p < 0.00001). As widening participation covers 42% of postcodes, the "contextualise everyone" method achieves equity. Conventionally, contextual admissions identify individuals belonging to under-represented groups and gives them preferential treatment. Changing the rules for everyone, by using a relative attainment instead of simple absolute attainment metric, benefits from treating applicants equally; and could promote equity through widening participation.

Residency Selection in Otolaryngology: Past, Present, & Future

OBJECTIVES

To examine the otolaryngology residency selection process, including past experiences based on the medical literature and survey analysis of our present practices to generate recommendations for future selection system design.

METHODS

A mixed-methods study, including a scoping review and a cross-sectional survey, was completed. Four databases were assessed for articles on otolaryngology residency selection published from January 1, 2016 through December 31, 2020. A 36-question survey was developed and distributed to 114 otolaryngology program directors. Descriptive and thematic analysis was performed.

RESULTS

Ultimately, 67 of 168 articles underwent data abstraction and assessment. Three themes surfaced during the analysis: effectiveness, efficiency, and equity. Regarding the survey, there were 62 participants (54.4% response rate). The three most important goals for the selection process were: (1) to fit the program culture, (2) to make good colleagues, and (3) to contribute to the program's diversity. The three biggest 'pain points' were as follows: (1) Large volume of applications, (2) Lack of reliable information about personal characteristics, and (3) Lack of reliable information about a genuine interest in the program.

CONCLUSIONS

Within this study, the depth and breadth of the literature on otolaryngology residency selection have been synthesized. Additionally, baseline data on selection practices within our specialty has been captured. With an informed understanding of our past and present, we can look to the future. Built upon the principles of person-environment fit theory, our proposed framework can guide research and policy discussions regarding the design of selection systems in otolaryngology, as we work to achieve more effective, efficient, and equitable outcomes.

LEVEL OF EVIDENCE

N/A Laryngoscope, 133:2929-2941, 2023.

Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153

Selection is the first assessment of medical education and training. Medical schools must select from a pool of academically successful applicants and ensure that the way in which they choose future clinicians is robust, defensible, fair to all who apply and cost-effective. However, there is no comprehensive and evidence-informed guide to help those tasked with setting up or rejuvenating their local selection process. To address this gap, our guide draws on the latest research, international case studies and consideration of common dilemmas to provide practical guidance for designing, implementing and evaluating an effective medical school selection system. We draw on a model from the field of instructional design to frame the many different activities involved in doing so: the ADDIE model. ADDIE provides a systematic framework of Analysis (of the outcomes to be achieved by the selection process, and the barriers and facilitators to achieving these), Design (what tools and content are needed so the goals of selection are achieved), Development (what materials and resources are needed and available), Implementation (plan [including piloting], do study and adjust) and Evaluation (quality assurance is embedded throughout but the last step involves extensive evaluation of the entire process and its outcomes).HIGHLIGHTSRobust, defensible and fair selection into medical school is essential. This guide systematically covers the processes required to achieve this, from needs analysis through design, development and implementation, to evaluation of the success of a selection process.

Impact of medical students' socioeconomic backgrounds on medical school application, admission and migration in Japan: a web-based survey

OBJECTIVES

The aim of this study was to clarify and compare the socioeconomic backgrounds of medical students with those of other health professions and non-health faculty students in an era of increasing inequity in Japanese society.

DESIGN

This was a quantitative nationwide study. Data were collected by a cross-sectional web-based anonymous questionnaire.

SETTING

Data from years 3 and 4 medical, health professions and non-health faculty students across Japan were collected in 2021.

PARTICIPANTS

Participants were 1991 students from medical schools, 224 from dental, 419 from pharmacy, 326 from nursing, 144 from other health professions and 207 from non-health faculties.

RESULTS

The proportion of high-income families (>18 million yen: ca. US$140 000) among medical students was 25.6%, higher than that of pharmacy (8.7%) and nursing students (4.1%) (p<0.01). One-third of medical students had a physician parent, more common than in non-medical students (p<0.01). Students who only applied to public medical schools and a regional quota 'Chiiki-waku' students with scholarship had lower family income and physician parents compared with those who applied to private medical schools (p<0.01), but they still had higher physician parents compared with non-medical students (p<0.01). Logistic regression revealed that having a physician parent (p<0.01), aspiring to the present profession during elementary school (p<0.01) and private upper secondary school graduation (p<0.01) predicted the likelihood of studying medicine. There were regional differences of backgrounds among medical students, and 80% of medical students with urban backgrounds intended to work in urban localities after graduation.

CONCLUSIONS

This study provides evidence that medical students in Japan hail from urban and higher income classes and physicians' families. This finding has implications for the health workforce maldistribution in Japan. Widening the diversity of medical students is essential for solving physician workforce issues and meeting broad healthcare needs.

Does medical education reform change who is selected? A national cross-sectional survey from China

OBJECTIVES

Approximately 10 years ago, China introduced an education plan to improve the overall quality of medical education and to better serve the population's health needs. Many medical schools were then recognised and financed by China's Ministry of Education to develop and operationalise new pilot programmes (PPs) aligned with this plan. These ran in parallel with the traditional programmes (TPs). One way to achieve the plan's first aim, improving the quality of medical education, is to select academically stronger candidates. We, thus, examined and compared who were selected into PPs and TPs.

DESIGN

Cross-sectional study.

SETTING

Data were collected from 123 medical schools across China via the 2021 China Medical Student Survey.

PARTICIPANTS

Participants were undergraduate clinical medicine students across all year groups.

PRIMARY AND SECONDARY OUTCOME MEASURES

Medical school selection was via the National College Entrance Examination (NCEE). Medical students' NCEE performance and their sociodemographics were used as the primary and secondary outcome measures. Mann-Whitney or χ2 tests were used to compare the means between educational programmes (PPs vs TPs) and various selection outcomes. Multilevel mixed-effects regressions were employed to account for school idiosyncratic selection results.

RESULTS

Of the 204 817 respondents, 194 163 (94.8%) were in a TP and 10 654 (5.2%) a PP. PP respondents (median=75.2, IQR=69.5-78.8) had significantly higher NCEE scores than their TP counterparts (median=73.9, IQR=68.5-78.7). Holding constant their NCEE score, PP respondents were significantly more likely to come from urban areas, not be first-generation college students, and have parents with higher occupational status and income.

CONCLUSIONS

Assuming quality can be indicated by prior academic achievement at the point of selection, PPs achieved this mission. However, doing so limited medical students' diversity. This may be unhelpful in achieving the Education Plan's goal to better serve China's health needs.

Examining the nexus between medical education and complexity: a systematic review to inform practice and research

BACKGROUND

Medical education is a multifarious endeavour integrating a range of pedagogies and philosophies. Complexity as a science or theory ('complexity') signals a move away from a reductionist paradigm to one which appreciates that interactions in multi-component systems, such as healthcare systems, can result in adaptive and emergent outcomes. This examination of the nexus between medical education and complexity theory aims to discover ways that complexity theory can inform medical education and medical education research.

METHODS

A structured literature review was conducted to examine the nexus between medical education and complexity; 5 databases were searched using relevant terms. Papers were included if they engaged fully with complexity as a science or theory and were significantly focused on medical education. All types of papers were included, including conceptual papers (e.g. opinion and theoretical discussions), case studies, program evaluations and empirical research. A narrative and thematic synthesis was undertaken to create a deep understanding of the use of complexity in medical education.

RESULTS

Eighty-three papers were included; the majority were conceptual papers. The context and theoretical underpinnings of complexity as a relevant theory for medical education were identified. Bibliographic and temporal observations were noted regarding the entry of complexity into medical education. Complexity was relied upon as a theoretical framework for empirical studies covering a variety of elements within medical education including: knowledge and learning theories; curricular, program and faculty development; program evaluation and medical education research; assessment and admissions; professionalism and leadership; and learning for systems, about systems and in systems.

DISCUSSION

There is a call for greater use of theory by medical educators. Complexity within medical education is established, although not widespread. Individualistic cultures of medicine and comfort with reductionist epistemologies challenges its introduction. However, complexity was found to be a useful theory across a range of areas by a limited number of authors and is increasingly used by medical educators and medical education researchers. This review has further conceptualized how complexity is being used to support medical education and medical education research.

CONCLUSION

This literature review can assist in understanding how complexity can be useful in medical educationalists' practice.

What is the evidence for biology as the 'heart of eligibility' to study medicine? A retrospective analysis

The first, sometimes the only, selection tool for entry into undergraduate medicine is prior educational attainment (PEA). This is often further specified to include certain subjects, for example, biology is a prerequisite for entry into medicine in many Asian countries. However, there seems no clear evidence base for this prerequisite. Our aim, therefore, was to carry out a retrospective quantitative study comparing the performances of five cohorts of students (2015-2019 entry; n = 588) with and without biology PEA in Years 1 and 2 Bachelor of Medicine and Bachelor of Surgery (MBBS) integrated written assessments (n = 3) and anatomy practical examinations (APE) (n = 5). The study was conducted at one of Singapore's three medical schools. Data were analyzed using independent t-tests and Mann-Whitney U with p values of less than 0.05 were considered significant. There were no significant differences in performance on any Years 1 or 2 integrated written assessments. Similarly, in one of the APE, a significant difference was found for one cohort (academic year [AY] 2015-2016) out of five assessments. These results suggest that having a prior biology qualification does not make a difference in assessment performance in the early years of medical school. This information may help stakeholders and admissions committees decide whether biology is required for medical school entrance.

Violence by Burning Against Women and Girls: An Integrative Review

Violence against women and girls by burning is a serious and confronting form of gender-based violence. Often, perpetrators aim to disfigure their victims or cause great pain, rather than kill them. Little is known about the characteristics of females who are subjected to violence by burning. This study aimed to review the literature concerning the prevalence, demographic profile, injury event, contributing factors and health outcomes for women and girls who have experienced burn-related violence. A search across five databases (PubMed, CINAHL, PsycINFO, Scopus and LILACS) was conducted up to April 2021 to identify original peer-review research, with a focus on violence by burning against women and girls. The review was guided by the five-stage approach to integrative reviews developed by Whittemore and Knafl (2005). Fifteen studies were identified. Victims were predominantly married, with low socio-economic status, limited education, and high emotional and financial dependency on their partners or families. Burn injuries were mostly caused by flame or acid, with significant morbidity or high mortality. Motives included family/marital issues or property/financial disputes. This review identified the limited evidence available in the peer-reviewed literature related to burn-related violence against women and girls worldwide. Findings suggest the need for further research to provide a clearer understanding of the complex issues involved.

"Important but risky": attitudes of global thought leaders towards cost and value research in health professions education

Studies of cost and value can inform educational decision making, yet our understanding of the barriers to such research is incomplete. To address this gap, our aim was to explore the attitudes of global thought leaders in HPE towards cost and value research. This was a qualitative virtual interview study underpinned by social constructionism. In telephone or videoconference interviews in 2018-2019, we asked global healthcare professional thought leaders their views regarding HPE cost and value research, outstanding research questions in this area and why addressing these questions was important. Analysis was inductive and thematic, and incorporated review and comments from the original interviewees (member checking). We interviewed 11 thought leaders, nine of whom gave later feedback on our data interpretation (member checking). We identified four themes: Cost research is really important but potentially risky (quantifying and reporting costs provides evidence for decision-making but could lead to increased accountability and loss of autonomy); I don't have the knowledge and skills (lack of economic literacy); it's not what I went into education research to do (professional identity); and it's difficult to generate generalizable findings (the importance of context). This study contributes to a wider conversation in the literature about cost and value research by bringing in the views of global HPE thought leaders. Our findings provide insight to inform how best to engage and empower educators and researchers in the processes of asking and answering meaningful, acceptable and relevant cost and value questions in HPE.

BMAT's predictive validity for medical school performance: A retrospective cohort study

BACKGROUND

Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.

METHODS

This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students.

RESULTS

When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades.

DISCUSSION

Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Insights on selection of undergraduate dental students

INTRODUCTION

The goal of selection to dental education was to find those with the greatest aptitude for dentistry. Recently, schools have introduced a variety of tools; however, these have often been adopted without appropriate evaluation regarding existing evidence for fairness, reliability or validity. We explore dental admissions staff beliefs about the quality of different selection tools, with the objective of exploring their decision-making in implementing selection practices.

METHODS

This qualitative study is underpinned by a social constructionist epistemology, in which our principal concern is "explicating the processes by which people come to describe, explain or otherwise account for the world (including themselves) in which they live." We conducted individual interviews with 15 of the 16 UK dental admission leads to elicit their views around admissions processes and aims. Data coding and analysis were initially inductive, using thematic analysis. After the themes emerged, we applied a deductive framework of affordances to group themes and then examined these for heuristics.

RESULTS AND DISCUSSION

We identified three main themes; "Selection Tool Use," "Widening Participation Practices," and "Professionalising the Admission Lead Role." Admission leads spoke favourably of tools that allowed a "holistic" view of the applicants "potential". Selection tools were favoured if they enabled "Gut feeling". Leads spoke of evaluating candidates, making sure they were "rounded", and "know what dentistry is all about." In justifying the use of elements of their procedure, the use of heuristics was prominent.

CONCLUSION

In order to minimise the potential consequences of poor selection decisions, it is important to acknowledge that dental admission leads are at risk of depending on sub-optimal heuristics to make judgements about effective selection (shaped by previous practices) rather than using more rational decision-making processes based on the extant evidence (regarding the quality of different selection tools). Future research may be usefully informed by the knowledge translation literature to offer solutions for improving selection practices in dental education.

From struggle to opportunity: Reimagining medical education in a pandemic era

The COVID-19 pandemic has disrupted the international medical education community in unprecedented ways. The restrictions imposed to control the spread of the virus have upended our routines and forced us to reimagine our work structures, educational programming and delivery of patient care in ways that will likely continue to change how we live and work for the foreseeable future. Yet, despite these interruptions, the pandemic has additionally sparked a transformative impulse in some to actively engage in critical introspection around the future of their work, compelling us to consider what changes could (and perhaps should) occur after the pandemic is over. Drawing on key concepts associated with scholar Paulo Freire's critical pedagogy, this paper serves as a call to action, illuminating the critical imaginings that have come out of this collective moment of struggle and instability, suggesting that we can perhaps create a more just, compassionate world even in the wake of extraordinary hardship.

Meritocratic and fair? The discourse of UK and Australia's widening participation policies

INTRODUCTION

Globally, people with the academic and personal attributes to successfully study medicine experience disadvantages associated with sociodemographic factors. Governments have attempted to address this issue via macrolevel policies aimed at widening participation (WP) to medicine. These policies differ by country, suggesting much can be learned from examining and comparing international policy discourses of WP. Our question was: How are discourses of WP to higher and medical education positioned in the UK and Australia?

METHODS

A systematic search strategy was guided by five a priori themes inspired by United Nations Sustainability Goals (2015). Seventeen policy documents (UK n = 9, Australia n = 8) published between 2008 and 2018 were identified. Analysis involved two over-arching, iterative stages: a document analysis then a Foucauldian critical discourse analysis, the latter with the aim of unveiling the power dynamics at play within policy-related discourses.

RESULTS

Discourses of social mobility and individual responsibility within a meritocracy are still paramount in the UK. In contrast, the dominant discourse in Australia is social accountability in achieving equity and workforce diversity, prioritising affirmative action and community values. Similarities between the two countries in terms of WP policy and policy levers have changed over time, linked to the divergence of internal drivers for societal change. Both nations recognise tensions inherent in striving to achieve both local and global goals, but Australia appears to prioritise community values in working towards 'nation building' whereas in the UK the focus on individuality and meritocracy at times seems at odds with achieving parity for disadvantaged individuals.

DISCUSSION

WP policies and practices are situated and contextual so caution must be taken when extrapolating lessons from one context to another. The history of a country and the nature of marginalisation in that country must be scrutinised when trying to understand what drives WP policy.

"You can't always get what you want…": economic thinking, constrained optimization and health professions education

Every choice we make in health professions education has a cost, whether it be financial or otherwise; by choosing one action (e.g., integrating more simulation, studying more for a summative examination) we lose the opportunity to take an alternative action (e.g., freeing up time for other teaching, leisure time). Economics significantly shapes the way we behave and think as educators and learners and so there is increasing interest in using economic ways of thinking and approaches to examine and understand how choices are made, the influence of constraints and boundaries in educational decision making, and how costs are felt. Thus, in this article, we provide a brief historical overview of modern economics, to illustrate how the core concepts of economics-scarcity (and desirability), rationality, and optimization-developed over time. We explain the important concept of bounded rationality, which explains how individual, meso-factors and contextual factors influence decision making. We then consider the opportunities that these concepts afford for health professions education and research. We conclude by proposing that embracing economic thinking opens up new questions and new ways of approaching old questions which can add knowledge about how choice is enacted in contemporary health professions education.

Outcomes-Based Selection Into Medical School: Predicting Excellence in Multiple Competencies During the Clinical Years

PURPOSE

Medical school selection committees aim to identify the best possible students and, ultimately, the best future doctors from a large, well-qualified, generally homogeneous pool of applicants. Constructive alignment of medical school selection, curricula, and assessment with the ultimate outcomes (e.g., CanMEDS roles) has been proposed as means to attain this goal. Whether this approach is effective has not yet been established. The authors addressed this gap by assessing the relationship between performance in an outcomes-based selection procedure and performance during the clinical years of medical school.

METHOD

Two groups of students were compared: (1) those admitted into Maastricht University Medical School via an outcomes-based selection procedure and (2) those rejected through this procedure who were admitted into the program through a national, grade-point-average-based lottery. The authors compared performance scores of students from the 2 groups on all 7 CanMEDS roles, using assessment data gathered during clinical rotations. The authors examined data from 3 cohorts (2011-2013).

RESULTS

Students admitted through the local, outcomes-based selection procedure significantly outperformed the initially rejected but lottery-admitted students in all years, and the differences between groups increased over time. The selected students performed significantly better in the CanMEDS roles of Communicator, Collaborator, and Professional in the first year of clinical rotations; in these 3 roles-plus Organizer-in the second year; and in 2 additional roles (Advocate and Scholar-all except Medical Expert) at the end of their clinical training.

CONCLUSIONS

A constructively aligned selection procedure has increasing predictive value across the clinical years of medical school compared with a GPA-based lottery procedure. The data reported here suggest that constructive alignment of selection, curricula, and assessment to ultimate outcomes is effective in creating a selection procedure predictive of clinical performance.

Getting to know our non-traditional and rejected medical school applicants

None

Demographic and socioeconomic characteristics of Canadian medical students: a cross-sectional study

BACKGROUND

While the importance of medical students' demographic characteristics in influencing the scope and location of their future practice is recognized, these data are not systematically collected in Canada. This study aimed to characterize and compare the demographics of Canadian medical students with the Canadian population.

METHODS

Through an online survey, delivered in 2018, medical students at 14 English-speaking Canadian medical schools provided their age, sex, gender identity, ethnicity, educational background, and rurality of the area they grew up in. Respondents also provided information on parental income, occupation, and education as markers of socioeconomic status. Data were compared to the 2016 Canadian Census.

RESULTS

A total of 1388 students responded to the survey, representing a response rate of 16.6%. Most respondents identified as women (63.1%) and were born after 1989 (82.1%). Respondents were less likely, compared to the Canadian Census population, to identify as black (1.7% vs 6.4%) (P < 0.001) or Aboriginal (3.5% vs. 7.4%) (P < 0.001), and have grown up in a rural area (6.4% vs. 18.7%) (P < 0.001). Respondents had higher socioeconomic status, indicated by parental education (29.0% of respondents' parents had a master's or doctoral degree, compared to 6.6% of Canadians aged 45-64), occupation (59.7% of respondents' parents were high-level managers or professionals, compared to 19.2% of Canadians aged 45-64), and income (62.9% of respondents grew up in households with income >$100,000/year, compared to 32.4% of Canadians). Assessment of non-response bias showed that our sample was representative of all students at English-speaking Canadian medical schools with respect to age, though a higher proportion of respondents were female. Additionally, there were no differences between early and late respondents with respect to ethnicity, rurality, and parental income, occupation, and education.

CONCLUSIONS

Canadian medical students have different socioeconomic characteristics compared to the Canadian population. Collecting and analyzing these characteristics can inform evidence-based admissions policies.

Mitigating Violence Against First Responder Teams: Results and Ideas From the Hackmanathon

First responders are on the front line of patient care and service, but research has shown that they are also on the front line of exposure to violence. Currently, there is a lack of evidence-based interventions that prepare first responders to handle violence on the job. With the increase in emergency medical services (EMS) call volume and reports of at least 57% of the EMS responders having experienced workplace violence, there is a need to develop scientifically systematic solutions to improve emergency responder safety. Using an adapted version of the hackathon method, academic scholars and practitioner conference attendees at the Interdisciplinary Network for Group Research (INGRoup) Conference were deployed into three multidisciplinary teams to analyze the issue and develop specific solutions. These solutions offer unique interventions to improve first responder safety.

2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions

Selection and recruitment into healthcare education and practice is a key area of interest for educators with significant developments in research, policy, and practice in recent years. This updated consensus statement, developed through a multi-stage process, examines future opportunities and challenges in selection and recruitment. There is both a gap in the literature around and a compelling case for further theoretical and empirical literature to underpin the development of overall selection philosophes and policies and their enactment. More consistent evidence has emerged regarding the quality of different selection methods. Approaches to selection are context-dependent, requiring the consideration of an institution's philosophy regarding what they are trying to achieve, the communities it purports to serve, along with the system within which they are used. Diversity and globalization issues continue to be critically important topics. Further research is required to explore differential attainment and explain why there are substantial differences in culturally acceptable ways of approaching diversity and widening access. More sophisticated evaluation approaches using multi-disciplinary theoretical frameworks are required to address the issues. Following a discussion of these areas, 10 recommendations are presented to guide future research and practice and to encourage debate between colleagues across the globe.