Transition to multiple mini interview (MMI) interviewing for medical school admissions

Scoping review of current challenges and circumstances impacting Indigenous applications to Canadian medical schools

Introduction

Considering the relevant 2015 Truth and Reconciliation Commission recommendations, this paper reviews the current state of Canadian medical schools' Indigenous admissions processes and explores continued barriers faced by Indigenous applicants.

Methods

A summary of literature illustrating disadvantages for Indigenous applicants of current admissions tools is presented. A grey literature search of current admissions requirements, interview processes, and other relevant data from each medical school was performed. Tables comparing differences in their approaches are included. A calculation of Indigenous access to medical school seats compared to the broader Canadian population was conducted. Gaps in execution are explored, culminating in a table of recommendations.

Results

Despite formal commitments to reduce barriers, Indigenous applicants to medical school in Canada still face barriers that non-Indigenous applicants do not. Most programs use tools for admission known to disadvantage Indigenous applicants. Indigenous applicants do not have equitable access to medical school seats. Facilitated Indigenous stream processes first ensure Indigenous applicants meet all minimum requirements of Canadian students, and then require further work.

Discussion

Seven years after the Truth and Reconciliation Commission called on Canadian universities and governments to train more Indigenous health care providers, there has been limited progress to reduce the structural disadvantages Indigenous students face when applying to medical school. Based on best practices observed in Canada and coupled with relevant Indigenous-focused literature, recommendations are made for multiple stakeholders.

Conclusions

The study was limited by the data available on numbers of Indigenous applicants and matriculants. Where available, data are not encouraging as to equitable access to medical school for Indigenous populations in Canada. These findings were presented at the International Congress of Academic Medicine 2023 Conference, April 2023, Quebec City, Canada.

Has a national recruitment scheme created a positive intervention for Black, Asian or other Minority Ethnic pharmacy trainees?

OBJECTIVES

A National Recruitment Scheme (NRS) for Pharmacy trainees was introduced in England and Wales in 2017, standardising recruitment processes on behalf of employers and with the aim of reducing bias for candidates applying to training posts within the National Health Service (NHS). This research attempted to identify whether the introduction of the NRS had an impact on the recruitment of Black, Asian, or other Minority Ethnic applicants into the most sought-after posts within the Scheme (hospital posts).

METHODS

An observational study was undertaken. Anonymised pharmacist trainee recruitment data between the cohort intakes of 2015-16 and 2020-21 was obtained from the pharmacy regulator the General Pharmaceutical Council and a comparison of proportional representations of ethnicities was undertaken, to ascertain whether a greater proportion of applicants from minority backgrounds attained the most sought-after posts in the NHS after the NRS was introduced. A robust generalised linear model was then used to analyse the data using binomial as the variance function and logit as a link function, where the proportion of hospital recruitment was an outcome with a two-way interaction between intervention and ethnicity after adjusting for overall proportion.

KEY FINDINGS

The statistical analysis of 18 283 pharmacy trainees in total, of whom 4446 were in hospital, shows a significant overall impact of intervention, with a significant positive change in the proportions of Asian-Pakistani applicants (P-value < 0.001) and Black-African applicants (P-value < 0.001) recruited to hospital posts.

CONCLUSIONS

Since the introduction of the NRS there has been a statistically significant impact on the correlation between the overall number of Black, Asian or other Minority Ethnic applicants and their proportion in hospital. That is, not only is the makeup of the hospital cohort increasingly reflecting the diversity of the overall cohort, but also a larger percentage of each ethnic cohort is attaining hospital training places.

Comparison of the Multiple Mini-Interview and the Traditional Interview in Medical School Admissions: Lessons Learned Using a Hybrid Model at One Institution

PURPOSE

Medical school admissions interviews are a critical form of assessment; however, the most effective interview strategy is debated. This study compares the traditional interview (TI) and multiple mini-interview (MMI) within a hybrid TI-MMI model at one medical school to determine whether the interview approaches reveal different information about applicants and whether a hybrid model results in a more diversified applicant pool.

METHOD

Admissions data from 3 application cycles at the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell were used. The TI was used in 2017-2018 and the hybrid TI-MMI model in 2018-2019 and 2019-2020. Applicants were scored on a 5-point scale and referred to a voting committee for acceptance consideration if interview scores met threshold criteria. Changes in the number of students referred to the committee using the TI vs the TI-MMI score criteria were analyzed.

RESULTS

In 2017-2018 (TI only), 683 applicants were interviewed; in 2018-2019 (TI-MMI), 844 applicants were interviewed; and in 2019-2020 (TI-MMI), 805 applicants were interviewed. Medium correlations were found between total MMI and TI scores in 2018-2019 ( ρ = 0.37, P < .001) and 2019-2020 ( ρ = 0.33, P < .001). No differences were found in TI scores between 2017-2018 and 2018-2019 ( P = .30), but TI scores were significantly lower in 2019-2020 vs 2017-2018 ( P < .001) and 2018-2019 ( P = .002). Overall, a 10% to 18% increase was found in the number of applicants referred to the voting committee when using hybrid criteria, with a 19% to 27% increase in underrepresented in medicine applicants.

CONCLUSIONS

The TI-MMI model may allow for a more holistic interview approach and an expanded pool of applicants, particularly underrepresented in medicine applicants, considered for acceptance.

A mixed methods evaluation of Multiple Mini Interviews for entry into the Bachelor of Midwifery

BACKGROUND

Multiple-mini interviews (MMI) are increasingly used as part of the admission process into healthcare degrees. Evaluations have found MMIs to be a fair assessment tool in terms of reliability and validity and viewed positively by those involved in the MMI process. The use of MMIs in midwifery is novel and evaluation is lacking.

AIM

To evaluate the use of MMIs as part of the admission process for the Bachelor of Midwifery in one Australian university.

METHODS

A basic convergent mixed methods study design was utilised. Data included linked data sets, Likert scale responses to survey questions, focus groups and open-ended survey questions. Integration took place at the interpretation and reporting stage.

FINDINGS

Participants viewed the MMI experience positively. The study confirmed the reliability of the MMIs as an assessment tool. Most variance in MMI scores was attributed to the candidate at 31.4 % with the interviewer and the interview station having less influence on the MMI score at 11 % and 6.4 % variance. Older applicants on average achieved higher MMI scores, and those who spoke a language other than English at home or were first in family to attend university had lower on average MMI scores. Being born overseas did not impact an applicant's MMI score. The overall experience was seen as fair, offering further opportunity to gain entry into the Bachelor of Midwifery.

CONCLUSION

MMIs were viewed positively and findings support the use of MMIs as part of an admissions process for the Bachelor of Midwifery.

Evaluating situational judgment test use and diversity in admissions at a southern US medical school

INTRODUCTION

Situational judgment tests have been adopted by medical schools to assess decision-making and ethical characteristics of applicants. These tests are hypothesized to positively affect diversity in admissions by serving as a noncognitive metric of evaluation. The purpose of this study was to evaluate the performance of the Computer-based Assessment for Sampling Personal Characteristics (CASPer) scores in relation to admissions interview evaluations.

METHODS

This was a cohort study of applicants interviewing at a public school of medicine in the southeastern United States in 2018 and 2019. Applicants took the CASPer test prior to their interview day. In-person interviews consisted of a traditional interview and multiple-mini-interview (MMI) stations. Between subjects, analyses were used to compare scores from traditional interviews, MMIs, and CASPer across race, ethnicity, and gender.

RESULTS

1,237 applicants were interviewed (2018: n = 608; 2019: n = 629). Fifty-seven percent identified as female. Self-identified race/ethnicity included 758 White, 118 Black or African-American, 296 Asian, 20 Native American or Alaskan Native, 1 Native Hawaiian or Other Pacific Islander, and 44 No response; 87 applicants identified as Hispanic. Black or African-American, Native American or Alaskan Native, and Hispanic applicants had significantly lower CASPer scores than other applicants. Statistically significant differences in CASPer percentiles were identified for gender and race; however, between subjects, comparisons were not significant.

CONCLUSIONS

The CASPer test showed disparate scores across racial and ethnic groups in this cohort study and may not contribute to minimizing bias in medical school admissions.

Evaluating a Global Assessment Measure Created by Standardized Patients for the Multiple Mini Interview in Medical School Admissions: Mixed Methods Study

Background

Standardized patients (SPs) are essential stakeholders in the multiple mini interviews (MMIs) that are increasingly used to assess medical school applicants’ interpersonal skills. However, there is little evidence for their inclusion in the development of instruments.

Objective

This study aimed to describe the process and evaluate the impact of having SPs co-design and cocreate a global measurement question that assesses medical school applicants’ readiness for medical school and acceptance status.

Methods

This study used an exploratory, sequential, and mixed methods study design. First, we evaluated the initial MMI program and determined the next quality improvement steps. Second, we held a collaborative workshop with SPs to codevelop the assessment question and response options. Third, we evaluated the created question and the additional MMI rubric items through statistical tests based on 1084 applicants’ data from 3 cohorts of applicants starting in the 2018-2019 academic year. The internal reliability of the MMI was measured using a Cronbach α test, and its prediction of admission status was tested using a forward stepwise binary logistic regression.

Results

Program evaluation indicated the need for an additional quantitative question to assess applicant readiness for medical school. In total, 3 simulation specialists, 2 researchers, and 21 SPs participated in a workshop leading to a final global assessment question and responses. The Cronbach α’s were >0.8 overall and in each cohort year. The final stepwise logistic model for all cohorts combined was statistically significant (P<.001), explained 9.2% (R²) of the variance in acceptance status, and correctly classified 65.5% (637/972) of cases. The final model consisted of 3 variables: empathy, rank of readiness, and opening the encounter.

Conclusions

The collaborative nature of this project between stakeholders, including nonacademics and researchers, was vital for the success of this project. The SP-created question had a significant impact on the final model predicting acceptance to medical school. This finding indicates that SPs bring a critical perspective that can improve the process of evaluating medical school applicants.

Weight management counseling experiences of first year medical students before starting medical school and their self-perceived impact on treating patients with obesity

Physicians can play a vital role in counseling patients on overweight and obesity. This secondary analysis examined whether experiences in patient care specific to weight management before starting medical school were associated with students' intentions and confidence to provide weight management counseling (WMC) to patients who have overweight or obesity, and perceived impact as future physicians on patients' motivation to manage weight. First-year medical students (n = 1305) in the entering class of 2020 at eight medical schools nationwide completed questions relating to their prior experiences in patient care and WMC using the 5As. Also assessed were their intentions to treat patients with overweight or obesity, and confidence in counseling patients to help manage their weight. Over half the students (58.3%) who completed the survey had prior experience in patient care and nearly half (47.4%) began medical school with prior WMC experiences. Prior experiences correlated positively with higher confidence in performing WMC and students' intentions to treat patients with overweight or obesity. Given the relatively high rates of exposure to some type of weight management or lifestyle counseling among students before enrolling in medical school, the curriculum could build on established student interest and experience by offering treatment strategies including counseling for patients with overweight and obesity. By making prior experiences advantageous for admission, medical schools could gravitate towards admitting students who have brief but valuable insights about weight management in health care, thus increasing the possibility of filling important gaps.