Lessons learned from 15 years of non-grades-based selection for medical school

Improving selection procedures in health professions education from the applicant perspective: an interview study

INTRODUCTION

Applicant perceptions of selection impact motivation and performance during selection, and student diversity. However, in-depth insight into which values underly these perceptions is lacking, creating challenges for aligning selection procedures with applicant perceptions. This qualitative interview study aimed to identify values applicants believe should underlie selection, and how, according to applicants, these values should be used to make specific improvements to selection procedures in undergraduate health professions education (HPE).

METHODS

Thirty-one applicants to five undergraduate HPE programs in the Netherlands participated in semi-structured interviews using Appreciative Inquiry, an approach that focuses on what goes well to create vision for improvement, to guide the interviews. Transcriptions were analyzed using thematic analysis, adopting a constructivist approach.

RESULTS

Applicants' values related to the aims of selection, the content of selection, and the treatment of applicants. Applicants believed that selection procedures should aim to identify students who best fit the training and profession, and generate diverse student populations to fulfill societal needs. According to applicants, the content of selection should be relevant for the curriculum and profession, assess a comprehensive set of attributes, be of high quality, allow applicants to show who they are, and be adapted to applicants' current developmental state. Regarding treatment, applicants believed that selection should be a two-way process that fosters reflection on study choice, be transparent about what applicants can expect, safeguard applicants' well-being, treat all applicants equally, and employ an equitable approach by taking personal circumstances into account. Applicants mentioned specific improvements regarding each value.

DISCUSSION

Applicants' values offer novel insights into what they consider important preconditions for the design of selection procedures. Their suggested improvements can support selection committees in better meeting applicants' needs.

How can healthcare organisations increase doctors' research engagement? A scoping review

PURPOSE

Clinician engagement in research has positive impacts for healthcare, but is often difficult for healthcare organisations to support in light of limited resources. This scoping review aimed to describe the literature on health service-administered strategies for increasing research engagement by medical practitioners.

DESIGN/METHODOLOGY/APPROACH

Medline, EMBASE and Web of Science databases were searched from 2000 to 2021 and two independent reviewers screened each record for inclusion. Inclusion criteria were that studies sampled medically qualified clinicians; reported empirical data; investigated effectiveness of an intervention in improving research engagement and addressed interventions implemented by an individual health service/hospital.

FINDINGS

Of the 11,084 unique records, 257 studies were included. Most (78.2%) studies were conducted in the USA, and were targeted at residents (63.0%). Outcomes were measured in a variety of ways, most commonly publication-related outcomes (77.4%), though many studies used more than one outcome measure (70.4%). Pre-post (38.8%) and post-only (28.7%) study designs were the most common, while those using a contemporaneous control group were uncommon (11.5%). The most commonly reported interventions included Resident Research Programs (RRPs), protected time, mentorship and education programs. Many articles did not report key information needed for data extraction (e.g. sample size).

ORIGINALITY/VALUE

This scoping review demonstrated that, despite a large volume of research, issues like poor reporting, infrequent use of robust study designs and heterogeneous outcome measures limited application. The most compelling available evidence pointed to RRPs, protected time and mentorship as effective interventions. Further high-quality evidence is needed to guide healthcare organisations on increasing medical research engagement.

A "Red Flag" system adds value to medical school admissions interviews

INTRODUCTION

Non-cognitive traits should be considered when selecting candidates to study medicine. However, evaluating these traits remains difficult. We explored whether measuring undesirable non-cognitive behaviour ('Red Flags') added value to a medical school admissions system. Red Flags included rudeness, ignoring the contributions of others, disrespectful behaviour, or poor communication.

METHODS

Following an admissions interview testing non-cognitive attributes in 648 applicants to a UK medical school, we measured the association between interview score and Red Flag frequency. We tested linear and polynomial regression models to evaluate whether the association was linear or non-linear.

RESULTS

In total, 1126 Red Flags were observed. While Red Flags were concentrated among low-scorers, candidates in the highest- and second-highest deciles for interview score still received Red Flags (six and twenty-two, respectively). The polynomial regression model indicated candidates with higher scores received fewer Red Flags, but the association was not linear (F(3644) = 159.8, p = .001, adjusted R2 = 0.42).

CONCLUSIONS

The non-linear association between interview score and Red Flag frequency shows some candidates with desirable non-cognitive attributes will still display undesirable-or even exclusionary-non-cognitive attributes. Recording Red Flag behaviour reduces the likelihood such candidates will be offered a place at medical school.

What makes a 'good doctor'? A critical discourse analysis of perspectives from medical students with lived experience as patients

What constitutes a 'good doctor' varies widely across groups and contexts. While patients prioritise communication and empathy, physicians emphasise medical expertise, and medical students describe a combination of the two as professional ideals. We explored the conceptions of the 'good doctor' held by medical learners with chronic illnesses or disabilities who self-identify as patients to understand how their learning as both patients and future physicians aligns with existing medical school curricula. We conducted 10 semistructured interviews with medical students with self-reported chronic illness or disability and who self-identified as patients. We used critical discourse analysis to code for dimensions of the 'good doctor'. In turn, using concepts of Bakhtinian intersubjectivity and the hidden curriculum we explored how these discourses related to student experiences with formal and informal curricular content.According to participants, dimensions of the 'good doctor' included empathy, communication, attention to illness impact and boundary-setting to separate self from patients. Students reported that formal teaching on empathy and illness impact were present in the formal curriculum, however ultimately devalued through day-to-day interactions with faculty and peers. Importantly, teaching on boundary-setting was absent from the formal curriculum, however participants independently developed reflective practices to cultivate these skills. Moreover, we identified two operating discourses of the 'good doctor': an institutionalised discourse of the 'able doctor' and a counterdiscourse of the 'doctor with lived experience' which created a space for reframing experiences with illness and disability as a source of expertise rather than a source of stigma. Perspectives on the 'good doctor' carry important implications for how we define professional roles, and hold profound consequences for medical school admissions, curricular teaching and licensure. Medical students with lived experiences of illness and disability offer critical insights about curricular messages of the 'good doctor' based on their experiences as patients, providing important considerations for curriculum and faculty development.

Inequality of opportunity in selection procedures limits diversity in higher education: An intersectional study of Dutch selective higher education programs

Selection for higher education (HE) programs may hinder equal opportunities for applicants and thereby reduce student diversity and representativeness. However, variables which could play a role in inequality of opportunity are often studied separately from each other. Therefore, this retrospective cohort study conducts an innovative intersectional analysis of the inequality of opportunity in admissions to selective HE programs. Using a combination of multivariable logistic regression analyses and descriptive statistics, we aimed to investigate 1) the representativeness of student populations of selective HE programs, as compared to both the applicant pool and the demographics of the age cohort; 2) the demographic background variables which are associated with an applicant's odds of admission; and 3) the intersectional acceptance rates of applicants with all, some or none of the background characteristics positively associated with odds of admission. The study focused on all selective HE programs (n = 96) in The Netherlands in 2019 and 2020, using Studielink applicant data (N = 85,839) and Statistics Netherlands microdata of ten background characteristics. The results show that student diversity in selective HE programs is limited, partly due to the widespread inequality of opportunity in the selection procedures, and partly due to self-selection. Out of all ten variables, migration background was most often (negatively) associated with the odds of receiving an offer of admission. The intersectional analyses provide detailed insight into how (dis)advantage has different effects for different groups. We therefore recommend the implementation of equitable admissions procedures which take intersectionality into account.

Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153

Selection is the first assessment of medical education and training. Medical schools must select from a pool of academically successful applicants and ensure that the way in which they choose future clinicians is robust, defensible, fair to all who apply and cost-effective. However, there is no comprehensive and evidence-informed guide to help those tasked with setting up or rejuvenating their local selection process. To address this gap, our guide draws on the latest research, international case studies and consideration of common dilemmas to provide practical guidance for designing, implementing and evaluating an effective medical school selection system. We draw on a model from the field of instructional design to frame the many different activities involved in doing so: the ADDIE model. ADDIE provides a systematic framework of Analysis (of the outcomes to be achieved by the selection process, and the barriers and facilitators to achieving these), Design (what tools and content are needed so the goals of selection are achieved), Development (what materials and resources are needed and available), Implementation (plan [including piloting], do study and adjust) and Evaluation (quality assurance is embedded throughout but the last step involves extensive evaluation of the entire process and its outcomes).HIGHLIGHTSRobust, defensible and fair selection into medical school is essential. This guide systematically covers the processes required to achieve this, from needs analysis through design, development and implementation, to evaluation of the success of a selection process.

Selection tools and student diversity in health professions education: a multi-site study

Student diversity in health professions education (HPE) can be affected by selection procedures. Little is known about how different selection tools impact student diversity across programs using different combinations of traditional and broadened selection criteria. The present multi-site study examined the chances in selection of subgroups of applicants to HPE undergraduate programs with distinctive selection procedures, and their performance on corresponding selection tools. Probability of selection of subgroups (based on gender, migration background, prior education, parental education) of applicants (N = 1935) to five selection procedures of corresponding Dutch HPE undergraduate programs was estimated using multilevel logistic regression. Multilevel linear regression was used to analyze performance on four tools: prior-education grade point average (pe-GPA), biomedical knowledge test, curriculum-sampling test, and curriculum vitae (CV). First-generation Western immigrants and applicants with a foreign education background were significantly less likely to be selected than applicants without a migration background and with pre-university education. These effects did not vary across programs. More variability in effects was found between different selection tools. Compared to women, men performed significantly poorer on CVs, while they had higher scores on biomedical knowledge tests. Applicants with a non-Western migration background scored lower on curriculum-sampling tests. First-generation Western immigrants had lower CV-scores. First-generation university applicants had significantly lower pe-GPAs. There was a variety in effects for applicants with different alternative forms of prior education. For curriculum-sampling tests and CVs, effects varied across programs. Our findings highlight the need for continuous evaluation, identifying best practices within existing tools, and applying alternative tools.

Does medical education reform change who is selected? A national cross-sectional survey from China

OBJECTIVES

Approximately 10 years ago, China introduced an education plan to improve the overall quality of medical education and to better serve the population's health needs. Many medical schools were then recognised and financed by China's Ministry of Education to develop and operationalise new pilot programmes (PPs) aligned with this plan. These ran in parallel with the traditional programmes (TPs). One way to achieve the plan's first aim, improving the quality of medical education, is to select academically stronger candidates. We, thus, examined and compared who were selected into PPs and TPs.

DESIGN

Cross-sectional study.

SETTING

Data were collected from 123 medical schools across China via the 2021 China Medical Student Survey.

PARTICIPANTS

Participants were undergraduate clinical medicine students across all year groups.

PRIMARY AND SECONDARY OUTCOME MEASURES

Medical school selection was via the National College Entrance Examination (NCEE). Medical students' NCEE performance and their sociodemographics were used as the primary and secondary outcome measures. Mann-Whitney or χ2 tests were used to compare the means between educational programmes (PPs vs TPs) and various selection outcomes. Multilevel mixed-effects regressions were employed to account for school idiosyncratic selection results.

RESULTS

Of the 204 817 respondents, 194 163 (94.8%) were in a TP and 10 654 (5.2%) a PP. PP respondents (median=75.2, IQR=69.5-78.8) had significantly higher NCEE scores than their TP counterparts (median=73.9, IQR=68.5-78.7). Holding constant their NCEE score, PP respondents were significantly more likely to come from urban areas, not be first-generation college students, and have parents with higher occupational status and income.

CONCLUSIONS

Assuming quality can be indicated by prior academic achievement at the point of selection, PPs achieved this mission. However, doing so limited medical students' diversity. This may be unhelpful in achieving the Education Plan's goal to better serve China's health needs.

The association between how medical students were selected and their perceived stress levels in Year-1 of medical school

BACKGROUND

The prevalence of medical students' mental distress is high. While schools apply various methods to select a well-performing and diverse student population, little is known about the association between different selection methods and the well-being of these students during medical school. The present retrospective multi-cohort study assessed whether students selected by high grades, assessment, or weighted lottery showed different stress perception levels in Year-1 of medical school.

METHODS

Of 1144 Dutch Year-1 medical students, 650 (57%) of the cohorts 2013, 2014, and 2018 who were selected by high grades, assessment, or weighted lottery completed a stress perception questionnaire (PSS-14). A multilevel regression analysis assessed the association between selection method (independent variable) and stress perception levels (dependent variable) while controlling for gender and cohort. In a post-hoc analysis, academic performance (optimal vs. non-optimal) was included in the multilevel model.

RESULTS

Students selected by assessment (B = 2.25, p < .01, effect size (ES) = small) or weighted lottery (B = 3.95, p < .01, ES = medium) had higher stress perception levels than students selected by high grades. Extending the regression model with optimal academic performance (B=-4.38, p < .001, ES = medium), eliminated the statistically significant difference in stress perception between assessment and high grades and reduced the difference between weighted lottery and high grades from 3.95 to 2.45 (B = 2.45, p < .05, ES = small).

CONCLUSIONS

Selection methods intended to create a diverse student population - assessment and lottery - are associated with higher stress perception levels in Year-1 of medical school. These findings offer medical schools insights into fulfilling their responsibility to take care of their students' well-being.

The added value of free preparatory activities for widening access to medical education: a multi-cohort study

BACKGROUND

Medical schools are reported to be less accessible to students with non-traditional backgrounds. These students face barriers when applying for and transitioning to medical school, which may be reduced by offering free preparatory activities. By equalizing access to resources, these activities are expected to reduce disparities in selection outcomes and early academic performance. In the present study, four free institutionally-provided preparatory activities were evaluated by comparing the demographic composition of participating and non-participating applicants. Additionally, the association between participation and selection outcomes and early academic performance was investigated for subgroups (based on sex, migration background and parental education).

METHODS

Participants were applicants to a Dutch medical school in 2016-2019 (N = 3592). Free preparatory activities included Summer School (N = 595), Coaching Day (N = 1794), Pre-Academic Program (N = 217), and Junior Med School (N = 81), supplemented with data on participation in commercial coaching (N = 65). Demographic compositions of participants and non-participants were compared using chi-squared tests. Regression analyses were performed to compare selection outcomes (curriculum vitae [CV], selection test score, probability of enrolment) and early academic performance (first-course grade) between participants and non-participants of demographic subgroups, controlling for pre-university grades and participation in other activities.

RESULTS

Generally, no differences in sociodemographic compositions of participants and non-participants were found, but males participated less often in Summer School and Coaching Day. Applicants with a non-Western background participated less often in commercial coaching, but the overall participation rate was low and participation had negligible effects on selection outcomes. Participation in Summer School and Coaching Day were stronger related with selection outcomes. In some cases, this association was even stronger for males and candidates with a migration background. After controlling for pre-university grades, none of the preparatory activities were positively associated with early academic performance.

CONCLUSIONS

Free institutionally-provided preparatory activities may contribute to student diversity in medical education, because usage was similar across sociodemographic subgroups, and participation was positively associated with selection outcomes of underrepresented and non-traditional students. However, since participation was not associated with early academic performance, adjustments to activities and/or curricula are needed to ensure inclusion and retention after selection.

Relationship between selection criteria and trainee performance in medical specialty training: A retrospective longitudinal study

BACKGROUND

Investigation of tools used for candidate selection as predictors of future performance in medical education has been primarily undertaken within the undergraduate setting, but little is known about the selection to medical specialist programs. This retrospective longitudinal study aims to explore correlations between selection tools and the performance of trainees enrolled in the Australasian College of Dermatologists (ACD), the accredited specialist medical college for training in dermatology in Australia.

METHODS

Data were collected from consecutive cohorts of ACD trainees commencing from 2007 to 2015. Predictive variables were trainee demographics and selection tools (prior academic qualification; research experience; clinical experience; curriculum vitae [CV] rating; interview scores). The outcome variables were exam performance. First Year ([Pharmacology and Clinical Sciences] and Fellowship Exam [Written, Clinical, and Overall Score]). Statistical analyses included: descriptive statistics summaries; bivariate correlation of selection criteria and demographic data with exam performance; and linear regression analysis to identify predictors of exam performance.

RESULTS

Demographic analysis (N = 172) showed that 64% of trainees were female, the average age was 30.5 years (± 3.47) and trainees living in high socioeconomic status (SES) areas were over-represented. Using Pearson correlation analysis, interview scores were significantly positively correlated with First Year Pharmacology results (p = 0.018), Fellowship Written results (p = 0.002), and Fellowship Overall Scores (p = 0.006). First Year Pharmacology Exam performance was most highly correlated with Fellowship Exam performance (p = 0.000). No association was identified between exam performance and gender, SES, prior academic qualification, research experience or CV rating. Linear regression analysis showed that interview score (p = 0.012), entry age (p = 0.026) and First Year Pharmacology score (p = 0.002) were predictors of Fellowship exam performance.

CONCLUSION

These results suggest that the comprehensive selection tools used by ACD are effective, with interviews being a highly valid tool linked to exam performance. These findings have important practical implications for assessing the selection process in specialist dermatology training.

Applicant perceptions of selection methods for health professions education: Rationales and subgroup differences

CONTEXT

Applicant perceptions of selection methods can affect motivation, performance and withdrawal and may therefore be of relevance in the context of widening access. However, it is unknown how applicant subgroups perceive different selection methods.

OBJECTIVES

Using organisational justice theory, the present multi-site study examined applicant perceptions of various selection methods, rationales behind perceptions and subgroup differences.

METHODS

Applicants to five Dutch undergraduate health professions programmes (N = 704) completed an online survey including demographics and a questionnaire on applicant perceptions applied to 11 commonly used selection methods. Applicants rated general favourability and justice dimensions (7-point Likert scale) and could add comments for each method.

RESULTS

Descriptive statistics revealed a preference for selection methods on which applicants feel more 'in control': General favourability ratings were highest for curriculum-sampling tests (mean [M] = 5.32) and skills tests (M = 5.13), while weighted lottery (M = 3.05) and unweighted lottery (M = 2.97) were perceived least favourable. Additionally, applicants preferred to distinguish themselves on methods that assess attributes beyond cognitive abilities. Qualitative content analysis of comments revealed several conflicting preferences, including a desire for multiple selection methods versus concerns of experiencing too much stress. Results from a linear mixed model of general favourability indicated some small subgroup differences in perceptions (based on gender, migration background, prior education and parental education), but practical meaning of these differences was negligible. Nevertheless, concerns were expressed that certain selection methods can hinder equitable admission due to inequal access to resources.

CONCLUSIONS

Our findings illustrate that applicants desire to demonstrate a variety of attributes on a combination of selection tools, but also observe that this can result in multiple drawbacks. The present study can help programmes in deciding which selection methods to include, which more negatively perceived methods should be better justified to applicants, and how to adapt methods to meet applicants' needs.

Influence of social networks in healthcare on preparation for selection procedures of health professions education: a Dutch interview study

OBJECTIVES

Health professions education (HPE) students are often not representative of the populations they will serve. The underrepresentation of non-traditional students is problematic because diversity is essential for promoting excellence in health education and care. This study aimed to understand the perceptions of traditional and non-traditional students regarding facilitators and barriers in preparing for HPE selection procedures, and to determine the role of social networks in their decision-making and preparations to apply.

METHODS

A qualitative study was conducted with 26 Dutch youth who were interested in university-level HPE programmes. Semistructured interviews and sociograms were analysed using thematic analysis, adopting a constructivist approach.

RESULTS

Twenty-six high school students participated, with traditional and non-traditional backgrounds, with and without social networks in healthcare and higher education. Two themes were constructed. First, four high-impact facilitators helped to overcome barriers to apply and in preparation for selection: access to a social network connection working or studying in healthcare, to correct information, to healthcare experience and to a social network connection in higher education. Lack of information was the main barrier while access to social network connections in healthcare was the main facilitator to overcome this barrier. However, this access was unevenly distributed. Second, access alone is not enough: the need for agency to make use of available facilitators is also essential.

CONCLUSIONS

The themes are discussed using intersectionality. Traditional students with access to facilitators develop their self-efficacy and agency within social structures that privilege them, whereas non-traditional students must develop those skills without such structures. Our findings provide recommendations for the ways in which universities can remove barriers that cause unequal opportunities to prepare for the selection of HPE programmes. Along with equitable admissions, these recommendations can help to achieve a more representative student population and subsequently a better quality of health education and care.

Selection process for admission to an academic nursing programme - A meta-synthesis

BACKGROUND

Student nurse selection has been a topic of debate for years. Poor nursing care has been attributed to uncaring attitudes, lack of motivation and poor training and mentoring. Student nurse selection requirements and the steps involved in the process have been a revolving debate. Internationally, the recruitment and selection process are being refined to align with recommendations from the Francis Inquiry to include a values assessment.

OBJECTIVE

To select, review and interpret evidence-based studies on student nurse selection; and develop and provide recommendations for nurse educators when selecting first-year student nurses as a holistic, integrative process fitting with caring and providing quality care.

DESIGN

Qualitative meta-synthesis.

DATA SOURCES

Electronic databases were used during data collection: Ebscohost, ScienceDirect, SAGE, Cinahl and Sabinet (SAePublications). Qualitative articles published in English or Afrikaans reporting on student selection processes and experiences were included in the data collection process.

REVIEW METHODS

Qualitative articles were systematically identified using the STARLITE principles. The articles were critically appraised for quality by employing the CASP method.

RESULTS

Fourteen articles were included. Thirteen of the articles were published in English. One PhD thesis was included, a seminal work conducted on student nurse selection in South Africa. Data analysis revealed three main themes: a reliance on the interview, alternative selection methods, and attributes as criteria for nursing selection. Within these themes, many different methods of student selection emerged. The benefits and challenges of these methods were identified, and a holistic admission process was ultimately most successful in selecting the correct student nurse.

CONCLUSIONS

The findings indicate there are many different student nurse selection processes. A holistic admission process is needed to select the best student nurse that displays caring attributes and provides quality nursing care.

Selection for health professions education leads to increased inequality of opportunity and decreased student diversity in The Netherlands, but lottery is no solution: A retrospective multi-cohort study

BACKGROUND

Concerns exist about the role of selection in the lack of diversity in health professions education (HPE). In The Netherlands, the gradual transition from weighted lottery to selection allowed for investigating the variables associated with HPE admission, and whether the representativeness of HPE students has changed.

METHOD

We designed a retrospective multi-cohort study using Statistics Netherlands microdata of all 16-year-olds on 1 October 2008, 2012, and 2015 (age cohorts, N > 600,000) and investigated whether they were eligible students for HPE programs (n > 62,000), had applied (n > 14,000), and were HPE students at age 19 (n > 7500). We used multivariable logistic regression to investigate which background variables were associated with becoming an HPE student.

RESULTS

HPE students with ≥1 healthcare professional (HP) parent, ≥1 top-10% income/wealth parent, and women are overrepresented compared to all age cohorts. During hybrid lottery/selection (cohort-2008), applicants with ≥1 top-10% wealth parent and women had higher odds of admission. During 100% selection (cohort-2015) this remained the case. Additionally, applicants with ≥1 HP parent had higher odds, those with a migration background had lower odds.

CONCLUSIONS

Odds of admission are increasingly influenced by applicants' backgrounds. Targeted recruitment and equitable admissions procedures are required to increase matriculation of underrepresented students.

Medical empathy in medical students in Madrid: A proposal for empathy level cut-off points for Spain

This study evaluates the degree of empathy among medical students and its influencing factors at three critical moments of their degree studies (beginning of first year and end of third and sixth years) as well as establishes low-, medium-, and high-empathy cut-off points to obtain valid and reliable results that can be extrapolated to the general population. This cross-sectional study of the eight (public and private) medical schools in the province of Madrid, used an electronic questionnaire with the Jefferson Scale of Empathy (JSE), Medical Student Well-Being Index, and other independent characteristics as measuring instruments. Of the 2,264 student participants, 1,679 (74.0%) were women, with a 50.7% participation rate. No significant differences were found in empathy levels by academic year. Regarding range, percentile and cut-off point tables were established to identify students with high, medium, and low empathy levels. Women (p<0.001), volunteer workers (p<0.001), and those preferring general specialties (internal medicine, psychiatry, pediatrics, or family medicine) scored higher on the JSE (p<0.02). Moreover, 41.6% presented high level of psychological distress. Women reported a lower well-being level and a higher risk of psychological distress (p = 0.004). In sum, the empathy of medical students in Madrid did not differ among the three critical moments of their university studies. The established cut-off points could be taken into account when accessing the medical degree and identifying students with low levels of empathy to implement curricular interventions to rectify this perceived deficiency. There was a high percentage of medical students with high levels of psychological distress.

Accepting randomness in medical school admissions: The case for a lottery

McMaster University's medical school, faced with the need to socially distance during the COVID-19 pandemic, recently replaced their structured admission interview process with a partial lottery. At first, it may seem that leaving medical school admissions partly to chance could erode autonomy and meritocracy. Yet our current system for selecting medical students is strained by a limited predictive ability. In the search for good doctors, we lack meaningful, quantifiable, and comparable criteria. Partial or weighted admissions lotteries can offer us an escape. They have the potential to reduce mental and financial burdens on both applicants and medical schools, avoiding an overemphasis on marginal differences between applicants. Lotteries are also a simple way to address persistent admissions disparities by being truly non-discriminatory. At the very least, lotteries represent a useful benchmark against which we can rigorously compare current and future selection methods.

Effects of raising the bar on medical student study progress: An intersectional approach

CONTEXT

Medical schools seek for measures to improve their students' study progress and are responsible for a diverse student population.

OBJECTIVES

The effect of a stricter academic dismissal (AD) policy in medical school on short-term and long-term study progress was investigated in a longitudinal cohort study. In addition, differential effects for subgroups were assessed by intersecting gender, ethnicity and prior education (intersectional framework).

METHODS

Participants were first-year Bachelor students enrolled in 2011 to 2016 in a Dutch medical school. For cohorts 2011-2013, the AD policy consisted of a minimum of 67% of Year-1 credits required to remain enrolled (67%-policy, n = 1189), and for cohorts 2014-2016, this bar was raised to 100% of Year-1 credits (100%-policy, n = 1233). Outcome measures on study progress were Year-1 completion and dropout (short term) and Bachelor completion in three and four years (long term).

RESULTS

Overall, Year-1 completion rates increased under the 100%-policy compared to the 67%-policy (OR = 2.50, 95%-CI:2.06-3.03, P < .001). Yet, this increase was not present for students with non-standard prior education - except for males with a migration background (OR = 7.19, 95%-CI:2.33-25.73, P < .01). The dropout rate doubled under the 100%-policy (OR = 2.41, 95%-CI:1.68-3.53, P < .001). Mainly students with standard prior education dropped out more often (OR = 3.68, 95%-CI:2.37-5.89, P < .001), except for males with a migration background. Bachelor completion rates after three and four years were not positively affected by the 100%-policy. Notably, females without a migration background and with non-standard prior education suffered from the 100%-policy regarding Bachelor completion after three years (OR = 0.29, 95%-CI:0.11-0.76, P < .05).

CONCLUSIONS

Despite increased dropout rates, the stricter AD policy improved Year-1 completion rates - especially for under-represented subgroups, thereby improving study progress without harming student diversity on the short term. However, these positive effects did not hold regarding Bachelor completion rates indicating that long-term effects require higher performance standards throughout the Bachelor, which in turn may harm other subgroups and thereby student diversity.

Unraveling the medical residency selection game

The diversity of modern society is often not represented in the medical workforce. This might be partly due to selection practices. We need to better understand decision-making processes by selection committees in order to improve selection procedures with regard to diversity. This paper reports on a qualitative study with a socio-constructivist perspective conducted in 2015 that explored how residency selection decision-making occurred within four specialties in two regions in the Netherlands. Data included transcripts of the decision-making meetings and of one-on-one interviews with committee members before and after the group decision-making meetings. Candidates struggled to portray themselves favorably as they had to balance playing by the rules and being authentic; between fitting in and standing out. Although admissions committees had a welcoming stance to diversity, their practices were unintentionally preventing them from hiring underrepresented minority (URM) candidates. While negotiating admissions is difficult for all candidates, it is presumably even more complicated for URM candidates. This seems to be having a negative influence on attaining workforce diversity. Current beliefs, which make committees mistakenly feel they are acting fairly, might actually justify biased practices. Awareness of the role of committee members in these processes is an essential first step.

Outcomes-Based Selection Into Medical School: Predicting Excellence in Multiple Competencies During the Clinical Years

PURPOSE

Medical school selection committees aim to identify the best possible students and, ultimately, the best future doctors from a large, well-qualified, generally homogeneous pool of applicants. Constructive alignment of medical school selection, curricula, and assessment with the ultimate outcomes (e.g., CanMEDS roles) has been proposed as means to attain this goal. Whether this approach is effective has not yet been established. The authors addressed this gap by assessing the relationship between performance in an outcomes-based selection procedure and performance during the clinical years of medical school.

METHOD

Two groups of students were compared: (1) those admitted into Maastricht University Medical School via an outcomes-based selection procedure and (2) those rejected through this procedure who were admitted into the program through a national, grade-point-average-based lottery. The authors compared performance scores of students from the 2 groups on all 7 CanMEDS roles, using assessment data gathered during clinical rotations. The authors examined data from 3 cohorts (2011-2013).

RESULTS

Students admitted through the local, outcomes-based selection procedure significantly outperformed the initially rejected but lottery-admitted students in all years, and the differences between groups increased over time. The selected students performed significantly better in the CanMEDS roles of Communicator, Collaborator, and Professional in the first year of clinical rotations; in these 3 roles-plus Organizer-in the second year; and in 2 additional roles (Advocate and Scholar-all except Medical Expert) at the end of their clinical training.

CONCLUSIONS

A constructively aligned selection procedure has increasing predictive value across the clinical years of medical school compared with a GPA-based lottery procedure. The data reported here suggest that constructive alignment of selection, curricula, and assessment to ultimate outcomes is effective in creating a selection procedure predictive of clinical performance.

University Admission Test Associates with Academic Performance at the End of Medical Course in a PBL Medical Hybrid Curriculum

PURPOSE

Most studies assessing the value of the university admissions test (UAT) to predict academic performance at the end of a medical course were carried out on lecture-based medical courses. However, the association between performance in the UAT with academic achievement at the end of medical course in a problem-based learning (PBL) medical hybrid curriculum remains controversial. The aim of this study was to correlate marks in the UAT with those obtained in the Organized Structured Clinical Examination (OSCE), in the progress testing (PT), and in the final marks of the clerkship (FMC).

METHODS

We used data from 48 medical students. A single and a multiple dependency studies were performed to assess bivariate and multiple correlation between the UAT or the essay scores (dependent variables) and the OSCE, PT, and FMC (independent variables). Pearson test, multiple linear regression, and ANOVA tests were used and a p-value < 0.05 was considered significant.

RESULTS

In the bivariate analysis, only the UAT and FMC marks were correlated (r=0.34; p=0.02). However, the multiple dependency study showed a moderate correlation among UAT, OSCE, PT, and FMC marks (r=0.46; p=0.01). No correlation was found between the essay scores and PT, FMC, and OSCE scores.

CONCLUSION

Our study shows that UAT marks, but not essay scores, can predict academic achievement, particularly in terms of clinical competence (FMC) at the end of a medical course in a PBL hybrid curriculum.

Reliability of personality and values tests: The effects of "high stakes" selection conditions, and of four years in medical school

This study investigated if scores on tests of personal qualities are affected by whether they will determine selection decisions ("high stakes") or not; and whether they are stable for individuals and groups across a four-year medical course. Two tests, one assessing values and one assessing components of personality, were administered either at the same time as a medical university entrance exam (first cohort; N = 216), or after entry was confirmed (second cohort; N = 142). Both cohorts took the tests again after four years of medical school. Analysis of variance was used to compare group mean scores and interactions, and correlation coefficients to measure temporal reliability. The high stakes cohort initially presented themselves in a significantly more positive light on the personality test. After four years of medical school scores on both tests changed significantly, towards more communitarian values and less empathic attitudes. Thus, personality scores were affected by both the conditions under which the initial tests were conducted and by the passage of time, but values only by the passage of time. Before and after scores were significantly correlated.

Bachelor completion and dropout rates of selected, rejected and lottery-admitted medical students in the Netherlands

BACKGROUND

Evidence for the effectiveness of the selection of medical students is weak. This study aimed to examine the added value of a two-step selection procedure (first step non-academic, second step academic tests) to a pre-university GPA-based lottery procedure. Because previous research has suggested that participation in selection (regardless of the outcome) is a predictor of study success, this study is the first to include students who initially applied for selection, then refrained from (actively) participating in selection and were eventually admitted through lottery.

METHODS

Bachelor completion and dropout rates of selected (n = 416) and lottery-admitted students from four cohorts (2006-2009) were compared using logistic regression analysis. Four groups of lottery-admitted students were distinguished: students who were rejected after step 2 (n = 57), were rejected after step 1 (n = 169), withdrew during selection step 1 (n = 42) and students who only applied for lottery (n = 366). Covariates included gender, age, pre-university GPA and cohort.

RESULTS

There was a significant association between admission group and obtaining a bachelor degree in three years. Selected students were more likely to obtain a bachelor degree within three years (64.2% versus 51.6%; OR = 1.7) or four years (81.5% versus 74.3%; OR = 1.6) than students who only applied to a lottery (p <  0.05); selected students also seemed more likely to obtain all Year-1 course credits than students who withdrew during step 1 (40.4% versus 21.4%; OR = 2.3; p <  0.05). We found no significant association between dropout and admission groups. Students rejected at step 1 or 2 did not perform significantly different from selected students on any of the outcome measures.

CONCLUSIONS

The findings indicated that students at risk for study delay in the preclinical phase in our context were more likely to refrain from applying to a demanding selection procedure when a less demanding alternative was available. We found no significant associations between the non-academic and academic selection steps and bachelor completion and dropout rates. These findings suggest that the presence of the selection was more important than these specific selection components. In follow-up research, we plan to investigate the associations between the admission groups and outcome measures in the clinical phase.

Evaluation of critical thinking course for premedical students using literature and film

PURPOSE

Premedical education is one stage in which various educational approaches are used to promote critical thinking. Given that critical thinking ability could be regarded as one of the intended outcomes of social science and humanities education, this study explored the effectiveness of a course to promote critical thinking in a premedical curriculum using both literature and film.

METHODS

Fifty-one 2nd year premedical students enrolled in a 'Critical Thinking for Premeds' course. Students were required to read or watch a selected material, submit group discussion agendas, attend five group discussion sessions, and write critical essays. Five tutors facilitated the group discussions, observed and assessed the students' performance and critical essay. Students' critical thinking disposition and opinion on assigned reading materials were examined before and after the course. A program evaluation survey was conducted to investigate the students' reaction after the course.

RESULTS

On average, students appreciated 78.6% of the total assigned materials. The students indicated that group discussions and the narrative comments of facilitators contributed the most to develop critical thinking. After the course, the students' tendency preferring cheaper price books, as well as medicine-related books decreased significantly. Students who had critical essay scores greater than or equal to the median demonstrated a significant improvement in critical thinking disposition scores.

CONCLUSION

The course was well-accepted by premedical students and had several positive outcomes. A more effective use of the course could be anticipated with a clearer explanation of the purpose, the consideration of previous reading experience, and use of complementary learning activities.

Selection and lottery in medical school admissions: who gains and who loses?

This article was migrated. The article was marked as recommended. Concerns related to fairness of medical school admissions through selection have led some scholars to consider selection as an expensive lottery and suggest that lottery may be fairer. This paper considers the issue of selection versus lottery from the perspectives of three groups of stakeholders: 1) applicants, 2) medical schools, and 3) society. This paper contributes to the discussion by addressing advantages and disadvantages of the use of selection and lottery for these stakeholder groups, grounded in the findings from research. Themes that are discussed are reliability and validity issues, perceived influence on selection outcomes and student uptake, effects on student diversity, financial costs, impact on rejected applicants, transparency, and strategic behaviour. For each stakeholder group both lottery and selection yield a combination of advantages and disadvantages, which implies that none of the currently available admissions strategies completely fulfils stakeholders' needs. Research indicates that selection yields only small gains compared to a lottery procedure, while the student diversity, necessary for serving the increasingly diverse patient population, may be compromised. We argue that society's needs should drive admissions policies rather than institutional gains, which means that until a selection procedure is developed that does not disadvantage certain types of students, a lottery procedure should be preferred.

Does selection pay off? A cost-benefit comparison of medical school selection and lottery systems

CONTEXT

Resources for medical education are becoming more constrained, whereas accountability in medical education is increasing. In this constrictive environment, medical schools need to consider and justify their selection procedures in terms of costs and benefits. To date, there have been no studies focusing on this aspect of selection.

OBJECTIVES

We aimed to examine and compare the costs and benefits of two different approaches to admission into medical school: a tailored, multimethod selection process versus a lottery procedure. Our goal was to assess the relative effectiveness of each approach and to compare these in terms of benefits and costs from the perspective of the medical school.

METHODS

The study was conducted at Maastricht University Medical School, at which the selection process and a weighted lottery procedure ran in parallel for 3 years (2011-2013). The costs and benefits of the selection process were compared with those of the lottery procedure over three student cohorts throughout the Bachelor's programme. The extra costs of selection represented the monetary investment of the medical school in conducting the selection procedure; the benefits were derived from the increase in income generated by the prevention of dropout and the reductions in extra costs facilitated by decreases in the repetition of blocks and objective structured clinical examinations.

RESULTS

The tailor-made selection procedure cost about €139 000 when extrapolated to a full cohort of students (n = 286). The lottery procedure came with negligible costs for the medical school. However, the average benefits of selection compared with the lottery system added up to almost €207 000.

CONCLUSIONS

This study not only shows that conducting a cost-benefit comparison is feasible in the context of selection for medical school, but also that an 'expensive' selection process can be cost-beneficial in comparison with an 'inexpensive' lottery system. We encourage other medical schools to examine the cost-effectiveness of their own selection processes in relation to student outcomes in order to extend knowledge on this important topic.

2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions

Selection and recruitment into healthcare education and practice is a key area of interest for educators with significant developments in research, policy, and practice in recent years. This updated consensus statement, developed through a multi-stage process, examines future opportunities and challenges in selection and recruitment. There is both a gap in the literature around and a compelling case for further theoretical and empirical literature to underpin the development of overall selection philosophes and policies and their enactment. More consistent evidence has emerged regarding the quality of different selection methods. Approaches to selection are context-dependent, requiring the consideration of an institution's philosophy regarding what they are trying to achieve, the communities it purports to serve, along with the system within which they are used. Diversity and globalization issues continue to be critically important topics. Further research is required to explore differential attainment and explain why there are substantial differences in culturally acceptable ways of approaching diversity and widening access. More sophisticated evaluation approaches using multi-disciplinary theoretical frameworks are required to address the issues. Following a discussion of these areas, 10 recommendations are presented to guide future research and practice and to encourage debate between colleagues across the globe.