Differential attainment in the MRCPsych according to ethnicity and place of qualification between 2013 and 2018: a UK cohort study

Re-examining single-moment-in-time high-stakes examinations in specialist training: A critical narrative review

In this critical narrative review, we challenge the belief that single-moment-in-time high-stakes examinations (SMITHSEx) are an essential component of contemporary specialist training. We explore the arguments both for and against SMITHSEx, examine potential alternatives, and discuss the barriers to change.SMITHSEx are viewed as the "gold standard" assessment of competence but focus excessively on knowledge assessment rather than capturing essential competencies required for safe and competent workplace performance. Contrary to popular belief, regulatory bodies do not mandate SMITHSEx in specialist training. Though acting as significant drivers of learning and professional identity formation, these attributes are not exclusive to SMITHSEx.Skills such as crisis management, procedural skills, professionalism, communication, collaboration, lifelong learning, reflection on practice, and judgement are often overlooked by SMITHSEx. Their inherent design raises questions about the validity and objectivity of SMITHSEx as a measure of workplace competence. They have a detrimental impact on trainee well-being, contributing to burnout and differential attainment.Alternatives to SMITHSEx include continuous low-stakes assessments throughout training, ongoing evaluation of competence in the workplace, and competency-based medical education (CBME) concepts. These aim to provide a more comprehensive and context-specific assessment of trainees' competence while also improving trainee welfare.Specialist training colleges should evolve from exam providers to holistic education sources. Assessments should emphasise essential practical knowledge over trivia, align with clinical practice, aid learning, and be part of a diverse toolkit. Eliminating SMITHSEx from specialist training will foster a competency-based approach, benefiting future medical professionals' well-being and success.

Differential attainment in specialty training recruitment in the UK: an observational analysis of the impact of psychometric testing assessment in Public Health postgraduate selection

OBJECTIVES

To determine how current psychometric testing approaches used in selection of postgraduate training in UK Public Health are associated with socioeconomic and sociocultural background of applicants (including ethnicity).

DESIGN

Observational study using contemporaneous data collected during recruitment and psychometric test scores.

SETTING

Assessment centre of UK national Public Health recruitment for postgraduate Public Health training. The assessment centre element of selection comprises three psychometric assessments: Rust Advanced Numerical Reasoning, Watson-Glaser Critical Thinking Assessment II and Public Health situational judgement test.

PARTICIPANTS

629 applicants completed the assessment centre in 2021. 219 (34.8%) were UK medical graduates, 73 (116%) were international medical graduates and 337 (53.6%) were from backgrounds other than medicine.

MAIN OUTCOME MEASURE

Multivariable-adjusted progression statistics in the form of adjusted OR (aOR), accounting for age, sex, ethnicity, professional background and surrogate measures of familial socioeconomic and sociocultural status.

RESULTS

357 (56.8%) candidates passed all three psychometric tests. Candidate characteristics negatively associated with progression were black ethnicity (aOR 0.19, 0.08 to 0.44), Asian ethnicity (aOR 0.35, 0.16 to 0.71) and coming from a non-UK medical graduate background (aOR 0.05, 0.03 to 0.12); similar differential attainment was observed in each of the psychometric tests. Even within the UK-trained medical cohort, candidates from white British backgrounds were more likely to progress than those from ethnic minorities (89.2% vs 75.0%, p=0.003).

CONCLUSION

Although perceived to mitigate the risks of conscious and unconscious bias in selection to medical postgraduate training, these psychometric tests demonstrate unexplained variation that suggests differential attainment. Other specialties should enhance their data collection to evaluate the impact of differential attainment on current selection processes and take forward opportunities to mitigate differential attainment where possible.

Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study

Objective

A recent independent review on diversity and inclusivity highlighted concerns that barriers to surgical career progression exist for some groups of individuals and not others. Group-level differences in performance at the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examinations have been identified but are yet to be investigated. We aimed to characterise the relationship between sociodemographic differences and performance at MRCS.

Design

Retrospective cohort study.

Setting

Secondary care.

Participants

All UK MRCS candidates attempting Part A (n = 5780) and Part B (n = 2600) between 2013 and 2019 with linked sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk).

Main outcome measures

Chi-square tests established univariate associations with MRCS performance. Multiple logistic regression identified independent predictors of success, adjusted for medical school performance.

Results

Statistically significant differences in MRCS pass rates were found according to gender, ethnicity, age, graduate status, educational background and socioeconomic status (all p < 0.05). After adjusting for prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were independent predictors of MRCS Part A success and being a non-graduate (OR 1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51, 95% CI 1.08–2.10) were independent predictors of Part B success. Black and minority ethnic groups were significantly less likely to pass MRCS Part B at their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR 0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White candidates.

Conclusions

There is significant group-level differential attainment at MRCS, likely to represent the accumulation of privilege and disadvantage experienced by individuals throughout their education and training. Those leading surgical education now have a responsibility to identify and address the causes of these attainment differences.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination

The COVID-19 pandemic has affected how clinical examinations are conducted, resulting in the Royal College of Psychiatrists delivering the Clinical Assessment of Skills and Competence virtually. Although this pragmatic step has allowed for progression of training, it has come at the cost of a significantly altered examination experience. This article aims to explore the fairness of such an examination, the difference in trainee experience, and the use of telemedicine to consider what might be lost as well as gained at a time when medical education and delivery of healthcare are moving toward the digitised frontier.

Uncovering trends in training progression for a national cohort of psychiatry trainees: discrete-time survival analysis

BACKGROUND

The global rise in mental health issues calls for a strong psychiatry workforce. Yet, psychiatry training worldwide is facing recruitment challenges, causing unfilled consultant posts and possibly threatening the quality of patient care. An in-depth understanding of trainees' progression through training is warranted to explore what happens to recruited trainees during training.

AIMS

To uncover current trends in psychiatry trainees' progression through training in the UK.

METHOD

This national retrospective cohort study with data from the UK Medical Education Database used discrete-time survival analysis to analyse training progression for those trainees who started their core psychiatry post in 2012-2017 (2820 trainees; 59.6% female, 67.6% UK graduates (UKGs)). The impact of sociodemographic characteristics on training progression were also investigated.

RESULTS

The overall probability of completing training in 6 years (minimum years required to complete psychiatry training in the UK) was 17.2% (ranging from 4.8% for non-UKG females to 29% for UKG males). The probability to not progress was highest (57.1%) from core to specialty training. For UKGs, trainees from ethnicities other than White, trainees with a disability, and trainees who had experienced childhood social deprivation (measured as entitlement to free school meals) had a significantly (P ≤ 0.02) lower probability of completing training in 6 years.

CONCLUSIONS

Less than one in five psychiatry trainees are likely to complete training in 6 years and this probability varies across groups of doctors. Completing psychiatry training in 6 years is, therefore, the exception rather than the norm and this has important implications for trainees, those planning psychiatry workforces or responsible for psychiatry training.