The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study

Re-examining single-moment-in-time high-stakes examinations in specialist training: A critical narrative review

In this critical narrative review, we challenge the belief that single-moment-in-time high-stakes examinations (SMITHSEx) are an essential component of contemporary specialist training. We explore the arguments both for and against SMITHSEx, examine potential alternatives, and discuss the barriers to change.SMITHSEx are viewed as the "gold standard" assessment of competence but focus excessively on knowledge assessment rather than capturing essential competencies required for safe and competent workplace performance. Contrary to popular belief, regulatory bodies do not mandate SMITHSEx in specialist training. Though acting as significant drivers of learning and professional identity formation, these attributes are not exclusive to SMITHSEx.Skills such as crisis management, procedural skills, professionalism, communication, collaboration, lifelong learning, reflection on practice, and judgement are often overlooked by SMITHSEx. Their inherent design raises questions about the validity and objectivity of SMITHSEx as a measure of workplace competence. They have a detrimental impact on trainee well-being, contributing to burnout and differential attainment.Alternatives to SMITHSEx include continuous low-stakes assessments throughout training, ongoing evaluation of competence in the workplace, and competency-based medical education (CBME) concepts. These aim to provide a more comprehensive and context-specific assessment of trainees' competence while also improving trainee welfare.Specialist training colleges should evolve from exam providers to holistic education sources. Assessments should emphasise essential practical knowledge over trivia, align with clinical practice, aid learning, and be part of a diverse toolkit. Eliminating SMITHSEx from specialist training will foster a competency-based approach, benefiting future medical professionals' well-being and success.

Examining the diversity of MRCS examiners

BACKGROUND

MRCS examiners are the face of the Royal College of Surgeons for early-career surgeons and should therefore represent the workforce they are examining as not to marginalise or negatively impact on the assessment experience of candidates from minoritised groups. This study aimed to explore the diversity of MRCS examiners and whether they represent the demographics of the MRCS candidates.

METHODS

A retrospective observational study including all active examiners and examination candidates who attempted MRCS Part A or Part B between January 2020 and July 2021. Self-declared demographic data collected by the Intercollegiate Committee for Basic Surgical Examinations (ICBSE) included gender, sexual orientation, disability status and ethnicity. Following data anonymisation, total group response frequencies were made available to the research team for statistical analysis.

RESULTS

Chi-squared analyses showed statistically significant differences in the representation of gender, disability and ethnicity between candidates and examiners (all p < 0.001). Men (83.9% (n = 1121) vs 70.9% (n = 6017) respectively), individuals without disability (98.7% (n = 917) vs 96.1% (n = 6847)) and individuals of White ethnicity (36.6% (n = 346) vs 20.4% (n = 1223)) were significantly overrepresented in the examiners compared to the examination candidates. There was no statistically significant difference in sexual orientation between examiners and candidates (p = 0.712).

CONCLUSIONS

Broadly speaking, the socio-demographic profile of MRCS examiners reflects that seen in senior and leadership positions in surgery in the UK - that is, predominantly male and White - but not that seen in early-career surgeons. Positive action is now required in examiner recruitment by the Royal Colleges to ensure that the cohort of MRCS examiners reflects the modern surgical workforce.

Do medical students with a disability experience adverse educational outcomes on UK medical courses?

BACKGROUND

International data demonstrate that medical students with disabilities experience differential awarding. One cross-sectional study finds lower graduation rates and scores in written exams, with no impact on clinical exams. Disabled students graduated with lower decile scores. This quantitative, retrospective, longitudinal study explored the impact of disability on exam performance, course performance and course discontinuation.

METHOD

Anonymised data were obtained for 1743 students on a UK graduate medical programme from 2011 to present. Statistical tests, including t-tests and one-way ANOVA were conducted for main effects of demographic variables on exam results and categorical outcomes. Regression models established the effects of variables and sub-categories of variables on results and categorical outcomes.

RESULTS

Significant main effects of disability on exam scores were identified, as well as failure probability. Regressions showed significant differences in outcomes between different types of disability, with mental health conditions predicting course discontinuation. A significant amplifying effect was found for BAME students with disability.

CONCLUSION

Disability had a significant negative impact on all course outcomes, illustrating inequity in medical training and an area of focus for curriculum development. Intersectional data identified a key disadvantaged subgroup of medical students.

Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study

Objective

A recent independent review on diversity and inclusivity highlighted concerns that barriers to surgical career progression exist for some groups of individuals and not others. Group-level differences in performance at the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examinations have been identified but are yet to be investigated. We aimed to characterise the relationship between sociodemographic differences and performance at MRCS.

Design

Retrospective cohort study.

Setting

Secondary care.

Participants

All UK MRCS candidates attempting Part A (n = 5780) and Part B (n = 2600) between 2013 and 2019 with linked sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk).

Main outcome measures

Chi-square tests established univariate associations with MRCS performance. Multiple logistic regression identified independent predictors of success, adjusted for medical school performance.

Results

Statistically significant differences in MRCS pass rates were found according to gender, ethnicity, age, graduate status, educational background and socioeconomic status (all p < 0.05). After adjusting for prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were independent predictors of MRCS Part A success and being a non-graduate (OR 1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51, 95% CI 1.08–2.10) were independent predictors of Part B success. Black and minority ethnic groups were significantly less likely to pass MRCS Part B at their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR 0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White candidates.

Conclusions

There is significant group-level differential attainment at MRCS, likely to represent the accumulation of privilege and disadvantage experienced by individuals throughout their education and training. Those leading surgical education now have a responsibility to identify and address the causes of these attainment differences.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.