Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK

The educational value of situational judgement tests (SJTs) when used during undergraduate medical training: A systematic review and narrative synthesis

INTRODUCTION

Situational judgement tests (SJTs) are a recognised assessment method for admission into medical school, selection into postgraduate training programs, and postgraduate competency assessment. More recently, however, SJTs have been used during undergraduate medical training (UMT). This systematic review identifies, describes, and appraises the evidence for SJTs in UMT to determine educational associations and outcomes.

METHODS

MEDLINE, EMBASE, ERIC, PsycINFO, SCOPUS, Web of Science, and grey literature were searched for original research studies evaluating SJTs implemented within UMT to 1 November 2022. Studies reporting evaluation outcomes were included. Narrative data syntheses were undertaken. Risk of Bias was appraised using the Quality in Prognosis Studies tool.

RESULTS

24 studies were included. National database-derived SJTs (n = 14) assessed against professionalism, postgraduate attainment, construct of medical degree, medical school admissions scores, personality attributes, and declaration of disability. In-house derived SJTs (n = 10) assessed against professionalism, clinical skills, and personality attributes. Most evidence evaluated and reported inverse SJT associations with professionalism and were moderate risk of bias.

CONCLUSION

SJTs may have utility for developing professional behaviours in medical students. However, further research testing SJT robustness, standard setting methodologies, and prospectively evaluating SJTs against objective outcome measures within the context of UMT is warranted.

Prior degree and academic performance in medical school: evidence for prioritising health students and moving away from a bio-medical science-focused entry stream

BACKGROUND

Given the importance of the selection process, many medical schools are reviewing their selection criteria. The traditional pathway for post-graduate medicine has been from science-based undergraduate degrees, however some programs are expanding their criteria. In this study we investigated academic success across all years and themes of the Deakin University medical degree, based on the type of degree undertaken prior to admission. We evaluated whether the traditional pathway of biomedical science into medicine should remain the undergraduate degree of choice, or whether other disciplines should be encouraged.

METHODS

Data from 1159 students entering the degree from 2008 to 2016 was collected including undergraduate degree, grade point average (GPA), Graduate Medical Schools Admission Test (GAMSAT) score and academic outcomes during the 4 years of the degree. Z-scores were calculated for each assessment within each cohort and analysed using a one sample t-test to determine if they differed from the cohort average. Z-scores between groups were analysed by 1-way ANOVA with LSD post-hoc analysis correcting for multiple comparisons.

RESULTS

The majority of students had Science (34.3%) or Biomedical Science (31.0%) backgrounds. 27.9% of students had a Health-related undergraduate degree with smaller numbers of students from Business (3.5%) and Humanities (3.4%) backgrounds. At entry, GPA and GAMSAT scores varied significantly with Biomedical Science and Science students having significantly higher scores than Health students. Health students consistently outperformed students from other disciplines in all themes while Biomedical Science students underperformed.

CONCLUSIONS

Our data suggest that a Health-related undergraduate degree results in the best performance throughout medical school, whereas a Biomedical Science background is associated with lower performance. These findings challenge the traditional Biomedical Science pathway into medicine and suggest that a health background might be more favourable when determining the selection criteria for graduate entry into medicine.

Publication in the Australian medical student journal is associated with future academic success: a matched-cohort study

BACKGROUND

Medical student journals (MSJs) help to introduce the fundamentals of academic research and publication to future doctors. It has recently been shown that MSJs can influence doctors' future academic and professional success, however these findings have not been replicated in an Australian cohort. The aim of this study was to examine the association between publication in the Australian Medical Student Journal (AMSJ) and markers of a student's future academic trajectory, including future publication, attainment of higher academic degree or entry into specialist training.

METHODS

Articles authored by medical students in the AMSJ from 2010 to 2015 were retrospectively identified. A list of these student authors was made, with university- and year- matched control students randomly selected from university graduation databases. For all students, data related to academic success were obtained from multiple sources including PubMed®, Google, university databases and author affiliation information from publications. A multivariable conditional logistic regression model was used to assess correlation between variables. The primary outcome measure was the number of postgraduate PubMed®-indexed publications. Secondary outcome measures included attainment of higher degree or faculty position. Clinical speciality was also recorded.

RESULTS

Fifty-five AMSJ authors (14 case reports, 17 original research, 21 review articles) from 14 Australian universities were included. Publication in the AMSJ was associated with future PubMed® indexed publications (OR 3.43, 95% CI 1.74-6.77, P < 0.001) and higher degree attainment (OR 4.05, 95% CI 1.99-8.22, P = 0.0001). AMSJ authors were also significantly more likely to enter into surgical training (OR 2.53, 95% CI 1.10-5.84, P = 0.029). A multivariable conditional logistic regression model demonstrated that publication in the AMSJ was predictive of future PubMed indexed publication, independent of higher degree or faculty position attainment (OR 2.56, CI 1.22-5.39, P = 0.01).

CONCLUSION

We have shown that publication in a MSJ is associated with markers of academic success in an Australian cohort. PubMed®-indexed publications, attainment of a Masters degree, and entry in to surgical training were all significantly correlated to AMSJ publication. A conditional logistic regression model demonstrated that medical student publication in the AMSJ influences the number of future PubMed®-indexed publications, independent of major academic confounding variables.

Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis

BACKGROUND

The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels.

METHODS

A literature search was conducted in April 2020 using eight electronic databases: MEDLINE, APA PsycInfo, SCOPUS, Web of Science, EThOS, OpenGrey, PROSPERO, and the UCAT website. Data were extracted from selected studies and tabulated as results matrices. A narrative synthesis was performed.

RESULTS

Twenty-four studies satisfied our inclusion criteria, 23 of which were deemed to be of good quality (using the Newcastle-Ottawa Scale). For over 70% of univariate data points, the UCAT exerted no statistically significant predictive validity; for the remainder, predictive power was weak. The cognitive total and verbal reasoning subtests had the largest evidence base as weakly positive predictors of academic performance. The SJT subtest was a weak predictor of professional behaviour during medical school. Studies specific to dental schools demonstrated variable findings across the five studies. Only 1 study looked at post-graduate outcome measures and demonstrated that the UCAT was not a predictor of health- or conduct-related fitness to practice declarations at GMC registration.

CONCLUSIONS

These data provide some support for the use of cognitive total and verbal reasoning subtests as part of medical school selection. Further research is needed to investigate outcomes beyond professional registration and for dental students.

Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002-2018 (UKMED54)

OBJECTIVES

To examine factors associated with declaration of disability by medical students and doctors, and the association of declared disability with academic performance.

DESIGN

Observational study using record-linked data collected between 2002 and 2018.

SETTING

UK Medical Education Database is a repository of data relating to training of medical students and doctors. Disability and other data are record-linked.

PARTICIPANTS

All students starting at a UK medical school between 2002 and 2018 (n=135 930).

MAIN OUTCOME MEASURES

Declared disability was categorised by the Higher Education Statistics Authority. Outcomes related to undergraduate academic performance included scores in the educational performance measure (EPM), prescribing safety assessment and situational judgement test. Performance in postgraduate examinations was studied, as well as prior attainment in school examinations and aptitude tests.

RESULTS

Specific learning disability (SLD) was the most commonly declared disability (3.5% compared with the next most commonly declared disability at 1.0% of n=129 345 all cases in the study), and during the period covered by the data, SLD declarations increased from 1.4% (n=6440 for students starting in 2002) to 4.6% (n=8625 for students starting in 2018). In a logistic regression, the following factors predicted recording of SLD on entry to medical school ((exp(B)±95% CI), p<0.0001 unless otherwise stated): attendance at a fee-paying school (2.306±0.178), graduate status (1.806±0.205), participation of local areas quintile (1.089±0.030), age (1.034±0.012). First year medical students were less likely to declare SLD if they were from a non-white ethnic background (Asian/Asian British 0.324±0.034, black/black British 0.571±0.102, mixed 0.731±0.108, other ethnic groups 0.566±0.120), female (0.913±0.059; p=0.007) or from a low index of multiple deprivation quintile (0.963±0.029); p=0.017. In univariate analysis with Bonferroni corrections applied for multiple tests, no significant difference was observed in the recording of SLD according to socioeconomic class (χ²=5.637, p=1), whether or not a student's parents had a higher education (χ²=0.140, p=1), or whether or not a student had received a United Kingdom Clinical Aptitude Test (UKCAT) bursary (χ²=7.661, p=0.068). Students who declared SLD at some point in medical school (n=4830) had lower EPM normalised deviate values (-0.390) than those who did not (-0.119) (F=189.872, p<0.001). Those for whom SLD was recorded were as likely to complete the course successfully as those who did not declare disability (93.0% successful completion by those for whom SLD declared from year 1 (n=2480), 92.2% by those for whom SLD declared after year 1 (n=2350), 91.6% by those for whom SD not declared at any point (n=85 180)) (χ²=6.905, p=0.032). Of 3580 first year students who declared SLD, 43.1% had not sat the UKCAT Special Educational Needs aptitude test (which gives extra time for those with special educational needs), while 28% of 2400 registrants for whom SLD was recorded as medical students did not declare it at General Medical Council registration.

CONCLUSIONS

Substantial increases in declaration of SLD may reflect changes in the social and legal environment during the period of the study. Those who declare SLD are just as likely to gain a primary medical qualification as those who do not. For some individuals, disability declaration appears to depend on context, based on differences in numbers declaring SLD before, during and after medical school.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

Predictive validity of A-level grades and teacher-predicted grades in UK medical school applicants: a retrospective analysis of administrative data in a time of COVID-19

OBJECTIVES

To compare in UK medical students the predictive validity of attained A-level grades and teacher-predicted A levels for undergraduate and postgraduate outcomes. Teacher-predicted A-level grades are a plausible proxy for the teacher-estimated grades that replaced UK examinations in 2020 as a result of the COVID-19 pandemic. The study also models the likely future consequences for UK medical schools of replacing public A-level examination grades with teacher-predicted grades.

DESIGN

Longitudinal observational study using UK Medical Education Database data.

SETTING

UK medical education and training.

PARTICIPANTS

Dataset 1: 81 202 medical school applicants in 2010-2018 with predicted and attained A-level grades. Dataset 2: 22 150 18-year-old medical school applicants in 2010-2014 with predicted and attained A-level grades, of whom 12 600 had medical school assessment outcomes and 1340 had postgraduate outcomes available.

OUTCOME MEASURES

Undergraduate and postgraduate medical examination results in relation to attained and teacher-predicted A-level results.

RESULTS

Dataset 1: teacher-predicted grades were accurate for 48.8% of A levels, overpredicted in 44.7% of cases and underpredicted in 6.5% of cases. Dataset 2: undergraduate and postgraduate outcomes correlated significantly better with attained than with teacher-predicted A-level grades. Modelling suggests that using teacher-estimated grades instead of attained grades will mean that 2020 entrants are more likely to underattain compared with previous years, 13% more gaining the equivalent of the lowest performance decile and 16% fewer reaching the equivalent of the current top decile, with knock-on effects for postgraduate training.

CONCLUSIONS

The replacement of attained A-level examination grades with teacher-estimated grades as a result of the COVID-19 pandemic may result in 2020 medical school entrants having somewhat lower academic performance compared with previous years. Medical schools may need to consider additional teaching for entrants who are struggling or who might need extra support for missed aspects of A-level teaching.

Is Academic Attainment or Situational Judgment Test Performance in Medical School Associated With the Likelihood of Disciplinary Action? A National Retrospective Cohort Study

PURPOSE

Disciplinary action imposed on physicians indicates their fitness to practice medicine is impaired and patient safety is potentially at risk. This national retrospective cohort study sought to examine whether there was an association between academic attainment or performance on a situational judgment test (SJT) in medical school and the risk of receiving disciplinary action within the first 5 years of professional practice in the United Kingdom.

METHOD

The authors included data from the UK Medical Education Database for 34,865 physicians from 33 U.K. medical schools that started the UK Foundation Programme (similar to internship) between 2014 and 2018. They analyzed data from 2 undergraduate medical assessments used in the United Kingdom: the Educational Performance Measure (EPM), which is based on academic attainment, and SJT, which is an assessment of professional attributes. The authors calculated hazard ratios (HRs) for EPM and SJT scores.

RESULTS

The overall rate of disciplinary action was low (65/34,865, 0.19%) and the mean time to discipline was 810 days (standard deviation [SD] = 440). None of the physicians with fitness to practice concerns identified as students went on to receive disciplinary action after they qualified as physicians. The multivariate survival analysis demonstrated that a score increase of 1 SD (approximately 7.6 percentage points) on the EPM reduced the hazard of disciplinary action by approximately 50% (HR = 0.51; 95% confidence interval [CI]: 0.38, 0.69; P < .001). There was not a statistically significant association between the SJT score and the hazard of disciplinary action (HR = 0.84; 95% CI: 0.62, 1.13; P = .24).

CONCLUSIONS

An increase in EPM score was significantly associated with a reduced hazard of disciplinary action, whereas performance on the SJT was not. Early identification of increased risk of disciplinary action may provide an opportunity for remediation and avoidance of patient harm.

Mental illness and suicide among physicians

The COVID-19 pandemic has heightened interest in how physician mental health can be protected and optimised, but uncertainty and misinformation remain about some key issues. In this Review, we discuss the current literature, which shows that despite what might be inferred during training, physicians are not immune to mental illness, with between a quarter and a third reporting increased symptoms of mental ill health. Physicians, particularly female physicians, are at an increased risk of suicide. An emerging consensus exists that some aspects of physician training, working conditions, and organisational support are unacceptable. Changes in medical training and health systems, and the additional strain of working through a pandemic, might have amplified these problems. A new evidence-informed framework for how individual and organisational interventions can be used in an integrated manner in medical schools, in health-care settings, and by professional colleagues is proposed. New initiatives are required at each of these levels, with an urgent need for organisational-level interventions, to better protect the mental health and wellbeing of physicians.

Predictors of ophthalmology career success (POCS) study

Objective

Ophthalmology is the busiest outpatient specialty with demand predicted to rise over 40% in the next 20 years. A significant increase in the number of trainee ophthalmologists is required to fill currently vacant consultant posts and meet the UK's workforce demands by 2038. Our aim was to understand what determines success in ophthalmology training, in order to inform future ophthalmologists, refine recruitment and facilitate workforce planning.

Methods and Analysis

This was a retrospective longitudinal cohort study using routinely collected data available from UK Medical Education Database (UKMED) (https://www.ukmed.ac.uk/). Data were analysed on 1350 candidates who had applied for ophthalmology specialty training (OST) between 2012 and 2018, as well as 495 candidates who had attempted Fellow of the Royal College of Ophthalmologists (FRCOphth) Part 1 between 2013 and 2018. Participants who had not obtained their primary medical qualification from the UK medical schools were excluded. Primary outcome measures included gaining a place on the OST programme and passing the FRCOphth Part 1 examination on first attempt.

Results

Higher education performance measure decile scores at medical school are strongly predictive in securing an OST post and passing the part 1 examination first time (p<0.001). Candidates who attempt FRCOphth Part 1 prior to their ST1 application are more likely to get a place on OST on first attempt. Socioeconomic factors, gender and ethnicity do not influence success in OST entry. Male trainees are more likely to pass FRCOphth Part 1 on their first attempt.

Conclusion

This study is the first quantitative assessment of the factors that determine success in OST recruitment and ophthalmology postgraduate examinations in the UK. Similar studies should be undertaken in all other medical and surgical specialties to understand what factors predict success.

Factors relating to working hours restriction that have impacted the professional identity of trainees in the last decade

Ever-developing changes to the working hours of junior doctors by the European Working Time Directive, the junior doctor contract of 2019 and most recently the COVID-19 pandemic have impacted the professional identity of doctors. There has been little investigation into its influence on the multifaceted aspects of postgraduate medical training, which feeds into how trainees consider themselves professionally and the concept of professional identity or 'being a doctor'. A review of the medical, socio-political and educational literature reveals that the impact on the professional identity development of trainees is influenced by several perspectives from the trainee, trainer and the public. Gross reduction in working hours has no doubt decreased the raw volume of clinical experiences. However, to counteract this, smarter learning processes have evolved, including narrative reflection, supervised learning events, and a greater awareness of coaching and training among trainers.

Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise

BACKGROUND

Medical schools differ, particularly in their teaching, but it is unclear whether such differences matter, although influential claims are often made. The Medical School Differences (MedDifs) study brings together a wide range of measures of UK medical schools, including postgraduate performance, fitness to practise issues, specialty choice, preparedness, satisfaction, teaching styles, entry criteria and institutional factors.

METHOD

Aggregated data were collected for 50 measures across 29 UK medical schools. Data include institutional history (e.g. rate of production of hospital and GP specialists in the past), curricular influences (e.g. PBL schools, spend per student, staff-student ratio), selection measures (e.g. entry grades), teaching and assessment (e.g. traditional vs PBL, specialty teaching, self-regulated learning), student satisfaction, Foundation selection scores, Foundation satisfaction, postgraduate examination performance and fitness to practise (postgraduate progression, GMC sanctions). Six specialties (General Practice, Psychiatry, Anaesthetics, Obstetrics and Gynaecology, Internal Medicine, Surgery) were examined in more detail.

RESULTS

Medical school differences are stable across time (median alpha = 0.835). The 50 measures were highly correlated, 395 (32.2%) of 1225 correlations being significant with p < 0.05, and 201 (16.4%) reached a Tukey-adjusted criterion of p < 0.0025. Problem-based learning (PBL) schools differ on many measures, including lower performance on postgraduate assessments. While these are in part explained by lower entry grades, a surprising finding is that schools such as PBL schools which reported greater student satisfaction with feedback also showed lower performance at postgraduate examinations. More medical school teaching of psychiatry, surgery and anaesthetics did not result in more specialist trainees. Schools that taught more general practice did have more graduates entering GP training, but those graduates performed less well in MRCGP examinations, the negative correlation resulting from numbers of GP trainees and exam outcomes being affected both by non-traditional teaching and by greater historical production of GPs. Postgraduate exam outcomes were also higher in schools with more self-regulated learning, but lower in larger medical schools. A path model for 29 measures found a complex causal nexus, most measures causing or being caused by other measures. Postgraduate exam performance was influenced by earlier attainment, at entry to Foundation and entry to medical school (the so-called academic backbone), and by self-regulated learning. Foundation measures of satisfaction, including preparedness, had no subsequent influence on outcomes. Fitness to practise issues were more frequent in schools producing more male graduates and more GPs.

CONCLUSIONS

Medical schools differ in large numbers of ways that are causally interconnected. Differences between schools in postgraduate examination performance, training problems and GMC sanctions have important implications for the quality of patient care and patient safety.

A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses

BACKGROUND

Gateway courses are increasingly popular widening participation routes into medicine. These six year courses provide a more accessible entry route into medical school and aim to support under-represented students' progress and graduation as doctors. There is little evidence on the performance of gateway students and this study compares attainment and aptitude on entry, and outcomes at graduation of students on the UK's three longest running gateway courses with students studying on a standard entry medical degree (SEMED) course at the same institutions.

METHODS

Data were obtained from the UK Medical Education Database for students starting between 2007 and 2012 at three UK institutions. These data included A-levels and Universities Clinical Aptitude Test scores on entry to medical school and the Educational Performance Measure (EPM) decile, Situational Judgement Test (SJT) and Prescribing Safety Assessment (PSA) scores as outcomes measures. Multiple regression models were used to test for difference in outcomes between the two types of course, controlling for attainment and aptitude on entry.

RESULTS

Four thounsand three hundred forty students were included in the analysis, 560 on gateway courses and 3785 on SEMED courses. Students on SEMED courses had higher attainment (Cohen's d = 1.338) and aptitude (Cohen's d = 1.078) on entry. On exit SEMED students had higher EPM scores (Cohen's d = 0.616) and PSA scores (Cohen's d = 0.653). When accounting for attainment and aptitude on entry course type is still a significant predictor of EPM and PSA, but the proportion of the variation in outcome explained by course type drops from 6.4 to 1.6% for EPM Decile and from 5.3% to less than 1% for the PSA score. There is a smaller significant difference in SJT scores, with SEMED having higher scores (Cohen's d = 0.114). However, when measures of performance on entry are accounted for, course type is no longer a significant predictor of SJT scores.

CONCLUSIONS

This study shows the differences of the available measures between gateway students and SEMED students on entry to their medical degrees are greater than the differences on exit. This provides modest evidence that gateway courses allow students from under-represented groups to achieve greater academic potential.