Assessment at UK medical schools varies substantially in volume, type and intensity and correlates with postgraduate attainment

Medical school origins of award-winning physicians; analysis of a complete national dataset

BACKGROUND

Educators and medical students share the same objective of achieving success in medical practice. Both groups consider doctors' successes to include optimum patient care outcomes and positive career progressions. Accordingly, identifying common educational features of such high-achieving doctors facilitates the generation of excellence amongst future medical trainees. In this study we use data from the British clinical merit award schemes as outcome measures in order to identify medical school origins of doctors who have achieved national or international prominence.

METHODS

Britain has Clinical Excellence Awards/Distinction Awards schemes that financially reward all National Health Service doctors in England, Scotland and Wales who are classified as high achievers. We used these outcome measures in a quantitative observational analysis of the 2019-20 dataset of all 901 national award-winning doctors. Where appropriate, Pearson's Chi-Square test was applied.

RESULTS

The top five medical schools (London university medical schools, Glasgow, Edinburgh, Oxford and Cambridge) were responsible for 51.2% of the physician merit award-winners in the 2019-20 round, despite the dataset representing 85 medical schools. 91.4% of the physician merit award-winners were from European medical schools. The lowest national award-winners (tier 3) originated from 61 medical schools representing six continents. International medical graduates comprised 11.4% of all award-winners.

CONCLUSIONS

The majority of physicians who were national merit award-winners originated from only five, apparently overrepresented, UK university medical schools. In contrast, there was a greater diversity in medical school origin among the lower grade national merit awards; the largest number of international medical graduates were in these tier 3 awards (13.3%). As well as ranking educationally successful university medical schools, this study assists UK and international students, by providing a roadmap for rational decision making when selecting physician and non-physician medical education pathways that are more likely to fulfil their career ambitions.

Medical school origins of award-winning psychiatrists; analysis of a complete national dataset

BACKGROUND

Britain attracts doctors from all over the world to work in the National Health Service. Elucidating the educational backgrounds of award-winning doctors working in the country is potentially an important medical education issue and a merit award audit. Using the British clinical merit award schemes as outcome measures, we identify medical school origins of award-winning doctors who have been identified as having achieved national or international prominence.

METHODS

The Clinical Excellence Awards/Distinction Awards schemes select doctors in Britain who are classified as high achievers, with categories for national prominence and above. We used this outcome measure in a quantitative observational analysis of the 2019-20 dataset of all 901 award-winning doctors. Pearson's Chi-Square test was used where appropriate.

RESULTS

Five university medical schools (London university medical schools, Glasgow, Edinburgh, Aberdeen and Cambridge) accounted for 59.1% of the psychiatrist award-winning doctors in the 2019 round, despite the dataset representing 85 medical schools. 84.1% of the psychiatrist award-winners were from European medical schools, compared to 92.1% of the non-psychiatrist award-winners. International medical graduates accounted for 22.7% of the award-winning psychiatrists. Psychiatrists with the lower grade national awards came from a more diverse educational background of 17 medical schools. IMGs represented diverse medical schools from five continents and were most represented in the lowest grade of national merit awards at 24.2%.

CONCLUSIONS

The majority of the award-winning psychiatrists originated from only five medical schools. A greater diversity of medical school origin existed for the lowest grade national psychiatrist award-winners. International medical graduates contributed substantially to these award-winners; psychiatrist award-winners were more likely to be international medical graduates (22.7%) than non-psychiatrist award-winners (10.8%). This study not only indicates educational centres associated with the production of award-winners but also provides students with a roadmap for rational decision making when selecting medical schools.

The educational value of situational judgement tests (SJTs) when used during undergraduate medical training: A systematic review and narrative synthesis

INTRODUCTION

Situational judgement tests (SJTs) are a recognised assessment method for admission into medical school, selection into postgraduate training programs, and postgraduate competency assessment. More recently, however, SJTs have been used during undergraduate medical training (UMT). This systematic review identifies, describes, and appraises the evidence for SJTs in UMT to determine educational associations and outcomes.

METHODS

MEDLINE, EMBASE, ERIC, PsycINFO, SCOPUS, Web of Science, and grey literature were searched for original research studies evaluating SJTs implemented within UMT to 1 November 2022. Studies reporting evaluation outcomes were included. Narrative data syntheses were undertaken. Risk of Bias was appraised using the Quality in Prognosis Studies tool.

RESULTS

24 studies were included. National database-derived SJTs (n = 14) assessed against professionalism, postgraduate attainment, construct of medical degree, medical school admissions scores, personality attributes, and declaration of disability. In-house derived SJTs (n = 10) assessed against professionalism, clinical skills, and personality attributes. Most evidence evaluated and reported inverse SJT associations with professionalism and were moderate risk of bias.

CONCLUSION

SJTs may have utility for developing professional behaviours in medical students. However, further research testing SJT robustness, standard setting methodologies, and prospectively evaluating SJTs against objective outcome measures within the context of UMT is warranted.

Medical school origins of award-winning surgeons; analysis of a complete national dataset

BACKGROUND

Britain attracts doctors from all over the world to work in the National Health Service. Elucidating the educational backgrounds of award-winning doctors working in the country is potentially an important medical education issue and merit award audit. Using the British clinical merit award schemes as outcome measures, we identify medical school origins of award-winning doctors who have been identified as having achieved national or international prominence.

METHODS

The Clinical Excellence Awards/Distinction Awards schemes select doctors in Britain who are classified as high achievers, with categories for national prominence and above. We used this outcome measure in a quantitative observational analysis of the 2019 dataset of all 901 award-winning doctors. Pearson's Chi-Square test was used where appropriate.

RESULTS

Seven medical schools (London university medical schools, Glasgow, Edinburgh, Aberdeen, Oxford, Cambridge and Manchester) accounted for 52.7% of the surgical award-winning doctors in the 2019 round, despite the dataset representing 85 medical schools. Surgeons with the lower grade national awards came from a more diverse educational background of 43 medical schools. International medical graduates accounted for 16.1% of the award-winning surgeons and 9.8% of the award-winning non-surgeons. 87.1% of the surgical award-winners were from European medical schools, whereas 93.2% of the non-surgical award-winners were from European medical schools.

CONCLUSIONS

The majority of the award-winning surgeons originated from only seven, overrepresented, medical schools. A greater diversity of medical school origin existed for the lowest grade national merit awards. These comprised 43 medical schools and indicated greater globalization effects in this category. International medical graduates contributed substantially to these award holders; surgical award-winners were more likely to be international medical graduates (16.1%) than non-surgical award-winners (9.8%). This study not only indicates educational centres associated with the production of award-winners but also provides students with a roadmap for rational decision making when selecting medical schools.

A national analysis of the medical schools of training for merit award-winning laboratory medical doctors working in Britain

AIMS

To inform the discussion regarding the origins of Laboratory Medical Consultant clinical merit award holders (LMC) whether the awards came from the Clinical Excellence Awards (CEA) or Distinction Awards (DA) schemes.

METHODS

Setting - CEA is a scheme to financially reward senior doctors in England and Wales who are assessed to be working over and above the standard expected of their role. The DA scheme is the parallel and equivalent scheme in Scotland. Participants - All of the merit award holders in the 2019 round. Design - This involved a secondary analysis of the complete 2019 published dataset of award winners. Statistical analyses were performed with Chi-square tests set at p < 0.05 level for statistical significance.

RESULTS

The top five medical schools (London University, Glasgow, Edinburgh, Aberdeen and Oxford) were responsible for 68.4% of the LMC merit award holders in the 2019 round. 97.9% of the LMC merit award holders were from European medical schools, whereas 90.9% of the non-LMC award holders were from European medical schools. The LMCs with A plus or platinum awards came from only six medical schools: Aberdeen, Edinburgh, London University, Oxford, Sheffield and Southampton. In contrast, the B or silver/bronze LMC award holders came from a more diverse background of 13 medical schools.

CONCLUSIONS

The majority of LMC merit award holders originated from only five university medical schools. All the LMCs with A plus or platinum awards came from only six university medical schools. There is an apparent overrepresentation of a small number of medical schools of origin amongst those LMCs that hold national merit awards.

Medical leadership training varies substantially between UK medical schools: Report of the leadership in undergraduate medical education national survey (LUMENS)

BACKGROUND

Doctors are increasingly expected to demonstrate medical leadership and management (MLM) skills. The Faculty of Medical Leadership and Management (FMLM) has published an indicative undergraduate curriculum to guide the development of MLM content at UK medical schools.

METHOD

Students from 30 medical schools were surveyed to determine their understanding of MLM teaching at their school. Timetables for 21 schools were searched for MLM-related keywords. Student-reported teaching and timetabled teaching were coded according to predefined themes. Aggregated demographic and postgraduate performance data were obtained through collaboration with the Medical Student Investigators Collaborative (msico.org).

RESULTS

Whilst 88% of medical students see MLM teaching as relevant, only 18% believe it is well integrated into their curriculum. MLM content represented ∼2% of timetabled teaching in each 5-year undergraduate medical course. Most of this teaching was dedicated to teamwork, performance/reflection and communication skills. There was minimal association between how much of a topic students believed they were taught, and how much they were actually taught. We found no association between the volume of MLM teaching and performance in postgraduate examinations, trainee career destinations or fitness to practice referrals.

CONCLUSION

Our findings demonstrate limited and variable teaching of MLM content. Delivery was independent of broader teaching and assessment factors.

Anatomy education for medical students in the United Kingdom and Republic of Ireland in 2019: A 20-year follow-up

Anatomical education in the United Kingdom (UK) and Ireland has long been under scrutiny, especially since the reforms triggered in 1993 by the General Medical Council's "Tomorrow's Doctors." The aim of the current study was to investigate the state of medical student anatomy education in the UK and Ireland in 2019. In all, 39 medical schools completed the survey (100% response rate) and trained 10,093 medical students per year cohort. The teachers comprised 760 individuals, of these 143 were employed on full-time teaching contracts and 103 were employed on education and research contracts. Since a previous survey in 1999, the number of part-time staff has increased by 300%, including a significant increase in the number of anatomy demonstrators. In 2019, anatomy was predominantly taught to medical students in either a system-based or hybrid curriculum. In all, 34 medical schools (87%) used human cadavers to teach anatomy, with a total of 1,363 donors being used per annum. Gross anatomy teaching was integrated with medical imaging in 95% of medical schools, embryology in 81%, living anatomy in 78%, neuroanatomy in 73%, and histology in 68.3%. Throughout their five years of study, medical students are allocated on average 85 h of taught time for gross anatomy, 24 h for neuroanatomy, 24 h for histology, 11 h for living anatomy, and 10 for embryology. In the past 20 years, there has been an average loss of 39 h dedicated to gross anatomy teaching and a reduction in time dedicated to all other anatomy sub-disciplines.

Institutional choice among medical applicants: a profile paper for The United Kingdom Medical Applicant Cohort Study (UKMACS) prospective longitudinal cohort study

PURPOSE

To generate a large cohort of those in 2019 seriously considering applying to study Medicine, collecting data on a range of socioeconomic and other demographic factors that influence choice of medical schools and to link to other datasets to form a longitudinal study of progress through medical school and careers in medicine.

DESIGN

Cross-sectional questionnaire studies, part of the longitudinal UK Medical Applicant Cohort Study (UKMACS).

SETTING

UK medical school admissions in 2020.

PARTICIPANTS

UK residents aged 16+ and seriously considering applying to study Medicine. The cohort was primarily drawn from those registering in 2019 for the U(K)CAT (University Clinical Aptitude Test (formerly the UK Clinical Aptitude Test)) with additional potential applicants responding to an open call. Participants consented to their data being linked within the UK Medical Education Database.

FINDINGS TO DATE

UKMACS Wave 1 questionnaire respondents consisted of 6391 consenting respondents from across the UK. In 2019, 14 980 of the 17 470 UK-domiciled medicine applicants were first-time applicants. The questionnaires show that many of these applicants have a need for more help and guidance to make informed choices, with less advantaged groups reporting themselves as being at a disadvantage when applying due to limited understanding of information and limited access to guidance to enable informed and effective decision-making.

FUTURE PLANS

To link the cohort with successive Universities and Colleges Admissions Service and other datasets to analyse outcomes of applications and establish national longitudinal evidence to understand how medical choices are made and how they impact on educational, career and workforce outcomes.

Dialogic Problematization of Academic Integrity Education

Many university educators have argued for a need for academic integrity education as an alternative to a focus on students' and scholars' compliance with academic rules and conventions (Brimble, 2016; Christensen Hughes & Bertram-Gallant, 2016; Hutton, 2006). I argue that the universal ethical-moral discourse of academic integrity disciplines subjects to comply with frequently alienating academic practices. This ethical discourse focuses on individual responsibility, in turn rendering invisible the authority of sometimes dysfunctional and oppressive instructional and summative assessment practices. Taking a Bakhtinian dialogic authorial perspective, the paper calls on students, scholars, instructors, and academic advisors to engage in critical ontological dialogue on diverse responses and motivations in regard to academic demands and deeds. Dialogue on situated instead of universal ethics in academic settings contextualizes and problematizes not just individual actions but also the ethics of the summative assessment regime, the instruction, the curriculum, authority dynamics, and the educational system as a whole. This discussion on academic integrity violations calls on educators to consider the ethical value of separating summative assessment from instruction.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

Measuring differential attainment: a longitudinal analysis of assessment results for 1512 medical students at four Scottish medical schools

OBJECTIVE

To measure Differential Attainment (DA) among Scottish medical students and to explore whether attainment gaps increase or decrease during medical school.

DESIGN

A retrospective analysis of undergraduate medical student performance on written assessment, measured at the start and end of medical school.

SETTING

Four Scottish medical schools (universities of Aberdeen, Dundee, Edinburgh and Glasgow).

PARTICIPANTS

1512 medical students who attempted (but did not necessarily pass) final written assessment.

MAIN OUTCOME MEASURES

The study modelled the change in attainment gap during medical school for four student demographical categories (white/non-white, international/Scottish domiciled, male/female and with/without a known disability) to test whether the attainment gap grew, shrank or remained stable during medical school. Separately, the study modelled the expected versus actual frequency of different demographical groups in the top and bottom decile of the cohort.

RESULTS

The attainment gap grew significantly for white versus non-white students (t(449.39)=7.37, p=0.001, d=0.49 and 95% CI 0.34 to 0.58), for internationally domiciled versus Scottish-domiciled students (t(205.8) = -7, p=0.01, d=0.61 and 95% CI -0.75 to -0.42) and for male versus female students (t(1336.68)=3.54, p=0.01, d=0.19 and 95% CI 0.08 to 0.27). International, non-white and male students received higher marks than their comparison group at the start of medical school but lower marks by final assessment. No significant differences were observed for disability status. Students with a known disability, Scottish students and non-white students were over-represented in the bottom decile and under-represented in the top decile.

CONCLUSIONS

The tendency for attainment gaps to grow during undergraduate medical education suggests that educational factors at medical schools may-however inadvertently-contribute to DA. It is of critical importance that medical schools investigate attainment gaps within their cohorts and explore potential underlying causes.

Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study

BACKGROUND

Identifying predictors of success in postgraduate examinations can help guide the career choices of medical students and may aid early identification of trainees requiring extra support to progress in specialty training. We assessed whether performance on the educational performance measurement (EPM) and situational judgement test (SJT) used for selection into foundation training predicted success at the Membership of the Royal College of Surgeons (MRCS) examination.

METHODS

This was a longitudinal, cohort study using data from the UK Medical Education Database (https://www.ukmed.ac.uk). UK medical graduates who had attempted Part A (n=2585) and Part B (n=755) of the MRCS between 2014 and 2017 were included. χ² and independent t-tests were used to examine the relationship between medical school performance and sociodemographic factors with first-attempt success at MRCS Part A and B. Multivariate logistic regression was employed to identify independent predictors of MRCS performance.

RESULTS

The odds of passing MRCS increased by 55% for Part A (OR 1.55 (95% CI 1.48 to 1.61)) and 23% for Part B (1.23 (1.14 to 1.32)) for every additional EPM decile point gained. For every point awarded for additional degrees in the EPM, candidates were 20% more likely to pass MRCS Part A (1.20 (1.13 to 1.29)) and 17% more likely to pass Part B (1.17 (1.04 to 1.33)). For every point awarded for publications in the EPM, candidates were 14% more likely to pass MRCS Part A (1.14 (1.01 to 1.28)). SJT score was not a statistically significant independent predictor of MRCS success.

CONCLUSION

This study has demonstrated the EPM's independent predictive power and found that medical school performance deciles are the most significant measure of predicting later success in the MRCS. These findings can be used by medical schools, training boards and workforce planners to inform evidence-based and contemporary selection and assessment strategies.

Variation in performance on common content items at UK medical schools

BACKGROUND

Due to differing assessment systems across UK medical schools, making meaningful cross-school comparisons on undergraduate students' performance in knowledge tests is difficult. Ahead of the introduction of a national licensing assessment in the UK, we evaluate schools' performances on a shared pool of "common content" knowledge test items to compare candidates at different schools and evaluate whether they would pass under different standard setting regimes. Such information can then help develop a cross-school consensus on standard setting shared content.

METHODS

We undertook a cross-sectional study in the academic sessions 2016-17 and 2017-18. Sixty "best of five" multiple choice 'common content' items were delivered each year, with five used in both years. In 2016-17 30 (of 31 eligible) medical schools undertook a mean of 52.6 items with 7,177 participants. In 2017-18 the same 30 medical schools undertook a mean of 52.8 items with 7,165 participants, creating a full sample of 14,342 medical students sitting common content prior to graduation. Using mean scores, we compared performance across items and carried out a "like-for-like" comparison of schools who used the same set of items then modelled the impact of different passing standards on these schools.

RESULTS

Schools varied substantially on candidate total score. Schools differed in their performance with large (Cohen's d around 1) effects. A passing standard that would see 5 % of candidates at high scoring schools fail left low-scoring schools with fail rates of up to 40 %, whereas a passing standard that would see 5 % of candidates at low scoring schools fail would see virtually no candidates from high scoring schools fail.

CONCLUSIONS

Candidates at different schools exhibited significant differences in scores in two separate sittings. Performance varied by enough that standards that produce realistic fail rates in one medical school may produce substantially different pass rates in other medical schools - despite identical content and the candidates being governed by the same regulator. Regardless of which hypothetical standards are "correct" as judged by experts, large institutional differences in pass rates must be explored and understood by medical educators before shared standards are applied. The study results can assist cross-school groups in developing a consensus on standard setting future licensing assessment.

The attitudes, perceptions and experiences of medical school applicants following the closure of schools and cancellation of public examinations in 2020 due to the COVID-19 pandemic: a cross-sectional questionnaire study of UK medical applicants

OBJECTIVE

Describe the experiences and views of medical applicants from diverse social backgrounds following the closure of schools and universities and the cancellation of public examinations in the UK due to COVID-19.

DESIGN

Cross-sectional questionnaire study, part of the longitudinal UK Medical Applicant Cohort Study (UKMACS).

SETTING

UK medical school admissions in 2020.

PARTICIPANTS

2887 participants completed an online questionnaire from 8 April to 22 April 2020. Eligible participants had registered to take the University Clinical Admissions Test in 2019 and agreed to be invited to take part, or had completed a previous UKMACS questionnaire, had been seriously considering applying to medicine in the UK for entry in 2020, and were UK residents.

MAIN OUTCOME MEASURES

Views on calculated grades, views on medical school admissions and teaching in 2020 and 2021, reported experiences of education during the national lockdown.

RESULTS

Respondents were concerned about the calculated grades that replaced A-level examinations: female and Black Asian and Minority Ethnic applicants felt teachers would find it difficult to grade and rank students accurately, and applicants from non-selective state schools and living in deprived areas had concerns about the standardisation process. Calculated grades were generally not considered fair enough to use in selection, but were considered fair enough to use in combination with other measures including interview and aptitude test scores. Respondents from non-selective state (public) schools reported less access to educational resources compared with private/selective school pupils, less online teaching in real time and less time studying during lockdown.

CONCLUSIONS

The COVID-19 pandemic has and will have significant and long-term impacts on the selection, education and performance of our medical workforce. It is important that the views and experiences of applicants from diverse backgrounds are considered in decisions affecting their future and the future of the profession.

Clay-Based Modeling in the Anatomist's Toolkit: A Systematic Review

Anatomical education has suffered from reduced teaching time and poor availability of staff and resources over the past thirty years. Clay-based modeling (CBM) is an alternative technique for teaching anatomy that can improve student knowledge and experience. This systematic review aimed to summarize and appraise the quality of the literature describing the uses, advantages, and limitations of CBM compared to alternative methods of teaching human gross anatomy to students or qualified healthcare professionals. A systematic search of Embase, MEDLINE, Scopus, and Web of Science databases was conducted, and the Medical Education Research Quality Instrument (MERSQI) was used to assess study quality. Out of the 829 studies identified, 12 papers met the inclusion criteria and were eligible for this review. The studies were of high quality, with a mean MERSQI score of 11.50/18. Clay-based modeling can be used to teach all gross anatomical regions, and 11 studies demonstrated a significant improvement in short-term knowledge gain in students who used CBM in comparison to other methods of learning anatomy. Eight studies that included subjective assessment showed that CBM is rated highly. However, some studies showed that students viewed CBM as juvenile and experienced difficulty making the models. Additionally, there is no evidence to suggest that CBM improves long-term knowledge. Clay-based modeling is an effective learning method for human gross anatomy and should be incorporated into the anatomists' toolkit. In the future, more randomized controlled studies with transparent study designs investigating the long-term impact of CBM are needed.

Predictors of performance on the pediatric board certification examination

BACKGROUND

Examining the predictors of summative assessment performance is important for improving educational programs and structuring appropriate learning environments for trainees. However, predictors of certification examination performance in pediatric postgraduate education have not been comprehensively investigated in Japan.

METHODS

The Pediatric Board Examination database in Japan, which includes 1578 postgraduate trainees from 2015 to 2016, was analyzed. The examinations included multiple-choice questions (MCQs), case summary reports, and an interview, and the predictors for each of these components were investigated by multiple regression analysis.

RESULTS

The number of examination attempts and the training duration were significant negative predictors of the scores for the MCQ, case summary, and interview. Employment at a community hospital or private university hospital were negative predictors of the MCQ and case summary score, respectively. Female sex and the number of academic presentations positively predicted the case summary and interview scores. The number of research publications was a positive predictor of the MCQ score, and employment at a community hospital was a positive predictor of the case summary score.

CONCLUSION

This study found that delayed and repeated examination taking were negative predictors, while the scholarly activity of trainees was a positive predictor, of pediatric board certification examination performance.

Medical student assessments-frequency of radiological images used: a national study

Objectives:

Assessments are a key part of life for medical students at University. We know there is variation in these assessments across Universities. The aims of this study were to expatiate summative assessments in Scottish Medical Schools and to examine how frequently radiological images feature in them.

Methods:

All Scottish medical schools were invited to participate in the study. Data on objective structured clinical examinations (OSCEs; 5 years) and written assessments (3 years) were retrospectively collected for each university and results were collated. Each University was randomly assigned a letter from A to E and anonymised for data presentation.

Results:

10,534 multiple choice questions (MCQ) and 1083 OSCE stations were included in this study. There was wide variation in the number, type and timing of assessments across Scottish medical schools. There were highly significant differences in the number of OSCE stations and the number of MCQs set over the study period (p < 0.0001). Radiological images were used on average 0.6 times (range 0–1.1) in each OSCE examination and 2.4 times (range 0.1–3.7) for written assessments.

Conclusion:

In this detailed study, we demonstrated significant differences in medical student assessments across Scottish Universities. Given the importance of Radiology in modern medicine, the frequency and differences in which radiological images were used in assessments across Universities should be addressed.

Advances in knowledge:

This is the first national longitudinal study to quantify the role of radiological images in summative Medical Student Assessments. Great variability exists in the extent and how (clinical versus written assessments) radiological images are used to assess Scottish medical students. Radiological images are used infrequently in clinical assessments, but are present in every written assessment. These findings could help inform medical schools and academic radiologists as they prepare medical students for the imminent unified medical licensing examination, where Clinical Imaging is a subject with one of the highest number of associated conditions examinable.

The Analysis of Teaching of Medical Schools (AToMS) survey: an analysis of 47,258 timetabled teaching events in 25 UK medical schools relating to timing, duration, teaching formats, teaching content, and problem-based learning

BACKGROUND

What subjects UK medical schools teach, what ways they teach subjects, and how much they teach those subjects is unclear. Whether teaching differences matter is a separate, important question. This study provides a detailed picture of timetabled undergraduate teaching activity at 25 UK medical schools, particularly in relation to problem-based learning (PBL).

METHOD

The Analysis of Teaching of Medical Schools (AToMS) survey used detailed timetables provided by 25 schools with standard 5-year courses. Timetabled teaching events were coded in terms of course year, duration, teaching format, and teaching content. Ten schools used PBL. Teaching times from timetables were validated against two other studies that had assessed GP teaching and lecture, seminar, and tutorial times.

RESULTS

A total of 47,258 timetabled teaching events in the academic year 2014/2015 were analysed, including SSCs (student-selected components) and elective studies. A typical UK medical student receives 3960 timetabled hours of teaching during their 5-year course. There was a clear difference between the initial 2 years which mostly contained basic medical science content and the later 3 years which mostly consisted of clinical teaching, although some clinical teaching occurs in the first 2 years. Medical schools differed in duration, format, and content of teaching. Two main factors underlay most of the variation between schools, Traditional vs PBL teaching and Structured vs Unstructured teaching. A curriculum map comparing medical schools was constructed using those factors. PBL schools differed on a number of measures, having more PBL teaching time, fewer lectures, more GP teaching, less surgery, less formal teaching of basic science, and more sessions with unspecified content.

DISCUSSION

UK medical schools differ in both format and content of teaching. PBL and non-PBL schools clearly differ, albeit with substantial variation within groups, and overlap in the middle. The important question of whether differences in teaching matter in terms of outcomes is analysed in a companion study (MedDifs) which examines how teaching differences relate to university infrastructure, entry requirements, student perceptions, and outcomes in Foundation Programme and postgraduate training.

Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise

BACKGROUND

Medical schools differ, particularly in their teaching, but it is unclear whether such differences matter, although influential claims are often made. The Medical School Differences (MedDifs) study brings together a wide range of measures of UK medical schools, including postgraduate performance, fitness to practise issues, specialty choice, preparedness, satisfaction, teaching styles, entry criteria and institutional factors.

METHOD

Aggregated data were collected for 50 measures across 29 UK medical schools. Data include institutional history (e.g. rate of production of hospital and GP specialists in the past), curricular influences (e.g. PBL schools, spend per student, staff-student ratio), selection measures (e.g. entry grades), teaching and assessment (e.g. traditional vs PBL, specialty teaching, self-regulated learning), student satisfaction, Foundation selection scores, Foundation satisfaction, postgraduate examination performance and fitness to practise (postgraduate progression, GMC sanctions). Six specialties (General Practice, Psychiatry, Anaesthetics, Obstetrics and Gynaecology, Internal Medicine, Surgery) were examined in more detail.

RESULTS

Medical school differences are stable across time (median alpha = 0.835). The 50 measures were highly correlated, 395 (32.2%) of 1225 correlations being significant with p < 0.05, and 201 (16.4%) reached a Tukey-adjusted criterion of p < 0.0025. Problem-based learning (PBL) schools differ on many measures, including lower performance on postgraduate assessments. While these are in part explained by lower entry grades, a surprising finding is that schools such as PBL schools which reported greater student satisfaction with feedback also showed lower performance at postgraduate examinations. More medical school teaching of psychiatry, surgery and anaesthetics did not result in more specialist trainees. Schools that taught more general practice did have more graduates entering GP training, but those graduates performed less well in MRCGP examinations, the negative correlation resulting from numbers of GP trainees and exam outcomes being affected both by non-traditional teaching and by greater historical production of GPs. Postgraduate exam outcomes were also higher in schools with more self-regulated learning, but lower in larger medical schools. A path model for 29 measures found a complex causal nexus, most measures causing or being caused by other measures. Postgraduate exam performance was influenced by earlier attainment, at entry to Foundation and entry to medical school (the so-called academic backbone), and by self-regulated learning. Foundation measures of satisfaction, including preparedness, had no subsequent influence on outcomes. Fitness to practise issues were more frequent in schools producing more male graduates and more GPs.

CONCLUSIONS

Medical schools differ in large numbers of ways that are causally interconnected. Differences between schools in postgraduate examination performance, training problems and GMC sanctions have important implications for the quality of patient care and patient safety.

Preparation for practice and the arguments for standardisation in view of the forthcoming medical licensing exam: A literature review

Aims: The aim of this paper is to address whether the introduction of a national licensing exam will improve preparedness for practice. This review will explore and identify the areas in which UK medical graduates are feeling underprepared. In addition, this review will discuss the need for the Medical Licensing Assessment in the UK.Methods: A literature search of several databases was performed in December 2018. Studies which identified the areas in which medical graduates were prepared/unprepared for, and those which explored whether UK medical schools were adequately preparing their graduates for practice, were included. Studies exploring the preparedness of international medical graduates were excluded.Results: Sixteen articles met the final inclusion criteria and are included in this review. All the final studies are cross-sectional and are varied in their use of methods used, with the vast majority using questionnaires to identify the preparedness of medical students and junior doctors.Conclusions: This review conveys that junior doctors are mostly underprepared in emergency work. This can be associated to a lack of exposure to emergency situations during their undergraduate training. Moreover, this review suggests that the preparedness of medical graduates may remain unaffected by the implementation of the MLA.

Osler Centenary Papers: William Osler in medical education

William Osler combined many excellent characteristics of a clinical educator being a scientific scholar, a motivational speaker and writer and a proficient physician. As we celebrate his life a century on, many of his educational ideals are as pertinent today as they were in those Victorian times. Osler's contributions to modern medicine go beyond his legacy of quotable aphorisms to a doctor, educator and leader whose proponent use of bedside teaching, careful clinical methods, and clinicopathological correlation was a great inspiration for students and junior doctors. He was also a great advocate of patient-centred care-listening to and closely observing his patients, an important message for modern medicine as the reliance on investigations strains modern healthcare systems. This review of Osler's contribution to medical education summarises his development as an educator and provides reflection on his influences to modern clinical education.

The impact of medical school assessment on preparedness for practice

PURPOSE

As assessment is known to drive learning, this paper looks at the relationship between assessment practice across UK medical schools and graduates preparedness for practice.

MATERIALS AND METHODS

It uses data on written and practical assessment at each medical school and the association with students' self-reported preparedness for working as a foundation doctor on graduation, and in particular the preparation related to clinical skills.

RESULTS AND CONCLUSIONS

A negative correlation (β= -0.003, p < 0.001) was observed between total duration of written assessment and preparedness, while a positive relationship (β = 0.461, p < 0.001) was seen between "adequately prepared" and the proportion of all assessment time focusing on practical skills. This suggests that graduates from medical schools with a greater emphasis on practical skills in their assessment plan are better prepared to practice as a junior doctor on gradation; something that may be of relevance when designing a national licensing examination.

Artificial or intelligent? Machine learning and medical selection: possibilities and risks

This article was migrated. The article was marked as recommended. Machine learning approaches form the basis of "artificial intelligence" and have been increasingly applied in health services settings. It has been shown that such approaches may produce more accurate predictions in some contexts, compared to conventional statistical approaches, and may also reduce the costs of decision-making through automation.  Nevertheless, there are both general limitations to developing and implementing machine learning approaches that must be borne in mind. To date, relatively little research has been published on the potential for machine learning to support personnel selection. Moreover, there are particular challenges and issues that need to be considered if such methods are to be used to support decision-making in medical selection scenarios. This article describes some of these potential advantages and challenges and presents an illustrative example, based on real-world data, related to the selection of medical undergraduates.

Formative student-authored question bank: perceptions, question quality and association with summative performance

Purpose of the study

There are few studies on the value of authoring questions as a study method, the quality of the questions produced by students and student perceptions of student-authored question banks. Here we evaluate PeerWise, a widely used and free online resource that allows students to author, answer and discuss multiple-choice questions.

Study design

We introduced two undergraduate medical student cohorts to PeerWise (n=603). We looked at their patterns of PeerWise usage; identified associations between student engagement and summative exam performance; and used focus groups to assess student perceptions of the value of PeerWise for learning. We undertook item analysis to assess question difficulty and quality.

Results

Over two academic years, the two cohorts wrote 4671 questions, answered questions 606 658 times and posted 7735 comments. Question writing frequency correlated most strongly with summative performance (Spearman’s rank: 0.24, p=<0.001). Student focus groups found that: (1) students valued curriculum specificity; and (2) students were concerned about student-authored question quality. Only two questions of the 300 ’most-answered' questions analysed had an unacceptable discriminatory value (point-biserial correlation <0.2).

Conclusions

Item analysis suggested acceptable question quality despite student concerns. Quantitative and qualitative methods indicated that PeerWise is a valuable study tool.