Graduates of different UK medical schools show substantial differences in performance on MRCP(UK) Part 1, Part 2 and PACES examinations

Medical school origins of award-winning physicians; analysis of a complete national dataset

BACKGROUND

Educators and medical students share the same objective of achieving success in medical practice. Both groups consider doctors' successes to include optimum patient care outcomes and positive career progressions. Accordingly, identifying common educational features of such high-achieving doctors facilitates the generation of excellence amongst future medical trainees. In this study we use data from the British clinical merit award schemes as outcome measures in order to identify medical school origins of doctors who have achieved national or international prominence.

METHODS

Britain has Clinical Excellence Awards/Distinction Awards schemes that financially reward all National Health Service doctors in England, Scotland and Wales who are classified as high achievers. We used these outcome measures in a quantitative observational analysis of the 2019-20 dataset of all 901 national award-winning doctors. Where appropriate, Pearson's Chi-Square test was applied.

RESULTS

The top five medical schools (London university medical schools, Glasgow, Edinburgh, Oxford and Cambridge) were responsible for 51.2% of the physician merit award-winners in the 2019-20 round, despite the dataset representing 85 medical schools. 91.4% of the physician merit award-winners were from European medical schools. The lowest national award-winners (tier 3) originated from 61 medical schools representing six continents. International medical graduates comprised 11.4% of all award-winners.

CONCLUSIONS

The majority of physicians who were national merit award-winners originated from only five, apparently overrepresented, UK university medical schools. In contrast, there was a greater diversity in medical school origin among the lower grade national merit awards; the largest number of international medical graduates were in these tier 3 awards (13.3%). As well as ranking educationally successful university medical schools, this study assists UK and international students, by providing a roadmap for rational decision making when selecting physician and non-physician medical education pathways that are more likely to fulfil their career ambitions.

Gender and graduating results in the Anesthesiology and Intensive Care Abomey-Calavi (Cotonou, Benin) program

Background: For unknown reasons female participation in anesthesiology is very low in Sub-Saharan Africa (SSA), especially in West Africa, and few women reach top academic or clinical positions. Objective: Women reduced professional perspectives. Design: Male and female residents’ performances were compared when they presented for their first try the graduating exams of the specialty. Settings: The Cotonou anesthesiology and intensive care training center, the second largest in French-speaking SSA, where 146 anesthesiologists from 14 African countries graduated since its creation in 1996. Method: All results at their final exams (consisting in 3 written questions and 2 clinical evaluations) were retrieved for the 125 men and 21 women who graduated. Scores obtained by women and males were compared using Student’s t tests. Their total of points was used to divide graduates into deciles. The proportion of women was counted in each decile. Results: Women performed better at both anesthesia and intensive care clinical evaluations taken separately and together (total 68.2% vs. 64.2% p=0.004) and were even with men for the three written exams (anesthesia, intensive care and basic sciences - total 66.2 % vs. 66.1% p=0.99). When clinical and written scores are added in each sector, women scored better than males for anesthesiology (69.2% vs. 65.2% p=0.01) and were even for intensive care (65.0% vs. 64.1% p=0.51). Globally women and men results were similar (67% vs. 65%, p=0.1) The proportion of women in each decile increased from the lowest to the best deciles, but the absolute low number of women gives this trend borderline significance (slope +1,56 % women per decile, p=0.046)**. Conclusion: Women performance at end-specialty exams is unlikely to explain their subsequent underrepresentation at the academic level in anesthesia and intensive care in SSA**.

Academic-faculty environment and graduate employability: variation of work-readiness perceptions

Graduates with a high level of competence can cope better with the disequilibria triggered by events such as shifting labor processes and job transitions. This study examined the relationship between prospective graduates' perceived job preparedness and the university's role in preparing students for the workforce. A sample of 335 South African university students was used. We discovered that skill inequalities vary greatly between faculties. Collegiate skill preparation of students for the workplace can explain the disparities in graduate skill development and recruitment opportunities. Universities and their faculties must reimagine themselves as the primary drivers of graduate skill development and expand pipelines for the most vulnerable prospective graduates to contribute to global skills need.

Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study

OBJECTIVE

To determine whether scores on two undergraduate admissions tests (BioMedical Admissions Test (BMAT) and University Clinical Aptitude Test (UCAT)) predict performance on the postgraduate Membership of the Royal Colleges of Physicians (MRCP) examination, including the clinical examination Practical Assessment of Clinical Examination Skills (PACES).

DESIGN

National cohort study.

SETTING

Doctors who graduated medical school between 2006 and 2018.

PARTICIPANTS

3045 doctors who had sat BMAT, UCAT and the MRCP.

PRIMARY OUTCOME MEASURES

Passing each section of the MRCP at the first attempt, including the clinical assessment PACES.

RESULTS

Several BMAT and UCAT subtest scores displayed incremental predictive validity for performance on the first two (written) parts of the MRCP. Only aptitude and skills on BMAT (OR 1.34, 1.08 to 1.67, p=0.01) and verbal reasoning on UCAT (OR 1.34, 1.04 to 1.71, p=0.02) incrementally predicted passing PACES at the first attempt.

CONCLUSIONS

Our results imply that the abilities assessed by aptitude and skills and verbal reasoning may be the most important cognitive attributes, of those routinely assessed at selection, for predicting future clinical performance. Selectors may wish to consider placing particular weight on scales assessing these attributes if they wish to select applicants likely to become more competent clinicians. These results are potentially relevant in an international context too, since many admission tests used globally, such as the Medical College Admission Test, assess similar abilities.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

Measuring differential attainment: a longitudinal analysis of assessment results for 1512 medical students at four Scottish medical schools

OBJECTIVE

To measure Differential Attainment (DA) among Scottish medical students and to explore whether attainment gaps increase or decrease during medical school.

DESIGN

A retrospective analysis of undergraduate medical student performance on written assessment, measured at the start and end of medical school.

SETTING

Four Scottish medical schools (universities of Aberdeen, Dundee, Edinburgh and Glasgow).

PARTICIPANTS

1512 medical students who attempted (but did not necessarily pass) final written assessment.

MAIN OUTCOME MEASURES

The study modelled the change in attainment gap during medical school for four student demographical categories (white/non-white, international/Scottish domiciled, male/female and with/without a known disability) to test whether the attainment gap grew, shrank or remained stable during medical school. Separately, the study modelled the expected versus actual frequency of different demographical groups in the top and bottom decile of the cohort.

RESULTS

The attainment gap grew significantly for white versus non-white students (t(449.39)=7.37, p=0.001, d=0.49 and 95% CI 0.34 to 0.58), for internationally domiciled versus Scottish-domiciled students (t(205.8) = -7, p=0.01, d=0.61 and 95% CI -0.75 to -0.42) and for male versus female students (t(1336.68)=3.54, p=0.01, d=0.19 and 95% CI 0.08 to 0.27). International, non-white and male students received higher marks than their comparison group at the start of medical school but lower marks by final assessment. No significant differences were observed for disability status. Students with a known disability, Scottish students and non-white students were over-represented in the bottom decile and under-represented in the top decile.

CONCLUSIONS

The tendency for attainment gaps to grow during undergraduate medical education suggests that educational factors at medical schools may-however inadvertently-contribute to DA. It is of critical importance that medical schools investigate attainment gaps within their cohorts and explore potential underlying causes.

Predictors of ophthalmology career success (POCS) study

Objective

Ophthalmology is the busiest outpatient specialty with demand predicted to rise over 40% in the next 20 years. A significant increase in the number of trainee ophthalmologists is required to fill currently vacant consultant posts and meet the UK's workforce demands by 2038. Our aim was to understand what determines success in ophthalmology training, in order to inform future ophthalmologists, refine recruitment and facilitate workforce planning.

Methods and Analysis

This was a retrospective longitudinal cohort study using routinely collected data available from UK Medical Education Database (UKMED) (https://www.ukmed.ac.uk/). Data were analysed on 1350 candidates who had applied for ophthalmology specialty training (OST) between 2012 and 2018, as well as 495 candidates who had attempted Fellow of the Royal College of Ophthalmologists (FRCOphth) Part 1 between 2013 and 2018. Participants who had not obtained their primary medical qualification from the UK medical schools were excluded. Primary outcome measures included gaining a place on the OST programme and passing the FRCOphth Part 1 examination on first attempt.

Results

Higher education performance measure decile scores at medical school are strongly predictive in securing an OST post and passing the part 1 examination first time (p<0.001). Candidates who attempt FRCOphth Part 1 prior to their ST1 application are more likely to get a place on OST on first attempt. Socioeconomic factors, gender and ethnicity do not influence success in OST entry. Male trainees are more likely to pass FRCOphth Part 1 on their first attempt.

Conclusion

This study is the first quantitative assessment of the factors that determine success in OST recruitment and ophthalmology postgraduate examinations in the UK. Similar studies should be undertaken in all other medical and surgical specialties to understand what factors predict success.

Variation in performance on common content items at UK medical schools

BACKGROUND

Due to differing assessment systems across UK medical schools, making meaningful cross-school comparisons on undergraduate students' performance in knowledge tests is difficult. Ahead of the introduction of a national licensing assessment in the UK, we evaluate schools' performances on a shared pool of "common content" knowledge test items to compare candidates at different schools and evaluate whether they would pass under different standard setting regimes. Such information can then help develop a cross-school consensus on standard setting shared content.

METHODS

We undertook a cross-sectional study in the academic sessions 2016-17 and 2017-18. Sixty "best of five" multiple choice 'common content' items were delivered each year, with five used in both years. In 2016-17 30 (of 31 eligible) medical schools undertook a mean of 52.6 items with 7,177 participants. In 2017-18 the same 30 medical schools undertook a mean of 52.8 items with 7,165 participants, creating a full sample of 14,342 medical students sitting common content prior to graduation. Using mean scores, we compared performance across items and carried out a "like-for-like" comparison of schools who used the same set of items then modelled the impact of different passing standards on these schools.

RESULTS

Schools varied substantially on candidate total score. Schools differed in their performance with large (Cohen's d around 1) effects. A passing standard that would see 5 % of candidates at high scoring schools fail left low-scoring schools with fail rates of up to 40 %, whereas a passing standard that would see 5 % of candidates at low scoring schools fail would see virtually no candidates from high scoring schools fail.

CONCLUSIONS

Candidates at different schools exhibited significant differences in scores in two separate sittings. Performance varied by enough that standards that produce realistic fail rates in one medical school may produce substantially different pass rates in other medical schools - despite identical content and the candidates being governed by the same regulator. Regardless of which hypothetical standards are "correct" as judged by experts, large institutional differences in pass rates must be explored and understood by medical educators before shared standards are applied. The study results can assist cross-school groups in developing a consensus on standard setting future licensing assessment.

Effect of medical school attended on the chances of successfully embarking on a clinical-academic career in the UK

PURPOSE OF THE STUDY

This study aimed to investigate whether, in the UK, medical school attended influences the propensity to apply to and be successful in obtaining an offer from the Academic Foundation Programme (AFP), thus taking the first step to embarking on a clinical-academic career.

STUDY DESIGN

A retrospective observational study was performed. Using the UK Foundation Programme's yearly statistical report data, mean application rates to, and mean offer rates from the AFP were calculated by medical school, between the years 2017-2019. Mean application and mean offer rates were subsequently correlated with metrics of medical school academic performance and research focus.

RESULTS

Mean application rates to the AFP were higher in medical schools that had a mandatory intercalated degree as part of the undergraduate medical curriculum (mean=33.99%, SD=13.93 vs mean=19.44%, SD=6.88, p<0.001), lower numerical rank in the Times Higher Education 2019 World Rankings (correlation with higher numerical rank, r=-0.50, p=0.004), and lower numerical rank in the Research Excellence Framework 2014 UK rankings (correlation with higher numerical rank, r=-0.37, p=0.004). Mean offer rates from the AFP were not correlated with any metric of medical school academic performance or research focus.

CONCLUSIONS

Students attending a medical school with greater academic performance and research focus are more likely to apply and subsequently embark on a clinical-academic career. However, students wishing to embark a clinical-academic career from any medical school have an equal chance of success.

The Analysis of Teaching of Medical Schools (AToMS) survey: an analysis of 47,258 timetabled teaching events in 25 UK medical schools relating to timing, duration, teaching formats, teaching content, and problem-based learning

BACKGROUND

What subjects UK medical schools teach, what ways they teach subjects, and how much they teach those subjects is unclear. Whether teaching differences matter is a separate, important question. This study provides a detailed picture of timetabled undergraduate teaching activity at 25 UK medical schools, particularly in relation to problem-based learning (PBL).

METHOD

The Analysis of Teaching of Medical Schools (AToMS) survey used detailed timetables provided by 25 schools with standard 5-year courses. Timetabled teaching events were coded in terms of course year, duration, teaching format, and teaching content. Ten schools used PBL. Teaching times from timetables were validated against two other studies that had assessed GP teaching and lecture, seminar, and tutorial times.

RESULTS

A total of 47,258 timetabled teaching events in the academic year 2014/2015 were analysed, including SSCs (student-selected components) and elective studies. A typical UK medical student receives 3960 timetabled hours of teaching during their 5-year course. There was a clear difference between the initial 2 years which mostly contained basic medical science content and the later 3 years which mostly consisted of clinical teaching, although some clinical teaching occurs in the first 2 years. Medical schools differed in duration, format, and content of teaching. Two main factors underlay most of the variation between schools, Traditional vs PBL teaching and Structured vs Unstructured teaching. A curriculum map comparing medical schools was constructed using those factors. PBL schools differed on a number of measures, having more PBL teaching time, fewer lectures, more GP teaching, less surgery, less formal teaching of basic science, and more sessions with unspecified content.

DISCUSSION

UK medical schools differ in both format and content of teaching. PBL and non-PBL schools clearly differ, albeit with substantial variation within groups, and overlap in the middle. The important question of whether differences in teaching matter in terms of outcomes is analysed in a companion study (MedDifs) which examines how teaching differences relate to university infrastructure, entry requirements, student perceptions, and outcomes in Foundation Programme and postgraduate training.

Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise

BACKGROUND

Medical schools differ, particularly in their teaching, but it is unclear whether such differences matter, although influential claims are often made. The Medical School Differences (MedDifs) study brings together a wide range of measures of UK medical schools, including postgraduate performance, fitness to practise issues, specialty choice, preparedness, satisfaction, teaching styles, entry criteria and institutional factors.

METHOD

Aggregated data were collected for 50 measures across 29 UK medical schools. Data include institutional history (e.g. rate of production of hospital and GP specialists in the past), curricular influences (e.g. PBL schools, spend per student, staff-student ratio), selection measures (e.g. entry grades), teaching and assessment (e.g. traditional vs PBL, specialty teaching, self-regulated learning), student satisfaction, Foundation selection scores, Foundation satisfaction, postgraduate examination performance and fitness to practise (postgraduate progression, GMC sanctions). Six specialties (General Practice, Psychiatry, Anaesthetics, Obstetrics and Gynaecology, Internal Medicine, Surgery) were examined in more detail.

RESULTS

Medical school differences are stable across time (median alpha = 0.835). The 50 measures were highly correlated, 395 (32.2%) of 1225 correlations being significant with p < 0.05, and 201 (16.4%) reached a Tukey-adjusted criterion of p < 0.0025. Problem-based learning (PBL) schools differ on many measures, including lower performance on postgraduate assessments. While these are in part explained by lower entry grades, a surprising finding is that schools such as PBL schools which reported greater student satisfaction with feedback also showed lower performance at postgraduate examinations. More medical school teaching of psychiatry, surgery and anaesthetics did not result in more specialist trainees. Schools that taught more general practice did have more graduates entering GP training, but those graduates performed less well in MRCGP examinations, the negative correlation resulting from numbers of GP trainees and exam outcomes being affected both by non-traditional teaching and by greater historical production of GPs. Postgraduate exam outcomes were also higher in schools with more self-regulated learning, but lower in larger medical schools. A path model for 29 measures found a complex causal nexus, most measures causing or being caused by other measures. Postgraduate exam performance was influenced by earlier attainment, at entry to Foundation and entry to medical school (the so-called academic backbone), and by self-regulated learning. Foundation measures of satisfaction, including preparedness, had no subsequent influence on outcomes. Fitness to practise issues were more frequent in schools producing more male graduates and more GPs.

CONCLUSIONS

Medical schools differ in large numbers of ways that are causally interconnected. Differences between schools in postgraduate examination performance, training problems and GMC sanctions have important implications for the quality of patient care and patient safety.

Preparation for practice and the arguments for standardisation in view of the forthcoming medical licensing exam: A literature review

Aims: The aim of this paper is to address whether the introduction of a national licensing exam will improve preparedness for practice. This review will explore and identify the areas in which UK medical graduates are feeling underprepared. In addition, this review will discuss the need for the Medical Licensing Assessment in the UK.Methods: A literature search of several databases was performed in December 2018. Studies which identified the areas in which medical graduates were prepared/unprepared for, and those which explored whether UK medical schools were adequately preparing their graduates for practice, were included. Studies exploring the preparedness of international medical graduates were excluded.Results: Sixteen articles met the final inclusion criteria and are included in this review. All the final studies are cross-sectional and are varied in their use of methods used, with the vast majority using questionnaires to identify the preparedness of medical students and junior doctors.Conclusions: This review conveys that junior doctors are mostly underprepared in emergency work. This can be associated to a lack of exposure to emergency situations during their undergraduate training. Moreover, this review suggests that the preparedness of medical graduates may remain unaffected by the implementation of the MLA.

Unexplained variations in general practitioners' perceptions and practices regarding vaccination in France

Background

Given geographical disparities in vaccination coverage (VC) and the crucial role general practitioners (GPs) play in vaccination in France, we sought to: 1) determine the existence of geographical variations in GPs' perceptions of vaccines, their trust in information sources, and the frequency of their recommendations to patients by comparing data from southeast (SE), central-west (CW), northwest (NW), and the rest of France; and 2) identify individual and contextual factors associated with regional variations in GPs' recommendations.

Methods

This cross-sectional observational study in 2014 collected data from a panel of 2586 French GPs in private practice: 3 specific regional samples and a fourth sample for the rest of France. We calculated a composite score summarizing GPs' vaccine recommendation frequency for 6 vaccine situations and used a five-step hierarchical linear regression to study the score's links with practice location and individual and contextual factors.

Results

GPs' vaccine recommendation frequency score was highest in NW France and lowest in the SE. The low SE score was explained by GPs' greater doubts about vaccine utility and risks and lower trust in information sources. The high NW score was partially explained by greater adherence to guidelines by GPs there. The contextual factors studied did not explain regional differences.

Conclusion

The geographical variations in GPs' vaccination-related attitudes and practices suggest that vaccine hesitancy among GPs differs in prevalence between regions. These variations coincide with north/south trends in population VC. Intervention strategies to restore confidence in vaccines should target GPs and must be adapted to each regional context.

Effect of sex on specialty training application outcomes: a longitudinal administrative data study of UK medical graduates

OBJECTIVES

To examine sex differences in the specialty training recruitment outcomes of UK medical graduates; and whether sex differences were explained by prior academic attainment and previous fitness to practise (FtP) declarations.

DESIGN

Retrospective longitudinal cohort study.

SETTING

Administrative data on entrants to all UK medical schools from the UK Medical Education Database.

PARTICIPANTS

10 559 doctors (6 155; 58% female) who entered a UK medical school in 2007 or 2008 and were eligible to apply for specialty training by 2015.

PRIMARY OUTCOME MEASURE

Odds of application, offer and acceptance to any specialty training programme, and on to each of the nine largest training programmes, adjusting for sex, other demographics, prior academic attainment, FtP declaration and medical school.

RESULTS

Across all specialties, there were no sex differences in applications for specialty training, but women had increased odds of getting an offer (OR=1.40; 95% CI=1.25 to 1.57; p<0.001) and accepting one (OR=1.43; 95% CI=1.19 to 1.71; p<0.001). Seven of the nine largest specialties showed significant sex differences in applications, which remained after adjusting for other factors. In the adjusted models, Paediatrics (OR=1.57; 95% CI=1.01 to 2.46; p=0.046) and general practice (GP) (OR=1.23; 95% CI=1.03 to 1.46; p=0.017) were the only specialties to show sex differences in offers, both favouring women. GP alone showed sex differences in acceptances, with women being more likely to accept (OR=1.34; 95% CI=1.03 to 1.76; p=0.03). Doctors with an FtP declaration were slightly less likely to apply to specialty training overall (OR=0.84; 95% CI=0.71 to 1.00; p=0.048) and less likely to accept an offer to any programme (OR=0.71; 95% CI=0.52 to 0.98; p=0.036), after adjusting for confounders.

CONCLUSIONS

Sex segregation between medical specialties is due to differential application, although research is needed to understand why men are less likely to be offered a place on to GP and Paediatrics training, and if offered GP are less likely to accept.

The effects of human factor related issues on assessors during the recruitment process for general and vascular surgery in the UK

INTRODUCTION

Healthcare professionals increasingly recognise how human factors (HF) can contribute to medical error. An understanding of HF is also important during other high stakes activities such as summative assessments or examinations. National organisations hosting such events need to be aware of potential fatigue, boredom and stress that can occur in interviewers during these often repetitive activities.

METHODS

A previously validated questionnaire based around the well known HF analysis and classification system (HFACS) was used to evaluate four factors (care and support, asking questions about the role as an interviewer, working within the rules and boundaries, and stress and pressure) at the 2018 UK general and vascular surgery trainee national selection process.

RESULTS

A total of 92 questionnaires were completed and analysed (48% response rate). After recoding for negative phrased questions, no significant differences were found between years of experience in the selection process and mean scores obtained for all four factor items. Interviewers had a positive experience during national selection with mean factor scores ranging from 3.84 to 3.98 (out of a maximum satisfaction score of 5).

CONCLUSIONS

Organisations need to plan carefully and recognise the human element to ensure that their interviewers are cared for during any high stakes assessment such as national selection. Our work suggests that a positive assessor experience will further help contribute to a reliable and fair recruitment process.

The International Landscape of Medical Licensing Examinations: A Typology Derived From a Systematic Review

BACKGROUND

National licensing examinations (NLEs) are large-scale examinations usually taken by medical doctors close to the point of graduation from medical school. Where NLEs are used, success is usually required to obtain a license for full practice. Approaches to national licensing, and the evidence that supports their use, varies significantly across the globe. This paper aims to develop a typology of NLEs, based on candidacy, to explore the implications of different examination types for workforce planning.

METHODS

A systematic review of the published literature and medical licensing body websites, an electronic survey of all medical licensing bodies in highly developed nations, and a survey of medical regulators.

RESULTS

The evidence gleaned through this systematic review highlights four approaches to NLEs: where graduating medical students wishing to practice in their national jurisdiction must pass a national licensing exam before they are granted a license to practice; where all prospective doctors, whether from the national jurisdiction or international medical graduates, are required to pass a national licensing exam in order to practice within that jurisdiction; where international medical graduates are required to pass a licensing exam if their qualifications are not acknowledged to be comparable with those students from the national jurisdiction; and where there are no NLEs in operation. This typology facilitates comparison across systems and highlights the implications of different licensing systems for workforce planning.

CONCLUSION

The issue of national licensing cannot be viewed in isolation from workforce planning; future research on the efficacy of national licensing systems to drive up standards should be integrated with research on the implications of such systems for the mobility of doctors to cross borders.

FRCS first pass variance: deanery and specialty contrariety

BACKGROUND

FRCS exit examination success may be interpreted as a surrogate marker for UK Deanery-related training quality. The aim of this study was to evaluate relative FRCS examination pass rates related to Deanery and Surgical Specialty.

METHODS

Joint Committee on Surgical Training-published examination first attempt pass rates were scrutinised for type I higher surgical trainees and outcomes compared related to Deanery and Surgical Specialty.

RESULTS

Of 9363 FRCS first attempts, 3974 were successful (42.4%). Median and mean pass rates related to Deanery were 42.1% and 30.7%, respectively, and ranged from 26.7% to 45.6%. Median (range) pass rates by specialty were urology 76.3% (60%-100%), trauma and orthopaedic surgery 74.7% (58.2%-100%), general surgery 70.0% (63.1%-86%), ENT 62.5% (50%-100%), cardiothoracic surgery 50.0% (25%-100%), oral and maxillofacial surgery 50% (40.0%-100%), neurosurgery 50% (22.7%-100%), plastic surgery 47.6% (30.0%-100%) and paediatric surgery 25% (16.7%-100%). Significant variance was observed across all specialties and deaneries (p=0.001).

CONCLUSION

As much as threefold variance exists related to FRCS examination first attempt success, trainees should be aware of this spectrum when preferencing deaneries during national selection.

Trends in national licensing examinations in medicine

CONTEXT

As a contribution to this special issue commemorating the journal's 50th volume, this paper seeks to explore directions for national licensing examinations (NLEs) in medicine. Increases in the numbers of new medical schools and the mobility of doctors across national borders mean that NLEs are becoming even more important to ensuring physician competence.

OBJECTIVES

The purpose of this paper is to explore the use of NLEs in the future in the context of global changes in medical education and health care delivery.

METHODS

Because the literature related to NLEs is so large, we have not attempted a comprehensive review, but have focused instead on a small number of topics on which we think we have something useful to say. The paper is organised around five predicted trends for NLEs.

DISCUSSION

The first section discusses reasons why we think the use of NLEs will increase in the coming years. The second section discusses the ongoing problem of content specificity and its implications for the design of NLEs. The third section examines the evolution of large-scale, standardised cognitive assessments in NLEs and suggests some future directions. Reflecting the fact that NLEs are, increasingly, attempting to assess more than just knowledge, the fourth section addresses the future of large-scale clinical skills assessments in NLEs, predicting both increases in their use and some shifts in the nature of the stations used. The fifth section discusses workplace-based assessments, predicting increases in their use for formative assessment and identifying some limitations in their direct application in NLEs. The concluding section discusses the cost of NLEs and indulges in some further speculations about their evolution.

Assessment at UK medical schools varies substantially in volume, type and intensity and correlates with postgraduate attainment

BACKGROUND

In the United Kingdom (UK), medical schools are free to develop local systems and policies that govern student assessment and progression. Successful completion of an undergraduate medical degree results in the automatic award of a provisional licence to practice medicine by the General Medical Council (GMC). Such a licensing process relies heavily on the assumption that individual schools develop similarly rigorous assessment policies. Little work has evaluated variability of undergraduate medical assessment between medical schools. That absence is important in the light of the GMC's recent announcement of the introduction of the UKMLA (UK Medical Licensing Assessment) for all doctors who wish to practise in the UK. The present study aimed to quantify and compare the volume, type and intensity of summative assessment across medicine (A100) courses in the United Kingdom, and to assess whether intensity of assessment correlates with the postgraduate attainment of doctors from these schools.

METHODS

Locally knowledgeable students in each school were approached to take part in guided-questionnaire interviews via telephone or Skype(TM). Their understanding of assessment at their medical school was probed, and later validated with the assessment department of the respective medical school. We gathered data for 25 of 27 A100 programmes in the UK and compared volume, type and intensity of assessment between schools. We then correlated these data with the mean first-attempt score of graduates sitting MRCGP and MRCP(UK), as well as with UKFPO selection measures.

RESULTS

The median written assessment volume across all schools was 2000 min (mean = 2027, SD = 586, LQ = 1500, UQ = 2500, range = 1000-3200) and 1400 marks (mean = 1555, SD = 463, LQ = 1200, UQ = 1800, range = 1100-2800). The median practical assessment volume was 400 min (mean = 472, SD = 207, LQ = 400, UQ = 600, range = 200-1000). The median intensity (minutes per mark ratio) of summative written assessment was 1.24 min per mark (mean = 1.28, SD = 0.30, LQ = 1.11, UQ = 1.37, range = 0.85-2.08). An exploratory analysis suggested a significant correlation of total assessment time with mean first-attempt score on both the knowledge and the clinical assessments of MRCGP and of MRCP(UK).

CONCLUSIONS

There are substantial differences in the volume, format and intensity of undergraduate assessment between UK medical schools. These findings suggest a potential for differences in the reliability of detecting poorly performing students, or differences in identifying and stratifying academically equivalent students for ranking in the Foundation Programme Application System (FPAS). Furthermore, these differences appear to directly correlate with performance in postgraduate examinations. Taken together, our findings highlight highly variable local assessment procedures that warrant further investigation to establish their potential impact on students.

UK medical selection: lottery or meritocracy?

From senior school through to consultancy, a plethora of assessments shape medical careers. Multiple methods of assessment are used to discriminate between applicants. Medical selection in the UK appears to be moving increasingly towards non-knowledge-based testing at all career stages. We review the evidence for non-knowledge-based tests and discuss their perceived benefits. We raise the question: is the current use of non-knowledge-based tests within the UK at risk of undermining more robust measures of medical school and postgraduate performance?

UK postgraduate medicine examinations: opportunities for international candidates

The medical profession is global, and ambitious trainee physicians around the world are eager to attain internationally recognised postgraduate medical qualifications. The MRCP(UK) and specialty certificate examinations of the Federation of Royal Colleges of Physicians of the United Kingdom provide such qualifications, and between 2002 and 2013, the number of international candidates attempting these examinations grew substantially. Delivering these proven and reliable UK-based examinations in other countries has many local benefits: it enhances careers, strengthens medical training and improves standards of patient care. In collaboration with international colleagues, the Federation is committed to continued growth that extends these benefits to all physicians, wherever they work and live.

A survey of factors influencing career preference in new-entrant and exiting medical students from four UK medical schools

BACKGROUND

Workforce planning is a central issue for service provision and has consequences for medical education. Much work has been examined the career intentions, career preferences and career destinations of UK medical graduates but there is little published about medical students career intentions. How soon do medical students formulate careers intentions? How much do these intentions and preferences change during medical school? If they do change, what are the determining factors? Our aim was to compare medical students' career preferences upon entry into and exit from undergraduate medical degree programmes.

METHODS

This was a cross-sectional questionnaire survey. Two cohorts [2009-10, 2010-11] of first and final year medical students at the four Scottish graduating medical schools took part in career preference questionnaire surveys. Questions were asked about demographic factors, career preferences and influencing factors.

RESULTS

The response rate was 80.9% [2682/3285]. Significant differences were found across the four schools, most obviously in terms of student origin [Scotland, rest of UK or overseas], age group, and specialty preferences in Year 1 and Year 5. Year 1 and Year 5 students' specialty preferences also differed within each school and, while there were some common patterns, each medical school had a different profile of students' career preferences on exit. When the analysis was adjusted for demographic and job-related preferences, specialty preferences differed by gender, and wish for work-life balance and intellectual satisfaction.

CONCLUSIONS

This is the first multi-centre study exploring students' career preferences and preference influences upon entry into and exit from undergraduate medical degree programmes. We found various factors influenced career preference, confirming prior findings. What this study adds is that, while acknowledging student intake differs by medical school, medical school itself seems to influence career preference. Comparisons across medical school populations must therefore control for differences in input [the students] as well as context and process [the medical school] when looking at output [e.g., performance]. A robust, longitudinal study is required to explore how medical students' career preferences change as they progress through medical school and training to understand the influence of the learning environment on training choice and outcomes.

Annual Review of Competence Progression (ARCP) performance of doctors who passed Professional and Linguistic Assessments Board (PLAB) tests compared with UK medical graduates: national data linkage study

OBJECTIVE

To determine whether use of the Professional and Linguistic Assessments Board (PLAB) examination system used to grant registration for international medical graduates results in equivalent postgraduate medical performance, as evaluated at Annual Review of Competence Progression (ARCP), between UK based doctors who qualified overseas and those who obtained their primary medical qualification from UK universities.

DESIGN

Observational study linking ARCP outcome data from the UK deaneries with PLAB test performance and demographic data held by the UK General Medical Council (GMC).

SETTING

Doctors in postgraduate training for a medical specialty or general practice in the UK and doctors obtaining GMC registration via the PLAB system.

PARTICIPANTS

53,436 UK based trainee doctors with at least one competency related ARCP outcome reported during the study period, of whom 42,017 were UK medical graduates and 11,419 were international medical graduates who were registered following a pass from the PLAB route.

MAIN OUTCOME MEASURE

Probability of obtaining a poorer versus a more satisfactory category of outcome at ARCP following successful registration as a doctor in the UK.

RESULTS

International medical graduates were more likely to obtain a less satisfactory outcome at ARCP compared with UK graduates. This finding persisted even after adjustment for the potential influence of sex, age, years of UK based practice, and ethnicity and exclusion of outcomes associated with postgraduate examination failure (odds ratio 1.63, 95% confidence interval 1.30 to 2.06). However, international medical graduates who scored in the highest twelfth at part 1 of the PLAB (at least 32 points above the pass mark) had ARCP outcomes that did not differ significantly from those of UK graduates.

CONCLUSIONS

These findings suggest that the PLAB test used for registration of international medical graduates is not generally equivalent to the requirements for UK graduates. The differences in postgraduate performance, as captured at ARCP, following the two routes to registration might be levelled out by raising the standards of English language competency required as well as the pass marks for the two parts of the PLAB test. An alternative might be to introduce a different testing system.

PLAB and UK graduates' performance on MRCP(UK) and MRCGP examinations: data linkage study

OBJECTIVES

To assess whether international medical graduates passing the two examinations set by the Professional and Linguistic Assessments Board (PLAB1 and PLAB2) of the General Medical Council (GMC) are equivalent to UK graduates at the end of the first foundation year of medical training (F1), as the GMC requires, and if not, to assess what changes in the PLAB pass marks might produce equivalence.

DESIGN

Data linkage of GMC PLAB performance data with data from the Royal Colleges of Physicians and the Royal College of General Practitioners on performance of PLAB graduates and UK graduates at the MRCP(UK) and MRCGP examinations.

SETTING

Doctors in training for internal medicine or general practice in the United Kingdom.

PARTICIPANTS

7829, 5135, and 4387 PLAB graduates on their first attempt at MRCP(UK) Part 1, Part 2, and PACES assessments from 2001 to 2012 compared with 18,532, 14,094, and 14,376 UK graduates taking the same assessments; 3160 PLAB1 graduates making their first attempt at the MRCGP AKT during 2007-12 compared with 14,235 UK graduates; and 1411 PLAB2 graduates making their first attempt at the MRCGP CSA during 2010-12 compared with 6935 UK graduates.

MAIN OUTCOME MEASURES

Performance at MRCP(UK) Part 1, Part 2, and PACES assessments, and MRCGP AKT and CSA assessments in relation to performance on PLAB1 and PLAB2 assessments, as well as to International English Language Testing System (IELTS) scores. MRCP(UK), MRCGP, and PLAB results were analysed as marks relative to the pass mark at the first attempt.

RESULTS

PLAB1 marks were a valid predictor of MRCP(UK) Part 1, MRCP(UK) Part 2, and MRCGP AKT (r=0.521, 0.390, and 0.490; all P<0.001). PLAB2 marks correlated with MRCP(UK) PACES and MRCGP CSA (r=0.274, 0.321; both P<0.001). PLAB graduates had significantly lower MRCP(UK) and MRCGP assessments (Glass's Δ=0.94, 0.91, 1.40, 1.01, and 1.82 for MRCP(UK) Part 1, Part 2, and PACES and MRCGP AKT and CSA), and were more likely to fail assessments and to progress more slowly than UK medical graduates. IELTS scores correlated significantly with later performance, multiple regression showing that the effect of PLAB1 (β=0.496) was much stronger than the effect of IELTS (β=0.086). Changes to PLAB pass marks that would result in international medical graduate and UK medical graduate equivalence were assessed in two ways. Method 1 adjusted PLAB pass marks to equate median performance of PLAB and UK graduates. Method 2 divided PLAB graduates into 12 equally spaced groups according to PLAB performance, and compared these with mean performance of graduates from individual UK medical schools, assessing which PLAB groups were equivalent in MRCP(UK) and MRCGP performance to UK graduates. The two methods produced similar results. To produce equivalent performance on the MRCP and MRGP examinations, the pass mark for PLAB1 would require raising by about 27 marks (13%) and for PLAB2 by about 15-16 marks (20%) above the present standard.

CONCLUSIONS

PLAB is a valid assessment of medical knowledge and clinical skills, correlating well with performance at MRCP(UK) and MRCGP. PLAB graduates' knowledge and skills at MRCP(UK) and MRCGP are over one standard deviation below those of UK graduates, although differences in training quality cannot be taken into account. Equivalent performance in MRCGP(UK) and MRCGP would occur if the pass marks of PLAB1 and PLAB2 were raised considerably, but that would also reduce the pass rate, with implications for medical workforce planning. Increasing IELTS requirements would have less impact on equivalence than raising PLAB pass marks.

The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors

BACKGROUND

Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.

METHODS

Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.

RESULTS

Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.

CONCLUSIONS

The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital' and 'medical capital'). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

BACKGROUND

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.

METHODS

Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.

RESULTS

Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.

CONCLUSIONS

Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools

BACKGROUND

Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.

METHODS

A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.

RESULTS

UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.

CONCLUSIONS

This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

BACKGROUND

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.

METHODS

Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.

RESULTS

Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.

CONCLUSIONS

Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

Performance at MRCP(UK): when should trainees sit examinations?

This paper analyses candidate performance in the three components of the MRCP(UK) examination to establish when they are most likely to pass. Using data from the 2010 and 2011 MRCP(UK) examinations, pass rates of candidates who gained their primary medical qualification (PMQ) in 2005 or later were analysed. Results from a total of 22,827 candidates were included in the study: 12,517 (54.8%) from Part 1, 5,545 (24.3%) from Part 2 written and 4,765 (20.9%) from the Part 2 practical assessment of clinical examination skills (PACES). The results show that candidates are more likely to pass Part 1 and Part 2 written 12-24 months after graduation and to pass Part 2 PACES 25-36 months after graduation. When we consider the training programme for physicians in the UK, successful candidates are likely to be in foundation programmes or early core or specialty training when they achieve success. At the moment, some candidates are dissuaded from taking the examination during their foundation programme, but our data show that their likelihood of success is highest during this period of training. The analysis also shows that for candidates who fail their first attempt, delaying their next attempt by one diet significantly increases the likelihood of them passing at their next attempt.

Perceptions of UK medical graduates' preparedness for practice: a multi-centre qualitative study reflecting the importance of learning on the job

BACKGROUND

There is evidence that graduates of different medical schools vary in their preparedness for their first post. In 2003 Goldacre et al. reported that over 40% of UK medical graduates did not feel prepared and found large differences between graduates of different schools. A follow-up survey showed that levels of preparedness had increased yet there was still wide variation. This study aimed to examine whether medical graduates from three diverse UK medical schools were prepared for practice.

METHODS

This was a qualitative study using a constructivist grounded theory approach. Prospective and cross-sectional data were collected from the three medical schools.A sample of 60 medical graduates (20 from each school) was targeted. They were interviewed three times: at the end of medical school (n = 65) and after four (n = 55) and 12 months (n = 46) as a Year 1 Foundation Programme doctor. Triangulated data were collected from clinicians via interviews across the three sites (n = 92). In addition three focus groups were conducted with senior clinicians who assess learning portfolios. The focus was on identifying areas of preparedness for practice and any areas of lack of preparedness.

RESULTS

Although selected for being diverse, we did not find substantial differences between the schools. The same themes were identified at each site. Junior doctors felt prepared in terms of communication skills, clinical and practical skills and team working. They felt less prepared for areas of practice that are based on experiential learning in clinical practice: ward work, being on call, management of acute clinical situations, prescribing, clinical prioritisation and time management and dealing with paperwork.

CONCLUSIONS

Our data highlighted the importance of students learning on the job, having a role in the team in supervised practice to enable them to learn about the duties and responsibilities of a new doctor in advance of starting work.

Associations between medical school and career preferences in Year 1 medical students in Scotland

OBJECTIVES

Little is known about the relationship between the career preferences of medical students and the medical schools at which they are enrolled. Our aim was to explore this relationship early in students' medical training.

METHODS

Year 1 (2009-2010) medical students at the five Scottish medical schools were invited to take part in a career preference questionnaire survey. Questions were asked about demographic factors, career preferences and influencing factors.

RESULTS

The response rate was 87.9% (883/1005). No significant differences were found among medical schools with regard to first-choice specialty. Surgery (22.5%), medicine (19.0%), general practice (17.6%) and paediatrics (16.1%) were the top career choices. Work-life balance, perceived aptitude and skills, intellectual satisfaction, and amount of patient contact were rated as the most important job-related factors by most respondents. Few differences were found among schools in terms of the impact of job-related factors on future career preferences. Students for whom the work-life balance was extremely important (odds ratio [OR]=0.6) were less likely to prefer surgery. Students for whom the work-life balance (OR=2.2) and continuity of care (OR=2.1) were extremely important were more likely to prefer general practice.

CONCLUSIONS

Students' early career preferences were similar across the five medical schools. These preferences result from the interplay among demographic factors and the perceived characteristics of the various specialties. Maintaining a satisfactory work-life balance is very important to tomorrow's doctors, and the data hint that this may be breaking down some of the traditional gender differences in specialty choice. Longitudinal work is required to explore whether students' career preferences change as they progress through medical school and training.

The mediators of minority ethnic underperformance in final medical school examinations

BACKGROUND

UK-trained medical students and doctors from minority ethnic groups underperform academically. It is unclear why this problem exists, which makes it difficult to know how to address it.

AIM

To investigate whether demographic and psychological factors mediate the relationship between ethnicity and final examination scores.

SAMPLE

Two consecutive cohorts of Year 5 (final year) UCL Medical School students (n= 703; 51% minority ethnic). A total of 587 (83%) had previously completed a questionnaire in Year 3.

METHODS

Participants were administered a questionnaire in 2005 and 2006 that included a short version of the NEO-PI-R, the Study Process Questionnaire, and the General Health Questionnaire (GHQ) as well as socio-demographic measures. Participants were then followed up to final year (2007-2010). White and minority ethnic students' questionnaire responses and final examination grades were compared using univariate tests. The effect of ethnicity on final year grades after taking into account the questionnaire variables was calculated using hierarchical multiple linear regression.

RESULTS

Univariate ethnic differences were found on age, personality, learning styles, living at home, first language, parental factors, and prior education. Minority ethnic students had lower final exam scores, were more likely to fail, and less likely to achieve a merit or distinction in finals. Multivariate analyses showed ethnicity predicted final exam scores even after taking into account questionnaire factors.

CONCLUSIONS

Ethnic differences in the final year performance of two cohorts of UCL medical students were not due to differences in psychological or demographic factors, which suggests alternative explanations are responsible for the ethnic attainment gap in medicine.

Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis

OBJECTIVE

To determine whether the ethnicity of UK trained doctors and medical students is related to their academic performance.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Online databases PubMed, Scopus, and ERIC; Google and Google Scholar; personal knowledge; backwards and forwards citations; specific searches of medical education journals and medical education conference abstracts.

STUDY SELECTION

The included quantitative reports measured the performance of medical students or UK trained doctors from different ethnic groups in undergraduate or postgraduate assessments. Exclusions were non-UK assessments, only non-UK trained candidates, only self reported assessment data, only dropouts or another non-academic variable, obvious sampling bias, or insufficient details of ethnicity or outcomes. Results 23 reports comparing the academic performance of medical students and doctors from different ethnic groups were included. Meta-analyses of effects from 22 reports (n = 23,742) indicated candidates of "non-white" ethnicity underperformed compared with white candidates (Cohen's d = -0.42, 95% confidence interval -0.50 to -0.34; P<0.001). Effects in the same direction and of similar magnitude were found in meta-analyses of undergraduate assessments only, postgraduate assessments only, machine marked written assessments only, practical clinical assessments only, assessments with pass/fail outcomes only, assessments with continuous outcomes only, and in a meta-analysis of white v Asian candidates only. Heterogeneity was present in all meta-analyses.

CONCLUSION

Ethnic differences in academic performance are widespread across different medical schools, different types of exam, and in undergraduates and postgraduates. They have persisted for many years and cannot be dismissed as atypical or local problems. We need to recognise this as an issue that probably affects all of UK medical and higher education. More detailed information to track the problem as well as further research into its causes is required. Such actions are necessary to ensure a fair and just method of training and of assessing current and future doctors.

Predicting medical student performance from attributes at entry: a latent class analysis

OBJECTIVES

The objectives of this study were two-fold. Firstly, we aimed to model 'typologies' of student examination performance by grouping students into like categories based on measures of prior academic achievement (particularly in the science subjects) and interview rating at time of entry to a medical degree course, and outcome measures of subsequent performance across the course. Secondly, we aimed to illustrate and evidence the utility of the latent class analysis (LCA) clustering technique to provide meaningful information on the effectiveness of a student selection process with respect to the likelihood of poor examination performance.

METHODS

For this retrospective study, anonymised data on two sequential cohorts of students who graduated from a 5-year Bachelor of Medicine, Bachelor of Surgery degree course were analysed using LCA. In order to triangulate the findings, the same data were analysed using the more conventional approach of logistic regression.

RESULTS

The LCA identified three distinct classes or typologies of student examination performance using measures of prior academic achievement and interview rating at time of course entry. Measures of prior academic achievement and score on a structured admissions interview made significant contributions to the model's ability to discriminate between typologies. Strong prior academic achievement, especially in chemistry, and high interview score were positively related to the likelihood of successful test performance. These findings were supported by the logistic regression analysis.

CONCLUSIONS

The LCA clustering technique provided meaningful information on the performance of a selection process. As a complementary tool to existing methods used in this area of research, LCA has the potential to empirically inform the selection process.

Making sense of differences between medical schools through Bourdieu's concept of 'field'

OBJECTIVES

Most basic medical education studies focus on one medical school, and the social and educational significance of differences between schools remains poorly understood. Sociologists have called for more comparative studies and for research on the institutional context of medical education. This paper introduces Bourdieu's concept of 'field' as a tool for conducting such analyses.

CONTEXT

A 'field' is any arena in which there is a struggle over resources (capital), akin to a 'game' between players who occupy different positions depending on the resources they have. Prior studies show that higher education institutions compete in a field for various forms of capital which are reflected in their curricula and the students they attract.

METHODS

This paper argues that medical education can be conceptualised as a field within which medical schools compete for different forms of capital, such as students, funding and prestige. The structure of the UK medical education field is analysed as a specific example of how Bourdieu's framework can be applied. It is argued that UK medical schools' varying curricula and admissions criteria serve to distinguish them from their competitors and to facilitate access to different forms of capital. Competition within the field helps to maintain inter-school differences, with implications for both curriculum reform and students' beliefs and aspirations.

CONCLUSIONS

Medical schools have varying curricula, reputations, and types and levels of resources. They compete with one another on all these fronts and attract different types of students and staff. Research and practice in basic medical education must take account of the position of any given medical school in relation to its competitors and to external agencies in order to critically consider the ethos of its curriculum and the perspectives of its students and staff. Bourdieu's concept of field offers one useful way of accomplishing this.

An evaluation of the performance in the UK Royal College of Anaesthetists primary examination by UK medical school and gender

BACKGROUND

There has been comparatively little consideration of the impact that the changes to undergraduate curricula might have on postgraduate academic performance. This study compares the performance of graduates by UK medical school and gender in the Multiple Choice Question (MCQ) section of the first part of the Fellowship of the Royal College of Anaesthetists (FRCA) examination.

METHODS

Data from each sitting of the MCQ section of the primary FRCA examination from June 1999 to May 2008 were analysed for performance by medical school and gender.

RESULTS

There were 4983 attempts at the MCQ part of the examination by 3303 graduates from the 19 United Kingdom medical schools. Using the standardised overall mark minus the pass mark graduates from five medical schools performed significantly better than the mean for the group and five schools performed significantly worse than the mean for the group. Males performed significantly better than females in all aspects of the MCQ - physiology, mean difference = 3.0% (95% CI 2.3, 3.7), p < 0.001; pharmacology, mean difference = 1.7% (95% CI 1.0, 2.3), p < 0.001; physics with clinical measurement, mean difference = 3.5% (95% CI 2.8, 4.1), p < 0.001; overall mark, mean difference = 2.7% (95% CI 2.1, 3.3), p < 0.001; and standardised overall mark minus the pass mark, mean difference = 2.5% (95% CI 1.9, 3.1), p < 0.001. Graduates from three medical schools that have undergone the change from Traditional to Problem Based Learning curricula did not show any change in performance in any aspects of the MCQ pre and post curriculum change.

CONCLUSION

Graduates from each of the medical schools in the UK do show differences in performance in the MCQ section of the primary FRCA, but significant curriculum change does not lead to deterioration in post graduate examination performance. Whilst females now outnumber males taking the MCQ, they are not performing as well as the males.

Licensing examinations in North America: is external audit valuable?

The United States and Canada both have long-standing, highly developed national systems of assessment for medical-licensure based outside the institutions of medical education. This commentary reviews those programs and explores some of the reasons for their implementation and retention for nearly a century. The North American experience may be relevant to dialog about national or European assessments for medical practice.