Compellingly negative: Bayesian analysis shows handedness and dementia are not associated

This scientific commentary refers to "The role of hand preference in cognition and neuropsychiatric symptoms in neurodegenerative diseases" by Saari & Vuoksimaa (https://doi.org/10.1093/braincomms/fcad137).

Institutional choice among medical applicants: a profile paper for The United Kingdom Medical Applicant Cohort Study (UKMACS) prospective longitudinal cohort study

PURPOSE

To generate a large cohort of those in 2019 seriously considering applying to study Medicine, collecting data on a range of socioeconomic and other demographic factors that influence choice of medical schools and to link to other datasets to form a longitudinal study of progress through medical school and careers in medicine.

DESIGN

Cross-sectional questionnaire studies, part of the longitudinal UK Medical Applicant Cohort Study (UKMACS).

SETTING

UK medical school admissions in 2020.

PARTICIPANTS

UK residents aged 16+ and seriously considering applying to study Medicine. The cohort was primarily drawn from those registering in 2019 for the U(K)CAT (University Clinical Aptitude Test (formerly the UK Clinical Aptitude Test)) with additional potential applicants responding to an open call. Participants consented to their data being linked within the UK Medical Education Database.

FINDINGS TO DATE

UKMACS Wave 1 questionnaire respondents consisted of 6391 consenting respondents from across the UK. In 2019, 14 980 of the 17 470 UK-domiciled medicine applicants were first-time applicants. The questionnaires show that many of these applicants have a need for more help and guidance to make informed choices, with less advantaged groups reporting themselves as being at a disadvantage when applying due to limited understanding of information and limited access to guidance to enable informed and effective decision-making.

FUTURE PLANS

To link the cohort with successive Universities and Colleges Admissions Service and other datasets to analyse outcomes of applications and establish national longitudinal evidence to understand how medical choices are made and how they impact on educational, career and workforce outcomes.

Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study

OBJECTIVE

To determine whether scores on two undergraduate admissions tests (BioMedical Admissions Test (BMAT) and University Clinical Aptitude Test (UCAT)) predict performance on the postgraduate Membership of the Royal Colleges of Physicians (MRCP) examination, including the clinical examination Practical Assessment of Clinical Examination Skills (PACES).

DESIGN

National cohort study.

SETTING

Doctors who graduated medical school between 2006 and 2018.

PARTICIPANTS

3045 doctors who had sat BMAT, UCAT and the MRCP.

PRIMARY OUTCOME MEASURES

Passing each section of the MRCP at the first attempt, including the clinical assessment PACES.

RESULTS

Several BMAT and UCAT subtest scores displayed incremental predictive validity for performance on the first two (written) parts of the MRCP. Only aptitude and skills on BMAT (OR 1.34, 1.08 to 1.67, p=0.01) and verbal reasoning on UCAT (OR 1.34, 1.04 to 1.71, p=0.02) incrementally predicted passing PACES at the first attempt.

CONCLUSIONS

Our results imply that the abilities assessed by aptitude and skills and verbal reasoning may be the most important cognitive attributes, of those routinely assessed at selection, for predicting future clinical performance. Selectors may wish to consider placing particular weight on scales assessing these attributes if they wish to select applicants likely to become more competent clinicians. These results are potentially relevant in an international context too, since many admission tests used globally, such as the Medical College Admission Test, assess similar abilities.

Predictive validity of A-level grades and teacher-predicted grades in UK medical school applicants: a retrospective analysis of administrative data in a time of COVID-19

OBJECTIVES

To compare in UK medical students the predictive validity of attained A-level grades and teacher-predicted A levels for undergraduate and postgraduate outcomes. Teacher-predicted A-level grades are a plausible proxy for the teacher-estimated grades that replaced UK examinations in 2020 as a result of the COVID-19 pandemic. The study also models the likely future consequences for UK medical schools of replacing public A-level examination grades with teacher-predicted grades.

DESIGN

Longitudinal observational study using UK Medical Education Database data.

SETTING

UK medical education and training.

PARTICIPANTS

Dataset 1: 81 202 medical school applicants in 2010-2018 with predicted and attained A-level grades. Dataset 2: 22 150 18-year-old medical school applicants in 2010-2014 with predicted and attained A-level grades, of whom 12 600 had medical school assessment outcomes and 1340 had postgraduate outcomes available.

OUTCOME MEASURES

Undergraduate and postgraduate medical examination results in relation to attained and teacher-predicted A-level results.

RESULTS

Dataset 1: teacher-predicted grades were accurate for 48.8% of A levels, overpredicted in 44.7% of cases and underpredicted in 6.5% of cases. Dataset 2: undergraduate and postgraduate outcomes correlated significantly better with attained than with teacher-predicted A-level grades. Modelling suggests that using teacher-estimated grades instead of attained grades will mean that 2020 entrants are more likely to underattain compared with previous years, 13% more gaining the equivalent of the lowest performance decile and 16% fewer reaching the equivalent of the current top decile, with knock-on effects for postgraduate training.

CONCLUSIONS

The replacement of attained A-level examination grades with teacher-estimated grades as a result of the COVID-19 pandemic may result in 2020 medical school entrants having somewhat lower academic performance compared with previous years. Medical schools may need to consider additional teaching for entrants who are struggling or who might need extra support for missed aspects of A-level teaching.

The Analysis of Teaching of Medical Schools (AToMS) survey: an analysis of 47,258 timetabled teaching events in 25 UK medical schools relating to timing, duration, teaching formats, teaching content, and problem-based learning

BACKGROUND

What subjects UK medical schools teach, what ways they teach subjects, and how much they teach those subjects is unclear. Whether teaching differences matter is a separate, important question. This study provides a detailed picture of timetabled undergraduate teaching activity at 25 UK medical schools, particularly in relation to problem-based learning (PBL).

METHOD

The Analysis of Teaching of Medical Schools (AToMS) survey used detailed timetables provided by 25 schools with standard 5-year courses. Timetabled teaching events were coded in terms of course year, duration, teaching format, and teaching content. Ten schools used PBL. Teaching times from timetables were validated against two other studies that had assessed GP teaching and lecture, seminar, and tutorial times.

RESULTS

A total of 47,258 timetabled teaching events in the academic year 2014/2015 were analysed, including SSCs (student-selected components) and elective studies. A typical UK medical student receives 3960 timetabled hours of teaching during their 5-year course. There was a clear difference between the initial 2 years which mostly contained basic medical science content and the later 3 years which mostly consisted of clinical teaching, although some clinical teaching occurs in the first 2 years. Medical schools differed in duration, format, and content of teaching. Two main factors underlay most of the variation between schools, Traditional vs PBL teaching and Structured vs Unstructured teaching. A curriculum map comparing medical schools was constructed using those factors. PBL schools differed on a number of measures, having more PBL teaching time, fewer lectures, more GP teaching, less surgery, less formal teaching of basic science, and more sessions with unspecified content.

DISCUSSION

UK medical schools differ in both format and content of teaching. PBL and non-PBL schools clearly differ, albeit with substantial variation within groups, and overlap in the middle. The important question of whether differences in teaching matter in terms of outcomes is analysed in a companion study (MedDifs) which examines how teaching differences relate to university infrastructure, entry requirements, student perceptions, and outcomes in Foundation Programme and postgraduate training.

Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise

BACKGROUND

Medical schools differ, particularly in their teaching, but it is unclear whether such differences matter, although influential claims are often made. The Medical School Differences (MedDifs) study brings together a wide range of measures of UK medical schools, including postgraduate performance, fitness to practise issues, specialty choice, preparedness, satisfaction, teaching styles, entry criteria and institutional factors.

METHOD

Aggregated data were collected for 50 measures across 29 UK medical schools. Data include institutional history (e.g. rate of production of hospital and GP specialists in the past), curricular influences (e.g. PBL schools, spend per student, staff-student ratio), selection measures (e.g. entry grades), teaching and assessment (e.g. traditional vs PBL, specialty teaching, self-regulated learning), student satisfaction, Foundation selection scores, Foundation satisfaction, postgraduate examination performance and fitness to practise (postgraduate progression, GMC sanctions). Six specialties (General Practice, Psychiatry, Anaesthetics, Obstetrics and Gynaecology, Internal Medicine, Surgery) were examined in more detail.

RESULTS

Medical school differences are stable across time (median alpha = 0.835). The 50 measures were highly correlated, 395 (32.2%) of 1225 correlations being significant with p < 0.05, and 201 (16.4%) reached a Tukey-adjusted criterion of p < 0.0025. Problem-based learning (PBL) schools differ on many measures, including lower performance on postgraduate assessments. While these are in part explained by lower entry grades, a surprising finding is that schools such as PBL schools which reported greater student satisfaction with feedback also showed lower performance at postgraduate examinations. More medical school teaching of psychiatry, surgery and anaesthetics did not result in more specialist trainees. Schools that taught more general practice did have more graduates entering GP training, but those graduates performed less well in MRCGP examinations, the negative correlation resulting from numbers of GP trainees and exam outcomes being affected both by non-traditional teaching and by greater historical production of GPs. Postgraduate exam outcomes were also higher in schools with more self-regulated learning, but lower in larger medical schools. A path model for 29 measures found a complex causal nexus, most measures causing or being caused by other measures. Postgraduate exam performance was influenced by earlier attainment, at entry to Foundation and entry to medical school (the so-called academic backbone), and by self-regulated learning. Foundation measures of satisfaction, including preparedness, had no subsequent influence on outcomes. Fitness to practise issues were more frequent in schools producing more male graduates and more GPs.

CONCLUSIONS

Medical schools differ in large numbers of ways that are causally interconnected. Differences between schools in postgraduate examination performance, training problems and GMC sanctions have important implications for the quality of patient care and patient safety.

Fitness to practise sanctions in UK doctors are predicted by poor performance at MRCGP and MRCP(UK) assessments: data linkage study

BACKGROUND

The predictive validity of postgraduate examinations, such as MRCGP and MRCP(UK) in the UK, is hard to assess, particularly for clinically relevant outcomes. The sanctions imposed on doctors by the UK's General Medical Council (GMC), including erasure from the Medical Register, are indicators of serious problems with fitness to practise (FtP) that threaten patient safety or wellbeing. This data linkage study combined data on GMC sanctions with data on postgraduate examination performance.

METHODS

Examination results were obtained for UK registered doctors taking the MRCGP Applied Knowledge Test (AKT; n = 27,561) or Clinical Skills Assessment (CSA; n = 17,365) at first attempt between 2010 and 2016 or taking MRCP(UK) Part 1 (MCQ; n = 37,358), Part 2 (MCQ; n = 28,285) or Practical Assessment of Clinical Examination Skills (PACES; n = 27,040) at first attempt between 2001 and 2016. Exam data were linked with GMC actions on a doctor's registration from September 2008 to January 2017, sanctions including Erasure, Suspension, Conditions on Practice, Undertakings or Warnings (ESCUW). Examination results were only considered at first attempts. Multiple logistic regression assessed the odds ratio for ESCUW in relation to examination results. Multiple imputation was used for structurally missing values.

RESULTS

Doctors sanctioned by the GMC performed substantially less well on MRCGP and MRCP(UK), with a mean Cohen's d across the five exams of - 0.68. Doctors on the 2.5th percentile of exam performance were about 12 times more likely to have FtP problems than those on the 97.5th percentile. Knowledge assessments and clinical assessments were independent predictors of future sanctions, with clinical assessments predicting ESCUW significantly better. The log odds of an FtP sanction were linearly related to examination marks over the entire range of performance, additional performance increments lowering the risk of FtP sanctions at all performance levels.

CONCLUSIONS

MRCGP and MRCP(UK) performance are valid predictors of professionally important outcomes that transcend simple knowledge or skills and the GMC puts under the headings of conduct and trust. Postgraduate examinations may predict FtP sanctions because the psychological processes involved in successfully studying, understanding and practising medicine at a high level share similar mechanisms to those underlying conduct and trust.

Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK

BACKGROUND

Little research has compared the profile, success, or specialty destinations of graduates entering UK medical schools via accelerated, 4-yr, standard 5-yr and 6-yr programmes. Four research questions directed this investigation:- What are the success rates for graduates entering graduate-entry vs. undergraduate medicine courses? How does the sociodemographic and educational profile differ between these two groups? Is success - in medical school and foundation training - dependent on prior degree, demographic factors, or aptitude test performance at selection? What specialty do graduate entry medicine students subsequently enter?

METHODS

The data from two cohorts of graduates entering medical school in 2007 and 2008 (n = 2761) in the UKMED (UK Medical Education Database) database were studied: 1445 taking 4-yr and 1150 taking 5-yr medicine courses, with smaller numbers following other programmes.

RESULTS

Completion rates for degree programmes were high at 95%, with no significant difference between programme types. 4-yr entrants were older, less likely to be from Asian communities, had lower HESA (Higher Education Statistics Agency) tariff scores, but higher UKCAT (UK Clinical Aptitude Test) and GAMSAT (Graduate Medical School Admissions Test) scores, than 5-yr entrants. Higher GAMSAT scores, black or minority ethnicity (BME), and younger age were independent predictors of successful completion of medical school. Foundation Programme (FPAS) selection measures (EPM - educational performance measure; SJT - situational judgment test) were positively associated with female sex, but negatively with black or minority ethnicity. Higher aptitude test scores were associated with EPM and SJT, GAMSAT with EPM, UKCAT with SJT. Prior degree subject, class of degree, HESA tariff, and type of medicine programme were not related to success.

CONCLUSIONS

The type of medicine programme has little effect on graduate entrant completion, or EPM or SJT scores, despite differences in student profile. Aptitude test score has some predictive validity, as do sex, age and BME, but not prior degree subject or class. Further research is needed to disentangle the influences of BME.

'Best friends forever'? Friendship stability across school transition and associations with mental health and educational attainment

BACKGROUND

Friendships have been linked to mental health and school attainment in children. The effects of friendlessness and friendship quality have been well researched, but less is known about the role of friendship stability (i.e., maintaining the same friend over time), an aspect of friendship which is often interrupted by the transition between phases of schooling. Many children report concerns about the secondary school transition which introduces a number of new social and academic challenges for children.

AIMS

To explore rates of friendship stability and whether maintaining a stable best friend across the primary to secondary school transition provided benefits to children's adjustment during this period.

SAMPLE

Data were from 593 children (M age = 11 years 2 months).

METHODS

This study used longitudinal data from children transitioning into 10 UK secondary schools and explored the association between self-reported friendship stability and three outcomes: academic attainment, emotional problems and conduct problems. Analyses controlled for friendship quality and pre-transition psychological adjustment or attainment as appropriate.

RESULTS

Rates of friendship stability were relatively low during this period. Children who kept the same best friend had higher academic attainment and lower levels of conduct problems. Exploratory analyses indicated that secondary school policies that group children based on friendships may support friendship stability.

CONCLUSIONS

Helping maintain children's best friendships during the transition to secondary school may contribute to higher academic performance and better mental health.

Meta-analytic findings reveal lower means but higher variances in visuospatial ability in dyslexia

Conflicting empirical and theoretical accounts suggest that dyslexia is associated with either average, enhanced, or impoverished high-level visuospatial processing relative to controls. Such heterogeneous results could be due to the presence of wider variability in dyslexic samples, which is unlikely to be identified at the single study level, due to lack of power. To address this, this study reports a meta-analysis of means and variances in high-level visuospatial ability in 909 non-dyslexic and 956 dyslexic individuals. The findings suggest that dyslexia is associated not only with a lower mean performance on visuospatial tasks but also with greater variability in performance. Through novel meta-analytic techniques, we demonstrate a negative effect size for mean differences (-.457), but a positive effect size for SD differences (+.118; SD ratio = 1.107). In doing so, this is the first study to demonstrate impoverished visuospatial processing of the majority of individuals with dyslexia in addition to greater variance in performance in this group. The findings advocate for further consideration of both the presence of, and reasons for, increased variance in perception, attention, and memory across neurodevelopmental disorders.

Would changing the selection process for GP trainees stem the workforce crisis? A cohort study using multiple-imputation and simulation

BACKGROUND

There is currently a shortage of qualified GPs in the UK and not all of the training posts available each year are filled. Changing the way in which GP trainees are selected could help increase the training post fill rate and the number of new entrants to the GP Register. The aim of this study was to model the impact of changing the selection process for GP training on the number of trainees obtaining GP Registration, either with or without extensions.

METHOD

This was a cohort study using UK applications for GP training in 2011-14. Application data were linked using GMC numbers to training outcome data where available, and imputed using multiple imputation where missing. The number of trainees appointed and GP Registrations within three and five years' full-time-equivalent were estimated for four different selection processes.

RESULTS

The cut scores used in the actual 2015 selection process makes it impossible to fill all training posts. Random selection is the worst option, but the difference between this and other processes modelled falls as more trainees are selected. There are large marginal effects on outcomes: those with the highest selection scores are more likely to obtain GP Registration than those with the lowest scores.

CONCLUSIONS

Changing the selection process alone would have a small impact on the number of GP Registrations; reducing/removing cut scores would have a much larger impact. This would also increase the number of trainees requiring extensions and being released from training which would have adverse consequences for the profession.

Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment

BACKGROUND

Fairness is a critical component of defensible assessment. Candidates should perform according to ability without influence from background characteristics such as ethnicity or sex. However, performance differs by candidate background in many assessment environments. Many potential causes of such differences exist, and examinations must be routinely analysed to ensure they do not present inappropriate progression barriers for any candidate group. By analysing the individual questions of an examination through techniques such as Differential Item Functioning (DIF), we can test whether a subset of unfair questions explains group-level differences. Such items can then be revised or removed.

METHODS

We used DIF to investigate fairness for 13,694 candidates sitting a major international summative postgraduate examination in internal medicine. We compared (a) ethnically white UK graduates against ethnically non-white UK graduates and (b) male UK graduates against female UK graduates. DIF was used to test 2773 questions across 14 sittings.

RESULTS

Across 2773 questions eight (0.29%) showed notable DIF after correcting for multiple comparisons: seven medium effects and one large effect. Blinded analysis of these questions by a panel of clinician assessors identified no plausible explanations for the differences. These questions were removed from the question bank and we present them here to share knowledge of questions with DIF. These questions did not significantly impact the overall performance of the cohort. Group-level differences in performance between the groups we studied in this examination cannot be explained by a subset of unfair questions.

CONCLUSIONS

DIF helps explore fairness in assessment at the question level. This is especially important in high-stakes assessment where a small number of unfair questions may adversely impact the passing rates of some groups. However, very few questions exhibited notable DIF so differences in passing rates for the groups we studied cannot be explained by unfairness at the question level.

Faking handedness: Individual differences in ability to fake handedness, social cognitions of the handedness of others, and a forensic application using Bayes' theorem

People usually describe their handedness honestly, but that need not necessarily be the case. A legal case is described of a murder said by the pathologist to be committed by a left-hander but the defendant claimed to be right-handed, and the first author assessed the defendant's handedness as an expert witness. We know of no previous work on faking handedness, and so we tested 30 right-handers and 25 left-handers on various handedness tasks, and then asked the participants to repeat the tasks while faking being of opposite handedness. Social cognitions of handedness were assessed from participants' knowledge of how other right- and left-handers would answer handedness questionnaires. Fake handedness was best differentiated using cursive lower-case sentence writing, upper-case written letters being less good at distinguishing, as also were simple motor tasks. Participants differed in social cognitions of handedness, and those with more accurate social cognitions were better able to fake. Personality measures did not predict faking ability. For forensic purposes a Bayesian analysis was carried out to evaluate the likelihood of right and left hand performance being true rather than faked, and the cursive lower-case writing provided strong posterior odds that, as claimed, the particular defendant was a true right-hander.

Stigmatization, Physical Illness and Mental Health in Primary Ciliary Dyskinesia

Primary ciliary dyskinesia (PCD) causes chronic cough, sinusitis and bronchiectasis, and half of patients also show situs inversus. The genetic basis and visible and concealed chronic symptoms provide potential for stigmatization. We describe a structural equation model linking a questionnaire measure of stigmatization to sex, age, personality (Big Five), symptoms (St George’s Respiratory Questionnaire), health status (SF-36) and stress (GHQ-12). Stigma did not relate to physical symptoms or health, or to situs, but correlated with mental health and the social impact of symptoms. Neuroticism, extroversion, openness to experience, age, age at diagnosis and being female indirectly affected stigmatization via mental health.

A longitudinal study of self-control at the transition to secondary school: Considering the role of pubertal status and parenting

Higher self-control in children and adolescents is associated with a range of positive outcomes in adulthood. However, little is known about the naturalistic development of self-control during early adolescence and the factors that affect this. We examined the role of puberty and parenting style as theoretically important influences on stability and change in self-control. A longitudinal (3 waves), multiple-informant dataset of children entering early adolescence (M = 11 years) was used to explore longitudinal change in self-control using latent growth curve modelling. Children's self-control declined during the one-year study period and declines were associated with children's behavioural and social functioning. Associations with self-control were found for pubertal status and parental warmth and hostility, but not for parental discipline. The findings suggest that during early adolescence, when children make the transition to secondary school, self-control declines. This is particularly the case for those experiencing puberty earlier than their peers. Parent warmth influences the trajectory of self-control during this period.

Assessment at UK medical schools varies substantially in volume, type and intensity and correlates with postgraduate attainment

BACKGROUND

In the United Kingdom (UK), medical schools are free to develop local systems and policies that govern student assessment and progression. Successful completion of an undergraduate medical degree results in the automatic award of a provisional licence to practice medicine by the General Medical Council (GMC). Such a licensing process relies heavily on the assumption that individual schools develop similarly rigorous assessment policies. Little work has evaluated variability of undergraduate medical assessment between medical schools. That absence is important in the light of the GMC's recent announcement of the introduction of the UKMLA (UK Medical Licensing Assessment) for all doctors who wish to practise in the UK. The present study aimed to quantify and compare the volume, type and intensity of summative assessment across medicine (A100) courses in the United Kingdom, and to assess whether intensity of assessment correlates with the postgraduate attainment of doctors from these schools.

METHODS

Locally knowledgeable students in each school were approached to take part in guided-questionnaire interviews via telephone or Skype(TM). Their understanding of assessment at their medical school was probed, and later validated with the assessment department of the respective medical school. We gathered data for 25 of 27 A100 programmes in the UK and compared volume, type and intensity of assessment between schools. We then correlated these data with the mean first-attempt score of graduates sitting MRCGP and MRCP(UK), as well as with UKFPO selection measures.

RESULTS

The median written assessment volume across all schools was 2000 min (mean = 2027, SD = 586, LQ = 1500, UQ = 2500, range = 1000-3200) and 1400 marks (mean = 1555, SD = 463, LQ = 1200, UQ = 1800, range = 1100-2800). The median practical assessment volume was 400 min (mean = 472, SD = 207, LQ = 400, UQ = 600, range = 200-1000). The median intensity (minutes per mark ratio) of summative written assessment was 1.24 min per mark (mean = 1.28, SD = 0.30, LQ = 1.11, UQ = 1.37, range = 0.85-2.08). An exploratory analysis suggested a significant correlation of total assessment time with mean first-attempt score on both the knowledge and the clinical assessments of MRCGP and of MRCP(UK).

CONCLUSIONS

There are substantial differences in the volume, format and intensity of undergraduate assessment between UK medical schools. These findings suggest a potential for differences in the reliability of detecting poorly performing students, or differences in identifying and stratifying academically equivalent students for ranking in the Foundation Programme Application System (FPAS). Furthermore, these differences appear to directly correlate with performance in postgraduate examinations. Taken together, our findings highlight highly variable local assessment procedures that warrant further investigation to establish their potential impact on students.

Cross-comparison of MRCGP & MRCP(UK) in a database linkage study of 2,284 candidates taking both examinations: assessment of validity and differential performance by ethnicity

BACKGROUND

MRCGP and MRCP(UK) are the main entry qualifications for UK doctors entering general [family] practice or hospital [internal] medicine. The performance of MRCP(UK) candidates who subsequently take MRCGP allows validation of each assessment. In the UK, underperformance of ethnic minority doctors taking MRCGP has had a high political profile, with a Judicial Review in the High Court in April 2014 for alleged racial discrimination. Although the legal challenge was dismissed, substantial performance differences between white and BME (Black and Minority Ethnic) doctors undoubtedly exist. Understanding ethnic differences can be helped by comparing the performance of doctors who take both MRCGP and MRCP(UK).

METHODS

We identified 2,284 candidates who had taken one or more parts of both assessments, MRCP(UK) typically being taken 3.7 years before MRCGP. We analyzed performance on knowledge-based MCQs (MRCP(UK) Parts 1 and 2 and MRCGP Applied Knowledge Test (AKT)) and clinical examinations (MRCGP Clinical Skills Assessment (CSA) and MRCP(UK) Practical Assessment of Clinical Skills (PACES)).

RESULTS

Correlations between MRCGP and MRCP(UK) were high, disattenuated correlations for MRCGP AKT with MRCP(UK) Parts 1 and 2 being 0.748 and 0.698, and for CSA and PACES being 0.636. BME candidates performed less well on all five assessments (P < .001). Correlations disaggregated by ethnicity were complex, MRCGP AKT showing similar correlations with Part1/Part2/PACES in White and BME candidates, but CSA showing stronger correlations with Part1/Part2/PACES in BME candidates than in White candidates. CSA changed its scoring method during the study; multiple regression showed the newer CSA was better predicted by PACES than the previous CSA.

CONCLUSIONS

High correlations between MRCGP and MRCP(UK) support the validity of each, suggesting they assess knowledge cognate to both assessments. Detailed analyses by candidate ethnicity show that although White candidates out-perform BME candidates, the differences are largely mirrored across the two examinations. Whilst the reason for the differential performance is unclear, the similarity of the effects in independent knowledge and clinical examinations suggests the differences are unlikely to result from specific features of either assessment and most likely represent true differences in ability.

Implementing statistical equating for MRCP(UK) Parts 1 and 2

BACKGROUND

The MRCP(UK) exam, in 2008 and 2010, changed the standard-setting of its Part 1 and Part 2 examinations from a hybrid Angoff/Hofstee method to statistical equating using Item Response Theory, the reference group being UK graduates. The present paper considers the implementation of the change, the question of whether the pass rate increased amongst non-UK candidates, any possible role of Differential Item Functioning (DIF), and changes in examination predictive validity after the change.

METHODS

Analysis of data of MRCP(UK) Part 1 exam from 2003 to 2013 and Part 2 exam from 2005 to 2013.

RESULTS

Inspection suggested that Part 1 pass rates were stable after the introduction of statistical equating, but showed greater annual variation probably due to stronger candidates taking the examination earlier. Pass rates seemed to have increased in non-UK graduates after equating was introduced, but was not associated with any changes in DIF after statistical equating. Statistical modelling of the pass rates for non-UK graduates found that pass rates, in both Part 1 and Part 2, were increasing year on year, with the changes probably beginning before the introduction of equating. The predictive validity of Part 1 for Part 2 was higher with statistical equating than with the previous hybrid Angoff/Hofstee method, confirming the utility of IRT-based statistical equating.

CONCLUSIONS

Statistical equating was successfully introduced into the MRCP(UK) Part 1 and Part 2 written examinations, resulting in higher predictive validity than the previous Angoff/Hofstee standard setting. Concerns about an artefactual increase in pass rates for non-UK candidates after equating were shown not to be well-founded. Most likely the changes resulted from a genuine increase in candidate ability, albeit for reasons which remain unclear, coupled with a cognitive illusion giving the impression of a step-change immediately after equating began. Statistical equating provides a robust standard-setting method, with a better theoretical foundation than judgemental techniques such as Angoff, and is more straightforward and requires far less examiner time to provide a more valid result. The present study provides a detailed case study of introducing statistical equating, and issues which may need to be considered with its introduction.

PLAB and UK graduates' performance on MRCP(UK) and MRCGP examinations: data linkage study

OBJECTIVES

To assess whether international medical graduates passing the two examinations set by the Professional and Linguistic Assessments Board (PLAB1 and PLAB2) of the General Medical Council (GMC) are equivalent to UK graduates at the end of the first foundation year of medical training (F1), as the GMC requires, and if not, to assess what changes in the PLAB pass marks might produce equivalence.

DESIGN

Data linkage of GMC PLAB performance data with data from the Royal Colleges of Physicians and the Royal College of General Practitioners on performance of PLAB graduates and UK graduates at the MRCP(UK) and MRCGP examinations.

SETTING

Doctors in training for internal medicine or general practice in the United Kingdom.

PARTICIPANTS

7829, 5135, and 4387 PLAB graduates on their first attempt at MRCP(UK) Part 1, Part 2, and PACES assessments from 2001 to 2012 compared with 18,532, 14,094, and 14,376 UK graduates taking the same assessments; 3160 PLAB1 graduates making their first attempt at the MRCGP AKT during 2007-12 compared with 14,235 UK graduates; and 1411 PLAB2 graduates making their first attempt at the MRCGP CSA during 2010-12 compared with 6935 UK graduates.

MAIN OUTCOME MEASURES

Performance at MRCP(UK) Part 1, Part 2, and PACES assessments, and MRCGP AKT and CSA assessments in relation to performance on PLAB1 and PLAB2 assessments, as well as to International English Language Testing System (IELTS) scores. MRCP(UK), MRCGP, and PLAB results were analysed as marks relative to the pass mark at the first attempt.

RESULTS

PLAB1 marks were a valid predictor of MRCP(UK) Part 1, MRCP(UK) Part 2, and MRCGP AKT (r=0.521, 0.390, and 0.490; all P<0.001). PLAB2 marks correlated with MRCP(UK) PACES and MRCGP CSA (r=0.274, 0.321; both P<0.001). PLAB graduates had significantly lower MRCP(UK) and MRCGP assessments (Glass's Δ=0.94, 0.91, 1.40, 1.01, and 1.82 for MRCP(UK) Part 1, Part 2, and PACES and MRCGP AKT and CSA), and were more likely to fail assessments and to progress more slowly than UK medical graduates. IELTS scores correlated significantly with later performance, multiple regression showing that the effect of PLAB1 (β=0.496) was much stronger than the effect of IELTS (β=0.086). Changes to PLAB pass marks that would result in international medical graduate and UK medical graduate equivalence were assessed in two ways. Method 1 adjusted PLAB pass marks to equate median performance of PLAB and UK graduates. Method 2 divided PLAB graduates into 12 equally spaced groups according to PLAB performance, and compared these with mean performance of graduates from individual UK medical schools, assessing which PLAB groups were equivalent in MRCP(UK) and MRCGP performance to UK graduates. The two methods produced similar results. To produce equivalent performance on the MRCP and MRGP examinations, the pass mark for PLAB1 would require raising by about 27 marks (13%) and for PLAB2 by about 15-16 marks (20%) above the present standard.

CONCLUSIONS

PLAB is a valid assessment of medical knowledge and clinical skills, correlating well with performance at MRCP(UK) and MRCGP. PLAB graduates' knowledge and skills at MRCP(UK) and MRCGP are over one standard deviation below those of UK graduates, although differences in training quality cannot be taken into account. Equivalent performance in MRCGP(UK) and MRCGP would occur if the pass marks of PLAB1 and PLAB2 were raised considerably, but that would also reduce the pass rate, with implications for medical workforce planning. Increasing IELTS requirements would have less impact on equivalence than raising PLAB pass marks.

The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors

BACKGROUND

Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.

METHODS

Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.

RESULTS

Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.

CONCLUSIONS

The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital' and 'medical capital'). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

BACKGROUND

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.

METHODS

Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.

RESULTS

Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.

CONCLUSIONS

Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools

BACKGROUND

Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.

METHODS

A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.

RESULTS

UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.

CONCLUSIONS

This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

BACKGROUND

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.

METHODS

Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.

RESULTS

Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.

CONCLUSIONS

Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

Genome-wide association study of handedness excludes simple genetic models

Handedness is a human behavioural phenotype that appears to be congenital, and is often assumed to be inherited, but for which the developmental origin and underlying causation(s) have been elusive. Models of the genetic basis of variation in handedness have been proposed that fit different features of the observed resemblance between relatives, but none has been decisively tested or a corresponding causative locus identified. In this study, we applied data from well-characterised individuals studied at the London Twin Research Unit. Analysis of genome-wide SNP data from 3940 twins failed to identify any locus associated with handedness at a genome-wide level of significance. The most straightforward interpretation of our analyses is that they exclude the simplest formulations of the 'right-shift' model of Annett and the 'dextral/chance' model of McManus, although more complex modifications of those models are still compatible with our observations. For polygenic effects, our study is inadequately powered to reliably detect alleles with effect sizes corresponding to an odds ratio of 1.2, but should have good power to detect effects at an odds ratio of 2 or more.

The fighting hypothesis in combat: how well does the fighting hypothesis explain human left-handed minorities?

The strong population bias in hand preference in favor of right-handedness seems to be a typical human trait. An elegant evolutionary hypothesis explaining this trait is the so-called fighting hypothesis that postulates that left-handedness is under frequency-dependent selection. The fighting hypothesis assumes that left-handers, being in the minority because of health issues, are still maintained in the population since they would have a greater chance of winning in fights than right-handers due to a surprise effect. This review critically evaluates the assumptions and evidence for this hypothesis and concludes that some evidence, although consistent with the fighting hypothesis, does not directly support it and may also be interpreted differently. Other supportive data are ambiguous or open for both statistical and theoretical criticism. We conclude that, presently, evidence for the fighting hypothesis is not particularly strong, but that there is little evidence to reject it either. The hypothesis thus remains an intuitively plausible explanation for the persistent left-hand preference in the population. We suggest alternative explanations and several ways forward for obtaining more crucial data for testing this frequently cited hypothesis.

Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP(UK) PACES and nPACES examinations

BACKGROUND

Bias of clinical examiners against some types of candidate, based on characteristics such as sex or ethnicity, would represent a threat to the validity of an examination, since sex or ethnicity are 'construct-irrelevant' characteristics. In this paper we report a novel method for assessing sex and ethnic bias in over 2000 examiners who had taken part in the PACES and nPACES (new PACES) examinations of the MRCP(UK).

METHOD

PACES and nPACES are clinical skills examinations that have two examiners at each station who mark candidates independently. Differences between examiners cannot be due to differences in performance of a candidate because that is the same for the two examiners, and hence may result from bias or unreliability on the part of the examiners. By comparing each examiner against a 'basket' of all of their co-examiners, it is possible to identify examiners whose behaviour is anomalous. The method assessed hawkishness-doveishness, sex bias, ethnic bias and, as a control condition to assess the statistical method, 'even-number bias' (i.e. treating candidates with odd and even exam numbers differently). Significance levels were Bonferroni corrected because of the large number of examiners being considered.

RESULTS

The results of 26 diets of PACES and six diets of nPACES were examined statistically to assess the extent of hawkishness, as well as sex bias and ethnicity bias in individual examiners. The control (odd-number) condition suggested that about 5% of examiners were significant at an (uncorrected) 5% level, and that the method therefore worked as expected. As in a previous study (BMC Medical Education, 2006, 6:42), some examiners were hawkish or doveish relative to their peers. No examiners showed significant sex bias, and only a single examiner showed evidence consistent with ethnic bias. A re-analysis of the data considering only one examiner per station, as would be the case for many clinical examinations, showed that analysis with a single examiner runs a serious risk of false positive identifications probably due to differences in case-mix and content-specificity.

CONCLUSIONS

In examinations where there are two independent examiners at a station, our method can assess the extent of bias against candidates with particular characteristics. The method would be far less sensitive in examinations with only a single examiner per station as examiner variance would be confounded with candidate performance variance. The method however works well when there is more than one examiner at a station and in the case of the current MRCP(UK) clinical examination, nPACES, found possible sex bias in no examiners and possible ethnic bias in only one.

Multilocus genetic models of handedness closely resemble single-locus models in explaining family data and are compatible with genome-wide association studies

Right- and left-handedness run in families, show greater concordance in monozygotic than dizygotic twins, and are well described by single-locus Mendelian models. Here we summarize a large genome-wide association study (GWAS) that finds no significant associations with handedness and is consistent with a meta-analysis of GWASs. The GWAS had 99% power to detect a single locus using the conventional criterion of P < 5 × 10(-8) for the single locus models of McManus and Annett. The strong conclusion is that handedness is not controlled by a single genetic locus. A consideration of the genetic architecture of height, primary ciliary dyskinesia, and intelligence suggests that handedness inheritance can be explained by a multilocus variant of the McManus DC model, classical effects on family and twins being barely distinguishable from the single locus model. Based on the ENGAGE meta-analysis of GWASs, we estimate at least 40 loci are involved in determining handedness.

Local processing enhancements associated with superior observational drawing are due to enhanced perceptual functioning, not weak central coherence

Individuals with drawing talent have previously been shown to exhibit enhanced local visual processing ability. The aim of the current study was to assess whether local processing biases associated with drawing ability result from a reduced ability to cohere local stimuli into global forms, or an increased ability to disregard global aspects of an image. Local and global visual processing ability was assessed in art students and controls using the Group Embedded Figures Task, Navon shape stimuli, the Block Design Task and the Autism Spectrum Quotient, whilst controlling for nonverbal IQ and artistic ability. Local processing biases associated with drawing appear to arise from an enhancement of local processing alongside successful filtering of global information, rather than a reduction in global processing. The relationship between local processing and drawing ability is independent of individual differences in nonverbal IQ and artistic ability. These findings have implications for bottom-up and attentional theories of observational drawing, as well as explanations of special skills in autism.

Resitting a high-stakes postgraduate medical examination on multiple occasions: nonlinear multilevel modelling of performance in the MRCP(UK) examinations

BACKGROUND

Failure rates in postgraduate examinations are often high and many candidates therefore retake examinations on several or even many times. Little, however, is known about how candidates perform across those multiple attempts. A key theoretical question to be resolved is whether candidates pass at a resit because they have got better, having acquired more knowledge or skills, or whether they have got lucky, chance helping them to get over the pass mark. In the UK, the issue of resits has become of particular interest since the General Medical Council issued a consultation and is considering limiting the number of attempts candidates may make at examinations.

METHODS

Since 1999 the examination for Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) has imposed no limit on the number of attempts candidates can make at its Part 1, Part 2 or PACES (Clinical) examination. The present study examined the performance of candidates on the examinations from 2002/2003 to 2010, during which time the examination structure has been stable. Data were available for 70,856 attempts at Part 1 by 39,335 candidates, 37,654 attempts at Part 2 by 23,637 candidates and 40,303 attempts at PACES by 21,270 candidates, with the maximum number of attempts being 26, 21 and 14, respectively. The results were analyzed using multilevel modelling, fitting negative exponential growth curves to individual candidate performance.

RESULTS

The number of candidates taking the assessment falls exponentially at each attempt. Performance improves across attempts, with evidence in the Part 1 examination that candidates are still improving up to the tenth attempt, with a similar improvement up to the fourth attempt in Part 2 and the sixth attempt at PACES. Random effects modelling shows that candidates begin at a starting level, with performance increasing by a smaller amount at each attempt, with evidence of a maximum, asymptotic level for candidates, and candidates showing variation in starting level, rate of improvement and maximum level. Modelling longitudinal performance across the three diets (sittings) shows that the starting level at Part 1 predicts starting level at both Part 2 and PACES, and the rate of improvement at Part 1 also predicts the starting level at Part 2 and PACES.

CONCLUSION

Candidates continue to show evidence of true improvement in performance up to at least the tenth attempt at MRCP(UK) Part 1, although there are individual differences in the starting level, the rate of improvement and the maximum level that can be achieved. Such findings provide little support for arguments that candidates should only be allowed a fixed number of attempts at an examination. However, unlimited numbers of attempts are also difficult to justify because of the inevitable and ever increasing role that luck must play with increasing numbers of resits, so that the issue of multiple attempts might be better addressed by tackling the difficult question of how a pass mark should increase with each attempt at an exam.

The misinterpretation of the standard error of measurement in medical education: a primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement

BACKGROUND

In high-stakes assessments in medical education, such as final undergraduate examinations and postgraduate assessments, an attempt is frequently made to set confidence limits on the probable true score of a candidate. Typically, this is carried out using what is referred to as the standard error of measurement (SEM). However, it is often the case that the wrong formula is applied, there actually being three different formulae for use in different situations.

AIMS

To explain and clarify the calculation of the SEM, and differentiate three separate standard errors, which here are called the standard error of measurement (SEmeas), the standard error of estimation (SEest) and the standard error of prediction (SEpred).

RESULTS

Most accounts describe the calculation of SEmeas. For most purposes, though, what is required is the standard error of estimation (SEest), which has to be applied not to a candidate's actual score but to their estimated true score after taking into account the regression to the mean that occurs due to the unreliability of an assessment. A third formula, the standard error of prediction (SEpred) is less commonly used in medical education, but is useful in situations such as counselling, where one needs to predict a future actual score on an examination from a previous actual score on the same examination.

CONCLUSIONS

The various formulae can produce predictions that differ quite substantially, particularly when reliability is not particularly high, and the mark in question is far removed from the average performance of candidates. That can have important, unintended consequences, particularly in a medico-legal context.

Perceptions of Humility: A Preliminary Study

Coren and Porac (1976) reported that objects looked larger in the right eye of right-eye dominant subjects and in the left eye of left-eye dominant subjects. This paper attempts to repeat that finding. Two circles of same or different size were presented haploscopically in a binocular three-field tachistoscope, to right or left visual half-field and to the upper or lower visual field, one to the right eye and one to the left. A total of 43 subjects reported which of the two circles was the larger, each subject carrying out 120 trials of the experiment. Overall subjects reported that the stimulus to the left eye was significantly larger than that presented to the right eye. There was no association with eye dominance, and therefore the Coren and Porac finding could not be repeated. There was however a very significant association with handedness, left-handed subjects tending to report that the stimulus in the right eye looked larger, and right-handed subjects reporting that the stimulus in the left eye looked larger.

Arnheim's Gestalt theory of visual balance: Examining the compositional structure of art photographs and abstract images

In Art and Visual Perception, Rudolf Arnheim, following on from Denman Ross's A Theory of Pure Design, proposed a Gestalt theory of visual composition. The current paper assesses a physicalist interpretation of Arnheim's theory, calculating an image's centre of mass (CoM). Three types of data are used: a large, representative collection of art photographs of recognised quality; croppings by experts and non-experts of photographs; and Ross and Arnheim's procedure of placing a frame around objects such as Arnheim's two black disks. Compared with control images, the CoM of art photographs was closer to an axis (horizontal, vertical, or diagonal), as was the case for photographic croppings. However, stronger, within-image, paired comparison studies, comparing art photographs with the CoM moved on or off an axis (the 'gamma-ramp study'), or comparing adjacent croppings on or off an axis (the 'spider-web study'), showed no support for the Arnheim-Ross theory. Finally, studies moving a frame around two disks, of different size, greyness, or background, did not support Arnheim's Gestalt theory. Although the detailed results did not support the Arnheim-Ross theory, several significant results were found which clearly require explanation by any adequate theory of the aesthetics of visual composition.

Vocation and avocation: leisure activities correlate with professional engagement, but not burnout, in a cross-sectional survey of UK doctors

BACKGROUND

Sir William Osler suggested in 1899 that avocations (leisure activities) in doctors are related to an increased sense of vocation (professional engagement) and a decreased level of burnout. This study evaluated those claims in a large group of doctors practicing in the UK while taking into account a wide range of background variables.

METHODS

A follow-up questionnaire was sent to 4,457 UK-qualified doctors who had been included in four previous studies of medical school selection and training, beginning in 1980, 1985, 1990 and 1989/1991. A total of 2,845 (63.8%) doctors returned the questionnaire. Questions particularly asked about work engagement, satisfaction with medicine as a career, and personal achievement (Vocation/engagement), stress, emotional exhaustion, and depersonalization (BurnedOut), and 29 different leisure activities (Avocation/Leisure), as well as questions on personality, empathy, work experience, and demography.

RESULTS

Doctors reporting more Avocation/Leisure activities tended to be women, to have older children, to be less surface-rational, more extravert, more open to experience, less agreeable, and to fantasize more. Doctors who were more BurnedOut tended to be men, to be more sleep-deprived, to report a greater workload and less choice and independence in their work, to have higher neuroticism, lower extraversion and lower agreeableness scores, and to have lower self-esteem. In contrast, doctors with a greater sense of Vocation/engagement, tended to see more patients, to have greater choice and independence at work, to have a deep approach to work, to have a more supportive-receptive work environment, to be more extravert and more conscientious, and to report greater self-esteem.Avocation/Leisure activities correlated significantly with Vocation/engagement, even after taking into account 25 background variables describing demography, work, and personality, whereas BurnedOut showed no significant correlation with Avocation/Leisure activities. Popular Culture and High Culture did not differ in their influence on Vocation/engagement, although there was a suggestion that Depersonalization was correlated with more interest in Popular Culture and less interest in High Culture.

CONCLUSION

In this cross-sectional study there is evidence, even after taking into account a wide range of individual difference measures, that doctors with greater Avocation/Leisure activities also have a greater sense of Vocation/Engagement. In contrast, being BurnedOut did not relate to Avocation/Leisure activities (but did relate to many other measures). Osler was probably correct in recommending to doctors that, 'While medicine is to be your vocation, or calling, see to it that you also have an avocation'.

Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis

OBJECTIVE

To determine whether the ethnicity of UK trained doctors and medical students is related to their academic performance.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Online databases PubMed, Scopus, and ERIC; Google and Google Scholar; personal knowledge; backwards and forwards citations; specific searches of medical education journals and medical education conference abstracts.

STUDY SELECTION

The included quantitative reports measured the performance of medical students or UK trained doctors from different ethnic groups in undergraduate or postgraduate assessments. Exclusions were non-UK assessments, only non-UK trained candidates, only self reported assessment data, only dropouts or another non-academic variable, obvious sampling bias, or insufficient details of ethnicity or outcomes. Results 23 reports comparing the academic performance of medical students and doctors from different ethnic groups were included. Meta-analyses of effects from 22 reports (n = 23,742) indicated candidates of "non-white" ethnicity underperformed compared with white candidates (Cohen's d = -0.42, 95% confidence interval -0.50 to -0.34; P<0.001). Effects in the same direction and of similar magnitude were found in meta-analyses of undergraduate assessments only, postgraduate assessments only, machine marked written assessments only, practical clinical assessments only, assessments with pass/fail outcomes only, assessments with continuous outcomes only, and in a meta-analysis of white v Asian candidates only. Heterogeneity was present in all meta-analyses.

CONCLUSION

Ethnic differences in academic performance are widespread across different medical schools, different types of exam, and in undergraduates and postgraduates. They have persisted for many years and cannot be dismissed as atypical or local problems. We need to recognise this as an issue that probably affects all of UK medical and higher education. More detailed information to track the problem as well as further research into its causes is required. Such actions are necessary to ensure a fair and just method of training and of assessing current and future doctors.

Predictive validity of the Biomedical Admissions Test: an evaluation and case study

There has been an increase in the use of pre-admission selection tests for medicine. Such tests need to show good psychometric properties. Here, we use a paper by Emery and Bell [2009. The predictive validity of the Biomedical Admissions Test for pre-clinical examination performance. Med Educ 43:557-564] as a case study to evaluate and comment on the reporting of psychometric data in the field of medical student selection (and the comments apply to many papers in the field). We highlight pitfalls when reliability data are not presented, how simple zero-order associations can lead to inaccurate conclusions about the predictive validity of a test, and how biases need to be explored and reported. We show with BMAT that it is the knowledge part of the test which does all the predictive work. We show that without evidence of incremental validity it is difficult to assess the value of any selection tests for medicine.

Aesthetic activities and aesthetic attitudes: Influences of education, background and personality on interest and involvement in the arts

There have been few studies of why some people are frequently involved in aesthetic activities such as going to the theatre, reading or playing musical instruments, whereas others are less involved. This study assesses the broad roles of education, personality and demographic factors such as social class, age and sex. More aesthetic activity was associated with music and art education, whereas science education had a substantial negative relationship with aesthetic activity, both directly and also indirectly via reduced art education. More aesthetic activity was particularly related to higher scores on the personality factor of openness, and also to lower scores on agreeableness and conscientiousness. Higher parental social class was also associated with more aesthetic activity, as also was lower age. Sex had no relationship to aesthetic activity, as neither did masculinity-femininity. Positive aesthetic attitudes were also related moderately to aesthetic activity, but were particularly strongly related to openness to experience, and somewhat less to extraversion. Class, age and sex had no direct relationship to aesthetic attitudes.