1
|
Toale C, Morris M, Gross S, O'Keeffe DA, Ryan DM, Boland F, Doherty EM, Traynor OJ, Kavanagh DO. Performance in Irish Selection and Future Performance in Surgical Training. JAMA Surg 2024; 159:538-545. [PMID: 38446454 PMCID: PMC10918576 DOI: 10.1001/jamasurg.2024.0034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 12/04/2023] [Indexed: 03/07/2024]
Abstract
Importance Selection processes for surgical training should aim to identify candidates who will become competent independent practitioners and should aspire to high standards of reliability and validity. Objective To determine the association between measured candidate factors at the time of an Irish selection and assessment outcomes in surgical training, examined via rate of progression to Higher Specialist Training (HST), attrition rates, and performance as assessed through a multimodal framework of workplace-based and simulation-based assessments. Design, Setting, and Participants This retrospective observational cohort study included data from all successful applicants to the Royal College of Surgeons in Ireland (RCSI) national Core Surgical Training (CST) program. Participants included all trainees recruited to dedicated postgraduate surgical training from 2016 to 2020. These data were analyzed from July 11, 2016, through July 10, 2022. Exposures Selection decisions were based on a composite score that was derived from technical aptitude assessments, undergraduate academic performance, and a 4-station multiple mini-interview. Main outcomes and measures Assessment data, attrition rates, and rates of progression to HST were recorded for each trainee. CST performance was assessed using workplace-based and simulation-based technical and nontechnical skill assessments. Potential associations between selection and assessment measures were explored using Pearson correlation, logistic regression, and multiple linear-regression analyses. Results Data were available for 303 trainees. Composite scores were positively associated with progression to HST (odds ratio [OR], 1.09; 95% CI, 1.05-1.13). There was a weak positive correlation, ranging from 0.23 to 0.34, between scores and performance across all CST assessments. Multivariable linear regression analysis showed technical aptitude scores at application were associated with future operative performance assessment scores, both in the workplace (β = 0.31; 95% CI, 0.14-0.48) and simulated environments (β = 0.57; 95% CI, 0.33-0.81). There was evidence that the interpersonal skills interview station was associated with future performance in simulated communication skill assessments (β = 0.55; 95% CI, 0.22-0.87). Conclusions and Relevance In this study, performance at the time of Irish national selection, measured across technical and nontechnical domains in a multimodal fashion, was associated with future performance in the workplace and in simulated environments. Future studies will be required to explore the consequential validity of selection, including potential unintended effects of selection and ranking on candidate performance.
Collapse
Affiliation(s)
- Conor Toale
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Marie Morris
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Sara Gross
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Dara A O'Keeffe
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Donncha M Ryan
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Fiona Boland
- Data Science Centre, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Eva M Doherty
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Oscar J Traynor
- Department of Surgical Affairs, RCSI University of Medicine and Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland
| | | |
Collapse
|
2
|
Abstract
In this critical narrative review, we challenge the belief that single-moment-in-time high-stakes examinations (SMITHSEx) are an essential component of contemporary specialist training. We explore the arguments both for and against SMITHSEx, examine potential alternatives, and discuss the barriers to change.SMITHSEx are viewed as the "gold standard" assessment of competence but focus excessively on knowledge assessment rather than capturing essential competencies required for safe and competent workplace performance. Contrary to popular belief, regulatory bodies do not mandate SMITHSEx in specialist training. Though acting as significant drivers of learning and professional identity formation, these attributes are not exclusive to SMITHSEx.Skills such as crisis management, procedural skills, professionalism, communication, collaboration, lifelong learning, reflection on practice, and judgement are often overlooked by SMITHSEx. Their inherent design raises questions about the validity and objectivity of SMITHSEx as a measure of workplace competence. They have a detrimental impact on trainee well-being, contributing to burnout and differential attainment.Alternatives to SMITHSEx include continuous low-stakes assessments throughout training, ongoing evaluation of competence in the workplace, and competency-based medical education (CBME) concepts. These aim to provide a more comprehensive and context-specific assessment of trainees' competence while also improving trainee welfare.Specialist training colleges should evolve from exam providers to holistic education sources. Assessments should emphasise essential practical knowledge over trivia, align with clinical practice, aid learning, and be part of a diverse toolkit. Eliminating SMITHSEx from specialist training will foster a competency-based approach, benefiting future medical professionals' well-being and success.
Collapse
Affiliation(s)
- Navdeep S Sidhu
- Department of Anaesthesiology, School of Medicine, University of Auckland, Auckland, New Zealand
- Department of Anaesthesia and Perioperative Medicine, North Shore Hospital, Auckland, New Zealand
| | - Simon Fleming
- Department of Hand Surgery, Royal North Shore Hospital, Sydney, New South Wales, Australia
| |
Collapse
|
3
|
Ellis R, Cleland J, Scrimgeour DS, Lee AJ, Hines J, Brennan PA. Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS Part A. Surgeon 2023; 21:323-330. [PMID: 37544852 DOI: 10.1016/j.surge.2023.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/08/2023]
Abstract
Successful completion of the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination is mandatory for surgical trainees entering higher specialist training in the United Kingdom. Despite its international reputation, and the value placed on the examination in surgical training, there has been little evidence of its predictive validity until recently. In this review, we present a summary of findings of four recent Intercollegiate studies assessing the predictive validity of the MRCS Part A (written) examination. Data from all four studies showed statistically significant positive correlations between the MRCS Part A and other written examinations taken by surgical trainees over the course of their education. The studies summarised in this review provide compelling evidence for the predictive validity of this gatekeeping examination. This review will be of interest to trainees, training institutions and the Royal Colleges given the value placed on the examination by surgical training programmes.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB25 2ZD, United Kingdom; Urology Department, Nottingham University Hospitals, Nottingham, United Kingdom.
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB25 2ZD, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, United Kingdom.
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom.
| | - John Hines
- Urology Department, University College Hospital, London, W1G 8PH, United Kingdom.
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, United Kingdom.
| |
Collapse
|
4
|
Ellis R, Cleland J, Scrimgeour DS, Lee AJ, Hines J, Brennan PA. Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS part B. Surgeon 2023; 21:278-284. [PMID: 37517979 DOI: 10.1016/j.surge.2023.07.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/01/2023]
Abstract
The Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) is a high-stakes postgraduate examination taken by thousands of surgical trainees worldwide every year. The MRCS is a challenging assessment, highly regarded by surgical training programmes and valued as a gatekeeper to the surgical profession. The examination is taken at considerable personal, social and financial cost to surgical trainees, and failure has significant implications for career progression. Given the value placed on MRCS, it must be a reliable and valid assessment of the knowledge and skills of early-career surgeons. Our first article 'Establishing the Predictive Validity of the Intercollegiate Membership of the Royal Colleges of Surgeons Written Examination: MRCS Part A' discussed the principles of assessment reliability and validity and outlined the mounting evidence supporting the predictive validity of the MRCS Part A (the multiple-choice questionnaire component of the examination). This, the second article in the series discusses six recently published studies investigating the predictive validity of the MRCS Part B (the clinical component of the examination). All national longitudinal cohort studies reviewed have demonstrated significant correlations between MRCS Part B and other assessments taken during the UK surgical training pathway, supporting the predictive validity of MRCS Part B. This review will be of interest to trainees, trainers and Royal Colleges given the value placed on the examination by surgical training programmes.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; Urology Department, Nottingham University Hospitals, Nottingham, United Kingdom.
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, United Kingdom.
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, United Kingdom.
| | - John Hines
- University College Hospital London, United Kingdom.
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, United Kingdom.
| |
Collapse
|
5
|
Abstract
BACKGROUND In the UK, core surgical training (CST) is the first specialty experience that early-career surgeons receive but training differs significantly across CST deaneries. To identify the impact these differences have on trainee performance, we assessed whether success at the Membership of the Royal College of Surgeons (MRCS) examinations is associated with CST deanery. METHODS A retrospective cohort study of UK trainees in CST who attempted MRCS between 2014 and 2020 (n = 1104). Chi-squared tests examined associations between locality and first-attempt MRCS performance. Multivariate logistic regression models identified the likelihood of MRCS success depending on CST deanery. RESULTS MRCS Part A and Part B pass rates were associated with CST deanery (p < 0.001 and p = 0.013, respectively). Candidates that trained in Thames Valley (Odds Ratio [OR] 2.52 (95% Confidence Interval [CI] 1.00-6.42), North Central and East London (OR 2.37 [95% CI 1.04-5.40]) or South London (OR 2.36 [95% CI 1.09-5.10]) were each more than twice as likely to pass MRCS Part A at first attempt. Trainees from North Central and East London were more than ten times more likely to pass MRCS Part B at first attempt (OR 10.59 [95% CI 1.23-51.00]). However, 68% of candidates attempted Part A prior to CST and 48% attempted Part B before or during the first year of CST. CONCLUSION MRCS performance is associated with CST deanery; however, many candidates passed the exam with little or any CST experience suggesting that some deaneries attract high academic performers. MRCS performance is therefore not a suitable marker of CST training quality.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | | | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| |
Collapse
|
6
|
Ellis R, Brennan PA, Lee AJ, Scrimgeour DS, Cleland J. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study. J R Soc Med 2022; 115:257-272. [PMID: 35171739 PMCID: PMC9234885 DOI: 10.1177/01410768221079018] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Objective A recent independent review on diversity and inclusivity highlighted concerns
that barriers to surgical career progression exist for some groups of
individuals and not others. Group-level differences in performance at the
Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS)
examinations have been identified but are yet to be investigated. We aimed
to characterise the relationship between sociodemographic differences and
performance at MRCS. Design Retrospective cohort study. Setting Secondary care. Participants All UK MRCS candidates attempting Part A (n = 5780) and Part
B (n = 2600) between 2013 and 2019 with linked
sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk). Main outcome measures Chi-square tests established univariate associations with MRCS performance.
Multiple logistic regression identified independent predictors of success,
adjusted for medical school performance. Results Statistically significant differences in MRCS pass rates were found according
to gender, ethnicity, age, graduate status, educational background and
socioeconomic status (all p < 0.05). After adjusting for
prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence
interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were
independent predictors of MRCS Part A success and being a non-graduate (OR
1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51,
95% CI 1.08–2.10) were independent predictors of Part B success. Black and
minority ethnic groups were significantly less likely to pass MRCS Part B at
their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR
0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White
candidates. Conclusions There is significant group-level differential attainment at MRCS, likely to
represent the accumulation of privilege and disadvantage experienced by
individuals throughout their education and training. Those leading surgical
education now have a responsibility to identify and address the causes of
these attainment differences.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Urology Department, Nottingham University Hospitals, Nottingham, NG5 1PB, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, UK
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB24 3FX, UK
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, UK
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, 308232, Singapore
| |
Collapse
|
7
|
Ellis R, Brennan PA, Scrimgeour DSG, Lee AJ, Cleland J. Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study. BMJ Open 2022; 12:e054616. [PMID: 34987044 PMCID: PMC8734024 DOI: 10.1136/bmjopen-2021-054616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/19/2021] [Accepted: 12/02/2021] [Indexed: 12/03/2022] Open
Abstract
OBJECTIVES The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values. DESIGN AND PARTICIPANTS A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database. METHODS We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt. RESULTS MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05). CONCLUSIONS There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.
Collapse
Affiliation(s)
- Ricky Ellis
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| | - Duncan S G Scrimgeour
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, UK
| | - Amanda J Lee
- Medical Statistics Team, University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
8
|
Ellis R, Scrimgeour DSG, Brennan PA, Lee AJ, Cleland J. Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study. BMJ Open 2021; 11:e046615. [PMID: 34400449 PMCID: PMC8370550 DOI: 10.1136/bmjopen-2020-046615] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
BACKGROUND Identifying predictors of success in postgraduate examinations can help guide the career choices of medical students and may aid early identification of trainees requiring extra support to progress in specialty training. We assessed whether performance on the educational performance measurement (EPM) and situational judgement test (SJT) used for selection into foundation training predicted success at the Membership of the Royal College of Surgeons (MRCS) examination. METHODS This was a longitudinal, cohort study using data from the UK Medical Education Database (https://www.ukmed.ac.uk). UK medical graduates who had attempted Part A (n=2585) and Part B (n=755) of the MRCS between 2014 and 2017 were included. χ2 and independent t-tests were used to examine the relationship between medical school performance and sociodemographic factors with first-attempt success at MRCS Part A and B. Multivariate logistic regression was employed to identify independent predictors of MRCS performance. RESULTS The odds of passing MRCS increased by 55% for Part A (OR 1.55 (95% CI 1.48 to 1.61)) and 23% for Part B (1.23 (1.14 to 1.32)) for every additional EPM decile point gained. For every point awarded for additional degrees in the EPM, candidates were 20% more likely to pass MRCS Part A (1.20 (1.13 to 1.29)) and 17% more likely to pass Part B (1.17 (1.04 to 1.33)). For every point awarded for publications in the EPM, candidates were 14% more likely to pass MRCS Part A (1.14 (1.01 to 1.28)). SJT score was not a statistically significant independent predictor of MRCS success. CONCLUSION This study has demonstrated the EPM's independent predictive power and found that medical school performance deciles are the most significant measure of predicting later success in the MRCS. These findings can be used by medical schools, training boards and workforce planners to inform evidence-based and contemporary selection and assessment strategies.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Department of Urology, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Duncan S G Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| | - Amanda J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
9
|
Ellis R, Cleland J, Scrimgeour DSG, Lee AJ, Brennan PA. A cross-sectional study examining the association between MRCS performance and surgeons receiving sanctions against their medical registration. Surgeon 2021; 20:211-215. [PMID: 34030984 DOI: 10.1016/j.surge.2021.04.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 03/20/2021] [Accepted: 04/06/2021] [Indexed: 11/18/2022]
Abstract
BACKGROUND Fitness to practice (FtP) investigations by the General Medical Council (GMC) safeguard patients and maintain the integrity of the medical profession. The likelihood of FtP sanctions is influenced by specialty and socio-demographic factors and can be predicted by performance at postgraduate examinations. This is the first study to characterise the prevalence of FtP sanctions in early-career surgeons and to examine the association with performance at the Membership of the Royal College of Surgeons (MRCS) examination. METHODS All UK graduates who attempted MRCS between September 2007-January 2020 were matched to the GMC list of registered medical practitioners. Clinicians who had active FtP sanctions between 28th August 2018 and 28th August 2020 were identified. Data were anonymised by RCS England prior to analysis. RESULTS Of 11,660 candidates who attempted MRCS within the study period, only 31 (0.3%) had FtP sanctions between 2018 and 2020. Of these, 12 had active conditions on registration, seven had undertakings and 14 had warnings. There was no statistically significant difference in MRCS performance in either Parts A or B of the examination for those with and those free from FtP sanctions (P > 0.05). CONCLUSIONS In this, the largest study of MRCS candidates to date, the prevalence of active FtP sanctions in early-career surgeons was 0.3%, significantly lower than the prevalence of sanctions across more experienced UK surgeons (0.9%). These data highlight early-career surgeons as a low-risk group for disciplinary action and should reassure patients and medical professionals of the rarity of FtP sanctions.
Collapse
Affiliation(s)
- R Ellis
- Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom; Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom.
| | - J Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - D S G Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, United Kingdom
| | - A J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom
| | - P A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, United Kingdom
| |
Collapse
|
10
|
Hardie JA, Green G, Bor R, Brennan PA. Cutting edge selection: learning from high reliability organisations for virtual recruitment in surgery during the COVID-19 pandemic. Ann R Coll Surg Engl 2021; 103:385-389. [PMID: 33955275 DOI: 10.1308/rcsann.2021.0034] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
INTRODUCTION National selection for higher surgical training (ST3+) recruitment in the UK is competitive. The process must prioritise patient safety while being credible, impartial and fair. During the COVID-19 pandemic, all face-to-face interviews were cancelled. Selection was based on a controversial isolated self-assessment score with no evidence checking taking place. From 2021, selection will take place entirely online. Although this has cost and time advantages, new challenges emerge. METHODS We review surgical selection as it transitions to an online format and suggest validated methods that could be adapted from High Reliability Organisations (HRO). FINDINGS Virtual selection methods include video interviewing, online examinations and aptitude testing. These tools have been used in business for many years, but their predictive value in surgery is largely unknown. In healthcare, the established online Multi-Specialty Recruitment Assessment (MSRA) examines generic professional capabilities. Its scope, however, is too limited to be used in isolation. Candidates and interviewers alike may have concerns about the technical aspects of virtual recruitment. The significance of human factors must not be overlooked in the online environment. Surgery can learn from HROs, such as aviation. Pilot and air traffic control selection is integral to ensuring safety. These organisations have already established digital selection methods for psychological aptitude, professional capabilities and manual dexterity. CONCLUSION National selection for higher surgical training (ST3+) can learn from HROs, using validated methods to prioritise patient safety while being acceptable to candidates, trainers and health service recruiters.
Collapse
Affiliation(s)
| | - G Green
- Centre for Aviation Psychology, UK
| | - R Bor
- Centre for Aviation Psychology, UK.,Royal Free Hospital NHS Foundation Trust, UK
| | - P A Brennan
- Portsmouth Hospitals University NHS Trust, UK
| |
Collapse
|
11
|
Ellis R, Brennan P, Scrimgeour DS, Lee AJ, Cleland J. Performance at medical school selection correlates with success in Part A of the intercollegiate Membership of the Royal College of Surgeons (MRCS) examination. Postgrad Med J 2021; 98:e19. [PMID: 33692157 DOI: 10.1136/postgradmedj-2021-139748] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 02/15/2021] [Accepted: 02/20/2021] [Indexed: 11/04/2022]
Abstract
Medical schools in the UK typically use prior academic attainment and an admissions test (University Clinical Aptitude Test (UCAT), Biomedical Admissions Test (BMAT) or the Graduate Medical School Admissions Test (GAMSAT)) to help select applicants for interview. To justify their use, more information is needed about the predictive validity of these tests. Thus, we investigated the relationship between performance in admissions tests and the Membership of the Royal College of Surgeons (MRCS) examination.The UKMED database (https://www.ukmed.ac.uk) was used to access medical school selection data for all UK graduates who attempted MRCS Part A (n=11 570) and Part B (n=5690) between 2007 and 2019. Univariate and multivariate logistic regression models identified independent predictors of MRCS success. Pearson correlation coefficients examined the linear relationship between test scores and MRCS performance.Successful MRCS Part A candidates scored higher in A-Levels, UCAT, BMAT and GAMSAT (p<0.05). No significant differences were observed for MRCS Part B. All admissions tests were found to independently predict MRCS Part A performance after adjusting for prior academic attainment (A-Level performance) (p<0.05). Admission test scores demonstrated statistically significant correlations with MRCS Part A performance (p<0.001).The utility of admissions tests is clear with respect to helping medical schools select from large numbers of applicants for a limited number of places. Additionally, these tests appear to offer incremental value above A-Level performance alone. We expect this data to guide medical schools' use of admissions test scores in their selection process.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK .,Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Peter Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, Portsmouth, UK
| | - Duncan Sg Scrimgeour
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, Aberdeen, UK
| | - Amanda J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research & Scholarship Unit, Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
12
|
Fonseka T, Ellis R, Salem H, Brennan PA, Terry T. The effects of COVID-19 on training within urology: Lessons learned in virtual learning, human factors, non-technical skills and reflective practice. J Clin Urol 2021; 14:29-35. [PMID: 38603044 PMCID: PMC7435206 DOI: 10.1177/2051415820950109] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Accepted: 07/21/2020] [Indexed: 11/16/2022]
Abstract
The COVID-19 pandemic has changed training and recruitment in urology in unprecedented ways. As efforts are made to ensure trainees can continue to progress, lessons can be learned to improve training and urological practice even after the acute phase of the pandemic is over. Novel methods of education through virtual learning have burgeoned amidst the social distancing the pandemic has brought. The importance of training in human factors and non-technical skills has also been brought to the fore while operating under the constraints of personal protective equipment and working in new teams and unfamiliar environments. This paper critically appraises the available evidence of how urological training has been affected by COVID-19 and the lessons we have learned and continue to learn going forward. Level of Evidence Not Applicable.
Collapse
Affiliation(s)
- T Fonseka
- Urology Department, Royal Derby
Hospital, University Hospitals of Derby and Burton, UK
| | - R Ellis
- Urology Department, Royal Derby
Hospital, University Hospitals of Derby and Burton, UK
| | - H Salem
- Urology Department, Royal Derby
Hospital, University Hospitals of Derby and Burton, UK
| | - PA Brennan
- Maxillofacial Unit, Queen Alexandra
Hospital, UK
| | - T Terry
- Urology Department, Nottingham
University Hospitals, UK
| |
Collapse
|
13
|
Brennan P, Scrimgeour D, Shakib K. Editorial: exploring evidence-based options for reducing the duration of UK OMFS training. Br J Oral Maxillofac Surg 2020; 58:1216-1218. [DOI: 10.1016/j.bjoms.2020.01.013] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2019] [Accepted: 01/02/2020] [Indexed: 11/16/2022]
|
14
|
Singh RP. Should we recruit trainees into the Oral and Maxillofacial Surgery pathway at the beginning of the second degree to ensure the long term viability of the specialty? Br J Oral Maxillofac Surg 2020; 58:1353-1354. [PMID: 33208287 DOI: 10.1016/j.bjoms.2020.10.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 10/07/2020] [Indexed: 12/13/2022]
Abstract
There has been a sustained decline in the number of trainees applying for entry into the Oral and Maxillofacial Surgery (OMFS) training program, which has sparked further debate on the issue of OMFS training and the future of the specialty in the United Kingdom. Here I discuss the option of recruiting the trainees into the program at the beginning of their second degree to streamline the training, and to improve the recruitment drive to ensure long term viability of the specialty.
Collapse
Affiliation(s)
- Rabindra P Singh
- Department of Oral and Maxillofacial Surgery, University Hospital Southampton, Tremona Road, Southampton, SO16 6YD, United Kingdom.
| |
Collapse
|
15
|
Scrimgeour DSG, Cleland J, Lee AJ, Brennan PA. Prediction of success at UK Specialty Board Examinations using the mandatory postgraduate UK surgical examination. BJS Open 2019; 3:865-871. [PMID: 31832594 PMCID: PMC6887704 DOI: 10.1002/bjs5.50212] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2018] [Accepted: 06/26/2019] [Indexed: 11/09/2022] Open
Abstract
Background While performance in other mandatory examinations taken at the beginning of a doctor's career are predictive of final training outcomes, the influence early postgraduate surgical examinations might have on success at Specialty Board Exams in the UK is currently unknown. The aim was to investigate whether performance at the mandatory Membership of the Royal College of Surgeons (MRCS) examination, and other variables, are predictive of success at the Fellowship of the Royal College of Surgeons (FRCS) examination, thus potentially identifying those who may benefit from early academic intervention. Methods Pearson correlation coefficients examined the linear relationship between both examinations and logistic regression analysis identified potential independent predictors of FRCS success. All UK medical graduates who attempted either section of FRCS (Sections 1 and 2) between 2012 and 2018 were included. Results First attempt pass rates for Sections 1 and 2 FRCS were 87.4 per cent (n = 854) and 91.8 per cent (n = 797) respectively. In logistic regression analysis, sex (male: odds ratio (OR) 2.32, 95 per cent c.i 1.43 to 3.76), age (less than 29 years at graduation: OR 3.22, 1.88 to 5.51), Part B MRCS attempts (1 attempt: OR 1.77, 1.08 to 3.00), Part A score (OR 1.14, 1.09 to 1.89) and Part B score (OR 1.06, 1.03 to 1.09) were independent predictors of Section 1 FRCS success. Predictors of Section 2 FRCS success were age (less than 29 years at graduation: OR 3.55, 2.00 to 6.39), Part A score (OR 1.06, 1.02 to 1.11) and Section 1 FRCS score (OR 1.13, 1.07 to 1.18). Conclusion Part A and B MRCS performance were independent predictors of FRCS success, providing further evidence to support the predictive validity of this mandatory postgraduate exam. However, future research must explore the reasons between the attainment gaps observed for different groups of doctors.
Collapse
Affiliation(s)
- D S G Scrimgeour
- Centre for Healthcare Education Research and Innovation University of Aberdeen Aberdeen UK.,Department of Colorectal Surgery Aberdeen Royal Infirmary Aberdeen UK.,Intercollegiate Committee for Basic Surgical Examinations
| | - J Cleland
- Centre for Healthcare Education Research and Innovation University of Aberdeen Aberdeen UK
| | - A J Lee
- Department of Medical Statistics University of Aberdeen Aberdeen UK
| | - P A Brennan
- Intercollegiate Committee for Basic Surgical Examinations
| |
Collapse
|
16
|
Woolf K, Jayaweera H, Unwin E, Keshwani K, Valerio C, Potts H. Effect of sex on specialty training application outcomes: a longitudinal administrative data study of UK medical graduates. BMJ Open 2019; 9:e025004. [PMID: 30837254 PMCID: PMC6429837 DOI: 10.1136/bmjopen-2018-025004] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES To examine sex differences in the specialty training recruitment outcomes of UK medical graduates; and whether sex differences were explained by prior academic attainment and previous fitness to practise (FtP) declarations. DESIGN Retrospective longitudinal cohort study. SETTING Administrative data on entrants to all UK medical schools from the UK Medical Education Database. PARTICIPANTS 10 559 doctors (6 155; 58% female) who entered a UK medical school in 2007 or 2008 and were eligible to apply for specialty training by 2015. PRIMARY OUTCOME MEASURE Odds of application, offer and acceptance to any specialty training programme, and on to each of the nine largest training programmes, adjusting for sex, other demographics, prior academic attainment, FtP declaration and medical school. RESULTS Across all specialties, there were no sex differences in applications for specialty training, but women had increased odds of getting an offer (OR=1.40; 95% CI=1.25 to 1.57; p<0.001) and accepting one (OR=1.43; 95% CI=1.19 to 1.71; p<0.001). Seven of the nine largest specialties showed significant sex differences in applications, which remained after adjusting for other factors. In the adjusted models, Paediatrics (OR=1.57; 95% CI=1.01 to 2.46; p=0.046) and general practice (GP) (OR=1.23; 95% CI=1.03 to 1.46; p=0.017) were the only specialties to show sex differences in offers, both favouring women. GP alone showed sex differences in acceptances, with women being more likely to accept (OR=1.34; 95% CI=1.03 to 1.76; p=0.03). Doctors with an FtP declaration were slightly less likely to apply to specialty training overall (OR=0.84; 95% CI=0.71 to 1.00; p=0.048) and less likely to accept an offer to any programme (OR=0.71; 95% CI=0.52 to 0.98; p=0.036), after adjusting for confounders. CONCLUSIONS Sex segregation between medical specialties is due to differential application, although research is needed to understand why men are less likely to be offered a place on to GP and Paediatrics training, and if offered GP are less likely to accept.
Collapse
Affiliation(s)
- Katherine Woolf
- Research Department of Medical Education, University College London Medical School, London, UK
| | - Hirosha Jayaweera
- Research Department of Medical Education, University College London Medical School, London, UK
- Centre for Clinical Research in Neuropsychiatry, University of Western Australia, Crawley, Western Australia, Australia
| | - Emily Unwin
- Research Department of Medical Education, University College London Medical School, London, UK
| | - Karim Keshwani
- Research Department of Medical Education, University College London Medical School, London, UK
- North Middlesex University Hospital NHS Trust, London, UK
| | - Christopher Valerio
- Research Department of Medical Education, University College London Medical School, London, UK
- North Middlesex University Hospital NHS Trust, London, UK
| | - Henry Potts
- Centre for Health Informatics and Multiprofessional Education, UCL, London, UK
| |
Collapse
|
17
|
Scrimgeour DSG, Brennan PA, Griffiths G, Lee AJ, Smith FCT, Cleland J. Does the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination predict 'on-the-job' performance during UK higher specialty surgical training? Ann R Coll Surg Engl 2018; 100:1-7. [PMID: 30286650 PMCID: PMC6204508 DOI: 10.1308/rcsann.2018.0153] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/02/2018] [Indexed: 11/22/2022] Open
Abstract
INTRODUCTION The Intercollegiate Membership of the Royal College of Surgeons (MRCS) is a mandatory examination to enter higher surgical specialty training in the UK. It is designed to help to ensure that successful candidates are competent to practice as higher surgical trainees. The annual review of competence progression (ARCP) assesses trainees' competence to progress to the next level of training and can be interpreted as a measure of 'on-the-job' performance. We investigated the relationship between MRCS performance and ARCP outcomes. MATERIALS AND METHODS All UK medical graduates who passed MRCS (Parts A and B) from 2007 to 2016 were included. MRCS scores, attempts and sociodemographics for each candidate were crosslinked with ARCP outcomes (satisfactory, unsatisfactory and insufficient evidence). Multinomial logistic regression was used to identify potential independent predictors of ARCP outcomes. RESULTS A total of 2570 trainees underwent 11,064 ARCPs; 1589 (61.8%) had only satisfactory outcomes recorded throughout training; 510 (19.9%) had at least one unsatisfactory outcome; and 471 (18.3%) supplied insufficient evidence. After adjusting for age, gender, first language and Part A performance, ethnicity (non-white vs white, OR 1.36, 95% CI 1.08 to 1.71), Part B passing score (OR 0.98, 95% CI 0.98 to 1.00) and number of attempts at Part B (two or more attempts vs one attempt, OR 1.50, 95% CI 1.16 to 1.94) were found to be independent predictors of an unsatisfactory ARCP outcome. CONCLUSIONS This is the first study to identify predictors of ARCP outcomes during higher surgical specialty training in the UK and provides further evidence of the predictive validity of the MRCS examination.
Collapse
Affiliation(s)
- DSG Scrimgeour
- Centre for Healthcare Education Research and Innovation, University of Aberdeen, Aberdeen, Scotland, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, Scotland, UK
- Intercollegiate Committee for Basic Surgical Examinations, UK
| | - PA Brennan
- Intercollegiate Committee for Basic Surgical Examinations, UK
| | - G Griffiths
- Department of Vascular Surgery, Ninewells Hospital, Dundee, Scotland, UK
| | - AJ Lee
- Medical Statistics Team, University of Aberdeen, Aberdeen, Scotland, UK
| | - FCT Smith
- Intercollegiate Committee for Basic Surgical Examinations, UK
| | - J Cleland
- Centre for Healthcare Education Research and Innovation, University of Aberdeen, Aberdeen, Scotland, UK
| |
Collapse
|
18
|
Ajah JL. Low Pass-rate in postgraduate surgical examination in Nigeria and its contribution to the low surgeon workforce in the country; a review article. SICOT J 2018; 4:36. [PMID: 30176165 PMCID: PMC6120751 DOI: 10.1051/sicotj/2018008] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2017] [Accepted: 02/24/2018] [Indexed: 11/15/2022] Open
Abstract
Surgical postgraduate examiners and examinees in Nigeria complain of the low pass rate at all levels of the postgraduate surgical training examinations to which several factors are contributing. For several years there has been being a persistently low surgeon workforce in the country despite having two surgeon producing institutions been for at least 37 years. A review of the probable causes was carried out to shed more light on the matter. At the time of writing there are 52 National Postgraduate Medical College of Nigeria (NPMCN) and 46 West African College of Surgeons (WACS) accredited post graduate surgery training programs in Nigeria compared with 99 in the United Kingdom (UK) and 1056 in the United States (US). Based on available data Nigeria has approximately 572 surgery residency training slots yearly compared with approximately 646 in the UK and 4225 in the US. Examination pass rate was less than 40% for primary WACS compared with 98% pass rate in USMLE (United States Medical Licensing Examination) 3, pass rate at part I was 28.8% for WACS compared with 37% at MRCS (Membership Royal College of Surgeons) part A and 57% for MRCS part B. For the exit examination or part II WACS pass rate was 31.5% (general surgery) while it was 64% for Fellowship Royal College of Surgeons (FRCS) cumulative and 70% in the American board of surgery (ABS). Surgeon per 100 000 population was 0.69 for Nigeria compared with 11.7 and 25.6 for the UK and US respectively. In the last 35 years WACS has produced 1638 surgeons (2.8 times more than NPMCN) in surgery and NPMCN has produced 572. The frequency of examination were twice per year for both WACS and NPMCN examinations, 3 times per year for the USMLE step 3, MRCS (A & B) and Fellowship Royal College of Surgeons (FRCS) general surgery. The American Board of Surgery (ABS) is once per year for Qualifying Examination (QE) and 5 times per year for Certifying Examination (CE).
Collapse
Affiliation(s)
- Jonathan L. Ajah
- Specialty Registrar in Surgery, Jos University Teaching Hospital,
Jos Nigeria
| |
Collapse
|
19
|
Scrimgeour D, Cleland J, Lee A, Brennan P. Which factors predict success in the mandatory UK postgraduate surgical exam: The Intercollegiate Membership of the Royal College of Surgeons (MRCS)? Surgeon 2018; 16:220-226. [DOI: 10.1016/j.surge.2017.10.001] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2017] [Revised: 09/29/2017] [Accepted: 10/02/2017] [Indexed: 11/17/2022]
|
20
|
Cleland JA, Durning SJ, Driessen E. Medical education research: aligning design and research goals. Med J Aust 2018; 208:473-475. [PMID: 29902401 DOI: 10.5694/mja17.00941] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2017] [Accepted: 02/07/2018] [Indexed: 11/17/2022]
Affiliation(s)
- Jennifer A Cleland
- Centre for Healthcare Education Research and Innovation, University of Aberdeen, Aberdeen, UK
| | - Steven J Durning
- Uniformed Services University of the Health Sciences, Bethesda, MD, USA
| | | |
Collapse
|