Assessment methods in surgical training in the United Kingdom

MCQs in-training examination scores of the surgical residency program in Thailand: the relationship between medical school vs public health-based training institutions

BACKGROUND

This study aims at investigating and evaluating the categorical knowledge of residency using an internet-based examination.

METHODS

All in-training examinees from 32 Thailand's general surgery residency training institutions participating in the online examination. One hundred fifty Multiple Choice questions (MCQs) were selected for this examination from a pool of previous MCQs used for board certification examinations. Baseline chracteristic of the examinee including residency year, training institution (medical schools and public health-based training institutions), regional-based area of the institution, overall test score, scoring by subcategory, total time to complete the examination, and the length of accredited as a training centre time were collected and analysed.

RESULTS

Total 613 examinees. The mean total score of 1st and 3rd year residency of a public health hospital institution differed from those with medical school-based training. On average, the scores in 4 out of 10 categories of residency from medical school training institutions were higher. However, residency from institutions with over 7 years of experience tended to score higher in the Liver, Biliary and Pancreas category.

CONCLUSIONS

The average scores of MCQs exams which reflect the medical knowledge for general surgical residency training at medical schools do not significantly differ from those of individuals undergoing public health training. Additionally, the mean scores of MCQs exams did not differ between high-experience training institutions and recent accredited training centers.

Exploring the surgical residents' experience of teaching and learning process in the operating room: A grounded theory study

BACKGROUND

Despite the development of valuable teaching methods and assessment tools in the field of surgery education, yet unpreparedness for independence practice by surgical residents remains a problem in most of the countries.

AIMS

This study aims to explain the surgical residents' experience of the teaching and learning process in the operating rooms in Iran.

MATERIAL AND METHODS

This study is a qualitative research that was done from May 2019 to June 2020 in Tehran and eight other cities in Iran. The selected methodology was constructivist grounded theory. The study setting was 11 teaching hospitals. Data were collected through 36 in-depth interviews and 132 h of observation. Participants were selected initially by purposive sampling and then by theoretical sampling for covering gaps and completing categories.

RESULTS

Findings showed that the confused educational system was the main concern of the residents' experience of the teaching and learning process in the operating room. This concern is investigated by dividing into four subcategories: Education in the shadow of treatment, inefficient education, patient safety versus trusting residents for independent practice, and unstructured assessment.

CONCLUSION

Based on residents' experience about challenges such as inefficient education, the influence of patient safety versus trusting residents for independent practice, and unstructured assessment, the surgical residents' education in the operating rooms needs to revise.

Influence of Trainer Role, Subspecialty and Hospital Status on Consultant Workplace-based Assessment Completion

OBJECTIVE

Performance assessment is challenging to administer and validate, yet remains central to patient safety and quality of care. The aim of this study was to evaluate Consultant Surgeon trainer performance with respect to Workplace Based Assessment (WBA) completion.

DESIGN

All WBAs for 60 Core Surgical Trainees (n = 2932) recorded in one academic year were analyzed using the Intercollegiate Surgical Curriculum Progamme. Primary outcome measures were numbers of WBAs performed related to trainer role (Assigned Educational Supervisor vs. Clinical Supervisor vs. No Training Role), gender, surgical subspecialty, hospital status (teaching vs. district general), and trainer RCSEng. TrACE course accreditation.

SETTING

A core surgical training program serving a single UK (Wales) deanery.

PARTICIPANTS

Sixty consecutively appointed core surgical trainees.

RESULTS

Median WBA number performed irrespective of trainer role was 6 (range 0-51), consisting of CBD 2 (0-18), mini-CEX 2 (0-22), DOPS 2 (0-32), and PBA 0 (0-10). Assigned Educational Supervisor trainers were more likely to complete the full range of WBAs compared with Clinical Supervisor and No Training Role assessors; WBA 17 vs. 6 vs. 3; CBD 5 vs. 2 vs. 1; mini-CEX 5 vs. 2 vs. 1; DOPS 4 vs. 2 vs. 1; and PBA 0 vs. 0 vs. 0 (p < 0.001). WBAs completed varied by subspecialty; first quartile performance: ENT, Plastic Surgery, (median 12, interquartile range 13), compared with fourth quartile: OMFS, Urology, T&O, and Cardiothoracic Surgery (median 5, interquartile range 11, p = 0.016). Hospital status, gender, and TrACE accreditation were not associated with WBA performance.

CONCLUSIONS

Important variations in trainer WBA completion were apparent; training programme directors and trainees alike should be aware of this when agreeing educational contracts.

CORTRAK Superuser Competency Assessment and Training Recommendations

BACKGROUND

Blind insertion of feeding tubes remains unsafe. Electromagnetic placement devices such as the CORTRAK Enteral Access System allow operators to interpret placement of feeding tubes in real time. However, pneumothoraces have been reported and inadequate user expertise is a concern.

OBJECTIVE

To explore factors influencing competency of CORTRAK-assisted feeding tube insertion.

METHODS

A prospective, observational pilot study was conducted. Data collection included demographics, self-confidence, clinical judgment regarding CORTRAK-assisted feeding tube insertion, and general self-efficacy. CORTRAK-assisted feeding tube insertions were performed with the Anatomical Box and CORMAN task trainers.

RESULTS

Twenty nurses who had inserted a mean of 53 CORTRAK feeding tubes participated. Participants inserted a mean of 2 CORTRAK feeding tubes weekly; each had inserted a feeding tube in the past 7 days. All superusers were competent; 1 required remediation for improper receiver unit placement. Mean (SD) scores were 35 (3.68) on a 40-point scale for self-efficacy, 4.6 (0.68) on a 5-point scale for self-reported feeding tube insertion confidence, and 4.85 (0.49) on a 5-point scale for demonstrated confidence. Participants estimated that 8 CORTRAK-assisted insertions were needed before they felt competent as super users. Confidence with the CORTRAK tracing was estimated to require 10 feeding tube insertions. Six participants continued to assess placement by auscultation, suggesting low confidence in their interpretation of the tracing.

CONCLUSIONS

At least 3 observations should be performed to assess initial competency; the number should be individualized to the operator. Interpretation of the insertion tracing is complex and requires multiple performance opportunities to gain competency and confidence for this high-risk skill.

Agreement between two raters' evaluation for integrated Traditional Prosthodontic Practical Exam with Directly Observed Procedural Skills in Egypt

PURPOSE

It aimed to determine the agreement between two raters evaluating students at prosthodontic clinical practical exam integrated with directly observed procedural skills (DOPS).

METHODS

A sample of 76 students was monitored by two raters to evaluate the process and the final registered maxillomandibular relation for completely edentulous patient in Mansoura Dental School, Egypt at practical exam of the Bachelor students on May 15, till June 28, 2017. Each registered relation was evaluated from total 60 marks subdivided to three score-categories: occlusal plane orientation (OPO), vertical dimension registration (VDR), and centric relation registration (CRR). The marks of each category included mark of DOPS. The marks of OPO and VDR for both raters were compared by graph method to measure reliability using the Bland and Altman analysis. The reliability of CRR marks was evaluated by Krippendorff's alpha ratio.

RESULTS

The results revealed similarity between raters for OPO (mean = 18.1) and closes of limits of agreement (0.73 and -0.78). For VDR, there were closeness of means (mean= 17.4 and 17.1 for examiner 1 and 2 respectively); with limits of agreement (2.7and-2.2). There was a strong correlation (Krippendorff's alpha ratio= 0.92; 95% CI [0.79-0.99]) among raters at evaluating CRR.

CONCLUSION

The two raters' evaluation of clinical traditional practical exam integrated with directly observed procedural skills revealed not to be different to evaluate candidate at the end of the clinical prosthodontic course. The limits of agreement between raters would be optimum at exclusion subjective evaluation parameters and complicated cases from examination procedures.

The Role of Simulation in Microsurgical Training

Simulation has been established as an integral part of microsurgical training. The aim of this study was to assess and categorize the various simulation models in relation to the complexity of the microsurgical skill being taught and analyze the assessment methods commonly employed in microsurgical simulation training. Numerous courses have been established using simulation models. These models can be categorized, according to the level of complexity of the skill being taught, into basic, intermediate, and advanced. Microsurgical simulation training should be assessed using validated assessment methods. Assessment methods vary significantly from subjective expert opinions to self-assessment questionnaires and validated global rating scales. The appropriate assessment method should carefully be chosen based on the simulation modality. Simulation models should be validated, and a model with appropriate fidelity should be chosen according to the microsurgical skill being taught. Assessment should move from traditional simple subjective evaluations of trainee performance to validated tools. Future studies should assess the transferability of skills gained during simulation training to the real-life setting.

Methodological shortcomings in the literature evaluating the role and applications of 3D training for surgical trainees

AIM

There has been an increased interest in the use of three-dimensional (3D) technology in surgical training. We wish to appraise the methodological rigor applied to evaluating the role and applications of 3D technology in surgical training, in particular, on the validity of these models and assessment methods in simulated surgical training.

METHODS

Literature search was performed using MEDLINE with the following terms: "3D"; "surgery"; and "training". Only studies evaluating the role of 3D technology in surgical training were eligible for inclusion and assessed for the level of evidence, validity of the simulation model, and assessment method used.

RESULTS

A total of 93 studies were analyzed, and majority of reviewed articles focused on 3D displays (36) and 3D printing (35). Most of these studies were case series, the most common assessment was subjective (69), with objective assessment used by 57 studies. Very few studies provided evidence for validity of the model or the assessment methods used.

CONCLUSIONS

3D technology has a great potential in simulated surgical training. However, the validity of this technology and strong evidence for its beneficial effects in surgical training is lacking. Further work on validation of 3D technology and assessment tools is needed.

Performance optimization of force feedback control system in virtual vascular intervention surgery

In virtual surgery of minimally invasive vascular intervention, the force feedback is transmitted through the flexible guide wire. The disturbance caused by the flexible deformation would affect the fidelity of the VR (virtual reality) training. SMC (sliding mode control) strategy with delayed-output observer is adopted to suppress the effect of flexible deformation. In this study, the control performance of the strategy is assessed when the length of guide wire between actuator and the operating point changes. The performance assessment results demonstrate the effectiveness of the proposed method and find the optimal length of guide wire for the force feedback control.