Development of a train-to-proficiency curriculum for the technical skills component of the fundamentals of endoscopic surgery exam

Current status of resident simulation training curricula: pearls and pitfalls

BACKGROUND

Residency programs are required to incorporate simulation into their training program. Ideally, simulation provides a safe environment for a trainee to be exposed to both common and challenging clinical scenarios. The purpose of this review is to detail the current state of the most commonly used laparoscopic, endoscopic, and robotic surgery simulation programs in general surgery residency education, including resources required for successful implementation and benchmarks for evaluation.

MATERIALS AND METHODS

Members of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) Resident and Fellow Task Force (RAFT) Committee performed a literature review using PubMed and training websites. Information regarding the components of the most commonly used laparoscopic, endoscopic, and/or robotic simulation curriculum, including both formal and informal benchmarks for evaluating training competence, were collected.

RESULTS

Laparoscopic simulation revolves around the Fundamentals of Laparoscopic Surgery (FLS). Proficiency-based as well as virtual simulation have been utilized for FLS training curricula. Challenges include less direct translation to the technical complexities that can arise in laparoscopic surgery. Endoscopic simulation focuses on the Fundamentals of Endoscopic Surgery. There are virtual reality simulation platforms that can be used for skills assessment and training. Challenges include simulator types and access, as well as structured mentoring and feedback. Robotic simulation training curricula have not been standardized. Simulation includes one primary technology, which can be prohibitive based on cost and requirements for onboarding.

CONCLUSIONS

While surgical simulation seems to be a fundamental and integrated part of surgical training, it requires a significant number of resources, which can be daunting for residency training programs. Regardless of the barriers outlined, the need for surgical simulation in laparoscopy, endoscopy, and robotics at surgical education training programs is clear.

The Impact of Simulation Based Training on the Fundamentals of Endoscopic Surgery Performance Examination

OBJECTIVE

To determine if simulation training is required to pass the FES skills test and assess the relationship between simulation training, clinical training, and FES skills test performance.

SUMMARY OF BACKGROUND DATA

The ABS began requiring completion of the Flexible Endoscopy Curriculum for all applicants beginning in 2018. The role of simulation-based training in FES skills test performance after this requirement has not been evaluated.

METHODS

De-identified data from the initial FES skills tests after the Flexible Endoscopy Curriculum requirement was reviewed, and 731 unique participants with reported simulation experience demographics were identified. Self-reported data included sex, upper (UE) and lower (LE) endoscopy experience, and simulator training hours (SE). Final FES skills exam scores and pass/fail designations for each participant were reported by the FES program staff.

RESULTS

There was a statistically discernible difference in mean FES total scores between those reporting no SE and more experienced groups ( P = 0.002), and between less and more experienced UE and LE groups ( P < 0.001). There was no statistically discernible difference in FES skills exam pass rates between SE groups ( P = 0.2), but there was a strong relationship between clinical experience (UE & LE) and pass rate ( P < 0.001). Finally, on logistic regression analysis, LE was a discernible predictor of passing [odds ratio (OR) = 1.4, 95% confidence interval (CI) 1.1-1.8, P = 0.02], while UE [odds ratio (OR) = 1, 95% CI 0.8-1.3, P = 0.9] and SE (OR = 1,95% CI 0.9- 1.3, P = 0.7) were not.

CONCLUSIONS

There is no threat to the validity of the FES skills test from a need for simulation training to pass the FES skills test. Similarly, the amount of simulation practice is not predictive of passing, but can improve performance on certain FES tasks.

The American Board of Surgery flexible endoscopy curriculum prepares individuals to pass the fundamentals of endoscopic surgery manual skills test

BACKGROUND

The American Board of Surgery (ABS) has required Fundamentals of Endoscopic Surgery (FES) certification for general surgery applicants since 2018. Flexible Endoscopy Curriculum (FEC) completion is recommended prior to taking the FES exam. The objective of the study was to determine if FEC completion prepares individuals to pass the FES manual skills test.

METHODS

Participants included first-attempt FES examinees from June 2014 to February 2019. De-identified data were reviewed, Self-reported data included gender, PGY, glove size, upper (UE) and lower (LE) endoscopy experience, simulation training time, and participation in an endoscopy rotation (ER). FES skills exam performance was reported by FES staff. Those completing all vs. none of the FEC were compared.

RESULTS

Of 2023 participants identified, 809 (40.0%) reported completion of all FEC components, 1053 (52.1%) completed of some, and 161 (8.0%) completed none. Men and candidates taking FES later in residency were more likely to complete all FEC requirements (p = 0.002, p < 0.001). FES pass rates were higher for those who completed all FEC components compared to those who completed none (88.4% vs 72.7%, p < 0.001). On logistic regression analysis, completion of all components (OR 2.3, 95% CI 1.5-3.7, p < 0.001) and male gender (OR 3.1, 95% CI 1.7-5.7, p < 0.001) were predictors of passing, while glove size (OR 1.5, 95% CI 1.0-2.5, p = 0.08), simulator time (OR 1.1, 95% CI 0.9-1.4, p = 0.37) and PGY were not (OR 1.1, 95% CI 0.9-1.4, p = 0.38). On multivariate analysis controlling for glove size and gender, completion of all FEC components was still associated with a higher likelihood of passing the FES skills exam (OR 1.6, 95% CI 1.2-2.1, p < 0.001).

CONCLUSIONS

Completion of FEC is strongly associated with passing the FES skills test. This study supports the ABS recommendation for completion of FEC prior to taking the FES skills test.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training

BACKGROUND

Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.

METHODS

Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.

RESULTS

Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.

CONCLUSION

A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Democratizing Flexible Endoscopy Training: Noninferiority Randomized Trial Comparing a Box-Trainer vs a Virtual Reality Simulator to Prepare for the Fundamental of Endoscopic Surgery Exam

BACKGROUND

A considerable number of surgical residents fail the mandated endoscopy exam despite having completed the required clinical cases. Low-cost endoscopy box trainers (BTs) could democratize training; however, their effectiveness has never been compared with higher-cost virtual reality simulators (VRSs).

STUDY DESIGN

In this randomized noninferiority trial, endoscopy novices trained either on the VRS used in the Fundamental of Endoscopic Surgery manual skills (FESms) exam or a validated BT-the Basic Endoscopic Skills Training (BEST) box. Trainees were tested at fixed timepoints on the FESms and on standardized ex vivo models. The primary endpoint was FESms improvement at 1 week. Secondary endpoints were FESms improvement at 2 weeks, FESms pass rates, ex vivo tests performance, and trainees' feedback.

RESULTS

Seventy-seven trainees completed the study. VRS and BT trainees showed comparable FESms improvements (25.16 ± 14.29 vs 25.58 ± 11.75 FESms points, respectively; p = 0.89), FESms pass rates (76.32% vs 61.54%, respectively; p = 0.16) and total ex vivo tasks completion times (365.76 ± 237.56 vs 322.68 ± 186.04 seconds, respectively; p = 0.55) after 1 week. Performances were comparable also after 2 weeks of training, but FESms pass rates increased significantly only in the first week. Trainees were significantly more satisfied with the BT platform (3.97 ± 1.20 vs 4.81 ± 0.40 points on a 5-point Likert scale for the VRS and the BT, respectively; p < 0.001).

CONCLUSIONS

Simulation-based training is an effective means to develop competency in endoscopy, especially at the beginning of the learning curve. Low-cost BTs like the BEST box compare well with high-tech VRSs and could help democratize endoscopy training.

Resident Endoscopy Experience Correlates Poorly with Performance on a Virtual Reality Simulator

Background  Endoscopy training has become increasingly emphasized during general surgery residency as reflected by introduction of the Fundamentals of Endoscopic Surgery (FES) examination, which includes testing of skills on virtual reality (VR) simulators. Although studies exist to assess the ability of the simulator to differentiate between novices and experienced endoscopists, it is not well understood how simulators can differentiate skills among resident cohort.

Objective  To assess the utility of the VR simulator, we evaluated the correlation between resident endoscopy experience and performance on two VR simulator colonoscopy modules on the GI-BRONCH Mentor (Simbionix Ltd, Airport City, Israel).

Methods  Postgraduate years 2 to 5 residents completed “easy” and “difficult” VR colonoscopies, and performance metrics were recorded from October 2017 to February 2018 at Rutgers' two general surgery residency programs. Resident endoscopy experience was obtained through Accreditation Council for Graduate Medical Education case logs. Correlations between resident endoscopy experience and VR colonoscopy performance metrics were assessed using Spearman's rho (ρ) correlation statistic and bivariate logistic regression.

Results  Fifty-five residents out of 65 (84.6%) eligible participants completed the study. There were limited correlations found between resident endoscopy experience and FES performance metrics and no correlations were found between resident endoscopy experience and binary metrics of colonoscopy—ability to complete colonoscopy, ability to retroflex, and withdrawal time of less than 6 minutes.

Conclusion  The VR simulator may have a limited ability to discriminate between experience levels among resident cohort. Future studies are needed to further understand how well the VR simulator metrics correlate with resident endoscopy experience.

Fact or Fiction? Practice on a Simulator is Not Required to Pass the Fundamentals of Endoscopic Surgery (FES)TM Skills Exam

PURPOSE

Initial work on the validity evidence used to support the Fundamentals of Endoscopic Surgery (FES) performance exam as a measure of technical competency showed a strong relationship to clinical experience. Despite this evidence, there is a perception among some program directors that the exam cannot be successfully passed without practice on a simulator. We assess the validity of this perception.

METHODS

Deidentified data from the initial FES skills examination (prior to the 2014 FEC requirement) was reviewed, and 335 unique participants with reported simulation experience demographics were identified. Self reported data analyzed included gender, total clinical endoscopy procedure experience (1-150, 151-300, >300), and endoscopy simulator training hours (0, 1-5, 6-10, 11-20, >20). Final FES skills exam scores, and pass/fail designations for each participant were reported by the FES program staff. Continuous variables were compared between groups using one-way analysis of variance with post-hoc analysis where appropriate. Categorical variables were compared using Pearson Chi-Squared. The effect of variables on pass rate was assessed using univariate and multivariate logistic regression.

RESULTS

Simulation training experience (SE) was categorically reported in hours(n,%): 0 (98, 29%), 1-5 (135, 40%), 6-10 (52, 16%), 11-20 (24, 7%), and >20 (26, 8%). Clinical endoscopy experience (CE), reported categorically as total cases performed (n,%), was available for 323 of 355 identified participants: 1-150 (126, 39%), 151-300 (99, 31%), >300 (98, 30%). There was no statistically discernible differences in mean FES total or task scores across the SE groups (total score 0:72 ± 15, 1-5:72 ± 13, 6-10:71 ± 14, 11-20:71 ± 16, 20:78 ± 13; p = 0.28), while both total score and task scores were discernibly higher in the more experienced CE groups (>151) compared to the least experienced group (total score; <150:67 ± 15, 151-300:75 ± 1, >300:77 ± 14; p < 0.01). Similarly, there was no statistically discernible difference in FES skills exam pass rates between SE groups (0: 80%, 1-5: 82%, 6-10: 79%, 11-20: 75%, >20: 85%; x2 = 2.5, p = 0.6), but there was a strong relationship between clinical experience and pass rate (<150: 70%, 151-300: 87%, >300: 89%; x2 = 15.8, p < 0.001). Finally, on both univariate and multivariate logistic regression, CE remained a discernible predictor of passing, even when controlling for SE (odds ratio = 2, 95% confidence interval 1.4-2.9, p < 0.001).

CONCLUSIONS

FES skills examination data collected on participants completing the examination before the FEC requirement shows no demonstrable relationship with self-reported training experience on a simulator but confirms a strong relationship with clinical endoscopy experience. This lends further evidence to the validity of the FES exam as a marker of clinical endoscopic skill.

Confidence and Competence in Volunteer Examiners of the American Board of Surgery

BACKGROUND

Previous work has demonstrated the impact of sex on the relationship between confidence and competence, with women having less confidence and less self-perceived competence. Recent recruitment of examiners by the American Board of Surgery (ABS) allowed exploration of confidence and perceived competence.

STUDY DESIGN

We solicited 23,869 eligible board-certified surgeons (4,382 women) via email to serve as examiners for the ABS General Surgery Certifying Examination (GCE). Volunteers were asked about practice and comfort examining in the 14 GCE content areas. Regression models investigated the relationship between reported practice and confidence examining.

RESULTS

There were 2,157 surgeons who volunteered, of whom 420 (19.5%) were female. Men reported practicing in a greater number of content areas than women (6.59 vs 5.36, p < 0.001) and selected more content areas in which they felt comfortable examining (7.69 vs 6.64, p < 0.001). The average male volunteer was comfortable examining in 0.70 more content areas than women, controlling for self-reported practice.

CONCLUSIONS

Men reported practicing more broadly and were more confident than women in their ability to examine across ABS Certifying Examination content areas. However, the confidence of male surgeons was not restricted to those areas in which they reported practicing. These data have important implications for training, leadership, and professional development.

An Inexpensive, Portable Physical Endoscopic Simulator: Description and Initial Evaluation

BACKGROUND

Surgeons and gastroenterologists in training benefit from simulation-based endoscopy education, yet the price of most training endoscopy simulators is prohibitive. We set out to create and evaluate a low-cost endoscopic simulator and box model trainer for learning fundamental endoscopic skills.

METHODS

After adding a wireless network-enabled camera (total cost, $20) to a discarded clinical endoscope, we paired this with an easily constructed box trainer (cost $32) to generate an endoscopic simulator system (YazanoScope) for simulation training. Participants (general surgery interns, research fellows, and medical and college students) used the YazanoScope to train to mastery on 5 FES tasks. Outcomes included skill assessments on a computer simulator and trainees' perceptions comparing the physical model to the computer simulator.

RESULTS

Forty trainees participated. The median (range) training time was 110 (60-180) min. Only 10% of trainees were able to reach the cecum at baseline compared to 100% after training. The mean (SD) time was 253 (154) s at baseline (including completers and non-completers) and 249 (89) after training (P = 0.88). On a retention test 2 wk later, 21 of 22 (96%) completed the computer simulator assessment (endoscope tip reached the cecum). Mean time was 214 (67) s (P = 0.32 compared with immediate posttraining). All 40 trainees believed the YazanoScope provided better haptic feedback than the computer simulator.

CONCLUSIONS

Training with this inexpensive, portable endoscopic simulator (YazanoScope) was associated with increased procedure completion with no change in procedure time. All participants favored the haptic feedback of the $52 YazanoScope over a computer simulator.

Assessing the Value of Endoscopy Simulator Modules Designed to Prepare Residents for the Fundamentals of Endoscopic Surgery Examination

BACKGROUND

The Fundamentals of Endoscopic Surgery examination is required for all general surgery residents. The test modules are not available for practice before the examination; however, similar modules are commercially available.

OBJECTIVE

This study aims to determine which modules are most valuable for resident training and preparation for the examination by evaluating which correlates best with experience level.

DESIGN

This was a single-institution study.

SETTING

A virtual reality endoscopy simulator was utilized.

PARTICIPANTS

General surgery residents and faculty endoscopists performed endoscopy simulator modules (Endobasket 2, Endobubble 1 and 2, Mucosal Evaluation 2, and Basic Navigation) designed to prepare residents for the Fundamentals of Endoscopic Surgery examination. Residents were assigned into junior and senior groups based on the completion of a dedicated endoscopy rotation.

MAIN OUTCOME MEASURES

The primary outcomes measured were the mean time to completion, mean number of balloons popped, and mean number of wall hits for the 3 groups.

RESULTS

A total of 21 junior residents, 11 senior residents, and 3 faculty participated. There were significant differences among groups in the mean time to completion for the Endobasket, Endobubble, and Mucosal Evaluation modules. The modules that correlated best with experience level were Endobubble 2 and Mucosal Evaluation 2. For Endobubble 2, juniors were slower than seniors, who were in turn slower than faculty (junior 118.8 ± 20.55 seconds, senior 100.3 ± 11.78 seconds, faculty 87.67 ± 2.848 seconds; p < 0.01). Juniors popped fewer balloons than seniors, who popped fewer balloons than faculty (junior 9.441 ± 3.838, senior 15.62 ± 4.133, faculty 28.78 ± 1.712; p < 0.001). For Mucosal Evaluation 2, juniors were slower than seniors, who were in turn slower than faculty (junior 468.8 ± 123.5 seconds, senior 368.6 ± 63.42 seconds, faculty 233.1 ± 70.45 seconds; p < 0.01).

LIMITATIONS

Study residents have not completed the Fundamentals of Endoscopic Surgery examinations, so correlation with examination performance is not yet possible.

CONCLUSIONS

Performance on Endobasket, Endobubble, and Mucosal Evaluation correlated well with experience level, providing benchmarks for each level to attain in preparation for the Fundamentals of Endoscopic Surgery examination. See Video Abstract at http://links.lww.com/DCR/A823.

Training to Proficiency in the WHO Hand Hygiene Technique

OBJECTIVES

Hand hygiene is critical to patient safety, but low performance in terms of the quantity and quality of hand hygiene is often reported. Training-to-proficiency is common for other clinical skills, but no proficiency-based training program for hand hygiene has been reported in the literature. This study developed a proficiency-based training program to improve hand hygiene quality in line with World Health Organization (WHO) guidelines and assessed the amount of training required to reach proficiency. The training was delivered as part of a 5-day induction for students on the Physician Assistant online program.

METHODS

A total of 42 students used a simulator to objectively measure hand hygiene technique over a 5-day period. Proficiency was achieved when students demonstrated all 6 steps of the WHO technique in less than 42 seconds. The students also completed a postintervention questionnaire.

RESULTS

The average training episode lasted 2.5 minutes and consisted of 4.5 hand hygiene exercises. The average student completed 5 training episodes (1 per day) taking a total of 17 minutes. A total of 40% (17) of the students achieved proficiency within the 5 days. Proficiency was strongly correlated with the number of training exercises completed (r = 0.79, P < .001) and the total time spent training (r = 0.75, P < .001). Linear regression predicted that the 32 hand hygiene exercises or a total of 23-minute training were required to achieve proficiency.

CONCLUSIONS

This is the first study to develop a train-to-proficiency program for hand hygiene quality and estimate the amount of training required. Given the importance of hand hygiene quality to preventing health care-associated infections (HAIs), medical education programs should consider using proficiency-based training in hand hygiene technique.