Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the Endoscopy Training System

Novel Use of Objective Sensor Technology: Creation of Individualized Education Plans to Develop Operative Mastery

BACKGROUND

Motion tracking has been shown to correlate with expert and novice performance but has not been used for skill development. For skill development, performance goals must be defined. We hypothesize that by using wearable sensor technology, motion-tracking outcomes can be identified in those deemed practice ready (PR) and used as benchmarks for precision learning.

STUDY DESIGN

At the American College of Surgeons Clinical Congress, surgeons and surgeons in training volunteered to wear motion-tracking sensors while performing intracorporeal suturing and knot tying, laparoscopic pattern cutting, and simulated laparoscopic cholecystectomy. Coach raters evaluated the participants using a modified SIMPL scale. Participants were divided into 2 groups based on coach ratings: PR and not PR. Motion results were compared. Benchmark performance for each motion-tracking outcome of the simulated laparoscopic cholecystectomy was set at the median (±median absolute deviation) of the PR cohort. For all participants, each motion-tracking outcome was compared with the benchmark.

RESULTS

A total of 94 participants were recruited. Fifty-three were rated PR. Differences between groups were identified in 2 of 10 metrics for intracorporeal suturing and knot tying, 4 of 10 metrics for pattern cutting, and 5 of 10 metrics for laparoscopic cholecystectomy. Set benchmarks for the metrics were compared with each participant's score for all individual metrics. A not PR individual was less likely to achieve the benchmarks (chi-square = 55.48, p < 0.00001).

CONCLUSIONS

Wearable sensor technology can identify differences between surgeons rated PR and not PR. More importantly, motion metric results can be used to develop benchmarks for training endpoints. This will allow for the development of an individualized report card and training protocol focused on areas in need of improvement instead of the current model of generalized training. Implementing such focused training may expedite competency and mastery of surgical skills.

Can my surgeon scope? Trends in endoscopy training volume and experience among general surgery residents in the United States: a nationwide analysis

INTRODUCTION

Endoscopy is a major part of surgical training. Accreditation Council for Graduate Medical Education (ACGME) has set standards regarding the minimum volume of endoscopy cases required for graduation. However, there is paucity of high-quality data on the number of cases that most surgical graduates perform.

METHODS

We conducted a retrospective analysis of operative case logs of all general surgery residents graduating from ACGME-accredited programs from 2010 to 2023. Data on mean number of endoscopy cases, including mean in each subcategory, were retrieved. Mann-Kendall trend test was used to investigate trends in endoscopy experience.

RESULTS

Between 2010 and 2023, the mean overall endoscopy procedures per resident remained stable, with 129.5 in 2010 and 132.1 in 2023 (t = 0.429; p-value = 0.037). The majority of these cases were performed as surgeon junior (76.6% in 2010; 80.9% in 2023), while the remaining cases were logged as surgeon chief. The most substantial contribution to the overall volume was from flexible colonoscopy (mean: 64.1 in 2010 and 67.2 in 2023). The volume for colonoscopy remained fairly stable (t = 0.429; p-value = 0.036). This was followed by esophagogastroduodenoscopy (mean: 35.3 in 2010 and 35.5 in 2023), which saw a significant increase in volume (t = 0.890; p-value ≤ 0.001). There was a significant increase in the number of overall upper endoscopic procedures (t = 0.791; p-value ≤ 0.001), while lower endoscopic procedures did not change significantly (t = 0.319; p-value = 0.125). The procedural volume for endoscopic retrograde cholangiography, sigmoidoscopy, cystoscopy/ureteroscopy, laryngoscopy, and bronchoscopy decreased significantly (p-value < 0.05 for all).

CONCLUSION

The overall endoscopy volume for general surgery residents has largely remained stable, with a minor increase in esophagogastroduodenoscopy and no change in colonoscopy. Future research should investigate whether simulation-based exercises can bridge the gap between procedural volume and learning curve requirements for endoscopy.

National Cohort Study of Resource Utilization in Older Adults With Emergency General Surgery Conditions

INTRODUCTION

Prior studies have sought to describe Emergency General Surgery (EGS) burden, but a detailed description of resource utilization for both operative and nonoperative management of EGS conditions has not been undertaken.

METHODS

Patient and hospital characteristics were extracted from Medicare data, 2015-2018. Operations, nonsurgical procedures, and other resources (i.e., radiology) were defined using Current Procedural Terminology codes.

RESULTS

One million eight hundred two thousand five hundred forty-five patients were included in the cohort. The mean age was 74.7 y and the most common diagnoses were upper gastrointestinal. The majority of hospitals were metropolitan (75.1%). Therapeutic radiology services were available in 78.4% of hospitals and operating rooms or endoscopy suites were available in 92.5% of hospitals. There was variability in resource utilization across EGS subconditions, with hepatobiliary (26.4%) and obstruction (23.9%) patients most frequently undergoing operation.

CONCLUSIONS

Treatment of EGS diseases in older adults involves several interventional resources. Changes in EGS models, acute care surgery training, and interhospital care coordination may be beneficial to the treatment of EGS patients.

"Into the fire" approach to teaching endoscopic foreign body removal using a modular simulation curriculum

BACKGROUND

As flexible endoscopy becomes an increasingly valuable minimally invasive approach to surgical challenges, an efficient and comprehensive training curriculum is needed to train surgeons in therapeutic endoscopy. We developed a modular curriculum utilizing a simulation-based, "into the fire" approach to endoscopic foreign body removal for practicing physicians with task performance pre- and post-testing.

METHODS

From 2020 to 2021, two sessions of our advanced flexible endoscopy course were taught by two expert surgical endoscopists using ex-vivo porcine models. The course focused on safe removal techniques for various foreign bodies as part of an overall endoscopy curriculum that uses hands-on simulation-based pre-testing, didactics, and mentored practice sessions, followed by post-course examination. Pre- and post-course assessments and surveys were used to evaluate knowledge, performance, and confidence of participants, and subsequently analyzed using the Wilcoxon-signed rank test.

RESULTS

Of the 16 practicing physicians who participated in the course, 43.8% were certified in Fundamentals of Endoscopic Surgery, and 62.5% had completed > 200 prior upper endoscopies. Upon course completion, scoring on knowledge-based written examinations improved from 3.4 ± 1.9 to 5.8 ± 2.0 (p < 0.001). Technical facility of each participant demonstrated significant overall improvement with post-course score increased from 15.8 ± 2.5 to 23.6 ± 1.6 (p < 0.001), with skill refinement noted in technical subcategories of appropriate instrument use (p < 0.001), foreign body manipulation (p < 0.001), and successful foreign body removal (p < 0.001). Confidence surveys likewise demonstrated significant increase in confidence after completion of the curriculum 11.6 ± 3.4 to 23.0 ± 5.5 (p < 0.001).

CONCLUSIONS

The "into the fire" approach to teaching endoscopic foreign body removal utilizing our simulation module provides an effective curriculum to improve knowledge, confidence, and overall technical performance. Our methodology utilizes hands-on, simulation-based pre-testing prior to instruction. This introduces clinical scenarios and technical challenges, while accounting for and tailoring to provider-specific variation in knowledge and experience, facilitating training efficiency.

Teaching endoscopic management of gastrointestinal hemorrhage using a modular simulation curriculum

BACKGROUND

As flexible endoscopy is increasingly adopted as a minimally invasive approach to surgical challenges, an efficient curriculum is needed to train surgeons in therapeutic endoscopy. We developed a simulation-based approach to teaching endoscopic management of gastrointestinal hemorrhage as part of a modular curriculum, complete with task performance pre- and post-testing.

METHODS

Two sessions of our advanced flexible endoscopy course were taught using ex vivo porcine models to simulate active gastrointestinal hemorrhage and allow for training in hands-on endoscopic management. The module is composed of hands-on pretesting, didactics, mentored practice sessions, and postcourse assessments. Pre- and postcourse tests and surveys evaluated knowledge, confidence, and performance of participants and results were analyzed using the paired t test.

RESULTS

Sixteen practicing surgeons participated in the course. After course completion, overall knowledge-based assessments improved from 3.4 (±1.9) to 5.8 (±2.0) (P < .001). Although participants with glove sizes >7.0 and ≥2 years in practice had higher pretest evaluator scores (P = .045 and P = .020), all participants demonstrated overall improvement in endoscopic management of hemorrhage, with postcourse evaluator score increases from 20.9 (±1.6) to 23.6 (±2.0) (P = .001) and specific improvements in identification of target bleeding (P = .015), endoscopic clip setup (P < .001), and clip deployment (P = .002). Surveys also found increased confidence in competency after curriculum completion, 11.6 (±3.4)-23.0 (±5.5) (P < .001).

CONCLUSION

Our simulation-based approach to teaching the endoscopic management of gastrointestinal bleeding emphasizes hands-on pretesting and provides an effective training model to improve the knowledge, confidence, and technical performance of practicing surgeons.

Simulation-based mastery learning in gastrointestinal endoscopy training

Simulation-based mastery learning (SBML) is an emerging form of competency-based training that has been proposed as the next standard method for procedural task training, including that in gastrointestinal endoscopy. Current basic gastrointestinal endoscopy training relies on the number of procedures performed, and it has been criticized for its lack of objective standards that result in variable skills among trainees and its association with patient safety risk. Thus, incorporating simulators into a competency-based curriculum seems ideal for gastrointestinal endoscopy training. The curriculum for SBML in gastrointestinal endoscopy is currently being developed and has promising potential to translate into the clinical performance. Unlike the present apprenticeship model of "see one, do one, teach one," SBML integrates a competency-based curriculum with specific learning objectives alongside simulation-based training. This allows trainees to practice essential skills repeatedly, receive feedback from experts, and gradually develop their abilities to achieve mastery. Moreover, trainees and trainers need to understand the learning targets of the program so that trainees can focus their learning on the necessary skills and trainers can provide structured feedback based on the expected outcomes. In addition to learning targets, an assessment plan is essential to provide trainees with future directions for their improvement and ensure patient safety by issuing a passing standard. Finally, the SBML program should be planned and managed by a specific team and conducted within a developed and tested curriculum. This review discusses the current state of gastrointestinal endoscopy training and the role of SBML in that field.

A modular simulation curriculum to teach endoscopic stenting to practicing surgeons: an "Into the fire" approach

BACKGROUND

Flexible endoscopy is a valuable tool for the gastrointestinal (GI) surgeon, creating a need for effective and efficient training curricula in therapeutic endoscopic techniques for trainees and practicing providers. Here, we present a simulation-based modular curriculum using an "into the fire" approach with hands-on pre- and post-testing to teach endoscopic stenting to practicing surgeons.

METHODS

Three advanced flexible endoscopy courses were taught by expert surgical endoscopists from 2018 to 2019. The stenting module involved using self-expandable metal stents to manage simulated esophageal and gastroduodenal strictures on a non-tissue GI model. Based on the educational theories of inquiry-based learning, the simulation curriculum was designed with a series of pre-tests, didactics, mentored hands-on instructions, and post-tests. Assessments included a confidence survey, knowledge-based written test, and evaluation form specific to the hands-on performance of endoscopic stenting.

RESULTS

Twenty-eight practicing surgeons with varying endoscopic experiences participated in the course. Most of the participants (67.9%) had completed over 100 upper endoscopic procedures and 57.1% were certified in Fundamentals of Endoscopic Surgery. After completing the modular curriculum, participant confidence survey scores improved from 11.4 ± 4.2 to 20.7 ± 4.0 (p < 0.001). Knowledge-based written test scores also improved from 7.1 ± 1.2 to 8.4 ± 0.9 (p < 0.001). In terms of technical performance, overall hands-on performance scores improved from 21.3 ± 2.7 to 28.9 ± 1.2 (p < 0.001) with significant improvement in each individual component of the assessment (all p values < 0.01) and the greatest improvement seen in equipment handling (88%) and flow of procedure (54%).

CONCLUSION

Our modular simulation curriculum using an "into the fire" approach to teach endoscopic stenting is effective in improving learner knowledge, confidence, and hands-on performance of endoscopic stenting. This approach to simulation is effective, efficient, and adaptable to teaching practicing surgeons with varying levels of experience.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training

BACKGROUND

Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.

METHODS

Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.

RESULTS

Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.

CONCLUSION

A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Comparison of Two Simulated Procedural Assessment Formats in Attending Emergency Physicians

Background

Emergency physicians must be proficient at inserting central venous catheters and performing lumbar punctures to provide life-saving therapies to critically ill patients. An assessment of procedural skill is rarely performed after an emergency physician has completed residency. Current board certification exams for emergency medicine focus only on verbal descriptions of procedures to assess skill. We compared two methods of procedural skill assessment, simulated task trainer and verbal description, to assess the range of skill in central venous catheter insertion and lumbar punctures of emergency attending physicians at a large, urban, academic tertiary care institution.

Methodology

This is a prospective cohort study of simulated internal jugular central venous catheter insertion and lumbar puncture skill by emergency attending physicians on a task trainer versus verbal description. A total of 17 attending emergency medicine physicians consented to participate in the study during a yearly procedural skills session. For each subject, two expert raters used previously published checklists to assess procedural skill and give a global rating score.

Results

More checklist items were performed correctly on the task trainer than on verbal assessment for central line (task trainer = 78.4% ± 8.32% and verbal = 68.26% ± 8.9%) and lumbar puncture (task trainer = 85.57% ± 7.6% and verbal = 73.53%4 ± 10.34%) procedures, both with significant differences (p < 0.001). Of the participants, 82% strongly preferred the task trainer format to the verbal description assessment format.

Conclusions

The higher scores on the simulated format compared to the current verbal format imply that a shift towards simulated procedural assessment techniques may benefit examinees. More work is needed to determine if objective checklist scores for practicing attending emergency physicians correlate with subjective expert assessments of their procedural skills.

Early implementation of Fundamentals of Endoscopic Surgery training using a simulation-based mastery learning curriculum

BACKGROUND

After the American Board of Surgery announcement of the Flexible Endoscopy Curriculum requirement in 2014, we implemented a dedicated endoscopy rotation at the post graduate year (PGY)2 level including a simulation curriculum for Fundamentals of Endoscopic Surgery skills. Here we evaluate the outcomes of this implementation.

METHODS

Beginning in 2015, we developed a clinical endoscopy and simulation-based rotation to prepare for Fundamentals of Endoscopic Surgery testing. Originally, our curriculum was based on the published Texas Association of Surgical Skills Laboratories curriculum using the GI Mentor and transitioned to a mastery learning curriculum using the Endoscopy Training System in 2016. We evaluated the success of the curriculum in terms of first-time pass rates, training time required, and comparison to previously published benchmarks based on clinical experience.

RESULTS

Since 2015, a total of 37 general surgery residents in our program were Fundamentals of Endoscopic Surgery tested (PGY2 = 24, PGY3 = 4, PGY5 = 9); 84% (31) completed the Endoscopy Training System curriculum. At the time of testing, 73% (27) had performed <25 esophagogastroduodenoscopies, and 46% had performed <25 colonoscopies. Ninety-two percent (34) spent 10 hours or less completing the curriculum. The first-time pass rate for those completing the Endoscopy Training System curriculum was 97% vs 67% for those not completing the Endoscopy Training System curriculum (P = .01). For residents completing the Endoscopy Training System curriculum, total Fundamentals of Endoscopic Surgery scores were discernibly higher (472 vs 389, P < .01), as were 3/5 task scores (Nav1 80 vs 67, P = .02; Loop2 36 vs 8, P = .02; Retro3 89 vs 71, P = .02). Despite clinical inexperience (<25 esophagogastroduodenoscopies and <50 colonoscopies), PGY2s yielded a mean score of 454 and a pass rate of 92%. This was similar to PGY5s (427, 89%; P = .3) and compares to benchmark data of endoscopists with >300 cases.

CONCLUSION

Early implementation of flexible endoscopy training with a simulation-based curriculum results in Fundamentals of Endoscopic Surgery performance equal to a clinical experience not often gained during surgical residency. Often requiring <10 hours, this represents a fantastic return on investment for this training.