Colonoscopy performance correlates with scores on the FES™ manual skills test

Study on impact of flexible endoscopy training course for surgeons in India

Context

Competence in flexible endoscopy is essential for all surgeons during this era of minimal access surgery. However, fewer surgeons have expertise in endoscopy due to a lack of training and interest. The Indian Association of Gastrointestinal Endo Surgeons devised a short-structured training course in the art and science of endoscopy.

Aims

This study aimed to find the impact of the endoscopy training course (Endoscopic Fellowship of Indian Association of Gastrointestinal Endo Surgeons [EFIAGES]) in improving the endoscopic skill of surgeons.

Settings and Design

Twenty-two-part electronic survey forms were sent to all 375 candidates who took the course between 2016 and 2019 for this retrospective observational study.

Subjects and Methods

The following outcome measures were noted, namely technical competence in endoscopy before the course, delegate feedback about the course modules, volume of endoscopies before and after the course and quality indicators such as reaching up to duodenum (D2) and caecum before and after the course.

Statistical Analysis Used

Statistical analysis of the impact of the course was done using Chi-square test.

Results

Responses from 262 out of a total of 375 candidates were received. Seventy-seven per cent of trainees were pleased with content and mode of conduct of the course. The quality indicator of gastroscopy with the ability to reach D2 in 90% of the caseload was achieved by only 28% of trainees before the EFIAGES. This increased to 72% of candidates after the course and similar results were seen with colonoscopy also. Most of the candidates noted a distinct improvement in their endoscopic navigation skills subsequent to the course.

Conclusions

Endoscopy skill transfer was possible with a short-structured endoscopy course. The surgical fraternity should realise the importance of endoscopy skills in the current era of surgical practice.

The Impact of Simulation Based Training on the Fundamentals of Endoscopic Surgery Performance Examination

OBJECTIVE

To determine if simulation training is required to pass the FES skills test and assess the relationship between simulation training, clinical training, and FES skills test performance.

SUMMARY OF BACKGROUND DATA

The ABS began requiring completion of the Flexible Endoscopy Curriculum for all applicants beginning in 2018. The role of simulation-based training in FES skills test performance after this requirement has not been evaluated.

METHODS

De-identified data from the initial FES skills tests after the Flexible Endoscopy Curriculum requirement was reviewed, and 731 unique participants with reported simulation experience demographics were identified. Self-reported data included sex, upper (UE) and lower (LE) endoscopy experience, and simulator training hours (SE). Final FES skills exam scores and pass/fail designations for each participant were reported by the FES program staff.

RESULTS

There was a statistically discernible difference in mean FES total scores between those reporting no SE and more experienced groups ( P = 0.002), and between less and more experienced UE and LE groups ( P < 0.001). There was no statistically discernible difference in FES skills exam pass rates between SE groups ( P = 0.2), but there was a strong relationship between clinical experience (UE & LE) and pass rate ( P < 0.001). Finally, on logistic regression analysis, LE was a discernible predictor of passing [odds ratio (OR) = 1.4, 95% confidence interval (CI) 1.1-1.8, P = 0.02], while UE [odds ratio (OR) = 1, 95% CI 0.8-1.3, P = 0.9] and SE (OR = 1,95% CI 0.9- 1.3, P = 0.7) were not.

CONCLUSIONS

There is no threat to the validity of the FES skills test from a need for simulation training to pass the FES skills test. Similarly, the amount of simulation practice is not predictive of passing, but can improve performance on certain FES tasks.

The American Board of Surgery flexible endoscopy curriculum prepares individuals to pass the fundamentals of endoscopic surgery manual skills test

BACKGROUND

The American Board of Surgery (ABS) has required Fundamentals of Endoscopic Surgery (FES) certification for general surgery applicants since 2018. Flexible Endoscopy Curriculum (FEC) completion is recommended prior to taking the FES exam. The objective of the study was to determine if FEC completion prepares individuals to pass the FES manual skills test.

METHODS

Participants included first-attempt FES examinees from June 2014 to February 2019. De-identified data were reviewed, Self-reported data included gender, PGY, glove size, upper (UE) and lower (LE) endoscopy experience, simulation training time, and participation in an endoscopy rotation (ER). FES skills exam performance was reported by FES staff. Those completing all vs. none of the FEC were compared.

RESULTS

Of 2023 participants identified, 809 (40.0%) reported completion of all FEC components, 1053 (52.1%) completed of some, and 161 (8.0%) completed none. Men and candidates taking FES later in residency were more likely to complete all FEC requirements (p = 0.002, p < 0.001). FES pass rates were higher for those who completed all FEC components compared to those who completed none (88.4% vs 72.7%, p < 0.001). On logistic regression analysis, completion of all components (OR 2.3, 95% CI 1.5-3.7, p < 0.001) and male gender (OR 3.1, 95% CI 1.7-5.7, p < 0.001) were predictors of passing, while glove size (OR 1.5, 95% CI 1.0-2.5, p = 0.08), simulator time (OR 1.1, 95% CI 0.9-1.4, p = 0.37) and PGY were not (OR 1.1, 95% CI 0.9-1.4, p = 0.38). On multivariate analysis controlling for glove size and gender, completion of all FEC components was still associated with a higher likelihood of passing the FES skills exam (OR 1.6, 95% CI 1.2-2.1, p < 0.001).

CONCLUSIONS

Completion of FEC is strongly associated with passing the FES skills test. This study supports the ABS recommendation for completion of FEC prior to taking the FES skills test.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training

BACKGROUND

Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.

METHODS

Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.

RESULTS

Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.

CONCLUSION

A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Democratizing Flexible Endoscopy Training: Noninferiority Randomized Trial Comparing a Box-Trainer vs a Virtual Reality Simulator to Prepare for the Fundamental of Endoscopic Surgery Exam

BACKGROUND

A considerable number of surgical residents fail the mandated endoscopy exam despite having completed the required clinical cases. Low-cost endoscopy box trainers (BTs) could democratize training; however, their effectiveness has never been compared with higher-cost virtual reality simulators (VRSs).

STUDY DESIGN

In this randomized noninferiority trial, endoscopy novices trained either on the VRS used in the Fundamental of Endoscopic Surgery manual skills (FESms) exam or a validated BT-the Basic Endoscopic Skills Training (BEST) box. Trainees were tested at fixed timepoints on the FESms and on standardized ex vivo models. The primary endpoint was FESms improvement at 1 week. Secondary endpoints were FESms improvement at 2 weeks, FESms pass rates, ex vivo tests performance, and trainees' feedback.

RESULTS

Seventy-seven trainees completed the study. VRS and BT trainees showed comparable FESms improvements (25.16 ± 14.29 vs 25.58 ± 11.75 FESms points, respectively; p = 0.89), FESms pass rates (76.32% vs 61.54%, respectively; p = 0.16) and total ex vivo tasks completion times (365.76 ± 237.56 vs 322.68 ± 186.04 seconds, respectively; p = 0.55) after 1 week. Performances were comparable also after 2 weeks of training, but FESms pass rates increased significantly only in the first week. Trainees were significantly more satisfied with the BT platform (3.97 ± 1.20 vs 4.81 ± 0.40 points on a 5-point Likert scale for the VRS and the BT, respectively; p < 0.001).

CONCLUSIONS

Simulation-based training is an effective means to develop competency in endoscopy, especially at the beginning of the learning curve. Low-cost BTs like the BEST box compare well with high-tech VRSs and could help democratize endoscopy training.

Colonoscopy competence assessment tools: a systematic review of validity evidence

BACKGROUND

Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity.

METHODS

We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI).

RESULTS

From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5).

CONCLUSIONS

The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.

Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence

PURPOSE

Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries.

METHOD

Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries.

RESULTS

From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified.

CONCLUSIONS

This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.

Fact or Fiction? Practice on a Simulator is Not Required to Pass the Fundamentals of Endoscopic Surgery (FES)TM Skills Exam

PURPOSE

Initial work on the validity evidence used to support the Fundamentals of Endoscopic Surgery (FES) performance exam as a measure of technical competency showed a strong relationship to clinical experience. Despite this evidence, there is a perception among some program directors that the exam cannot be successfully passed without practice on a simulator. We assess the validity of this perception.

METHODS

Deidentified data from the initial FES skills examination (prior to the 2014 FEC requirement) was reviewed, and 335 unique participants with reported simulation experience demographics were identified. Self reported data analyzed included gender, total clinical endoscopy procedure experience (1-150, 151-300, >300), and endoscopy simulator training hours (0, 1-5, 6-10, 11-20, >20). Final FES skills exam scores, and pass/fail designations for each participant were reported by the FES program staff. Continuous variables were compared between groups using one-way analysis of variance with post-hoc analysis where appropriate. Categorical variables were compared using Pearson Chi-Squared. The effect of variables on pass rate was assessed using univariate and multivariate logistic regression.

RESULTS

Simulation training experience (SE) was categorically reported in hours(n,%): 0 (98, 29%), 1-5 (135, 40%), 6-10 (52, 16%), 11-20 (24, 7%), and >20 (26, 8%). Clinical endoscopy experience (CE), reported categorically as total cases performed (n,%), was available for 323 of 355 identified participants: 1-150 (126, 39%), 151-300 (99, 31%), >300 (98, 30%). There was no statistically discernible differences in mean FES total or task scores across the SE groups (total score 0:72 ± 15, 1-5:72 ± 13, 6-10:71 ± 14, 11-20:71 ± 16, 20:78 ± 13; p = 0.28), while both total score and task scores were discernibly higher in the more experienced CE groups (>151) compared to the least experienced group (total score; <150:67 ± 15, 151-300:75 ± 1, >300:77 ± 14; p < 0.01). Similarly, there was no statistically discernible difference in FES skills exam pass rates between SE groups (0: 80%, 1-5: 82%, 6-10: 79%, 11-20: 75%, >20: 85%; x2 = 2.5, p = 0.6), but there was a strong relationship between clinical experience and pass rate (<150: 70%, 151-300: 87%, >300: 89%; x2 = 15.8, p < 0.001). Finally, on both univariate and multivariate logistic regression, CE remained a discernible predictor of passing, even when controlling for SE (odds ratio = 2, 95% confidence interval 1.4-2.9, p < 0.001).

CONCLUSIONS

FES skills examination data collected on participants completing the examination before the FEC requirement shows no demonstrable relationship with self-reported training experience on a simulator but confirms a strong relationship with clinical endoscopy experience. This lends further evidence to the validity of the FES exam as a marker of clinical endoscopic skill.

The Fundamentals of Endoscopic Surgery (FES) skills test: factors associated with first-attempt scores and pass rate

BACKGROUND

The Fundamentals of Endoscopic Surgery (FES) program became required for American Board of Surgery certification as part of the Flexible Endoscopy Curriculum (FEC) for residents graduating in 2018. This study expands prior psychometric investigation of the FES skills test.

METHODS

We analyzed de-identified first-attempt skills test scores and self-reported demographic characteristics of 2023 general surgery residents who were required to pass FES.

RESULTS

The overall pass rate was 83%. "Loop Reduction" was the most difficult sub-task. Subtasks related to one another only modestly (Spearman's ρ ranging from 0.11 to 0.42; coefficient α = .55). Both upper and lower endoscopic procedural experience had modest positive association with scores (ρ = 0.14 and 0.15) and passing. Examinees who tested on the GI Mentor Express simulator had lower total scores and a lower pass rate than those tested on the GI Mentor II (pass rates = 73% vs. 85%). Removing an Express-specific scoring rule that had been applied eliminated these differences. Gender, glove size, and height were closely related. Women scored lower than men (408- vs. 489-point averages) and had a lower first-attempt pass rate (71% vs. 92%). Glove size correlated positively with score (ρ = 0.31) and pass rate. Finally, height correlated positively with score (r = 0.27) and pass rate. Statistically controlling for glove size and height did not eliminate gender differences, with men still having 3.2 times greater odds of passing.

CONCLUSIONS

FES skills test scores show both consistencies with the assessment's validity argument and several remarkable findings. Subtasks reflect distinct skills, so passing standards should perhaps be set for each subtask. The Express simulator-specific scoring penalty should be removed. Differences seen by gender are concerning. We argue those differences do not reflect measurement bias, but rather highlight equity concerns in surgical technology, training, and practice.

ASGE EndoVators Summit: simulators and the future of endoscopic training

Interest in the use of simulation for acquiring, maintaining, and assessing skills in GI endoscopy has grown over the past decade, as evidenced by recent American Society for Gastrointestinal Endoscopy (ASGE) guidelines encouraging the use of endoscopy simulation training and its incorporation into training standards by a key accreditation organization. An EndoVators Summit, partially supported by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) of the National Institutes of Health, (NIH) was held at the ASGE Institute for Training and Technology from November 19 to 20, 2017. The summit brought together over 70 thought leaders in simulation research and simulator development and key decision makers from industry. Proceedings opened with a historical review of the role of simulation in medicine and an outline of priority areas related to the emerging role of simulation training within medicine broadly. Subsequent sessions addressed the summit's purposes: to review the current state of endoscopy simulation and the role it could play in endoscopic training, to define the role and value of simulators in the future of endoscopic training and to reach consensus regarding priority areas for simulation-related education and research and simulator development. This white paper provides an overview of the central points raised by presenters, synthesizes the discussions on the key issues under consideration, and outlines actionable items and/or areas of consensus reached by summit participants and society leadership pertinent to each session. The goal was to provide a working roadmap for the developers of simulators, the investigators who strive to define the optimal use of endoscopy-related simulation and assess its impact on educational outcomes and health care quality, and the educators who seek to enhance integration of simulation into training and practice.

Development of a fundamentals of endoscopic surgery proficiency-based skills curriculum for general surgery residents

BACKGROUND

Fundamentals of Endoscopic Surgery (FES) has become a board certification requirement for general surgery residents. While the FES program provides a robust didactic curriculum, an endoscopic skills curriculum is lacking for this high stakes assessment. The aims of this study were to develop a proficiency-based endoscopic skills curriculum and assess its effectiveness on success in the FES exam.

METHODS

Endoscopy experts developed a multiple-choice questionnaire based on the FES online didactics. Five training cases from the GI Mentor II simulator were selected, and expert performance defined proficiency levels for each case. Participating surgery residents were required to review online didactics and achieve proficiency twice on selected simulator cases. The multiple-choice questionnaire, simulator-generated metrics of two endoscopy cases, Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), NASA-Task Load Index (TLX), and the manual portion of the FES exam were used for assessment before and after training. The curriculum was implemented either alongside a clinical endoscopy rotation or independent of the rotation. Clinical endoscopic skills of participants with a dedicated rotation were assessed using GAGES.

RESULTS

Twenty-eight general surgery residents (PGY 2-5) participated in the study, of which 25 (89%) completed the curriculum. Scores of the multiple-choice questionnaire and all simulator-generated metrics improved in the post-training assessment, with the exception of the percentage of mucosal surface examined, which was slightly decreased. Simulated and clinical GAGES scores and the NASA-TLX score improved after training. Performance scores on four of five FES exam tasks were significantly improved.

CONCLUSIONS

The proficiency-based endoscopic skills curriculum was successfully implemented both alongside the clinical endoscopy rotation and independent of the rotation. Participating residents acquired skills to pass the FES exam. This curriculum will be valuable to general surgery residency programs.

The FES Test: Are We Ready?

PURPOSE

The FES hands-on skills test is administered using a $100,000 computer-based simulator. Few of our trainees have practiced on this device. Our aim was to evaluate our GS residents' baseline endoscopic skills and eventually develop a simulation-based endoscopy curriculum and clarify performance-based assessment criteria.

METHODS

General surgery residents' colonoscopy skills were assessed using a computer-based endoscopy simulator (CBES) during their biannual simulation-based OSCE-type assessments. Trainees were asked to reach the ileum in <5 minutes with minimal patient pain and complications. Module 1 (easy) was assigned to PGY 1-4 residents and module 5 (hard) to both PGY 4s and 5s. The colonoscope insertion length, % time with no pain, % time in "red out", and complications were recorded. Performance grading criteria were driven by literature review and expert opinion. Residents were assessed in the fall 2017; they were then given scoring criteria, a step-by-step instruction manual, and a voluntary hands-on session with the CBES. Residents repeated the same assessment in the spring 2018.

RESULTS

30 PGY-1s, 12 PGY-2s, 8 PGY-3s, 9 PGY-4s and 7 PGY-5s GS residents participated in the fall colonoscopy assessment. In module 1, 66% of PGY-4s, 50% of PGY-3s, 8% of PGY-2s and 0% of the PGY-1s intubated the ileum (p<0.05). In module 5, 30% of PGY 5 and 22% of PGY 4 residents completed the task (p<0.05). 15 PGY-1s, 5 PGY-2s, 1 PGY-3, 2 PGY-4s, and 1 PGY-5 participated in the voluntary hands-on session. All residents completing the fall assessment undertook the same task in the spring. In module 1, 89% of PGY-4s, 100% of the PGY-3s, 75% of PGY-2s and 70% of the PGY-1s completed the task. In module 5, 30% of PGY 5 and 34 % of PGY 4 residents completed the task. Residents who participated in the voluntary hands-on session (n= 24, 96% task completion) outperformed residents (n= 42, 64% task completion) that did not participate (p<0.05).

CONCLUSIONS

Most of our GS residents could not initially intubate the ileum using the CBES. Prior experience with the CBES was the only factor strongly correlated with successful task completion. A voluntary hands-on teaching session allowed 96% of participating trainees to subsequently achieve CBES task completion. Developing a formal simulation-based curriculum suggests we can better prepare surgical trainees for the FES exam.

Computerized feedback during colonoscopy training leads to improved performance: a randomized trial

BACKGROUND AND AIMS

Simulation-based training in colonoscopy is increasingly replacing the traditional apprenticeship method to avoid patient-related risk. Mentoring during simulation is necessary to provide feedback and to motivate, but expert supervisors are a scarce resource. We aimed to determine whether computerized feedback in simulated colonoscopy would improve performance, optimize time spent practicing, and optimize the pattern of training.

METHODS

Forty-four participants were recruited and randomized to either a feedback group (FG) or a control group (CG). Participants were allowed 2 hours of self-practice during which they could practice as they saw fit on 2 different cases: 1 easy and 1 difficult. The CG practiced without feedback, but the participants in the FG were given a score of progression every time they reached the cecum. All participants were tested on a different case after end of training. The primary outcome was the progression score in the final case, and secondary outcomes were time spent practicing and the training pattern.

RESULTS

Regression analysis adjusting for sex was done because of an uneven sex distribution between groups (P = .026) and significantly higher performance scores by men (37.6, standard deviation [SD] 25.9) compared with women (19.7, SD 18.7); P = .012. The FG outperformed the CG in the final case, FG scoring 14.4 points (95% confidence interval [CI], 1.2-27.6) more than the CG; P = .033, and they spent more time practicing, FG practicing 25.8 minutes (95% CI, 11.6-39.9) more than the CG; P = .001. The FG practiced more on the easy case and reached the cecum 3.2 times more (95% CI, 2-4.5) during practice (P < .001).

CONCLUSIONS

Our findings of this study revealed that an automatic, computerized score of progression during simulated colonoscopy motivates the novices to improve performance, optimizes time spent practicing, and optimizes their pattern of training. (Clinical trial registration number: NCT03248453.).

The SAGES flexible endoscopy course for fellows: a worthwhile investment in furthering surgical endoscopy

BACKGROUND

The SAGES flexible endoscopy course for minimally invasive surgery (MIS) fellows improves confidence and skills in performing gastrointestinal (GI) endoscopy. This study evaluated the long-term retention of these confidence levels and investigated how fellows changed practices within their fellowships due to the course.

METHODS

Participating MIS fellows completed surveys 6 months after the course. Respondents rated their confidence to independently perform 16 endoscopic procedures (1 = not at all; 5 = very), barriers to use of endoscopy, and current uses of endoscopy. Respondents also noted participation in additional skills courses and status of fundamentals of endoscopic surgery (FES) certification. Comparisons of responses from the immediate post-course survey to the 6-month follow-up survey were examined. McNemar and paired t tests were used for analyses.

RESULTS

23 of 57 (40%) course participants returned to the 6-month survey. No major barriers to endoscopy use were identified. Fellows reported less competition with GI providers as a barrier to practice compared to their original post-course expectations (50% vs. 86%, p < 0.01). In addition, confidence was maintained in performing the majority of the 16 endoscopic procedures, although fellows reported significant decreases in confidence in independently performing snare polypectomy (- 26%; p < 0.05), control of variceal bleeding (- 39%; p < 0.05), colonic stenting (- 48%; p < 0.01), BARRX (- 40%; p < 0.05), and TIF (- 31%; p < 0.05). Fewer fellows used the GI suite to manage surgical problems than was anticipated post course (26% vs. 74%, p < 0.01). Fellows who passed FES noted no significant loss of independence, changes in use, or barriers to use. 18% made additional partnerships with industry after the course. 41% stated flexible endoscopy has influenced their post-fellowship job choice.

CONCLUSIONS

The SAGES flexible endoscopy course for MIS fellows results in long-term practice changes with participating fellows maintaining confidence to perform the majority of taught endoscopic procedures 6 months later. Additionally, fellows experienced no major barriers to implementing endoscopy into practice.

Methods for certification in colonoscopy - a systematic review

INTRODUCTION

Reliable, valid, and feasible assessment tools are essential to ensure competence in colonoscopy. This study aims to provide an overview of the existing assessment methods and the validity evidence that supports them.

METHODS

A systematic search was conducted in October 2016. Pubmed, EMBASE, and PsycINFO were searched for studies evaluating assessment methods to ensure competency in colonoscopy. Outcome variables were described and evidence of validity was explored using a contemporary framework.

RESULTS

Twenty-five observational studies were included in the systematic review. Most studies were based on small sample sizes. The studies were categorized after outcome measures into five groups: Clinical process related outcome metrics (n = 2), direct observational colonoscopy assessment (n = 8), simulator based metrics (n = 11), automatic computerized metrics (n = 2), and self-assessment (n = 1). Validity score varied among the studies and only five studies presented sufficient evidence to recommend the tool for clinical assessment.

CONCLUSIONS

The objectives vary throughout the presented tools. Some tools are global tools where others focus on procedural technical skill assessment or even part-task skills. There is a tendency in the most recent studies towards more specific assessment of technical skills. The majority of assessment methods lack sufficient validity evidence.

Development of a train-to-proficiency curriculum for the technical skills component of the fundamentals of endoscopic surgery exam

BACKGROUND

The demonstration of competency in endoscopy is required prior to obtaining American Board of Surgery Certification. To demonstrate competency, the resident must pass a national high-stakes cognitive test and a technical skills exam on a virtual reality simulator. The purpose of this preliminary study was to design a proficiency-based endoscopy simulation curriculum to meet this competency requirement.

METHODS

This is a mixed methods prospective cohort study at a single academic medical institution. Prior to taking the national exam, surgery residents were required to participate in a skills lab and demonstrate proficiency on 10 simulation tasks. Proficiency was based on time and percent of objects targeted/mucosa seen. Simulation practice time, number of task repetitions to proficiency, and prior endoscopic experience were recorded. Resident's self-reported confidence scores in endoscopic skills prior to and following simulation lab training were obtained.

RESULTS

From January 1, 2016 through August 1, 2017, 20 surgical residents (8 PGY2, 8 PGY3, 4 PGY4) completed both a faculty-supervised endoscopy skills lab and independent learning with train-to-proficiency simulation tasks. Median overall simulator time per resident was 306 min (IQR: 247-405 min). Median overall time to proficiency in all tasks was 235 min (IQR: 208-283 min). The median time to proficiency decreased with increasing PGY status (r = 0.4, P = 0.05). There was no correlation between prior real-time endoscopic experience and time to proficiency. Reported confidence in endoscopic skills increased significantly from mean of 5.75 prior to 7.30 following the faculty-supervised endoscopy skills lab (P = 0.0002). All 20 residents passed the national exam.

CONCLUSIONS

In this preliminary study, a train-to-proficiency curriculum in endoscopy improved surgical resident's confidence in their endoscopic skills and 100% of residents passed the FES technical skills test on their first attempt. Our findings also indicate that uniform proficiency was not achieved by real-time experience alone.

A proficiency-based virtual reality endoscopy curriculum improves performance on the fundamentals of endoscopic surgery examination

INTRODUCTION

The fundamentals of endoscopic surgery (FES) examination is a national test of knowledge and skill in flexible gastrointestinal endoscopy. The skill portion of the examination involves five tasks that assesses the following skills: scope navigation, loop reduction, mucosal inspection, retroflexion, and targeting. This project aimed to assess the efficacy of a proficiency-based virtual reality (VR) curriculum in preparing residents for the FES skills exam.

METHODS

Experienced (>100 career colonoscopies) and inexperienced endoscopists (<50 career colonoscopies) were recruited to participate. Six VR modules were identified as reflecting the skills tested in the exam. All participants were asked to perform each of the selected modules twice, and median performance was compared between the two groups. Inexperienced endoscopists were subsequently randomized in matched pairs into a repetition (10 repetitions of each task) or proficiency curriculum. After completion of the respective curriculum, FES scores and pass rates were compared to national data and historical institutional control data (endoscopy-rotation training alone).

RESULTS

Five experienced endoscopists and twenty-three inexperienced endoscopists participated. Construct valid metrics were identified for six modules and proficiency benchmarks were set at the median performance of experienced endoscopists. FES scores of inexperienced endoscopists in the proficiency group had significantly higher FES scores (530 ± 86) versus historical control (386.7 ± 92.2, p = 0.0003) and higher pass rate (proficiency: 100%, historical control 61.5%, p = 0.01).

CONCLUSION

Trainee engagement in a VR curriculum yields superior FES performance compared to an endoscopy rotation alone. Compared to the 2012-2016 national resident pass rate of 80, 100% of trainees in a proficiency-based curriculum passed the FES manual skills examination.

In-training gastrointestinal endoscopy competency assessment tools: Types of tools, validation and impact

The ability to perform endoscopy procedures safely, effectively and efficiently is a core element of gastroenterology practice. Training programs strive to ensure learners demonstrate sufficient competence to deliver high quality endoscopic care independently at completion of training. In-training assessments are an essential component of gastrointestinal endoscopy education, required to support training and optimize learner's capabilities. There are several approaches to in-training endoscopy assessment from direct observation of procedural skills to monitoring of surrogate measures of endoscopy skills such as procedural volume and quality metrics. This review outlines the current state of evidence as it pertains to in-training assessment of competency in performing gastrointestinal endoscopy as part of an overall endoscopy quality and skills training program.

Training and Assessment in Pediatric Endoscopy

A key aspect of pediatric gastroenterology practice is the ability to perform endoscopy procedures safely, effectively, and efficiently. Similar to adult endoscopy, performance of pediatric endoscopy requires the acquisition of related technical, cognitive, and integrative competencies to effectively diagnose and manage gastrointestinal disorders in children. However, the distinctive requirements of pediatric patients and their families and the differential spectrum of disease highlight the need for a pediatric-specific training curriculum and assessment framework to ensure endoscopic procedures are performed safely and successfully in children. This review outlines the current state of evidence as it pertains to pediatric endoscopy training and assessment.

Endoscopy services and training: a national survey of general surgeons

BACKGROUND

Delivering high-quality endoscopy services depends largely on the competence of endoscopists. General surgery residency training in endoscopy and the associated quality of endoscopy services being delivered by general surgeons have been the subject of considerable controversy. In conjunction with the Canadian Association of General Surgeons (CAGS) executive board, we formulated a survey to evaluate the general state of endoscopy practice and training among general surgeons in Canada.

METHODS

The study was designed as a cross-sectional survey. General surgeons who are members of CAGS were selected to participate in the study and were emailed a link to the online questionnaire regarding the importance of endoscopy. They were asked to compare their training to resident training today.

RESULTS

Sixty-nine surveys were completed. The majority of general surgeons (95.7%) indicated that endoscopy was an important skill to possess, and more than 85.5% used endoscopy in their own practices. However, nearly half (46.4%) felt that general surgery endoscopy training in Canada is currently inadequate to produce competent endoscopists. The main qualitative themes emerging from the survey were the inadequacy of current postgraduate endoscopy training (37.5%) and the absence of standardization in training (25.0%).

CONCLUSION

Endoscopy is considered integral to academic and community general surgeons' practices; however, the adequacy of training seems to be questioned. Postgraduate training in endoscopy needs to be formalized and standardized, with a greater emphasis placed on teaching endoscopy.

Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test

BACKGROUND

The fundamentals of endoscopic surgery (FES) examination measures the knowledge and skills required to perform safe flexible endoscopy. A potential limitation of the FES skills test is the size and cost of the simulator on which it was developed (GI Mentor II virtual reality endoscopy simulator; Simbionix LTD, Israel). A more compact and lower-cost alternative (GI Mentor Express) was developed to address this issue. The purpose of this study was to obtain evidence for the validity of scores obtained on the Express platform, so that it can be used for testing.

STUDY DESIGN

General surgery residents at various levels of training and practicing endoscopists at five institutions participated. Each completed the five FES tasks on both simulator platforms in random order, with 3-14 days between tests. Scores were calculated using the same standardized computer-generated algorithm and compared using Pearson's correlation coefficient.

RESULTS

There were 58 participants (mean age 32; 76% male) with a broad range of endoscopic experience. The mean (95% confidence interval) FES scores were 72 (67:77) on the GI Mentor II and 66 (60:71) on the Express. The correlation between scores on the two platforms was 0.86 (0.77:0.91; p < 0.0001).

CONCLUSION

There is a high correlation between FES manual skills scores measured on the original platform and the new Express, providing evidence to support the use of the GI Mentor Express for FES testing.