Gastrointestinal Endoscopy Competency Assessment Tool: development of a procedure-specific assessment tool for colonoscopy

Endoscopic Education, Training, and Simulation

This article summarizes the evolution of endoscopy training from its origins of simple bedside instruction to its current state of competency-based curricula and incorporation of novel training methods, including simulation, gamification for the trainee, and improving education skills of the tutors.

Supporting self-regulated-learning in colonoscopy training-A comparison cohort trial

Aim

This study aimed to demonstrate that using a self-regulated learning (SRL) approach can improve colonoscopy performance skills.

Background

Colonoscopy is the gold standard for detecting colorectal cancer and removing its precursors: polyps. Acquiring proficiency in colonoscopy is challenging, requiring completion of several hundred procedures. SRL seems to be beneficial to help trainees acquire competencies in regulating their future learning processes and enhance the outcomes of current learning situations. SRL is a learner-centred approach that refers to a trainee's ability to understand and control their learning environment, including cognitions, motivations and emotions. The key abilities include self- and situational awareness, task analysis, and strategic planning. This study is the first to use an SRL approach for workplace-based colonoscopy training.

Methods

In this comparison cohort trial, participants used two SRL supports: a self-review of videotaped performance, and an online learning platform with procedural and conceptual knowledge about colonoscopy. In the control cohort, participants performed patient-based colonoscopy as usual in their departments. Improvement was monitored via three video-based ratings (study start, end of the study period, and follow-up) using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Outcomes were analysed using two-way analysis of variance with repeated measurements.

Results

This study recruited 21 participants (12, intervention cohort; nine, control cohort); 58 videos were recorded. The intraclass correlation coefficient was 0.88 (95% CI 0.61-0.98; p < 0.001). The global rating scale (GRS) and checklist (CL) in GiECAT were analysed separately. No statistically significant main effects of cohort (GRS: F(1,16) = 2.84, p = 0.11; CL: F(1,16) = 1.06, p = 0.32), test (GRS: F(2,32) = 2.56, p = 0.09; CL: F(2,32) = 0.76, p = 0.48), or interactions between cohort and test were observed (GRS: F(2,32) = 1.16, p = 0.33; CL: F(2,32) = 1.01, p = 0.37).

Conclusions

SRL in patient-based colonoscopy is feasible; however, no clear effect on performance scores was observed.

Training in basic gastrointestinal endoscopic procedures: a European Society of Gastrointestinal Endoscopy (ESGE) and European Society of Gastroenterology and Endoscopy Nurses and Associates (ESGENA) Position Statement

This ESGE Position Statement provides structured and evidence-based guidance on the essential requirements and processes involved in training in basic gastrointestinal (GI) endoscopic procedures. The document outlines definitions; competencies required, and means to their assessment and maintenance; the structure and requirements of training programs; patient safety and medicolegal issues. 1:  ESGE and ESGENA define basic endoscopic procedures as those procedures that are commonly indicated, generally accessible, and expected to be mastered (technically and cognitively) by the end of any core training program in gastrointestinal endoscopy. 2:  ESGE and ESGENA consider the following as basic endoscopic procedures: diagnostic upper and lower GI endoscopy, as well as a limited range of interventions such as: tissue acquisition via cold biopsy forceps, polypectomy for lesions ≤ 10 mm, hemostasis techniques, enteral feeding tube placement, foreign body retrieval, dilation of simple esophageal strictures, and India ink tattooing of lesion location. 3:  ESGE and ESGENA recommend that training in GI endoscopy should be subject to stringent formal requirements that ensure all ESGE key performance indicators (KPIs) are met. 4:  Training in basic endoscopic procedures is a complex process and includes the development and acquisition of cognitive, technical/motor, and integrative skills. Therefore, ESGE and ESGENA recommend the use of validated tools to track the development of skills and assess competence. 5:  ESGE and ESGENA recommend incorporating a multimodal approach to evaluating competence in basic GI endoscopic procedures, including procedural thresholds and the measurement and documentation of established ESGE KPIs. 7:  ESGE and ESGENA recommend the continuous monitoring of ESGE KPIs during GI endoscopy training to ensure the trainee's maintenance of competence. 9:  ESGE and ESGENA recommend that GI endoscopy training units fulfil the ESGE KPIs for endoscopy units and, furthermore, be capable of providing the dedicated personnel, infrastructure, and sufficient case volume required for successful training within a structured training program. 10:  ESGE and ESGENA recommend that trainers in basic GI endoscopic procedures should be endoscopists with formal educational training in the teaching of endoscopy, which allows them to successfully and safely teach trainees.

Impact of an artificial intelligence-aided endoscopic diagnosis system on improving endoscopy quality for trainees in colonoscopy: Prospective, randomized, multicenter study

OBJECTIVE

This study was performed to evaluate whether the use of CAD EYE (Fujifilm, Tokyo, Japan) for colonoscopy improves colonoscopy quality in gastroenterology trainees.

METHODS

The patients in this multicenter randomized controlled trial were divided into Group A (observation using CAD EYE) and Group B (standard observation). Six trainees performed colonoscopies using a back-to-back method in pairs with gastroenterology experts. The primary end-point was the trainees' adenoma detection rate (ADR), and the secondary end-points were the trainees' adenoma miss rate (AMR) and Assessment of Competency in Endoscopy (ACE) tool scores. Each trainee's learning curve was evaluated using a cumulative sum (CUSUM) control chart.

RESULTS

We analyzed data for 231 patients (Group A, n = 113; Group B, n = 118). The ADR was not significantly different between the two groups. Group A had a significantly lower AMR (25.6% vs. 38.6%, P = 0.033) and number of missed adenomas per patient (0.5 vs. 0.9, P = 0.004) than Group B. Group A also had significantly higher ACE tool scores for pathology identification (2.26 vs. 2.07, P = 0.030) and interpretation and identification of pathology location (2.18 vs. 2.00, P = 0.038). For the CUSUM learning curve, Group A showed a trend toward a lower number of cases of missed multiple adenomas by the six trainees.

CONCLUSION

CAD EYE did not improve ADR but decreased the AMR and improved the ability to accurately locate and identify colorectal adenomas. CAD EYE can be assumed to be beneficial for improving colonoscopy quality in gastroenterology trainees.

TRIAL REGISTRATION

University Hospital Medical Information Network Clinical Trials Registry (UMIN000044031).

Promoting Research that Supports High-Quality Gastrointestinal Endoscopy in Children

PURPOSE OF REVIEW

Defining and measuring the quality of endoscopic care is a key component of performing gastrointestinal endoscopy in children. The purpose of this review is to discuss quality metrics for pediatric gastrointestinal endoscopy and identify where additional research is needed.

RECENT FINDINGS

Pediatric-specific standards and indicators were recently defined by the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) working group through a rigorous guideline consensus process. Although the aim of these guidelines is to facilitate best practices for safe and high-quality gastrointestinal endoscopy in children, they highlight the pressing need to expand upon the body of evidence supporting these standards and indicators as predictors of clinically relevant outcomes. In this review, we propose and discuss ideas for several high-yield research topics to engage pediatric endoscopists and promote best practices in pediatric endoscopy.

Validity evidence for observational ERCP competency assessment tools: a systematic review

BACKGROUND : Assessment of competence in endoscopic retrograde cholangiopancreatography (ERCP) is critical for supporting learning and documenting attainment of skill. Validity evidence supporting ERCP observational assessment tools has not been systematically evaluated. METHODS : We conducted a systematic search using electronic databases and hand-searching from inception until August 2021 for studies evaluating observational assessment tools of ERCP performance. We used a unified validity framework to characterize validity evidence from five sources: content, response process, internal structure, relations to other variables, and consequences. Each domain was assigned a score of 0-3 (maximum score 15). We assessed educational utility and methodological quality using the Accreditation Council for Graduate Medical Education framework and the Medical Education Research Quality Instrument, respectively. RESULTS : From 2769 records, we included 17 studies evaluating 7 assessment tools. Five tools were studied for clinical ERCP, one for simulated ERCP, and one for simulated and clinical ERCP. Validity evidence scores ranged from 2 to 12. The Bethesda ERCP Skills Assessment Tool (BESAT), ERCP Direct Observation of Procedural Skills Tool (ERCP DOPS), and The Endoscopic Ultrasound (EUS) and ERCP Skills Assessment Tool (TEESAT) had the strongest validity evidence, with scores of 10, 12, and 11, respectively. Regarding educational utility, most tools were easy to use and interpret, and required minimal additional resources. Overall methodological quality (maximum score 13.5) was strong, with scores ranging from 10 to 12.5. CONCLUSIONS : The BESAT, ERCP DOPS, and TEESAT had strong validity evidence compared with other assessments. Integrating tools into training may help drive learners' development and support competency decision making.

Methods for Gastrointestinal Endoscopy Quantification: A Focus on Hands and Fingers Kinematics

Gastrointestinal endoscopy is a complex procedure requiring the mastery of several competencies and skills. This procedure is in increasing demand, but there exist important management and ethical issues regarding the training of new endoscopists. Nowadays, this requires the direct involvement of real patients and a high chance of the endoscopists themselves suffering from musculoskeletal conditions. Colonoscopy quantification can be useful for improving these two issues. This paper reviews the literature regarding efforts to quantify gastrointestinal procedures and focuses on the capture of hand and finger kinematics. Current technologies to support the capture of data from hand and finger movements are analyzed and tested, considering smart gloves and vision-based solutions. Manus VR Prime II and Stretch Sense MoCap reveal the main problems with smart gloves related to the adaptation of the gloves to different hand sizes and comfortability. Regarding vision-based solutions, Vero Vicon cameras show the main problem in gastrointestinal procedure scenarios: occlusion. In both cases, calibration and data interoperability are also key issues that limit possible applications. In conclusion, new advances are needed to quantify hand and finger kinematics in an appropriate way to support further developments.

Colonoscopy competence assessment tools: a systematic review of validity evidence

BACKGROUND

Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity.

METHODS

We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI).

RESULTS

From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5).

CONCLUSIONS

The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.

Impact of a Virtual Endoscopy Training Curriculum in Novice Endoscopists: First Experience in Argentina

BACKGROUND

Virtual reality simulation in gastrointestinal endoscopy is an educational tool that allows repetitive instruction in a non-patient care environment.

AIM

To determine the impact of a virtual endoscopy training curriculum applying an objective pre- and post-training analysis on trainee endoscopists.

METHODS

A before-after training study was carried out. Subjects were first year fellows of gastroenterology, who completed a questionnaire and then performed two pre-training simulated cases. The virtual endoscopy training curriculum consisted of an 8-h workday utilizing two GI MENTOR™ in a specialized clinical simulation center. After the training, all subjects completed the same two cases they did in the pre-training. Pre- and post-training results' comparisons were made by paired t test.

RESULTS

Totally, 126 subjects were included (mean age 30 years, 61% female). A significant improvement from pre- to post-training was observed in psychomotor skills (total time, percentage, and number of balloons exploded) and endoscopic skills (cecal intubation time, percentage of examined mucosa, and efficacy of screening). There was also an improvement in the quality of the endoscopic study; percentage of examined mucosa over 85% showed a significant improvement post-training with an adjusted OR of 2.72 (95% CI 1.51-4.89, p = 0.001).

CONCLUSIONS

Virtual endoscopy training curriculum produces a significant improvement in the trainee endoscopists performance and their psychomotor skills and introduces the concept of a quality endoscopic study in a non-patient, risk-free environment.

Development of the Diabetic Wound Assessment Learning Tool (DiWALT) and validity evidence

OBJECTIVE

Diabetic foot wounds account for up to one-third of diabetes-related health care expenditure and are the greatest cause of extremity amputation in Canada. Physicians encounter patients with such wounds in all specialties, particularly as generalists in medical wards and emergency departments. However, there is a dearth of literature on the optimal way to teach and to assess the management of these patients. Given the importance of assessment for learning in the shift toward competency-based medical education, we aimed to develop an assessment tool and to build validity evidence for its use in this context.

METHODS

A consensus process involving nine Canadian experts in diabetic wound management was used to develop the Diabetic Wound Assessment Learning Tool (DiWALT) items and two 10-minute simulation-based testing scenarios. The simulators used were modified from commercially available models to serve the testing scenarios. Validity evidence for the DiWALT was subsequently evaluated by assessing 24 physician participants' performance during the two scenarios. All participants were novices (<50 cases managed). Two assessors independently rated participants using the DiWALT. Evidence was organized using Kane's validity framework and included Cronbach α for interitem consistency as well as test-retest and inter-rater reliability using the intra-class correlation coefficient (ICC).

RESULTS

Cronbach α was 0.92, implying high internal consistency. Test-retest reliability was also excellent with ICC of 0.89 (confidence interval [CI], 0.76-0.95) for single measures and ICC of 0.94 (CI, 0.86-0.98) for average measures. Inter-rater reliability was fair for single measures with ICC of 0.68 (CI, 0.65-0.71) and good for average measures with ICC of 0.81 (CI, 0.79-0.83).

CONCLUSIONS

These results demonstrate that the DiWALT consistently and reliably evaluates competence in diabetic wound management during simulated cases using a small, homogeneous sample of physicians. Further work is necessary to quantify sources of error in the assessment scores, to establish validity evidence when it is used to assess larger and more heterogeneous participants, and to identify how well the DiWALT differentiates between different experience levels.

Colorectal cancer prevention by a CLEAR principles-based colonoscopy protocol: an observational study

BACKGROUND AND AIMS

Colorectal cancer (CRC) prevention by colonoscopy has been lower than expected. We studied CRC prevention outcomes of a colonoscopy protocol based on Clean the colon, Look Everywhere, and complete Abnormality Removal (CLEAR) principles.

METHODS

This observational follow-up study studied patients provided screening colonoscopy at a free-standing private ambulatory surgery center in South Carolina by 80 endoscopists from October 2001 to December 2014, followed through December 2015. The colonoscopy protocol, optimized for polyp clearance, featured in-person bowel preparation instructions reinforced by phone, polyp search and removal throughout insertion and gradual withdrawal with circumferential tip movements, and a team approach using all personnel present to maximize polyp detection, patient safety, and clear-margin polypectomy including requesting repeat inspection or additional tissue removal. Outcome measures were postscreening lifetime CRC risk relative to Surveillance Epidemiology and End Results (SEER)-18 and interval cancer rate (postcolonoscopy CRCs among cancer-free patients at screening).

RESULTS

Of 25,862 patients (mean age, 58.1 years; 52% black; 205,522 person-years of observation), 159 had CRC at screening and 67 patients developed interval CRC. The interval CRC rate was 3.34 per 10,000 person-years of observation, 5.79 and 2.24 among patients with and without adenomas, respectively. The rate was similar among older patients (mean age 68.5 years at screening) and with prolonged follow-up. Postscreening lifetime CRC risk was 1.6% (bootstrap 95% confidence interval, 1.3%-1.8%) versus 4.7% in SEER-18, 67% lower. Subgroups with mean screening ages of 50 and 68.5 years showed risk reductions of 80% and 72%, respectively. The adverse event rate was less than usually reported rates: perforation 2.6 per 10,000, bleeding with hospitalization 2.4 per 10,000, and no deaths.

CONCLUSIONS

A colonoscopy protocol optimized for polyp clearance prevented 67% of CRC compared with a SEER-18 population given ongoing population screening.

Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study

BACKGROUND

Colonoscopy performance varies among endoscopists, impairing the discovery of colorectal cancers and precursor lesions. We aimed to construct a real-time quality improvement system (ENDOANGEL) to monitor real-time withdrawal speed and colonoscopy withdrawal time and to remind endoscopists of blind spots caused by endoscope slipping. We also aimed to evaluate the effectiveness of this system for improving adenoma yield of everyday colonoscopy.

METHODS

The ENDOANGEL system was developed using deep neural networks and perceptual hash algorithms. We recruited consecutive patients aged 18-75 years from Renmin Hospital of Wuhan University in China who provided written informed consent. We randomly assigned patients (1:1) using computer-generated random numbers and block randomisation (block size of four) to either colonoscopy with the ENDOANGEL system or unassisted colonoscopy (control). Endoscopists were not masked to the random assignment but analysts and patients were unaware of random assignments. The primary endpoint was the adenoma detection rate (ADR), which is the proportion of patients having one or more adenomas detected at colonoscopy. The primary analysis was done per protocol (ie, in all patients having colonoscopy done in accordance with the assigned intervention) and by intention to treat (ie, in all randomised patients). This trial is registered with http://www.chictr.org.cn, ChiCTR1900021984.

FINDINGS

Between June 18, 2019, and Sept 6, 2019, 704 patients were randomly allocated colonoscopy with the ENDOANGEL system (n=355) or unassisted (control) colonoscopy (n=349). In the intention-to-treat population, ADR was significantly greater in the ENDOANGEL group than in the control group, with 58 (16%) of 355 patients allocated ENDOANGEL-assisted colonoscopy having one or more adenomas detected, compared with 27 (8%) of 349 allocated control colonoscopy (odds ratio [OR] 2·30, 95% CI 1·40-3·77; p=0·0010). In the per-protocol analysis, findings were similar, with 54 (17%) of 324 patients assigned ENDOANGEL-assisted colonoscopy and 26 (8%) of 318 patients assigned control colonoscopy having one or more adenomas detected (OR 2·18, 95% CI 1·31-3·62; p=0·0026). No adverse events were reported.

INTERPRETATION

The ENDOANGEL system significantly improved the adenoma yield during colonoscopy and seems to be effective and safe for use during routine colonoscopy.

FUNDING

Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Hubei Province Major Science and Technology Innovation Project, and the National Natural Science Foundation of China.

Gastrointestinal endoscopy in early diagnosis and treatment of gastrointestinal tumors

Objective

To explore the value of gastrointestinal endoscopy in the early diagnosis and treatment of gastrointestinal tumors and lay a foundation for the diagnosis and treatment of gastrointestinal tumors.

Methods

One hundred and eight patients with gastrointestinal tumors who were admitted to our hospital from August 2016 to April 2018 were retrospectively analyzed and divided into observation group and control group according to different diagnostic methods, 54 cases in each group. The control group was treated with traditional endoscopy (white light imaging) and traditional surgery, while the observation group underwent narrow band imaging (NBI) based on endoscopic examination and endoscopic mucosal resection. The image quality scores (morphological image, gastric pit image and capillary image), diagnostic accuracy, surgery related clinical indicators (operation time, intraoperative bleeding volume, hospitalization days) and complications were observed and compared between the two groups.

Results

The morphological image, gastric pit image and capillary image scores of the observation group were higher than those of the control group (P<0.05). The diagnostic accuracy rate of the observation group was 96.30%, which was significantly higher than 75.93% (P<0.05). The operation time and hospitalization days of the observation group were shorter than those of the control group, and the intraoperative bleeding volume of the observation group was less than that of the control group; the differences were statistically significant (P<0.05). The incidence of complications of the observation group was lower than that of the control group, and the difference was statistically significant (P<0.05).

Conclusion

Gastrointestinal endoscopy can accurately identify the pathological changes of tumors in the early diagnosis and treatment of gastrointestinal tumors, improve the diagnostic accuracy rate, and guide the implementation of treatment measures to improve clinical indicators. Moreover the incidence of postoperative complications is low. It is worth clinical promotion.

ASGE EndoVators Summit: simulators and the future of endoscopic training

Interest in the use of simulation for acquiring, maintaining, and assessing skills in GI endoscopy has grown over the past decade, as evidenced by recent American Society for Gastrointestinal Endoscopy (ASGE) guidelines encouraging the use of endoscopy simulation training and its incorporation into training standards by a key accreditation organization. An EndoVators Summit, partially supported by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) of the National Institutes of Health, (NIH) was held at the ASGE Institute for Training and Technology from November 19 to 20, 2017. The summit brought together over 70 thought leaders in simulation research and simulator development and key decision makers from industry. Proceedings opened with a historical review of the role of simulation in medicine and an outline of priority areas related to the emerging role of simulation training within medicine broadly. Subsequent sessions addressed the summit's purposes: to review the current state of endoscopy simulation and the role it could play in endoscopic training, to define the role and value of simulators in the future of endoscopic training and to reach consensus regarding priority areas for simulation-related education and research and simulator development. This white paper provides an overview of the central points raised by presenters, synthesizes the discussions on the key issues under consideration, and outlines actionable items and/or areas of consensus reached by summit participants and society leadership pertinent to each session. The goal was to provide a working roadmap for the developers of simulators, the investigators who strive to define the optimal use of endoscopy-related simulation and assess its impact on educational outcomes and health care quality, and the educators who seek to enhance integration of simulation into training and practice.

Simulation in endoscopy: Practical educational strategies to improve learning

In gastrointestinal endoscopy, simulation-based training can help endoscopists acquire new skills and accelerate the learning curve. Simulation creates an ideal environment for trainees, where they can practice specific skills, perform cases at their own pace, and make mistakes with no risk to patients. Educators also benefit from the use of simulators, as they can structure training according to learner needs and focus solely on the trainee. Not all simulation-based training, however, is effective. To maximize benefits from this instructional modality, educators must be conscious of learners' needs, the potential benefits of training, and associated costs. Simulation should be integrated into training in a manner that is grounded in educational theory and empirical data. In this review, we focus on four best practices in simulation-based education: deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational strategies. For each topic, we provide definitions, supporting evidence, and practical tips for implementation.

Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation

BACKGROUND

Competency-based training has gained ground in surgical training and with it assessment tools to ensure that training objectives are met. Very few assessment tools are available for evaluating performance in thoracoscopic procedures. Video recordings would provide the possibility of blinded assessment and limited rater bias. This study aimed to provide validity evidence for a newly developed and dedicated tool for assessing competency in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS

Participants with varying experience with VATS lobectomy were included from different countries. Video recordings from participants' performance of a VATS right upper lobe lobectomy on a virtual reality simulator were rated by three raters using a modified version of a newly developed VATS lobectomy assessment tool (the VATSAT) and analyzed in relation to the unitary framework (content, response process, internal structure, relation to other variables, and consequences of testing).

RESULTS

Fifty-three participants performed two consecutive simulated VATS lobectomies on the virtual reality simulator, leaving a total of 106 videos. Content established in previously published studies. Response process Standardized data collection was ensured by using an instructional element, uniform data collection, a special rating program, and automatic generation of the results to a database. Raters were carefully instructed in using the VATSAT, and tryout ratings were carried out. Internal structure Inter-rater reliability was calculated as intra-class correlation coefficients, to 0.91 for average measures (p < 0.001). Test/re-test reliability was calculated as Pearson's r of 0.70 (p < 0.001). G-coefficient was calculated to be 0.79 with two procedures and three raters. By performing D-theory was found that either three procedures rated by two raters or five procedures rated by one rater were enough to reach an acceptable G-coefficient of ≥ 0.8. Relation to other variables Significant differences between groups were found (p < 0.001). The participants' VATS lobectomy experience correlated significantly to their VATSAT score (p = 0.016). Consequences of testing The pass/fail score was found to be 14.9 points by the contrasting groups' method, leaving five false positive (29%) and six false negatives (43%).

CONCLUSION

Validity evidence was provided for the VATSAT according to the unitary framework. The VATSAT provides supervisors and assessors with a procedure-specific assessment tool for evaluating VATS lobectomy performance and aids with the decision of when the trainee is ready for unsupervised performance.

Non-technical skills and gastrointestinal endoscopy: a review of the literature

BACKGROUND

Non-technical skills (NTS) have gained increasing recognition in recent years for their role in safe, effective team performance in healthcare. Gastrointestinal endoscopy is a procedure-based specialty with rapidly advancing technology, significant operational pressures and rapidly changing 'teams of experts'. However, to date there has been little focus on the effect of NTS in this field.

OBJECTIVES

This review aims to examine the existing literature on NTS in gastrointestinal endoscopy and identify areas for further research.

METHOD

A systematic search of MEDLINE, Embase, Cochrane Library, PsychINFO, CINAHL Plus and PubMed databases was performed using search terms Non-Technical Skills, Team Performance or Team Skills, and Endoscopy, Colonoscopy, OGD, Gastroscopy, Endoscopic Retrograde Cholangio-Pancreatography or Endoscopic Ultrasound.

RESULTS

Eighteen relevant publications were found. NTS are deemed an essential component of practice, but so far there is little evidence of their integration into training or competency assessment. Those studies examining the effects of NTS and team training in endoscopy are small and have variable outcome measures with limited evidence of improvement in skills or clinical outcomes. NTS assessment in endoscopy is in its early phases with a few tools in development.

CONCLUSIONS

The current literature on NTS in gastrointestinal endoscopy is limited. NTS, however, are deemed an essential component of practice, with potential positive effects on team performance and clinical outcomes. A validated reliable tool would enable evaluation of training and investigation into the effects of NTS on outcomes. There is a clear need for further research in this field.

Methods for certification in colonoscopy - a systematic review

INTRODUCTION

Reliable, valid, and feasible assessment tools are essential to ensure competence in colonoscopy. This study aims to provide an overview of the existing assessment methods and the validity evidence that supports them.

METHODS

A systematic search was conducted in October 2016. Pubmed, EMBASE, and PsycINFO were searched for studies evaluating assessment methods to ensure competency in colonoscopy. Outcome variables were described and evidence of validity was explored using a contemporary framework.

RESULTS

Twenty-five observational studies were included in the systematic review. Most studies were based on small sample sizes. The studies were categorized after outcome measures into five groups: Clinical process related outcome metrics (n = 2), direct observational colonoscopy assessment (n = 8), simulator based metrics (n = 11), automatic computerized metrics (n = 2), and self-assessment (n = 1). Validity score varied among the studies and only five studies presented sufficient evidence to recommend the tool for clinical assessment.

CONCLUSIONS

The objectives vary throughout the presented tools. Some tools are global tools where others focus on procedural technical skill assessment or even part-task skills. There is a tendency in the most recent studies towards more specific assessment of technical skills. The majority of assessment methods lack sufficient validity evidence.

A prospective comparison of live and video-based assessments of colonoscopy performance

BACKGROUND AND AIMS

Colonoscopy performance is typically assessed by a supervisor in the clinical setting. There are limitations of this approach, however, because it allows for rater bias and increases supervisor workload demand during the procedure. Video-based assessment of recorded procedures has been proposed as a complementary means by which to assess colonoscopy performance. This study sought to investigate the reliability, validity, and feasibility of video-based assessments of competence in performing colonoscopy compared with live assessment.

METHODS

Novice (<50 previous colonoscopies), intermediate (50-500), and experienced (>1000) endoscopists from 5 hospitals participated. Two views of each colonoscopy were videotaped: an endoscopic (intraluminal) view and a recording of the endoscopist's hand movements. Recorded procedures were independently assessed by 2 blinded experts using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT), a validated procedure-specific assessment tool comprising a global rating scale (GRS) and checklist (CL). Live ratings were conducted by a non-blinded expert endoscopist. Outcomes included agreement between live and blinded video-based ratings of clinical colonoscopies, intra-rater reliability, inter-rater reliability and discriminative validity of video-based assessments, and perceived ease of assessment.

RESULTS

Forty endoscopists participated (20 novices, 10 intermediates, and 10 experienced). There was good agreement between the live and video-based ratings (total, intra-class correlation [ICC] = 0.847; GRS, ICC = 0.868; CL, ICC = 0.749). Intra-rater reliability was excellent (total, ICC = 0.99; GRS, ICC = 0.99; CL, ICC = 0.98). Inter-rater reliability between the 2 blinded video-based raters was high (total, ICC = 0.91; GRS, ICC = 0.918; CL, ICC = 0.862). GiECAT total, GRS, and CL scores differed significantly among novice, intermediate, and experienced endoscopists (P < .001). Video-based assessments were perceived as "fairly easy," although live assessments were rated as significantly easier (P < .001).

CONCLUSIONS

Video-based assessments of colonoscopy procedures using the GiECAT have strong evidence of reliability and validity. In addition, assessments using videos were feasible, although live assessments were easier.

Impact of cap-assisted colonoscopy on the learning curve and quality in colonoscopy: a randomized controlled trial

BACKGROUND AND AIMS

Colonoscopy competency assessment in trainees traditionally has been informal. Comprehensive metrics such as the Assessment of Competency in Endoscopy (ACE) tool suggest that competency thresholds are higher than assumed. Cap-assisted colonoscopy (CAC) may improve competency, but data regarding novice trainees are lacking. We compared CAC versus standard colonoscopy (SC) performed by novice trainees in a randomized controlled trial.

METHODS

All colonoscopies performed by 3 gastroenterology fellows without prior experience were eligible for the study. Exclusion criteria included patient age <18 or >90 years, pregnancy, prior colon resection, diverticulitis, colon obstruction, severe hematochezia, referral for EMR, or a procedure done without patient sedation. Patients were randomized to either CAC or SC in a 1:1 fashion. The primary outcome was the independent cecal intubation rate (ICIR). Secondary outcomes were cecal intubation time, polyp detection rate, polyp miss rate, adenoma detection rate, ACE tool scores, and cumulative summation learning curves.

RESULTS

A total of 203 colonoscopies were analyzed, 101 in CAC and 102 in SC. CAC resulted in a significantly higher cecal intubation rate, at 79.2% in CAC compared with 66.7% in SC (P = .04). Overall cecal intubation time was significantly shorter at 13.7 minutes for CAC versus 16.5 minutes for SC (P =.02). Cecal intubation time in the case of successful independent fellow intubation was not significantly different between CAC and SC (11.6 minutes vs 12.7 minutes; P = .29). Overall ACE tool motor and cognitive scores were higher with CAC. Learning curves for ICIR approached the competency threshold earlier with cap use but reached competency for only 1 fellow. The polyp detection rate, polyp miss rate, and adenoma detection rate were not significantly different between groups.

CONCLUSIONS

CAC resulted in significant improvement in ICIR, overall ACE tool scores, and trend toward competency on learning curves when compared with SC in colonoscopy trainees without prior colonoscopy experience. (Clinical trial registration number: NCT02472730.).

Impact of experience on self-assessment accuracy of clinical colonoscopy competence

BACKGROUND AND AIMS

Self-assessment is important for life-long learning and a recommended assessment method for endoscopy skills. Prior literature has not investigated self-assessment accuracy of colonoscopic competence in the clinical setting. This study aimed to determine the self-assessment accuracy of novice, intermediate, and experienced endoscopists.

METHODS

Novice (performed <50 previous colonoscopies), intermediate (50-500), and experienced (>1000) endoscopists from 5 hospitals each performed a clinical colonoscopy. Video recordings of procedures were independently assessed by 2 blinded expert endoscopists by using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Externally assessed and self-assessed GiECAT scores were defined as the mean of the 2 video-based ratings and as participants' own assigned ratings, respectively. Self-assessment accuracy between the externally assessed and self-assessed scores was evaluated by using absolute difference scores, intraclass correlation coefficients, and the Bland-Altman analysis.

RESULTS

Twenty novice, 10 intermediate, and 10 experienced endoscopists participated. There was moderate agreement of externally assessed and self-assessed GiECAT scores, with an intraclass correlation coefficient of 0.65 (95% confidence interval, 0.44-0.80). The absolute difference scores among the 3 groups were significantly different (P = .002), with experienced endoscopists demonstrating a more accurate self-assessment ability compared with novices (P = .002). Bland-Altman plots suggest that novice and experienced endoscopists tend to overrate and underrate their clinical competence, respectively; no specific trends were associated with intermediates.

CONCLUSION

Participants demonstrated moderate self-assessment accuracy of clinical competence. Endoscopist experience was positively associated with self-assessment accuracy; novices demonstrated lower self-assessment accuracy compared with experienced endoscopists. Moreover, novices tended to overestimate their performances. Novice endoscopists may benefit from targeted interventions to improve self-assessment accuracy.

A System to Assess the Competency for Interpretation of Esophageal Manometry Identifies Variation in Learning Curves

BACKGROUND & AIMS

Quality esophageal high-resolution manometry (HRM) studies require competent interpretation of data. However, there is little understanding of learning curves, training requirements, or measures of competency for HRM. We aimed to develop and use a competency assessment system to examine learning curves for interpretation of HRM data.

METHODS

We conducted a prospective multicenter study of 20 gastroenterology trainees with no experience in HRM, from 8 centers, over an 8-month period (May through December 2015). We designed a web-based HRM training and competency assessment system. After reviewing the training module, participants interpreted 50 HRM studies and received answer keys at the fifth and then at every second interpretation. A cumulative sum procedure produced individual learning curves with preset acceptable failure rates of 10%; we classified competency status as competency not achieved, competency achieved, or competency likely achieved.

RESULTS

Five (25%) participants achieved competence, 4 (20%) likely achieved competence, and 11 (55%) failed to achieve competence. A minimum case volume to achieve competency was not identified. There was no significant agreement between diagnostic accuracy and accuracy for individual HRM skills.

CONCLUSIONS

We developed a competency assessment system for HRM interpretation; using this system, we found significant variation in learning curves for HRM diagnosis and individual skills. Our system effectively distinguished trainee competency levels for HRM interpretation and contrary to current recommendations, found that competency for HRM is not case-volume specific.

Achieving Consensus on Principles of Good Practice for School Health in Independent Schools: A Delphi Study

BACKGROUND

Although the 8 components of the coordinated school health (CSH) framework have been implemented to various degrees in the nation's public schools, principles of good practice (PGPs) to guide health promotion efforts in independent schools do not exist. The purpose of this study was to generate PGPs and rate their feasibility of implementation.

METHODS

A modified Delphi process was used to generate PGPs for school health in independent schools and develop consensus regarding their feasibility of implementation. A working group of 6 independent school health professionals (ISHPs) was convened to propose draft PGPs based on the CSH framework. The proposed PGPs were then reviewed by a national and geographically diverse expert panel of 10 school health researchers and 23 ISHPs that completed 3 Delphi rounds to achieve consensus on the PGPs. Of 33 participants originally invited to participate, 27 completed the Delphi process.

RESULTS

A total of 27 panelists rated 67 PGPs, provided 399 comments, and achieved consensus using interquartile range on 56 (84%) of the proposed PGPs, 41 of which were rated feasible and 15 somewhat feasible.

CONCLUSION

This study has generated empirical support for the feasibility of PGPs for school health in independent schools.

Simulation-Based Training of Non-Technical Skills in Colonoscopy: Protocol for a Randomized Controlled Trial

BACKGROUND

Non-technical skills (NTS), such as communication and professionalism, contribute to the safe and effective completion of procedures. NTS training has previously been shown to improve surgical performance. Moreover, increases in NTS have been associated with improved clinical endoscopic performance. Despite this evidence, NTS training has not been tested as an intervention in endoscopy.

OBJECTIVE

The aim of this study is to evaluate the effectiveness of a simulation-based training (SBT) curriculum of NTS on novice endoscopists' performance of clinical colonoscopy.

METHODS

Novice endoscopists were randomized to 2 groups. The control group received 4 hours of interactive didactic sessions on colonoscopy theory and 6 hours of SBT. Hours 5 and 6 of the SBT were integrated scenarios, wherein participants interacted with a standardized patient and nurse, while performing a colonoscopy on the virtual reality (VR) simulator. The NTS (intervention) group received the same teaching sessions but the last hour was focused on NTS teaching. The NTS group also reviewed a checklist of tasks relevant to NTS concepts prior to each integrated scenario case and was provided with dedicated feedback on their NTS performance during the integrated scenario practice. All participants were assessed at baseline, immediately after training, and 4 to 6 weeks post-training. The primary outcome measure is colonoscopy-specific performance in the clinical setting.

RESULTS

In total, 42 novice endoscopists completed the study. Data collection and analysis is ongoing. We anticipate completion of all assessments by August 2017. Data analysis, manuscript writing, and subsequent submission for publication is expected to be completed by December 2017.

CONCLUSIONS

Results from this study may inform the implementation of NTS training into postgraduate gastrointestinal curricula. NTS curricula may improve attitudes towards patient safety and self-reflection among trainees. Moreover, enhanced NTS may lead to superior clinical performance and outcomes in colonoscopy.

TRIAL REGISTRATION

Clinicaltrial.gov NCT02877420; https://www.clinicaltrials.gov/ct2/show/NCT02877420 (Archived by WebCite at http://www.webcitation.org/6rw94ubXX NCT02877420).

Combining different methods improves assessment of competence in colonoscopy

OBJECTIVES

To develop a reliable method of assessing competence in colonoscopy based on multiple sources.

MATERIALS AND METHODS

Physicians with varying degrees of experience in colonoscopy performed two colonoscopies each in a standardized simulated environment. Their performances were assessed under direct observation by an expert rater and by automatic computerized analysis of operator movements and scope movements, respectively. Reliability (Cronbach's alpha) for subjective assessment, time to cecum, analysis of operator movement and analysis of scope movements were calculated. Composite score calculations were used to explore different combinations of the measures.

RESULTS

Twenty physicians were included in the study. The reliability (Cronbach's alpha) were 0.92, 0.57, 0.87 and 0.55 for the subjective score assessed under direct observation, time to cecum, distance between operator's hands and colonoscopy progression score, respectively. Equal weight (=25%) to all four methods resulted in a reliability of 0.91 and optimal weighting of the methods (55%, 10%, 25% and 10%, respectively) resulted in a maximum reliability of 0.95.

CONCLUSION

Combining subjective expert ratings with automated objective assessments results in a less biased and more reliable assessment of competence in colonoscopy.

Automatic and unbiased assessment of competence in colonoscopy: exploring validity of the Colonoscopy Progression Score (CoPS)

Background and aims: Colonoscopy is a difficult procedure to master. Increasing demands for colonoscopy, due to screening and surveillance programs, have highlighted the need for competent performers. Valid methods for assessing technical skills are pivotal for training and assessment. This study is the first clinical descriptive report of a novel colonoscopy assessment tool based on Magnetic Endoscopic Imaging (MEI) data and the aim was to gather validity evidence based on the data collected using the "Colonoscopy Progression Score" (CoPS). Methods: We recorded 137 colonoscopy procedures performed by 31 endoscopists at three university hospitals. The participants performed more than two procedures each (range 2 - 12) and had an experience of 0 - 10 000 colonoscopies. The CoPS was calculated for each recording and validity was explored using a widely accepted contemporary framework. The following sources of validity evidence were explored: response process (data collection), internal structure (reliability), relationship to other variables (i. e. operator experience), and consequences of testing (pass/fail). Results: Identical set-ups at all three locations ensured uniform data collection. The Generalizability coefficient (G-coefficient) was 0.80, and a Decision-study (D-study) revealed that four recordings were sufficient to ensure a G-coefficient above 0.80. We showed a positive correlation between CoPS and experience with Pearson's r of 0.61 (P < 0.001). A pass/fail standard of 107 points was established using the contrasting group method to explore the consequences of testing. Conclusion: This study provides evidence supporting the validity of the CoPS for use in assessing technical colonoscopy performance in the clinical setting.

STUDY REGISTRATION

NCT01997177.

Expert opinions and scientific evidence for colonoscopy key performance indicators

Colonoscopy is a widely performed procedure with procedural volumes increasing annually throughout the world. Many procedures are now performed as part of colorectal cancer screening programmes. Colonoscopy should be of high quality and measures of this quality should be evidence based. New UK key performance indicators and quality assurance standards have been developed by a working group with consensus agreement on each standard reached. This paper reviews the scientific basis for each of the quality measures published in the UK standards.

In-training gastrointestinal endoscopy competency assessment tools: Types of tools, validation and impact

The ability to perform endoscopy procedures safely, effectively and efficiently is a core element of gastroenterology practice. Training programs strive to ensure learners demonstrate sufficient competence to deliver high quality endoscopic care independently at completion of training. In-training assessments are an essential component of gastrointestinal endoscopy education, required to support training and optimize learner's capabilities. There are several approaches to in-training endoscopy assessment from direct observation of procedural skills to monitoring of surrogate measures of endoscopy skills such as procedural volume and quality metrics. This review outlines the current state of evidence as it pertains to in-training assessment of competency in performing gastrointestinal endoscopy as part of an overall endoscopy quality and skills training program.

Effective teaching of endoscopy: a qualitative study of the perceptions of gastroenterology fellows and attending gastroenterologists

Background: There is little information describing the perceptions of gastroenterology fellows and attending gastroenterologists of what constitutes effective teaching of endoscopy. We sought to identify common themes regarding endoscopy training methods and their impact among fellows and attendings.

Methods: Focus group exercises and surveys were conducted among fellows, about educational resources, teaching techniques and ways of improving the teaching of endoscopy. The fellows identified the ‘best' teachers of endoscopy, who were interviewed regarding their training in endoscopy, their teaching methods, key points of information, and opinions on endoscopy curriculum.

Results: Nineteen fellows (68%) had attended the American Society for Gastrointestinal Endoscopy First Year Fellows’ Endoscopy course and found it very helpful. Thirteen fellows(46%) had exposure to an endoscopy simulator, but their median duration of use was only 1 hour. Only two out of five fellowship programs used a formal endoscopic skill assessment tool and none of the programs had an endoscopy curriculum of which the fellows were aware. Fellows reported that they learned endoscopy best by performing procedures. They also volunteered that attending gastroenterologists used variable teaching methods, and might benefit from instruction on how to teach endoscopy. Ten attending gastroenterologists (77%) had received training in advanced procedures; none received formal training on teaching endoscopy: they all felt that such training would be beneficial.

Conclusions: A standardized endoscopy curriculum may be beneficial to fellows, who prefer to learn endoscopy by performing procedures—but they want explicit and specific instruction. Both those attending and the fellows thought that formal instruction for attending gastroenterologists on how to teach endoscopy would be beneficial, indicating a role for a ‘teach-the-teacher' curriculum.

Measuring Quality in Pediatric Endoscopy

Measuring quality in endoscopy includes the assessment of appropriateness of a procedure and the skill with which it is performed. High-quality pediatric endoscopy is safe and efficient, used effectively to make proper diagnoses, is useful for excluding other diagnoses, minimizes adverse events, and is accompanied by appropriate documentation from beginning through end of the procedure. There are no standard quality metrics for pediatric endoscopy, but proposed candidates are both process and outcomes oriented. Both are likely to be used in the near future to increase transparency about patient outcomes, as well as to influence payments for the procedure.

Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: a randomized trial

BACKGROUND

GI endoscopy simulation-based training augments early clinical performance; however, the optimal manner by which to deliver training is unknown.

OBJECTIVE

We aimed to validate a simulation-based structured comprehensive curriculum (SCC) designed to teach technical, cognitive, and integrative competencies in colonoscopy.

DESIGN

Single-blinded, randomized, controlled trial.

SETTING

Endoscopic simulation course at an academic hospital.

PARTICIPANTS AND INTERVENTIONS

Thirty-three novice endoscopists were allocated to an SCC group or self-regulated learning (SRL) group. The SCC group received a curriculum consisting of 6 hours of didactic lectures and 8 hours of virtual reality simulation-based training with expert feedback. The SRL group was provided a list of desired objectives and was instructed to practice on the simulator for an equivalent time (8 hours).

MAIN OUTCOME MEASUREMENTS

Clinical transfer was assessed during 2 patient colonoscopies using the Joint Advisory Group Direct Observation of Procedural Skills (JAG DOPS) scale. Secondary outcome measures included differences in procedural knowledge, immediate post-training simulation performance, and delayed post-training (4-6 weeks) performance during an integrated scenario test on the JAG DOPS communication and integrated scenario global rating scales.

RESULTS

There was no significant difference in baseline or post-training performance on the simulator task. The SCC group performed superiorly during their first and second clinical colonoscopies. Additionally, the SCC group demonstrated significantly better knowledge and colonoscopy-specific performance, communication, and global performance during the integrated scenario.

LIMITATIONS

We were unable to measure SRL participants' effort outside of mandatory training. In addition, feedback metrics and number of available simulation cases are limited.

CONCLUSIONS

These results support integration of endoscopy simulation into a structured curriculum incorporating instructional feedback and complementary didactic knowledge as a means to augment technical, cognitive, and integrative skills acquisition, as compared with SRL on virtual reality simulators. (

CLINICAL TRIAL REGISTRATION NUMBER

NCT01991522.)

A competency framework for colonoscopy training derived from cognitive task analysis techniques and expert review

BACKGROUND

Colonoscopy is a difficult cognitive-perceptual-motor task. Designing an appropriate instructional program for such a task requires an understanding of the knowledge, skills and attitudes underpinning the competency required to perform the task. Cognitive task analysis techniques provide an empirical means of deriving this information.

METHODS

Video recording and a think-aloud protocol were conducted while 20 experienced endoscopists performed colonoscopy procedures. "Cued-recall" interviews were also carried out post-procedure with nine of the endoscopists. Analysis of the resulting transcripts employed the constant comparative coding method within a grounded theory framework. The resulting draft competency framework was modified after review during semi-structured interviews conducted with six expert endoscopists.

RESULTS

The proposed colonoscopy competency framework consists of twenty-seven skill, knowledge and attitude components, grouped into six categories (clinical knowledge; colonoscope handling; situation awareness; heuristics and strategies; clinical reasoning; and intra- and inter-personal).

CONCLUSIONS

The colonoscopy competency framework provides a principled basis for the design of a training program, and for the design of formative assessment to gauge progress towards attaining the knowledge, skills and attitudes underpinning the achievement of colonoscopy competence.

Operationalising elaboration theory for simulation instruction design: a Delphi study

OBJECTIVE

The aim of this study was to assess the feasibility of incorporating the Delphi process within the simplifying conditions method (SCM) described in elaboration theory (ET) to identify conditions impacting the complexity of procedural skills for novice learners.

METHODS

We generated an initial list of conditions impacting the complexity of lumbar puncture (LP) from key informant interviews (n = 5) and a literature review. Eighteen clinician-educators from six different medical specialties were subsequently recruited as expert panellists. Over three Delphi rounds, these panellists rated: (i) their agreement with the inclusion of the simple version of the conditions in a representative ('epitome') training scenario, and (ii) how much the inverse (complex) version increases LP complexity for a novice. Cronbach's α-values were used to assess inter-rater agreement.

RESULTS

All panellists completed Rounds 1 and 2 of the survey and 17 completed Round 3. In Round 1, Cronbach's α-values were 0.89 and 0.94 for conditions that simplify and increase LP complexity, respectively; both values increased to 0.98 in Rounds 2 and 3. With the exception of 'high CSF (cerebral spinal fluid) pressure', panellists agreed with the inclusion of all conditions in the simplest (epitome) training scenario. Panellists rated patient movement, spinal anatomy, patient cooperativeness, body habitus, and the presence or absence of an experienced assistant as having the greatest impact on the complexity of LP.

CONCLUSIONS

This study demonstrated the feasibility of using expert consensus to establish conditions impacting the complexity of procedural skills, and the benefits of incorporating the Delphi method into the SCM. These data can be used to develop and sequence simulation scenarios in a progressively challenging manner. If the theorised learning gains associated with ET are realised, the methods described in this study may be applied to the design of simulation training for other procedural and non-procedural skills, thereby advancing the agenda of theoretically based instruction design in health care simulation.

The gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy

OBJECTIVES

Validated assessment tools are required to support competency-based education. We aimed to assess the reliability and validity of the Gastrointestinal Endoscopy Competency Assessment Tool for Pediatric Colonoscopy (GiECATKIDS), an instrument developed by 41 North American experts using Delphi methodology.

METHODS

GiECATKIDS consists of a 7-item global rating scale (GRS) and an 18-item checklist (CL). An attending physician assessed 104 colonoscopies performed at 3 North American hospitals by 56 endoscopists, including 25 novices (<50 previous procedures), 21 intermediates (50-250), and 10 advanced endoscopists (>500). Another observer rated procedures to assess interrater reliability using intraclass correlation coefficient (ICC). Test-retest reliability was measured with ICC comparing endoscopists' first and second procedure scores. Discriminative validity was examined by comparing experience level with scores. Concurrent validity was assessed by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment.

RESULTS

Interrater reliability of the GiECATKIDS was high (total: ICC = 0.88; GRS: ICC = 0.79; CL: ICC = 0.89). Test-retest reliability was excellent (total: ICC = 0.94; GRS: ICC = 0.94; CL: ICC = 0.84). GiECATKIDS total, GRS, and CL scores differed significantly among novice, intermediate, and advanced endoscopists (P < 0.001). There was a significant positive correlation (P < 0.001) between scores and number of previous colonoscopies (total: ρ = 0.91, GRS: ρ = 0.92, CL: ρ = 0.84), cecal intubation rate (total: ρ = 0.82, GRS: ρ = 0.85, CL: ρ = 0.77), ileal intubation rate (total: ρ = 0.82, GRS: ρ = 0.82, CL: ρ = 0.80), and physician global assessment (total: ρ = 0.95, GRS: ρ = 0.94, CL: ρ = 0.89).

CONCLUSIONS

The GiECATKIDS demonstrates strong reliability and validity as a measure of performance of pediatric colonoscopy that can be used to support training and assessment.

Gastrointestinal Endoscopy Competency Assessment Tool: reliability and validity evidence

BACKGROUND

Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice.

OBJECTIVE

To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting.

DESIGN

Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment.

SETTING

A total of 116 colonoscopies performed by 33 novice (<50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (>1000 previous procedures) endoscopists from 6 Canadian hospitals.

MAIN OUTCOME MEASUREMENTS

Interrater and test-retest reliability, discriminative, and concurrent validity.

RESULTS

Interrater reliability was high (total: ICC=0.85; GRS: ICC=0.85; CL: ICC=0.81). Test-retest reliability was excellent (total: ICC=0.91; GRS: ICC=0.93; CL: ICC=0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P<.001). There was a significant positive correlation (P<.001) between scores and number of previous colonoscopies (total: ρ=0.78, GRS: ρ=0.80, CL: Spearman's ρ=0.71); cecal intubation rate (total: ρ=0.81, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.75); ileal intubation rate (total: Spearman's ρ=0.82, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.77); and physician global assessment (total: Spearman's ρ=0.90, GRS: Spearman's ρ=0.94, CL: Spearman's ρ=0.77).

LIMITATIONS

Nonblinded assessments.

CONCLUSION

This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.

Development of the gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy (GiECAT KIDS)

OBJECTIVES

Many aspects of pediatric colonoscopy differ from adult practice. To date, there is no validated measure of endoscopic competence for use in pediatrics. Using Delphi methodology, we aimed to determine expert consensus regarding items required on a checklist and global rating scale designed to assess the competence of clinicians performing colonoscopy on pediatric patients.

METHODS

A total of 41 North American pediatric endoscopy experts rated potential checklist and global rating items for their importance as indicators of the competence of trainees learning to perform pediatric colonoscopy. Responses were analyzed and re-sent to the panel for further ratings until consensus was reached. Items that ≥ 80% of experts rated as ≥ 4 out of 5 were included in the final instrument. Consensus items were compared with those items deemed by adult endoscopy experts as fundamental to assessing the performance of adult colonoscopy.

RESULTS

Five rounds of surveys were completed with response rates ranging from 76% to 100%. Seventy-five checklist and 38 global rating items were reduced to 18 checklist and 7 global rating items that reached consensus. Three pediatric checklist items differed from those considered to be critical adult indicators, whereas 4 items on the latter did not reach consensus among pediatric experts.

CONCLUSIONS

Delphi methodology allowed for achievement of expert consensus regarding essential items to be included in the Gastrointestinal Endoscopy Competency Assessment Tool for Pediatric Colonoscopy (GiECATKIDS), a measure of endoscopic competence specific to performing pediatric colonoscopy. Key differences in the checklist items, compared with items reaching consensus during a separate adult Delphi process using the same indicators, emphasize the need for a pediatric-specific tool.

Assessment of competence in pediatric gastrointestinal endoscopy

Gastrointestinal endoscopy is an invaluable tool for the diagnosis and treatment of upper and lower gastrointestinal diseases in children. Pediatric and adult endoscopy differ in several respects including differences in procedural indications, sedation practices, pre-procedure preparation, equipment, and the importance of routine tissue sampling and terminal ileum intubation. In the same way that performance of endoscopy in children requires pediatric-specific training, assessment of pediatric endoscopists requires an approach that is tailored to pediatric practice and the use of assessment methods and measures that have been developed and validated specifically within the pediatric context.