Influence of video-based feedback on self-assessment accuracy of endoscopic skills: a randomized controlled trial

Concordance Between Trainee Self-Assessment and Expert Assessment of Cold Snare Polypectomy Competence: Results From a Randomized Clinical Study

PURPOSE

Many trainees lack competence in performing cold snare polypectomy (CSP), and longer observation periods using assessment tools, such as the Cold Snare Polypectomy Assessment Tool (CSPAT), may be required. However, these tools are not commonly used in busy academic endoscopy practices. This study evaluates the concordance between trainee self-assessment of CSP with expert assessment and assesses factors associated with concordance.

METHOD

Consecutive CSPs performed by gastroenterology trainees from 2 institutions were video recorded and rated by 8 blinded experts from 4 different academic institutions in the United States and Singapore using the CSPAT from August 2017 to February 2020. Trainees self-assessed competence for each CSP immediately after the procedure. Concordance between trainee and trainer was reported as percentage of agreement in competence (score of 3 or 4) or not yet competent (score of 1 or 2).

RESULTS

Twenty-two trainees performed 765 colonoscopies with 2,267 CSPs. Concordance was found between expert and trainee assessment of competence for 1,380 CSPs (60.9%; 95% CI, 58.8%-62.9%; weighted κ = 0.12; 95% CI, 0.08-0.16). Trainees underassessed competence for 541 CSPs (23.9%; 95% CI, 22.1%-25.7%) and overassessed competence for 326 CSPs (15.3%; 95% CI, 13.8%-16.8%). Career plan of private practice general gastroenterology (adjusted odds ratio, 0.40; 95% CI, 0.17-0.92) and lower colonoscopy volume before study (adjusted odds ratio, 0.35; 95% CI, 0.22-0.62) were independently associated with lower likelihood of concordance between expert and trainee assessment of competence. Structured feedback was not associated with concordance, and there was no change as trainees gained experience; however, feedback decreased the proportion of overassessments.

CONCLUSIONS

There is poor concordance between trainee self-assessment and trainer assessment of CSP. Self-assessment can be used as an initial part of competency assessment, which should then be reconciled with external assessments to improve concordance in assessments.

Video recording in GI endoscopy

The current approach to procedure reporting in endoscopy aims to capture essential findings and interventions but inherently sacrifices the rich detail and nuance of the entire endoscopic experience. Endoscopic video recording (EVR) provides a complete archive of the procedure, extending the utility of the encounter beyond diagnosis and intervention, and potentially adding significant value to the care of the patient and the field in general. This white paper outlines the potential of EVR in clinical care, quality improvement, education, and artificial intelligence-driven innovation, and addresses critical considerations surrounding technology, regulation, ethics, and privacy. As with other medical imaging modalities, growing adoption of EVR is inevitable, and proactive engagement of professional societies and practitioners is essential to harness the full potential of this technology toward improving clinical care, education, and research.

Disinfection of flexible fibre-optic endoscopes out-of-hours: confidential telephone survey of ENT units in England - 20 years on

BACKGROUND

Flexible upper aerodigestive endoscopy is often performed in the emergency setting. To prevent nosocomial infection on-call clinicians must have access to decontaminated endoscopes.

METHODS

A telephone survey of 104 ENT units in England replicated previous cycles conducted 10 and 20 years ago. The on-call clinician was asked about decontamination practices, training and cross-cover.

RESULTS

Seventy-one clinicians participated of which 68 had an endoscope available out-of-hours. Twenty-five (36.8 per cent) used single-use endoscopes. Twenty-three (51.1 per cent) of the 45 clinicians using re-usable endoscopes decontaminated them themselves, an increase from 43.3 per cent in 2013 and from 35.1 per cent in 2002. Overall 91.2 per cent had safe practices, up from 68.7 per cent in 2013 and 48 per cent in 2002. One hundred per cent had been trained in decontamination, compared to 37.3 per cent in 2013 and 12.1 per cent in 2002. On-call clinicians from the ENT department increased to 91.5 per cent, compared to 63 per cent in 2013.

CONCLUSION

There has been a dramatic increase in patient safety, underpinned by the introduction of single-use endoscopes, increased training and reduced cross-cover.

Training in basic gastrointestinal endoscopic procedures: a European Society of Gastrointestinal Endoscopy (ESGE) and European Society of Gastroenterology and Endoscopy Nurses and Associates (ESGENA) Position Statement

This ESGE Position Statement provides structured and evidence-based guidance on the essential requirements and processes involved in training in basic gastrointestinal (GI) endoscopic procedures. The document outlines definitions; competencies required, and means to their assessment and maintenance; the structure and requirements of training programs; patient safety and medicolegal issues. 1:  ESGE and ESGENA define basic endoscopic procedures as those procedures that are commonly indicated, generally accessible, and expected to be mastered (technically and cognitively) by the end of any core training program in gastrointestinal endoscopy. 2:  ESGE and ESGENA consider the following as basic endoscopic procedures: diagnostic upper and lower GI endoscopy, as well as a limited range of interventions such as: tissue acquisition via cold biopsy forceps, polypectomy for lesions ≤ 10 mm, hemostasis techniques, enteral feeding tube placement, foreign body retrieval, dilation of simple esophageal strictures, and India ink tattooing of lesion location. 3:  ESGE and ESGENA recommend that training in GI endoscopy should be subject to stringent formal requirements that ensure all ESGE key performance indicators (KPIs) are met. 4:  Training in basic endoscopic procedures is a complex process and includes the development and acquisition of cognitive, technical/motor, and integrative skills. Therefore, ESGE and ESGENA recommend the use of validated tools to track the development of skills and assess competence. 5:  ESGE and ESGENA recommend incorporating a multimodal approach to evaluating competence in basic GI endoscopic procedures, including procedural thresholds and the measurement and documentation of established ESGE KPIs. 7:  ESGE and ESGENA recommend the continuous monitoring of ESGE KPIs during GI endoscopy training to ensure the trainee's maintenance of competence. 9:  ESGE and ESGENA recommend that GI endoscopy training units fulfil the ESGE KPIs for endoscopy units and, furthermore, be capable of providing the dedicated personnel, infrastructure, and sufficient case volume required for successful training within a structured training program. 10:  ESGE and ESGENA recommend that trainers in basic GI endoscopic procedures should be endoscopists with formal educational training in the teaching of endoscopy, which allows them to successfully and safely teach trainees.

Pediatric endoscopy: how can we improve patient outcomes and ensure best practices?

INTRODUCTION

Strategies to promote high-quality endoscopy in children require consensus around pediatric-specific quality standards and indicators. Using a rigorous guideline development process, the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) was developed to support continuous quality improvement efforts within and across pediatric endoscopy services.

AREAS COVERED

This review presents a framework, informed by the PEnQuIN guidelines, for assessing endoscopist competence, granting procedural privileges, audit and feedback, and for skill remediation, when required. As is critical for promoting quality, PEnQuIN indicators can be benchmarked at the individual endoscopist, endoscopy facility, and endoscopy community levels. Furthermore, efforts to incorporate technologies, including electronic medical records and artificial intelligence, into endoscopic quality improvement processes can aid in creation of large-scale networks to facilitate comparison and standardization of quality indicator reporting across sites.

EXPERT OPINION

PEnQuIN quality standards and indicators provide a framework for continuous quality improvement in pediatric endoscopy, benefiting individual endoscopists, endoscopy facilities, and the broader endoscopy community. Routine and reliable measurement of data, facilitated by technology, is required to identify and drive improvements in care. Engaging all stakeholders in endoscopy quality improvement processes is crucial to enhancing patient outcomes and establishing best practices for safe, efficient, and effective pediatric endoscopic care.

Promoting Research that Supports High-Quality Gastrointestinal Endoscopy in Children

PURPOSE OF REVIEW

Defining and measuring the quality of endoscopic care is a key component of performing gastrointestinal endoscopy in children. The purpose of this review is to discuss quality metrics for pediatric gastrointestinal endoscopy and identify where additional research is needed.

RECENT FINDINGS

Pediatric-specific standards and indicators were recently defined by the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) working group through a rigorous guideline consensus process. Although the aim of these guidelines is to facilitate best practices for safe and high-quality gastrointestinal endoscopy in children, they highlight the pressing need to expand upon the body of evidence supporting these standards and indicators as predictors of clinically relevant outcomes. In this review, we propose and discuss ideas for several high-yield research topics to engage pediatric endoscopists and promote best practices in pediatric endoscopy.

Video-based interventions to improve self-assessment accuracy among physicians: A systematic review

PURPOSE

Self-assessment of a physician's performance in both procedure and non-procedural activities can be used to identify their deficiencies to allow for appropriate corrective measures. Physicians are inaccurate in their self-assessments, which may compromise opportunities for self- development. To improve this accuracy, video-based interventions of physicians watching their own performance, an experts' performance or both, have been proposed to inform their self-assessment. We conducted a systematic review of the effectiveness of video-based interventions targeting improved self-assessment accuracy among physicians.

MATERIALS AND METHODS

The authors performed a systematic search of MEDLINE, Embase, EBM reviews, and Scopus databases from inception to August 23, 2022, using combinations of terms for "self-assessment", "video-recording", and "physician". Eligible studies were empirical investigations assessing the effect of video-based interventions on physicians' self-assessment accuracy with a comparison of self-assessment accuracy pre- and post- video intervention. We defined self-assessment accuracy as a "direct comparison between an external evaluator and self-assessment that was quantified using formal statistical analysis". Two reviewers independently screened records, extracted data, assessed risk of bias, and evaluated quality of evidence. A narrative synthesis was conducted, as variable outcomes precluded a meta-analysis.

RESULTS

A total of 2,376 papers were initially retrieved. Of these, 22 papers were selected for full-text review; a final 9 studies met inclusion criteria for data extraction. Across studies, 240 participants from 5 specialties were represented. Video-based interventions included self-video review (8/9), benchmark video review (3/9), and/or a combination of both types (1/9). Five out of nine studies reported that participants had inaccurate self-assessment at baseline. After the intervention, 5 of 9 studies found a statistically significant improvement in self-assessment accuracy.

CONCLUSIONS

Overall, current data suggests video-based interventions can improve self-assessment accuracy. Benchmark video review may enable physicians to improve self-assessment accuracy, especially for those with limited experience performing a particular clinical skill. In contrast, self-video review may be able to provide improvement in self-assessment accuracy for more experience physicians. Future research should use standardized methods of comparison for self-assessment accuracy, such as the Bland-Altman analysis, to facilitate meta-analytic summation.

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis

BACKGROUND

Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists.

METHODS

This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland-Altman studies.

RESULTS

After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (-41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (-45.0 % to 35.1 % and -54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (-14.2 % to 21.4 %).

CONCLUSIONS

GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

Improving robotic skills by video review

Background Since the transition of surgical robot systems into the medical field, physicians have had to develop new dexterity skills. The ideal learning environment for robotic surgery has yet to be discovered. Virtual-reality (VR) simulation is a possible safe and economic method. In VR-simulator training human feedback is hardly used, an alternative may be found in video revision. The purpose of this study is to investigate whether adding video review to VR-simulation-based training in novice physicians improves their ability to complete a complex robot task. In addition, the secondary goal is to investigate whether the skills learned on the robotic simulator can be transferred to a real robotic system. Methods 40 participants, medical students and, medical-PhD candidates, from one university hospital, were included. Baseline dexterity skills were measured through completion of a vesico-urethral anastomosis on a VR-robot simulator and the DaVinci robot. Participants were randomized into a video and control group. The video group practiced skills on the robot simulator with intermediate video revision, whereas the control group had intermediate pause instead. Post-intervention dexterity skills were measured using the same exercises as the baseline tests. Results No significant differences were found in baseline performance. Post-intervention results on the VR-simulator show that the video group commits significantly fewer injuries to the urethra and sutures at a greater optimal depth. The control group was significantly faster, had less camera travel, and had their instruments less out of view. On the DaVinci robot, participants in both groups performed significantly faster and had better GEARS-score after the training sessions on the VR simulator. Conclusions Video revision significantly improves the quality of robotic skills in novice surgeons on the VR-simulator, though at the expense of time. Furthermore, both groups demonstrated enhanced skills on the DaVinci robot after training sessions, which advocates the transferability of skill.

Self-assessment of Competence in Endoscopy: Challenges and Insights

BACKGROUND

Endoscopists use self-assessment to monitor the development and maintenance of their skills. The accuracy of these self-assessments, which reflects how closely one's own rating corresponds to an external rating, is unclear.

METHODS

In this narrative review, we critically examine the current literature on self-assessment in gastrointestinal endoscopy with the aim of informing training and practice and identifying opportunities to improve the methodological rigor of future studies.

RESULTS

In the seven included studies, the evidence regarding self-assessment accuracy was mixed. When stratified by experience level, however, novice endoscopists were least accurate in their self-assessments and tended to overestimate their performance. Studies examining the utility of video-based interventions using observation of expert benchmark performances show promise as a mechanism to improve self-assessment accuracy among novices.

CONCLUSIONS

Based on the results of this review, we highlight problematic areas, identify opportunities to improve the methodological rigor of future studies on endoscopic self-assessment and outline potential avenues for further exploration.

COVID-19 and Canadian Gastroenterology Trainees

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic has impacted endoscopy services and education worldwide. This study aimed to characterize the impact of COVID-19 on gastroenterology trainees in Canada.

METHODS

An analysis of Canadian respondents from the international EndoTrain survey, open from April 11 to May 2 2020 and distributed by program directors, trainees, and national and international gastroenterology societies' representatives, was completed. The survey included questions on monthly endoscopy volume, personal protective equipment availability, trainee well-being and educational resources. The primary outcome was change in procedural volume during the COVID-19 pandemic. Secondary outcomes included trainee's professional and personal concerns, anxiety and burnout.

RESULTS

Thirty-four Canadian trainees completed the survey. Per month, participants completed a median of 30 esophagogastroduodenoscopies (interquartile range 16 to 50) prior to the pandemic compared to 2 (0 to 10) during the pandemic, 20 (8 to 30) compared to 2 (0 to 5) colonoscopies and 3 (1 to 10) compared to 0 (0 to 3) upper gastrointestinal bleeding procedures. There was a significant decrease in procedural volumes between the pre-COVID-19 and COVID-19 time periods for all procedures (P < 0.001). Thirty (88%) trainees were concerned about personal COVID-19 exposure, 32 (94%) were concerned about achieving and/or maintaining clinical competence and 24 (71%) were concerned about prolongation of training time due to the pandemic. Twenty-six (79%) respondents experienced some degree of anxiety, and 10 (31%) experienced some degree of burnout.

CONCLUSION

The COVID-19 pandemic has substantially impacted gastroenterology trainees in Canada. As the pandemic eases, it important for gastrointestinal programs to adapt to maximize resident learning, maintain effective clinical care and ensure development of endoscopic competence.

Impact of a simulation-based ergonomics training curriculum on work-related musculoskeletal injury risk in colonoscopy

BACKGROUND AND AIMS

Endoscopists are at risk of developing musculoskeletal injuries (MSIs), and few receive training on ergonomics. The aim of this study was to determine the impact of a simulation-based ergonomics training curriculum (ETC) on work-related MSI risk during clinical colonoscopy.

METHODS

Novice endoscopists underwent a simulation-based ETC and were compared with an historical control group who received simulation-based training without ergonomics training. The ETC included a didactic lecture and video on ergonomics in colonoscopy, feedback from supervisors on ergonomics, and an ergonomics checklist to augment feedback and promote self-reflection. Participants were assessed using the rapid entire body assessment (REBA) and rapid upper limb assessment (RULA). The primary outcome was participants' REBA scores during 2 clinical colonoscopies 4 to 6 weeks after training.

RESULTS

In clinical colonoscopy, the ETC group had superior REBA scores (clinical procedure 1: median score, 6 vs 11; P < .001; clinical procedure 2: median score, 6 vs 10; P < .001). In a simulated colonoscopy, the ETC group did not have significantly different REBA or RULA scores between baseline, immediately after training, and 4 to 6 weeks after (REBA: median scores of 5, 5, and 5, respectively; P > .05; RULA: median scores of 6, 6, and 6, respectively; P > .05). The control group had worsening REBA and RULA scores during the study timeline (REBA: median scores of 5 at baseline, 9 immediately after training, and 9 at 4-6 weeks after training; P < .001; RULA: median scores of 6, 7, and 7, respectively; P = .04) during simulated procedures.

CONCLUSIONS

A simulation-based ETC is associated with reduced risk of MSI during endoscopy. Although the REBA score was improved, the intervention group was still within the medium-risk range.

Impact of COVID-19 on endoscopy trainees: an international survey

BACKGROUND AND AIMS

Although coronavirus disease 2019 (COVID-19) has affected endoscopy services globally, the impact on trainees has not been evaluated. We aimed to assess the impact of COVID-19 on procedural volumes and on the emotional well-being of endoscopy trainees worldwide.

METHODS

An international survey was disseminated over a 3-week period in April 2020. The primary outcome was the percentage reduction in monthly procedure volume before and during COVID-19. Secondary outcomes included potential variation of COVID-19 impact between different continents and rates and predictors of anxiety and burnout among trainees.

RESULTS

Across 770 trainees from 63 countries, 93.8% reported a reduction in endoscopy case volume. The median percentage reduction in total procedures was 99% (interquartile range, 85%-100%), which varied internationally (P < .001) and was greatest for colonoscopy procedures. Restrictions in case volume and trainee activity were common barriers. A total of 71.9% were concerned that the COVID-19 pandemic could prolonged training. Anxiety was reported in 52.4% of respondents and burnout in 18.8%. Anxiety was independently associated with female gender (odds ratio [OR], 2.15; P < .001), adequacy of personal protective equipment (OR, 1.75; P = .005), lack of institutional support for emotional health (OR, 1.67; P = .008), and concerns regarding prolongation of training (OR, 1.60; P = .013). Modifying existing national guidelines to support adequate endoscopy training during the pandemic was supported by 68.9%.

CONCLUSIONS

The COVID-19 pandemic has led to restrictions in endoscopic volumes and endoscopy training, with high rates of anxiety and burnout among endoscopy trainees worldwide. Targeted measures by training programs to address these key issues are warranted to improve trainee well-being and support trainee education.