The effect of model fidelity on colonoscopic skills acquisition. A randomized controlled study

W, A. K. The story so far………- current opinion in the use and applications of interactive storytelling in physiology and clinical education

Physiology and clinical practice are subjects of study which demand integration of multiple sources of systems working knowledge and information on the performance of those systems to come to meaningful conclusions. This is made more complex by the interpretation and actions as a result of this conclusion having direct impact on the sum of the component systems, the human, thereby integrating significant social and psychological considerations into an already complex situation. As higher education educators, it is a significant challenge to provide our learners with training and most importantly, practice, in these knowledge, skills and behaviours in the classroom. There has been a significant interest in recent years in providing active learning opportunities which allow learners to apply subject knowledge to multi-faceted, immersive, continuously evolving stories which reflect a graduate's professional aspirations. This review highlights practices from the literature of storytelling education which the higher education educator can utilise in promoting "meaning making" in the classroom. Here, the case for interactive storytelling in physiology and clinical education is argued, as well as presenting commonly utilised techniques and practices with which educators can embed storytelling into their pedagogy as well as highlighting future directions in this field.

Vesicovaginal Fistula Repair Simulation Model and Hierarchical Task Analysis

IMPORTANCE

There is a need for surgeons skilled in vesicovaginal fistula (VVF) repair, yet training opportunities are limited.

OBJECTIVES

This study aimed to create a low-fidelity simulation model for transvaginal VVF repair, identify essential steps of VVF repair, and evaluate the model's ability to replicate essential steps.

STUDY DESIGN

First, a low-fidelity VVF repair simulation model was designed and built by the authors. Next, a hierarchical task analysis was performed by urogynecologic surgeons with expertise in VVF repair. Each expert submitted an outline of tasks required to perform VVF repair. To control for bias, an education specialist de-identified, reviewed, and collated the submitted outlines. The education specialist then led a focus group, and through a modified Delphi process, the experts reached consensus on the essential steps. A separate group of urogynecologic surgeons then tested the model and completed an anonymous questionnaire assessing how well the model replicated the essential steps. Descriptive analyses were performed.

RESULTS

Five experts submitted an outline of steps for transvaginal VVF repair, and 4 experts participated in a focus group to reach consensus on the essential steps. Nine urogynecologic surgeons, with a median of 10 years in practice (interquartile range, 7-12 years), tested the model and completed the postsimulation questionnaire. Most testers thought that tasks involving identification and closure of the fistula were replicated by the model. Testers thought that tasks involving cystoscopy or bladder filling were not replicated by the model.

CONCLUSIONS

We developed a novel, low-fidelity transvaginal VVF repair simulation model that consistently replicated tasks involving identification and closure of the fistula.

Simulator Fidelity Does Not Affect Training for Robot-Assisted Minimally Invasive Surgery

This study was undertaken to compare performance using a surgical robot after training with one of three simulators of varying fidelity. Methods: Eight novice operators and eight expert surgeons were randomly assigned to one of three simulators. Each participant performed two exercises using a simulator and then using a surgical robot. The primary outcome of this study is performance assessed by time and GEARS score. Results: Participants were randomly assigned to one of three simulators. Time to perform the suturing exercise (novices vs. experts) was significantly different for all 3 simulators. Using the da Vinci robot, peg transfer showed no significant difference between novices and experts and all participants combined (mean time novice 2.00, expert 2.21, p = 0.920). The suture exercise had significant differences in each group and all participants combined (novice 3.54, expert 1.90, p = 0.001). ANOVA showed p-Values for suturing (novice 0.523, expert 0.123) and peg transfer (novice 0.742, expert 0.131) are not significantly different. GEARS scores were different (p < 0.05) for novices and experts. Conclusion: Training with simulators of varying fidelity result in similar performance using the da Vinci robot. A dry box simulator may be as effective as a virtual reality simulator for training. Further studies are needed to validate these results.

Outcomes, Measurement Instruments, and Their Validity Evidence in Randomized Controlled Trials on Virtual, Augmented, and Mixed Reality in Undergraduate Medical Education: Systematic Mapping Review

BACKGROUND

Extended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence.

OBJECTIVE

Our aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education.

METHODS

We conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants' attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%).

CONCLUSIONS

RCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

A systematic review of low-cost simulators in ENT surgery

BACKGROUND

Simulation training has become a key part of the surgical curriculum over recent years. Current trainees face significantly reduced operating time as a result of the coronavirus disease 2019 pandemic, alongside increased costs to surgical training, thus creating a need for low-cost simulation models.

METHODS

A systematic review of the literature was performed using multiple databases. Each model included was assessed for the ease and expense of its construction, as well as its validity and educational value.

RESULTS

A total of 18 low-cost simulation models were identified, relating to otology, head and neck surgery, laryngeal surgery, rhinology, and tonsil surgery. In only four of these models (22.2 per cent) was an attempt made to demonstrate the educational impact of the model. Validation was rarely formally assessed.

CONCLUSION

More efforts are required to standardise validation methods and demonstrate the educational value of the available low-cost simulation models in otorhinolaryngology.

Using Fully-Immersive Simulation Training with Structured Debrief to Improve Nontechnical Skills in Emergency Endovascular Surgery

OBJECTIVE

Assess whether fully-immersive simulation training with structured debriefing of a standardized emergency thoracic endovascular aortic repair (TEVAR) scenario improves team-work performance of the lead surgeon. Secondary aims: assess whether technical skills (TS) and radiation safety behaviors (RSB) improved concurrently.

DESIGN

Pre-post study.

SETTING

UK-based training days.

PARTICIPANTS

General and vascular surgical trainees (n = 16).

INTERVENTION(S)

Fully-immersive simulation training with structured debriefing of a standardized emergency TEVAR scenario. Following standardized emergency TEVAR technical training, trainees led 2 standardized fully-immersive endovascular surgery simulations, with scripted support from a passive surgical team. A non-TS (NTS) structured debrief was delivered following simulations. NTS were assessed using the validated observational teamwork assessment for surgery tool post hoc using video recordings of simulations. TS were assessed through time taken to complete each step of the procedure, as defined during technical training. RSB were assessed through checking for presence of pre-defined actions and the length of time fluoroscopy was used during each simulation.

RESULTS

Total observational teamwork assessment for surgery scores improved following structured debrief (p = 0.005, median 52.55/90 vs 73.0/90), alongside all constituent domains - communication (p < 0.001, median 11.7/20 vs 16.6/20), coordination (p < 0.001, median 8.6/15 vs 13.4/15), cooperation (p < 0.001, median 13.15/20 vs 16.35/20), leadership (p < 0.001, median 8.70/15 vs 11.30/15) and monitoring (p < 0.001, median 9.85/20 vs 14.85/20). TS improved; time to complete 12 of 13 procedural steps improved (p < 0.027). Fluoroscopy time (seconds) decreased (p = 0.339, 543.6 vs 495.5), frequency lead surgeons checked the team were wearing leads increased (p = 0.125, 3 vs 7) and asked the team to step back before screening increased (p = 0.003, frequency team asked to step back/total angiography runs before = 2/36 vs after = 14/44).

CONCLUSIONS

fully-immersive endovascular simulation with structured debrief is a robust tool to improve NTS and TS. Incorporation into surgical training may reduce operating theatres errors, increase efficiency, and improve RSB. However effective translation into the clinical workplace must be demonstrated to see these benefits.

Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy

BACKGROUND

Endoscopy has traditionally been taught with novices practicing on real patients under the supervision of experienced endoscopists. Recently, the growing awareness of the need for patient safety has brought simulation training to the forefront. Simulation training can provide trainees with the chance to practice their skills in a learner-centred, risk-free environment. It is important to ensure that skills gained through simulation positively transfer to the clinical environment. This updated review was performed to evaluate the effectiveness of virtual reality (VR) simulation training in gastrointestinal endoscopy.

OBJECTIVES

To determine whether virtual reality simulation training can supplement and/or replace early conventional endoscopy training (apprenticeship model) in diagnostic oesophagogastroduodenoscopy, colonoscopy, and/or sigmoidoscopy for health professions trainees with limited or no prior endoscopic experience.

SEARCH METHODS

We searched the following health professions, educational, and computer databases until 12 July 2017: the Cochrane Central Register of Controlled Trials, Ovid MEDLINE, Ovid Embase, Scopus, Web of Science, BIOSIS Previews, CINAHL, AMED, ERIC, Education Full Text, CBCA Education, ACM Digital Library, IEEE Xplore, Abstracts in New Technology and Engineering, Computer and Information Systems Abstracts, and ProQuest Dissertations and Theses Global. We also searched the grey literature until November 2017.

SELECTION CRITERIA

We included randomised and quasi-randomised clinical trials comparing VR endoscopy simulation training versus any other method of endoscopy training with outcomes measured on humans in the clinical setting, including conventional patient-based training, training using another form of endoscopy simulation, or no training. We also included trials comparing two different methods of VR training.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed the eligibility and methodological quality of trials, and extracted data on the trial characteristics and outcomes. We pooled data for meta-analysis where participant groups were similar, studies assessed the same intervention and comparator, and had similar definitions of outcome measures. We calculated risk ratio for dichotomous outcomes with 95% confidence intervals (CI). We calculated mean difference (MD) and standardised mean difference (SMD) with 95% CI for continuous outcomes when studies reported the same or different outcome measures, respectively. We used GRADE to rate the quality of the evidence.

MAIN RESULTS

We included 18 trials (421 participants; 3817 endoscopic procedures). We judged three trials as at low risk of bias. Ten trials compared VR training with no training, five trials with conventional endoscopy training, one trial with another form of endoscopy simulation training, and two trials compared two different methods of VR training. Due to substantial clinical and methodological heterogeneity across our four comparisons, we did not perform a meta-analysis for several outcomes. We rated the quality of evidence as moderate, low, or very low due to risk of bias, imprecision, and heterogeneity.Virtual reality endoscopy simulation training versus no training: There was insufficient evidence to determine the effect on composite score of competency (MD 3.10, 95% CI -0.16 to 6.36; 1 trial, 24 procedures; low-quality evidence). Composite score of competency was based on 5-point Likert scales assessing seven domains: atraumatic technique, colonoscope advancement, use of instrument controls, flow of procedure, use of assistants, knowledge of specific procedure, and overall performance. Scoring range was from 7 to 35, a higher score representing a higher level of competence. Virtual reality training compared to no training likely provides participants with some benefit, as measured by independent procedure completion (RR 1.62, 95% CI 1.15 to 2.26; 6 trials, 815 procedures; moderate-quality evidence). We evaluated overall rating of performance (MD 0.45, 95% CI 0.15 to 0.75; 1 trial, 18 procedures), visualisation of mucosa (MD 0.60, 95% CI 0.20 to 1.00; 1 trial, 55 procedures), performance time (MD -0.20 minutes, 95% CI -0.71 to 0.30; 2 trials, 29 procedures), and patient discomfort (SMD -0.16, 95% CI -0.68 to 0.35; 2 trials, 145 procedures), all with very low-quality evidence. No trials reported procedure-related complications or critical flaws (e.g. bleeding, luminal perforation) (3 trials, 550 procedures; moderate-quality evidence).Virtual reality endoscopy simulation training versus conventional patient-based training: One trial reported composite score of competency but did not provide sufficient data for quantitative analysis. Virtual reality training compared to conventional patient-based training resulted in fewer independent procedure completions (RR 0.45, 95% CI 0.27 to 0.74; 2 trials, 174 procedures; low-quality evidence). We evaluated performance time (SMD 0.12, 95% CI -0.55 to 0.80; 2 trials, 34 procedures), overall rating of performance (MD -0.90, 95% CI -4.40 to 2.60; 1 trial, 16 procedures), and visualisation of mucosa (MD 0.0, 95% CI -6.02 to 6.02; 1 trial, 18 procedures), all with very low-quality evidence. Virtual reality training in combination with conventional training appears to be advantageous over VR training alone. No trials reported any procedure-related complications or critical flaws (3 trials, 72 procedures; very low-quality evidence).Virtual reality endoscopy simulation training versus another form of endoscopy simulation: Based on one study, there were no differences between groups with respect to composite score of competency, performance time, and visualisation of mucosa. Virtual reality training in combination with another form of endoscopy simulation training did not appear to confer any benefit compared to VR training alone.Two methods of virtual reality training: Based on one study, a structured VR simulation-based training curriculum compared to self regulated learning on a VR simulator appears to provide benefit with respect to a composite score evaluating competency. Based on another study, a progressive-learning curriculum that sequentially increases task difficulty provides benefit with respect to a composite score of competency over the structured VR training curriculum.

AUTHORS' CONCLUSIONS

VR simulation-based training can be used to supplement early conventional endoscopy training for health professions trainees with limited or no prior endoscopic experience. However, we found insufficient evidence to advise for or against the use of VR simulation-based training as a replacement for early conventional endoscopy training. The quality of the current evidence was low due to inadequate randomisation, allocation concealment, and/or blinding of outcome assessment in several trials. Further trials are needed that are at low risk of bias, utilise outcome measures with strong evidence of validity and reliability, and examine the optimal nature and duration of training.

Endoscopic training: A nationwide survey of French fellows in gastroenterology

BACKGROUND

During their 4 years of training, French fellows in gastroenterology should acquire theoretical and practical competency in gastrointestinal (GI) endoscopy.

AIMS

To evaluate the delivery of endoscopy training to French GI fellows and perception of learning.

METHODS

A nationwide electronic survey was carried out of French GI fellows using an anonymous, 17-item electronic questionnaire.

RESULTS

A total of 291 out of 484 (60%) GI fellows responded to the survey. Only 40% of subjects had access to theoretical training and/or virtual simulators. Only 49% and 35% of fourth year fellows had reached the threshold numbers of EGD and colonoscopies recommended by the European section and Board of gastroenterology and hepatology. Sixty-two percent and 57% of trainees reported having insufficient knowledge in interpreting gastric and colic lesions. Access to dedicated endoscopy activity for at least 8 weeks during the year was the only independent factor associated with the achievement of the recommended annual threshold number of procedures.

CONCLUSION

The access of fellows to theoretical training and to preclinical virtual simulators is still insufficient. Personalized support and regular assessment of cognitive and technical acquisition over the 4 years of training seems to be necessary.

Simulator training in gastrointestinal endoscopy - From basic training to advanced endoscopic procedures

Simulator-based gastrointestinal endoscopy training has gained acceptance over the last decades and has been extensively studied. Several types of simulators have been validated and it has been demonstrated that the use of simulators in the early training setting accelerates the learning curve in acquiring basic skills. Current GI endoscopy simulators lack the degree of realism that would be necessary to provide training to achieve full competency or to be applicable in certification. Virtual Reality and mechanical simulators are commonly used in basic flexible endoscopy training, whereas ex vivo and in vivo models are used in training the most advanced endoscopic procedures. Validated models for the training of more routine therapeutic interventions like polypectomy, EMR, stenting and haemostasis are lacking or scarce and developments in these areas should be encouraged.

Training and competence assessment in GI endoscopy: a systematic review

INTRODUCTION

Training procedural skills in GI endoscopy once focused on threshold numbers. As threshold numbers poorly reflect individual competence, the focus gradually shifts towards a more individual approach. Tools to assess and document individual learning progress are being developed and incorporated in dedicated training curricula. However, there is a lack of consensus and training guidelines differ worldwide, which reflects uncertainties on optimal set-up of a training programme.

AIMS

The primary aim of this systematic review was to evaluate the currently available literature for the use of training and assessment methods in GI endoscopy. Second, we aimed to identify the role of simulator-based training as well as the value of continuous competence assessment in patient-based training. Third, we aimed to propose a structured training curriculum based on the presented evidence.

METHODS

A literature search was carried out in the available medical and educational literature databases. The results were systematically reviewed and studies were included using a predefined protocol with independent assessment by two reviewers and a final consensus round.

RESULTS

The literature search yielded 5846 studies. Ninety-four relevant studies on simulators, assessment methods, learning curves and training programmes for GI endoscopy met the inclusion criteria. Twenty-seven studies on simulator validation were included. Good validity was demonstrated for four simulators. Twenty-three studies reported on simulator training and learning curves, including 17 randomised control trials. Increased performance on a virtual reality (VR) simulator was shown in all studies. Improved performance in patient-based assessment was demonstrated in 14 studies. Four studies reported on the use of simulators for assessment of competence levels. Current simulators lack the discriminative power to determine competence levels in patient-based endoscopy. Eight out of 14 studies on colonoscopy, endoscopic retrograde cholangiopancreatography and endosonography reported on learning curves in patient-based endoscopy and proved the value of this approach for measuring performance. Ten studies explored the numbers needed to gain competence, but the proposed thresholds varied widely between them. Five out of nine studies describing the development and evaluation of assessment tools for GI endoscopy provided insight into the performance of endoscopists. Five out of seven studies proved that intense training programmes result in good performance.

CONCLUSIONS

The use of validated VR simulators in the early training setting accelerates the learning of practical skills. Learning curves are valuable for the continuous assessment of performance and are more relevant than threshold numbers. Future research will strengthen these conclusions by evaluating simulation-based as well as patient-based training in GI endoscopy. A complete curriculum with the assessment of competence throughout training needs to be developed for all GI endoscopy procedures.

Simulated colonoscopy training using a low-cost physical model improves responsiveness of surgery interns

AIM

Surgery residents are required to become proficient in colonoscopy before completing training. The aim of this study was to evaluate the responsiveness of surgery interns to simulated colonoscopy training.

METHOD

Interns, defined as postgraduate year 1 residents without exposure to endoscopy, underwent training in a physical model including colonoscopy, synthetic anatomy trays with luminal tattoos and a hybrid simulator. After baseline testing and mentored training, final testing was performed using five predetermined proficiency criteria. Content-valid metrics defined by the extent of departure from clinical reality were evaluated by two blinded assessors. Responsiveness was defined as change in performance over time and assessed comparing baseline testing with nonmentored final testing.

RESULTS

Twelve interns (eight male, mean age 26, 80% right-handed) performed 48 colonoscopies each over 1 year. Improvement was seen in the overall procedure time (24 min 46 s vs 20 min 54 s; P = 0.03), passing the splenic flexure (20 min 33 s vs 10 min 45 s; P = 0.007), passing the hepatic flexure (23 min 31 s vs 12 min 45 s; P = 0.003), caecal intubation time (23 min 38 s vs 13 min 26 s; P = 0.008), the duration of loss of view of the lumen (75% vs 8.3%; P = 0.023), incomplete colonoscopy (100% vs 33.3%; P = 0.042), colonoscope withdrawal < 6 min (16.7% vs 8.3%; P = 0.052). Tattoo identification time (9 min 16 s vs 12 min 25 s; P = 0.50), colon looped time (2 min 12 s vs 1 min 45 s; P = 0.50) and rate of colon perforation (8.3% vs 8.3%; P = 1) remained unchanged. Interrater reliability was 1.0 for all measures.

CONCLUSION

Simulated colonoscopy training in a low-cost physical model improved the performance of surgery interns with decreased procedure time, increased rates of complete colonoscopy and appropriate scope withdrawal.

Simulated colonoscopy training leads to improved performance during patient-based assessment

BACKGROUND

Virtual reality (VR) endoscopy simulators are increasingly being used in the training of novice endoscopists. There are, however, insufficient data regarding the effect of simulator training on the early learning curve of novice endoscopists.

OBJECTIVE

The aim of this study was to assess the clinical performance of novice endoscopists during colonoscopy after intensive and prolonged training on a VR endoscopy simulator.

DESIGN

Prospective study.

SETTING

Single university medical center.

PATIENTS

Patient-based assessment (PBA) of performance was carried out on patients routinely scheduled for colonoscopy.

INTERVENTIONS

Eighteen trainees without any endoscopic experience were included in the study. They were divided into 2 groups. The simulator-training program consisted of either 50 (group I) or 100 (group II) VR colonoscopies. After 10, 30, and 50 (group I) and after 20, 60, and 100 (group II) VR colonoscopies, trainees underwent both simulator-based assessment and PBA.

MAIN OUTCOME MEASUREMENTS

Cecal intubation time, colonic insertion depth, and cecal intubation rate.

RESULTS

Eighteen novices participated in the study. All completed VR training and assessments. The mean cecal intubation time on the SBA decreased from a baseline of 9.50 minutes to 2.20 minutes at completion of the training (P = .002). Colonic insertion depth during PBA improved from 29.4 cm to 63.7 cm (P < .001). The learning effect of simulator training ceased after 60 colonoscopies.

LIMITATIONS

Single-center study, no formal sample size calculation.

CONCLUSIONS

VR training by using a colonoscopy simulator leads to a significant improvement in performance with the simulator itself and, more importantly, to significantly improved performances during patient-based colonoscopy. This study demonstrates the rationale for intensive simulator training in the early learning curve of novices performing colonoscopy.

Effects of simulation-based training in gastrointestinal endoscopy: a systematic review and meta-analysis

BACKGROUND & AIMS

Simulation-based training (SBT) in gastrointestinal endoscopy has been increasingly adopted by gastroenterology fellowship programs. However, the effectiveness of SBT in enhancing trainee skills remains unclear. We performed a systematic review with a meta-analysis of published literature on SBT in gastrointestinal endoscopy.

METHODS

We performed a systematic search of multiple electronic databases for all original studies that evaluated SBT in gastrointestinal endoscopy in comparison with no intervention or alternative instructional approaches. Outcomes included skills (in a test setting), behaviors (in clinical practice), and effects on patients. We pooled effect size (ES) using random-effects meta-analysis.

RESULTS

From 10,903 articles, we identified 39 articles, including 21 randomized trials of SBT, enrolling 1181 participants. Compared with no intervention (n = 32 studies), SBT significantly improved endoscopic process skills in a test setting (ES, 0.79; n = 22), process behaviors in clinical practice (ES, 0.49; n = 8), time to procedure completion in both a test setting (ES, 0.79; n = 16) and clinical practice (ES, 0.75; n = 5), and patient outcomes (procedural completion and risk of major complications; ES, 0.45; n = 10). Only 5 studies evaluated the comparative effectiveness of different SBT approaches; which provided inconclusive evidence regarding feedback and simulation modalities.

CONCLUSIONS

Simulation-based education in gastrointestinal endoscopy is associated with improved performance in a test setting and in clinical practice, and improved patient outcomes compared with no intervention. Comparative effectiveness studies of different simulation modalities are limited.