A comparative study of skills in virtual laparoscopy and endoscopy

Resident Endoscopy Experience Correlates Poorly with Performance on a Virtual Reality Simulator

Background  Endoscopy training has become increasingly emphasized during general surgery residency as reflected by introduction of the Fundamentals of Endoscopic Surgery (FES) examination, which includes testing of skills on virtual reality (VR) simulators. Although studies exist to assess the ability of the simulator to differentiate between novices and experienced endoscopists, it is not well understood how simulators can differentiate skills among resident cohort.

Objective  To assess the utility of the VR simulator, we evaluated the correlation between resident endoscopy experience and performance on two VR simulator colonoscopy modules on the GI-BRONCH Mentor (Simbionix Ltd, Airport City, Israel).

Methods  Postgraduate years 2 to 5 residents completed “easy” and “difficult” VR colonoscopies, and performance metrics were recorded from October 2017 to February 2018 at Rutgers' two general surgery residency programs. Resident endoscopy experience was obtained through Accreditation Council for Graduate Medical Education case logs. Correlations between resident endoscopy experience and VR colonoscopy performance metrics were assessed using Spearman's rho (ρ) correlation statistic and bivariate logistic regression.

Results  Fifty-five residents out of 65 (84.6%) eligible participants completed the study. There were limited correlations found between resident endoscopy experience and FES performance metrics and no correlations were found between resident endoscopy experience and binary metrics of colonoscopy—ability to complete colonoscopy, ability to retroflex, and withdrawal time of less than 6 minutes.

Conclusion  The VR simulator may have a limited ability to discriminate between experience levels among resident cohort. Future studies are needed to further understand how well the VR simulator metrics correlate with resident endoscopy experience.

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Background

Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access.

Objective

This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence.

Methods

For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed.

Results

A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67).

Conclusions

The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

Correlation of surgical trainee performance on laparoscopic versus endoscopic simulation

BACKGROUND

Both laparoscopy and endoscopy are image-based procedures, which are less intuitive than traditional open surgery and require extensive training to reach adequate proficiency. Currently, there is lack of understanding as to how the skills in one image-based procedure translate to another, such as endoscopy to laparoscopy and vice versa. The aim of our study was to explore the relationship between endoscopic and laparoscopic skills using a Fundamentals of Laparoscopic Surgery (FLS) trainer, a traditional virtual reality endoscopic trainer and a "desk-top" endoscopic physical simulator.

METHODS

Senior surgical residents from across Canada participating in an advanced laparoscopic foregut training course were enrolled in the study. Participants were assessed performing the FLS laparoscopic suturing task, the Endobubble 2 task (Simbionix, GI Mentor), and a forward viewing peg transfer on the novel Basics in Endoscopic Skills Training Box (BEST Box).

RESULTS

There was significant correlation between the participant's skill in simulated laparoscopic suturing and simulated endoscopic skill using the BEST box (Pearson coefficient (r) was 0.551 (p = 0.033) and the coefficient of determination (r²) was 0.304). There was a trend towards correlation between laparoscopic suturing time and Endobubble 2 score, but this did not reach statistical significance (r = 0.458, p = 0.086; r² = 0.210).

CONCLUSIONS

Performance in the two physical simulators, laparoscopic suturing and simulated flexible endoscopy using the BEST box, showed a correlation. This study adds to the growing body of evidence that laparoscopic and endoscopic skills are complementary and has the potential to impact simulation training involving both skill sets.

Effect of a short preclinical laparoscopy course for interns in surgery: a randomized controlled trial

OBJECTIVES

Surgical interns are often not well prepared and have high anxiety about the execution of basic technical skills. This study investigates whether a short preclinical course focusing on laparoscopic camera-navigating skills is useful in the preparation for internship.

DESIGN

Through randomization, an experimental group who attended a short laparoscopic training session and a control group were created. Students' interest for this training and their confidence for laparoscopic exposure during surgical internship were inquired. During internship, camera-navigating skills were assessed by the operating surgeons (using a validated global rating scale) as well as by the students themselves (using a 10-points Likert scale).

SETTING

All research was performed in the Center for Surgical Technologies, Leuven, Belgium.

PARTICIPANTS

A total of 205 fifth-year medical students at the University of Leuven, Belgium.

RESULTS

Of the control group students, 80% were interested in attending the training session. There was no difference in confidence between experimental and control group. According to the surgeons and students, there was a significant improvement in clinical performance from the first (scores on global rating and Likert scales ±50%) to the last procedure (scores ±70%) for both groups. However, there was no difference in performance between groups.

CONCLUSIONS

Students are very interested in attending a preclinical laparoscopic training session. However, trained students did not display higher confidence or better clinical performance during internship. Even without previous training, students are fast to acquire the necessary skills during surgical internship.

Do basic psychomotor skills transfer between different image-based procedures?

Background

Surgical techniques that draw from multiple types of image-based procedures (IBP) are increasing, such as Natural Orifice Transluminal Endoscopic Surgery, fusing laparoscopy and flexible endoscopy. However, little is known about the relation between psychomotor skills for performing different types of IBP. For example, do basic psychomotor colonoscopy and laparoscopy skills interact?

Methods

Following a cross-over study design, 29 naïve endoscopists were trained on the Simbionix GI Mentor and the SimSurgery SEP simulators. Group C (n = 15) commenced with a laparoscopy session, followed by four colonoscopy sessions and a second laparoscopy session. Group L (n = 14) started with a colonoscopy session, followed by four laparoscopy sessions and a second colonoscopy session.

Results

No significant differences were found between the performances of group L and group C in their first training sessions on either technique. With additional colonoscopy training, group C outperformed group L in the second laparoscopy training session on the camera navigation task.

Conclusions

Overall, training in the basic colonoscopy tasks does not affect performance of basic laparoscopy tasks (and vice versa). However, to limited extent, training of basic psychomotor skills for colonoscopy do appear to contribute to the performance of angled laparoscope navigation tasks. Thus, training and assessment of IBP type-specific skills should focus on each type of tasks independently. Future research should further investigate the influence of psychometric abilities on the performance of IBP and the transfer of skills for physicians who are experienced in one IBP type and would like to become proficient in another type of IBP.

Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy

BACKGROUND

Simulators may improve the efficiency, safety, and quality of endoscopic training. However, no objective, reliable, and valid tool exists to assess clinical endoscopic skills. Such a tool to measure the outcomes of educational strategies is a necessity. This multicenter, multidisciplinary trial aimed to develop instruments for evaluating basic flexible endoscopic skills and to demonstrate their reliability and validity.

METHODS

The Global Assessment of Gastrointestinal Endoscopic Skills (GAGES) Upper Endoscopy (GAGES-UE) and Colonoscopy (GAGES-C) are rating scales developed by expert endoscopists. The GAGES scale was completed by the attending endoscopist (A) and an observer (O) in self-assessment (S) during procedures to establish interrater reliability (IRR, using the intraclass correlation coefficient [ICC]) and internal consistency (IC, using Cronbach's alpha). Instrumentation was evaluated when possible and correlated with total scores. Construct and external validity were examined by comparing novice (NOV) and experienced (EXP) endoscopists (Student's t-test). Correlations were calculated for GAGES-UE and GAGES-C with participants who had performed both.

RESULTS

For the 139 completed evaluations (60 NOV, 79 EXP), IRR (A vs. O) was 0.96 for GAGES-UE and 0.97 for GAGES-C. The IRR between S and A was 0.78 for GAGES-UE and 0.89 for GAGES-C. The IC was 0.89 for GAGES-UE, and 0.95 for GAGES-C. There were mean differences between the NOV and the EXP endoscopists for GAGE-UE (14.4 +/- 3.7 vs. 18.5 +/- 1.6; p < 0.001) and GAGE-C (11.8 +/- 3.8 vs. 18.8 +/- 1.3; p < 0.001). Good correlation was found between the scores for the GAGE-UE and the GAGE-C (r = 0.75; n = 37). Instrumentation, when performed, demonstrated correlations with total scores of 0.84 (GAGE-UE; n = 73) and 0.86 (GAGE-C; n = 45).

CONCLUSIONS

The GAGES-UE and GAGES-C are easy to administer and consistent and meet high standards of reliability and validity. They can be used to measure the effectiveness of simulator training and to provide specific feedback. The GAGES results can be generalized to North American and European endoscopists and may contribute to the definition of technical proficiency in endoscopy.

Virtual Reality Laparoscopic Simulator as an Aid in Surgical Resident Education: Two Years' Experience

Background and Aims:

Virtual reality (VR) laparoscopic simulator training has been shown to augment the learning of skills needed in real laparoscopic operations. We report here our two-year experience of using a VR simulator in the training of surgical residents.

Methods:

A VR laparoscopic simulator was purchased for Päijät-Häme Central Hospital at the end of November 2005. From 1 December 2005 onwards surgical residents in our hospital were encouraged to voluntarily practise with the VR simulator. After the VR simulator had been in use for two years all the data stored in the simulator's computer memory was collected and analysed.

Results:

In this two-year period a total of 79 persons practised with the simulator. The total number of performed tasks stored in the computer was 2,090. The training activity varied greatly between residents (6–171 tasks). The performance in simulator tasks differentiated between residents and GI surgeons. The learning curve of the residents in basic tasks was steep but their performance failed to reach the level of experienced laparoscopic surgeons in most tasks.

Conclusions:

The VR simulator was well used. Practising with a VR simulator on a voluntary basis can result in inadequate training. The VR laparoscopic simulator differentiates between subjects with different laparoscopic skills and shows good construct validity.

Learning curve and interobserver reproducibility evaluation of liver stiffness measurement by transient elastography

BACKGROUND/AIMS

Fibroscan allows liver stiffness examination (LSE) that is well correlated with fibrosis stages. Our main objective was to evaluate LSE learning curve.

METHODS

LSE results of five novice observers with different medical status were compared with those of five expert observers (physicians with >100 examinations) in 250 patients with chronic liver disease. Each novice-expert pair had to blindly examine 50 consecutive patients divided into five consecutive subgroups of 10 patients.

RESULTS

In each observer group, novice-expert agreement [intraclass correlation coefficient (Ric)] for LSE results was excellent from the first to the last subgroup. Novice-expert agreement for LSE results varied with liver stiffness level: <9 kPa: Ric=0.49; >or=9 kPa: Ric=0.87. Relative difference (%) between novice and expert LSE results was independently associated with the number of valid LSE measurements, and stabilizes around 20-30% after the fourth valid measurement. In each observer group, novice-expert agreement (Ric) for LSE success rate progressively increased as a function of time.

CONCLUSION

LSE requires no learning curve: a novice is able to obtain a reliable result after a single training session, whatever the professional status. However, success rate will progressively increase. An LSE with less than four valid measurements should not be considered as reliable.

Simulation and surgical training

The aim of this review was to outline current forms of surgical simulation and methods of assessing technical skills using these forms of simulation. To review this subject, a literature search was done using key words 'assessment', 'simulation', 'surgery', 'technical skills' and 'virtual reality'. Simulation in surgery has several forms, inorganic (synthetic & computer) and organic (animal or cadaver). Surgical simulation is a mode of training which is promising and may be effective. Technical errors in the simulated environment do not have clinical consequences and does not have a morbidity or mortality. We must ensure that the competent skills learnt in the simulation environment are translated to the real environment. This can be achieved if the same assessment tools are used in both environments. Surgical training is entering a new era, with increased scrutiny and an evolving work and training environment. We as surgical teachers must ensure that the surgeons of the future are as competent as or better than their predecessors using these new modes of training which we have access to.

Construct validity for eye-hand coordination skill on a virtual reality laparoscopic surgical simulator

BACKGROUND

This study was carried out to investigate whether eye-hand coordination skill on a virtual reality laparoscopic surgical simulator (the LAP Mentor) was able to differentiate among subjects with different laparoscopic experience and thus confirm its construct validity.

METHODS

A total of 31 surgeons, who were all right-handed, were divided into the following two groups according to their experience as an operator in laparoscopic surgery: experienced surgeons (more than 50 laparoscopic procedures) and novice surgeons (fewer than 10 laparoscopic procedures). The subjects were tested using the eye-hand coordination task of the LAP Mentor, and performance was compared between the two groups. Assessment of the laparoscopic skills was based on parameters measured by the simulator.

RESULTS

The experienced surgeons completed the task significantly faster than the novice surgeons. The experienced surgeons also achieved a lower number of movements (NOM), better economy of movement (EOM) and faster average speed of the left instrument than the novice surgeons, whereas there were no significant differences between the two groups for the NOM, EOM and average speed of the right instrument.

CONCLUSIONS

Eye-hand coordination skill of the nondominant hand, but not the dominant hand, measured using the LAP Mentor was able to differentiate between subjects with different laparoscopic experience. This study also provides evidence of construct validity for eye-hand coordination skill on the LAP Mentor.

Virtual Reality Simulation Training can Improve Inexperienced Surgeons' Endovascular Skills

PURPOSE

The aim of this study was to evaluate virtual reality (VR) simulation for endovascular training of surgeons inexperienced in this technique.

METHODS

Twenty consultant vascular surgeons were divided into those who had performed >50 endovascular procedures (e.g. aortic and carotid stent) as primary operator (n=8), and those having performed <10 procedures (n=12). To test for endovascular skill rather than procedural knowledge, all subjects performed a renal artery balloon angioplasty and stent procedure. The simulator uses real tools with active force feedback, and provides a realistic image of the virtual patient. Surgeons with endovascular skills performed two repetitions and those without completed six repetitions of the same task. The simulator recorded time taken for the procedure, the amount of contrast fluid used and total fluoroscopy time.

RESULTS

Initially, surgeons with endovascular skills were significantly faster (median 571.5 vs. 900.0 s, p=0.039) and used less contrast fluid (19.1 vs. 42.9 ml, p=0.047) than inexperienced operators, though differences for fluoroscopy time were not significant (273 vs. 441 s, p=0.305). Over the six sessions, the inexperienced group made significant improvements in performance for time taken (p=0.007) and contrast fluid usage (p=0.021), achieving similar scores at the end of the training program to the experienced group.

CONCLUSIONS

Surgeons with minimal endovascular experience can improve their time taken and contrast usage during short-phase training on a VR endovascular task. VR simulation may be useful for the early part of the learning curve for surgeons who wish to expand their endovascular interests.

Objective assessment of visuospatial and psychomotor ability and flow of residents and senior endoscopists in simulated gastroscopy

BACKGROUND

Advanced medical simulators have predominantly been used to shorten the learning curve of endoscopy for medical students and young residents. Rarely have the effects of visuospatial ability and attitudes of intermediately experienced and experienced specialists been studied with regard to simulator training. The aim of this study was to assess the effects of visuospatial ability and attitude on performance in simulator training.

METHODS

Eighteen surgical residents were included in the study. Prior to the simulated gastroscopy task, they performed a visuospatial test (the card rotation test). After the simulated gastroscopy task, they completed a questionnaire regarding flow experiences. Their results were compared with those of 11 expert endoscopists who performed the same tests.

RESULTS

Total gastroscopy time was significantly shorter for the expert endoscopists compared to residents (2 min 11 sec, p = 0.003). There was also a trend of more mucosa inspected (p = 0.088) and higher efficiency of screening (p = 0.069) by the experts. The residents made fewer errors in the card rotation test than the expert endoscopists (2.5 +/- 0.8 vs 5.5 +/- 1.2, respectively; p = 0.034), and their visuospatial card rotation test results correlated better with their performance in the simulated gastroscopy.

CONCLUSIONS

A virtual gastroscopy task presents more of an emotional as well as a psychomotoric challenge to intermediately experienced endoscopists than to senior experts. Our study demonstrates that these differences can be objectively assessed by the use of visuospatial ability tests, flowsheets, and an endoscopic simulator.

Consensus guidelines for validation of virtual reality surgical simulators

The Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes is a newly formed sub-group of the European Association of Endoscopic Surgeons (EAES). This work group undertook a review of validation evidence for surgical simulators and the resulting consensus is presented in this article. Using clinical guidelines criteria, the evidence for validation for six different simulators was rated and subsequently translated to a level of recommendation for each system. The simulators could be divided into two basic types; systems for laparoscopic general surgery and flexible gastrointestinal endoscopy. Selection of simulators for inclusion in this consensus was based on their availability and relatively widespread usage as of July 2004. Whilst level 2 recommendations were achieved for a few systems, it was clear that there was an overall lack of published validation studies with rigorous experimental methodology. Since the consensus meeting, there have been a number of new articles, system upgrades and new devices available. The work group intends to update these consensus guidelines on a regular basis, with the resulting article available on the EAES website (http://www.eaes-eur.org ).