Use of Augmented Reality and Virtual Reality Technologies in Endoscopic Training

Impact and assessment of training models in interventional endoscopic ultrasound

Interventional endoscopic ultrasound (IEUS) has gained significant popularity in recent years because of its diagnostic and therapeutic capabilities. The proper training of endoscopists is critical to ensure safe and effective procedures. This review study aims to assess the impact of different training models on the competence of trainees performing IEUS. Eight studies that evaluated simulators for IEUS were identified in the medical literature. Various training models have been used, including the EASIE-R, Mumbai EUS, EUS Magic Box, EndoSim, Thai Association for Gastrointestinal Endoscopy model, and an ex vivo porcine model (HiFi SAM). The trainees underwent traditional didactic lectures, hands-on training using simulators, and direct supervision by experienced endoscopists. The effectiveness of these models has been evaluated based on objective and subjective parameters such as technical proficiency, operative time, diagnostic success, and participant feedback. As expected, the majority of skills were improved after the training sessions concluded, although the risk of bias is high in the absence of external validation. It is difficult to determine the ideal simulator among the existing ones because of the wide variation between them in terms of costs, reusability, design, fidelity of anatomical structures and feedback, and types of procedures performed. There is a need for a standardized approach for the evaluation of IEUS simulators and the ways skills are acquired by trainees, as well as a clearer definition of the key personal attributes necessary for developing a physician into a skilled endoscopist capable of performing basic and advanced therapeutic EUS interventions.

Simulation Method for the Physical Deformation of a Three-Dimensional Soft Body in Augmented Reality-Based External Ventricular Drainage

OBJECTIVES

Intraoperative navigation reduces the risk of major complications and increases the likelihood of optimal surgical outcomes. This paper presents an augmented reality (AR)-based simulation technique for ventriculostomy that visualizes brain deformations caused by the movements of a surgical instrument in a three-dimensional brain model. This is achieved by utilizing a position-based dynamics (PBD) physical deformation method on a preoperative brain image.

METHODS

An infrared camera-based AR surgical environment aligns the real-world space with a virtual space and tracks the surgical instruments. For a realistic representation and reduced simulation computation load, a hybrid geometric model is employed, which combines a high-resolution mesh model and a multiresolution tetrahedron model. Collision handling is executed when a collision between the brain and surgical instrument is detected. Constraints are used to preserve the properties of the soft body and ensure stable deformation.

RESULTS

The experiment was conducted once in a phantom environment and once in an actual surgical environment. The tasks of inserting the surgical instrument into the ventricle using only the navigation information presented through the smart glasses and verifying the drainage of cerebrospinal fluid were evaluated. These tasks were successfully completed, as indicated by the drainage, and the deformation simulation speed averaged 18.78 fps.

CONCLUSIONS

This experiment confirmed that the AR-based method for external ventricular drain surgery was beneficial to clinicians.

Endoscopy training in COVID-19: Challenges and hope for a better age

The COVID-19 pandemic is a unique challenge that has disrupted endoscopy training. Initial infection control measures aimed at protecting patients and staff meant nonessential endoscopic activity was suspended in many countries. The decrease in elective caseload from the pandemic also reduced training numbers during this period. While hands-on training took a backseat, more efforts were directed to didactic training of cognitive competencies. We review the literature describing the impact of COVID-19 on endoscopy training and summarize key measures aimed at mitigating this effect. These include leveraging on web-based didactic material and video-conferences, increased use of simulation and models to hone technical competencies, and a shift in focus from numbers-based accreditation to competency-based accreditation. While COVID-19 was hoped to be short-lived, it is clear the impact is long-lasting. Hence, it is crucial for training programs to take stock of how endoscopy training is evolving and use this opportunity to implement new paradigms into their endoscopic training curricula. COVID-19 might just be the catalyst that transforms endoscopy training into a new digital era.

360° video recording inside a GI endoscopy room: Technical feasibility and its potential use for the acquisition of gastrointestinal endoscopy skills. Pilot experience

New advances in video processing, 3-dimensional designs, and augmented/virtual reality are exciting and evolving fields. These new tools can facilitate the learning phase of basic or advanced endoscopic procedures. Herein, we explain our initial experience, creating an immersive virtual reality (IVR) by using 360-degree recording videos from an interventional endoscopy room. Some common terms used around this technology, such as Augmented reality (AR), Virtual Reality (VR), Three hundred sixty videos, and Mixed Reality (MR), are discussed below. Three examples of VR 360 endoscopic room videos are included in this article.

SlicerVR for Medical Intervention Training and Planning in Immersive Virtual Reality

Virtual reality (VR) provides immersive visualization that has proved to be useful in a variety of medical applications. Currently, however, no free open-source software platform exists that would provide comprehensive support for translational clinical researchers in prototyping experimental VR scenarios in training, planning or guiding medical interventions. By integrating VR functions in 3D Slicer, an established medical image analysis and visualization platform, SlicerVR enables virtual reality experience by a single click. It provides functions to navigate and manipulate the virtual scene, as well as various settings to abate the feeling of motion sickness. SlicerVR allows for shared collaborative VR experience both locally and remotely. We present illustrative scenarios created with SlicerVR in a wide spectrum of applications, including echocardiography, neurosurgery, spine surgery, brachytherapy, intervention training and personalized patient education. SlicerVR is freely available under BSD type license as an extension to 3D Slicer and it has been downloaded over 7,800 times at the time of writing this article.

Immersive Virtual Colonoscopy

Virtual colonoscopy (VC) is a non-invasive screening tool for colorectal polyps which employs volume visualization of a colon model reconstructed from a CT scan of the patient's abdomen. We present an immersive analytics system for VC which enhances and improves the traditional desktop VC through the use of VR technologies. Our system, using a head-mounted display (HMD), includes all of the standard VC features, such as the volume rendered endoluminal fly-through, measurement tool, bookmark modes, electronic biopsy, and slice views. The use of VR immersion, stereo, and wider field of view and field of regard has a positive effect on polyp search and analysis tasks in our immersive VC system, a volumetric-based immersive analytics application. Navigation includes enhanced automatic speed and direction controls, based on the user's head orientation, in conjunction with physical navigation for exploration of local proximity. In order to accommodate the resolution and frame rate requirements for HMDs, new rendering techniques have been developed, including mesh-assisted volume raycasting and a novel lighting paradigm. Feedback and further suggestions from expert radiologists show the promise of our system for immersive analysis for VC and encourage new avenues for exploring the use of VR in visualization systems for medical diagnosis.

Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of medical imaging. Radiology departments have begun exploring the use of these technologies to help with radiology education and clinical care. The purpose of this review article is to summarize how three institutions have explored using virtual and augmented reality for radiology.

Endoscopy training in Korea

It is essential to maintain high-quality endoscopy given the increasing number of endoscopic screens performed in Korea. The training of fellows to perform endoscopies is challenging. The rapid development of endoscopic techniques and rising patient complexity increase the training pressures. At the end of training, all practitioners must perform endoscopy safely and effectively. Here, we examine the current status of endoscopy training in Korea. Although our system produces many competent endoscopists, there is room for improvement. Formal training programs should be developed to train the trainers. Specific assessment tools measuring performance and improving training are required. Changes should be made at all levels to improve our endoscopy training system.