Teleproctoring for Neurovascular Procedures: Demonstration of Concept Using Optical See-Through Head-Mounted Display, Interactive Mixed Reality, and Virtual Space Sharing-A Critical Need Highlighted by the COVID-19 Pandemic

Magic Leap 1 versus Microsoft HoloLens 2 for the Visualization of 3D Content Obtained from Radiological Images

The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the most popular MR head-mounted displays, Magic Leap 1 and Microsoft HoloLens 2, for the visualization of 3D medical imaging data. We evaluate the functionalities and performance of both devices through a user-study in which surgeons and residents assessed the visualization of 3D computer-generated anatomical models. The digital content is obtained through a dedicated medical imaging suite (Verima imaging suite) developed by the Italian start-up company (Witapp s.r.l.). According to our performance analysis in terms of frame rate, there are no significant differences between the two devices. The surgical staff expressed a clear preference for Magic Leap 1, particularly for the better visualization quality and the ease of interaction with the 3D virtual content. Nonetheless, even though the results of the questionnaire were slightly more positive for Magic Leap 1, the spatial understanding of the 3D anatomical model in terms of depth relations and spatial arrangement was positively evaluated for both devices.

Live Virtual Surgery and Virtual Reality in Surgery: Potential Applications in Hand Surgery Education

Accelerated in part by the coronavirus disease 2019 pandemic, medical education has increasingly moved into the virtual sphere in recent years. Virtual surgical education encompasses several domains, including live virtual surgery and virtual and augmented reality. These technologies range in complexity from streaming audio and video of surgeries in real-time to fully immersive virtual simulations of surgery. This article reviews the current use of virtual surgical education and its possible applications in hand surgery. Applications of virtual technologies for preoperative planning and intraoperative guidance, as well as care in underresourced settings, are discussed. The authors describe their experience creating a virtual surgery subinternship with live virtual surgeries. There are many roles virtual technology can have in surgery, and this review explores potential value these technologies may have in hand surgery.

The Advantage of Using an Optical See-Through Head-Mounted Display in Ultrasonography-Guided Needle Biopsy Procedures: A Prospective Randomized Study

An optical see-through head-mounted display (OST-HMD) can potentially improve the safety and accuracy of ultrasonography (US)-guided fine-needle aspiration. We aimed to evaluate the usefulness of an OST-HMD in US-guided needle-puncture procedures. We conducted a prospective randomized controlled study in which we compared the accuracy and safety of the US-guided needle puncture procedure and the stress on the practitioner when using OST-HMD versus standard US display (SUD). Inexperienced medical students were enrolled and randomly divided into two groups. A breast phantom was used to evaluate the required time and accuracy of the US-guided needle puncture. Practitioner stress was quantified using a visual analog scale (VAS). When the procedure was performed for the first time, the time required to reach the target lesion at a shallow depth was significantly shorter in the OST-HMD group (39.8 ± 39.9 s) than in the SUD group (71.0 ± 81.0 s) (p = 0.01). Using the OST-HMD significantly reduced the unintentional puncture of a non-target lesion (p = 0.01). Furthermore, the stress felt by the practitioners when capturing the image of the target lesion (p < 0.001), inserting and advancing the needle more deeply (p < 0.001), and puncturing the target lesion (p < 0.001) was significantly reduced in the OST-HMD group compared with that in the SUD group. Use of OST-HMD may improve the accuracy and safety of US-guided needle puncture procedures and may reduce practitioner stress during the procedure.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

BACKGROUND

Mixed reality (MR) and its potential applications have gained increasing interest within the medical community over the recent years. The ability to integrate virtual objects into a real-world environment within a single video-see-through display is a topic that sparks imagination. Given these characteristics, MR could facilitate preoperative and preinterventional planning, provide intraoperative and intrainterventional guidance, and aid in education and training, thereby improving the skills and merits of surgeons and residents alike.

OBJECTIVE

In this narrative review, we provide a broad overview of the different applications of MR within the entire spectrum of surgical and interventional practice and elucidate on potential future directions.

METHODS

A targeted literature search within the PubMed, Embase, and Cochrane databases was performed regarding the application of MR within surgical and interventional practice. Studies were included if they met the criteria for technological readiness level 5, and as such, had to be validated in a relevant environment.

RESULTS

A total of 57 studies were included and divided into studies regarding preoperative and interventional planning, intraoperative and interventional guidance, as well as training and education.

CONCLUSIONS

The overall experience with MR is positive. The main benefits of MR seem to be related to improved efficiency. Limitations primarily seem to be related to constraints associated with head-mounted display. Future directions should be aimed at improving head-mounted display technology as well as incorporation of MR within surgical microscopes, robots, and design of trials to prove superiority.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.