Intraoperative holography navigation using a mixed-reality wearable computer during laparoscopic cholecystectomy

Leveraging Space-Flown Technologies to Deliver Healthcare with Holographic Physical Examinations

INTRODUCTION: Musculoskeletal injuries are one of the more common injuries in spaceflight. Physical assessment of an injury is essential for diagnosis and treatment. Unfortunately, when musculoskeletal injuries occur in space, the flight surgeon is limited to two-dimensional videoconferencing and, potentially, observations made by the crew medical officer. To address these limitations, we investigated the feasibility of performing physical examinations on a three-dimensional augmented reality projection using a mixed-reality headset, specifically evaluating a standard shoulder examination.METHODS: A simulated patient interaction was set up between Western University in London, Ontario, Canada, and Huntsville, AL, United States. The exam was performed by a medical student, and a healthy adult man volunteered to enable the physical exam.RESULTS: All parts of the standard shoulder physical examination according to the Bates Guide to the Physical Exam were performed with holoportation. Adaptation was required for the palpation and some special tests.DISCUSSION: All parts of the physical exam were able to be completed. The true to anatomical size of the holograms permitted improved inspection of the anatomy compared to traditional videoconferencing. Palpation was completed by instructing the patient to palpate themselves and comment on relevant findings asked by the examiner. Range of motion and special tests for specific pathologies were also able to be completed with some modifications due to the examiner not being present to provide resistance. Future work should aim to improve the graphics, physician communication, and haptic feedback during holoportation.Levschuk A, Whittal J, Trejos AL, Sirek A. Leveraging space-flown technologies to deliver healthcare with holographic physical examinations. Aerosp Med Hum Perform. 2024; 95(4):214-218.

Comparing Different Registration and Visualization Methods for Navigated Common Femoral Arterial Access-A Phantom Model Study Using Mixed Reality

Mixed reality (MxR) enables the projection of virtual three-dimensional objects into the user's field of view via a head-mounted display (HMD). This phantom model study investigated three different workflows for navigated common femoral arterial (CFA) access and compared it to a conventional sonography-guided technique as a control. A total of 160 punctures were performed by 10 operators (5 experts and 5 non-experts). A successful CFA puncture was defined as puncture at the mid-level of the femoral head with the needle tip at the central lumen line in a 0° coronary insertion angle and a 45° sagittal insertion angle. Positional errors were quantified using cone-beam computed tomography following each attempt. Mixed effect modeling revealed that the distance from the needle entry site to the mid-level of the femoral head is significantly shorter for navigated techniques than for the control group. This highlights that three-dimensional visualization could increase the safety of CFA access. However, the navigated workflows are infrastructurally complex with limited usability and are associated with relevant cost. While navigated techniques appear as a potentially beneficial adjunct for safe CFA access, future developments should aim to reduce workflow complexity, avoid optical tracking systems, and offer more pragmatic methods of registration and instrument tracking.

Application of Extended Reality (Virtual Reality and Mixed Reality) Technology in Laparoscopic Liver Resections

Background and purpose Laparoscopic liver resection (LLR) has recently gained popularity owing to advances in surgical techniques. Difficulties in LLR may be influenced by anatomical factors. This study presents a comprehensive overview of LLR performed using extended reality (XR) technology. Methods Six patients underwent LLR performed wearing HoloLens2® XR (Microsoft Corporation, Redmond, Washington, United States) technology. We performed dynamic contrast-enhanced CT scans before surgery and used the data to construct three-dimensional images. Results Of the six patients, two were diagnosed with colorectal liver metastases, two with hepatocellular carcinoma, and one with intrahepatic cholangiocarcinoma. The median maximum tumor diameter was 31 mm (range, 23-80 mm). One patient had liver cirrhosis, with Child-Pugh classification grade B. Anatomical resection was performed in three patients (60%), with a median difficulty score of 7 (intermediate). No conversions to open surgery were necessary. The median operative time and estimated blood loss were 444 minutes (range, 337-597 minutes) and 200 mL (range, 100-1000 mL), respectively. Postoperative complications (Clavien-Dindo classification grade II) were observed in one patient. All six cases achieved negative surgical margins. Conclusions LLR using XR technology enhances surgical visualization and anatomical recognition. The incorporation of XR technology into LLR offers advantages over traditional two-dimensional imaging.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

BACKGROUND

Mixed reality (MR) and its potential applications have gained increasing interest within the medical community over the recent years. The ability to integrate virtual objects into a real-world environment within a single video-see-through display is a topic that sparks imagination. Given these characteristics, MR could facilitate preoperative and preinterventional planning, provide intraoperative and intrainterventional guidance, and aid in education and training, thereby improving the skills and merits of surgeons and residents alike.

OBJECTIVE

In this narrative review, we provide a broad overview of the different applications of MR within the entire spectrum of surgical and interventional practice and elucidate on potential future directions.

METHODS

A targeted literature search within the PubMed, Embase, and Cochrane databases was performed regarding the application of MR within surgical and interventional practice. Studies were included if they met the criteria for technological readiness level 5, and as such, had to be validated in a relevant environment.

RESULTS

A total of 57 studies were included and divided into studies regarding preoperative and interventional planning, intraoperative and interventional guidance, as well as training and education.

CONCLUSIONS

The overall experience with MR is positive. The main benefits of MR seem to be related to improved efficiency. Limitations primarily seem to be related to constraints associated with head-mounted display. Future directions should be aimed at improving head-mounted display technology as well as incorporation of MR within surgical microscopes, robots, and design of trials to prove superiority.

Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects

In the world reference context, although virtual reality, augmented reality and mixed reality have been emerging methodologies for several years, only today technological and scientific advances have made them suitable to revolutionize clinical care and medical contexts through the provision of enhanced functionalities and improved health services. This systematic review provides the state-of-the-art applications of the Microsoft^® HoloLens 2 in a medical and healthcare context. Focusing on the potential that this technology has in providing digitally supported clinical care, also but not only in relation to the COVID-19 pandemic, studies that proved the applicability and feasibility of HoloLens 2 in a medical and healthcare scenario were considered. The review presents a thorough examination of the different studies conducted since 2019, focusing on HoloLens 2 medical sub-field applications, device functionalities provided to users, software/platform/framework used, as well as the study validation. The results provided in this paper could highlight the potential and limitations of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as telemedicine, remote control and motor rehabilitation.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

Digital Transformation Will Change Medical Education and Rehabilitation in Spine Surgery

The concept of minimally invasive spine therapy (MIST) has been proposed as a treatment strategy to reduce the need for overall patient care, including not only minimally invasive spine surgery (MISS) but also conservative treatment and rehabilitation. To maximize the effectiveness of patient care in spine surgery, the educational needs of medical students, residents, and patient rehabilitation can be enhanced by digital transformation (DX), including virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), three-dimensional (3D) medical images and holograms; wearable sensors, high-performance video cameras, fifth-generation wireless system (5G) and wireless fidelity (Wi-Fi), artificial intelligence, and head-mounted displays (HMDs). Furthermore, to comply with the guidelines for social distancing due to the unexpected COVID-19 pandemic, the use of DX to maintain healthcare and education is becoming more innovative than ever before. In medical education, with the evolution of science and technology, it has become mandatory to provide a highly interactive educational environment and experience using DX technology for residents and medical students, known as digital natives. This study describes an approach to pre- and intraoperative medical education and postoperative rehabilitation using DX in the field of spine surgery that was implemented during the COVID-19 pandemic and will be utilized thereafter.

Mixed-Reality-Assisted Puncture of the Common Femoral Artery in a Phantom Model

Percutaneous femoral arterial access is daily practice in a variety of medical specialties and enables physicians worldwide to perform endovascular interventions. The reported incidence of percutaneous femoral arterial access complications is 3–18% and often results from suboptimal puncture location due to insufficient visualization of the target vessel. The purpose of this proof-of-concept study was to evaluate the feasibility and the positional error of a mixed-reality (MR)-assisted puncture of the common femoral artery in a phantom model using a commercially available navigation system. In total, 15 MR-assisted punctures were performed. Cone-beam computed tomography angiography (CTA) was used following each puncture to allow quantification of positional error of needle placements in the axial and sagittal planes. Technical success was achieved in 14/15 cases (93.3%) with a median axial positional error of 1.0 mm (IQR 1.3) and a median sagittal positional error of 1.1 mm (IQR 1.6). The median duration of the registration process and needle insertion was 2 min (IQR 1.0). MR-assisted puncture of the common femoral artery is feasible with acceptable positional errors in a phantom model. Future studies should aim to measure and reduce the positional error resulting from MR registration.