Editorial: Challenges for the usability of AR and VR for clinical neurosurgical procedures

Clinical trainee performance on task-based AR/VR-guided surgical simulation is correlated with their 3D image spatial reasoning scores

This paper describes a methodology for the assessment of training simulator-based computer-assisted intervention skills on an AR/VR-guided procedure making use of CT axial slice views for a neurosurgical procedure: external ventricular drain (EVD) placement. The task requires that trainees scroll through a stack of axial slices and form a mental representation of the anatomical structures in order to subsequently target the ventricles to insert an EVD. The process of observing the 2D CT image slices in order to build a mental representation of the 3D anatomical structures is the skill being taught, along with the cognitive control of the subsequent targeting, by planned motor actions, of the EVD tip to the ventricular system to drain cerebrospinal fluid (CSF). Convergence is established towards the validity of this assessment methodology by examining two objective measures of spatial reasoning, along with one subjective expert ranking methodology, and comparing these to AR/VR guidance. These measures have two components: the speed and accuracy of the targeting, which are used to derive the performance metric. Results of these correlations are presented for a population of PGY1 residents attending the Canadian Neurosurgical "Rookie Bootcamp" in 2019.

Feasibility of Augmented Reality for Pediatric Giant Supratentorial Tumors: A Report of Three Cases

Giant supratentorial brain tumors (GSBTs) in children are uncommon and extremely challenging entities unique to pediatric neurosurgery. Factors such as young patient age, need for urgent intervention, intraoperative blood loss, and ongoing raised intracranial pressure symptoms are examples of difficulties faced. Recently, there has been a growing body of literature on augmented reality (AR) in adult neurosurgery. In contrast, the use of AR in pediatric neurosurgery is comparatively less. Nonetheless, we postulate that AR systems will be helpful for understanding spatial relationships of complex GSBT anatomy for preoperative planning in a timely fashion. This study describes our experience in trialing AR as a potential tool for three cases of pediatric GSBTs. Overall, the AR platform offers our neurosurgical team excellent visuospatial insights for preoperative decision-making. However, we observe that substantial time is required to set up the AR system prior to each clinical case discussion by the neurosurgical team. In congruency with existing literature, our preliminary results report that there are still obstacles that need to be addressed before the technology can be seamlessly implemented into the clinical workflow for these time-sensitive childhood brain tumors. To our knowledge, this is the first study to report the potential use of AR for complex pediatric GSBT cases.

AR-AI assisted ophthalmic nursing: Preliminary usability study in clinical settings

Objective

Ophthalmic ward nursing work is onerous and busy, and many researchers have tried to introduce artificial intelligence (AI) technology to assist nurses in performing nursing tasks. This study aims to use augmented reality (AR) and AI technology to develop an intelligent assistant system for ophthalmic ward nurses and evaluate the usability and acceptability of the system in assisting clinical work for nurses.

Methods

Based on AR technology, under the framework of deep learning, the system management, functions, and interfaces were completed using acoustic recognition, voice interaction, and image recognition technologies. Finally, an intelligent assistance system with functions such as patient face recognition, automatic information matching, and nursing work management was developed. Ophthalmic day ward nurses were invited to participate in filling out the System Usability Scale (SUS). Using the AR-based intelligent assistance system (AR-IAS) as the experimental group and the existing personal digital assistant (PDA) system as the control group. The experimental results of the three subscales of learnability, efficiency, and satisfaction of the usability scale were compared, and the clinical usability score of the AR-IAS system was calculated.

Results

This study showed that the AR-IAS and the PDA systems had learnability subscale scores of 22.50/30.00 and 21.00/30.00, respectively; efficiency subscale scores of 29.67/40.00 and 28.67/40.00, respectively; and satisfaction subscale scores of 23.67/30.00 and 23.17/30.00, respectively. The overall usability score of the AR-IAS system was 75.83/100.00.

Conclusion

Based on the analysis results of the System Usability Scale, the AR-IAS system developed using AR and AI technology has good overall usability and can be accepted by clinical nurses. It is suitable for use in ophthalmic nursing tasks and has clinical promotion and further research value.

Bone biopsies guided by augmented reality: a pilot study

PURPOSE

To test the technical feasibility of an augmented reality (AR) navigation system to guide bone biopsies.

METHODS

We enrolled patients subjected to percutaneous computed tomography (CT)-guided bone biopsy using a novel AR navigation system. Data from prospectively enrolled patients (AR group) were compared with data obtained retrospectively from previous standard CT-guided bone biopsies (control group). We evaluated the following: procedure duration, number of CT passes, patient's radiation dose (dose-length product), complications, and specimen adequacy. Technical success was defined as the ability to complete the procedure as planned, reaching the target center. Technical efficacy was assessed evaluating specimen adequacy.

RESULTS

Eight patients (4 males) aged 58 ± 24 years (mean ± standard deviation) were enrolled in the AR group and compared with 8 controls (4 males) aged 60 ± 15 years. No complications were observed. Procedure duration, number of CT passes, and radiation dose were 22 ± 5 min, 4 (median) [4, 6 interquartile range] and 1,034 ± 672 mGy*cm for the AR group and 23 ± 5 min, 9 [7.75, 11.25], and 1,954 ± 993 mGy*cm for controls, respectively. No significant differences were observed for procedure duration (p = 0.878). Conversely, number of CT passes and radiation doses were significantly lower for the AR group (p < 0.001 and p = 0.021, respectively). Technical success and technical efficacy were 100% for both groups.

CONCLUSIONS

This AR navigation system is safe, feasible, and effective; it can decrease radiation exposure and number of CT passes during bone biopsies without increasing duration time.

RELEVANCE STATEMENT

This augmented reality (AR) navigation system is a safe and feasible guidance for bone biopsies; it may ensure a decrease in the number of CT passes and patient's radiation dose.

KEY POINTS

• This AR navigation system is a safe guidance for bone biopsies. • It ensures decrease of number of CT passes and patient's radiation exposure. • Procedure duration was similar to that of standard CT-guided biopsy. • Technical success was 100% as in all patients the target was reached. • Technical efficacy was 100% as the specimen was adequate in all patients.

Thermal Ablation of Liver Tumors Guided by Augmented Reality: An Initial Clinical Experience

Background: Over the last two decades, augmented reality (AR) has been used as a visualization tool in many medical fields in order to increase precision, limit the radiation dose, and decrease the variability among operators. Here, we report the first in vivo study of a novel AR system for the guidance of percutaneous interventional oncology procedures. Methods: Eight patients with 15 liver tumors (0.7−3.0 cm, mean 1.56 + 0.55) underwent percutaneous thermal ablations using AR guidance (i.e., the Endosight system). Prior to the intervention, the patients were evaluated with US and CT. The targeted nodules were segmented and three-dimensionally (3D) reconstructed from CT images, and the probe trajectory to the target was defined. The procedures were guided solely by AR, with the position of the probe tip was subsequently confirmed by conventional imaging. The primary endpoints were the targeting accuracy, the system setup time, and targeting time (i.e., from the target visualization to the correct needle insertion). The technical success was also evaluated and validated by co-registration software. Upon completion, the operators were assessed for cybersickness or other symptoms related to the use of AR. Results: Rapid system setup and procedural targeting times were noted (mean 14.3 min; 12.0−17.2 min; 4.3 min, 3.2−5.7 min, mean, respectively). The high targeting accuracy (3.4 mm; 2.6−4.2 mm, mean) was accompanied by technical success in all 15 lesions (i.e., the complete ablation of the tumor and 13/15 lesions with a >90% 5-mm periablational margin). No intra/periprocedural complications or operator cybersickness were observed. Conclusions: AR guidance is highly accurate, and allows for the confident performance of percutaneous thermal ablations.

A review of participant recruitment transparency for sound validation of hip surgery simulators: a novel umbrella approach

Malposition of implants is associated with complications, higher wear and increased revision rates in total hip replacement (THR) along with surgeon inexperience. Training THR residents to reach expert proficiency is affected by the high cost and resource limitations of traditional training techniques. Research in extended reality (XR) technologies can overcome such barriers. These offer a platform for learning, objective skill-monitoring and, potentially, for automated certification. Prior to their incorporation into curricula however, thorough validation must be undertaken. As validity is heavily dependent on the participants recruited, there is a need to review, scrutinise and define recruitment criteria in the absence of pre-defined standards, for sound simulator validation. A systematic review on PubMed and IEEE databases was conducted. Training simulator validation research in fracture, arthroscopy and arthroplasty relating to the hip was included. 46 validation studies were reviewed. It was observed that there was no uniformity in reporting or recruitment criteria, rendering cross-comparison challenging. This work developed Umbrella categories to help prioritise recruitment, and has formulated a detailed template of fields and guidelines for reporting criteria so that, in future, research may come to a consensus as to recruitment criteria for a hip "expert" or "novice".

Virtual reality-based evaluation of neurovascular conflict for the surgical planning of microvascular decompression in trigeminal neuralgia patients

OBJECTIVE

Trigeminal neuralgia (TN) is a lightning bolt of violent, electrifying, and stinging pain, often secondary to the neurovascular conflict (NVC). The vessels involved in NVC are mostly arteries and rarely veins. Evaluation of NVC in the deep infratentorial region is inseparably connected with cranial imaging. We retrospectively analyzed the potential influence of three-dimensional (3D) virtual reality (VR) reconstructions compared to conventional magnetic resonance imaging (MRI) scans on the evaluation of NVC for the surgical planning of microvascular decompression in patients with TN.

METHODS

Medical files were retrospectively analyzed regarding patient- and disease-related data. Preoperative MRI scans were retrospectively visualized via VR software to detect the characteristics of NVC. A questionnaire of experienced neurosurgeons evaluated the influence of VR visualization technique on identification of anatomical structures involved in NVC and on surgical strategy.

RESULTS

Twenty-four patients were included and 480 answer sheets were evaluated. Compared to conventional MRI, image presentation using 3D-VR modality significantly influenced the identification of the affected trigeminal nerve (p = 0.004), the vascular structure involved in the NVC (p = 0.0002), and the affected side of the trigeminal nerve (p = 0.005).

CONCLUSIONS

In patients with TN caused by NVC, the reconstruction of conventional preoperative MRI scans and the spatial and anatomical presentation in 3D-VR models offers the possibility of increased understanding of the anatomy and even more the underlying pathology, and thus influences operation planning and strategy.

Cerebral Anatomy Detection and Surgical Planning in Patients with Anterior Skull Base Meningiomas Using a Virtual Reality Technique

Anterior skull base meningiomas represent a wide cohort of tumors with different locations, extensions, configurations, and anatomical relationships. Diagnosis of these tumors and review of their therapies are inseparably connected with cranial imaging. We analyzed the influence of three-dimensional-virtual reality (3D-VR) reconstructions versus conventional computed tomography (CT) and magnetic resonance imaging (MRI) images (two-dimensional (2D) and screen 3D) on the identification of anatomical structures and on the surgical planning in patients with anterior skull base meningiomas. Medical files were retrospectively analyzed regarding patient- and disease-related data. Preoperative 2D-CT and 2D-MRI scans were retrospectively reconstructed to 3D-VR images and visualized via VR software to detect the characteristics of tumors. A questionnaire of experienced neurosurgeons evaluated the influence of the VR visualization technique on identification of tumor morphology and relevant anatomy and on surgical strategy. Thirty patients were included and 600 answer sheets were evaluated. The 3D-VR modality significantly influenced the detection of tumor-related anatomical structures (p = 0.002), recommended head positioning (p = 0.005), and surgical approach (p = 0.03). Therefore, the reconstruction of conventional preoperative 2D scans into 3D images and the spatial and anatomical presentation in VR models enabled greater understanding of anatomy and pathology, and thus influenced operation planning and strategy.

Retrospective Comparison of Minimally Invasive and Open Monosegmental Lumbar Fusion, and Impact of Virtual Reality on Surgical Planning and Strategy

BACKGROUND AND STUDY AIMS

 Spinal fusion for symptomatic lumbar spondylolisthesis can be accomplished using an open or minimally invasive surgical (MIS) technique. Evaluation of segmental spondylolisthesis and instabilities and review of their therapies are inseparably connected with lumbar tomographic imaging. We analyzed a cohort of patients who underwent MIS or open monosegmental dorsal fusion and compared surgical outcomes along with complication rates. We furthermore evaluated the influence of virtual reality (VR) visualization on surgical planning in lumbar fusion.

MATERIAL AND METHODS

 Patient files were retrospectively analyzed regarding patient- and disease-related data, operative performance, surgical outcomes, and perioperative surgical complications. Preoperative computed tomography (CT) and magnetic resonance imaging (MRI) scans were retrospectively visualized via VR software. A questionnaire evaluated the influence of three-dimensional (3D) VR images versus two-dimensional CT and MRI scans on therapy planning, fusion method, and surgical technique and procedure.

RESULTS

 Overall, 171 patients were included (MIS/open: 90/81). MIS was associated with less blood loss, shorter surgery time and hospital stay, lower complication rates, equivalent long-term patient-reported outcomes, but lower fusion rates and higher late reoperation rates than open surgery. Image presentation using VR significantly influenced the recommended surgical therapies (decompression only/decompression and fusion; p = 0.02), had no significant influence on the recommended fusion method (rigid/dynamic/stand-alone; p = 0.77), and, in cases of rigid fusion, a significant influence on the recommended technique (MIS/open; p = 0.03) and fusion procedure (p = 0.02).

CONCLUSION

 In patients with monosegmental degenerative or isthmic spondylolisthesis, MIS fusion was advantageous concerning perioperative complication rates and perioperative surgical outcomes, but disadvantageous regarding fusion and reoperation rates compared to open fusion. 3D-VR-based analysis of sectional images significantly influenced the recommended surgical planning.

Impact of Virtual Reality in Arterial Anatomy Detection and Surgical Planning in Patients with Unruptured Anterior Communicating Artery Aneurysms

Anterior-communicating artery (ACoA) aneurysms have diverse configurations and anatomical variations. The evaluation and operative treatment of these aneurysms necessitates a perfect surgical strategy based on review of three-dimensional (3D) angioarchitecture using several radiologic imaging methods. We analyzed the influence of 3D virtual reality (VR) reconstructions versus conventional computed tomography angiography (CTA) scans on the identification of vascular anatomy and on surgical planning in patients with unruptured ACoA aneurysms. Medical files were retrospectively analyzed regarding patient- and disease-related data. Preoperative CTA scans were retrospectively reconstructed to 3D-VR images and visualized via VR software to detect the characteristics of unruptured ACoA aneurysms. A questionnaire was used to evaluate the influence of VR on the identification of aneurysm morphology and relevant arterial anatomy and on surgical strategy. Twenty-six patients were included and 520 answer sheets were evaluated. The 3D-VR modality significantly influenced detection of the aneurysm-related vascular structure (p = 0.0001), the recommended head positioning (p = 0.005), and the surgical approach (p = 0.001) in the planning of microsurgical clipping. Thus, reconstruction of conventional preoperative CTA scans into 3D images and the spatial presentation in VR models enabled greater understanding of the anatomy and pathology, provided realistic haptic feedback for aneurysm surgery, and influenced operation planning and strategy.

The Potential Value of Mixed Reality in Neurosurgery

Mixed reality (MR) merges virtual information into the real world through computer technology, in which the real environment and virtual objects can get spliced in the same image or space at real time so that it can effectively express and integrate the virtual and real worlds and allow high feedback interaction. This technology combines the many advantages of virtual realityand augmented reality, and has a promising future in the medical field. At present, MR technology is just at the beginning stage in the medical field in the world, whose application in neurosurgery is also rarely reported. Given this, the authors described the research progress of MR in neurosurgery including preoperative planning and intraoperative guidance, doctor-patient communication, teaching rounds, physician training, and so on.

Comparison of stand-alone cage and cage-with-plate for monosegmental cervical fusion and impact of virtual reality in evaluating surgical results

OBJECTIVES

After ventral decompression of monosegmental cervical spondylotic stenosis, a stand-alone cage (SC) or cage-with-plate (CP) can be inserted for fusion. Postoperative radiological evaluation can be achieved using different imaging modalities. We retrospectively compared complications, as well as clinical and radiological outcomes for both fusion techniques, and analyzed the possible role of virtual reality (VR) in evaluating the postoperative results.

PATIENTS AND METHODS

One hundred seventeen patients were included (SC/CP: 54/63). Complications, as well as clinical and radiological outcomes of both fusion techniques were compared. Computed tomography (CT) scans were visualized via VR to measure the smallest cross-sectional area of the intervertebral neuroforamen in the lateral resection region, and to assess the degree of the intersegmental ossification.

RESULTS

There were no significant differences between the two fusion techniques regarding perioperative complication rates, fusion rates, or pain parameter (visual analogue scale (VAS) of arm pain, neck disability index). However, advantages regarding subsidence, kyphosis, and VAS of neck pain were found when using the CP versus SC. Using the VR technique, there was no significant difference between the two fusion techniques in the mean size of the cross-sectional area at the end of follow-up.

CONCLUSION

Due to the long-term advantages of CP fusion, we prefer a monosegmental cervical spinal fusion using CP. Reconstruction of postsurgical two-dimensional CT images into three-dimensional images, and the spatial and anatomical presentation in VR models, improved the evaluation of these postoperative results.

Immersing Patients in a Virtual Reality Environment for Brain Mapping During Awake Surgery: Safety Study

BACKGROUND

Brain mapping by direct electrical stimulation during awake craniotomy is now a standard procedure that reduces the risk of permanent neurologic deficits. Virtual reality technology immerses the patient in a virtually controlled, interactive world, offering a unique opportunity to develop innovative tasks for perioperative mapping of complex cognitive functions. The objective of this prospective single-center study was to evaluate the tolerance and safety of a virtual reality headset (VRH) and immersive virtual experiences in patients undergoing awake craniotomy and brain mapping by direct electrical stimulation.

METHODS

The study included 30 patients with a brain tumor near the language area. Language mapping was performed with a naming task, DO 80, presented on a digital tablet and then in two-dimensional and three-dimensional formats through a VRH. During wound closure, different virtual reality experiences were proposed to the patient, offering different types of virtual motion or interaction with an avatar piloted by a neuropsychologist.

RESULTS

Two patients could not use the VRH owing to technical issues. No procedure was aborted, no patient experienced virtual reality sickness and all patients reported they would repeat the procedure. Despite a high rate of intraoperative focal seizures, there was no argument to attribute the seizures to VRH use.

CONCLUSIONS

This study shows that it is possible during awake brain surgery to immerse the patient in a virtual environment and to interact with the patient, opening the field of new brain mapping procedures for complex cognitive functions.

Virtual Reality-Based Evaluation of Surgical Planning and Outcome of Monosegmental, Unilateral Cervical Foraminal Stenosis

BACKGROUND

Foraminal cervical nerve root compression can be caused by lateral disk herniation or osteophyte formation of the vertebrae. Improved diagnosis and evaluation can be achieved using different imaging techniques: radiographs, computed tomography (CT), and magnetic resonance imaging. We retrospectively evaluated the potential influence of a virtual reality (VR) visualization technique on surgery planning and evaluation of postoperative results in patients with monosegmental, unilateral osseous cervical neuroforaminal stenosis.

METHODS

Seventy-three patients were included. Ventral decompression of the neuroforamen was performed in 41 patients, dorsal decompression in 32 patients. Patients' files were evaluated. CT scans were visualized via VR software to measure the smallest cross-sectional area of the intervertebral neuroforamen in the lateral resection region. A questionnaire evaluated the influence of VR technique on surgical planning and strategy.

RESULTS

The VR-technique had a moderate influence on the choice of the approach (ventral or dorsal), a significant influence on the ventral approach strategy, and no influence on the positioning of the patient or the dorsal approach strategy. A significant difference was found in the size of the smallest cross-sectional area of the intervertebral neuroforamen in the lateral resection region between ventral and dorsal approaches, with no correlation to the clinical outcome.

CONCLUSIONS

Reconstruction of pre- and postoperative 2D-CT images of the cervical spine into 3D images, and the spatial and anatomical reconstructions in VR models, can be helpful in planning surgical approaches and treatment strategies for patients with cervical foraminal stenoses, and for evaluation of their postoperative results.

Application and Prospect of Mixed Reality Technology in Medical Field

Mixed reality (MR) technology is a new digital holographic image technology, which appears in the field of graphics after virtual reality (VR) and augmented reality (AR) technology, a new interdisciplinary frontier. As a new generation of technology, MR has attracted great attention of clinicians in recent years. The emergence of MR will bring about revolutionary changes in medical education training, medical research, medical communication, and clinical treatment. At present, MR technology has become the popular frontline information technology for medical applications. With the popularization of digital technology in the medical field, the development prospects of MR are inestimable. The purpose of this review article is to introduce the application of MR technology in the medical field and prospect its trend in the future.