Expanding Educational Frontiers in Neurosurgery: Current and Future Uses of Augmented Reality

Apple Vision Pro Augmented Reality-Assisted Minimally Invasive Surgical Treatment of Spinal Dural Arteriovenous Fistula

This report outlines the innovative use of augmented reality (AR) in the surgical planning and treatment of a spinal dural arteriovenous fistula (dAVF) via a minimally invasive technique. AR technology by way of an Apple Vision Pro headset was employed to enhance preoperative visualization and understanding of the pathology, leading to successful surgical ligation of the AVF. This case describes a 56-year-old male presenting with progressive weakness and thoracic myelopathy who showed marked improvement postoperatively, highlighting AR's potential to improve surgical approach and outcomes.

Augmented Reality-Guided External Ventricular Drain Placement: A Case Report

The placement of an external ventricular drain (EVD) is a critical neurosurgical procedure used to relieve intracranial pressure in patients with conditions such as hydrocephalus, traumatic brain injury, and intracranial hemorrhage. Traditional methods rely heavily on anatomical landmarks and the surgeon's experience, which can lead to variability in outcomes and increased risk of complications. Neuronavigation, while available, is infrequently used due to the size, cost, and set-up times associated with these devices. This report explores the use of a headset-based augmented reality (AR) system for guidance during the EVD placement procedure. We describe an AR system that overlays a 3D model of the patient's cranial anatomy, derived from preoperative imaging, onto the patient's head. This system is a head-mounted display and utilizes a rapid fiducial-less registration to provide the surgeon with visualization of 3D anatomy, and targeted trajectories. The system was used with a 32-year-old patient undergoing EVD placement prior to a cranioplasty. Due to the atypical cranial anatomy and due to prior procedures and midline shift, this relatively high-risk catheter placement was an ideal circumstance for the use of AR guidance during the EVD placement. This report described an early use of AR for EVD placement and represents a substantial advancement in neurosurgical practice, offering enhanced precision, efficiency, and safety. Further large-scale studies are warranted to validate these findings and explore the broader applicability of AR in other neurosurgical procedures.

Feasibility of a novel augmented reality overlay for cervical screw placement in phantom spine models

STUDY DESIGN

Feasibility study.

PURPOSE

A phantom model was used to evaluate the accuracy of a novel augmented reality (AR) system for cervical screw placement.

OVERVIEW OF LITERATURE

The use of navigation systems is becoming increasingly common in spine procedures. However, numerous factors limit the feasibility of regular and widespread use of navigation tools during spine surgery. AR is a new technology that has already demonstrated utility as a navigation tool during spine surgery. However, advancements in AR technology are needed to increase its adoption by the medical community.

METHODS

AR technology that uses a fiducial-less registration system was tested in a preclinical cervical spine phantom model study for accuracy during spinal screw placement. A three-dimensional reconstruction of the spine along with trajectory lines was superimposed onto the phantom model using an AR headset. Participants used the AR system to guide screw placement, and post-instrumentation scans were compared for accuracy assessment.

RESULTS

Twelve cervical screws were placed under AR guidance. All screws were placed in an acceptable anatomic position. The average distance error for the insertion point was 2.73±0.55 mm, whereas that for the endpoint was 2.71±0.69 mm. The average trajectory angle error for all insertions was 2.69°±0.59°.

CONCLUSIONS

This feasibility study describes a novel registration approach that superimposes spinal anatomy and trajectories onto the surgeon's real-world view of the spine. These results demonstrate reasonable accuracy in the preclinical model. The results of this study demonstrate that this technology can assist with accurate screw placement. Further investigation using cadaveric and clinical models is warranted.

Intraoperative Augmented Reality for Complex Glioma Resection: A Case Report

Augmented reality (AR) is an emerging technology that can display three-dimensional patient anatomy in the surgeons' field of view. The use of this technology has grown considerably for both presurgical and intraoperative guidance. A patient diagnosed with breast cancer started to experience numbness in the left hand, which progressed to weakness in the left hand and arm. An MRI was performed demonstrating a 2.9 cm X 1.8 cm lesion with extensive surrounding edema in the posterior fronto-parietal lobes. Surgery was recommended for presumed metastatic disease. Preoperatively, an AR system and Brainlab navigation were registered to the patient. AR, traditional navigation, and ultrasound were all used to localize the lesion and determine the craniotomy site and size. The tumor was removed along the direction of the lesion. Intraoperatively, we used AR to reexamine the tumor details and could appreciate that we had to redirect our surgical trajectory anteriorly and laterally in order to follow along the main axis of the tumor. In doing this, we were able to more confidently remain with the tumor, which by this time was poorly defined by 2D navigation and by direct vision. Postoperative MRI confirmed gross total removal of the tumor. The patient had an uneventful postoperative course with resolution of preoperative symptoms and the final surgical pathology was grade 4 glioblastoma. Here, we describe the valuable use of AR for the resection of a glioma. The system has a seamless registration process and provides the surgeon with a unique view of 3D anatomy overlaid onto the patient's head. This exciting technology can add tremendous value to complex cranial surgeries.

Feasibility of Augmented Reality for Pediatric Giant Supratentorial Tumors: A Report of Three Cases

Giant supratentorial brain tumors (GSBTs) in children are uncommon and extremely challenging entities unique to pediatric neurosurgery. Factors such as young patient age, need for urgent intervention, intraoperative blood loss, and ongoing raised intracranial pressure symptoms are examples of difficulties faced. Recently, there has been a growing body of literature on augmented reality (AR) in adult neurosurgery. In contrast, the use of AR in pediatric neurosurgery is comparatively less. Nonetheless, we postulate that AR systems will be helpful for understanding spatial relationships of complex GSBT anatomy for preoperative planning in a timely fashion. This study describes our experience in trialing AR as a potential tool for three cases of pediatric GSBTs. Overall, the AR platform offers our neurosurgical team excellent visuospatial insights for preoperative decision-making. However, we observe that substantial time is required to set up the AR system prior to each clinical case discussion by the neurosurgical team. In congruency with existing literature, our preliminary results report that there are still obstacles that need to be addressed before the technology can be seamlessly implemented into the clinical workflow for these time-sensitive childhood brain tumors. To our knowledge, this is the first study to report the potential use of AR for complex pediatric GSBT cases.

NnU-Net versus mesh growing algorithm as a tool for the robust and timely segmentation of neurosurgical 3D images in contrast-enhanced T1 MRI scans

PURPOSE

This study evaluates the nnU-Net for segmenting brain, skin, tumors, and ventricles in contrast-enhanced T1 (T1CE) images, benchmarking it against an established mesh growing algorithm (MGA).

METHODS

We used 67 retrospectively collected annotated single-center T1CE brain scans for training models for brain, skin, tumor, and ventricle segmentation. An additional 32 scans from two centers were used test performance compared to that of the MGA. The performance was measured using the Dice-Sørensen coefficient (DSC), intersection over union (IoU), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) metrics, with time to segment also compared.

RESULTS

The nnU-Net models significantly outperformed the MGA (p < 0.0125) with a median brain segmentation DSC of 0.971 [95CI: 0.945-0.979], skin: 0.997 [95CI: 0.984-0.999], tumor: 0.926 [95CI: 0.508-0.968], and ventricles: 0.910 [95CI: 0.812-0.968]. Compared to the MGA's median DSC for brain: 0.936 [95CI: 0.890, 0.958], skin: 0.991 [95CI: 0.964, 0.996], tumor: 0.723 [95CI: 0.000-0.926], and ventricles: 0.856 [95CI: 0.216-0.916]. NnU-Net performance between centers did not significantly differ except for the skin segmentations Additionally, the nnU-Net models were faster (mean: 1139 s [95CI: 685.0-1616]) than the MGA (mean: 2851 s [95CI: 1482-6246]).

CONCLUSIONS

The nnU-Net is a fast, reliable tool for creating automatic deep learning-based segmentation pipelines, reducing the need for extensive manual tuning and iteration. The models are able to achieve this performance despite a modestly sized training set. The ability to create high-quality segmentations in a short timespan can prove invaluable in neurosurgical settings.

Augmented Reality Integration in Skull Base Neurosurgery: A Systematic Review

Background and Objectives: To investigate the role of augmented reality (AR) in skull base (SB) neurosurgery. Materials and Methods: Utilizing PRISMA methodology, PubMed and Scopus databases were explored to extract data related to AR integration in SB surgery. Results: The majority of 19 included studies (42.1%) were conducted in the United States, with a focus on the last five years (77.8%). Categorization included phantom skull models (31.2%, n = 6), human cadavers (15.8%, n = 3), or human patients (52.6%, n = 10). Microscopic surgery was the predominant modality in 10 studies (52.6%). Of the 19 studies, surgical modality was specified in 18, with microscopic surgery being predominant (52.6%). Most studies used only CT as the data source (n = 9; 47.4%), and optical tracking was the prevalent tracking modality (n = 9; 47.3%). The Target Registration Error (TRE) spanned from 0.55 to 10.62 mm. Conclusion: Despite variations in Target Registration Error (TRE) values, the studies highlighted successful outcomes and minimal complications. Challenges, such as device practicality and data security, were acknowledged, but the application of low-cost AR devices suggests broader feasibility.

The Apple Vision Pro as a Neurosurgical Planning Tool: A Case Report

With its recent release, the Apple Vision Pro (Apple Inc., Cupertino, CA) represents a promising technological advancement of mixed reality in the field of neurosurgery and medicine more broadly. With all new technologies, it is critical to facilitate early use and assessment of the technology to facilitate adoption by the larger medical community. A 44-year-old female with a history of ruptured intracranial aneurysm status post anterior communicating artery aneurysm clipping presented with worsened confusion and intermittent headache. CT imaging revealed evidence of hydrocephalus due to the malfunction of a previous right parietal ventriculoperitoneal (VP) shunt. Prior to the case, the Apple Vision Pro was used in the operating room to visualize and interact with a 3D model of the patient's anatomy for the patient undergoing a VP shunt placement. A visualization of the 3D model through the headset was used to plan the approach and entry point. At the conclusion of the procedure, all clinicians and operating staff who used the technology for planning completed a survey about their initial impressions of the headset. Overall, users felt the 3D models felt realistic (4.5/5), that the display of the user's real-world view felt natural (4.3/5), and that the headset did not cause eye strain or fatigue (4.5/5). The majority of users responded that they would continue to use the headset for cases (4/5). This represents one of the first known clinical uses of the Apple Vision Pro. It is a cutting-edge technology that will likely provide immense value for healthcare providers as it becomes more integrated into clinical care.

Case report: Use of markerless augmented reality system for ventriculoperitoneal shunt placement

Background

Ventriculoperitoneal (VP) shunt placement is one of the most commonly performed neurosurgical procedures, yet failure rates remain very high. Surface landmarks are typically used to guide VP shunt placement, but they are not reliable in identifying the target anatomy. Augmented reality (AR) is a promising new technology that has the potential to improve the accuracy and effectiveness of neurosurgical procedures. We describe the use of AR for the surgical planning of a VP shunt.

Case Description

A 62-year-old male with a history of subarachnoid hemorrhage presented with delayed hydrocephalus. A computed tomography scan was obtained that confirmed dilated ventricles, requiring a right VP shunt. The patient was brought to the operating room, where the AR system was used for visualization and planning.

Conclusion

In this study, we describe the use of AR for VP shunt placement. The AR system consists of a Microsoft HoloLens 2 head-mounted display and a novel markerless registration system, which was used to register patient-specific 3D models onto the patient's head for visualizing target anatomy and planning an operative approach. The AR system was used to plan the VP shunt placement in the operating room. This system is easy to use and provides a visualization of the patient's anatomy, which can be used to plan an optimal trajectory. We believe that this has the potential to improve the accuracy and outcomes of VP shunt placements, and further studies are needed to characterize the system's accuracy and benefits.

Augmented Reality in Neurosurgery: A New Paradigm for Training

Augmented reality (AR) involves the overlay of computer-generated images onto the user's real-world visual field to modify or enhance the user's visual experience. With respect to neurosurgery, AR integrates preoperative and intraoperative imaging data to create an enriched surgical experience that has been shown to improve surgical planning, refine neuronavigation, and reduce operation time. In addition, AR has the potential to serve as a valuable training tool for neurosurgeons in a way that minimizes patient risk while facilitating comprehensive training opportunities. The increased use of AR in neurosurgery over the past decade has led to innovative research endeavors aiming to develop novel, more efficient AR systems while also improving and refining present ones. In this review, we provide a concise overview of AR, detail current and emerging uses of AR in neurosurgery and neurosurgical training, discuss the limitations of AR, and provide future research directions. Following the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), 386 articles were initially identified. Two independent reviewers (GH and AC) assessed article eligibility for inclusion, and 31 articles are included in this review. The literature search included original (retrospective and prospective) articles and case reports published in English between 2013 and 2023. AR assistance has shown promise within neuro-oncology, spinal neurosurgery, neurovascular surgery, skull-base surgery, and pediatric neurosurgery. Intraoperative use of AR was found to primarily assist with surgical planning and neuronavigation. Similarly, AR assistance for neurosurgical training focused primarily on surgical planning and neuronavigation. However, studies included in this review utilize small sample sizes and remain largely in the preliminary phase. Thus, future research must be conducted to further refine AR systems before widespread intraoperative and educational use.

Mixed Reality as a Teaching Tool for Medical Students in Neurosurgery

Background and Objectives: Simulation-based learning within neurosurgery provides valuable and realistic educational experiences in a safe environment, enhancing the current teaching model. Mixed reality (MR) simulation can deliver a highly immersive experience through head-mounted displays and has become one of the most promising teaching tools in medical education. We aimed to identify whether an MR neurosurgical simulation module within the setting of an undergraduate neurosurgical hands-on course could improve the satisfaction of medical students. Materials and Methods: The quasi-experimental study with 223 medical students [120 in the conventional group (CG) and 103 in the MR-group (MRG)] was conducted at the University Hospital Münster, Münster, Germany. An MR simulation module was presented to the intervention group during an undergraduate neurosurgical hands-on course. Images of a skull fracture were reconstructed into 3D formats compatible with the MR-Viewer (Brainlab, Munich, Germany). Participants could interact virtually with the model and plan a surgical strategy using Magic Leap goggles. The experience was assessed by rating the course on a visual analog scale ranging from 1 (very poor) to 100 (very good) and an additional Likert-scale questionnaire. Results: The satisfaction score for CG and MRG were 89.3 ± 13.3 and 94.2 ± 7.5, respectively. The Wilcoxon rank-sum test showed that MR users (Mdn = 97.0, IQR = 4, n = 103) were significantly more satisfied than CG users (Mdn = 93.0, IQR = 10, n = 120; ln(W) = 8.99, p < 0.001) with moderate effect size (r^biserial = 0.30, CI95 [0.15, 0.43]), thus indicating that the utilization of MR-simulation is associated with greater satisfaction. Conclusions: This study reports a positive response from medical students towards MR as an educational tool. Feedback from the medical students encourages the adoption of disruptive technologies into medical school curricula.

Augmented reality in ultrasound-guided regional anaesthesia: useful tool or expensive toy?

Use of augmented reality is increasingly applied in medical education and practice. The main advantage of this technology is the display of relevant information in the visual field of multiple operators. Here we provide a critical analysis of the potential application of augmented reality in regional anaesthesia.

Virtual and augmented reality in biomedical engineering

BACKGROUND

In the future, extended reality technology will be widely used. People will be led to utilize virtual reality (VR) and augmented reality (AR) technologies in their daily lives, hobbies, numerous types of entertainment, and employment. Medical augmented reality has evolved with applications ranging from medical education to picture-guided surgery. Moreover, a bulk of research is focused on clinical applications, with the majority of research devoted to surgery or intervention, followed by rehabilitation and treatment applications. Numerous studies have also looked into the use of augmented reality in medical education and training.

METHODS

Using the databases Semantic Scholar, Web of Science, Scopus, IEEE Xplore, and ScienceDirect, a scoping review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. To find other articles, a manual search was also carried out in Google Scholar. This study presents studies carried out over the previous 14 years (from 2009 to 2023) in detail. We classify this area of study into the following categories: (1) AR and VR in surgery, which is presented in the following subsections: subsection A: MR in neurosurgery; subsection B: spine surgery; subsection C: oral and maxillofacial surgery; and subsection D: AR-enhanced human-robot interaction; (2) AR and VR in medical education presented in the following subsections; subsection A: medical training; subsection B: schools and curriculum; subsection C: XR in Biomedicine; (3) AR and VR for rehabilitation presented in the following subsections; subsection A: stroke rehabilitation during COVID-19; subsection B: cancer and VR, and (4) Millimeter-wave and MIMO systems for AR and VR.

RESULTS

In total, 77 publications were selected based on the inclusion criteria. Four distinct AR and/or VR applications groups could be differentiated: AR and VR in surgery (N = 21), VR and AR in Medical Education (N = 30), AR and VR for Rehabilitation (N = 15), and Millimeter-Wave and MIMO Systems for AR and VR (N = 7), where N is number of cited studies. We found that the majority of research is devoted to medical training and education, with surgical or interventional applications coming in second. The research is mostly focused on rehabilitation, therapy, and clinical applications. Moreover, the application of XR in MIMO has been the subject of numerous research.

CONCLUSION

Examples of these diverse fields of applications are displayed in this review as follows: (1) augmented reality and virtual reality in surgery; (2) augmented reality and virtual reality in medical education; (3) augmented reality and virtual reality for rehabilitation; and (4) millimeter-wave and MIMO systems for augmented reality and virtual reality.