Navigation-Linked Heads-Up Display in Intracranial Surgery: Early Experience

Augmented reality technology shortens aneurysm surgery learning curve for residents

OBJECTIVES

We aimed to prospectively investigate the benefit of using augmented reality (AR) for surgery residents learning aneurysm surgery.

MATERIALS AND METHODS

Eight residents were included, and divided into an AR group and a control group (4 in each group). Both groups were asked to locate an aneurysm with a blue circle on the same screenshot after their viewing of surgery videos from both AR and non-AR tests. Only the AR group was allowed to inspect and manipulate an AR holographic representation of the aneurysm in AR tests. The actual location of the aneurysm was defined by a yellow circle by an attending physician after each test. Localization deviation was determined by the distance between the blue and yellow circle.

RESULTS

Localization deviation was lower in the AR group than in the control group in the last 2 tests (AR Test 2: 2.7 ± 1.0 mm vs. 5.8 ± 4.1 mm, p = 0.01, non-AR Test 2: 2.1 ± 0.8 mm vs. 5.9 ± 5.8 mm, p < 0.001). The mean deviation was lower in non-AR Test 2 as compared to non-AR Test 1 in both groups (AR: p < 0.001, control: p = 0.391). The localization deviation of the AR group decreased from 8.1 ± 3.8 mm in Test 2 to 2.7 ± 1.0 mm in AR Test 2 (p < 0.001).

CONCLUSION

AR technology provides an effective and interactive way for neurosurgery training, and shortens the learning curve for residents in aneurysm surgery.

Application of artificial intelligence in brain arteriovenous malformations: Angioarchitectures, clinical symptoms and prognosis prediction

BACKGROUND

Artificial intelligence (AI) has rapidly advanced in the medical field, leveraging its intelligence and automation for the management of various diseases. Brain arteriovenous malformations (AVM) are particularly noteworthy, experiencing rapid development in recent years and yielding remarkable results. This paper aims to summarize the applications of AI in the management of AVMs management.

METHODS

Literatures published in PubMed during 1999-2022, discussing AI application in AVMs management were reviewed.

RESULTS

AI algorithms have been applied in various aspects of AVM management, particularly in machine learning and deep learning models. Automatic lesion segmentation or delineation is a promising application that can be further developed and verified. Prognosis prediction using machine learning algorithms with radiomic-based analysis is another meaningful application.

CONCLUSIONS

AI has been widely used in AVMs management. This article summarizes the current research progress, limitations and future research directions.

Heads-Up Micronavigation Reliability of Preoperative Transcranial Magnetic Stimulation Maps for the Motor Function: Comparison With Direct Cortical Stimulation

BACKGROUND AND OBJECTIVES

We aimed to assess the reliability of preoperative navigated transcranial magnetic stimulation (nTMS) maps for motor function as visualized intraoperatively with augmented reality heads-up display and to assess its accuracy via direct point-by-point comparison with the gold-standard direct cortical stimulation (DCS).

METHODS

From January 2022 to January 2023, candidates for surgical removal of lesions involving the motor pathways underwent preoperative nTMS assessment to obtain cortical maps of motor function. Intraoperatively and before tumor removal, nTMS maps were superimposed on the cortical surface, and DCS was performed on positive points with increasing current intensity until obtaining a positive response at 16 mA. The outcome of each stimulation was recorded to obtain discrimination metrics.

RESULTS

Twelve patients were enrolled (5 females [42%] vs 7 males [58%], mean age 62.9 ± 12.8 years), for a total of 304 investigated points. Agreement between nTMS and DCS was moderate (κ = 0.43, P < .005), with 0.66 (0.53-0.78) sensitivity, 0.87 (0.82-0.90) specificity, 0.50 (0.39-0.62) positive predictive values, 0.93 (0.89-0.95) negative predictive value, and 0.83 (0.79-0.87) accuracy. A loss of accuracy was observed with higher DCS current intensities.

CONCLUSION

We performed a point-by-point validation of preoperative nTMS maps for motor function using augmented reality visualization. The high negative predictive value and low positive predictive values highlight nTMS reliability to visualize safe cortical zones but not to identify critical functional areas, confirming previous findings of nTMS maps for the language function and suggesting the need for combined use of nTMS maps and DCS for optimal maximal safe resection.

The Inaugural "Century" of Mixed Reality in Cranial Surgery: Virtual Reality Rehearsal/Augmented Reality Guidance and Its Learning Curve in the First 100-Case, Single-Surgeon Series

BACKGROUND AND OBJECTIVE

Virtual reality (VR) refers to a computer-generated three-dimensional space in which a surgeon can interact with patient-specific anatomic models for surgical planning. Augmented reality (AR) is the technology that places computer-generated objects, including those made in VR, into the surgeon's visual space. Together, VR and AR are called mixed reality (MxR), and it is gaining importance in neurosurgery. MxR is helpful for selecting and creating templates for an optimal surgical approach and identifying key anatomic landmarks intraoperatively. By reporting our experience with the first 100 consecutive cases planned with VR and executed with AR, our objective is to detail the learning curve and encountered obstacles while adopting the new technology.

METHODS

This series includes the first 100 consecutive complex cranial cases of a single surgeon for which MxR was intended for use. Effectiveness of the VR rehearsal and AR guidance was analyzed for four specific contributions: (1) opening size, (2) precise craniotomy placement, (3) guidance toward anatomic landmarks or target, and (4) antitarget avoidance. Seventeen cases in the study cohort were matched with historical non-MxR cases for comparison of outcome parameters. The cases in which MxR failed were plotted over time to determine the nature of the "learning curve."

RESULTS

AR guidance was abandoned in eight operations because of technical problems, but problem-free application of MxR increased between the 44th and 63rd cases. This provides some evidence of proficiency acquisition in between. Comparing the 17 pairs of matched MxR and non-MxR cases, no statistically significant differences exist in the groups regarding blood loss, length of stay nor duration of surgery. Cases where MxR had above-expectation performances are highlighted.

CONCLUSION

MxR is a powerful tool that can help tailor operations to patient-specific anatomy and provide efficient intraoperative guidance without additional time for surgery or hospitalization.

Evaluation Metrics for Augmented Reality in Neurosurgical Preoperative Planning, Surgical Navigation, and Surgical Treatment Guidance: A Systematic Review

BACKGROUND AND OBJECTIVE

Recent years have shown an advancement in the development of augmented reality (AR) technologies for preoperative visualization, surgical navigation, and intraoperative guidance for neurosurgery. However, proving added value for AR in clinical practice is challenging, partly because of a lack of standardized evaluation metrics. We performed a systematic review to provide an overview of the reported evaluation metrics for AR technologies in neurosurgical practice and to establish a foundation for assessment and comparison of such technologies.

METHODS

PubMed, Embase, and Cochrane were searched systematically for publications on assessment of AR for cranial neurosurgery on September 22, 2022. The findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

The systematic search yielded 830 publications; 114 were screened full text, and 80 were included for analysis. Among the included studies, 5% dealt with preoperative visualization using AR, with user perception as the most frequently reported metric. The majority (75%) researched AR technology for surgical navigation, with registration accuracy, clinical outcome, and time measurements as the most frequently reported metrics. In addition, 20% studied the use of AR for intraoperative guidance, with registration accuracy, task outcome, and user perception as the most frequently reported metrics.

CONCLUSION

For quality benchmarking of AR technologies in neurosurgery, evaluation metrics should be specific to the risk profile and clinical objectives of the technology. A key focus should be on using validated questionnaires to assess user perception; ensuring clear and unambiguous reporting of registration accuracy, precision, robustness, and system stability; and accurately measuring task performance in clinical studies. We provided an overview suggesting which evaluation metrics to use per AR application and innovation phase, aiming to improve the assessment of added value of AR for neurosurgical practice and to facilitate the integration in the clinical workflow.

Advantages of computed tomography-based navigation in clipping distal anterior cerebral artery aneurysms: a retrospective cohort study

Background

The occurrence rate of distal anterior cerebral artery (DACA) aneurysms is relatively low, primarily due to their deep-seated location, which makes surgical clamping challenging. The objective of this study was to investigate the efficacy and safety of computed tomography (CT) navigation-assisted clipping of DACA aneurysms compared to traditional clipping without navigation.

Methods

A retrospective cohort study involving retrospective data collection was performed. The retrospective analysis was conducted on 139 patients with ruptured DACA aneurysms who underwent clipping. From January 2013 to November 2021, 164 patients were retrieved at the Department of Neurosurgery, Renmin Hospital of Wuhan University. The inclusion criteria were patients diagnosed with DACA aneurysms via CT angiography (CTA) or digital subtraction angiography (DSA), those with complete clinical data, and those who underwent craniotomy for aneurysm clipping. Meanwhile, the exclusion criteria were as follows: aneurysm recurrence, traumatic brain injury or surgery history, blood disorders or recent anticoagulant use, and severe organ dysfunction. Data on gender, age, Hunt-Hess grade, Fisher grade, modified Rankin Scale (mRS) score, aneurysm location, hospitalization time, aneurysm found time (the duration from incision to aneurysm discovery), and intraoperative bleeding volume were collected from medical records and neurosurgical databases. Patients were followed up in the clinic or by telephone in May 2022. All patients were divided into a navigation group or a traditional group for statistical analysis.

Results

No statistically significant differences were observed in age, sex, Fisher grade, Hunt-Hess grade, hospitalization time, or aneurysm site between the navigation group and traditional group (P>0.05). Intraoperative blood loss was lower in the navigation group than in the traditional group {370 [280-460] vs. 430 [310-610] mL, P=0.045}. Patients in the traditional group had a shorter aneurysm found time than did those in the navigation group {49 [42-53] vs. 79 [63-84] min, P<0.001}. There was no significant difference in the mRS score at hospital discharge (P=0.336) or follow-up (P=0.157) between the two groups.

Conclusions

CT neuronavigation-assisted microsurgery for clipping DACA aneurysms may improve surgical accuracy, shorten the time to locate aneurysms, and reduce intraoperative blood loss. Although no significant difference in prognosis was observed, this technique shows promise as a safe and effective alternative to traditional clipping without navigation.

Augmented reality for intracranial meningioma resection: a mini-review

Augmented reality (AR) integrates computer-generated content and real-world scenarios. Artificial intelligence's continuous development has allowed AR to be integrated into medicine. Neurosurgery has progressively introduced image-guided technologies. Integration of AR into the operating room has permitted a new perception of neurosurgical diseases, not only for neurosurgical planning, patient positioning, and incision design but also for intraoperative maneuvering and identification of critical neurovascular structures and tumor boundaries. Implementing AR, virtual reality, and mixed reality has introduced neurosurgeons into a new era of artificial interfaces. Meningiomas are the most frequent primary benign tumors commonly related to paramount neurovascular structures and bone landmarks. Integration of preoperative 3D reconstructions used for surgical planning into AR can now be inserted into the microsurgical field, injecting information into head-up displays and microscopes with integrated head-up displays, aiming to guide neurosurgeons intraoperatively to prevent potential injuries. This manuscript aims to provide a mini-review of the usage of AR for intracranial meningioma resection.

Intra-operative applications of augmented reality in glioma surgery: a systematic review

Background

Augmented reality (AR) is increasingly being explored in neurosurgical practice. By visualizing patient-specific, three-dimensional (3D) models in real time, surgeons can improve their spatial understanding of complex anatomy and pathology, thereby optimizing intra-operative navigation, localization, and resection. Here, we aimed to capture applications of AR in glioma surgery, their current status and future potential.

Methods

A systematic review of the literature was conducted. This adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline. PubMed, Embase, and Scopus electronic databases were queried from inception to October 10, 2022. Leveraging the Population, Intervention, Comparison, Outcomes, and Study design (PICOS) framework, study eligibility was evaluated in the qualitative synthesis. Data regarding AR workflow, surgical application, and associated outcomes were then extracted. The quality of evidence was additionally examined, using hierarchical classes of evidence in neurosurgery.

Results

The search returned 77 articles. Forty were subject to title and abstract screening, while 25 proceeded to full text screening. Of these, 22 articles met eligibility criteria and were included in the final review. During abstraction, studies were classified as "development" or "intervention" based on primary aims. Overall, AR was qualitatively advantageous, due to enhanced visualization of gliomas and critical structures, frequently aiding in maximal safe resection. Non-rigid applications were also useful in disclosing and compensating for intra-operative brain shift. Irrespective, there was high variance in registration methods and measurements, which considerably impacted projection accuracy. Most studies were of low-level evidence, yielding heterogeneous results.

Conclusions

AR has increasing potential for glioma surgery, with capacity to positively influence the onco-functional balance. However, technical and design limitations are readily apparent. The field must consider the importance of consistency and replicability, as well as the level of evidence, to effectively converge on standard approaches that maximize patient benefit.

Augmented Reality Support for Anterior Decompression and Fusion Using Floating Method for Cervical Ossification of the Posterior Longitudinal Ligament

Anterior decompression and fusion (ADF) using the floating method for cervical ossification of the posterior longitudinal ligament (OPLL) is an ideal surgical technique, but it has a specific risk of insufficient decompression caused by the impingement of residual ossification. Augmented reality (AR) support is a novel technology that enables the superimposition of images onto the view of a surgical field. AR technology was applied to ADF for cervical OPLL to facilitate intraoperative anatomical orientation and OPLL identification. In total, 14 patients with cervical OPLL underwent ADF with microscopic AR support. The outline of the OPLL and the bilateral vertebral arteries was marked after intraoperative CT, and the reconstructed 3D image data were transferred and linked to the microscope. The AR microscopic view enabled us to visualize the ossification outline, which could not be seen directly in the surgical field, and allowed sufficient decompression of the ossification. Neurological disturbances were improved in all patients. No cases of serious complications, such as major intraoperative bleeding or reoperation due to the postoperative impingement of the floating OPLL, were registered. To our knowledge, this is the first report of the introduction of microscopic AR into ADF using the floating method for cervical OPLL with favorable clinical results.

Traditional analogue vs. three-dimensional digital visualization used in ophthalmic surgery

The visualization of the surgical process remains a topical issue in cataract surgery. The review presents the history of visualization technique in ophthalmic surgery and compares the main current analogue and 3D digital technologies. The advent of 3D imaging systems in clinical practice has helped solve many issues associated with the use of standard analogue microscopes. These issues include limited focus and field of vision, the need to use a large amount of light, which increases the risk of iatrogenic retinal phototoxicity, the surgeon's attachment to the microscope and, consequently, a high load on the surgeon's visual apparatus when using eyepieces, as well as on their back and neck muscles.

Augmented and virtual reality usage in awake craniotomy: a systematic review

Augmented and virtual reality (AR, VR) are becoming promising tools in neurosurgery. AR and VR can reduce challenges associated with conventional approaches via the simulation and mimicry of specific environments of choice for surgeons. Awake craniotomy (AC) enables the resection of lesions from eloquent brain areas while monitoring higher cortical and subcortical functions. Evidence suggests that both surgeons and patients benefit from the various applications of AR and VR in AC. This paper investigates the application of AR and VR in AC and assesses its prospective utility in neurosurgery. A systematic review of the literature was performed using PubMed, Scopus, and Web of Science databases in accordance with the PRISMA guidelines. Our search results yielded 220 articles. A total of six articles consisting of 118 patients have been included in this review. VR was used in four papers, and the other two used AR. Tumour was the most common pathology in 108 patients, followed by vascular lesions in eight patients. VR was used for intraoperative mapping of language, vision, and social cognition, while AR was incorporated in preoperative training of white matter dissection and intraoperative visualisation and navigation. Overall, patients and surgeons were satisfied with the applications of AR and VR in their cases. AR and VR can be safely incorporated during AC to supplement, augment, or even replace conventional approaches in neurosurgery. Future investigations are required to assess the feasibility of AR and VR in various phases of AC.

Seventy-five years of neurosurgery residency training at The Mount Sinai Hospital

The Department of Neurosurgery's residency program at The Mount Sinai Hospital was founded in 1946. The department has its origins in 1914 as a division of general surgery, with Charles Elsberg at the helm. Neurosurgery then became a separate department in 1932 under the leadership of Ira Cohen. Dr. Cohen oversaw the creation of the neurosurgery residency training program 75 years ago. Since its inception, the residency program has graduated 120 residents. For more than 100 years, The Mount Sinai Hospital has been a site of clinical excellence, groundbreaking research, and technological innovation in neurosurgery. Currently, the Department of Neurosurgery has 39 clinical faculty members, performs more than 5300 surgeries and endovascular procedures annually, and is in the top 25 neurosurgical departments for NIH funding.

Use of Neuronavigation and Augmented Reality in Transsphenoidal Pituitary Adenoma Surgery

The aim of this study was to report on the clinical experience with microscope-based augmented reality (AR) in transsphenoidal surgery compared to the classical microscope-based approach. AR support was established using the head-up displays of the operating microscope, with navigation based on fiducial-/surface- or automatic intraoperative computed tomography (iCT)-based registration. In a consecutive single surgeon series of 165 transsphenoidal procedures, 81 patients underwent surgery without AR support and 84 patients underwent surgery with AR support. AR was integrated straightforwardly within the workflow. ICT-based registration increased AR accuracy significantly (target registration error, TRE, 0.76 ± 0.33 mm) compared to the landmark-based approach (TRE 1.85 ± 1.02 mm). The application of low-dose iCT protocols led to a significant reduction in applied effective dosage being comparable to a single chest radiograph. No major vascular or neurological complications occurred. No difference in surgical time was seen, time to set-up patient registration prolonged intraoperative preparation time on average by twelve minutes (32.33 ± 13.35 vs. 44.13 ± 13.67 min), but seems justifiable by the fact that AR greatly and reliably facilitated surgical orientation and increased surgeon comfort and patient safety, not only in patients who had previous transsphenoidal surgery but also in cases with anatomical variants. Automatic intraoperative imaging-based registration is recommended.

Augmented reality–assisted microsurgical resection of brain arteriovenous malformations: illustrative case

BACKGROUND

Arteriovenous malformations (AVMs) of the brain are vessel conglomerates of feeding arteries and draining veins that carry a risk of spontaneous and intraoperative rupture. Augmented reality (AR)-assisted neuronavigation permits continuous, real-time, updated visualization of navigation information through a heads-up display, thereby potentially improving the safety of surgical resection of AVMs.

OBSERVATIONS

The authors report a case of a 37-year-old female presenting with a 2-year history of recurrent falls due to intermittent right-sided weakness and increasing clumsiness in the right upper extremity. Magnetic resonance imaging, magnetic resonance angiography, and cerebral angiography of the brain revealed a left parietal Spetzler-Martin grade III AVM. After endovascular embolization of the AVM, microsurgical resection using an AR-assisted neuronavigation system was performed. Postoperative angiography confirmed complete obliteration of arteriovenous shunting. The postsurgical course was unremarkable, and the patient remains in excellent health.

LESSONS

Our case describes the operative setup and intraoperative employment of AR-assisted neuronavigation for AVM resection. Application of this technology may improve workflow and enhance patient safety.

The use of augmented reality in transsphenoidal surgery: A systematic review

BACKGROUND

Augmented reality (AR) has become a promising tool in neurosurgery. It can minimise the anatomical challenges faced by conventional endoscopic or microscopic transsphenoidal reoperations and can assist in intraoperative guidance, preoperative planning, and surgical training.

OBJECTIVES

The aims of this systematic review are to describe, compare, and evaluate the use of AR in endoscopic and microscopic transsphenoidal surgery, incorporating the latest primary research.

METHODS

A systematic review was performed to explore and evaluate existing primary evidence for using AR in transsphenoidal surgery. A comprehensive search of MEDLINE and EMBASE was conducted from database inception to 11th August 2021 for primary data on the use of AR in microscopic and endoscopic endonasal skull base surgery. Additional articles were identified through searches on PubMed, Google Scholar, JSTOR, SCOPUS, Web of Science, Engineering Village, IEEE transactions, and HDAS. A synthesis without meta-analysis (SWiM) analysis was employed quantitatively and qualitatively on the impact of AR on landmark identification, intraoperative navigation, accuracy, time, surgeon experience, and patient outcomes.

RESULTS

In this systematic review, 17 studies were included in the final analysis. The main findings were that AR provides a convincing improvement to landmark identification, intraoperative navigation, and surgeon experience in transsphenoidal surgery, with a further positive effect on accuracy and time. It did not demonstrate a convincing positive effect on patient outcomes. No studies reported comparative mortalities, morbidities, or cost-benefit indications.

CONCLUSION

AR-guided transsphenoidal surgery, both endoscopic and microscopic, is associated with an overall improvement in the areas of intraoperative guidance and surgeon experience as compared with their conventional counterparts. However, literature on this area, particularly comparative data and evidence, is very limited. More studies with similar methodologies and quantitative outcomes are required to perform appropriate meta-analyses and to draw significant conclusions.

Current status of augmented reality in cerebrovascular surgery: a systematic review

Augmented reality (AR) is an adjuvant tool in neuronavigation to improve spatial and anatomic understanding. The present review aims to describe the current status of intraoperative AR for the treatment of cerebrovascular pathology. A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The following databases were searched: PubMed, Science Direct, Web of Science, and EMBASE up to December, 2020. The search strategy consisted of "augmented reality," "AR," "cerebrovascular," "navigation," "neurovascular," "neurosurgery," and "endovascular" in both AND and OR combinations. Studies included were original research articles with intraoperative application. The manuscripts were thoroughly examined for study design, outcomes, and results. Sixteen studies were identified describing the use of intraoperative AR in the treatment of cerebrovascular pathology. A total of 172 patients were treated for 190 cerebrovascular lesions using intraoperative AR. The most common treated pathology was intracranial aneurysms. Most studies were cases and there was only a case-control study. A head-up display system in the microscope was the most common AR display. AR was found to be useful for tailoring the craniotomy, dura opening, and proper identification of donor and recipient vessels in vascular bypass. Most AR systems were unable to account for tissue deformation. This systematic review suggests that intraoperative AR is becoming a promising and feasible adjunct in the treatment of cerebrovascular pathology. It has been found to be a useful tool in the preoperative planning and intraoperative guidance. However, its clinical benefits remain to be seen.

Visualising the Future of Orthopaedic Surgery: A Novel Application of Wireless Smart Glasses to Visualise Intraoperative Imaging

Smart glasses can provide a heads-up display of advanced imaging intraoperatively. In recent years, growing attention has been drawn to the use of smart glasses as an assistive technology to improve both efficiency and ergonomics in a surgical setting. Previous studies have reported improved surgical accuracy, efficiency, and ergonomics with its usage, but its effectiveness as a form of intraoperative heads-up display remains elusive in the context of orthopaedics. This study provides a novel account of a wireless set-up of the Moverio BT-35E Smart Glasses (Suwa, Japan: Epson Inc.), incorporated in a complex orthopaedic procedure.

Hind-foot nailing was performed on a patient with a complex open ankle fracture and multiple co-morbidities. Smart glasses were worn by the primary surgeon throughout the procedure to provide heads-up visualisation of the intraoperative fluoroscopy.

In our surgical case, the surgeon experienced improved ergonomics and reduced disruption to focus with the use of smart glasses. The wireless set-up provided excellent signal transmission throughout the duration of the procedure.

The wireless set-up of smart glasses is a potential solution for common occupational risks imposed on orthopaedic surgeons. Smart glasses minimise musculoskeletal strain from switching of vision from monitor to patient, whilst the wireless set-up allows for efficient use of space in an operating theatre and may potentially limit radiation exposure. Lastly, ergonomic benefits may increase the efficiency of movement for the surgeon, decreasing operative duration, and in turn minimising the risk of surgical complications for patients.

Segmentation techniques of brain arteriovenous malformations for 3D visualization: a systematic review

BACKGROUND

Visualization, analysis and characterization of the angioarchitecture of a brain arteriovenous malformation (bAVM) present crucial steps for understanding and management of these complex lesions. Three-dimensional (3D) segmentation and 3D visualization of bAVMs play hereby a significant role. We performed a systematic review regarding currently available 3D segmentation and visualization techniques for bAVMs.

METHODS

PubMed, Embase and Google Scholar were searched to identify studies reporting 3D segmentation techniques applied to bAVM characterization. Category of input scan, segmentation (automatic, semiautomatic, manual), time needed for segmentation and 3D visualization techniques were noted.

RESULTS

Thirty-three studies were included. Thirteen (39%) used MRI as baseline imaging modality, 9 used DSA (27%), and 7 used CT (21%). Segmentation through automatic algorithms was used in 20 (61%), semiautomatic segmentation in 6 (18%), and manual segmentation in 7 (21%) studies. Median automatic segmentation time was 10 min (IQR 33), semiautomatic 25 min (IQR 73). Manual segmentation time was reported in only one study, with the mean of 5-10 min. Thirty-two (97%) studies used screens to visualize the 3D segmentations outcomes and 1 (3%) study utilized a heads-up display (HUD). Integration with mixed reality was used in 4 studies (12%).

CONCLUSIONS

A golden standard for 3D visualization of bAVMs does not exist. This review describes a tendency over time to base segmentation on algorithms trained with machine learning. Unsupervised fuzzy-based algorithms thereby stand out as potential preferred strategy. Continued efforts will be necessary to improve algorithms, integrate complete hemodynamic assessment and find innovative tools for tridimensional visualization.

Augmented reality visualization in brain lesions: a prospective randomized controlled evaluation of its potential and current limitations in navigated microneurosurgery

Background

Augmented reality (AR) has the potential to support complex neurosurgical interventions by including visual information seamlessly. This study examines intraoperative visualization parameters and clinical impact of AR in brain tumor surgery.

Methods

Fifty-five intracranial lesions, operated either with AR-navigated microscope (n = 39) or conventional neuronavigation (n = 16) after randomization, have been included prospectively. Surgical resection time, duration/type/mode of AR, displayed objects (n, type), pointer-based navigation checks (n), usability of control, quality indicators, and overall surgical usefulness of AR have been assessed.

Results

AR display has been used in 44.4% of resection time. Predominant AR type was navigation view (75.7%), followed by target volumes (20.1%). Predominant AR mode was picture-in-picture (PiP) (72.5%), followed by 23.3% overlay display. In 43.6% of cases, vision of important anatomical structures has been partially or entirely blocked by AR information. A total of 7.7% of cases used MRI navigation only, 30.8% used one, 23.1% used two, and 38.5% used three or more object segmentations in AR navigation. A total of 66.7% of surgeons found AR visualization helpful in the individual surgical case. AR depth information and accuracy have been rated acceptable (median 3.0 vs. median 5.0 in conventional neuronavigation). The mean utilization of the navigation pointer was 2.6 × /resection hour (AR) vs. 9.7 × /resection hour (neuronavigation); navigation effort was significantly reduced in AR (P < 0.001).

Conclusions

The main benefit of HUD-based AR visualization in brain tumor surgery is the integrated continuous display allowing for pointer-less navigation. Navigation view (PiP) provides the highest usability while blocking the operative field less frequently. Visualization quality will benefit from improvements in registration accuracy and depth impression.

German clinical trials registration number.

DRKS00016955.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00701-021-05045-1.

Intraoperative use of heads-up display in skull base surgery

In this video, the authors highlight the applications of virtual reality and heads-up display in skull base surgery by presenting the case of a 45-year-old woman with an incidental large clinoid meningioma extending into the posterior fossa. The patient underwent preoperative endovascular tumor embolization to facilitate tumor resection and reduce blood loss, followed by a right pterional craniotomy. The use of intraoperative Doppler, intraoperative neurophysiological monitoring, and endoscope-assisted microsurgery is also featured. A subtotal resection was planned given tumor encasement of the posterior communicating and anterior choroidal arteries. No new neurological deficits were noted after the surgical procedure.

The video can be found here: https://stream.cadmore.media/r10.3171/2021.10.FOCVID21177

Augmented Reality-Assisted Craniotomy for Parasagittal and Convexity En Plaque Meningiomas and Custom-Made Cranio-Plasty: A Preliminary Laboratory Report

BACKGROUND

This report discusses the utility of a wearable augmented reality platform in neurosurgery for parasagittal and convexity en plaque meningiomas with bone flap removal and custom-made cranioplasty.

METHODS

A real patient with en plaque cranial vault meningioma with diffuse and extensive dural involvement, extracranial extension into the calvarium, and homogeneous contrast enhancement on gadolinium-enhanced T1-weighted MRI, was selected for this case study. A patient-specific manikin was designed starting with the segmentation of the patient's preoperative MRI images to simulate a craniotomy procedure. Surgical planning was performed according to the segmented anatomy, and customized bone flaps were designed accordingly. During the surgical simulation stage, the VOSTARS head-mounted display was used to accurately display the planned craniotomy trajectory over the manikin skull. The precision of the craniotomy was assessed based on the evaluation of previously prepared custom-made bone flaps.

RESULTS

A bone flap with a radius 0.5 mm smaller than the radius of an ideal craniotomy fitted perfectly over the performed craniotomy, demonstrating an error of less than ±1 mm in the task execution. The results of this laboratory-based experiment suggest that the proposed augmented reality platform helps in simulating convexity en plaque meningioma resection and custom-made cranioplasty, as carefully planned in the preoperative phase.

CONCLUSIONS

Augmented reality head-mounted displays have the potential to be a useful adjunct in tumor surgical resection, cranial vault lesion craniotomy and also skull base surgery, but more study with large series is needed.

Holographic mixed-reality neuronavigation with a head-mounted device: technical feasibility and clinical application

OBJECTIVE

The authors aimed to evaluate the technical feasibility of a mixed-reality neuronavigation (MRN) system with a wearable head-mounted device (HMD) and to determine its clinical application and accuracy.

METHODS

A semiautomatic registration MRN system on HoloLens smart glasses was developed and tested for accuracy and feasibility. Thirty-seven patients with intracranial lesions were prospectively identified. For each patient, multimodal imaging-based holograms of lesions, markers, and surrounding eloquent structures were created and then imported to the MRN HMD. After a point-based registration, the holograms were projected onto the patient's head and observed through the HMD. The contour of the holograms was compared with standard neuronavigation (SN). The projection of the lesion boundaries perceived by the neurosurgeon on the patient's scalp was then marked with MRN and SN. The distance between the two contours generated by MRN and SN was measured so that the accuracy of MRN could be assessed.

RESULTS

MRN localization was achieved in all patients. The mean additional time required for MRN was 36.3 ± 6.3 minutes, in which the mean registration time was 2.6 ± 0.9 minutes. A trend toward a shorter time required for preparation was observed with the increase of neurosurgeon experience with the MRN system. The overall median deviation was 4.1 mm (IQR 3.0 mm-4.7 mm), and 81.1% of the lesions localized by MRN were found to be highly consistent with SN (deviation < 5.0 mm). There was a significant difference between the supine position and the prone position (3.7 ± 1.1 mm vs 5.4 ± 0.9 mm, p = 0.001). The magnitudes of deviation vectors did not correlate with lesion volume (p = 0.126) or depth (p = 0.128). There was no significant difference in additional operating time between different operators (37.4 ± 4.8 minutes vs 34.6 ± 4.8 minutes, p = 0.237) or in localization deviation (3.7 ± 1.0 mm vs 4.6 ± 1.5 mm, p = 0.070).

CONCLUSIONS

This study provided a complete set of a clinically applicable workflow on an easy-to-use MRN system using a wearable HMD, and has shown its technical feasibility and accuracy. Further development is required to improve the accuracy and clinical efficacy of this system.

Do-It-Yourself Augmented Reality Heads-Up Display (DIY AR-HUD): A Technical Note

BACKGROUND

We present a "Do-It-Yourself" method to build an affordable augmented reality heads-up display system (AR-HUD) capable of displaying intraoperative images. All components are commercially available products, which the surgeons may use in their own practice for educational and research purposes.

METHODS

Moverio BT 35-E smart glasses were connected to operating room imaging modalities (ie, fluoroscopy and 3D navigation platforms) via a high-definition multimedia interface (HDMI) converter, allowing for continuous high-definition video transmission. The addition of an HDMI transmitter-receiver makes the AR-HUD system wireless.

RESULTS

We used our AR-HUD system in 3 patients undergoing instrumented spinal fusion. AR-HUD projected fluoroscopy images onto the surgical field, eliminating shift of surgeon focus and procedure interruption, with only a 40- to 100-ms delay in transmission, which was not clinically impactful.

CONCLUSIONS

An affordable AR-HUD capable of displaying real-time information into the surgeon's view can be easily designed, built, and tested in surgical practice. As wearable heads-up display technology continues to evolve rapidly, individual components presented here may be substituted to improve its functionality and usability. Surgeons are in a unique position to conduct clinical testing in the operating room environment to optimize the augmented reality system for surgical use.

The utility of augmented reality in lateral skull base surgery: A preliminary report

OBJECTIVE

To discuss the utility of augmented reality in lateral skull base surgery.

PATIENTS

Those undergoing lateral skull base surgery at our institution.

INTERVENTION(S)

Cerebellopontine angle tumor resection using an augmented reality interface.

MAIN OUTCOME MEASURE(S)

Ease of use, utility of, and future directions of augmented reality in lateral skull base surgery.

RESULTS

Anecdotally we have found an augmented reality interface helpful in simulating cerebellopontine angle tumor resection as well as assisting in planning the incision and craniotomy.

CONCLUSIONS

Augmented reality has the potential to be a useful adjunct in lateral skull base surgery, but more study is needed with large series.

Applications of augmented reality in the neurosurgical operating room: A systematic review of the literature

Advancements in imaging techniques are key forces of progress in neurosurgery. The importance of accurate visualization of intraoperative anatomy cannot be overemphasized and is commonly delivered through traditional neuronavigation. Augmented Reality (AR) technology has been tested and applied widely in various neurosurgical subspecialties in intraoperative, clinical use and shows promise for the future. This systematic review of the literature explores the ways in which AR technology has been successfully brought into the operating room (OR) and incorporated into clinical practice. A comprehensive literature search was performed in the following databases from inception-April 2020: Ovid MEDLINE, Ovid EMBASE, and The Cochrane Library. Studies retrieved were then screened for eligibility against predefined inclusion/exclusion criteria. A total of 54 articles were included in this systematic review. The studies were sub- grouped into brain and spine subspecialties and analyzed for their incorporation of AR in the neurosurgical clinical setting. AR technology has the potential to greatly enhance intraoperative visualization and guidance in neurosurgery beyond the traditional neuronavigation systems. However, there are several key challenges to scaling the use of this technology and bringing it into standard operative practice including accurate and efficient brain segmentation of magnetic resonance imaging (MRI) scans, accounting for brain shift, reducing coregistration errors, and improving the AR device hardware. There is also an exciting potential for future work combining AR with multimodal imaging techniques and artificial intelligence to further enhance its impact in neurosurgery.

Early Experience With Virtual and Synchronized Augmented Reality Platform for Preoperative Planning and Intraoperative Navigation: A Case Series

BACKGROUND

Virtual reality (VR) allows for presurgical planning. Intraoperatively, augmented reality (AR) enables integration of segmented anatomic information with neuronavigation into the microsurgical scene to provide guidance without workflow disruption. Combining VR and AR solutions may help guide microsurgical technique to improve safety, efficiency, and ergonomics.

OBJECTIVE

To describe a VR/AR platform that provides VR planning and intraoperative guidance via microscope ocular injection of a comprehensive AR overlay of patient-specific 360°/3D anatomic model aligned and synchronized with neuronavigation.

METHODS

Custom 360° models from preoperative imaging of 49 patients were utilized for preoperative planning using a VR-based surgical rehearsal platform. Each model was imported to SyncAR, the platform's intraoperative counterpart, which was coregistered with Medtronic StealthStation S8 and Zeiss or Leica microscope. The model was injected into the microscope oculars and referenced throughout by adjusting overlay opacity. For anatomic shifts or misalignment, the overlay was reregistered via manual realignment with known landmarks.

RESULTS

No SyncAR-related complications occurred. SyncAR contributed positively to the 3D understanding of patient-specific anatomy and ability to operate. Preoperative planning and intraoperative AR with 360° models allowed for more precise craniotomy planning and execution. SyncAR was useful for guiding dissection, identifying critical structures including hidden anatomy, understanding regional anatomy, and facilitating resection. Manual realignment was performed in 48/49 surgeries. Gross total resection was achieved in 34/40 surgeries. All aneurysm clipping and microvascular decompression procedures were completed without complications.

CONCLUSION

SyncAR combined with VR planning has potential to enhance surgical performance by providing critical information in a user-friendly, continuously available, heads-up display format.

Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

Background: While performing surgeries in the OR, surgeons and assistants often need to access several information regarding surgical planning and/or procedures related to the surgery itself, or the accessory equipment to perform certain operations. The accessibility of this information often relies on the physical presence of technical and medical specialists in the OR, which is increasingly difficult due to the number of limitations imposed by the COVID emergency to avoid overcrowded environments or external personnel. Here, we analyze several scenarios where we equipped OR personnel with augmented reality (AR) glasses, allowing a remote specialist to guide OR operations through voice and ad-hoc visuals, superimposed to the field of view of the operator wearing them.

Methods: This study is a preliminary case series of prospective collected data about the use of AR-assistance in spine surgery from January to July 2020. The technology has been used on a cohort of 12 patients affected by degenerative lumbar spine disease with lumbar sciatica co-morbidities. Surgeons and OR specialists were equipped with AR devices, customized with P2P videoconference commercial apps, or customized holographic apps. The devices were tested during surgeries for lumbar arthrodesis in a multicenter experience involving author's Institutions.

Findings: A total number of 12 lumbar arthrodesis have been performed while using the described AR technology, with application spanning from telementoring (3), teaching (2), surgical planning superimposition and interaction with the hologram using a custom application for Microsoft hololens (1). Surgeons wearing the AR goggles reported a positive feedback as for the ergonomy, wearability and comfort during the procedure; being able to visualize a 3D reconstruction during surgery was perceived as a straightforward benefit, allowing to speed-up procedures, thus limiting post-operational complications. The possibility of remotely interacting with a specialist on the glasses was a potent added value during COVID emergency, due to limited access of non-resident personnel in the OR.

Interpretation: By allowing surgeons to overlay digital medical content on actual surroundings, augmented reality surgery can be exploited easily in multiple scenarios by adapting commercially available or custom-made apps to several use cases. The possibility to observe directly the operatory theater through the eyes of the surgeon might be a game-changer, giving the chance to unexperienced surgeons to be virtually at the site of the operation, or allowing a remote experienced operator to guide wisely the unexperienced surgeon during a procedure.

Decision Support Capabilities of Telemedicine in Emergency Prehospital Care: Systematic Review

Background

Telemedicine offers a unique opportunity to improve coordination and administration for urgent patient care remotely. In an emergency setting, it has been used to support first responders by providing telephone or video consultation with specialists at hospitals and through the exchange of prehospital patient information. This technological solution is evolving rapidly, yet there is a concern that it is being implemented without a demonstrated clinical need and effectiveness as well as without a thorough economic evaluation.

Objective

Our objective is to systematically review whether the clinical outcomes achieved, as reported in the literature, favor telemedicine decision support for medical interventions during prehospital care.

Methods

This systematic review included peer-reviewed journal articles. Searches of 7 databases and relevant reviews were conducted. Eligibility criteria consisted of studies that covered telemedicine as data- and information-sharing and two-way teleconsultation platforms, with the objective of supporting medical decisions (eg, diagnosis, treatment, and receiving hospital decision) in a prehospital emergency setting. Simulation studies and studies that included pediatric populations were excluded. The procedures in this review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. The Risk Of Bias In Non-randomised Studies–of Interventions (ROBINS-I) tool was used for the assessment of risk of bias. The results were synthesized based on predefined aspects of medical decisions that are made in a prehospital setting, which include diagnostic decision support, receiving facility decisions, and medical directions for treatment. All data extractions were done by at least two reviewers independently.

Results

Out of 42 full-text reviews, 7 were found eligible. Diagnostic support and medical direction and decision for treatments were often reported. A key finding of this review was the high agreement between prehospital diagnoses via telemedicine and final in-hospital diagnoses, as supported by quantitative evidence. However, a majority of the articles described the clinical value of having access to remote experts without robust quantitative data. Most telemedicine solutions were evaluated within a feasibility or short-term preliminary study. In general, the results were positive for telemedicine use; however, biases, due to preintervention confounding factors and a lack of documentation on quality assurance and protocol for telemedicine activation, make it difficult to determine the direct effect on patient outcomes.

Conclusions

The information-sharing capacity of telemedicine enables access to remote experts to support medical decision making on scene or in prolonged field care. The influence of human and technology factors on patient care is poorly understood and documented.

Current Accuracy of Augmented Reality Neuronavigation Systems: Systematic Review and Meta-Analysis

BACKGROUND

Augmented reality neuronavigation (ARN) systems can overlay three-dimensional anatomy and disease without the need for a two-dimensional external monitor. Accuracy is crucial for their clinical applicability. We performed a systematic review regarding the reported accuracy of ARN systems and compared them with the accuracy of conventional infrared neuronavigation (CIN).

METHODS

PubMed and Embase were searched for ARN and CIN systems. For ARN, type of system, method of patient-to-image registration, accuracy method, and accuracy of the system were noted. For CIN, navigation accuracy, expressed as target registration error (TRE), was noted. A meta-analysis was performed comparing the TRE of ARN and CIN systems.

RESULTS

Thirty-five studies were included, 12 for ARN and 23 for CIN. ARN systems could be divided into head-mounted display and heads-up display. In ARN, 4 methods were encountered for patient-to-image registration, of which point-pair matching was the one most frequently used. Five methods for assessing accuracy were described. Ninety-four TRE measurements of ARN systems were compared with 9058 TRE measurements of CIN systems. Mean TRE was 2.5 mm (95% confidence interval, 0.7-4.4) for ARN systems and 2.6 mm (95% confidence interval, 2.1-3.1) for CIN systems.

CONCLUSIONS

In ARN, there seems to be lack of agreement regarding the best method to assess accuracy. Nevertheless, ARN systems seem able to achieve an accuracy comparable to CIN systems. Future studies should be prospective and compare TREs, which should be measured in a standardized fashion.

Virtual Reality in Neurosurgery: "Can You See It?"-A Review of the Current Applications and Future Potential

Virtual reality (VR) technology had its early development in the 1960s in the U.S. Air Force and has since evolved into a budding area of scientific research with many practical medical purposes. From medical education to resident training to the operating room, VR has provided tangible benefits to learners and trainees and has also improved surgery through enhanced preoperative planning and efficiency in the operating room. Neurosurgery is a particularly complex field of medicine, in which VR has blossomed into a tool with great usefulness and promise. In spinal surgery, VR simulation has allowed for the practice of innovative minimally invasive procedures. In cranial surgery, VR has excelled in helping neurosurgeons design unique patient-specific approaches to particularly challenging tumor excisions. In neurovascular surgery, VR has helped trainees practice and perfect procedures requiring high levels of dexterity to minimize intraoperative complications and patient radiation exposure. In peripheral nerve surgery, VR has allowed surgeons to gain increased practice and comfort with complex microsurgeries such as nerve decompression. Overall, VR continues to increase its potential in neurosurgery and is poised to benefit patients in a multitude of ways. Although cost-prohibiting, legal, and ethical challenges surrounding this technology must be considered, future research and more direct quantitative outcome comparisons between standard and VR-supplemented procedures would help provide more direction regarding the feasibility of widespread adoption of VR technology in neurosurgery.

Augmented Reality in Superficial Temporal Artery to Middle Cerebral Artery Bypass Surgery: Technical Note

BACKGROUND

Augmented reality (AR) applied to surgery refers to the virtual superimposition of computer-generated anatomical information on the surgical field. AR assistance in extracranial-intracranial (EC-IC) bypass revascularization surgery has been reported to be a helpful technical adjunct.

OBJECTIVE

To describe our experience of using AR in superficial temporal artery to middle cerebral artery (STA-MCA) bypass surgery with the additional implementation of new technical processes to improve the safety and efficacy of the procedure.

METHODS

Data sets from preoperative imaging were loaded and fused in a single 3-dimensional matrix using the neuronavigation system. Anatomical structures of interest (the STA, a selected M4 branch of the MCA, the middle meningeal artery [MMA], and the primary motor cortex [PMC]) were segmented. After the registration of the patient and the operating microscope, the structures of interest were projected into the eyepiece of the microscope and superimposed onto the patient's head, creating the AR surgical field.

RESULTS

AR was shown to be useful in patients undergoing EC-IC bypass revascularization, mostly during the following 4 surgical steps: (1) microsurgical dissection of the donor vessel (STA); (2) tailoring the craniotomy above the recipient vessel (M4 branch of the MCA); (3) tailoring the craniotomy to spare the MMA; and (4) tailoring the craniotomy and the anastomosis to spare the PMC.

CONCLUSION

AR assistance in EC-IC bypass revascularization is a versatile technical adjunct for helping surgeons to ensure the safety and efficacy of the procedure.

The utility of virtual reality and augmented reality in spine surgery

As the number of advances in surgical techniques increases, it becomes increasingly important to assess and research the technology regarding spine surgery techniques in order to increase surgical accuracy, decrease overall length of surgery, and minimize overall radiation exposure. Currently, augmented reality and virtual reality have shown promising results in regard to their applicability beyond their current functions. At present, VR has been generally applied to a teaching and preparatory role, while AR has been utilized in surgical settings. As such, the following review attempts to provide an overview of both virtual reality and augmented reality, followed by a discussion of their current applications and future direction.

Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation

Major hurdles for Microsoft's HoloLens as a tool in medicine have been accessing tracking data, as well as a relatively high-localisation error of the displayed information; cumulatively resulting in its limited use and minimal quantification. The following work investigates the augmentation of HoloLens with the proprietary image processing SDK Vuforia, allowing integration of data from its front-facing RGB camera to provide more spatially stable holograms for neuronavigational use. Continuous camera tracking was able to maintain hologram registration with a mean perceived drift of 1.41 mm, as well as a mean sub 2-mm surface point localisation accuracy of 53%, all while allowing the researcher to walk about a test area. This represents a 68% improvement for the later and a 34% improvement for the former compared with a typical HoloLens deployment used as a control. Both represent a significant improvement on hologram stability given the current state-of-the-art, and to the best of the authors knowledge are the first example of quantified measurements when augmenting hologram stability using data from the RGB sensor.