Surgical Navigation Technology Based on Augmented Reality and Integrated 3D Intraoperative Imaging: A Spine Cadaveric Feasibility and Accuracy Study

Use of an inertial measurement unit sensor in pedicle screw placement improves trajectory accuracy

Ascertaining the accuracy of the pedicle screw (PS) trajectories is important as PS malpositioning can cause critical complications. We aimed to determine the angle range over which estimation is unreliable; build a low-cost PS placement support system that uses an inertial measurement unit (IMU) to enable the monitoring of surgical tools and PS trajectories, and determine the situations where IMU support would be most beneficial. In PS insertion experiments, we used cadaver samples that included lumbar porcine spines. Computed tomography images obtained before and after PS insertion were viewed. Offsets between the planned and implanted PS trajectories in the freehand and IMU-assisted groups were analyzed. The PS cortical bone breaches were classified according to the Gertzbein and Robbins criteria (GRC). Added head-down tilted sample experiments were repeated wherein we expected a decreased rostro-caudal rotational accuracy of the PS according to the angle estimation ability results. Evaluation of the PS trajectory accuracy revealed no significant advantage of IMU-assisted rostro-caudal rotational accuracy versus freehand accuracy. According to the GRC, IMU assistance significantly increased the rate of clinically acceptable PS positions (RoCA) than the freehand technique. In the head-down tilted sample experiments, IMU assist provided increased accuracies with both rostro-caudal and medial rotational techniques when compared with the freehand technique. In the freehand group, RoCA was significantly decreased in samples with rostral tilting relative to that in the samples without. However, In the IMU-assisted group, no significant difference in RoCA between the samples with and without head-down tilting was observed. Even when the planned PS medial and/or rostro-caudal rotational angle was relatively large and difficult to reproduce manually, IMU-support helped maintain the PS trajectory accuracy and positioning safety. IMU assist in PS placement was more beneficial, especially for larger rostro-caudal and/or medial rotational pedicle angles.

Frameless Patient Tracking With Adhesive Optical Skin Markers for Augmented Reality Surgical Navigation in Spine Surgery

STUDY DESIGN

Observational study.

OBJECTIVE

The aim of this study was to evaluate the accuracy of a new frameless reference marker system for patient tracking by analyzing the effect of vertebral position within the surgical field.

SUMMARY OF BACKGROUND DATA

Most modern navigation systems for spine surgery rely on a dynamic reference frame attached to a vertebra for tracking the patient. This solution has the drawback of being bulky and obstructing the surgical field, while requiring that the dynamic reference frame is moved between vertebras to maintain accuracy.

METHODS

An augmented reality surgical navigation (ARSN) system with intraoperative cone beam computed tomography (CBCT) capability was installed in a hybrid operating room. The ARSN system used input from four video cameras for tracking adhesive skin markers placed around the surgical field. The frameless reference marker system was evaluated first in four human cadavers, and then in 20 patients undergoing navigated spine surgery. In each CBCT, the impact of vertebral position in the surgical field on technical accuracy was analyzed. The technical accuracy of the inserted pedicle devices was determined by measuring the distance between the planned position and the placed pedicle device, at the bone entry point.

RESULTS

The overall mean technical accuracy was 1.65 ± 1.24 mm at the bone entry point (n = 366). There was no statistically significant difference in technical accuracy between levels within CBCTs (P ≥ 0.12 for all comparisons). Linear regressions showed that null- to negligible parts of the effect on technical accuracy could be explained by the number of absolute levels away from the index vertebrae (r ≤ 0.007 for all, β ≤ 0.071 for all).

CONCLUSION

The frameless reference marker system based on adhesive skin markers is unobtrusive and affords the ARSN system a high accuracy throughout the navigated surgical field, independent of vertebral position.

LEVEL OF EVIDENCE

3.

Intraoperative cone beam computed tomography is as reliable as conventional computed tomography for identification of pedicle screw breach in thoracolumbar spine surgery

Objectives

To test the hypothesis that intraoperative cone beam computed tomography (CBCT) using the Allura augmented reality surgical navigation (ARSN) system in a dedicated hybrid operating room (OR) matches computed tomography (CT) for identification of pedicle screw breach during spine surgery.

Methods

Twenty patients treated with spinal fixation surgery (260 screws) underwent intraoperative CBCT as well as conventional postoperative CT scans (median 12 months after surgery) to identify and grade the degree of pedicle screw breach on both scan types, according to the Gertzbein grading scale. Blinded assessments were performed by three independent spine surgeons and the CT served as the standard of reference. Screws graded as Gertzbein 0 or 1 were considered clinically accurate while grades 2 or 3 were considered inaccurate. Sensitivity, specificity, and negative predictive value were the primary metrics of diagnostic performance.

Results

For this patient group, the negative predictive value of an intraoperative CBCT to rule out pedicle screw breach was 99.6% (CI 97.75–99.99%). Among 10 screws graded as inaccurate on CT, 9 were graded as such on the CBCT, giving a sensitivity of 90.0% (CI 55.5–99.75%). Among the 250 screws graded as accurate on CT, 244 were graded as such on the CBCT, giving a specificity of 97.6% (CI 94.85–99.11%).

Conclusions

CBCT, performed intraoperatively with the Allura ARSN system, is comparable and non-inferior to a conventional postoperative CT scan for ruling out misplaced pedicle screws in spinal deformity cases, eliminating the need for a postoperative CT.

Key Points

• Intraoperative cone beam computed tomography (CT) using the Allura ARSN is comparable with conventional CT for ruling out pedicle screw breaches after spinal fixation surgery.

• Intraoperative cone beam computed tomography can be used to assess need for revisions of pedicle screws making routine postoperative CT scans unnecessary.

• Using cone beam computed tomography, the specificity was 97.6% and the sensitivity was 90% for detecting pedicle screw breaches and the negative predictive value for ruling out a pedicle screw breach was 99.6%.

[Extended reality technologies for support of surgical workflows]

Current developments in the field of extended reality (XR) could prove useful in the optimization of surgical workflows, time effectiveness and postoperative outcome. Although still primarily a subject of research, the state of XR technologies is rapidly improving and approaching feasibility for a broad clinical application. Surgical fields of application of XR technologies are currently primarily training, preoperative planning and intraoperative assistance. For all three areas, products already exist (some clinically approved) and technical feasibility studies have been conducted. In teaching, the use of XR can already be assessed as fundamentally practical and meaningful but still needs to be evaluated in large multicenter studies. In preoperative planning XR can also offer advantages, although technical limitations often impede routine use; however, for cases of intraoperative use informative evaluation studies are mostly lacking, so that an assessment is not yet possible in a meaningful way. Furthermore, there is a lack of assessments regarding cost-effectiveness in all three areas. The XR technologies enable proven advantages in surgical workflows despite the lack of high-quality evaluation with respect to the practical and clinical use of XR. New concepts for effective interaction with XR media also need to be developed. In the future, further research progress and technical developments in the field can be expected.

Augmented reality in neurosurgical navigation: a survey

BACKGROUND

Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems.

METHODS

In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey.

RESULTS

Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks.

CONCLUSIONS

Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.

The Role of Machine Learning in Spine Surgery: The Future Is Now

The recent influx of machine learning centered investigations in the spine surgery literature has led to increased enthusiasm as to the prospect of using artificial intelligence to create clinical decision support tools, optimize postoperative outcomes, and improve technologies used in the operating room. However, the methodology underlying machine learning in spine research is often overlooked as the subject matter is quite novel and may be foreign to practicing spine surgeons. Improper application of machine learning is a significant bioethics challenge, given the potential consequences of over- or underestimating the results of such studies for clinical decision-making processes. Proper peer review of these publications requires a baseline familiarity of the language associated with machine learning, and how it differs from classical statistical analyses. This narrative review first introduces the overall field of machine learning and its role in artificial intelligence, and defines basic terminology. In addition, common modalities for applying machine learning, including classification and regression decision trees, support vector machines, and artificial neural networks are examined in the context of examples gathered from the spine literature. Lastly, the ethical challenges associated with adapting machine learning for research related to patient care, as well as future perspectives on the potential use of machine learning in spine surgery, are discussed specifically.

A Clinical Application Study of Mixed Reality Technology Assisted Lumbar Pedicle Screws Implantation

Background

This was a prospective comparative study of mixed reality (MR) technology assisted lumbar pedicle screws placement and traditional lumbar pedicle screws placement.

Material/Methods

Fifty cases of lumbar pedicle screws placement were randomly divided into 2 groups: 25 cases with MR technology in group A, and 25 cases without MR technology in group B. All patients had their scores on the Oswestry disability index (ODI) of low back pain and the visual analog scale (VAS) of the affected lower limb recorded at pre-operation. Blood loss, operative duration, success rate of first penetration by tap, and number of times C-arm fluoroscopy was performed were recorded at intraoperation. The postoperative drainage was recorded. The ODI of low back pain and VAS of the affected lower limb were recorded at 1, 3, and 6 months after operation.

Results

Group A had less bleeding, shorter operation time, higher success rate of first penetration by tap, and fewer times using C-arm fluoroscopy at intraoperation (P<0.05). There was significant difference in ODI scores and VAS scores at 1 mouth after operation (P<0.05). The postoperative drainage of group A was less than group B (P<0.05). The implantation accuracy of group A was higher than group B (P<0.05). The postoperative recovery rate of low back pain of group A was faster than group B (P<0.05).

Conclusions

The safety of spinal surgery and implantation accuracy of pedicle screw fixation system could be increased by MR technology.

Towards Optical Imaging for Spine Tracking without Markers in Navigated Spine Surgery

Surgical navigation systems are increasingly used for complex spine procedures to avoid neurovascular injuries and minimize the risk for reoperations. Accurate patient tracking is one of the prerequisites for optimal motion compensation and navigation. Most current optical tracking systems use dynamic reference frames (DRFs) attached to the spine, for patient movement tracking. However, the spine itself is subject to intrinsic movements which can impact the accuracy of the navigation system. In this study, we aimed to detect the actual patient spine features in different image views captured by optical cameras, in an augmented reality surgical navigation (ARSN) system. Using optical images from open spinal surgery cases, acquired by two gray-scale cameras, spinal landmarks were identified and matched in different camera views. A computer vision framework was created for preprocessing of the spine images, detecting and matching local invariant image regions. We compared four feature detection algorithms, Speeded Up Robust Feature (SURF), Maximal Stable Extremal Region (MSER), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB) to elucidate the best approach. The framework was validated in 23 patients and the 3D triangulation error of the matched features was < 0 . 5 mm. Thus, the findings indicate that spine feature detection can be used for accurate tracking in navigated surgery.

The state-of-the-art in ultrasound-guided spine interventions

During the last two decades, intra-operative ultrasound (iUS) imaging has been employed for various surgical procedures of the spine, including spinal fusion and needle injections. Accurate and efficient registration of pre-operative computed tomography or magnetic resonance images with iUS images are key elements in the success of iUS-based spine navigation. While widely investigated in research, iUS-based spine navigation has not yet been established in the clinic. This is due to several factors including the lack of a standard methodology for the assessment of accuracy, robustness, reliability, and usability of the registration method. To address these issues, we present a systematic review of the state-of-the-art techniques for iUS-guided registration in spinal image-guided surgery (IGS). The review follows a new taxonomy based on the four steps involved in the surgical workflow that include pre-processing, registration initialization, estimation of the required patient to image transformation, and a visualization process. We provide a detailed analysis of the measurements in terms of accuracy, robustness, reliability, and usability that need to be met during the evaluation of a spinal IGS framework. Although this review is focused on spinal navigation, we expect similar evaluation criteria to be relevant for other IGS applications.

Hyperspectral Imaging for Skin Feature Detection: Advances in Markerless Tracking for Spine Surgery

In spinal surgery, surgical navigation is an essential tool for safe intervention, including the placement of pedicle screws without injury to nerves and blood vessels. Commercially available systems typically rely on the tracking of a dynamic reference frame attached to the spine of the patient. However, the reference frame can be dislodged or obscured during the surgical procedure, resulting in loss of navigation. Hyperspectral imaging (HSI) captures a large number of spectral information bands across the electromagnetic spectrum, providing image information unseen by the human eye. We aim to exploit HSI to detect skin features in a novel methodology to track patient position in navigated spinal surgery. In our approach, we adopt two local feature detection methods, namely a conventional handcrafted local feature and a deep learning-based feature detection method, which are compared to estimate the feature displacement between different frames due to motion. To demonstrate the ability of the system in tracking skin features, we acquire hyperspectral images of the skin of 17 healthy volunteers. Deep-learned skin features are detected and localized with an average error of only 0.25 mm, outperforming the handcrafted local features with respect to the ground truth based on the use of optical markers.

Imaging Technologies in Spine Surgery

This article presents a comprehensive review of the evolution of both invasive and noninvasive imaging technologies that are part of the arsenal of spinal diagnostics and surgical therapy. The text provides not only a historical lens to the evolution of the imaging technologies that are part of routine contemporary practice but also provides a detailed sketch of emerging imaging technologies, such as endoscopic and exoscopic systems. Augmented reality, virtual reality, and mixed reality are new technologies that have enhanced the preparation of surgery and provide excellent case-specific training modules to break down each step of an operation in isolation.

Design and control of an image-guided robot for spine surgery in a hybrid OR

BACKGROUND

Minimally invasive spine (MIS) fusion surgery requires image guidance and expert manual dexterity for a successful, efficient, and accurate pedicle screw placement. Operating room (OR)-integrated robotic solution can provide precise assistance to potentially minimize complication rates and facilitate difficult MIS procedures.

METHODS

A 5-degrees of freedom robot was designed specifically for a hybrid OR with integrated surgical navigation for guiding pedicle screw pilot holes. The system automatically aligns an instrument following the surgical plan using only instrument tracking feedback. Contrary to commercially available robotic systems, no tracking markers on the robotic arm are required. The system was evaluated in a cadaver study.

RESULTS

The mean targeting error (N = 34) was 1.27±0.57 mm and 1.62±0.85°, with 100% of insertions graded as clinically acceptable.

CONCLUSIONS

A fully integrated robotic guidance system, including intra-op imaging, planning, and physical guidance with optimized robot design and control, can improve workflow and provide pedicle screw guidance with less than 2 mm targeting error.

Feasibility and accuracy of a robotic guidance system for navigated spine surgery in a hybrid operating room: a cadaver study

The combination of navigation and robotics in spine surgery has the potential to accurately identify and maintain bone entry position and planned trajectory. The goal of this study was to examine the feasibility, accuracy and efficacy of a new robot-guided system for semi-automated, minimally invasive, pedicle screw placement. A custom robotic arm was integrated into a hybrid operating room (OR) equipped with an augmented reality surgical navigation system (ARSN). The robot was mounted on the OR-table and used to assist in placing Jamshidi needles in 113 pedicles in four cadavers. The ARSN system was used for planning screw paths and directing the robot. The robot arm autonomously aligned with the planned screw trajectory, and the surgeon inserted the Jamshidi needle into the pedicle. Accuracy measurements were performed on verification cone beam computed tomographies with the planned paths superimposed. To provide a clinical grading according to the Gertzbein scale, pedicle screw diameters were simulated on the placed Jamshidi needles. A technical accuracy at bone entry point of 0.48 ± 0.44 mm and 0.68 ± 0.58 mm was achieved in the axial and sagittal views, respectively. The corresponding angular errors were 0.94 ± 0.83° and 0.87 ± 0.82°. The accuracy was statistically superior (p < 0.001) to ARSN without robotic assistance. Simulated pedicle screw grading resulted in a clinical accuracy of 100%. This study demonstrates that the use of a semi-automated surgical robot for pedicle screw placement provides an accuracy well above what is clinically acceptable.

Accuracy of augmented reality surgical navigation for minimally invasive pedicle screw insertion in the thoracic and lumbar spine with a new tracking device

BACKGROUND CONTEXT

Minimally invasive approaches are increasingly used in spine surgery. The purpose of navigation systems is to guide the surgeon and to reduce intraoperative x-ray exposure.

PURPOSE

This study aimed to determine the feasibility and clinical accuracy of a navigation technology based on augmented reality surgical navigation (ARSN) for minimally invasive thoracic and lumbar pedicle screw instrumentation compared with standard fluoroscopy-guided minimally invasive technique.

STUDY DESIGN/SETTING

Cadaveric laboratory study.

METHODS

ARSN was installed in a hybrid operating room, consisting of a flat panel detector c-arm with two dimensional/three dimensional imaging capabilities and four integrated cameras in its frame. The surface-referenced navigation device does not require a bony reference but uses video cameras and optical markers applied to the patient's skin for tracking. In four cadavers, a total of 136 pedicle screws were inserted in thoracic and lumbar vertebrae. The accuracy was assessed by three independent raters in postoperative conventional computed tomography.

RESULTS

The overall accuracy of ARSN was 94% compared with an accuracy of 88% for fluoroscopy. The difference was not statistically significant. In the thoracic region, accuracy with ARSN was 92% compared with 83% with fluoroscopy. With fluoroscopy, unsafe screws were observed in three normal cadavers and one with scoliosis. Using ARSN, unsafe screws were only observed in the scoliotic spine. No significant difference in the median of time for K-wire placement was recorded. As no intraoperative fluoroscopy was necessary in ARSN, the performing surgeon was not exposed to radiation.

CONCLUSIONS

In this limited cadaveric study minimally invasive screw placement using ARSN was demonstrated to be feasible and as accurate as fluoroscopy. It did not require any additional navigation time or use of any intraoperative x-ray imaging, thereby potentially permitting surgery in a protective lead garment-free environment. A well-powered clinical study is needed to demonstrate a significant difference in the accuracy between the two methods.

CLINICAL SIGNIFICANCE

ARSN offers real-time imaging of planned insertion paths, instrument tracking, and overlay of three dimensional bony anatomy and surface topography. The referencing procedure, by optical recognition of several skin markers is easy and does not require a solid bony reference as necessary for conventional navigation which saves time. Additionally, ARSN may foster the reduction of intraoperative x-ray exposure to spinal surgeons.

Robotic Spine Surgery and Augmented Reality Systems: A State of the Art

Instrumented spine procedures have been performed for decades to treat a wide variety of spinal disorders. New technologies have been employed to obtain a high degree of precision, to minimize risks of damage to neurovascular structures and to diminish harmful exposure of patients and the operative team to ionizing radiations. Robotic spine surgery comprehends 3 major categories: telesurgical robotic systems, robotic-assisted navigation (RAN) and virtual augmented reality (AR) systems, including AR and virtual reality. Telesurgical systems encompass devices that can be operated from a remote command station, allowing to perform surgery via instruments being manipulated by the robot. On the other hand, RAN technologies are characterized by the robotic guidance of surgeon-operated instruments based on real-time imaging. Virtual AR systems are able to show images directly on special visors and screens allowing the surgeon to visualize information about the patient and the procedure (i.e., anatomical landmarks, screw direction and inclination, distance from neurological and vascular structures etc.). The aim of this review is to focus on the current state of the art of robotics and AR in spine surgery and perspectives of these emerging technologies that hold promises for future applications.

Simultaneous Optimization of Patient-Image Registration and Hand-Eye Calibration for Accurate Augmented Reality in Surgery

OBJECTIVE

Augmented reality (AR) navigation using a position sensor in endoscopic surgeries relies on the quality of patient-image registration and hand-eye calibration. Conventional methods collect the necessary data to compute two output transformation matrices separately. However, the AR display setting during surgery generally differs from that during preoperative processes. Although conventional methods can identify optimal solutions under initial conditions, AR display errors are unavoidable during surgery owing to the inherent computational complexity of AR processes, such as error accumulation over successive matrix multiplications, and tracking errors of position sensor.

METHODS

We propose the simultaneous optimization of patient-image registration and hand-eye calibration in an AR environment before surgery. The relationship between the endoscope and a virtual object to overlay is first calculated using an endoscopic image, which also functions as a reference during optimization. After including the tracking information from the position sensor, patient-image registration and hand-eye calibration are optimized in terms of least-squares.

RESULTS

Experiments with synthetic data verify that the proposed method is less sensitive to computation and tracking errors. A phantom experiment with a position sensor is also conducted. The accuracy of the proposed method is significantly higher than that of the conventional method.

CONCLUSION

The AR accuracy of the proposed method is compared with those of the conventional ones, and the superiority of the proposed method is verified.

SIGNIFICANCE

This study demonstrates that the proposed method exhibits substantial potential for improving AR navigation accuracy.

Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: a matched-control study comparing accuracy

This study aimed to compare screw placement accuracy and clinical aspects between Augmented Reality Surgical Navigation (ARSN) and free-hand (FH) technique. Twenty patients underwent spine surgery with screw placement using ARSN and were matched retrospectively to a cohort of 20 FH technique cases for comparison. All ARSN and FH cases were performed by the same surgeon. Matching was based on clinical diagnosis and similar proportions of screws placed in the thoracic and lumbosacral vertebrae in both groups. Accuracy of screw placement was assessed on postoperative scans according to the Gertzbein scale and grades 0 and 1 were considered accurate. Procedure time, blood loss and length of hospital stay, were collected as secondary endpoints. A total of 262 and 288 screws were assessed in the ARSN and FH groups, respectively. The share of clinically accurate screws was significantly higher in the ARSN vs FH group (93.9% vs 89.6%, p < 0.05). The proportion of screws placed without a cortical breach was twice as high in the ARSN group compared to the FH group (63.4% vs 30.6%, p < 0.0001). No statistical difference was observed for the secondary endpoints between both groups. This matched-control study demonstrated that ARSN provided higher screw placement accuracy compared to free-hand.

Towards non-invasive patient tracking: optical image analysis for spine tracking during spinal surgery procedures

Surgical navigation systems can enhance surgeon vision and form a reliable image-guided tool for complex interventions as spinal surgery. The main prerequisite is successful patient tracking which implies optimal motion compensation. Nowadays, optical tracking systems can satisfy the need of detecting patient position during surgery, allowing navigation without the risk of damaging neurovascular structures. However, the spine is subject to vertebrae movements which can impact the accuracy of the system. The aim of this paper is to investigate the feasibility of a novel approach for offering a direct relationship to movements of the spinal vertebra during surgery. To this end, we detect and track patient spine features between different image views, captured by several optical cameras, for vertebrae rotation and displacement reconstruction. We analyze patient images acquired in a real surgical scenario by two gray-scale cameras, embedded in the flat-panel detector of the C-arm. Spine segmentation is performed and anatomical landmarks are designed and tracked between different views, while experimenting with several feature detection algorithms (e.g. SURF, MSER, etc.). The 3D positions for the matched features are reconstructed and the triangulation errors are computed for an accuracy assessment. The analysis of the triangulation accuracy reveals a mean error of 0.38 mm, which demonstrates the feasibility of spine tracking and strengthens the clinical application of optical imaging for spinal navigation.

Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking

Objective

Surgical navigation is a well-established tool in endoscopic skull base surgery. However, navigational and endoscopic views are usually displayed on separate monitors, forcing the surgeon to focus on one or the other. Aiming to provide real-time integration of endoscopic and diagnostic imaging information, we present a new navigation technique based on augmented reality with fusion of intraoperative cone beam computed tomography (CBCT) on the endoscopic view. The aim of this study was to evaluate the accuracy of the method.

Material and methods

An augmented reality surgical navigation system (ARSN) with 3D CBCT capability was used. The navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. Intra-operative CBCT images were fused with the view of the surgical field obtained by the endoscope’s camera. Accuracy of CBCT image co-registration was tested using a custom-made grid with incorporated 3D spheres.

Results

Co-registration of the CBCT image on the endoscopic view was performed. Accuracy of the overlay, measured as mean target registration error (TRE), was 0.55 mm with a standard deviation of 0.24 mm and with a median value of 0.51mm and interquartile range of 0.39˗˗0.68 mm.

Conclusion

We present a novel augmented reality surgical navigation system, with fusion of intraoperative CBCT on the endoscopic view. The system shows sub-millimeter accuracy.

Augmented Reality Surgical Navigation in Spine Surgery to Minimize Staff Radiation Exposure

STUDY DESIGN

Prospective observational study.

OBJECTIVE

To assess staff and patient radiation exposure during augmented reality surgical navigation in spine surgery.

SUMMARY OF BACKGROUND DATA

Surgical navigation in combination with intraoperative three-dimensional imaging has been shown to significantly increase the clinical accuracy of pedicle screw placement. Although this technique may increase the total radiation exposure compared with fluoroscopy, the occupational exposure can be minimized, as navigation is radiation free and staff can be positioned behind protective shielding during three-dimensional imaging. The patient radiation exposure during treatment and verification of pedicle screw positions can also be reduced.

METHODS

Twenty patients undergoing spine surgery with pedicle screw placement were included in the study. The staff radiation exposure was measured using real-time active personnel dosimeters and was further compared with measurements using a reference dosimeter attached to the C-arm (i.e., a worst-case staff exposure situation). The patient radiation exposures were recorded, and effective doses (ED) were determined.

RESULTS

The average staff exposure per procedure was 0.21 ± 0.06 μSv. The average staff-to-reference dose ratio per procedure was 0.05% and decreased to less than 0.01% after a few procedures had been performed. The average patient ED was 15.8 ± 1.8 mSv which mainly correlated with the number of vertebrae treated and the number of cone-beam computed tomography acquisitions performed. A low-dose protocol used for the final 10 procedures yielded a 32% ED reduction per spinal level treated.

CONCLUSION

This study demonstrated significantly lower occupational doses compared with values reported in the literature. Real-time active personnel dosimeters contributed to a fast optimization and adoption of protective measures throughout the study. Even though our data include both cone-beam computed tomography for navigation planning and intraoperative screw placement verification, we find low patient radiation exposure levels compared with published data.

LEVEL OF EVIDENCE

3.

Augmented reality in orthopaedics

Aims

Computer-based applications are increasingly being used by orthopaedic surgeons in their clinical practice. With the integration of technology in surgery, augmented reality (AR) may become an important tool for surgeons in the future. By superimposing a digital image on a user’s view of the physical world, this technology shows great promise in orthopaedics. The aim of this review is to investigate the current and potential uses of AR in orthopaedics.

Materials and Methods

A systematic review of the PubMed, MEDLINE, and Embase databases up to January 2019 using the keywords ‘orthopaedic’ OR ‘orthopedic AND augmented reality’ was performed by two independent reviewers.

Results

A total of 41 publications were included after screening. Applications were divided by subspecialty: spine (n = 15), trauma (n = 16), arthroplasty (n = 3), oncology (n = 3), and sports (n = 4). Out of these, 12 were clinical in nature. AR-based technologies have a wide variety of applications, including direct visualization of radiological images by overlaying them on the patient and intraoperative guidance using preoperative plans projected onto real anatomy, enabling hands-free real-time access to operating room resources, and promoting telemedicine and education.

Conclusion

There is an increasing interest in AR among orthopaedic surgeons. Although studies show similar or better outcomes with AR compared with traditional techniques, many challenges need to be addressed before this technology is ready for widespread use. Cite this article: Bone Joint J 2019;101-B:1479–1488

A Novel Augmented-Reality-Based Surgical Navigation System for Spine Surgery in a Hybrid Operating Room: Design, Workflow, and Clinical Applications

BACKGROUND

Treatment of several spine disorders requires placement of pedicle screws. Detailed 3-dimensional (3D) anatomic information facilitates this process and improves accuracy.

OBJECTIVE

To present a workflow for a novel augmented-reality-based surgical navigation (ARSN) system installed in a hybrid operating room for anatomy visualization and instrument guidance during pedicle screw placement.

METHODS

The workflow includes surgical exposure, imaging, automatic creation of a 3D model, and pedicle screw path planning for instrument guidance during surgery as well as the actual screw placement, spinal fixation, and wound closure and intraoperative verification of the treatment results. Special focus was given to process integration and minimization of overhead time. Efforts were made to manage staff radiation exposure avoiding the need for lead aprons. Time was kept throughout the procedure and subdivided to reflect key steps. The navigation workflow was validated in a trial with 20 cases requiring pedicle screw placement (13/20 scoliosis).

RESULTS

Navigated interventions were performed with a median total time of 379 min per procedure (range 232-548 min for 4-24 implanted pedicle screws).

The total procedure time was subdivided into surgical exposure (28%), cone beam computed tomography imaging and 3D segmentation (2%), software planning (6%), navigated surgery for screw placement (17%) and non-navigated instrumentation, wound closure, etc (47%).

CONCLUSION

Intraoperative imaging and preparation for surgical navigation totaled 8% of the surgical time. Consequently, ARSN can routinely be used to perform highly accurate surgery potentially decreasing the risk for complications and revision surgery while minimizing radiation exposure to the staff.

[Value of virtual reality technology in preoperative planning of transtrochanteric curved varus osteotomy for avascular necrosis of femoral head in adults]

OBJECTIVE

To research the value of virtual reality (VR) technology in the preoperative planning of transtrochanteric curved varus osteotomy for avascular necrosis of the femoral head (ANFH) in adults.

METHODS

Between June 2018 and November 2018, 7 patients (11 hips) with ANFH, who were treated with transtrochanteric curved varus osteotomy, were enrolled in the study. There were 4 males (7 hips) and 3 females (4 hips) with an average age of 31.9 years (range, 14-46 years). Among them, 3 patients were unilateral ANFH and 4 patients were bilateral ANFH. There was 1 patient (1 hip) of traumatic ANFH, 2 patients (4 hips) of alcohol-induced ANFH, 2 patients (3 hips) of hormonal ANFH, and 2 patients (3 hips) of idiopathic ANFH. All hips were Association Research Circulation Osseous (ARCO) stage Ⅲ. There were 5 hips for Japanese Investigation Committee (JIC) type C1 and 6 hips for type C2. There were 5 hips for China-Japan Friendship Hospital (CJFH) type L1,1 for type L2, and 5 for type L3. The disease duration ranged from 5 to 12 months (mean, 8 months). Preoperative Harris score was 53.91±7.66. The neck-shaft angle ranged from 128 to 143° (mean, 133.9°). VR technology was adopted for the preoperative planning. CT data were imported into the software to construct the morphology of necrotic area, and the transrtrochanteric varus osteotomy was simulated. The varus angle was designed according to the integrity rate of femoral head. The planned varus angle was 6 to 16° (mean, 9.7°). The transtrochanteric curved varus osteotomy was performed according to the preoperative planning, and the varus angle and loading area were confirmed under fluoroscopy. If the planned varus angle was too small, it would continue to increase under the fluoroscopy until a satisfactory varus angle. Postoperative changes of the neck-shaft angle were calculated and compared with the preoperative planned varus angle (error). The hip function was assessed by using the Harris score.

RESULTS

All incisions healed by first intention. All patients were followed up 6-11 months with an average of 8 months. The X-ray film at 2 days after operation showed that the neck-shaft angle was 112-135° (mean, 123.4°). The difference of the neck-shaft angle between pre- and post-operation was 6-16° (mean, 11.0°). Among them, the difference of the neck-shaft angle was consistent with planned varus angle in 5 hips, while the error of the remaining 6 hips was 1-4°. There was 1 patient (1 hip) of osteotomy nonunion at 4 months after operation, 1 patient (1 hip) of proximal femur fracture at 2 months after operation. The rest 5 patients (9 hips) obtained union at the osteotomy. At last follow-up, the Harris score was 82.18±16.35, showing significant difference when compared with preoperative score ( t=-5.195, P=0.000).

CONCLUSION

VR technology is a brand-new preoperative planning method for transtrochanteric curved varus osteotomy in treating ANFH.

Augmented and Virtual Reality Instrument Tracking for Minimally Invasive Spine Surgery: A Feasibility and Accuracy Study

STUDY DESIGN

Cadaveric animal laboratory study.

OBJECTIVE

To evaluate the feasibility and accuracy of pedicle cannulation using an augmented reality surgical navigation (ARSN) system with automatic instrument tracking, yielding feedback of instrument position in relation to deep anatomy.

SUMMARY OF BACKGROUND DATA

Minimally invasive spine surgery (MISS) has the possibility of reducing surgical exposure resulting in shorter hospital stays, lower blood loss and infection rates compared with open surgery but the drawback of limiting visual feedback to the surgeon regarding deep anatomy. MISS is mainly performed using image-guided 2D fluoroscopy, thus exposing the staff to ionizing radiation.

METHODS

A hybrid operating room (OR) equipped with a robotic C-arm with integrated optical cameras for augmented reality instrument navigation was used. In two pig cadavers, cone beam computed tomography (CBCT) scans were performed, a 3D model generated, and pedicle screw insertions were planned. Seventy-eight insertions were performed. Technical accuracy was assessed on post-insertion CBCTs by measuring the distance between the navigated device and the corresponding pre-planned path as well as the angular deviations. Drilling and hammering into the pedicle were also compared. Navigation time was measured. An independent reviewer assessed a simulated clinical accuracy according to Gertzbein.

RESULTS

The technical accuracy was 1.7 ± 1.0 mm at the bone entry point and 2.0 ± 1.3 mm at the device tip. The angular deviation was 1.7 ± 1.7° in the axial and 1.6 ± 1.2° in the sagittal plane. Navigation time per insertion was 195 ± 93 seconds. There was no difference in accuracy between hammering and drilling into the pedicle. The clinical accuracy was 97.4% to 100% depending on the screw size considered for placement. No ionizing radiation was used during navigation.

CONCLUSION

ARSN with instrument tracking for MISS is feasible, accurate, and radiation-free during navigation.

LEVEL OF EVIDENCE

3.

Pedicle screw accuracy in clinical utilization of minimally invasive navigated robot-assisted spine surgery

In the emerging field of robot-assisted spine surgery, the radiographic evaluation of pedicle screw accuracy in clinical application is an area of high interest. This study describes the pedicle screw accuracy of the first 56 consecutive cases in which navigated robotic assistance was used in a private practice clinical setting. A retrospective, Institutional Review Board-exempt review of the first 56 navigated robot-assisted spine surgery cases was performed. Pedicle screw malposition, reposition, and return to operating room (OR) rates were collected. A CT-based Gertzbein and Robbins system (GRS) was used to classify pedicle screw accuracy. In the first 56 robotic cases, 356 total pedicle screws were placed. Eight screws were placed without the robot due to surgeon discretion. Of the 348 pedicle screws inserted by navigated robotic guidance, only 2.6% (9/348) were repositioned, resulting in a 97.4% (339/348) successful screw placement rate. The average age was 64, and 48% were female. Average body mass index was 31 kg/m². Based on the GRS CT-based grading, 97.7% (340/348) were graded A or B, 1.7% (6/348) screws were graded C, and only 0.6% (2/348) of screws were graded D. Two complications, explantation of interbody and vacuum-assisted wound closure, were reported as requiring a return to the OR, but these were not related to robotic guidance or pedicle screws. This study demonstrated a high level of accuracy (97.7%) in the first 56 cases using navigated, robot-assisted surgery based on the GRS. There were two non-screw-related complications requiring return to the operating room.

Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study

OBJECTIVE

Augmented reality (AR) is a novel technology that has the potential to increase the technical feasibility, accuracy, and safety of conventional manual and robotic computer-navigated pedicle insertion methods. Visual data are directly projected to the operator's retina and overlaid onto the surgical field, thereby removing the requirement to shift attention to a remote display. The objective of this study was to assess the comparative accuracy of AR-assisted pedicle screw insertion in comparison to conventional pedicle screw insertion methods.

METHODS

Five cadaveric male torsos were instrumented bilaterally from T6 to L5 for a total of 120 inserted pedicle screws. Postprocedural CT scans were obtained, and screw insertion accuracy was graded by 2 independent neuroradiologists using both the Gertzbein scale (GS) and a combination of that scale and the Heary classification, referred to in this paper as the Heary-Gertzbein scale (HGS). Non-inferiority analysis was performed, comparing the accuracy to freehand, manual computer-navigated, and robotics-assisted computer-navigated insertion accuracy rates reported in the literature. User experience analysis was conducted via a user experience questionnaire filled out by operators after the procedures.

RESULTS

The overall screw placement accuracy achieved with the AR system was 96.7% based on the HGS and 94.6% based on the GS. Insertion accuracy was non-inferior to accuracy reported for manual computer-navigated pedicle insertion based on both the GS and the HGS scores. When compared to accuracy reported for robotics-assisted computer-navigated insertion, accuracy achieved with the AR system was found to be non-inferior when assessed with the GS, but superior when assessed with the HGS. Last, accuracy results achieved with the AR system were found to be superior to results obtained with freehand insertion based on both the HGS and the GS scores. Accuracy results were not found to be inferior in any comparison. User experience analysis yielded "excellent" usability classification.

CONCLUSIONS

AR-assisted pedicle screw insertion is a technically feasible and accurate insertion method.

Machine learning for automated 3-dimensional segmentation of the spine and suggested placement of pedicle screws based on intraoperative cone-beam computer tomography

OBJECTIVE

The goal of this study was to develop and validate a system for automatic segmentation of the spine, pedicle identification, and screw path suggestion for use with an intraoperative 3D surgical navigation system.

METHODS

Cone-beam CT (CBCT) images of the spines of 21 cadavers were obtained. An automated model-based approach was used for segmentation. Using machine learning methodology, the algorithm was trained and validated on the image data sets. For measuring accuracy, surface area errors of the automatic segmentation were compared to the manually outlined reference surface on CBCT. To further test both technical and clinical accuracy, the algorithm was applied to a set of 20 clinical cases. The authors evaluated the system's accuracy in pedicle identification by measuring the distance between the user-defined midpoint of each pedicle and the automatically segmented midpoint. Finally, 2 independent surgeons performed a qualitative evaluation of the segmentation to judge whether it was adequate to guide surgical navigation and whether it would have resulted in a clinically acceptable pedicle screw placement.

RESULTS

The clinically relevant pedicle identification and automatic pedicle screw planning accuracy was 86.1%. By excluding patients with severe spinal deformities (i.e., Cobb angle > 75° and severe spinal degeneration) and previous surgeries, a success rate of 95.4% was achieved. The mean time (± SD) for automatic segmentation and screw planning in 5 vertebrae was 11 ± 4 seconds.

CONCLUSIONS

The technology investigated has the potential to aid surgeons in navigational planning and improve surgical navigation workflow while maintaining patient safety.

Augmented Reality in Orthopedics: Current State and Future Directions

Augmented reality (AR) comprises special hardware and software, which is used in order to offer computer-processed imaging data to the surgeon in real time, so that real-life objects are combined with computer-generated images. AR technology has recently gained increasing interest in the surgical practice. Preclinical research has provided substantial evidence that AR might be a useful tool for intra-operative guidance and decision-making. AR has been applied to a wide spectrum of orthopedic procedures, such as tumor resection, fracture fixation, arthroscopy, and component's alignment in total joint arthroplasty. The present study aimed to summarize the current state of the application of AR in orthopedics, in preclinical and clinical level, providing future directions and perspectives concerning potential further benefits from this technology.

Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of medical imaging. Radiology departments have begun exploring the use of these technologies to help with radiology education and clinical care. The purpose of this review article is to summarize how three institutions have explored using virtual and augmented reality for radiology.

Pedicle Screw Placement Using Augmented Reality Surgical Navigation With Intraoperative 3D Imaging: A First In-Human Prospective Cohort Study

STUDY DESIGN

Prospective observational study.

OBJECTIVE

The aim of this study was to evaluate the accuracy of pedicle screw placement using augmented reality surgical navigation (ARSN) in a clinical trial.

SUMMARY OF BACKGROUND DATA

Recent cadaveric studies have shown improved accuracy for pedicle screw placement in the thoracic spine using ARSN with intraoperative 3D imaging, without the need for periprocedural x-ray. In this clinical study, we used the same system to place pedicle screws in the thoracic and lumbosacral spine of 20 patients.

METHODS

The study was performed in a hybrid operating room with an integrated ARSN system encompassing a surgical table, a motorized flat detector C-arm with intraoperative 2D/3D capabilities, integrated optical cameras for augmented reality navigation, and noninvasive patient motion tracking. Three independent reviewers assessed screw placement accuracy using the Gertzbein grading on 3D scans obtained before wound closure. In addition, the navigation time per screw placement was measured.

RESULTS

One orthopedic spinal surgeon placed 253 lumbosacral and thoracic pedicle screws on 20 consenting patients scheduled for spinal fixation surgery. An overall accuracy of 94.1% of primarily thoracic pedicle screws was achieved. No screws were deemed severely misplaced (Gertzbein grade 3). Fifteen (5.9%) screws had 2 to 4 mm breach (Gertzbein grade 2), occurring in scoliosis patients only. Thirteen of those 15 screws were larger than the pedicle in which they were placed. Two medial breaches were observed and 13 were lateral. Thirteen of the grade 2 breaches were in the thoracic spine. The average screw placement time was 5.2 ± 4.1 minutes. During the study, no device-related adverse event occurred.

CONCLUSION

ARSN can be clinically used to place thoracic and lumbosacral pedicle screws with high accuracy and with acceptable navigation time. Consequently, the risk for revision surgery and complications could be minimized.

LEVEL OF EVIDENCE

3.

Radiation dose and image quality comparison during spine surgery with two different, intraoperative 3D imaging navigation systems

Careful protocol selection is required during intraoperative three-dimensional (3D) imaging for spine surgery to manage patient radiation dose and achieve clinical image quality. Radiation dose and image quality of a Medtronic O-arm commonly used during spine surgery, and a Philips hybrid operating room equipped with XperCT C-arm 3D cone-beam CT (hCBCT) are compared. The mobile O-arm (mCBCT) offers three different radiation dose settings (low, standard, and high), for four different patient sizes (small, medium, large, and extra large). The patient's radiation dose rate is constant during the entire 3D scan. In contrast, C-CBCT spine imaging uses three different field of views (27, 37, and 48 cm) using automatic exposure control (AEC) that modulates the patient's radiation dose rate during the 3D scan based on changing patient thickness. hCBCT uses additional x-ray beam filtration. Small, medium, and large trunk phantoms designed to mimic spine and soft tissue were imaged to assess radiation dose and image quality of the two systems. The estimated measured "patient" dose for the small, medium, and large phantoms imaged by the mCBCT considering all the dose settings ranged from 9.4-27.6 mGy, 8.9-33.3 mGy, and 13.8-40.6 mGy, respectively. The "patient" dose values for the same phantoms imaged with hCBCT were 2.8-4.6 mGy, 5.7-10.0 mGy, and 11.0-15.2 mGy. The CNR for the small, medium, and large phantoms was 2.9 to 3.7, 2.0 to 3.0, and 2.5 to 2.6 times higher with the hCBCT system, respectively. Hounsfield unit accuracy, noise, and uniformity of hCBCT exceeded the performance of the mCBCT; spatial resolution was comparable. Added x-ray beam filtration and AEC capability achieved clinical image quality for intraoperative spine surgery at reduced radiation dose to the patient in comparison to a reference O-arm system without these capabilities.

Diffuse reflectance spectroscopy, a potential optical sensing technology for the detection of cortical breaches during spinal screw placement

Safe and accurate placement of screws remains a critical issue in open and minimally invasive spine surgery. We propose to use diffuse reflectance (DR) spectroscopy as a sensing technology at the tip of a surgical instrument to ensure a safe path of the instrument through the cancellous bone of the vertebrae. This approach could potentially reduce the rate of cortical bone breaches, thereby resulting in fewer neural and vascular injuries during spinal fusion surgery. In our study, DR spectra in the wavelength ranges of 400 to 1600 nm were acquired from cancellous and cortical bone from three human cadavers. First, it was investigated whether these spectra can be used to distinguish between the two bone types based on fat, water, and blood content along with photon scattering. Subsequently, the penetration of the bone by an optical probe was simulated using the Monte-Carlo (MC) method, to study if the changes in fat content along the probe path would still enable distinction between the bone types. Finally, the simulation findings were validated via an experimental insertion of an optical screw probe into the vertebra aided by x-ray image guidance. The DR spectra indicate that the amount of fat, blood, and photon scattering is significantly higher in cancellous bone than in cortical bone (p  <  0.01), which allows distinction between the bone types. The MC simulations showed a change in fat content more than 1 mm before the optical probe came in contact with the cortical bone. The experimental insertion of the optical screw probe gave similar results. This study shows that spectral tissue sensing, based on DR spectroscopy at the instrument tip, is a promising technology to identify the transition zone from cancellous to cortical vertebral bone. The technology therefore has the potential to improve the safety and accuracy of spinal screw placement procedures.

Feasibility and Accuracy of Thoracolumbar Minimally Invasive Pedicle Screw Placement With Augmented Reality Navigation Technology

STUDY DESIGN

Cadaveric laboratory study.

OBJECTIVE

To assess the feasibility and accuracy of minimally invasive thoracolumbar pedicle screw placement using augmented reality (AR) surgical navigation.

SUMMARY OF BACKGROUND DATA

Minimally invasive spine (MIS) surgery has increasingly become the method of choice for a wide variety of spine pathologies. Navigation technology based on AR has been shown to be feasible, accurate, and safe in open procedures. AR technology may also be used for MIS surgery.

METHODS

The AR surgical navigation was installed in a hybrid operating room (OR). The hybrid OR includes a surgical table, a motorized flat detector C-arm with intraoperative 2D/3D imaging capabilities, integrated optical cameras for AR navigation, and patient motion tracking using optical markers on the skin. Navigation and screw placement was without any x-ray guidance. Two neurosurgeons placed 66 Jamshidi needles (two cadavers) and 18 cannulated pedicle screws (one cadaver) in the thoracolumbar spine. Technical accuracy was evaluated by measuring the distance between the tip of the actual needle position and the corresponding planned path as well as the angles between the needle and the desired path. Time needed for navigation along the virtual planned path was measured. An independent reviewer assessed the postoperative scans for the pedicle screws' clinical accuracy.

RESULTS

Navigation time per insertion was 90 ± 53 seconds with an accuracy of 2.2 ± 1.3 mm. Accuracy was not dependent on operator. There was no correlation between navigation time and accuracy. The mean error angle between the Jamshidi needles and planned paths was 0.9° ± 0.8°. No screw was misplaced outside the pedicle. Two screws breached 2 to 4 mm yielding an overall accuracy of 89% (16/18).

CONCLUSION

MIS screw placement directed by AR with intraoperative 3D imaging in a hybrid OR is accurate and efficient, without any fluoroscopy or x-ray imaging during the procedure.

LEVEL OF EVIDENCE

N/A.

A review of existing and potential computer user interfaces for modern radiology

Abstract

The digitalization of modern imaging has led radiologists to become very familiar with computers and their user interfaces (UI). New options for display and command offer expanded possibilities, but the mouse and keyboard remain the most commonly utilized, for usability reasons. In this work, we review and discuss different UI and their possible application in radiology. We consider two-dimensional and three-dimensional imaging displays in the context of interventional radiology, and discuss interest in touchscreens, kinetic sensors, eye detection, and augmented or virtual reality. We show that UI design specifically for radiologists is key for future use and adoption of such new interfaces. Next-generation UI must fulfil professional needs, while considering contextual constraints.

Teaching Points

• The mouse and keyboard remain the most utilized user interfaces for radiologists.

• Touchscreen, holographic, kinetic sensors and eye tracking offer new possibilities for interaction.

• 3D and 2D imaging require specific user interfaces.

• Holographic display and augmented reality provide a third dimension to volume imaging.

• Good usability is essential for adoption of new user interfaces by radiologists.

Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology

Context

Augmented reality (AR) devices such as the Microsoft HoloLens have not been well used in the medical field.

Objective

To test the HoloLens for clinical and nonclinical applications in pathology.

Design

A Microsoft HoloLens was tested for virtual annotation during autopsy, viewing 3D gross and microscopic pathology specimens, navigating whole slide images, telepathology, as well as real-time pathology-radiology correlation.

Results

Pathology residents performing an autopsy wearing the HoloLens were remotely instructed with real-time diagrams, annotations, and voice instruction. 3D-scanned gross pathology specimens could be viewed as holograms and easily manipulated. Telepathology was supported during gross examination and at the time of intraoperative consultation, allowing users to remotely access a pathologist for guidance and to virtually annotate areas of interest on specimens in real-time. The HoloLens permitted radiographs to be coregistered on gross specimens and thereby enhanced locating important pathologic findings. The HoloLens also allowed easy viewing and navigation of whole slide images, using an AR workstation, including multiple coregistered tissue sections facilitating volumetric pathology evaluation.

Conclusions

The HoloLens is a novel AR tool with multiple clinical and nonclinical applications in pathology. The device was comfortable to wear, easy to use, provided sufficient computing power, and supported high-resolution imaging. It was useful for autopsy, gross and microscopic examination, and ideally suited for digital pathology. Unique applications include remote supervision and annotation, 3D image viewing and manipulation, telepathology in a mixed-reality environment, and real-time pathology-radiology correlation.

Augmented-reality integrated robotics in neurosurgery: are we there yet?

Surgical robots have captured the interest—if not the widespread acceptance—of spinal neurosurgeons. But successful innovation, scientific or commercial, requires the majority to adopt a new practice. “Faster, better, cheaper” products should in theory conquer the market, but often fail. The psychology of change is complex, and the “follow the leader” mentality, common in the field today, lends little trust to the process of disseminating new technology. Beyond product quality, timing has proven to be a key factor in the inception, design, and execution of new technologies. Although the first robotic surgery was performed in 1985, scant progress was seen until the era of minimally invasive surgery. This movement increased neurosurgeons’ dependence on navigation and fluoroscopy, intensifying the drive for enhanced precision. Outside the field of medicine, various technology companies have made great progress in popularizing co-robots (“cobots”), augmented reality, and processor chips. This has helped to ease practicing surgeons into familiarity with and acceptance of these technologies. The adoption among neurosurgeons in training is a “follow the leader” phenomenon, wherein new surgeons tend to adopt the technology used during residency. In neurosurgery today, robots are limited to computers functioning between the surgeon and patient. Their functions are confined to establishing a trajectory for navigation, with task execution solely in the surgeon’s hands. In this review, the authors discuss significant untapped technologies waiting to be used for more meaningful applications. They explore the history and current manifestations of various modern technologies, and project what innovations may lie ahead.

From urban planning and emergency training to Pokémon Go: applications of virtual reality GIS (VRGIS) and augmented reality GIS (ARGIS) in personal, public and environmental health

The latest generation of virtual and mixed reality hardware has rekindled interest in virtual reality GIS (VRGIS) and augmented reality GIS (ARGIS) applications in health, and opened up new and exciting opportunities and possibilities for using these technologies in the personal and public health arenas. From smart urban planning and emergency training to Pokémon Go, this article offers a snapshot of some of the most remarkable VRGIS and ARGIS solutions for tackling public and environmental health problems, and bringing about safer and healthier living options to individuals and communities. The article also covers the main technical foundations and issues underpinning these solutions.