Hybrid navigation interface for orthopedic and trauma surgery

Is Overlain Display a Right Choice for AR Navigation? A Qualitative Study of Head-Mounted Augmented Reality Surgical Navigation on Accuracy for Large-Scale Clinical Deployment

BACKGROUND

During the course of the past two decades, head-mounted augmented reality surgical navigation (HMARSN) systems have been increasingly employed in a variety of surgical specialties as a result of both advancements in augmented reality-related technologies and surgeons' desires to overcome some drawbacks inherent to conventional surgical navigation systems. In the present time, most experimental HMARSN systems adopt overlain display (OD) that overlay virtual models and planned routes of surgical tools on corresponding physical tissues, organs, lesions, and so forth, in a surgical field so as to provide surgeons with an intuitive and direct view to gain better hand-eye coordination as well as avoid attention shift and loss of sight (LOS), among other benefits during procedures. Yet, its system accuracy, which is the most crucial performance indicator of any surgical navigation system, is difficult to ascertain because it is highly subjective and user-dependent. Therefore, the aim of this study was to review presently available experimental OD HMARSN systems qualitatively, explore how their system accuracy is affected by overlain display, and find out if such systems are suited to large-scale clinical deployment.

METHOD

We searched PubMed and ScienceDirect with the following terms: head mounted augmented reality surgical navigation, and 445 records were returned in total. After screening and eligibility assessment, 60 papers were finally analyzed. Specifically, we focused on how their accuracies were defined and measured, as well as whether such accuracies are stable in clinical practice and competitive with corresponding commercially available systems.

RESULTS AND CONCLUSIONS

The primary findings are that the system accuracy of OD HMARSN systems is seriously affected by a transformation between the spaces of the user's eyes and the surgical field, because measurement of the transformation is heavily individualized and user-dependent. Additionally, the transformation itself is potentially subject to changes during surgical procedures, and hence unstable. Therefore, OD HMARSN systems are not suitable for large-scale clinical deployment.

Review of Enhanced Handheld Surgical Drills

The handheld drill has been used as a conventional surgical tool for centuries. Alongside the recent successes of surgical robots, the development of new and enhanced medical drills has improved surgeon ability without requiring the high cost and consuming setup times that plague medical robot systems. This work provides an overview of enhanced handheld surgical drill research focusing on systems that include some form of image guidance and do not require additional hardware that physically supports or guides drilling. Drilling is reviewed by main contribution divided into audio-, visual-, or hardware-enhanced drills. A vision for future work to enhance handheld drilling systems is also discussed.

Augmented reality in the operating room: a clinical feasibility study

Background

Augmented Reality (AR) is a rapidly emerging technology finding growing acceptance and application in different fields of surgery. Various studies have been performed evaluating the precision and accuracy of AR guided navigation. This study investigates the feasibility of a commercially available AR head mounted device during orthopedic surgery.

Methods

Thirteen orthopedic surgeons from a Swiss university clinic performed 25 orthopedic surgical procedures wearing a holographic AR headset (HoloLens, Microsoft, Redmond, WA, USA) providing complementary three-dimensional, patient specific anatomic information. The surgeon’s experience of using the device during surgery was recorded using a standardized 58-item questionnaire grading different aspects on a 100-point scale with anchor statements.

Results

Surgeons were generally satisfied with image quality (85 ± 17 points) and accuracy of the virtual objects (84 ± 19 point). Wearing the AR device was rated as fairly comfortable (79 ± 13 points). Functionality of voice commands (68 ± 20 points) and gestures (66 ± 20 points) provided less favorable results. The greatest potential in the use of the AR device was found for surgical correction of deformities (87 ± 15 points). Overall, surgeons were satisfied with the application of this novel technology (78 ± 20 points) and future access to it was demanded (75 ± 22 points).

Conclusion

AR is a rapidly evolving technology with large potential in different surgical settings, offering the opportunity to provide a compact, low cost alternative requiring a minimum of infrastructure compared to conventional navigation systems. While surgeons where generally satisfied with image quality of the here tested head mounted AR device, some technical and ergonomic shortcomings were pointed out. This study serves as a proof of concept for the use of an AR head mounted device in a real-world sterile setting in orthopedic surgery.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12891-021-04339-w.

Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review

BACKGROUND & AIM

Utilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice.

METHODS & RESULTS

PRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits.

CONCLUSION

Whilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons' specific needs should increase uptake.

Hybrid trauma service: on the leading edge of damage Control

Trauma damage control seeks to limit life-threatening bleeding. Sequential diagnostic and therapeutic approaches are the current standard. Hybrid Room have reduced hemostasis time by integrating different specialties and technologies. Hybrid Rooms seek to control bleeding in an operating room equipped with specialized personnel and advanced technology including angiography, tomography, eFAST, radiography, endoscopy, infusers, cell retrievers, REBOA, etc. Trauma Hybrid Service is a concept that describes a vertical work scheme that begins with the activation of Trauma Code when admitting a severely injured patient, initiating a continuous resuscitation process led by the trauma surgeon who guides transfer to imaging, angiography and surgery rooms according to the patient's condition and the need for specific interventions. Hybrid rooms integrate different diagnostic and therapeutic tools in one same room, reducing the attention time and increasing all interventions effectiveness.

Augmented Reality Navigated Sacral-Alar-Iliac Screw Insertion

BACKGROUND

Sacral-alar-iliac (SAI) screws are increasingly used for lumbo-pelvic fixation procedures. Insertion of SAI screws is technically challenging, and surgeons often rely on costly and time-consuming navigation systems. We investigated the accuracy and precision of an augmented reality (AR)-based and commercially available head-mounted device requiring minimal infrastructure.

METHODS

A pelvic sawbone model served to drill pilot holes of 80 SAI screw trajectories by 2 surgeons, randomly either freehand (FH) without any kind of navigation or with AR navigation. The number of primary pilot hole perforations, simulated screw perforation, minimal axis/outer cortical wall distance, true sagittal cranio-caudal inclination angle (tSCCIA), true axial medio-lateral angle, and maximal screw length (MSL) were measured and compared to predefined optimal values.

RESULTS

In total, 1/40 (2.5%) of AR-navigated screw hole trajectories showed a perforation before passing the inferior gluteal line compared to 24/40 (60%) of FH screw hole trajectories (P < .05). The differences between FH- and AR-guided holes compared to optimal values were significant for tSCCIA with -10.8° ± 11.77° and MSL -65.29 ± 15 mm vs 55.04 ± 6.76 mm (P = .001).

CONCLUSIONS

In this study, the additional anatomical information provided by the AR headset and the superimposed operative plan improved the precision of drilling pilot holes for SAI screws in a laboratory setting compared to the conventional FH technique. Further technical development and validation studies are currently being performed to investigate potential clinical benefits of the AR-based navigation approach described here.

LEVEL OF EVIDENCE

4.

Augmented reality-based navigation increases precision of pedicle screw insertion

Background

Precise insertion of pedicle screws is important to avoid injury to closely adjacent neurovascular structures. The standard method for the insertion of pedicle screws is based on anatomical landmarks (free-hand technique). Head-mounted augmented reality (AR) devices can be used to guide instrumentation and implant placement in spinal surgery. This study evaluates the feasibility and precision of AR technology to improve precision of pedicle screw insertion compared to the current standard technique.

Methods

Two board-certified orthopedic surgeons specialized in spine surgery and two novice surgeons were each instructed to drill pilot holes for 40 pedicle screws in eighty lumbar vertebra sawbones models in an agar-based gel. One hundred and sixty pedicles were randomized into two groups: the standard free-hand technique (FH) and augmented reality technique (AR). A 3D model of the vertebral body was superimposed over the AR headset. Half of the pedicles were drilled using the FH method, and the other half using the AR method.

Results

The average minimal distance of the drill axis to the pedicle wall (MAPW) was similar in both groups for expert surgeons (FH 4.8 ± 1.0 mm vs. AR 5.0 ± 1.4 mm, p = 0.389) but for novice surgeons (FH 3.4 mm ± 1.8 mm, AR 4.2 ± 1.8 mm, p = 0.044).

Expert surgeons showed 0 primary drill pedicle perforations (PDPP) in both the FH and AR groups. Novices showed 3 (7.5%) PDPP in the FH group and one perforation (2.5%) in the AR group, respectively (p > 0.005).

Experts showed no statistically significant difference in average secondary screw pedicle perforations (SSPP) between the AR and the FH set 6-, 7-, and 8-mm screws (p > 0.05). Novices showed significant differences of SSPP between most groups: 6-mm screws, 18 (45%) vs. 7 (17.5%), p = 0.006; 7-mm screws, 20 (50%) vs. 10 (25%), p = 0.013; and 8-mm screws, 22 (55%) vs. 15 (37.5%), p = 0.053, in the FH and AR group, respectively. In novices, the average optimal medio-lateral convergent angle (oMLCA) was 3.23° (STD 4.90) and 0.62° (STD 4.56) for the FH and AR set screws (p = 0.017), respectively. Novices drilled with a higher precision with respect to the cranio-caudal inclination angle (CCIA) category (p = 0.04) with AR.

Conclusion

In this study, the additional anatomical information provided by the AR headset superimposed to real-world anatomy improved the precision of drilling pilot holes for pedicle screws in a laboratory setting and decreases the effect of surgeon’s experience. Further technical development and validations studies are currently being performed to investigate potential clinical benefits of the herein described AR-based navigation approach.

Instrument-mounted displays for reducing cognitive load during surgical navigation

PURPOSE

Surgical navigation systems rely on a monitor placed in the operating room to relay information. Optimal monitor placement can be challenging in crowded rooms, and it is often not possible to place the monitor directly beside the situs. The operator must split attention between the navigation system and the situs. We present an approach for needle-based interventions to provide navigational feedback directly on the instrument and close to the situs by mounting a small display onto the needle.

METHODS

By mounting a small and lightweight smartwatch display directly onto the instrument, we are able to provide navigational guidance close to the situs and directly in the operator's field of view, thereby reducing the need to switch the focus of view between the situs and the navigation system. We devise a specific variant of the established crosshair metaphor suitable for the very limited screen space. We conduct an empirical user study comparing our approach to using a monitor and a combination of both.

RESULTS

Results from the empirical user study show significant benefits for cognitive load, user preference, and general usability for the instrument-mounted display, while achieving the same level of performance in terms of time and accuracy compared to using a monitor.

CONCLUSION

We successfully demonstrate the feasibility of our approach and potential benefits. With ongoing technological advancements, instrument-mounted displays might complement standard monitor setups for surgical navigation in order to lower cognitive demands and for improved usability of such systems.

Towards intra-operative 3D nuclear imaging: reconstruction of 3D radioactive distributions using tracked gamma probes

Nuclear medicine imaging modalities assist commonly in surgical guidance given their functional nature. However, when used in the operating room they present limitations. Pre-operative tomographic 3D imaging can only serve as a vague guidance intra-operatively, due to movement, deformation and changes in anatomy since the time of imaging, while standard intra-operative nuclear measurements are limited to 1D or (in some cases) 2D images with no depth information. To resolve this problem we propose the synchronized acquisition of position, orientation and readings of gamma probes intra-operatively to reconstruct a 3D activity volume. In contrast to conventional emission tomography, here, in a first proof-of-concept, the reconstruction succeeds without requiring symmetry in the positions and angles of acquisition, which allows greater flexibility. We present our results in phantom experiments for sentinel node lymph node localization. The results indicate that 3D intra-operative nuclear images can be generated in such a setup up to an accuracy equivalent to conventional SPECT systems. This technology has the potential to advance standard procedures towards intra-operative 3D nuclear imaging and offers a novel approach for robust and precise localization of functional information to facilitate less invasive, image-guided surgery.

Virtually extended surgical drilling device: virtual mirror for navigated spine surgery

This paper introduces a new method for navigated spine surgery using a stereoscopic video see-through head-mounted display (HMD) and an optical tracking system. Vertebrae are segmented from volumetric CT data and visualized in-situ. A surgical drilling device is virtually extended with a mirror for intuitive planning of the drill canal, control of drill direction and insertion depth. The first designated application for the virtually extended drilling device is the preparation of canals for pedicle screw implantation in spine surgery. The objective of surgery is to install an internal fixateur for stabilization of injured vertebrae. We invited five surgeons of our partner clinic to test the system with realistic replica of lumbar vertebrae and compared the new approach with the classical, monitor-based navigation system providing three orthogonal slice views on the operation site. We measured time of procedure and scanned the drilled vertebrae with CT to verify accuracy of drilling.