Applicability of augmented reality in orthopedic surgery - A systematic review

Metaverse, virtual reality and augmented reality in total shoulder arthroplasty: a systematic review

PURPOSE

This systematic review aims to provide an overview of the current knowledge on the role of the metaverse, augmented reality, and virtual reality in reverse shoulder arthroplasty.

METHODS

A systematic review was performed using the PRISMA guidelines. A comprehensive review of the applications of the metaverse, augmented reality, and virtual reality in in-vivo intraoperative navigation, in the training of orthopedic residents, and in the latest innovations proposed in ex-vivo studies was conducted.

RESULTS

A total of 22 articles were included in the review. Data on navigated shoulder arthroplasty was extracted from 14 articles: seven hundred ninety-three patients treated with intraoperative navigated rTSA or aTSA were included. Also, three randomized control trials (RCTs) reported outcomes on a total of fifty-three orthopedics surgical residents and doctors receiving VR-based training for rTSA, which were also included in the review. Three studies reporting the latest VR and AR-based rTSA applications and two proof of concept studies were also included in the review.

CONCLUSIONS

The metaverse, augmented reality, and virtual reality present immense potential for the future of orthopedic surgery. As these technologies advance, it is crucial to conduct additional research, foster development, and seamlessly integrate them into surgical education to fully harness their capabilities and transform the field. This evolution promises enhanced accuracy, expanded training opportunities, and improved surgical planning capabilities.

Orthopaedic Sport Medicine Surgeons and Fellows Value Immersive Virtual Reality for Improving Surgical Training, Procedural Planning, and Distance Learning

BACKGROUND

Overall, the potential utility of iVR technology in orthopaedic surgery is promising. The attitudes of medical students and surgical trainees on VR simulated surgical training have been overwhelmingly positive. However, further research and understanding of the attitudes of practicing orthopaedic surgeons and Fellows are needed to appreciate its benefits for clinical practice.

PURPOSE

The purpose of this study was to assess the attitudes of Canadian orthopaedic surgeons and Fellows on the value of iVR for surgical training, clinical practice, and distance learning.

METHODS

Forty-three orthopedic surgeons and Fellows attended a VRthrscopy™ Knee Module (Conmed Corporation, Largo, USA) demonstration. The view and audio from the lead headset were cast to a large screen so the audience could follow the procedure in real-time. Immediately after the presentation, the audience members were given a paper questionnaire assessing their perceptions and attitudes toward iVR or use in orthopaedic learning, clinical practice and distance education and mentoring.

RESULTS

iVR was perceived to be valuable for the field of orthopaedic surgery. All 13 questions were rated with mean Likert scores of five or greater indicating observed value for all 13 questions. The respondents indicated that iVR had value (score of 5 or greater) in each questionnaire domain, with agreement ranging from 78-98% for teaching and learning, 66-97% for clinical practice, and 88-100% for distance education and mentoring questions.

CONCLUSION

This study has demonstrated that a group of Canadian sport medicine orthopedic surgeons and Fellows had favourable attitudes towards, and perceived that iVR has value in, orthopaedic surgical training, clinical practice, and distance learning and mentorship. The potential for utilizing iVR technology for distance learning, mentorship and global education appears promising.

Comparison of virtual reality and computed tomography in the preoperative planning of complex tibial plateau fractures

INTRODUCTION

Preoperative planning is a critical step in the success of any complex surgery. The pur-pose of this study is to evaluate the advantage of VR glasses in surgical planning of complex tibial plateau fractures compared to CT planning.

MATERIALS AND METHODS

Five orthopedic surgeons performed preoperative planning for 30 fractures using either conventional CT slices or VR visualization with a VR headset. Planning was performed in a randomized order with a 3-month interval between planning sessions. A standardized questionnaire assessed planned operative time, planning time, fracture classification and understanding, and surgeons' subjective confidence in surgical planning.

RESULTS

The mean planned operative time of 156 (SD 47) minutes was significantly lower (p < 0.001) in the VR group than in the CT group (172 min; SD 44). The mean planning time in the VR group was 3.48 min (SD 2.4), 17% longer than in the CT group (2.98 min, SD 1.9; p = 0.027). Relevant parameters influencing planning time were surgeon experience (-0.61 min) and estimated complexity of fracture treatment (+ 0.65 min).

CONCLUSION

The use of virtual reality for surgical planning of complex tibial plateau fractures resulted in significantly shorter planned operative time, while planning time was longer compared to CT planning. After VR planning, more surgeons felt (very) well prepared for surgery.

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

PURPOSE OF REVIEW

Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.

RECENT FINDINGS

To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict. AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.

An Image-Based Augmented Reality System for Achieving Accurate Bone Resection in Total Knee Arthroplasty

Background and objective With the steady advancement of computer-assisted surgical techniques, the importance of assessing and researching technology related to total knee arthroplasty (TKA) procedures has increased. Augmented reality (AR), a recently proposed next-generation technology, is expected to enhance the precision of orthopedic surgery by providing a more efficient and cost-effective approach. However, the accuracy of image-based AR in TKA surgery has not been established. Therefore, this study aimed to determine whether accurate bone resection can be achieved in TKA surgery using image-based AR. Methods In this study, we replaced traditional CT imaging and reconstructions for creating a bone 3D model by direct 3D scanning of the femur and tibia. The preoperative planning involved identifying anatomical landmarks and determining the surgical details. During surgery, markers were employed to create a local coordinate system for an AR-assisted surgical system using a Polaris camera. This approach helped minimize discrepancies between the 3D model and actual positioning, ensuring accurate alignment. Results The AR-assisted surgery using the image method resulted in fewer errors [average error: 0.32 mm; standard deviation (SD): 0.143] between the bone resection depth of the preoperative surgical plan and the bone model test results. Conclusions Our findings demonstrated the accuracy of bone resectioning by using image-based AR-assisted navigation for TKA surgery. Image-based AR-assisted navigation in TKA surgery is a valuable tool not only for enhancing accuracy by using smart glasses and sensors but also for improving the efficiency of the procedure. Therefore, we anticipate that image-based AR-assisted navigation in TKA surgery will gain wide acceptance in practice.

Conventional vs Augmented Reality-Guided Lateral Calcaneal Lengthening Simulated in a Foot Bone Model

BACKGROUND

Acquired adult flatfoot deformity (AAFD) results in a loss of the medial longitudinal arch of the foot and dysfunction of the posteromedial soft tissues. Hintermann osteotomy (H-O) is often used to treat stage II AAFD. The procedure is challenging because of variations in the subtalar facets and limited intraoperative visibility. We aimed to assess the impact of augmented reality (AR) guidance on surgical accuracy and the facet violation rate.

METHODS

Sixty AR-guided and 60 conventional osteotomies were performed on foot bone models. For AR osteotomies, the ideal osteotomy plane was uploaded to a Microsoft HoloLens 1 headset and carried out in strict accordance with the superimposed holographic plane. The conventional osteotomies were performed relying solely on the anatomy of the calcaneal lateral column. The rate and severity of facet joint violation was measured, as well as accuracy of entry and exit points. The results were compared across AR-guided and conventional osteotomies, and between experienced and inexperienced surgeons.

RESULTS

Experienced surgeons showed significantly greater accuracy for the osteotomy entry point using AR, with the mean deviation of 1.6 ± 0.9 mm (95% CI 1.26, 1.93) compared to 2.3 ± 1.3 mm (95% CI 1.87, 2.79) in the conventional method (P = .035). The inexperienced had improved accuracy, although not statistically significant (P = .064), with the mean deviation of 2.0 ± 1.5 mm (95% CI 1.47, 2.55) using AR compared with 2.7 ± 1.6 mm (95% CI 2.18, 3.32) in the conventional method. AR helped the experienced surgeons avoid full violation of the posterior facet (P = .011). Inexperienced surgeons had a higher rate of middle and posterior facet injury with both methods (P = .005 and .021).

CONCLUSION

Application of AR guidance during H-O was associated with improved accuracy for experienced surgeons, demonstrated by a better accuracy of the osteotomy entry point. More crucially, AR guidance prevented full violation of the posterior facet in the experienced group. Further research is needed to address limitations and test this technology on cadaver feet. Ultimately, the use of AR in surgery has the potential to improve patient and surgeon safety while minimizing radiation exposure.

CLINICAL RELEVANCE

Subtalar facet injury during lateral column lengthening osteotomy represents a real problem in clinical orthopaedic practice. Because of limited intraoperative visibility and variable anatomy, it is hard to resolve this issue with conventional means. This study suggests the potential of augmented reality to improve the osteotomy accuracy.

Single-center experience with Knee+™ augmented reality navigation system in primary total knee arthroplasty

BACKGROUND

Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years, as they enhance precision compared to conventional hardware. The expansion of computer assistance is evolving with the employment of augmented reality. Yet, the accuracy of augmented reality navigation systems has not been determined.

AIM

To examine the accuracy of component alignment and restoration of the affected limb's mechanical axis in primary total knee arthroplasty (TKA), utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.

METHODS

From May 2021 to December 2021, 30 patients, 25 women and five men, underwent a primary unilateral TKA. Revision cases were excluded. A preoperative radiographic procedure was performed to evaluate the limb's axial alignment. All patients were operated on by the same team, without a tourniquet, utilizing three distinct prostheses with the assistance of the Knee+™ augmented reality navigation system in every operation. Postoperatively, the same radiographic exam protocol was executed to evaluate the implants' position, orientation and coronal plane alignment. We recorded measurements in 3 stages regarding femoral varus and flexion, tibial varus and posterior slope. Firstly, the expected values from the Augmented Reality system were documented. Then we calculated the same values after each cut and finally, the same measurements were recorded radiologically after the operations. Concerning statistical analysis, Lin's concordance correlation coefficient was estimated, while Wilcoxon Signed Rank Test was performed when needed.

RESULTS

A statistically significant difference was observed regarding mean expected values and radiographic measurements for femoral flexion measurements only (Z score = 2.67, P value = 0.01). Nonetheless, this difference was statistically significantly lower than 1 degree (Z score = -4.21, P value < 0.01). In terms of discrepancies in the calculations of expected values and controlled measurements, a statistically significant difference between tibial varus values was detected (Z score = -2.33, P value = 0.02), which was also statistically significantly lower than 1 degree (Z score = -4.99, P value < 0.01).

CONCLUSION

The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized. Augmented reality navigation systems can bolster orthopaedic surgeons' accuracy in achieving precise axial alignment. However, further research is required to further evaluate their efficacy and potential.

Immersive virtual reality in orthopaedics-a narrative review

PURPOSE

This narrative review explores the applications and benefits of immersive virtual reality (VR) in orthopaedics, with a focus on surgical training, patient functional recovery, and pain management.

METHODS

The review examines existing literature and research studies on immersive VR in orthopaedics, analyzing both experimental and clinical studies.

RESULTS

Immersive VR provides a realistic simulation environment for orthopaedic surgery training, enhancing surgical skills, reducing errors, and improving overall performance. In post-surgical recovery and rehabilitation, immersive VR environments can facilitate motor learning and functional recovery through virtual embodiment, motor imagery during action observation, and virtual training. Additionally VR-based functional recovery programs can improve patient adherence and outcomes. Moreover, VR has the potential to revolutionize pain management, offering a non-invasive, drug-free alternative. Virtual reality analgesia acts by a variety of means including engagement and diverting patients' attention, anxiety reduction, and specific virtual-body transformations.

CONCLUSION

Immersive virtual reality holds significant promise in orthopaedics, demonstrating potential for improved surgical training, patient functional recovery, and pain management but further research is needed to fully exploit the benefits of VR technology in these areas.

Point-of-Care Orthopedic Oncology Device Development

BACKGROUND

The triad of 3D design, 3D printing, and xReality technologies is explored and exploited to collaboratively realize patient-specific products in a timely manner with an emphasis on designs with meta-(bio)materials.

METHODS

A case study on pelvic reconstruction after oncological resection (osteosarcoma) was selected and conducted to evaluate the applicability and performance of an inter-epistemic workflow and the feasibility and potential of 3D technologies for modeling, optimizing, and materializing individualized orthopedic devices at the point of care (PoC).

RESULTS

Image-based diagnosis and treatment at the PoC can be readily deployed to develop orthopedic devices for pre-operative planning, training, intra-operative navigation, and bone substitution.

CONCLUSIONS

Inter-epistemic symbiosis between orthopedic surgeons and (bio)mechanical engineers at the PoC, fostered by appropriate quality management systems and end-to-end workflows under suitable scientifically amalgamated synergies, could maximize the potential benefits. However, increased awareness is recommended to explore and exploit the full potential of 3D technologies at the PoC to deliver medical devices with greater customization, innovation in design, cost-effectiveness, and high quality.

Glenoid component placement in reverse shoulder arthroplasty assisted with augmented reality through a head-mounted display leads to low deviation between planned and postoperative parameters

BACKGROUND

Navigated augmented reality (AR) through a head-mounted display (HMD) may lead to accurate glenoid component placement in reverse shoulder arthroplasty (RSA). The purpose of this study is to evaluate the deviation between planned, intra- and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement assisted by a navigated AR through HMD during RSA.

METHODS

Both shoulders of 6 fresh frozen human cadavers, free from fractures or other bony pathologies, were used. Preoperative computed tomography (CT) scans were used for the 3-dimensional (3D) planning. The glenoid component placement was assisted using a navigated AR system through an HMD in all specimens. Intraoperative inclination, retroversion, depth, and rotation were measured by the system. A postoperative CT scan was performed. The pre- and postoperative 3D CT scan reconstructions were superimposed to calculate the deviation between planned and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement. Additionally, a comparison between intra- and postoperative values was calculated. Outliers were defined as >10° inclination, >10° retroversion, >3 mm entry point.

RESULTS

The registration algorithm of the scapulae prior to the procedure was correctly completed for all cases. The deviations between planned and postoperative values were 1.0° ± 0.7° for inclination, 1.8° ± 1.3° for retroversion, 1.1 ± 0.4 mm for entry point, 0.7 ± 0.6 mm for depth, and 1.7° ± 1.6° for rotation. The deviation between intra- and postoperative values were 0.9° ± 0.8° for inclination, 1.2° ± 1.1° for retroversion, 0.6 ± 0.5 mm for depth, and 0.3° ± 0.2° for rotation. There were no outliers between planned and postoperative parameters.

CONCLUSION

In this study, the use of a navigated AR system through an HMD for RSA led to low deviation between planned and postoperative values and between intra- and postoperative parameters.

Mixed Reality and Artificial Intelligence: A Holistic Approach to Multimodal Visualization and Extended Interaction in Knee Osteotomy

OBJECTIVE

Recent advancements in augmented reality led to planning and navigation systems for orthopedic surgery. However little is known about mixed reality (MR) in orthopedics. Furthermore, artificial intelligence (AI) has the potential to boost the capabilities of MR by enabling automation and personalization. The purpose of this work is to assess Holoknee prototype, based on AI and MR for multimodal data visualization and surgical planning in knee osteotomy, developed to run on the HoloLens 2 headset.

METHODS

Two preclinical test sessions were performed with 11 participants (eight surgeons, two residents, and one medical student) executing three times six tasks, corresponding to a number of holographic data interactions and preoperative planning steps. At the end of each session, participants answered a questionnaire on user perception and usability.

RESULTS

During the second trial, the participants were faster in all tasks than in the first one, while in the third one, the time of execution decreased only for two tasks ("Patient selection" and "Scrolling through radiograph") with respect to the second attempt, but without statistically significant difference (respectively [Formula: see text] = 0.14 and [Formula: see text] = 0.13, [Formula: see text]). All subjects strongly agreed that MR can be used effectively for surgical training, whereas 10 (90.9%) strongly agreed that it can be used effectively for preoperative planning. Six (54.5%) agreed and two of them (18.2%) strongly agreed that it can be used effectively for intraoperative guidance.

DISCUSSION/CONCLUSION

In this work, we presented Holoknee, the first holistic application of AI and MR for surgical planning for knee osteotomy. It reported promising results on its potential translation to surgical training, preoperative planning, and surgical guidance. Clinical and Translational Impact Statement - Holoknee can be helpful to support surgeons in the preoperative planning of knee osteotomy. It has the potential to impact positively the training of the future generation of residents and aid surgeons in the intraoperative stage.

Exposure to Extended Reality and Artificial Intelligence-Based Manifestations: A Primer on the Future of Hip and Knee Arthroplasty

BACKGROUND

Software-infused services, from robot-assisted and wearable technologies to artificial intelligence (AI)-laden analytics, continue to augment clinical orthopaedics - namely hip and knee arthroplasty. Extended reality (XR) tools, which encompass augmented reality, virtual reality, and mixed reality technology, represent a new frontier for expanding surgical horizons to maximize technical education, expertise, and execution. The purpose of this review is to critically detail and evaluate the recent developments surrounding XR in the field of hip and knee arthroplasty and to address potential future applications as they relate to AI.

METHODS

In this narrative review surrounding XR, we discuss (1) definitions, (2) techniques, (3) studies, (4) current applications, and (5) future directions. We highlight XR subsets (augmented reality, virtual reality, and mixed reality) as they relate to AI in the increasingly digitized ecosystem within hip and knee arthroplasty.

RESULTS

A narrative review of the XR orthopaedic ecosystem with respect to XR developments is summarized with specific emphasis on hip and knee arthroplasty. The XR as a tool for education, preoperative planning, and surgical execution is discussed with future applications dependent upon AI to potentially obviate the need for robotic assistance and preoperative advanced imaging without sacrificing accuracy.

CONCLUSION

In a field where exposure is critical to clinical success, XR represents a novel stand-alone software-infused service that optimizes technical education, execution, and expertise but necessitates integration with AI and previously validated software solutions to offer opportunities that improve surgical precision with or without the use of robotics and computed tomography-based imaging.

Factors affecting augmented reality head-mounted device performance in real OR

PURPOSE

Over the last years, interest and efforts to implement augmented reality (AR) in orthopedic surgery through head-mounted devices (HMD) have increased. However, the majority of experiments were preclinical and within a controlled laboratory environment. The operating room (OR) is a more challenging environment with various confounding factors potentially affecting the performance of an AR-HMD. The aim of this study was to assess the performance of an AR-HMD in a real-life OR setting.

METHODS

An established AR application using the HoloLens 2 HMD was tested in an OR and in a laboratory by two users. The accuracy of the hologram overlay, the time to complete the trial, the number of rejected registration attempts, the delay in live overlay of the hologram, and the number of completely failed runs were recorded. Further, different OR setting parameters (light condition, setting up partitions, movement of personnel, and anchor placement) were modified and compared.

RESULTS

Time for full registration was higher with 48 s (IQR 24 s) in the OR versus 33 s (IQR 10 s) in the laboratory setting (p < 0.001). The other investigated parameters didn't differ significantly if an optimal OR setting was used. Within the OR, the strongest influence on performance of the AR-HMD was different light conditions with direct light illumination on the situs being the least favorable.

CONCLUSION

AR-HMDs are affected by different OR setups. Standardization measures for better AR-HMD performance include avoiding direct light illumination on the situs, setting up partitions, and minimizing the movement of personnel.

The Accuracy and Absolute Reliability of a Knee Surgery Assistance System Based on ArUco-Type Sensors

Recent advances allow the use of Augmented Reality (AR) for many medical procedures. AR via optical navigators to aid various knee surgery techniques (e.g., femoral and tibial osteotomies, ligament reconstructions or menisci transplants) is becoming increasingly frequent. Accuracy in these procedures is essential, but evaluations of this technology still need to be made. Our study aimed to evaluate the system's accuracy using an in vitro protocol. We hypothesised that the system's accuracy was equal to or less than 1 mm and 1° for distance and angular measurements, respectively. Our research was an in vitro laboratory with a 316 L steel model. Absolute reliability was assessed according to the Hopkins criteria by seven independent evaluators. Each observer measured the thirty palpation points and the trademarks to acquire direct angular measurements on three occasions separated by at least two weeks. The system's accuracy in assessing distances had a mean error of 1.203 mm and an uncertainty of 2.062, and for the angular values, a mean error of 0.778° and an uncertainty of 1.438. The intraclass correlation coefficient was for all intra-observer and inter-observers, almost perfect or perfect. The mean error for the distance's determination was statistically larger than 1 mm (1.203 mm) but with a trivial effect size. The mean error assessing angular values was statistically less than 1°. Our results are similar to those published by other authors in accuracy analyses of AR systems.

Augmented reality and shoulder replacement: a state-of-the-art review article

Since its implementation, the rates of failure of total shoulder arthroplasty which may be due to malpositioning pushed to improve this surgery by creating new techniques and tools to help perioperatively. Augmented reality, a newly used tool in orthopedic surgery can help bypass this problem and reduce the rates of failure faced in shoulder replacement surgeries. Although this technology has revolutionized orthopedic surgery and helped improve the accuracy in shoulder prosthesis components positioning, it still has some limitations such as inaccurate over-imposition that should be addressed before it becomes of standard usage.

A Pilot Human Cadaveric Study on Accuracy of the Augmented Reality Surgical Navigation System for Thoracolumbar Pedicle Screw Insertion Using a New Intraoperative Rapid Registration Method

To evaluate the feasibility and accuracy of AR-assisted pedicle screw placement using a new intraoperative rapid registration method of combining preoperative CT scanning and intraoperative C-arm 2D fluoroscopy in cadavers. Five cadavers with intact thoracolumbar spines were employed in this study. Intraoperative registration was performed using anteroposterior and lateral views of preoperative CT scanning and intraoperative 2D fluoroscopic images. Patient-specific targeting guides were used for pedicle screw placement from Th1-L5, totaling 166 screws. Instrumentation for each side was randomized (augmented reality surgical navigation (ARSN) vs. C-arm) with an equal distribution of 83 screws in each group. CT was performed to evaluate the accuracy of both techniques by assessing the screw positions and the deviations between the inserted screws and planned trajectories. Postoperative CT showed that 98.80% (82/83) screws in ARSN group and 72.29% (60/83) screws in C-arm group were within the 2-mm safe zone (p < 0.001). The mean time for instrumentation per level in ARSN group was significantly shorter than that in C-arm group (56.17 ± 3.33 s vs. 99.22 ± 9.03 s, p < 0.001). The overall intraoperative registration time was 17.2 ± 3.5 s per segment. AR-based navigation technology can provide surgeons with accurate guidance of pedicle screw insertion and save the operation time by using the intraoperative rapid registration method of combining preoperative CT scanning and intraoperative C-arm 2D fluoroscopy.

Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery

Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.

Sonification as a reliable alternative to conventional visual surgical navigation

Despite the undeniable advantages of image-guided surgical assistance systems in terms of accuracy, such systems have not yet fully met surgeons' needs or expectations regarding usability, time efficiency, and their integration into the surgical workflow. On the other hand, perceptual studies have shown that presenting independent but causally correlated information via multimodal feedback involving different sensory modalities can improve task performance. This article investigates an alternative method for computer-assisted surgical navigation, introduces a novel four-DOF sonification methodology for navigated pedicle screw placement, and discusses advanced solutions based on multisensory feedback. The proposed method comprises a novel four-DOF sonification solution for alignment tasks in four degrees of freedom based on frequency modulation synthesis. We compared the resulting accuracy and execution time of the proposed sonification method with visual navigation, which is currently considered the state of the art. We conducted a phantom study in which 17 surgeons executed the pedicle screw placement task in the lumbar spine, guided by either the proposed sonification-based or the traditional visual navigation method. The results demonstrated that the proposed method is as accurate as the state of the art while decreasing the surgeon's need to focus on visual navigation displays instead of the natural focus on surgical tools and targeted anatomy during task execution.

Emerging Technologies in Shoulder Arthroplasty: Navigation, Mixed Reality, and Preoperative Planning

Shoulder arthroplasty is a rapidly improving and utilized management for end-stage arthritis that is associated with improved functional outcomes, pain relief, and long-term implant survival. Accurate placement of the glenoid and humeral components is critical for improved outcomes. Traditionally, preoperative planning was limited to radiographs and 2-dimensional computed tomography (CT); however, 3-dimensional CT is becoming more commonly utilized and necessary to understand complex glenoid and humeral deformities. To further increase accurate component placement, intraoperative assistive devices-patient-specific instrumentation, navigation, and mixed reality-minimize malpositioning, increase surgeon accuracy, and maximize fixation. These intraoperative technologies likely represent the future of shoulder arthroplasty.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Application of Virtual and Augmented Reality Technology in Hip Surgery: Systematic Review

BACKGROUND

Virtual and augmented reality (VAR) represents a combination of current state-of-the-art computer and imaging technologies and has the potential to be a revolutionary technology in many surgical fields. An increasing number of investigators have developed and applied VAR in hip-related surgery with the aim of using this technology to reduce hip surgery-related complications, improve surgical success rates, and reduce surgical risks. These technologies are beginning to be widely used in hip-related preoperative operation simulation and training, intraoperative navigation tools in the operating room, and postoperative rehabilitation.

OBJECTIVE

With the aim of reviewing the current status of virtual reality (VR) and augmented reality (AR) in hip-related surgery and summarizing its benefits, we discussed and briefly described the applicability, advantages, limitations, and future perspectives of VR and AR techniques in hip-related surgery, such as preoperative operation simulation and training; explored the possible future applications of AR in the operating room; and discussed the bright prospects of VR and AR technologies in postoperative rehabilitation after hip surgery.

METHODS

We searched the PubMed and Web of Science databases using the following key search terms: ("virtual reality" OR "augmented reality") AND ("pelvis" OR "hip"). The literature on basic and clinical research related to the aforementioned key search terms, that is, studies evaluating the key factors, challenges, or problems of using of VAR technology in hip-related surgery, was collected.

RESULTS

A total of 40 studies and reports were included and classified into the following categories: total hip arthroplasty, hip resurfacing, femoral neck fracture, pelvic fracture, acetabular fracture, tumor, arthroscopy, and postoperative rehabilitation. Quality assessment could be performed in 30 studies. Among the clinical studies, there were 16 case series with an average score of 89 out of 100 points (89%) and 1 case report that scored 81 (SD 10.11) out of 100 points (81%) according to the Joanna Briggs Institute Critical Appraisal Checklist. Two cadaveric studies scored 85 of 100 points (85%) and 92 of 100 points (92%) according to the Quality Appraisal for Cadaveric Studies scale.

CONCLUSIONS

VR and AR technologies hold great promise for hip-related surgeries, especially for preoperative operation simulation and training, feasibility applications in the operating room, and postoperative rehabilitation, and have the potential to assist orthopedic surgeons in operating more accurately and safely. More comparative studies are necessary, including studies focusing on clinical outcomes and cost-effectiveness.

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

How different augmented reality visualizations for drilling affect trajectory deviation, visual attention, and user experience

PURPOSE

Previous work has demonstrated the high accuracy of augmented reality (AR) head-mounted displays for pedicle screw placement in spinal fusion surgery. An important question that remains unanswered is how pedicle screw trajectories should be visualized in AR to best assist the surgeon.

METHODOLOGY

We compared five AR visualizations displaying the drill trajectory via Microsoft HoloLens 2 with different configurations of abstraction level (abstract or anatomical), position (overlay or small offset), and dimensionality (2D or 3D) against standard navigation on an external screen. We tested these visualizations in a study with 4 expert surgeons and 10 novices (residents in orthopedic surgery) on lumbar spine models covered by Plasticine. We assessed trajectory deviations ([Formula: see text]) from the preoperative plan, dwell times (%) on areas of interest, and the user experience.

RESULTS

Two AR visualizations resulted in significantly lower trajectory deviations (mixed-effects ANOVA, p<0.0001 and p<0.05) compared to standard navigation, whereas no significant differences were found between participant groups. The best ratings for ease of use and cognitive load were obtained with an abstract visualization displayed peripherally around the entry point and with a 3D anatomical visualization displayed with some offset. For visualizations displayed with some offset, participants spent on average only 20% of their time examining the entry point area.

CONCLUSION

Our results show that real-time feedback provided by navigation can level task performance between experts and novices, and that the design of a visualization has a significant impact on task performance, visual attention, and user experience. Both abstract and anatomical visualizations can be suitable for navigation when not directly occluding the execution area. Our results shed light on how AR visualizations guide visual attention and the benefits of anchoring information in the peripheral field around the entry point.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

Hemisphere Tabulation Method: An Ingenious Approach for Pose Evaluation of Instruments Using the Electromagnetic-Based Stereo Imaging Method

Drilling of a bone surface often occurs in clinical orthopaedic surgery. The position and orientation of the instrument are the most important factors in this process. Theoretically, some mechanical components may assist in orienting an instrument to certain bone shapes, such as the knee joint and caput femoris. However, the mechanical assisting component does not seem to work in some confined spaces where the bone shape is a free-form surface. In this paper, we propose an ingenious hemisphere tabulation method (HTM) for assessing the pose accuracy of an instrument. The acquisition and assessment of HTM is conducted based on an electromagnetic-based stereo imaging method using a custom-made optical measurement unit, and the operation steps of HTM are described in detail. Experimental results based on 50 tests show that the HTM can identify ideal poses and the evaluated pose of an instrument location on a hemisphere model. The mean error of pose localisation is 7.24 deg, with a range of 1.35 to 15.84 and a standard of 3.66 deg, which is more accurate than our previous method.

Translation of Medical AR Research into Clinical Practice

Translational research is aimed at turning discoveries from basic science into results that advance patient treatment. The translation of technical solutions into clinical use is a complex, iterative process that involves different stages of design, development, and validation, such as the identification of unmet clinical needs, technical conception, development, verification and validation, regulatory matters, and ethics. For this reason, many promising technical developments at the interface of technology, informatics, and medicine remain research prototypes without finding their way into clinical practice. Augmented reality is a technology that is now making its breakthrough into patient care, even though it has been available for decades. In this work, we explain the translational process for Medical AR devices and present associated challenges and opportunities. To the best knowledge of the authors, this concept paper is the first to present a guideline for the translation of medical AR research into clinical practice.

Next step trauma and orthopaedic surgery: integration of augmented reality for reduction and nail implantation of tibial fractures

INTRODUCTION

There is a tremendous scope of hardware and software development going on in augmented reality (AR), also in trauma and orthopaedic surgery. However, there are only a few systems available for intra-operative 3D imaging and guidance, most of them rely on peri- and intra-operative X-ray imaging. Especially in complex situations such as pelvic surgery or multifragmentary multilevel fractures, intra-operative 3D imaging and implant tracking systems have proven to be of great advantage for the outcome of the surgery and can help reduce X-ray exposure, at least for the surgical team (Ochs et al. in Injury 41:1297 1305, 2010). Yet, the current systems do not provide the ability to have a dynamic live view from the perspective of the surgeon. Our study describes a prototype AR-based system for live tracking which does not rely on X-rays.

MATERIALS AND METHODS

A protype live-view intra-operative guidance system using an AR head-mounted device (HMD) was developed and tested on the implantation of a medullary nail in a tibia fracture model. Software algorithms that allow live view and tracking of the implant, fracture fragments and soft tissue without the intra-operative use of X-rays were derived.

RESULTS

The implantation of a medullar tibia nail is possible while only relying on AR-guidance and live view without the intra-operative use of X-rays.

CONCLUSIONS

The current paper describes a feasibility study with a prototype of an intra-operative dynamic live tracking and imaging system that does not require intra-operative use of X-rays and dynamically adjust to the perspective of the surgeons due to an AR HMD. To our knowledge, the current literature does not describe any similar systems. This could be the next step in surgical imaging and education and a promising way to improve patient care.

Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study

BACKGROUND

To achieve an optimal clinical outcome in reverse total shoulder arthroplasty (RSA), accurate placement of the components is essential. The recently introduced navigation technology of augmented reality (AR) through head-mounted displays (HMD) offers a promising new approach to visualize the anatomy and navigate component positioning in various orthopedic surgeries. We hypothesized that AR through HMD is feasible, reliable, and accurate for guidewire placement in RSA baseplate positioning.

METHODS

Twelve human cadaver shoulders were scanned with computed tomography (CT) and RSA baseplate positioning was 3-D planned using dedicated software. The shoulders were prepared through a deltopectoral approach and an augmented reality hologram was superimposed using the HMD Microsoft HoloLense. The central guidewire was then navigated through the HMD to achieve the planned entry point and trajectory. Postoperatively, the shoulders were CT-scanned a second time and the deviation from the planning was calculated.

RESULTS

The mean deviation of the entry point was 3.5 mm  ± 1.7 mm (95% CI 2.4 mm; 4.6 mm). The mean deviation of the planned trajectory was 3.8°  ± 1.7° (95% CI 2.6°; 4.9°).

CONCLUSION

Augmented reality seems feasible and reliable for baseplate guidewire positioning in reverse total shoulder arthroplasty. The achieved values were accurate.

The effects of social support on depression and anxiety in the perinatal period: A mixed-methods systematic review

BACKGROUND

The social support offered to women in the perinatal period can buffer against depression and anxiety. The sources and types of support that reduce maternal levels of depression and anxiety are not well understood.

AIMS

To investigate the effects of structural and functional support on depression and anxiety in women in the perinatal period, and to explore the experiences of women around support during the perinatal period.

METHOD

A comprehensive search of six electronic databases was undertaken. Relevant studies published from January 2010 to April 2020 were included (PROSPERO reference number: CRD42020194228). Quantitative and qualitative studies were eligible if they reported the effects of, or had themes related to receiving functional or structural support in the perinatal period on women's levels of depression or anxiety. Qualitative data was synthesised using a thematic synthesis method. Quantitative data could not be pooled due to the lack of comparable RCTs or cohort studies and was thus presented in a narrative form.

RESULTS

Fifty-one articles (41 quantitative and 10 qualitative studies) were included. Analysis of quantitative studies demonstrated that insufficient support from partner, friends and family was associated with greater risk of symptoms of depression and anxiety. Distance-delivered interventions (via internet or telephone) diminished levels of depression and anxiety. Qualitative data revealed three overarching themes: appreciating all forms of support in the perinatal period, recognising appropriate and inappropriate support from health professionals and services, and having barriers to mobilising functional and structural support.

LIMITATIONS

Grey literature was not explored and search strategies only included English, Spanish and Portuguese language articles. The reviewed studies were heterogeneous and for this reason, quantitative assessments were not feasible.

CONCLUSION

Support from specific sources such as family members, friends who are also mothers, health professionals, neighbours, supervisors, co-workers and online communities still need to be further investigated. Implications for health professionals involved in the care of women in the perinatal period are discussed.

Mixed Reality Interfaces for Achieving Desired Views with Robotic X-ray Systems

Robotic X-ray C-arm imaging systems can precisely achieve any position and orientation relative to the patient. Informing the system, however, what pose exactly corresponds to a desired view is challenging. Currently these systems are operated by the surgeon using joysticks, but this interaction paradigm is not necessarily effective because users may be unable to efficiently actuate more than a single axis of the system simultaneously. Moreover, novel robotic imaging systems, such as the Brainlab Loop-X, allow for independent source and detector movements, adding even more complexity. To address this challenge, we consider complementary interfaces for the surgeon to command robotic X-ray systems effectively. Specifically, we consider three interaction paradigms: (1) the use of a pointer to specify the principal ray of the desired view relative to the anatomy, (2) the same pointer, but combined with a mixed reality environment to synchronously render digitally reconstructed radiographs from the tool's pose, and (3) the same mixed reality environment but with a virtual X-ray source instead of the pointer. Initial human-in-the-loop evaluation with an attending trauma surgeon indicates that mixed reality interfaces for robotic X-ray system control are promising and may contribute to substantially reducing the number of X-ray images acquired solely during "fluoro hunting" for the desired view or standard plane.

From Diagnosis to Therapy: The Role of Virtual and Augmented Reality in Orthopaedic Trauma Surgery

By reducing procedure-related problems, advancements in computer-assisted surgery (CAS) and surgical training aim to boost operative precision and enhance patient safety. Orthopaedic training and practice have started to change as a result of the incorporation of reality technologies like virtual reality (VR), augmented reality (AR), and mixed reality (MR) into CAS. Today's trainees can engage in realistic and highly involved operational simulations without supervision. With the coronavirus disease 2019 (COVID-19) pandemic, there is a greater need for breakthrough technology adoption. VR is an interactive technology that enables personalised care and could support successful patient-centered rehabilitation. It is a valid and trustworthy evaluation method for determining joint range of motion, function, and balance in physical rehabilitation. It may make it possible to customise care, encourage patients, boost compliance, and track their advancement. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. As little patient observation is needed, this may lessen the workload clinicians must bear. The ability to use it for home-based therapy is often available commercially as well.

The objectives of this review are to evaluate the technology available, comprehend the available evidence regarding the benefit, and take into account implementation problems in clinical practice. The use of this technology, its practical and moral ramifications, and how it will affect orthopaedic doctors and their patients are also covered. This review offers a current and thorough analysis of the reality technologies and their uses in orthopaedic surgery.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Spinal decompression with patient-specific guides

BACKGROUND CONTEXT

Patient-specific instruments (PSI) have been well established in spine surgery for pedicle screw placement. However, its utility in spinal decompression surgery is yet to be investigated.

PURPOSE

The purpose of this study was to investigate the feasibility and utility of PSI in spinal decompression surgery compared with conventional freehand (FH) technique for both expert and novice surgeons.

STUDY DESIGN

Human cadaver study.

METHODS

Thirty-two midline decompressions were performed on 4 fresh-frozen human cadavers. An expert spine surgeon and an orthopedic resident (novice) each performed 8 FH and 8 PSI-guided decompressions. Surgical time for each decompression method was measured. Postoperative decompression area, cranial decompression extent in relation to the intervertebral disc, and lateral recess bony overhang were measured on postoperative CT-scans. In the PSI-group, the decompression area and osteotomy accuracy were evaluated.

RESULTS

The surgical time was similar in both techniques, with 07:25 min (PSI) versus 06:53 min (FH) for the expert surgeon and 12:36 min (PSI) vs. 11:54 (FH) for the novice surgeon. The postoperative cranial decompression extent and the lateral recess bony overhang did not differ between both techniques and surgeons. Further, the postoperative decompression area was significantly larger with the PSI than with the FH for the novice surgeon (477 vs. 305 mm²; p=.01), but no significant difference was found between both techniques for the expert surgeon. The execution of the decompression differed from the preoperative plan in the decompression area by 5%, and the osteotomy planes had an accuracy of 1-3 mm.

CONCLUSION

PSI-guided decompression is feasible and accurate with similar procedure time to the standard FH technique in a cadaver model, which warrants further investigation in vivo. In comparison to the FH technique, a more extensive decompression was achieved with PSI in the novice surgeon's hands in this study.

CLINICAL SIGNIFICANCE

The PSI-guided spinal decompression technique may be a useful alternative to FH decompression in certain situations. A special potential of the PSI technique could lie in the technical aid for novice surgeons and in situations with unconventional anatomy or pathologies such as deformity or tumor. This study serves as a starting point toward PSI-guided spinal decompression, but further in vivo investigations are necessary.

The Surgical Treatment of Osteoarthritis

Osteoarthritis is a degenerative condition affecting the whole joint with the underlying bone, representing a major source of pain, disability, and socioeconomic cost worldwide. Age is considered the strongest risk factor, albeit abnormal biomechanics, morphology, congenital abnormality, deformity, malalignment, limb-length discrepancy, lifestyle, and injury may further increase the risk of the development and progression of osteoarthritis as well. Pain and loss of function are the main clinical features that lead to treatment. Although early manifestations of osteoarthritis are amenable to lifestyle modification, adequate pain management, and physical therapy, disease advancement frequently requires surgical treatment. The symptomatic progression of osteoarthritis with radiographical confirmation can be addressed either with arthroscopic interventions, (joint) preservation techniques, or bone fusion procedures, whereas (joint) replacement is preferentially reserved for severe and end-stage disease. The surgical treatment aims at alleviating pain and disability while restoring native biomechanics. Miscellaneous surgical techniques for addressing osteoarthritis exist. Advanced computer-integrated surgical concepts allow for patient personalization and optimization of surgical treatment. The scope of this article is to present an overview of the fundamentals of conventional surgical treatment options for osteoarthritis of the human skeleton, with emphasis on arthroscopy, preservation, arthrodesis, and replacement. Contemporary computer-assisted orthopaedic surgery concepts are further elucidated.

HoloLens 1 vs. HoloLens 2: Improvements in the New Model for Orthopedic Oncological Interventions

This work analyzed the use of Microsoft HoloLens 2 in orthopedic oncological surgeries and compares it to its predecessor (Microsoft HoloLens 1). Specifically, we developed two equivalent applications, one for each device, and evaluated the augmented reality (AR) projection accuracy in an experimental scenario using phantoms based on two patients. We achieved automatic registration between virtual and real worlds using patient-specific surgical guides on each phantom. They contained a small adaptor for a 3D-printed AR marker, the characteristic patterns of which were easily recognized using both Microsoft HoloLens devices. The newest model improved the AR projection accuracy by almost 25%, and both of them yielded an RMSE below 3 mm. After ascertaining the enhancement of the second model in this aspect, we went a step further with Microsoft HoloLens 2 and tested it during the surgical intervention of one of the patients. During this experience, we collected the surgeons’ feedback in terms of comfortability, usability, and ergonomics. Our goal was to estimate whether the improved technical features of the newest model facilitate its implementation in actual surgical scenarios. All of the results point to Microsoft HoloLens 2 being better in all the aspects affecting surgical interventions and support its use in future experiences.

Augmented Reality Spine Surgery Navigation: Increasing Pedicle Screw Insertion Accuracy for Both Open and Minimally Invasive Spine Surgeries

STUDY DESIGN

Collectively, seven cadavers were instrumented with 124 thoracolumbar pedicle screws using VisAR augmented reality/guidance. Sixty-five screws were inserted into four donors using open dissection spine surgery. Fifty-nine screws were positioned in three donors with a minimally invasive spine surgery (MISS) procedure. For both open and MISS, VisAR was used exclusively for pedicle screw navigation.

OBJECTIVE

The objective of this study was to determine the accuracy of pedicle screw placement using VisAR for open spine and MISS procedures.

SUMMARY OF BACKGROUND DATA

Pedicle screw placement can be challenging depending on anatomical location and a surgeon's experience. AR may minimize fluoroscopy use and speed screw insertion.

METHODS

Prior to computed tomography (CT) a series of four image visible April Tag optical fiducials were attached to the backs' of the donors. Resulting images were used preoperatively for planned virtual pedicle screw pathways including entry point, trajectory, and depth. The study link was encrypted on a quick response (QR) code, printed, and viewed in the operating room (OR) by the surgeon using VisAR (HoloLens 2 headset). Viewing the code wirelessly uploads and launches the study, converting the DICOM data to holographic images which register to the fiducials on the donor's back. The annotated pathways for each pedicle were called up by voice command and the surgeon positioned each screw by aligning with the virtual guidance hologram.

RESULTS

Overall, 124 pedicle screws were inserted with VisAR navigation with 96% accuracy (Gertzbein-Robbins grades A and B). The combined angle of error was 2.4° and the distance error was 1.9 mm.

CONCLUSION

Augmented reality is a highly accurate, emerging technology for navigating both open and minimally invasive spine surgery techniques with off-the-shelf headset hardware.

LEVEL OF EVIDENCE

N/A.

Augmented Reality Device for Preoperative Marking of Spine Surgery Can Improve the Accuracy of Level Identification

Introduction

Wrong-site spine surgery is an incident that could result in possible severe complications. In this present spinal surgery, the accurate spinal level is confirmed via preoperative or intraoperative radiographic marking. However, the location of radiographic marking has been determined from the manual palpation on the landmarks of the body surface. As a result, severe spine deformity can make it hard to identify the spinal level by manual palpation, thus leading to misidentification of the spinal level.

Recently, the use of mixed reality in spine surgery is gradually increasing. In this study, we will demonstrate a head-mounted display (HMD) device that can project a hologram (3D image) of the patient's bone onto the actual patient's body to improve the accuracy of level identification for spine surgery.

Technical Note

3D CT images are created preoperatively, and the bone's STL data (3D data) are generated with the workstation. The created STL data are downloaded to the augmented reality software Holoeyes, installed on the HMD. Through this device, surgeons can view the hologram (3D image) of a patient's bone overlaying on an actual patient's body.

We temporally estimated the spinous process level only by manual palpation without an HMD. Then, we estimated the spinous process level again after matching this hologram to a real bone with an HMD. The accuracy of the level identification with an HMD and without an HMD was examined by radiographic marking in order to evaluate the misidentification rate of the level. Without an HMD, the misidentification rate of the level was at 26.5%, while with it, the rate was reduced to 14.3%.

Conclusions

On preoperative marking, an HMD-projecting bone image onto a patient's body could allow us to estimate the spinal level more accurately. Identification of the spinal level using mixed reality is effective in preventing wrong-site spine surgery.

What is Significant in Modern Augmented Reality: A Systematic Analysis of Existing Reviews

Augmented reality (AR) is a field of technology that has evolved drastically during the last decades, due to its vast range of applications in everyday life. The aim of this paper is to provide researchers with an overview of what has been surveyed since 2010 in terms of AR application areas as well as in terms of its technical aspects, and to discuss the extent to which both application areas and technical aspects have been covered, as well as to examine whether one can extract useful evidence of what aspects have not been covered adequately and whether it is possible to define common taxonomy criteria for performing AR reviews in the future. To this end, a search with inclusion and exclusion criteria has been performed in the Scopus database, producing a representative set of 47 reviews, covering the years from 2010 onwards. A proper taxonomy of the results is introduced, and the findings reveal, among others, the lack of AR application reviews covering all suggested criteria.

Glenoid Component Placement Assisted by Augmented Reality Through a Head-Mounted Display During Reverse Shoulder Arthroplasty

Component positioning is a key factor for avoiding complications and improving functional outcomes in reverse shoulder arthroplasty. Preoperative planning can improve component positioning. However, translating the preoperative plan into the surgical procedure can be challenging. This is particularly the case for the glenoid component positioning in severe deformity or limited visualization of the scapula. Different computational-assisted techniques have been developed to aid implementation of the preoperative plan into the surgical procedure. Navigated augmented reality (AR) refers to the real world augmented with virtual real-time information about the position and orientation of instruments and components. This information can be presented through a head-mounted display (HMD), which enables the user to visualize the virtual information directly overlaid onto the real world. Navigated AR systems through HMD have been validated for shoulder arthroplasty using phantoms and cadavers. This article details a step-by-step guide use of a navigated AR system through HMD, in the placement of the glenoid bony-augmented component.

Precision surgery: the role of intra-operative real-time image guidance - outcomes from a multidisciplinary European consensus conference

Developments within the field of image-guided surgery are ever expanding, driven by collective involvement of clinicians, researchers, and industry. While the general conception of the potential of image-guided surgery is to improve surgical outcome, the specific motives and goals that drive can differ between the different expert groups. To establish the current and future role of intra-operative image guidance within the field of image-guided surgery a Delphi consensus survey was conducted during the 2^nd European Congress on Image-guided surgery. This multidisciplinary survey included questions on the conceptual potential and clinical value of image-guided surgery and was aimed at defining specific areas of research and development in the field in order to stimulate further advances towards precision surgery. Obtained results based on questionnaires filled in by 56 panel experts (clinicians: N=30, researchers: N=20 and industry: N=6) were discussed during a dedicated expert discussion session during the conference. The outcome of this Delphi consensus is indicative of the potential improvements offered by image-guided surgery and of the need for further research in this emerging field, that can be enriched by the identification of reliable molecular targets.

Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review

This article presents a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality (AR) surgery applications from 2013 to 2020. Articles were categorised by: OST-HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. 91 articles fulfilled all inclusion criteria. Some clear trends emerge. The Microsoft HoloLens increasingly dominates the field, with orthopaedic surgery being the most popular application (28.6%). By far the most common surgical context is surgical guidance (n=58) and segmented preoperative models dominate visualisation (n=40). Experiments mainly involve phantoms (n=43) or system setup (n=21), with patient case studies ranking third (n=19), reflecting the comparative infancy of the field. Experiments cover issues from registration to perception with very different accuracy results. Human factors emerge as significant to OST-HMD utility. Some factors are addressed by the systems proposed, such as attention shift away from the surgical site and mental mapping of 2D images to 3D patient anatomy. Other persistent human factors remain or are caused by OST-HMD solutions, including ease of use, comfort and spatial perception issues. The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed.

A Novel Technique of Mixed Reality Systems in the Treatment of Spinal Cord Tumors

Several reports have compared spinal cord tumor removal techniques but none have clearly described the appropriate site and level of indication for laminectomy or laminoplasty. The approach method for tumor removal depends on the type and localization of the tumor and the surgeon's skill. Therefore, a system that can suggest various surgical techniques is useful for spinal cord tumor surgery. The mixed reality system introduced in this paper is an excellent system that can suggest various surgical procedures. Using this system for spinal cord tumor removal, we made the surgery less invasive; therefore, we introduced this system and demonstrated its usefulness. Stereoscopic data of the patients with spinal cord tumors were obtained from preoperative myelogram-CT data. Stereoscopic laminectomy models including tumors were created using Blender, a free three-dimensional (3D) image editing software. We observed these data as 3D object images using a head-mounted display (HMD). This HMD is commercially available and relatively inexpensive. The surgical procedure is determined by considering those 3D images, radiological diagnosis, and the skill of surgeons. Intraoperative confirmation of the laminectomy site could be performed using the HMD. The 3D visualization of pathological conditions resulted in correct preoperative surgical planning and less invasive surgery in all five cases. Stereoscopic images using HMDs allow us a more intuitive understanding of the positional relationship between the tumor and spinal structure. These 3D object images can bring us more accurate preoperative planning and proper determination of surgical methods.

Biomechanical Studies for Glenoid Based Labral Repairs With Suture Anchors Do Not Use Consistent Testing Methods: A Critical Systematic Review

PURPOSE

The purpose of this systematic review was to investigate variability in biomechanical testing protocols for laboratory-based studies using suture anchors for glenohumeral shoulder instability and SLAP lesion repair.

METHODS

A systematic review of Medline, Embase, Scopus, and Google Scholar using Covidence software was performed for all biomechanical studies investigating labral-based suture anchor repair for shoulder instability and SLAP lesions. Clinical studies, technical notes or surgical technique descriptions, or studies treating glenoid bone loss or capsulorrhaphy were excluded. Risk of bias (ROB) was assessed with the ROBINS-I tool. Study quality was assessed with the Quality Appraisal for Cadaveric Studies. Heterogeneity was assessed with the I² statistic.

RESULTS

A total of 41 studies were included. ROB was serious and critical in 27 studies, moderate in 13, and low in 1; 6 studies had high quality, 21 good quality, 10 moderate quality, 2 low quality, and 2 very low quality. Thirty-one studies used and 22 studies included cyclic loading. Angle of anchor insertion was reported by 33 studies. The force vector for displacement varied. The most common directions were perpendicular to the glenoid (n = 9), and anteroinferior or anterior (n = 8). The most common outcome measures were load to failure (n = 35), failure mode (n = 23), and stiffness (n = 21). Other outcome measures included load at displacement, displacement at failure, tensile load at displacement, translation, energy absorbed, cycles to failure, contact pressure, and elongation.

CONCLUSION

This systematic review demonstrated a clear lack of consistency in those cadaver studies that investigated biomechanical properties after surgical repair with suture anchors for shoulder instability and SLAP lesions. Testing methods between studies varied substantially with no universally applied standard for preloading, load to failure and cyclic loading protocols, insertion angles of suture anchors, or direction of loading. To allow comparability between studies standardization of testing protocols is strongly recommended.

Head-Mounted Display-Based Augmented Reality for Image-Guided Media Delivery to the Heart: A Preliminary Investigation of Perceptual Accuracy

By aligning virtual augmentations with real objects, optical see-through head-mounted display (OST-HMD)-based augmented reality (AR) can enhance user-task performance. Our goal was to compare the perceptual accuracy of several visualization paradigms involving an adjacent monitor, or the Microsoft HoloLens 2 OST-HMD, in a targeted task, as well as to assess the feasibility of displaying imaging-derived virtual models aligned with the injured porcine heart. With 10 participants, we performed a user study to quantify and compare the accuracy, speed, and subjective workload of each paradigm in the completion of a point-and-trace task that simulated surgical targeting. To demonstrate the clinical potential of our system, we assessed its use for the visualization of magnetic resonance imaging (MRI)-based anatomical models, aligned with the surgically exposed heart in a motion-arrested open-chest porcine model. Using the HoloLens 2 with alignment of the ground truth target and our display calibration method, users were able to achieve submillimeter accuracy (0.98 mm) and required 1.42 min for calibration in the point-and-trace task. In the porcine study, we observed good spatial agreement between the MRI-models and target surgical site. The use of an OST-HMD led to improved perceptual accuracy and task-completion times in a simulated targeting task.

Augmented reality-guided positioning system for radiotherapy patients

In modern radiotherapy, error reduction in the patients’ daily setup error is important for achieving accuracy. In our study, we proposed a new approach for the development of an assist system for the radiotherapy position setup by using augmented reality (AR). We aimed to improve the accuracy of the position setup of patients undergoing radiotherapy and to evaluate the error of the position setup of patients who were diagnosed with head and neck cancer, and that of patients diagnosed with chest and abdomen cancer. We acquired the patient's simulation CT data for the three‐dimensional (3D) reconstruction of the external surface and organs. The AR tracking software detected the calibration module and loaded the 3D virtual model. The calibration module was aligned with the Linac isocenter by using room lasers. And then aligned the virtual cube with the calibration module to complete the calibration of the 3D virtual model and Linac isocenter. Then, the patient position setup was carried out, and point cloud registration was performed between the patient and the 3D virtual model, such the patient's posture was consistent with the 3D virtual model. Twenty patients diagnosed with head and neck cancer and 20 patients diagnosed with chest and abdomen cancer in the supine position setup were analyzed for the residual errors of the conventional laser and AR‐guided position setup. Results show that for patients diagnosed with head and neck cancer, the difference between the two positioning methods was not statistically significant (P > 0.05). For patients diagnosed with chest and abdomen cancer, the residual errors of the two positioning methods in the superior and inferior direction and anterior and posterior direction were statistically significant (t = −5.80, −4.98, P < 0.05). The residual errors in the three rotation directions were statistically significant (t = −2.29 to −3.22, P < 0.05). The experimental results showed that the AR technology can effectively assist in the position setup of patients undergoing radiotherapy, significantly reduce the position setup errors in patients diagnosed with chest and abdomen cancer, and improve the accuracy of radiotherapy.

Augmented Reality and Image-Guided Robotic Liver Surgery

Artificial intelligence makes surgical resection easier and safer, and, at the same time, can improve oncological results. The robotic system fits perfectly with these more or less diffused technologies, and it seems that this benefit is mutual. In liver surgery, robotic systems help surgeons to localize tumors and improve surgical results with well-defined preoperative planning or increased intraoperative detection. Furthermore, they can balance the absence of tactile feedback and help recognize intrahepatic biliary or vascular structures during parenchymal transection. Some of these systems are well known and are already widely diffused in open and laparoscopic hepatectomies, such as indocyanine green fluorescence or ultrasound-guided resections, whereas other tools, such as Augmented Reality, are far from being standardized because of the high complexity and elevated costs. In this paper, we review all the experiences in the literature on the use of artificial intelligence systems in robotic liver resections, describing all their practical applications and their weaknesses.

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections

Patient-specific instruments (PSIs) have become a valuable tool for osteotomy guidance in complex surgical scenarios such as pelvic tumor resection. They provide similar accuracy to surgical navigation systems but are generally more convenient and faster. However, their correct placement can become challenging in some anatomical regions, and it cannot be verified objectively during the intervention. Incorrect installations can result in high deviations from the planned osteotomy, increasing the risk of positive resection margins. In this work, we propose to use augmented reality (AR) to guide and verify PSIs placement. We designed an experiment to assess the accuracy provided by the system using a smartphone and the HoloLens 2 and compared the results with the conventional freehand method. The results showed significant differences, where AR guidance prevented high osteotomy deviations, reducing maximal deviation of 54.03 mm for freehand placements to less than 5 mm with AR guidance. The experiment was performed in two versions of a plastic three-dimensional (3D) printed phantom, one including a silicone layer to simulate tissue, providing more realism. We also studied how differences in shape and location of PSIs affect their accuracy, concluding that those with smaller sizes and a homogeneous target surface are more prone to errors. Our study presents promising results that prove AR's potential to overcome the present limitations of PSIs conveniently and effectively.