Grand Adventure of Augmented Reality in Landscape of Surgery

Extended reality for mapping perforator-based flaps in breast reconstruction: a systematic review and meta-analysis

Introduction

Extended reality technologies including augmented reality (AR) and virtual reality (VR) can be used in surgical settings for surgical planning, perioperative visualisation of patient anatomy and simulation of operative steps. This study aimed to ascertain the role of extended reality in perforator-based breast reconstruction.

Methods

A systematic search of the literature was performed. Screening was conducted independently by two reviewers, with conflicts resolved through consensus.

Results

In total, 4957 articles were identified of which 10 were included, comprising 229 flaps. Overall, localisation of perforator emergence with AR and VR were 4 mm (95% confidence interval [CI] 3.2-4.9) and 14 mm (95% CI 5.5-22.5), respectively, from the correct intraoperatively identified location. Two studies reported a significant reduction in harvesting time (88 min for VR and 19 min for AR). The pooled mean image processing time from four studies was 39±47 min. Preoperative VR was associated with shorter operative times than conventional Doppler ultrasound (478±56.94 vs 606.29 ± 81.94 min, P<0.05). Use of a simulated environment for mapping perforators appeared to reduce complications (wound breakdown, flap revision, flap loss, infection); however, this failed to reach statistical significance (odds ratio 0.6; 95% CI 0.3-1.3; P=0.20).

Conclusions

This study suggests that AR and VR offer limited benefit in improving accuracy of perforator identification; however, they may reduce flap harvesting and total operative time. Key limitations include heterogeneity and quality of the included studies. With larger sample sizes and higher quality evidence, definitive benefits and longitudinal outcomes relating to use of extended reality in perforator-based breast reconstruction may be established.

[Augmented reality as a method of neuronavigation in microsurgical treatment of cerebrovascular diseases: description of the method and clinical experience]

Augmented reality (AR) is a promising area in microsurgical treatment of cerebrovascular pathologies that can significantly facilitate preoperative planning and intraoperative understanding of anatomy.

OBJECTIVE

To describe AR-assisted neuronavigation in microsurgical treatment of intracranial aneurysms, arteriovenous malformations and cavernomas; to evaluate accuracy and applicability of AR-assisted neuronavigation.

MATERIAL AND METHODS

The study involved 22 patients with cerebral aneurysms, arteriovenous and cavernous malformations. Microsoft Hololens 2 HMD glasses and «Medgital» software for AR navigation were used. Accuracy of registration (TRE and FRE) and time for preoperative preparation were evaluated.

RESULTS. MEAN

TRE when using QR code was 0.6±0.2 cm, when combining through craniometric points - 1.4±0.6 cm. Time for preoperative image processing was 24.7±5.1 minutes, application setup in the operating theatre - 1.6±0.2 minutes. Combination using QR code provided higher accuracy of registration compared to craniometric points. AR-assisted navigation improved visualization and planning of surgeries for aneurysms, arteriovenous malformations, microvascular anastomoses and cavernous angiomas.

CONCLUSION

AR-assisted navigation is an innovative method with specific advantages that can potentially improve microsurgical treatment of cerebrovascular diseases. Further research is needed to confirm these findings and develop AR technology in neurosurgery.

Augmented Reality Integration in Skull Base Neurosurgery: A Systematic Review

Background and Objectives: To investigate the role of augmented reality (AR) in skull base (SB) neurosurgery. Materials and Methods: Utilizing PRISMA methodology, PubMed and Scopus databases were explored to extract data related to AR integration in SB surgery. Results: The majority of 19 included studies (42.1%) were conducted in the United States, with a focus on the last five years (77.8%). Categorization included phantom skull models (31.2%, n = 6), human cadavers (15.8%, n = 3), or human patients (52.6%, n = 10). Microscopic surgery was the predominant modality in 10 studies (52.6%). Of the 19 studies, surgical modality was specified in 18, with microscopic surgery being predominant (52.6%). Most studies used only CT as the data source (n = 9; 47.4%), and optical tracking was the prevalent tracking modality (n = 9; 47.3%). The Target Registration Error (TRE) spanned from 0.55 to 10.62 mm. Conclusion: Despite variations in Target Registration Error (TRE) values, the studies highlighted successful outcomes and minimal complications. Challenges, such as device practicality and data security, were acknowledged, but the application of low-cost AR devices suggests broader feasibility.

Exposure to Extended Reality and Artificial Intelligence-Based Manifestations: A Primer on the Future of Hip and Knee Arthroplasty

BACKGROUND

Software-infused services, from robot-assisted and wearable technologies to artificial intelligence (AI)-laden analytics, continue to augment clinical orthopaedics - namely hip and knee arthroplasty. Extended reality (XR) tools, which encompass augmented reality, virtual reality, and mixed reality technology, represent a new frontier for expanding surgical horizons to maximize technical education, expertise, and execution. The purpose of this review is to critically detail and evaluate the recent developments surrounding XR in the field of hip and knee arthroplasty and to address potential future applications as they relate to AI.

METHODS

In this narrative review surrounding XR, we discuss (1) definitions, (2) techniques, (3) studies, (4) current applications, and (5) future directions. We highlight XR subsets (augmented reality, virtual reality, and mixed reality) as they relate to AI in the increasingly digitized ecosystem within hip and knee arthroplasty.

RESULTS

A narrative review of the XR orthopaedic ecosystem with respect to XR developments is summarized with specific emphasis on hip and knee arthroplasty. The XR as a tool for education, preoperative planning, and surgical execution is discussed with future applications dependent upon AI to potentially obviate the need for robotic assistance and preoperative advanced imaging without sacrificing accuracy.

CONCLUSION

In a field where exposure is critical to clinical success, XR represents a novel stand-alone software-infused service that optimizes technical education, execution, and expertise but necessitates integration with AI and previously validated software solutions to offer opportunities that improve surgical precision with or without the use of robotics and computed tomography-based imaging.

Skin deformation analysis for pre-operative planning of DIEAP flap reconstruction surgery

Deep inferior epigastric artery perforator (DIEAP) flap reconstruction surgeries can potentially benefit from augmented reality (AR) in the context of surgery planning and outcomes improvement. Although three-dimensional (3D) models help visualize and map the perforators, the anchorage of the models to the patient's body during surgery does not consider eventual skin deformation from the moment of computed tomography angiography (CTA) data acquisition until the position of the patient while in surgery. In this work, we compared the 3D deformation registration from supine arms down (CTA position) to supine with arms at 90° degrees (surgical position), estimating the patient's skin deformation. We processed the data sets of 20 volunteers with a 3D rigid registration tool and performed a descriptive statistical analysis and statistical inference. With 2.45 mm of root mean square and 2.89 mm of standard deviation, results include 30% cases of deformation above 3 mm and 15% above 4 mm. Pose transformation deformation indicates that 3D surface data from the CTA scan position differs from data acquired in loco at the surgical table. Such results indicate that research should be conducted to construct accurate 3D models using CTA data to display on the patient, while considering projection errors when using AR technology.

Leading Transformation in Medical Education Through Extended Reality

Extended reality (XR) has exponentially developed over the past decades to incorporate technology whereby users can visualise, explore, and interact with 3-dimensional-generated computer environments, and superimpose virtual reality (VR) onto real-world environments, thus displaying information and data on various levels of the reality-virtuality continuum. In the context of medicine, VR tools allow for anatomical assessment and diagnosis, surgical training through lifelike procedural simulations, planning of surgeries and biopsies, intraprocedural guidance, and medical education. The following chapter aims to provide an overview of the currently available evidence and perspectives on the application of XR within medical education. It will focus on undergraduate and postgraduate teaching, medical education within Low-Middle Income Countries, key practical steps in implementing a successful XR programme, and the limitations and future of extended reality within medical education.

Augmented reality in medical education: students' experiences and learning outcomes

Augmented reality (AR) is a relatively new technology that allows for digitally generated three-dimensional representations to be integrated with real environmental stimuli. AR can make use of smart phones, tablets, or other devices to achieve a highly stimulating learning environment and hands-on immersive experience. The use of AR in industry is becoming widespread with applications being developed for use not just for entertainment and gaming but also healthcare, retail and marketing, education, military, travel and tourism, automotive industry, manufacturing, architecture, and engineering. Due to the distinct learning advantages that AR offers, such as remote learning and interactive simulations, AR-based teaching programs are also increasingly being adopted within medical schools across the world. These advantages are further highlighted by the current COVID-19 pandemic, which has caused an even greater shift towards online learning. In this review, we investigate the use of AR in medical training/education and its effect on students' experiences and learning outcomes. This includes the main goals of AR-based learning, such as to simplify the delivery and enhance the comprehension of complex information. We also describe how AR can enhance the experiences of medical students, by improving knowledge and understanding, practical skills and social skills. These concepts are discussed within the context of specific AR medical training programs, such as HoloHuman, OculAR SIM, and HoloPatient. Finally, we discuss the challenges of AR in learning and teaching and propose future directions for the use of this technology in medical education.

Current and Future Photography Techniques in Aesthetic Surgery

Background

The rapidly increasing modalities and mediums of clinical photography, use of 3-dimensional (3D) and 4-dimensional (4D) patient modeling, and widening implementation of cloud-based storage and artificial intelligence (AI) call for an overview of various methods currently in use as well as future considerations in the field.

Objectives

Through a close look at the methods used in aesthetic surgery photography, clinicians will be able to select the modality best suited to their practice and goals.

Methods

Review and discussion of current data pertaining to: 2-dimensional (2D) and 3D clinical photography, current photography software, augmented reality reconstruction, AI photography, and cloud-based storage.

Results

Important considerations for current image capture include a device with a gridded viewing screen and high megapixel resolution, a tripod with leveling base, studio lighting with dual-sourced light, standardized matte finish background, and consistency in patient orientation. Currently, 3D and 4D photography devices offer advantages such as improved communication to the patient on outcome expectation and better quality of patient service and safety. AI may contribute to post-capture processing and 3D printing of postoperative outcomes. Current smartphones distort patient perceptions about their appearance and should be used cautiously in an aesthetic surgery setting. Cloud-based storage provides flexibility, cost, and ease of service while remaining vulnerable to data breaches.

Conclusions

While there are advancements to be made in the physical equipment and preparation for the photograph, the future of clinical photography will be heavily influenced by innovations in software and 3D and 4D modeling of outcomes.

Plastic Surgery Lockdown Learning during Coronavirus Disease 2019: Are Adaptations in Education Here to Stay?

Summary: The novel coronavirus disease 2019 has had a major impact on human life and livelihood. The unprecedented challenges have expanded beyond just social and work life, and have grown to impact resident education. In this article, we review the structure of plastic surgery education before the pandemic, the different online learning opportunities for self-directed learning. A summary of the range of platforms and approaches of online remote access delivery of conferences and education that emerged or expanded as a result of the crisis has been reported. This article highlighted the rapid initiatives and efforts of programs and national and international societies to support continuing medical education in conjunction with the guidelines to “shelter at home” and maintain social distancing, and possible future for expanding the reach of online academic initiatives, in addition to the role of developing virtual technologies. The coronavirus disease 2019 crisis has created an opportunity to analyze and advance online learning options to overcome the associated challenges and continue as a reliable platform even following the resolution of the social distancing requirements.

Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking

Objective

Surgical navigation is a well-established tool in endoscopic skull base surgery. However, navigational and endoscopic views are usually displayed on separate monitors, forcing the surgeon to focus on one or the other. Aiming to provide real-time integration of endoscopic and diagnostic imaging information, we present a new navigation technique based on augmented reality with fusion of intraoperative cone beam computed tomography (CBCT) on the endoscopic view. The aim of this study was to evaluate the accuracy of the method.

Material and methods

An augmented reality surgical navigation system (ARSN) with 3D CBCT capability was used. The navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. Intra-operative CBCT images were fused with the view of the surgical field obtained by the endoscope’s camera. Accuracy of CBCT image co-registration was tested using a custom-made grid with incorporated 3D spheres.

Results

Co-registration of the CBCT image on the endoscopic view was performed. Accuracy of the overlay, measured as mean target registration error (TRE), was 0.55 mm with a standard deviation of 0.24 mm and with a median value of 0.51mm and interquartile range of 0.39˗˗0.68 mm.

Conclusion

We present a novel augmented reality surgical navigation system, with fusion of intraoperative CBCT on the endoscopic view. The system shows sub-millimeter accuracy.

Wearable Augmented Reality Application for Shoulder Rehabilitation

Augmented reality (AR) technology is gaining popularity and scholarly interest in the rehabilitation sector because of the possibility to generate controlled, user-specific environmental and perceptual stimuli which motivate the patient, while still preserving the possibility to interact with the real environment and other subjects, including the rehabilitation specialist. The paper presents the first wearable AR application for shoulder rehabilitation, based on Microsoft HoloLens, with real-time markerless tracking of the user’s hand. Potentialities and current limits of commercial head-mounted displays (HMDs) are described for the target medical field, and details of the proposed application are reported. A serious game was designed starting from the analysis of a traditional rehabilitation exercise, taking into account HoloLens specifications to maximize user comfort during the AR rehabilitation session. The AR application implemented consistently meets the recommended target frame rate for immersive applications with HoloLens device: 60 fps. Moreover, the ergonomics and the motivational value of the proposed application were positively evaluated by a group of five rehabilitation specialists and 20 healthy subjects. Even if a larger study, including real patients, is necessary for a clinical validation of the proposed application, the results obtained encourage further investigations and the integration of additional technical features for the proposed AR application.