Supporting mandibular resection with intraoperative navigation utilizing augmented reality technology – A proof of concept study

A cooperatively controlled robotic system with active constraints for enhancing efficacy in bilateral sagittal split osteotomy

PURPOSE

Precise osteotomies are vital in maxillofacial procedures such as the bilateral sagittal split osteotomy (BSSO) where surgical accuracy and precision directly impacts patient outcomes. Conventional freehand drilling can lead to unfavorable splits, negatively impacting surgical outcome.

METHODS

This paper presents the development work of a cooperatively controlled robot system designed to enhance the efficacy of osteotomies during BSSO. The system features two assistive modes for the execution of a patient-specific surgical plan: (1) a Haptic guidance mode that helps the surgeon align the surgical drill with the planned cutting plane to improve surgical accuracy of the cut and (2) an Active constraint mode that restricts deviations from the cutting plane to enhance surgical precision during drilling. We validated the system through feasibility experiments involving 36 mandible phantoms and a cadaveric specimen, with a surgeon, a surgical resident, and a medical student performing osteotomies freehand and with robotic assistance. Additionally, NASA TLX surveys were conducted to assess the perceived ease of use of the robotic system.

RESULTS

Compared to freehand methods, the robotic system improved the efficacy of the cut from 2.16 ± 0.98 to 0.71 ± 0.53  mm for the med student, 1.74 ± 0.95 to 0.53 ± 0.35  mm for the resident, and 1.64 ± 0.85 to 0.63 ± 0.24  mm for the surgeon while reducing the task load.

CONCLUSION

Our experimental results demonstrate that the proposed robotic system can enhance the precision of surgical drilling in the BSSO compared to a freehand approach. These findings indicate the potential of robotic systems to reduce errors and enhance patient outcomes in maxillofacial surgery.

Augmented Reality Navigation in Craniomaxillofacial/Head and Neck Surgery

Objective

This study aims to (1) develop an augmented reality (AR) navigation platform for craniomaxillofacial (CMF) and head and neck surgery; (2) apply it to a range of surgical cases; and (3) evaluate the advantages, disadvantages, and clinical opportunities for AR navigation.

Study Design

A multi-center retrospective case series.

Setting

Four tertiary care academic centers.

Methods

A novel AR navigation platform was collaboratively developed with Xironetic and deployed intraoperatively using only a head-mounted display (Microsoft HoloLens 2). Virtual surgical plans were generated from computed tomography/magnetic resonance imaging data and uploaded onto the AR platform. A reference array was mounted to the patient, and the virtual plan was registered to the patient intraoperatively. A retrospective review of all AR-navigated CMF cases since September 2023 was performed.

Results

Thirty-three cases were reviewed and classified as either trauma, orthognathic, tumor, or craniofacial. The AR platform had several advantages over traditional navigation including real-time 3D visualization of the surgical plan, identification of critical structures, and real-time tracking. Furthermore, this case series presents the first-known examples of (1) AR instrument tracking for midface osteotomies, (2) AR tracking of the zygomaticomaxillary complex during fracture reduction, (3) mandibular tracking in orthognathic surgery, (4) AR fibula cutting guides for mandibular reconstruction, and (5) integration of real-time infrared visualization in an AR headset for vasculature identification.

Conclusion

While still a developing technology, AR navigation provides several advantages over traditional navigation for CMF and head and neck surgery, including heads up, interactive 3D visualization of the surgical plan, identification of critical anatomy, and real-time tracking.

Intraoperative Real-Time Image-Guided Fibular Harvest and Mandibular Reconstruction: A Feasibility Study on Cadaveric Specimens

BACKGROUND

This study assesses the feasibility of real-time surgical navigation to plan and guide sequential steps during mandible reconstruction on a series of cadaveric specimens.

METHODS

An image-guided surgical (IGS) system was designed including customized mandible and fibula fixation devices with navigation reference frames and an accompanied image-guided software. The mandibular and fibular segmental osteotomies were performed using the IGS in all five cadaveric patients. Procedural time and cephalometric measurements were recorded.

RESULTS

Five real-time IGS mandibulectomy and fibular reconstruction were successfully performed. The mean Dice score and Hausdorff-95 distance between the planned and actual mandible reconstructions was 0.8 ± 0.08 and 7.29 ± 4.81 mm, respectively. Intercoronoid width, interangle width, and mandible projection differences were 1.15 ± 1.17 mm, 0.9 ± 0.56 mm, and 1.47 ± 1.62 mm, respectively.

CONCLUSION

This study presents the first demonstration of a comprehensive image-guided workflow for mandibulectomy and fibular flap reconstruction on cadaveric specimens and resulted in adequate cephalometric accuracy.

Is Overlain Display a Right Choice for AR Navigation? A Qualitative Study of Head-Mounted Augmented Reality Surgical Navigation on Accuracy for Large-Scale Clinical Deployment

BACKGROUND

During the course of the past two decades, head-mounted augmented reality surgical navigation (HMARSN) systems have been increasingly employed in a variety of surgical specialties as a result of both advancements in augmented reality-related technologies and surgeons' desires to overcome some drawbacks inherent to conventional surgical navigation systems. In the present time, most experimental HMARSN systems adopt overlain display (OD) that overlay virtual models and planned routes of surgical tools on corresponding physical tissues, organs, lesions, and so forth, in a surgical field so as to provide surgeons with an intuitive and direct view to gain better hand-eye coordination as well as avoid attention shift and loss of sight (LOS), among other benefits during procedures. Yet, its system accuracy, which is the most crucial performance indicator of any surgical navigation system, is difficult to ascertain because it is highly subjective and user-dependent. Therefore, the aim of this study was to review presently available experimental OD HMARSN systems qualitatively, explore how their system accuracy is affected by overlain display, and find out if such systems are suited to large-scale clinical deployment.

METHOD

We searched PubMed and ScienceDirect with the following terms: head mounted augmented reality surgical navigation, and 445 records were returned in total. After screening and eligibility assessment, 60 papers were finally analyzed. Specifically, we focused on how their accuracies were defined and measured, as well as whether such accuracies are stable in clinical practice and competitive with corresponding commercially available systems.

RESULTS AND CONCLUSIONS

The primary findings are that the system accuracy of OD HMARSN systems is seriously affected by a transformation between the spaces of the user's eyes and the surgical field, because measurement of the transformation is heavily individualized and user-dependent. Additionally, the transformation itself is potentially subject to changes during surgical procedures, and hence unstable. Therefore, OD HMARSN systems are not suitable for large-scale clinical deployment.

From Virtual Reality to Reality: Fine-Tuning the Taxonomy for Extended Reality Simulation in Dental Education

INTRODUCTION

Digital simulation in dental education has substantially evolved, addressing several educational challenges in dentistry. Following global lockdowns and sustainability concerns, dental educators are increasingly adopting digital simulation to enhance or replace traditional training methods. This review aimed to contribute to a uniform taxonomy for extended reality (XR) simulation within dental education.

METHODS

This scoping review followed the PRISMA and PRISMA-ScR guidelines. PubMed/MEDLINE, EMBASE, Web of Science and Google Scholar were searched. Eligible studies included English-written publications in indexed journals related to digital simulation in dental/maxillofacial education, providing theoretical descriptions of extended reality (XR) and/or immersive training tools (ITT). The outcomes of the scoping review were used as building blocks for a uniform of XR-simulation taxonomy.

RESULTS

A total of 141 articles from 2004 to 2024 were selected and categorised into Virtual Reality (VR), Mixed Reality (MR), Augmented Reality (AR), Augmented Virtuality (AV) and Computer Simulation (CS). Stereoscopic vision, immersion, interaction, modification and haptic feedback were identified as recurring features across XR-simulation in dentistry. These features formed the basis for a general XR-simulation taxonomy.

DISCUSSION

While XR-simulation features were consistent in the literature, the variety of definitions and classifications complicated the development of a taxonomy framework. VR was frequently used as an umbrella term. To address this, operational definitions were proposed for each category within the virtuality continuum, clarifying distinctions and commonalities.

CONCLUSION

This scoping review highlights the need for a uniform taxonomy in XR simulation within dental education. Establishing a consensus on XR-related terminology and definitions facilitates future research, allowing clear evidence reporting and analysis. The proposed taxonomy may also be of use for medical education, promoting alignment and the creation of a comprehensive body of evidence in XR technologies.

Co-Designing Dynamic Mixed Reality Drill Positioning Widgets: A Collaborative Approach with Dentists in a Realistic Setup

Mixed Reality (MR) is proven in the literature to support precise spatial dental drill positioning by superimposing 3D widgets. Despite this, the related knowledge about widget's visual design and interactive user feedback is still limited. Therefore, this study is contributed to by co-designed MR drill tool positioning widgets with two expert dentists and three MR experts. The results of co-design are two static widgets (SWs): a simple entry point, a target axis, and two dynamic widgets (DWs), variants of dynamic error visualization with and without a target axis (DWTA and DWEP). We evaluated the co-designed widgets in a virtual reality simulation supported by a realistic setup with a tracked phantom patient, a virtual magnifying loupe, and a dentist's foot pedal. The user study involved 35 dentists with various backgrounds and years of experience. The findings demonstrated significant results; DWs outperform SWs in positional and rotational precision, especially with younger generations and subjects with gaming experiences. The user preference remains for DWs (19) instead of SWs (16). However, findings indicated that the precision positively correlates with the time trade-off. The post-experience questionnaire (NASA-TLX) showed that DWs increase mental and physical demand, effort, and frustration more than SWs. Comparisons between DWEP and DWTA show that the DW's complexity level influences time, physical and mental demands. The DWs are extensible to diverse medical and industrial scenarios that demand precision.

Accuracy of the modified tooth-supported 3D printing surgical guides based on CT, CBCT, and intraoral scanning in maxillofacial region: A comparison study

BACKGROUND

Tooth-supported surgical guides have demonstrated superior accuracy compared with bone-supported guides. This study aimed to modify the fabrication of tooth-supported guides for compatibility with tumor resection procedures and investigate their accuracy.

METHODS

Patients with tumors who underwent osteotomy with the assistance of modified tooth- or bone-supported surgical guides were included. Virtual surgical planning (VSP) was employed to align three dimensional (3D) models extracted from intraoperative computed tomography (CT) images. The distances and angular deviations between the actual osteotomy plane and preoperative plane were recorded. A comparative analysis of osteotomy discrepancies between tooth-supported and bone-supported guides, as well as among tooth-supported guides based on CT, cone-beam CT (CBCT), or intraoral scanner (IOS) was conducted. The factors influencing the precision of the guides were analyzed.

RESULTS

Sixty patients with 81 resection planes were included in this study. In the tooth-supported group, the mean deviations in the osteotomy plane and angle were 1.39 mm and 4.30°, respectively, whereas those of the bone-supported group were 2.16 mm and 4.95°. In the tooth-supported isotype guide groups, the mean deviations of the osteotomy plane were 1.39 mm, 1.47 mm, 1.23 mm across CT, CBCT, and IOS, respectively. The accuracy of the modified tooth-supported guides remained consistent regardless of number and position of the teeth supporting the guide and location of the osteotomy lines.

CONCLUSIONS

The findings indicate that the modified tooth-supported surgical guides demonstrated high accuracy in the maxillofacial region, contributing to a reduction in the amount of surgically detached soft tissue.

Taxonomic discordance of immersive realities in dentistry: A systematic scoping review

OBJECTIVES

This review aimed to map taxonomy frameworks, descriptions, and applications of immersive technologies in the dental literature.

DATA

The Preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) guidelines was followed, and the protocol was registered at open science framework platform (https://doi.org/10.17605/OSF.IO/H6N8M).

SOURCES

Systematic search was conducted in MEDLINE (via PubMed), Scopus, and Cochrane Library databases, and complemented by manual search.

STUDY SELECTION

A total of 84 articles were included, with 81 % between 2019 and 2023. Most studies were experimental (62 %), including education (25 %), protocol feasibility (20 %), in vitro (11 %), and cadaver (6 %). Other study types included clinical report/technique article (24 %), clinical study (9 %), technical note/tip to reader (4 %), and randomized controlled trial (1 %). Three-quarters of the included studies were published in oral and maxillofacial surgery (38 %), dental education (26 %), and implant (12 %) disciplines. Methods of display included head mounted display device (HMD) (55 %), see through screen (32 %), 2D screen display (11 %), and projector display (2 %). Descriptions of immersive realities were fragmented and inconsistent with lack of clear taxonomy framework for the umbrella and the subset terms including virtual reality (VR), augmented reality (AR), mixed reality (MR), augmented virtuality (AV), extended reality, and X reality.

CONCLUSIONS

Immersive reality applications in dentistry are gaining popularity with a notable surge in the number of publications in the last 5 years. Ambiguities are apparent in the descriptions of immersive realities. A taxonomy framework based on method of display (full or partial) and reality class (VR, AR, or MR) is proposed.

CLINICAL SIGNIFICANCE

Understanding different reality classes can be perplexing due to their blurred boundaries and conceptual overlapping. Immersive technologies offer novel educational and clinical applications. This domain is fast developing. With the current fragmented and inconsistent terminologies, a comprehensive taxonomy framework is necessary.

Mixed Reality Combined with Surgical Navigation in Resection of Micro- and Mini-Tumors of the Parotid Gland: A Pilot Study

OBJECTIVE

This study aimed to evaluate the feasibility and outcomes of mixed reality combined with surgical navigation (MRSN) in the resection of parotid micro- and mini-tumors.

METHODS

Eighteen patients who underwent parotid tumor resection between December 2020 and November 2022 were included. Six patients were enrolled in MRSN group, and the surgeons performed the surgery with the help of MRSN technology. The surgical procedures include virtual planning, data transfer between mixed reality and surgical navigation, tumor localization and resection assisted by surgical navigation under mixed reality environment. Twelve patients were enrolled in control group, and intraoperative tumor localization and resection were performed according to the experience of the surgeon. Total surgery time and intraoperative bleeding were recorded. Perioperative complications were recorded during follow-up.

RESULTS

The mean surgery time of MRSN group (76.7 ± 14.0 min) and control group (65.4 ± 21.3 min) showed no significant difference (p = 0.220), so did the intraoperative bleeding of MRSN group (16.0 ± 8.0 mL) and control group (16.7 ± 6.6 mL) (p = 0.825). None of the patient in MRSN group underwent any complication, although one patient in control group suffered temporary facial paralysis. The mean deviation between the virtually marked and the intraoperative actual outermost point of tumor was 3.03 ± 0.83 mm.

CONCLUSION

MRSN technology can realize real-time three-dimensional visualization of the tumor, and it has the potential of enhancing the safety and accuracy of resection of micro- and mini-tumors of parotid gland.

LEVEL OF EVIDENCE

4 Laryngoscope, 134:1670-1678, 2024.

Indocyanine green and iohexol loaded hydroxyapatite in poly(L-lactide-co-caprolactone)-based composite for bimodal near-infrared fluorescence- and X-ray-based imaging

This study aimed to develop material for multimodal imaging by means of X-ray and near-infrared containing FDA- and EMA-approved iohexol and indocyanine green (ICG). The mentioned contrast agents (CAs) are hydrophilic and amphiphilic, respectively, which creates difficulties in fabrication of functional polymeric composites for fiducial markers (FMs) with usage thereof. Therefore, this study exploited for the first time the possibility of enhancing the radiopacity and introduction of the NIR fluorescence of FMs by adsorption of the CAs on hydroxyapatite (HAp) nanoparticles. The particles were embedded in the poly(L-lactide-co-caprolactone) (P[LAcoCL]) matrix resulting in the composite material for bimodal near-infrared fluorescence- and X-ray-based imaging. The applied method of material preparation provided homogenous distribution of both CAs with high iohexol loading efficiency and improved fluorescence signal due to hindered ICG aggregation. The material possessed profound contrasting properties for both imaging modalities. Its stability was evaluated during in vitro experiments in phosphate-buffered saline (PBS) and foetal bovine serum (FBS) solutions. The addition of HAp nanoparticles had significant effect on the fluorescence signal. The X-ray radiopacity was stable within minimum 11 weeks, even though the addition of ICG contributed to a faster release of iohexol. The stiffness of the material was not affected by iohexol or ICG, but incorporation of HAp nanoparticles elevated the values of bending modulus by approximately 70%. Moreover, the performed cell study revealed that all tested materials were not cytotoxic. Thus, the developed material can be successfully used for fabrication of FMs.

Application experience and research progress of different emerging technologies in plastic surgery

In the diagnosis and treatment of plastic surgery, there are structural processing problems, such as positioning, moving, and reconstructing complex three-dimensional structures. Doctors operate according to their own experience, and the inability to accurately locate these structures is an important problem in plastic surgery. Emerging digital technologies such as virtual reality, augmented reality, and three-dimensional printing are widely used in the medical field, particularly in plastic surgery. This article reviews the development of these three technical concepts, introduces the technical elements and specific applications required in plastic surgery, summarizes the application status of the three technologies in plastic surgery, and summarizes prospects for future development.

Mandibular resection and defect reconstruction guided by a contour registration-based augmented reality system: A preclinical trial

The aim of this study was to verify the feasibility and accuracy of a contour registration-based augmented reality (AR) system in jaw surgery. An AR system was developed to display the interaction between virtual planning and images of the surgical site in real time. Several trials were performed with the guidance of the AR system and the surgical guide. The postoperative cone beam CT (CBCT) data were matched with the preoperatively planned data to evaluate the accuracy of the system by comparing the deviations in distance and angle. All procedures were performed successfully. In nine model trials, distance and angular deviations for the mandible, reconstructed fibula, and fixation screws were 1.62 ± 0.38 mm, 1.86 ± 0.43 mm, 1.67 ± 0.70 mm, and 3.68 ± 0.71°, 5.48 ± 2.06°, 7.50 ± 1.39°, respectively. In twelve animal trials, results of the AR system were compared with the surgical guide. Distance deviations for the bilateral condylar outer poles were 0.93 ± 0.63 mm and 0.81 ± 0.30 mm, respectively (p = 0.68). Distance deviations for the bilateral mandibular posterior angles were 2.01 ± 2.49 mm and 2.89 ± 1.83 mm, respectively (p = 0.50). Distance and angular deviations for the mandible were 1.41 ± 0.61 mm, 1.21 ± 0.18 mm (p = 0.45), and 6.81 ± 2.21°, 6.11 ± 2.93° (p = 0.65), respectively. Distance and angular deviations for the reconstructed tibiofibular bones were 0.88 ± 0.22 mm, 0.84 ± 0.18 mm (p = 0.70), and 6.47 ± 3.03°, 6.90 ± 4.01° (p = 0.84), respectively. This study proposed a contour registration-based AR system to assist surgeons in intuitively observing the surgical plan intraoperatively. The trial results indicated that this system had similar accuracy to the surgical guide.

Mandible Angle Resection with the Retroauricular Approach

Square-shaped and large moon-shaped faces are commonly observed in Asians, and the contour of the mandible is associated with the shape of the lower part of the face. Mandible contouring surgery is performed to create a softer impression for East Asians. Currently, most surgeries are performed using an intraoral approach. External approaches have not been cosmetically attempted because of possible damage to the facial nerve and visible scarring and have been limited to mandible bone fracture reduction. This study included 42 patients who underwent mandibular angle reduction via classical intraoral incision and retroauricular incision between April 2019 and October 2021. Clinical outcomes were assessed using the Global Aesthetic Improvement Scale and Visual Analog Scale. Surgery was successful in all cases, with no significant complications. An appropriate mandibular contour was achieved postoperatively. All patients were satisfied with the outcome. Some patients experienced short-term complications, such as hematoma and wound disruption of the skin above the incision line. However, these improved within 3 weeks, and no serious long-term complications were observed. Mandible angle resection with the retroauricular approach is a promising alternative for patients, allowing speedy recovery and the resumption of routine daily life.

The impact of extended reality on surgery: a scoping review

PURPOSE

Extended reality (XR) is defined as a spectrum of technologies that range from purely virtual environments to enhanced real-world environments. In the past two decades, XR-assisted surgery has seen an increase in its use and also in research and development. This scoping review aims to map out the historical trends in these technologies and their future prospects, with an emphasis on the reported outcomes and ethical considerations on the use of these technologies.

METHODS

A systematic search of PubMed, Scopus, and Embase for literature related to XR-assisted surgery and telesurgery was performed using Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) guidelines. Primary studies, peer-reviewed articles that described procedures performed by surgeons on human subjects and cadavers, as well as studies describing general surgical education, were included. Non-surgical procedures, bedside procedures, veterinary procedures, procedures performed by medical students, and review articles were excluded. Studies were classified into the following categories: impact on surgery (pre-operative planning and intra-operative navigation/guidance), impact on the patient (pain and anxiety), and impact on the surgeon (surgical training and surgeon confidence).

RESULTS

One hundred and sixty-eight studies were included for analysis. Thirty-one studies investigated the use of XR for pre-operative planning concluded that virtual reality (VR) enhanced the surgeon's spatial awareness of important anatomical landmarks. This leads to shorter operating sessions and decreases surgical insult. Forty-nine studies explored the use of XR for intra-operative planning. They noted that augmented reality (AR) headsets highlight key landmarks, as well as important structures to avoid, which lowers the chance of accidental surgical trauma. Eleven studies investigated patients' pain and noted that VR is able to generate a meditative state. This is beneficial for patients, as it reduces the need for analgesics. Ten studies commented on patient anxiety, suggesting that VR is unsuccessful at altering patients' physiological parameters such as mean arterial blood pressure or cortisol levels. Sixty studies investigated surgical training whilst seven studies suggested that the use of XR-assisted technology increased surgeon confidence.

CONCLUSION

The growth of XR-assisted surgery is driven by advances in hardware and software. Whilst augmented virtuality and mixed reality are underexplored, the use of VR is growing especially in the fields of surgical training and pre-operative planning. Real-time intra-operative guidance is key for surgical precision, which is being supplemented with AR technology. XR-assisted surgery is likely to undertake a greater role in the near future, given the effect of COVID-19 limiting physical presence and the increasing complexity of surgical procedures.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

"Image to patient" equal-resolution surface registration supported by a surface scanner: analysis of algorithm efficiency for computer-aided surgery

PURPOSE

The "image to patient" registration procedure is crucial for the accuracy of surgical instrument tracking relative to the medical image while computer-aided surgery. The main aim of this work was to create an equal-resolution surface registration algorithm (ERSR) and analyze its efficiency.

METHODS

The ERSR algorithm provides two datasets with equal, high resolution and approximately corresponding points. The registered sets are obtained by projection of a user-designed rectangle(s)-shaped uniform clouds of points on DICOM and surface scanner datasets. The tests of the algorithm were performed on a phantom with titanium microscrews. We analyzed the influence of DICOM resolution on the effect of the ERSR algorithm and compared the ERSR to standard paired-points landmark transform registration. The methods of analysis were Target Registration Error, distance maps, and their histogram evaluation.

RESULTS

The mean TRE in case of ERSR equaled 0.8 ± 0.3 mm (resolution A), 0.8 ± 0.5 mm (resolution B), and 1.0 ± 0.7 mm (resolution C). The mean values were at least 0.4 mm lower than in the case of landmark transform registration. The distance maps between the model achieved from the scanner and the CT-based model were analyzed by histogram. The frequency of the first bin in a histogram of the distance map for ERSR was about 0.6 for all three resolutions of DICOM dataset and three times higher than in the case of landmark transform registration. The results were statistically analyzed using the Wilcoxon signed-rank test (alpha = 0.05).

CONCLUSION

The tests proved a statistically significant higher efficiency of equal resolution surface registration related to the landmark transform algorithm. It was proven that the lower resolution of the CT DICOM dataset did not degrade the efficiency of the ERSR algorithm. We observed a significantly lower response to decreased resolution than in the case of paired-points landmark transform registration.

Current Trends in the Reconstruction and Rehabilitation of Jaw following Ablative Surgery

Simple Summary

The Maxilla and mandible provide skeletal support for of the middle and lower third of our faces, allowing for the normal functioning of breathing, chewing, swallowing, and speech. The ablative surgery of jaws in the past often led to serious disfigurement and disruption in form and function. However, with recent strides made in computer-assisted surgery and patient-specific implants, the individual functional reconstruction of the jaw is evolving rapidly and the prompt rehabilitation of both the masticatory function and aesthetics after jaw resection has been made possible. In the present review, the recent advancements in jaw reconstruction technology and future perspectives will be discussed.

Abstract

The reconstruction and rehabilitation of jaws following ablative surgery have been transformed in recent years by the development of computer-assisted surgery and virtual surgical planning. In this narrative literature review, we aim to discuss the current state-of-the-art jaw reconstruction, and to preview the potential future developments. The application of patient-specific implants and the “jaw-in-a-day technique” have made the fast restoration of jaws’ function and aesthetics possible. The improved efficiency of primary reconstructive surgery allows for the rehabilitation of neurosensory function following ablative surgery. Currently, a great deal of research has been conducted on augmented/mixed reality, artificial intelligence, virtual surgical planning for soft tissue reconstruction, and the rehabilitation of the stomatognathic system. This will lead to an even more exciting future for the functional reconstruction and rehabilitation of the jaw following ablative surgery.

Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review

This article presents a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality (AR) surgery applications from 2013 to 2020. Articles were categorised by: OST-HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. 91 articles fulfilled all inclusion criteria. Some clear trends emerge. The Microsoft HoloLens increasingly dominates the field, with orthopaedic surgery being the most popular application (28.6%). By far the most common surgical context is surgical guidance (n=58) and segmented preoperative models dominate visualisation (n=40). Experiments mainly involve phantoms (n=43) or system setup (n=21), with patient case studies ranking third (n=19), reflecting the comparative infancy of the field. Experiments cover issues from registration to perception with very different accuracy results. Human factors emerge as significant to OST-HMD utility. Some factors are addressed by the systems proposed, such as attention shift away from the surgical site and mental mapping of 2D images to 3D patient anatomy. Other persistent human factors remain or are caused by OST-HMD solutions, including ease of use, comfort and spatial perception issues. The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed.

Accuracy of Mixed Reality Combined With Surgical Navigation Assisted Oral and Maxillofacial Tumor Resection

OBJECTIVE

To evaluate the feasibility and accuracy of mixed reality combined with surgical navigation in oral and maxillofacial tumor surgery.

METHODS

Retrospective analysis of data of seven patients with oral and maxillofacial tumors who underwent surgery between January 2019 and January 2021 using a combination of mixed reality and surgical navigation. Virtual surgical planning and navigation plan were based on preoperative CT datasets. Through IGT-Link port, mixed reality workstation was synchronized with surgical navigation, and surgical planning data were transferred to the mixed reality workstation. Osteotomy lines were marked with the aid of both surgical navigation and mixed reality images visualized through HoloLens. Frozen section examination was used to ensure negative surgical margins. Postoperative CT datasets were obtained 1 week after the surgery, and chromatographic analysis of virtual osteotomies and actual osteotomies was carried out. Patients received standard oncological postoperative follow-up.

RESULTS

Of the seven patients, four had maxillary tumors and three had mandibular tumors. There were total of 13 osteotomy planes. Mean deviation between the planned osteotomy plane and the actual osteotomy plane was 1.68 ± 0.92 mm; the maximum deviation was 3.46 mm. Chromatographic analysis showed error of ≤3 mm for 80.16% of the points. Mean deviations of maxillary and mandibular osteotomy lines were approximate (1.60 ± 0.93 mm vs. 1.86 ± 0.93 mm). While five patients had benign tumors, two had malignant tumors. Mean deviations of osteotomy lines was comparable between patients with benign and malignant tumors (1.48 ± 0.74 mm vs. 2.18 ± 0.77 mm). Intraoperative frozen pathology confirmed negative resection margins in all cases. No tumor recurrence or complications occurred during mean follow-up of 15.7 months (range, 6-26 months).

CONCLUSION

The combination of mixed reality technology and surgical navigation appears to be feasible, safe, and effective for tumor resection in the oral and maxillofacial region.

Biodegradable Fiducial Markers for Bimodal Near-Infrared Fluorescence- and X-ray-Based Imaging

This study aimed to evaluate, for the first time, implantable, biodegradable fiducial markers (FMs), which were designed for bimodal, near-infrared fluorescence-based (NIRF) and X-ray-based imaging. The developed FMs had poly(l-lactide-co-caprolactone)-based core-shell structures made of radiopaque (core) and fluorescent (shell) composites with a poly(l-lactide-co-caprolactone) matrix. The approved for human use contrast agents were utilized as fillers. Indocyanine green was applied to the shell material, whereas in the core materials, iohexol and barium sulfate were compared. Moreover, the possibility of tailoring the stability of the properties of the core materials by the addition of hydroxyapatite (HAp) was examined. The performed in situ (porcine tissue) and in vivo experiment (rat model) confirmed that the developed FMs possessed pronounced contrasting properties in NIRF and X-ray imaging. The presence of HAp improved the radiopacity of FMs at the initial state. It was also proved that, in iohexol-containing FMs, the presence of HAp slightly decreased the stability of contrasting properties, while in BaSO₄-containing ones, changes were less pronounced. A comprehensive material analysis explaining the differences in the stability of the contrasting properties was also presented. The tissue response around the FMs with composite cores was comparable to that of the FMs with a pristine polymeric core. The developed composite FMs did not cause serious adverse effects on the surrounding tissues even when irradiated in vivo. The developed FMs ensured good visibility for NIRF image-supported tumor surgery and the following X-ray image-guided radiotherapy. Moreover, this study replenishes a scanty report regarding similar biodegradable composite materials with a high potential for application.

Comparison of augmented reality and cutting guide technology in assisted harvesting of iliac crest grafts - A cadaver study

BACKGROUND

Harvesting vascularized bone grafts with computer-assisted surgery represents the gold standard for mandibular reconstruction. However, current augmented reality (AR) approaches are limited to invasive marker fixation. This trial compared a markerless AR-guided real-time navigation with virtually planned and 3D printed cutting guides for harvesting iliac crest grafts.

MATERIAL AND METHODS

Two commonly used iliac crest transplant configurations were virtually planned on 10 cadaver hips. Transplant harvest was performed with AR guidance and cutting guide technology. The harvested transplants were digitalized using cone beam CT. Deviations of angulation, distance and volume between the executed and planned osteotomies were measured.

RESULTS

Both AR and cutting guides accurately rendered the virtually planned transplant volume. However, the cumulative osteotomy plane angulation differed significantly (p = 0.018) between AR (14.99 ± 11.69°) and the cutting guides (8.49 ± 5.42°). The cumulative osteotomy plane distance showed that AR-guided navigation had lower accuracy (2.65 ± 3.32 mm) than the cutting guides (1.47 ± 1.36 mm), although without significant difference.

CONCLUSION

This study demonstrated the clinical usability of markerless AR-guided navigation for harvesting iliac crest grafts. Further improvement of accuracy rates might bring clinical implementation closer to reality.

Navigation of iliac crest graft harvest using markerless augmented reality and cutting guide technology: A pilot study

BACKGROUND

Defects of the facial skeleton often require complex reconstruction with vascularized grafts. This trial elucidated the usability, visual perception and accuracy of a markerless augmented reality (AR)-guided navigation for harvesting iliac crest transplants.

METHODS

Random CT scans were used to virtually plan two common transplant configurations on 10 iliac crest models, each printed four times. The transplants were harvested using projected AR and cutting guides. The duration and accuracies of the angulation, distance and volume between the planned and executed osteotomies were measured.

RESULTS

AR was characterized by the efficient use of time and accurate rendition of preoperatively planned geometries. However, vertical osteotomies and complex anatomical settings displayed significant inferiority of AR guidance compared to cutting guides.

CONCLUSIONS

This study demonstrated the usability of a markerless AR setup for harvesting iliac crest transplants. The visual perception and accuracy of the AR-guided osteotomies constituted remaining weaknesses against cutting guide technology.

Augmented reality in craniomaxillofacial surgery: added value and proposed recommendations through a systematic review of the literature

This systematic review provides an overview of augmented reality (AR) and its benefits in craniomaxillofacial surgery in an attempt to answer the question: Is AR beneficial for craniomaxillofacial surgery? This review includes a description of the studies conducted, the systems used and their technical characteristics. The search was performed in four databases: PubMed, Cochrane Library, Embase, and Web of Science. All journal articles published during the past 11 years related to AR, mixed reality, craniomaxillofacial, and surgery were considered in this study. From a total of 7067 articles identified using AR- and surgery-related keywords, 39 articles were finally selected. Based on these articles, a classification of study types, surgery types, devices used, metrics reported, and benefits were collected. The findings of this review indicate that AR could provide various benefits, addressing the challenges of conventional navigation systems, such as hand-eye coordination and depth perception. However, three main concerns were raised while performing this study: (1) it is complicated to aggregate the metrics reported in the articles, (2) it is difficult to obtain statistical value from the current studies, and (3) user evaluation studies are lacking. This article concludes with recommendations for future studies by addressing the latter points.

Artificial intelligence-enabled automatic segmentation of skull CT facilitates computer-assisted craniomaxillofacial surgery

BACKGROUND

The image segmentation of skull CT is the cornerstone for the computer-assisted craniomaxillofacial surgery in multiple aspects. This study aims to introduce an AI-enabled automatic segmentation and propose its prospect in facilitating the computer-assisted surgery.

METHODS

Three patients enrolled in a clinical trial of computer-assisted craniomaxillofacial surgery were randomly selected for this study. The preoperative helical CT scans of the head and neck region were subjected to the AI-enabled automatic segmentation in Mimics Viewer. The performance of AI segmentation was evaluated based on the requirements of computer-assisted surgery.

RESULTS

All three patients were successfully segmented by the AI-enabled automatic segmentation. The performance of AI segmentation was excellent regarding key anatomical structures. The overall quality of bone surface was satisfying. The median DICE coefficient was 92.4% for the maxilla, and 94.9% for the mandible, which fulfilled the requirements of computer-assisted craniomaxillofacial surgery.

CONCLUSIONS

The AI-enabled automatic segmentation could facilitate the preoperative virtual planning and postoperative outcome verification, which formed a feedback loop to enhance the current workflow of computer-assisted surgery. More studies are warranted to confirm the robustness of AI segmentation with more cases.

Augmented Reality Visualization for Image-Guided Surgery: A Validation Study Using a Three-Dimensional Printed Phantom

BACKGROUND

Oral and maxillofacial surgery currently relies on virtual surgery planning based on image data (CT, MRI). Three-dimensional (3D) visualizations are typically used to plan and predict the outcome of complex surgical procedures. To translate the virtual surgical plan to the operating room, it is either converted into physical 3D-printed guides or directly translated using real-time navigation systems.

PURPOSE

This study aims to improve the translation of the virtual surgery plan to a surgical procedure, such as oncologic or trauma surgery, in terms of accuracy and speed. Here we report an augmented reality visualization technique for image-guided surgery. It describes how surgeons can visualize and interact with the virtual surgery plan and navigation data while in the operating room. The user friendliness and usability is objectified by a formal user study that compared our augmented reality assisted technique to the gold standard setup of a perioperative navigation system (Brainlab). Moreover, accuracy of typical navigation tasks as reaching landmarks and following trajectories is compared.

RESULTS

Overall completion time of navigation tasks was 1.71 times faster using augmented reality (P = .034). Accuracy improved significantly using augmented reality (P < .001), for reaching physical landmarks a less strong correlation was found (P = .087). Although the participants were relatively unfamiliar with VR/AR (rated 2.25/5) and gesture-based interaction (rated 2/5), they reported that navigation tasks become easier to perform using augmented reality (difficulty Brainlab rated 3.25/5, HoloLens 2.4/5).

CONCLUSION

The proposed workflow can be used in a wide range of image-guided surgery procedures as an addition to existing verified image guidance systems. Results of this user study imply that our technique enables typical navigation tasks to be performed faster and more accurately compared to the current gold standard. In addition, qualitative feedback on our augmented reality assisted technique was more positive compared to the standard setup.?>.

Augmented Reality for Head and Neck Carcinoma Imaging: Description and Feasibility of an Instant Calibration, Markerless Approach

BACKGROUND AND OBJECTIVE

Augmented reality (AR) can help to overcome current limitations in computer assisted head and neck surgery by granting "X-ray vision" to physicians. Still, the acceptance of AR in clinical applications is limited by technical and clinical challenges. We aim to demonstrate the benefit of a marker-free, instant calibration AR system for head and neck cancer imaging, which we hypothesize to be acceptable and practical for clinical use.

METHODS

We implemented a novel AR system for visualization of medical image data registered with the head or face of the patient prior to intervention. Our system allows the localization of head and neck carcinoma in relation to the outer anatomy. Our system does not require markers or stationary infrastructure, provides instant calibration and allows 2D and 3D multi-modal visualization for head and neck surgery planning via an AR head-mounted display. We evaluated our system in a pre-clinical user study with eleven medical experts.

RESULTS

Medical experts rated our application with a system usability scale score of 74.8 ± 15.9, which signifies above average, good usability and clinical acceptance. An average of 12.7 ± 6.6 minutes of training time was needed by physicians, before they were able to navigate the application without assistance.

CONCLUSIONS

Our AR system is characterized by a slim and easy setup, short training time and high usability and acceptance. Therefore, it presents a promising, novel tool for visualizing head and neck cancer imaging and pre-surgical localization of target structures.

The use of 3D virtual surgical planning and computer aided design in reconstruction of maxillary surgical defects

PURPOSE OF REVIEW

The present review describes the latest development of 3D virtual surgical planning (VSP) and computer aided design (CAD) for reconstruction of maxillary defects with an aim of fully prosthetic rehabilitation. The purpose is to give an overview of different methods that use CAD in maxillary reconstruction in patients with head and neck cancer.

RECENT FINDINGS

3D VSP enables preoperative planning of resection margins and osteotomies. The current 3D VSP workflow is expanded with multimodal imaging, merging decision supportive information. Development of more personalized implants is possible using CAD, individualized virtual muscle modelling and topology optimization. Meanwhile the translation of the 3D VSP towards surgery is improved by techniques like intraoperative imaging and augmented reality. Recent improvements of preoperative 3D VSP enables surgical reconstruction and/or prosthetic rehabilitation of the surgical defect in one combined procedure.

SUMMARY

With the use of 3D VSP and CAD, ablation surgery, reconstructive surgery, and prosthetic rehabilitation can be planned preoperatively. Many reconstruction possibilities exist and a choice depends on patient characteristics, tumour location and experience of the surgeon. The overall objective in patients with maxillary defects is to follow a prosthetic-driven reconstruction with the aim to restore facial form, oral function, and do so in accordance with the individual needs of the patient.

Novel finite element-based plate design for bridging mandibular defects: Reducing mechanical failure

INTRODUCTION

When the application of a free vascularised flap is not possible, a segmental mandibular defect is often reconstructed using a conventional reconstruction plate. Mechanical failure of such reconstructions is mostly caused by plate fracture and screw pull-out. This study aims to develop a reliable, mechanically superior, yet slender patient-specific reconstruction plate that reduces failure due to these causes.

PATIENTS AND METHODS

Eight patients were included in the study. Indications were as follows: fractured reconstruction plate (2), loosened screws (1) and primary reconstruction of a mandibular continuity defect (5). Failed conventional reconstructions were studied using finite element analysis (FEA). A 3D virtual surgical plan (3D-VSP) with a novel patient-specific (PS) titanium plate was developed for each patient. Postoperative CBCT scanning was performed to validate reconstruction accuracy.

RESULTS

All PS plates were placed accurately according to the 3D-VSP. Mean 3D screw entry point deviation was 1.54 mm (SD: 0.85, R: 0.10-3.19), and mean screw angular deviation was 5.76° (SD: 3.27, R: 1.26-16.62). FEA indicated decreased stress and screw pull-out inducing forces. No mechanical failures appeared (mean follow-up: 16 months, R: 7-29).

CONCLUSION

Reconstructing mandibular continuity defects with bookshelf-reconstruction plates with FEA underpinning the design seems to reduce the risk of screw pull-out and plate fractures.

A surgical navigated cutting guide for mandibular osteotomies: accuracy and reproducibility of an image-guided mandibular osteotomy

PURPOSE

3D-printed cutting guides are the current standard to translate the virtual surgery plan to the intraoperative setting. The production of these patient-specific cutting guides is time-consuming and costly, and therefore, alternative approaches are currently subject of research. The aim of this study was to assess the accuracy and reproducibility of using a novel electromagnetic (EM) navigated surgical cutting guide to perform virtually planned osteotomies in mandible models.

METHODS

A novel 3D navigated cutting guide (dubbed Bladerunner) was designed and evaluated with a total of 20 osteotomies, performed on plaster mandibular models according to preoperative planning using EM navigation. The pre- and postoperative scans were registered, and the difference between the preoperatively planned osteotomy and the performed osteotomy was expressed as the distance between the planned and performed cutting planes, and the yaw and roll angles between the planes.

RESULTS

The mean difference in distance between the planned osteotomy and performed osteotomy was 1.1 mm (STD 0.6 mm), the mean yaw was 1.8° (STD 1.4°), and mean roll was 1.6° (STD 1.3°).

CONCLUSION

The proposed EM navigated cutting guide for mandibular osteotomies demonstrated accurate positioning of the cutting plane according to the preoperative virtual surgical plan with respect to distance, yaw and roll angles. This novel approach has the potential to make the use of 3D-printed cutting guides obsolete, thereby decreasing the interval between diagnosis and surgery, reduce cost and allow for adaptation of the virtual plan in case of rapid tumor proliferation or unanticipated in situ deviations from the preoperative CT/MR imaging.

Evaluating the accuracy of resection planes in mandibular surgery using a preoperative, intraoperative, and postoperative approach

In mandibular surgery, three-dimensionally printed patient-specific cutting guides are used to translate the preoperative virtually planned resection planes to the operating room. This study was performed to determine whether cutting guides are positioned according to the virtual plan and to compare the intraoperative position of the cutting guide with the resection performed. Nine patients were included. The exact positions of the resection planes were planned virtually and a patient-specific cutting guide was designed and printed. After surgical placement of the cutting guide, intraoperative cone beam computed tomography (CBCT) was performed. Postoperative CT was used to obtain the final resection planes. Distances and yaw and pitch angles between the preoperative, intraoperative, and postoperative resection planes were calculated. Cutting guides were positioned on the mandible with millimetre accuracy. Anterior osteotomies were performed more accurately than posterior osteotomies (intraoperatively positioned and final resection planes differed by 1.2±1.0mm, 4.9±6.6°, and 1.8±1.5°, respectively, and by 2.2±0.9mm, 9.3±9°, and 8.3±6.5° respectively). Differences between intraoperatively planned and final resection planes imply a directional freedom of the saw through the saw slots. Since cutting guides are positioned with millimetre accuracy compared to the virtual plan, the design of the saw slots in the cutting guides needs improvement to allow more accurate resections.

Facial model collection for medical augmented reality in oncologic cranio-maxillofacial surgery

Medical augmented reality (AR) is an increasingly important topic in many medical fields. AR enables x-ray vision to see through real world objects. In medicine, this offers pre-, intra- or post-interventional visualization of "hidden" structures. In contrast to a classical monitor view, AR applications provide visualization not only on but also in relation to the patient. However, research and development of medical AR applications is challenging, because of unique patient-specific anatomies and pathologies. Working with several patients during the development for weeks or even months is not feasible. One alternative are commercial patient phantoms, which are very expensive. Hence, this data set provides a unique collection of head and neck cancer patient PET-CT scans with corresponding 3D models, provided as stereolitography (STL) files. The 3D models are optimized for effective 3D printing at low cost. This data can be used in the development and evaluation of AR applications for head and neck surgery.