Simulation of Postoperative Facial Appearances via Geometric Deep Learning for Efficient Orthognathic Surgical Planning

3D evaluation model of facial aesthetics based on multi-input 3D convolution neural networks for orthognathic surgery

BACKGROUND

Quantitative evaluation of facial aesthetics is an important but also time-consuming procedure in orthognathic surgery, while existing 2D beauty-scoring models are mainly used for entertainment with less clinical impact.

METHODS

A deep-learning-based 3D evaluation model DeepBeauty3D was designed and trained using 133 patients' CT images. The customised image preprocessing module extracted the skeleton, soft tissue, and personal physical information from raw DICOM data, and the predicting network module employed 3-input-2-output convolution neural networks (CNN) to receive the aforementioned data and output aesthetic scores automatically.

RESULTS

Experiment results showed that this model predicted the skeleton and soft tissue score with 0.231 ± 0.218 (4.62%) and 0.100 ± 0.344 (2.00%) accuracy in 11.203 ± 2.824 s from raw CT images.

CONCLUSION

This study provided an end-to-end solution using real clinical data based on 3D CNN to quantitatively evaluate facial aesthetics by considering three anatomical factors simultaneously, showing promising potential in reducing workload and bridging the surgeon-patient aesthetics perspective gap.

Frontal facial analysis of female celebrity attractiveness standards through artificial intelligence

The contemporary significance of celebrities' facial aesthetics underscores their heightened importance in shaping attractiveness standards. This retrospective study aimed to investigate the impact of patterns on aesthetic canons in the profile views of female celebrities, using artificial intelligence. The study sought to compare different races and propose standards for attractive faces. In this retrospective cohort study, a Python-based algorithm was used to analyze frontal patterns and evaluate their influence on aesthetic norms in publicly accessible images of female global celebrities. Ten ideal angular or proportional measures were gathered from the literature, and were trained to serve as a benchmark for the analysis of facial attractiveness. Demographic characteristics were described statistically. A one-way ANOVA test was employed to assess data distribution. Differences in means between groups were evaluated using nonparametric independent-sample tests, with statistical significance set at < 0.05. The study involved facial analyses for 115 female celebrities. It revealed variations in facial features among races. The mean golden ratio differed, with African and Asian individuals showing lower ratios. Symmetry varied, with Latin and Caucasian faces considered the most symmetrical. The zygomatic-to-mandibular width ratio was similar across races, with a ratio close to 80% being associated with more attractive faces. Differences in nose-to-mouth ratio, lips, alar base width, and chin angle were noted among race groups. The study concluded that, regardless of race, an attractive female face is characterized by specific ratios and angles. Facial symmetry, though desirable, is not strictly necessary. Irrespective of race background, an appealing female face is characterized by a zygomatic-to-mandibular width ratio nearing 80%, a mid-facial third that is slightly larger than the lower third, and a distinctive chin angle of approximately 138°, contributing to a trapezoidal facial shape. The findings contribute valuable insights into attractiveness standards and the impact of frontal patterns on aesthetic canons in female celebrities.

Predicting optimal patient-specific postoperative facial landmarks for patients with craniomaxillofacial deformities

Orthognathic surgery primarily corrects skeletal anomalies and malocclusion to enhance facial aesthetics, aiming for an improved facial appearance. However, this traditional skeletal-driven approach may result in undesirable residual asymmetry. To address this issue, a soft tissue-driven planning methodology has been proposed. This technique estimates bone movements based on the envisioned optimal facial appearance, thereby enhancing surgical accuracy and effectiveness. This study investigates the initial implementation phase of the soft tissue-driven approach, simulating the patient's ideal appearance by realigning distorted facial landmarks to an ideal state. The algorithm employs symmetrization and weighted optimization strategies, aligning projected optimal landmarks with standard cephalometric values for both facial symmetry and form, which are essential in orthognathic surgery for facial aesthetics. It also incorporates regularization to preserve the patient's facial characteristics. Validation through retrospective analysis of preoperative patients and normal subjects demonstrates this method's efficacy in achieving facial symmetry, particularly in the lower face, and promoting a natural, harmonious contour. Adhering to soft tissue-driven principles, this novel approach shows promise in surpassing traditional methods, potentially leading to enhanced facial outcomes and patient satisfaction in orthognathic surgery.

Turn Your Vision into Reality-AI-Powered Pre-operative Outcome Simulation in Rhinoplasty Surgery

BACKGROUND

The increasing demand and changing trends in rhinoplasty surgery emphasize the need for effective doctor-patient communication, for which Artificial Intelligence (AI) could be a valuable tool in managing patient expectations during pre-operative consultations.

OBJECTIVE

To develop an AI-based model to simulate realistic postoperative rhinoplasty outcomes.

METHODS

We trained a Generative Adversarial Network (GAN) using 3,030 rhinoplasty patients' pre- and postoperative images. One-hundred-one study participants were presented with 30 pre-rhinoplasty patient photographs followed by an image set consisting of the real postoperative versus the GAN-generated image and asked to identify the GAN-generated image.

RESULTS

The study sample (48 males, 53 females, mean age of 31.6 ± 9.0 years) correctly identified the GAN-generated images with an accuracy of 52.5 ± 14.3%. Male study participants were more likely to identify the AI-generated images compared with female study participants (55.4% versus 49.6%; p = 0.042).

CONCLUSION

We presented a GAN-based simulator for rhinoplasty outcomes which used pre-operative patient images to predict accurate representations that were not perceived as different from real postoperative outcomes.

LEVEL OF EVIDENCE III

This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

The Accuracy of Three-Dimensional Soft Tissue Simulation in Orthognathic Surgery-A Systematic Review

Three-dimensional soft tissue simulation has become a popular tool in the process of virtual orthognathic surgery planning and patient-surgeon communication. To apply 3D soft tissue simulation software in routine clinical practice, both qualitative and quantitative validation of its accuracy are required. The objective of this study was to systematically review the literature on the accuracy of 3D soft tissue simulation in orthognathic surgery. The Web of Science, PubMed, Cochrane, and Embase databases were consulted for the literature search. The systematic review (SR) was conducted according to the PRISMA statement, and 40 articles fulfilled the inclusion and exclusion criteria. The Quadas-2 tool was used for the risk of bias assessment for selected studies. A mean error varying from 0.27 mm to 2.9 mm for 3D soft tissue simulations for the whole face was reported. In the studies evaluating 3D soft tissue simulation accuracy after a Le Fort I osteotomy only, the upper lip and paranasal regions were reported to have the largest error, while after an isolated bilateral sagittal split osteotomy, the largest error was reported for the lower lip and chin regions. In the studies evaluating simulation after bimaxillary osteotomy with or without genioplasty, the highest inaccuracy was reported at the level of the lips, predominantly the lower lip, chin, and, sometimes, the paranasal regions. Due to the variability in the study designs and analysis methods, a direct comparison was not possible. Therefore, based on the results of this SR, guidelines to systematize the workflow for evaluating the accuracy of 3D soft tissue simulations in orthognathic surgery in future studies are proposed.

Cataract-1K Dataset for Deep-Learning-Assisted Analysis of Cataract Surgery Videos

In recent years, the landscape of computer-assisted interventions and post-operative surgical video analysis has been dramatically reshaped by deep-learning techniques, resulting in significant advancements in surgeons' skills, operation room management, and overall surgical outcomes. However, the progression of deep-learning-powered surgical technologies is profoundly reliant on large-scale datasets and annotations. In particular, surgical scene understanding and phase recognition stand as pivotal pillars within the realm of computer-assisted surgery and post-operative assessment of cataract surgery videos. In this context, we present the largest cataract surgery video dataset that addresses diverse requisites for constructing computerized surgical workflow analysis and detecting post-operative irregularities in cataract surgery. We validate the quality of annotations by benchmarking the performance of several state-of-the-art neural network architectures for phase recognition and surgical scene segmentation. Besides, we initiate the research on domain adaptation for instrument segmentation in cataract surgery by evaluating cross-domain instrument segmentation performance in cataract surgery videos. The dataset and annotations are publicly available in Synapse.

Correspondence attention for facial appearance simulation

In orthognathic surgical planning for patients with jaw deformities, it is crucial to accurately simulate the changes in facial appearance that follow the bony movement. Compared with the traditional biomechanics-based methods like the finite-element method (FEM), which are both labor-intensive and computationally inefficient, deep learning-based methods offer an efficient and robust modeling alternative. However, current methods do not account for the physical relationship between facial soft tissue and bony structure, causing them to fall short in accuracy compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to predict facial changes by correlating facial soft tissue changes with bony movement through a point-to-point attentive correspondence matrix. To ensure efficient training, we also introduce a contrastive loss for self-supervised pre-training of the ACMT-Net with a k-Nearest Neighbors (k-NN) based clustering. Experimental results on patients with jaw deformities show that our proposed solution can achieve significantly improved computational efficiency over the state-of-the-art FEM-based method with comparable facial change prediction accuracy.

Image generation technology for functional occlusal pits and fissures based on a conditional generative adversarial network

The occlusal surfaces of natural teeth have complex features of functional pits and fissures. These morphological features directly affect the occlusal state of the upper and lower teeth. An image generation technology for functional occlusal pits and fissures is proposed to address the lack of local detailed crown surface features in existing dental restoration methods. First, tooth depth image datasets were constructed using an orthogonal projection method. Second, the optimization and improvement of the model parameters were guided by introducing the jaw position spatial constraint, the L1 loss and the perceptual loss functions. Finally, two image quality evaluation metrics were applied to evaluate the quality of the generated images, and deform the dental crown by using the generated occlusal pits and fissures as constraints to compare with expert data. The results showed that the images generated using the network constructed in this study had high quality, and the detailed pit and fissure features on the crown were effectively restored, with a standard deviation of 0.1802mm compared to the expert-designed tooth crown models.

Prediction of orthognathic surgery plan from 3D cephalometric analysis via deep learning

BACKGROUND

Preoperative planning of orthognathic surgery is indispensable for achieving ideal surgical outcome regarding the occlusion and jaws' position. However, orthognathic surgery planning is sophisticated and highly experience-dependent, which requires comprehensive consideration of facial morphology and occlusal function. This study aimed to investigate a robust and automatic method based on deep learning to predict reposition vectors of jawbones in orthognathic surgery plan.

METHODS

A regression neural network named VSP transformer was developed based on Transformer architecture. Firstly, 3D cephalometric analysis was employed to quantify skeletal-facial morphology as input features. Next, input features were weighted using pretrained results to minimize bias resulted from multicollinearity. Through encoder-decoder blocks, ten landmark-based reposition vectors of jawbones were predicted. Permutation importance (PI) method was used to calculate contributions of each feature to final prediction to reveal interpretability of the proposed model.

RESULTS

VSP transformer model was developed with 383 samples and clinically tested with 49 prospectively collected samples. Our proposed model outperformed other four classic regression models in prediction accuracy. Mean absolute errors (MAE) of prediction were 1.41 mm in validation set and 1.34 mm in clinical test set. The interpretability results of the model were highly consistent with clinical knowledge and experience.

CONCLUSIONS

The developed model can predict reposition vectors of orthognathic surgery plan with high accuracy and good clinically practical-effectiveness. Moreover, the model was proved reliable because of its good interpretability.

Bidirectional prediction of facial and bony shapes for orthognathic surgical planning

This paper proposes a deep learning framework to encode subject-specific transformations between facial and bony shapes for orthognathic surgical planning. Our framework involves a bidirectional point-to-point convolutional network (P2P-Conv) to predict the transformations between facial and bony shapes. P2P-Conv is an extension of the state-of-the-art P2P-Net and leverages dynamic point-wise convolution (i.e., PointConv) to capture local-to-global spatial information. Data augmentation is carried out in the training of P2P-Conv with multiple point subsets from the facial and bony shapes. During inference, network outputs generated for multiple point subsets are combined into a dense transformation. Finally, non-rigid registration using the coherent point drift (CPD) algorithm is applied to generate surface meshes based on the predicted point sets. Experimental results on real-subject data demonstrate that our method substantially improves the prediction of facial and bony shapes over state-of-the-art methods.

Three-Dimensional Postoperative Results Prediction for Orthognathic Surgery through Deep Learning-Based Alignment Network

To date, for the diagnosis of dentofacial dysmorphosis, we have relied almost entirely on reference points, planes, and angles. This is time consuming, and it is also greatly influenced by the skill level of the practitioner. To solve this problem, we wanted to know if deep neural networks could predict postoperative results of orthognathic surgery without relying on reference points, planes, and angles. We use three-dimensional point cloud data of the skull of 269 patients. The proposed method has two main stages for prediction. In step 1, the skull is divided into six parts through the segmentation network. In step 2, three-dimensional transformation parameters are predicted through the alignment network. The ground truth values of transformation parameters are calculated through the iterative closest points (ICP), which align the preoperative part of skull to the corresponding postoperative part of skull. We compare pointnet, pointnet++ and pointconv for the feature extractor of the alignment network. Moreover, we design a new loss function, which considers the distance error of transformed points for a better accuracy. The accuracy, mean intersection over union (mIoU), and dice coefficient (DC) of the first segmentation network, which divides the upper and lower part of skull, are 0.9998, 0.9994, and 0.9998, respectively. For the second segmentation network, which divides the lower part of skull into 5 parts, they were 0.9949, 0.9900, 0.9949, respectively. The mean absolute error of transverse, anterior-posterior, and vertical distance of part 2 (maxilla) are 0.765 mm, 1.455 mm, and 1.392 mm, respectively. For part 3 (mandible), they were 1.069 mm, 1.831 mm, and 1.375 mm, respectively, and for part 4 (chin), they were 1.913 mm, 2.340 mm, and 1.257 mm, respectively. From this study, postoperative results can now be easily predicted by simply entering the point cloud data of computed tomography.