Deep self-taught learning for facial beauty prediction

Artificial Intelligence: Singularity Approaches

SUMMARY

Artificial intelligence (AI) has been a disruptive technology within health care, from the development of simple care algorithms to complex deep-learning models. AI has the potential to reduce the burden of administrative tasks, advance clinical decision-making, and improve patient outcomes. Unlocking the full potential of AI requires the analysis of vast quantities of clinical information. Although AI holds tremendous promise, widespread adoption within plastic surgery remains limited. Understanding the basics is essential for plastic surgeons to evaluate the potential uses of AI. This review provides an introduction of AI, including the history of AI, key concepts, applications of AI in plastic surgery, and future implications.

Face beautification: Beyond makeup transfer

Facial appearance plays an important role in our social lives. Subjective perception of women's beauty depends on various face-related (e.g., skin, shape, hair) and environmental (e.g., makeup, lighting, angle) factors. Similarly to cosmetic surgery in the physical world, virtual face beautification is an emerging field with many open issues to be addressed. Inspired by the latest advances in style-based synthesis and face beauty prediction, we propose a novel framework for face beautification. For a given reference face with a high beauty score, our GAN-based architecture is capable of translating an inquiry face into a sequence of beautified face images with the referenced beauty style and the target beauty score values. To achieve this objective, we propose to integrate both style-based beauty representation (extracted from the reference face) and beauty score prediction (trained on the SCUT-FBP database) into the beautification process. Unlike makeup transfer, our approach targets many-to-many (instead of one-to-one) translation, where multiple outputs can be defined by different references with various beauty scores. Extensive experimental results are reported to demonstrate the effectiveness and flexibility of the proposed face beautification framework. To support reproducible research, the source codes accompanying this work will be made publicly available on GitHub.

Multi-View Graph Fusion for Semi-Supervised Learning: Application to Image-Based Face Beauty Prediction

Facial Beauty Prediction (FBP) is an important visual recognition problem to evaluate the attractiveness of faces according to human perception. Most existing FBP methods are based on supervised solutions using geometric or deep features. Semi-supervised learning for FBP is an almost unexplored research area. In this work, we propose a graph-based semi-supervised method in which multiple graphs are constructed to find the appropriate graph representation of the face images (with and without scores). The proposed method combines both geometric and deep feature-based graphs to produce a high-level representation of face images instead of using a single face descriptor and also improves the discriminative ability of graph-based score propagation methods. In addition to the data graph, our proposed approach fuses an additional graph adaptively built on the predicted beauty values. Experimental results on the SCUTFBP-5500 facial beauty dataset demonstrate the superiority of the proposed algorithm compared to other state-of-the-art methods.

Machine Learning-Based Facial Beauty Prediction and Analysis of Frontal Facial Images Using Facial Landmarks and Traditional Image Descriptors

The beauty industry has seen rapid growth in multiple countries and due to its applications in entertainment, the analysis and assessment of facial attractiveness have received attention from scientists, physicians, and artists because of digital media, plastic surgery, and cosmetics. An analysis of techniques is used in the assessment of facial beauty that considers facial ratios and facial qualities as elements to predict facial beauty. Here, the facial landmarks are extracted to calculate facial ratios according to Golden Ratios and Symmetry Ratios, and an ablation study is performed to find the best performing feature set from extracted ratios. Subsequently, Gray Level Covariance Matrix (GLCM), Hu's Moments, and Color Histograms in the HSV space are extracted as texture, shape, and color features, respectively. Another ablation study is performed to find out which feature performs the best when concatenated with the facial landmarks. Experimental results show that the concatenation of primary facial characteristics with facial landmarks improved the prediction score of facial beauty. Four models are trained, K-Nearest Neighbors (KNN), Linear Regression (LR), Random Forest (RF), and Artificial Neural Network (ANN) on a dataset of 5500 frontal facial images, and amongst them, KNN performs the best for the concatenated features achieving a Pearson's Correlation Coefficient of 0.7836 and a Mean Squared Error of 0.0963. Our analysis also provides us with insights into how different machine learning models can understand the concept of facial beauty.

Home Textile Pattern Emotion Labeling Using Deep Multi-View Feature Learning

Different home textile patterns have different emotional expressions. Emotion evaluation of home textile patterns can effectively improve the retrieval performance of home textile patterns based on semantics. It can not only help designers make full use of existing designs and stimulate creative inspiration but also help users select designs and products that are more in line with their needs. In this study, we develop a three-stage framework for home textile pattern emotion labeling based on artificial intelligence. To be specific, first of all, three kinds of aesthetic features, i.e., shape, texture, and salient region, are extracted from the original home textile patterns. Then, a CNN (convolutional neural network)-based deep feature extractor is constructed to extract deep features from the aesthetic features acquired in the previous stage. Finally, a novel multi-view classifier is designed to label home textile patterns that can automatically learn the weight of each view. The three-stage framework is evaluated by our data and the experimental results show its promising performance in home textile patterns labeling.

A novel facial attractiveness evaluation system based on face shape, facial structure features and skin

Facial attractiveness is an important research direction of genetic psychology and cognitive psychology, and its results are significant for the study of face evolution and human evolution. However, previous studies have not put forward a comprehensive evaluation system of facial attractiveness. Traditionally, the establishment of facial attractiveness evaluation system was based on facial geometric features, without facial skin features. In this paper, combined with big data analysis, evaluation of face in real society and literature research, we found that skin also have a significant impact on facial attractiveness, because skin could reflect age, wrinkles and healthful qualities, thus affected the human perception of facial attractiveness. Therefore, we propose a comprehensive and novel facial attractiveness evaluation system based on face shape structural features, facial structure features and skin texture feature. In order to apply face shape structural features to the evaluation of facial attractiveness, the classification of face shape is the first step. Face image dataset is divided according to face shape, and then facial structure features and skin texture features that represent facial attractiveness are extracted and fused. The machine learning algorithm with the best prediction performance is selected in the face shape structural subsets to predict facial attractiveness. Experimental results show that the facial attractiveness evaluation performance can be improved by the method based on classification of face shape and multi-features fusion, the facial attractiveness scores obtained by the proposed system correlates better with human ratings. Our evaluation system can help people project their cognition of facial attractiveness into artificial agents they interact with.

An insight analysis and detection of drug-abuse risk behavior on Twitter with self-taught deep learning

Drug abuse continues to accelerate towards becoming the most severe public health problem in the United States. The ability to detect drug-abuse risk behavior at a population scale, such as among the population of Twitter users, can help us to monitor the trend of drug-abuse incidents. Unfortunately, traditional methods do not effectively detect drug-abuse risk behavior, given tweets. This is because: (1) tweets usually are noisy and sparse and (2) the availability of labeled data is limited. To address these challenging problems, we propose a deep self-taught learning system to detect and monitor drug-abuse risk behaviors in the Twitter sphere, by leveraging a large amount of unlabeled data. Our models automatically augment annotated data: (i) to improve the classification performance and (ii) to capture the evolving picture of drug abuse on online social media. Our extensive experiments have been conducted on three million drug-abuse-related tweets with geo-location information. Results show that our approach is highly effective in detecting drug-abuse risk behaviors.

Subjectivity and complexity of facial attractiveness

The origin and meaning of facial beauty represent a longstanding puzzle. Despite the profuse literature devoted to facial attractiveness, its very nature, its determinants and the nature of inter-person differences remain controversial issues. Here we tackle such questions proposing a novel experimental approach in which human subjects, instead of rating natural faces, are allowed to efficiently explore the face-space and "sculpt" their favorite variation of a reference facial image. The results reveal that different subjects prefer distinguishable regions of the face-space, highlighting the essential subjectivity of the phenomenon. The different sculpted facial vectors exhibit strong correlations among pairs of facial distances, characterising the underlying universality and complexity of the cognitive processes, and the relative relevance and robustness of the different facial distances.

BeautyNet: Joint Multiscale CNN and Transfer Learning Method for Unconstrained Facial Beauty Prediction

Because of the lack of discriminative face representations and scarcity of labeled training data, facial beauty prediction (FBP), which aims at assessing facial attractiveness automatically, has become a challenging pattern recognition problem. Inspired by recent promising work on fine-grained image classification using the multiscale architecture to extend the diversity of deep features, BeautyNet for unconstrained facial beauty prediction is proposed in this paper. Firstly, a multiscale network is adopted to improve the discriminative of face features. Secondly, to alleviate the computational burden of the multiscale architecture, MFM (max-feature-map) is utilized as an activation function which can not only lighten the network and speed network convergence but also benefit the performance. Finally, transfer learning strategy is introduced here to mitigate the overfitting phenomenon which is caused by the scarcity of labeled facial beauty samples and improves the proposed BeautyNet's performance. Extensive experiments performed on LSFBD demonstrate that the proposed scheme outperforms the state-of-the-art methods, which can achieve 67.48% classification accuracy.

Art or Science? An Evidence-Based Approach to Human Facial Beauty a Quantitative Analysis Towards an Informed Clinical Aesthetic Practice

BACKGROUND

Patients often seek guidance from the aesthetic practitioners regarding treatments to enhance their 'beauty'. Is there a science behind the art of assessment and if so is it measurable? Through the centuries, this question has challenged scholars, artists and surgeons.

AIMS AND OBJECTIVES

This study aims to undertake a review of the evidence behind quantitative facial measurements in assessing beauty to help the practitioner in everyday aesthetic practice.

METHODS

A Medline, Embase search for beauty, facial features and quantitative analysis was undertaken.

SELECTION CRITERIA

Inclusion criteria were studies on adults, and exclusions included studies undertaken for dental, cleft lip, oncology, burns or reconstructive surgeries. The abstracts and papers were appraised, and further studies excluded that were considered inappropriate. The data were extracted using a standardised table. The final dataset was appraised in accordance with the PRISMA checklist and Holland and Rees' critique tools.

RESULTS

Of the 1253 studies screened, 1139 were excluded from abstracts and a further 70 excluded from full text articles. The remaining 44 were assessed qualitatively and quantitatively. It became evident that the datasets were not comparable. Nevertheless, common themes were obvious, and these were summarised.

CONCLUSION

Despite measures of the beauty of individual components to the sum of all the parts, such as symmetry and the golden ratio, we are yet far from establishing what truly constitutes quantitative beauty. Perhaps beauty is truly in the 'eyes of the beholder' (and perhaps in the eyes of the subject too).

LEVEL OF EVIDENCE V

This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .