1
|
Cîrneanu AL, Popescu D, Iordache D. New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:7092. [PMID: 37631629 PMCID: PMC10458371 DOI: 10.3390/s23167092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Revised: 07/29/2023] [Accepted: 08/02/2023] [Indexed: 08/27/2023]
Abstract
Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified.
Collapse
Affiliation(s)
- Andrada-Livia Cîrneanu
- Faculty of Automatic Control and Computers, University Politehnica of Bucharest, 060042 Bucharest, Romania;
| | - Dan Popescu
- Faculty of Automatic Control and Computers, University Politehnica of Bucharest, 060042 Bucharest, Romania;
| | - Dragoș Iordache
- The National Institute for Research & Development in Informatics-ICI Bucharest, 011455 Bucharest, Romania;
| |
Collapse
|
2
|
Application Research on Face Image Evaluation Algorithm of Deep Learning Mobile Terminal for Student Check-In Management. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:3961910. [PMID: 36017464 PMCID: PMC9398731 DOI: 10.1155/2022/3961910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/21/2022] [Revised: 06/18/2022] [Accepted: 06/24/2022] [Indexed: 11/28/2022]
Abstract
With the increase in the number of data, the traditional shallow image features cannot meet the needs of image representation. As an important means of image research, deep learning network has been paid attention to. In the field of face image evaluation, deep learning algorithm has been introduced, and the recognition technology has gradually matured. Based on this, this paper studies the application of face image evaluation algorithm of deep learning mobile terminal for student check-in management. A face image detection model for student check-in management is constructed, and a deep learning network is used to realize face detection. A face detection algorithm based on candidate region joint deep learning network is designed, and a face key point detection method based on cascaded convolution network is proposed. Aiming at the low efficiency of face recognition and detection, the existing loss function is optimized, the extraction algorithm of face binary features is proposed, and experiments are designed to analyze the performance of the algorithm. The simulation results show that the face detection based on the improved deep learning network can shorten the retrieval time and improve the accuracy of face image classification.
Collapse
|
3
|
Karpagam M, Jeyavathana RB, Chinnappan SK, Kanimozhi KV, Sambath M. A novel face recognition model for fighting against human trafficking in surveillance videos and rescuing victims. Soft comput 2022. [DOI: 10.1007/s00500-022-06931-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
4
|
Jiang L, Wu Z, Xu X, Zhan Y, Jin X, Wang L, Qiu Y. Opportunities and challenges of artificial intelligence in the medical field: current application, emerging problems, and problem-solving strategies. J Int Med Res 2021; 49:3000605211000157. [PMID: 33771068 PMCID: PMC8165857 DOI: 10.1177/03000605211000157] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
Recent advancements in the field of artificial intelligence have demonstrated
success in a variety of clinical tasks secondary to the development and
application of big data, supercomputing, sensor networks, brain science, and
other technologies. However, no projects can yet be used on a large scale in
real clinical practice because of the lack of standardized processes, lack of
ethical and legal supervision, and other issues. We analyzed the existing
problems in the field of artificial intelligence and herein propose possible
solutions. We call for the establishment of a process framework to ensure the
safety and orderly development of artificial intelligence in the medical
industry. This will facilitate the design and implementation of artificial
intelligence products, promote better management via regulatory authorities, and
ensure that reliable and safe artificial intelligence products are selected for
application.
Collapse
Affiliation(s)
- Lushun Jiang
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Zhe Wu
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Xiaolan Xu
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Yaqiong Zhan
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Xuehang Jin
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Li Wang
- Department of Rehabilitation Medicine, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China
| | - Yunqing Qiu
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, People's Republic of China.,Zhejiang Provincial Key Laboratory for Drug Clinical Research and Evaluation, Hangzhou, Zhejiang, People's Republic of China
| |
Collapse
|
5
|
Nguyen DT, Pham TD, Batchuluun G, Noh KJ, Park KR. Presentation Attack Face Image Generation Based on a Deep Generative Adversarial Network. SENSORS 2020; 20:s20071810. [PMID: 32218126 PMCID: PMC7180835 DOI: 10.3390/s20071810] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Revised: 03/12/2020] [Accepted: 03/19/2020] [Indexed: 11/16/2022]
Abstract
Although face-based biometric recognition systems have been widely used in many applications, this type of recognition method is still vulnerable to presentation attacks, which use fake samples to deceive the recognition system. To overcome this problem, presentation attack detection (PAD) methods for face recognition systems (face-PAD), which aim to classify real and presentation attack face images before performing a recognition task, have been developed. However, the performance of PAD systems is limited and biased due to the lack of presentation attack images for training PAD systems. In this paper, we propose a method for artificially generating presentation attack face images by learning the characteristics of real and presentation attack images using a few captured images. As a result, our proposed method helps save time in collecting presentation attack samples for training PAD systems and possibly enhance the performance of PAD systems. Our study is the first attempt to generate PA face images for PAD system based on CycleGAN network, a deep-learning-based framework for image generation. In addition, we propose a new measurement method to evaluate the quality of generated PA images based on a face-PAD system. Through experiments with two public datasets (CASIA and Replay-mobile), we show that the generated face images can capture the characteristics of presentation attack images, making them usable as captured presentation attack samples for PAD system training.
Collapse
|
6
|
Nguyen DT, Kang JK, Pham TD, Batchuluun G, Park KR. Ultrasound Image-Based Diagnosis of Malignant Thyroid Nodule Using Artificial Intelligence. SENSORS (BASEL, SWITZERLAND) 2020; 20:E1822. [PMID: 32218230 PMCID: PMC7180806 DOI: 10.3390/s20071822] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 03/09/2020] [Accepted: 03/23/2020] [Indexed: 12/14/2022]
Abstract
Computer-aided diagnosis systems have been developed to assist doctors in diagnosing thyroid nodules to reduce errors made by traditional diagnosis methods, which are mainly based on the experiences of doctors. Therefore, the performance of such systems plays an important role in enhancing the quality of a diagnosing task. Although there have been the state-of-the art studies regarding this problem, which are based on handcrafted features, deep features, or the combination of the two, their performances are still limited. To overcome these problems, we propose an ultrasound image-based diagnosis of the malignant thyroid nodule method using artificial intelligence based on the analysis in both spatial and frequency domains. Additionally, we propose the use of weighted binary cross-entropy loss function for the training of deep convolutional neural networks to reduce the effects of unbalanced training samples of the target classes in the training data. Through our experiments with a popular open dataset, namely the thyroid digital image database (TDID), we confirm the superiority of our method compared to the state-of-the-art methods.
Collapse
Affiliation(s)
| | | | - Tuyen Danh Pham
- Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 04620, Korea; (D.T.N.); (J.K.K.); (G.B.); (K.R.P.)
| | | | | |
Collapse
|
7
|
Kortli Y, Jridi M, Al Falou A, Atri M. Face Recognition Systems: A Survey. SENSORS (BASEL, SWITZERLAND) 2020; 20:E342. [PMID: 31936089 PMCID: PMC7013584 DOI: 10.3390/s20020342] [Citation(s) in RCA: 122] [Impact Index Per Article: 30.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 12/12/2019] [Accepted: 12/15/2019] [Indexed: 11/23/2022]
Abstract
Over the past few decades, interest in theories and algorithms for face recognition has been growing rapidly. Video surveillance, criminal identification, building access control, and unmanned and autonomous vehicles are just a few examples of concrete applications that are gaining attraction among industries. Various techniques are being developed including local, holistic, and hybrid approaches, which provide a face image description using only a few face image features or the whole facial features. The main contribution of this survey is to review some well-known techniques for each approach and to give the taxonomy of their categories. In the paper, a detailed comparison between these techniques is exposed by listing the advantages and the disadvantages of their schemes in terms of robustness, accuracy, complexity, and discrimination. One interesting feature mentioned in the paper is about the database used for face recognition. An overview of the most commonly used databases, including those of supervised and unsupervised learning, is given. Numerical results of the most interesting techniques are given along with the context of experiments and challenges handled by these techniques. Finally, a solid discussion is given in the paper about future directions in terms of techniques to be used for face recognition.
Collapse
Affiliation(s)
- Yassin Kortli
- AI-ED Department, Yncrea Ouest, 20 rue du Cuirassé de Bretagne, 29200 Brest, France; (M.J.); (A.A.F.)
- Electronic and Micro-electronic Laboratory, Faculty of Sciences of Monastir, University of Monastir, Monastir 5000, Tunisia
| | - Maher Jridi
- AI-ED Department, Yncrea Ouest, 20 rue du Cuirassé de Bretagne, 29200 Brest, France; (M.J.); (A.A.F.)
| | - Ayman Al Falou
- AI-ED Department, Yncrea Ouest, 20 rue du Cuirassé de Bretagne, 29200 Brest, France; (M.J.); (A.A.F.)
| | - Mohamed Atri
- College of Computer Science, King Khalid University, Abha 61421, Saudi Arabia;
| |
Collapse
|
8
|
Artificial Intelligence-Based Thyroid Nodule Classification Using Information from Spatial and Frequency Domains. J Clin Med 2019; 8:jcm8111976. [PMID: 31739517 PMCID: PMC6912332 DOI: 10.3390/jcm8111976] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Revised: 11/08/2019] [Accepted: 11/10/2019] [Indexed: 12/25/2022] Open
Abstract
Image-based computer-aided diagnosis (CAD) systems have been developed to assist doctors in the diagnosis of thyroid cancer using ultrasound thyroid images. However, the performance of these systems is strongly dependent on the selection of detection and classification methods. Although there are previous researches on this topic, there is still room for enhancement of the classification accuracy of the existing methods. To address this issue, we propose an artificial intelligence-based method for enhancing the performance of the thyroid nodule classification system. Thus, we extract image features from ultrasound thyroid images in two domains: spatial domain based on deep learning, and frequency domain based on Fast Fourier transform (FFT). Using the extracted features, we perform a cascade classifier scheme for classifying the input thyroid images into either benign (negative) or malign (positive) cases. Through expensive experiments using a public dataset, the thyroid digital image database (TDID) dataset, we show that our proposed method outperforms the state-of-the-art methods and produces up-to-date classification results for the thyroid nodule classification problem.
Collapse
|