Adaptive Convolutional Neural Network and Its Application in Face Recognition

Snow Parameters Inversion from Passive Microwave Remote Sensing Measurements by Deep Convolutional Neural Networks

This paper proposes a novel inverse method based on the deep convolutional neural network (ConvNet) to extract snow’s layer thickness and temperature via passive microwave remote sensing (PMRS). The proposed ConvNet is trained using simulated data obtained through conventional computational electromagnetic methods. Compared with the traditional inverse method, the trained ConvNet can predict the result with higher accuracy. Besides, the proposed method has a strong tolerance for noise. The proposed ConvNet composes three pairs of convolutional and activation layers with one additional fully connected layer to realize regression, i.e., the inversion of snow parameters. The feasibility of the proposed method in learning the inversion of snow parameters is validated by numerical examples. The inversion results indicate that the correlation coefficient (R2) ratio between the proposed ConvNet and conventional methods reaches 4.8, while the ratio for the root mean square error (RMSE) is only 0.18. Hence, the proposed method experiments with a novel path to improve the inversion of passive microwave remote sensing through deep learning approaches.

Hybridized sine cosine algorithm with convolutional neural networks dropout regularization application

Deep learning has recently been utilized with great success in a large number of diverse application domains, such as visual and face recognition, natural language processing, speech recognition, and handwriting identification. Convolutional neural networks, that belong to the deep learning models, are a subtype of artificial neural networks, which are inspired by the complex structure of the human brain and are often used for image classification tasks. One of the biggest challenges in all deep neural networks is the overfitting issue, which happens when the model performs well on the training data, but fails to make accurate predictions for the new data that is fed into the model. Several regularization methods have been introduced to prevent the overfitting problem. In the research presented in this manuscript, the overfitting challenge was tackled by selecting a proper value for the regularization parameter dropout by utilizing a swarm intelligence approach. Notwithstanding that the swarm algorithms have already been successfully applied to this domain, according to the available literature survey, their potential is still not fully investigated. Finding the optimal value of dropout is a challenging and time-consuming task if it is performed manually. Therefore, this research proposes an automated framework based on the hybridized sine cosine algorithm for tackling this major deep learning issue. The first experiment was conducted over four benchmark datasets: MNIST, CIFAR10, Semeion, and UPS, while the second experiment was performed on the brain tumor magnetic resonance imaging classification task. The obtained experimental results are compared to those generated by several similar approaches. The overall experimental results indicate that the proposed method outperforms other state-of-the-art methods included in the comparative analysis in terms of classification error and accuracy.

Artificial Intelligence-Aided Multiple Tumor Detection Method Based on Immunohistochemistry-Enhanced Dark-Field Imaging

The immunohistochemical method serves as one of the most practical tools in clinical cancer detection and thus has great application value to overcome the existing limits of the conventional method and further improve the detecting efficiency and sensitivity. This study employed 3,3'-diaminobenzidine (DAB), a conventional color indicator for immunohistochemistry, as a novel high-sensitive scattering reagent to provide a multidimensional image signal varying with the overexpression rate of tumor markers. Based on the scattering properties of DAB aggregates, an efficient and robust artificial intelligence-aided immunohistochemical method based on dark-field imaging has been established, with improvement in both the imaging quality and interpretation efficiency in comparison with the conventional manual-operated immunohistochemical method. Referencing the diagnosis from three independent pathologists, this method succeeded in detecting HER2 overexpressed breast tumors with a sensitivity of 95.2% and a specificity of 100.0%; meanwhile, it was found to be applicable for non-small-cell lung tumors and malignant lymphoma as well. As demonstrated, this study provided an effective and reliable means for making diagnostic suggestions, which exhibited great potential in multiple tumor pathological detection at low cost.

Performance of a Novel Chaotic Firefly Algorithm with Enhanced Exploration for Tackling Global Optimization Problems: Application for Dropout Regularization

Swarm intelligence techniques have been created to respond to theoretical and practical global optimization problems. This paper puts forward an enhanced version of the firefly algorithm that corrects the acknowledged drawbacks of the original method, by an explicit exploration mechanism and a chaotic local search strategy. The resulting augmented approach was theoretically tested on two sets of bound-constrained benchmark functions from the CEC suites and practically validated for automatically selecting the optimal dropout rate for the regularization of deep neural networks. Despite their successful applications in a wide spectrum of different fields, one important problem that deep learning algorithms face is overfitting. The traditional way of preventing overfitting is to apply regularization; the first option in this sense is the choice of an adequate value for the dropout parameter. In order to demonstrate its ability in finding an optimal dropout rate, the boosted version of the firefly algorithm has been validated for the deep learning subfield of convolutional neural networks, with respect to five standard benchmark datasets for image processing: MNIST, Fashion-MNIST, Semeion, USPS and CIFAR-10. The performance of the proposed approach in both types of experiments was compared with other recent state-of-the-art methods. To prove that there are significant improvements in results, statistical tests were conducted. Based on the experimental data, it can be concluded that the proposed algorithm clearly outperforms other approaches.

Chaotic Harris Hawks Optimization with Quasi-Reflection-Based Learning: An Application to Enhance CNN Design

The research presented in this manuscript proposes a novel Harris Hawks optimization algorithm with practical application for evolving convolutional neural network architecture to classify various grades of brain tumor using magnetic resonance imaging. The proposed improved Harris Hawks optimization method, which belongs to the group of swarm intelligence metaheuristics, further improves the exploration and exploitation abilities of the basic algorithm by incorporating a chaotic population initialization and local search, along with a replacement strategy based on the quasi-reflection-based learning procedure. The proposed method was first evaluated on 10 recent CEC2019 benchmarks and the achieved results are compared with the ones generated by the basic algorithm, as well as with results of other state-of-the-art approaches that were tested under the same experimental conditions. In subsequent empirical research, the proposed method was adapted and applied for a practical challenge of convolutional neural network design. The evolved network structures were validated against two datasets that contain images of a healthy brain and brain with tumors. The first dataset comprises well-known IXI and cancer imagining archive images, while the second dataset consists of axial T1-weighted brain tumor images, as proposed in one recently published study in the Q1 journal. After performing data augmentation, the first dataset encompasses 8.000 healthy and 8.000 brain tumor images with grades I, II, III, and IV and the second dataset includes 4.908 images with Glioma, Meningioma, and Pituitary, with 1.636 images belonging to each tumor class. The swarm intelligence-driven convolutional neural network approach was evaluated and compared to other, similar methods and achieved a superior performance. The obtained accuracy was over 95% in all conducted experiments. Based on the established results, it is reasonable to conclude that the proposed approach could be used to develop networks that can assist doctors in diagnostics and help in the early detection of brain tumors.

Introspective analysis of convolutional neural networks for improving discrimination performance and feature visualisation

Deep neural networks have been widely explored and utilised as a useful tool for feature extraction in computer vision and machine learning. It is often observed that the last fully connected (FC) layers of convolutional neural network possess higher discrimination power as compared to the convolutional and maxpooling layers whose goal is to preserve local and low-level information of the input image and down sample it to avoid overfitting. Inspired from the functionality of local binary pattern (LBP) operator, this paper proposes to induce discrimination into the mid layers of convolutional neural network by introducing a discriminatively boosted alternative to pooling (DBAP) layer that has shown to serve as a favourable replacement of early maxpooling layer in a convolutional neural network (CNN). A thorough research of the related works show that the proposed change in the neural architecture is novel and has not been proposed before to bring enhanced discrimination and feature visualisation power achieved from the mid layer features. The empirical results reveal that the introduction of DBAP layer in popular neural architectures such as AlexNet and LeNet produces competitive classification results in comparison to their baseline models as well as other ultra-deep models on several benchmark data sets. In addition, better visualisation of intermediate features can allow one to seek understanding and interpretation of black box behaviour of convolutional neural networks, used widely by the research community.

Research on influence mechanism of running clothing fatigue based on BP neural network

Dress fatigue can affect the efficiency of sports, especially for running, the dress fatigue has a greater impact on it. Moreover, at present, there are few studies on dress fatigue. Based on this, this study is based on BP neural network, and uses surface electromyography theory and muscle fatigue measurement method to perform fatigue measurement. The fatigue threshold analysis is mainly carried out by the experimental method, and the prediction model of the wearing fatigue threshold based on BP neural network is constructed based on the actual demand. Moreover, this paper verifies the reliability of threshold distribution by experimental analysis combined with model analysis. In addition, the study sets the organizational structure and clothing pressure as verification indicators to analyze the performance of the model. The research results show that the model constructed in this study can effectively analyze the mechanism of fatigue impact of running dress, and this paper can provide reference for the study of dress fatigue.

Achievements analysis of mooc English course based on fuzzy statistics and neural network clustering

At present, the field of natural language will also introduce in-depth learning, using the concept of word vector, so that the neural network can also complete the work in the field of statistics. It can be said that the neural network has begun to show its advantages in the field of natural language processing. In this paper, the author analyzes the multimedia English course based on fuzzy statistics and neural network clustering. Different factors were classified, and scores were classified according to the number of characteristics of different categories. It can be seen that with the popularization of the Internet, MOOC teaching meets the requirements of the current college English curriculum, is a breakthrough in the traditional teaching mode, improves students’ participation, and enables students to learn independently. It not only conforms to the characteristics of College students, but also improves their learning effect. In the automatic scoring stage, the quantitative text features are extracted by the feature extractor in the pre-processing stage, and then the weights of network connections obtained in the training stage are used to score the weights comprehensively. This model can better reflect students’ autonomous learning ability and language application ability.

Public opinion analysis of complex network information of local similarity clustering based on intelligent fuzzy system

With the rise of the network society, as the mapping Internet space, the public opinion has become the most active way of expressing social public opinion. It gradually gets deeply involved in the development and change of various social phenomena, social problems and social events, and evolves into the real politics and public management. In this context, it is of great practical significance to explore the evolution process and laws of online public opinions and systematically analyze the influence mechanism in the evolution process of online public opinions. This paper comprehensively uses the modeling simulation, empirical analysis, fuzzy systems and other research methods, adopts the reasonable abstraction of the main behavior characteristics, behavior motives and network relations of network users, and then constructs the evolution model of network public opinion in the complex social network. Besides, from the new research perspective of network members and network relations of the dynamic interaction between the government, media and netizen, this paper makes an in-depth study on the influence mechanism of the dynamic evolution of online public opinion.

Investigations of Object Detection in Images/Videos Using Various Deep Learning Techniques and Embedded Platforms—A Comprehensive Review

In recent years there has been remarkable progress in one computer vision application area: object detection. One of the most challenging and fundamental problems in object detection is locating a specific object from the multiple objects present in a scene. Earlier traditional detection methods were used for detecting the objects with the introduction of convolutional neural networks. From 2012 onward, deep learning-based techniques were used for feature extraction, and that led to remarkable breakthroughs in this area. This paper shows a detailed survey on recent advancements and achievements in object detection using various deep learning techniques. Several topics have been included, such as Viola–Jones (VJ), histogram of oriented gradient (HOG), one-shot and two-shot detectors, benchmark datasets, evaluation metrics, speed-up techniques, and current state-of-art object detectors. Detailed discussions on some important applications in object detection areas, including pedestrian detection, crowd detection, and real-time object detection on Gpu-based embedded systems have been presented. At last, we conclude by identifying promising future directions.

Densely Distilled Flow-Based Knowledge Transfer in Teacher-Student Framework for Image Classification

We propose a new teacherstudent framework (TSF)-based knowledge transfer method, in which knowledge in the form of dense flow across layers is distilled from a pre-trained "teacher" deep neural network (DNN) and transferred to another "student" DNN. In the case of distilled knowledge, multiple overlapped flow-based items of information from the pre-trained teacher DNN are densely extracted across layers. Transference of the densely extracted teacher information is then achieved in the TSF using repetitive sequential training from bottom to top between the teacher and student DNN models. In other words, to efficiently transmit extracted useful teacher information to the student DNN, we perform bottom-up step-by-step transfer of densely distilled knowledge. The performance of the proposed method in terms of image classification accuracy and fast optimization is compared with those of existing TSF-based knowledge transfer methods for application to reliable image datasets, including CIFAR-10, CIFAR-100, MNIST, and SVHN. When the dense flow-based sequential knowledge transfer scheme is employed in the TSF, the trained student ResNet more accurately reflects the rich information of the pre-trained teacher ResNet and exhibits superior accuracy to the existing TSF-based knowledge transfer methods for all benchmark datasets considered in this study.

DeepFHR: intelligent prediction of fetal Acidemia using fetal heart rate signals based on convolutional neural network

Background

Fetal heart rate (FHR) monitoring is a screening tool used by obstetricians to evaluate the fetal state. Because of the complexity and non-linearity, a visual interpretation of FHR signals using common guidelines usually results in significant subjective inter-observer and intra-observer variability. Objective: Therefore, computer aided diagnosis (CAD) systems based on advanced artificial intelligence (AI) technology have recently been developed to assist obstetricians in making objective medical decisions.

Methods

In this work, we present an 8-layer deep convolutional neural network (CNN) framework to automatically predict fetal acidemia. After signal preprocessing, the input 2-dimensional (2D) images are obtained using the continuous wavelet transform (CWT), which provides a better way to observe and capture the hidden characteristic information of the FHR signals in both the time and frequency domains. Unlike the conventional machine learning (ML) approaches, this work does not require the execution of complex feature engineering, i.e., feature extraction and selection. In fact, 2D CNN model can self-learn useful features from the input data with the prerequisite of not losing informative features, representing the tremendous advantage of deep learning (DL) over ML.

Results

Based on the test open-access database (CTU-UHB), after comprehensive experimentation, we achieved better classification performance using the optimal CNN configuration compared to other state-of-the-art methods: the averaged ten-fold cross-validation of the accuracy, sensitivity, specificity, quality index defined as the geometric mean of the sensitivity and specificity, and the area under the curve yielded results of 98.34, 98.22, 94.87, 96.53 and 97.82%, respectively

Conclusions

Once the proposed CNN model is successfully trained, the corresponding CAD system can be served as an effective tool to predict fetal asphyxia objectively and accurately.

Deep Convolutional Neural Network-Based Approaches for Face Recognition

Face recognition (FR) is defined as the process through which people are identified using facial images. This technology is applied broadly in biometrics, security information, accessing controlled areas, keeping of the law by different enforcement bodies, smart cards, and surveillance technology. The facial recognition system is built using two steps. The first step is a process through which the facial features are picked up or extracted, and the second step is pattern classification. Deep learning, specifically the convolutional neural network (CNN), has recently made commendable progress in FR technology. This paper investigates the performance of the pre-trained CNN with multi-class support vector machine (SVM) classifier and the performance of transfer learning using the AlexNet model to perform classification. The study considers CNN architecture, which has so far recorded the best outcome in the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in the past years, more specifically, AlexNet and ResNet-50. In order to determine performance optimization of the CNN algorithm, recognition accuracy was used as a determinant. Improved classification rates were seen in the comprehensive experiments that were completed on the various datasets of ORL, GTAV face, Georgia Tech face, labelled faces in the wild (LFW), frontalized labeled faces in the wild (F_LFW), YouTube face, and FEI faces. The result showed that our model achieved a higher accuracy compared to most of the state-of-the-art models. An accuracy range of 94% to 100% for models with all databases was obtained. Also, this was obtained with an improvement in recognition accuracy up to 39%.

Directly Connected Convolutional Neural Networks

Convolutional neural networks (CNNs) have better performance in feature extraction and classification. Most of the applications are based on a traditional structure of CNNs. However, due to the fixed structure, it may not be effective for large dataset which will spend much time for training. So, we use a new algorithm to optimize CNNs, called directly connected convolutional neural networks (DCCNNs). In DCCNNs, the down-sampling layer can directly connect the output layer with three-dimensional matrix operation, without full connection (i.e., matrix vectorization). Thus, DCCNNs have less weights and neurons than CNNs. We conduct the comparison experiments on five image databases: MNIST, COIL-20, AR, Extended Yale B, and ORL. The experiments show that the model has better recognition accuracy and faster convergence than CNNs. Furthermore, two applications (i.e., water quality evaluation and image classification) following the proposed concepts further confirm the generality and capability of DCCNNs.