TRA-ACGAN: A motor bearing fault diagnosis model based on an auxiliary classifier generative adversarial network and transformer network

Motor bearing fault diagnosis is essential to guarantee production efficiency and avoid catastrophic accidents. Deep learning-based methods have been developed and widely used for fault diagnosis, and these methods have proven to be very effective in accurately diagnosing bearing faults. In this paper, study the application of generative adversarial networks (GANs) in motor bearing fault diagnosis to address the practical issue of insufficient fault data in industrial testing. Focus on the auxiliary classifier generative adversarial network (ACGAN), and the data expansion is carried out for small datasets. This paper present a novel transformer network and auxiliary classifier generative adversarial network (TRA-ACGAN) for motor bearing fault diagnosis, where the TRA-ACGAN combines an ACGAN with a transformer network to avoid the traditional iterative and convolutional structures. The attention mechanism is fully utilized to extract more effective features, and the dual-task coupling problem encountered in classical ACGANs is avoided. Experimental results with the CWRU dataset and the PU dataset in the field of motor bearing fault diagnosis demonstrate the suitability and superiority of the TRA-ACGAN.

Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications, and Open Issues

Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This article aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we first illuminate EDL from DL and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from data preparation, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues, and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.

Event-Triggered Exponential Stabilization for State-Based Switched Inertial Complex-Valued Neural Networks With Multiple Delays

This article explores the exponential stabilization issue of a class of state-based switched inertial complex-valued neural networks with multiple delays via event-triggered control. First, the state-based switched inertial complex-valued neural networks with multiple delays are modeled. Second, by separating the real and imaginary parts of complex values, the state-based switched inertial complex-valued neural networks are transformed into two state-based switched inertial real-valued neural networks. Through the variable substitution method, the model of the second-order inertial neural networks is transformed into a model of the first-order neural networks. Third, an event-triggered controller with the transmission sequence is designed to study the exponential stabilization issue of neural networks constructed above. Then, by constructing the Lyapunov functions and based on some inequalities, we obtain sufficient conditions for exponential stabilization of the proposed neural networks. Furthermore, it is proved that the Zeno phenomenon cannot happen under the designed event-triggered controller. Finally, a simulation example is given to illustrate the correctness of the results.

On Adaptive Learning Framework for Deep Weighted Sparse Autoencoder: A Multiobjective Evolutionary Algorithm

In this article, an adaptive learning framework is established for a deep weighted sparse autoencoder (AE) by resorting to the multiobjective evolutionary algorithm (MOEA). The weighted sparsity is introduced to facilitate the design of the varying degrees of the sparsity constraints imposed on the hidden units of the AE. The MOEA is exploited to adaptively seek appropriate hyperparameters, where the divide-and-conquer strategy is implemented to enhance the MOEA's performance in the context of deep neural networks. Moreover, a sharing scheme is proposed to further reduce the time complexity of the learning process at the slight expense of the learning precision. It is shown via extensive experiments that the established adaptive learning framework is effective, where different sparse models are utilized to demonstrate the generality of the proposed results. Then, the generality of the proposed framework is examined on the convolutional AE and VGG-16 network. Finally, the developed framework is applied to the blind image quantity assessment that illustrates the applicability of the established algorithms.

Accuracy Versus Simplification in an Approximate Logic Neural Model

An approximate logic neural model (ALNM) is a novel single-neuron model with plastic dendritic morphology. During the training process, the model can eliminate unnecessary synapses and useless branches of dendrites. It will produce a specific dendritic structure for a particular task. The simplified structure of ALNM can be substituted by a logic circuit classifier (LCC) without losing any essential information. The LCC merely consists of the comparator and logic NOT, AND, and OR gates. Thus, it can be easily implemented in hardware. However, the architecture of ALNM affects the learning capacity, generalization capability, computing time and approximation of LCC. Thus, a Pareto-based multiobjective differential evolution (MODE) algorithm is proposed to simultaneously optimize ALNM's topology and weights. MODE can generate a concise and accurate LCC for every specific task from ALNM. To verify the effectiveness of MODE, extensive experiments are performed on eight benchmark classification problems. The statistical results demonstrate that MODE is superior to conventional learning methods, such as the backpropagation algorithm and single-objective evolutionary algorithms. In addition, compared against several commonly used classifiers, both ALNM and LCC are capable of obtaining promising and competitive classification performances on the benchmark problems. Besides, the experimental results also verify that the LCC obtains the faster classification speed than the other classifiers.

A Bilevel Learning Model and Algorithm for Self-Organizing Feed-Forward Neural Networks for Pattern Classification

Conventional artificial neural network (ANN) learning algorithms for classification tasks, either derivative-based optimization algorithms or derivative-free optimization algorithms work by training ANN first (or training and validating ANN) and then testing ANN, which are a two-stage and one-pass learning mechanism. Thus, this learning mechanism may not guarantee the generalization ability of a trained ANN. In this article, a novel bilevel learning model is constructed for self-organizing feed-forward neural network (FFNN), in which the training and testing processes are integrated into a unified framework. In this bilevel model, the upper level optimization problem is built for testing error on testing data set and network architecture based on network complexity, whereas the lower level optimization problem is constructed for network weights based on training error on training data set. For the bilevel framework, an interactive learning algorithm is proposed to optimize the architecture and weights of an FFNN with consideration of both training error and testing error. In this interactive learning algorithm, a hybrid binary particle swarm optimization (BPSO) taken as an upper level optimizer is used to self-organize network architecture, whereas the Levenberg-Marquardt (LM) algorithm as a lower level optimizer is utilized to optimize the connection weights of an FFNN. The bilevel learning model and algorithm have been tested on 20 benchmark classification problems. Experimental results demonstrate that the bilevel learning algorithm can significantly produce more compact FFNNs with more excellent generalization ability when compared with conventional learning algorithms.

Selective motor fascicle transfer and neural-machine interface: case report

An amputated nerve transferred to a nearby muscle produces a transcutaneously detectable electromyographic signal corresponding to the transferred nerve; this technique is known as targeted muscle reinnervation (TMR). There are 2 issues to overcome to improve this technique: the caliber and the selectivity of the transferred nerve. It is optimal to select and transfer each motor fascicle to achieve highly developed myoelectric arms with multiple degrees-of-freedom motion. The authors report on a case in which they first identified the remnant stumps of the amputated median and radial nerves and then identified the sensory fascicles using somatosensory evoked potentials. Each median nerve fascicle was transferred to the long head branch of the biceps or the brachialis branch, while the short head branch of the biceps was retained for elbow flexion. Each radial nerve fascicle was transferred to the medial or lateral head branch of the triceps, while the long head branch of the triceps was retained for elbow extension. Electrophysiological and functional tests were conducted in the reinnervated muscles. Functional and electrophysiological improvement was noted, with marked improvement in the identification rate for each digit, forearm, and elbow motion after the selective nerve transfers. The authors note that more selective nerve transfers may be required for the development of prostheses with multiple degrees of freedom.

An Adaptive-PSO-Based Self-Organizing RBF Neural Network

In this paper, a self-organizing radial basis function (SORBF) neural network is designed to improve both accuracy and parsimony with the aid of adaptive particle swarm optimization (APSO). In the proposed APSO algorithm, to avoid being trapped into local optimal values, a nonlinear regressive function is developed to adjust the inertia weight. Furthermore, the APSO algorithm can optimize both the network size and the parameters of an RBF neural network simultaneously. As a result, the proposed APSO-SORBF neural network can effectively generate a network model with a compact structure and high accuracy. Moreover, the analysis of convergence is given to guarantee the successful application of the APSO-SORBF neural network. Finally, multiple numerical examples are presented to illustrate the effectiveness of the proposed APSO-SORBF neural network. The results demonstrate that the proposed method is more competitive in solving nonlinear problems than some other existing SORBF neural networks.

YamiPred: A Novel Evolutionary Method for Predicting Pre-miRNAs and Selecting Relevant Features

MicroRNAs (miRNAs) are small non-coding RNAs, which play a significant role in gene regulation. Predicting miRNA genes is a challenging bioinformatics problem and existing experimental and computational methods fail to deal with it effectively. We developed YamiPred, an embedded classification method that combines the efficiency and robustness of support vector machines (SVM) with genetic algorithms (GA) for feature selection and parameters optimization. YamiPred was tested in a new and realistic human dataset and was compared with state-of-the-art computational intelligence approaches and the prevalent SVM-based tools for miRNA prediction. Experimental results indicate that YamiPred outperforms existing approaches in terms of accuracy and of geometric mean of sensitivity and specificity. The embedded feature selection component selects a compact feature subset that contributes to the performance optimization. Further experimentation with this minimal feature subset has achieved very high classification performance and revealed the minimum number of samples required for developing a robust predictor. YamiPred also confirmed the important role of commonly used features such as entropy and enthalpy, and uncovered the significance of newly introduced features, such as %A-U aggregate nucleotide frequency and positional entropy. The best model trained on human data has successfully predicted pre-miRNAs to other organisms including the category of viruses.

Evolutionary fuzzy ARTMAP neural networks for classification of semiconductor defects

Wafer defect detection using an intelligent system is an approach of quality improvement in semiconductor manufacturing that aims to enhance its process stability, increase production capacity, and improve yields. Occasionally, only few records that indicate defective units are available and they are classified as a minority group in a large database. Such a situation leads to an imbalanced data set problem, wherein it engenders a great challenge to deal with by applying machine-learning techniques for obtaining effective solution. In addition, the database may comprise overlapping samples of different classes. This paper introduces two models of evolutionary fuzzy ARTMAP (FAM) neural networks to deal with the imbalanced data set problems in a semiconductor manufacturing operations. In particular, both the FAM models and hybrid genetic algorithms are integrated in the proposed evolutionary artificial neural networks (EANNs) to classify an imbalanced data set. In addition, one of the proposed EANNs incorporates a facility to learn overlapping samples of different classes from the imbalanced data environment. The classification results of the proposed evolutionary FAM neural networks are presented, compared, and analyzed using several classification metrics. The outcomes positively indicate the effectiveness of the proposed networks in handling classification problems with imbalanced data sets.

Quantum-based algorithm for optimizing artificial neural networks

This paper presents a quantum-based algorithm for evolving artificial neural networks (ANNs). The aim is to design an ANN with few connections and high classification performance by simultaneously optimizing the network structure and the connection weights. Unlike most previous studies, the proposed algorithm uses quantum bit representation to codify the network. As a result, the connectivity bits do not indicate the actual links but the probability of the existence of the connections, thus alleviating mapping problems and reducing the risk of throwing away a potential candidate. In addition, in the proposed model, each weight space is decomposed into subspaces in terms of quantum bits. Thus, the algorithm performs a region by region exploration, and evolves gradually to find promising subspaces for further exploitation. This is helpful to provide a set of appropriate weights when evolving the network structure and to alleviate the noisy fitness evaluation problem. The proposed model is tested on four benchmark problems, namely breast cancer and iris, heart, and diabetes problems. The experimental results show that the proposed algorithm can produce compact ANN structures with good generalization ability compared to other algorithms.

Neural-fitted TD-leaf learning for playing Othello with structured neural networks

This paper describes a methodology for quickly learning to play games at a strong level. The methodology consists of a novel combination of three techniques, and a variety of experiments on the game of Othello demonstrates their usefulness. First, structures or topologies in neural network connectivity patterns are used to decrease the number of learning parameters and to deal more effectively with the structural credit assignment problem, which is to change individual network weights based on the obtained feedback. Furthermore, the structured neural networks are trained with the novel neural-fitted temporal difference (TD) learning algorithm to create a system that can exploit most of the training experiences and enhance learning speed and performance. Finally, we use the neural-fitted TD-leaf algorithm to learn more effectively when look-ahead search is performed by the game-playing program. Our extensive experimental study clearly indicates that the proposed method outperforms linear networks and fully connected neural networks or evaluation functions evolved with evolutionary algorithms.

Performance evaluation of multilayer perceptrons for discriminating and quantifying multiple kinds of odors with an electronic nose

This paper studies several types and arrangements of perceptron modules to discriminate and quantify multiple odors with an electronic nose. We evaluate the following types of multilayer perceptron. (A) A single multi-output (SMO) perceptron both for discrimination and for quantification. (B) An SMO perceptron for discrimination followed by multiple multi-output (MMO) perceptrons for quantification. (C) An SMO perceptron for discrimination followed by multiple single-output (MSO) perceptrons for quantification. (D) MSO perceptrons for discrimination followed by MSO perceptrons for quantification, called the MSO-MSO perceptron model, under the following conditions: (D1) using a simple one-against-all (OAA) decomposition method; (D2) adopting a simple OAA decomposition method and virtual balance step; and (D3) employing a local OAA decomposition method, virtual balance step and local generalization strategy all together. The experimental results for 12 kinds of volatile organic compounds at 85 concentration levels in the training set and 155 concentration levels in the test set show that the MSO-MSO perceptron model with the D3 learning procedure is the most effective of those tested for discrimination and quantification of many kinds of odors.