Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation

NGDE: A Niching-Based Gradient-Directed Evolution Algorithm for Nonconvex Optimization

Nonconvex optimization issues are prevalent in machine learning and data science. While gradient-based optimization algorithms can rapidly converge and are dimension-independent, they may, unfortunately, fall into local optimal solutions or saddle points. In contrast, evolutionary algorithms (EAs) gradually adapt the population of solutions to explore global optimal solutions. However, this approach requires substantial computational resources to perform numerous fitness function evaluations, which poses challenges for high-dimensional optimization in particular. This study introduces a novel nonconvex optimization algorithm, the niching-based gradient-directed evolution (NGDE) algorithm, designed specifically for high-dimensional nonconvex optimization. The NGDE algorithm generates potential solutions and divides them into multiple niches to explore distinct areas within the feasible region. Subsequently, each individual creates candidate offspring using the gradient-directed mutation operator we designed. The convergence properties of the NGDE algorithm are investigated in two scenarios: accessing the full gradient and approximating the gradient with mini-batch samples. The experimental studies demonstrate the superior performance of the NGDE algorithm in minimizing multimodal optimization functions. Additionally, when applied to train the neural networks of LeNet-5, NGDE shows significantly improved classification accuracy, especially in smaller training sizes.

Hyperspectral Anomaly Detection Using Reconstruction Fusion of Quaternion Frequency Domain Analysis

Most existing techniques consider hyperspectral anomaly detection (HAD) as background modeling and anomaly search problems in the spatial domain. In this article, we model the background in the frequency domain and treat anomaly detection as a frequency-domain analysis problem. We illustrate that spikes in the amplitude spectrum correspond to the background, and a Gaussian low-pass filter performing on the amplitude spectrum is equivalent to an anomaly detector. The initial anomaly detection map is obtained by the reconstruction with the filtered amplitude and the raw phase spectrum. To further suppress the nonanomaly high-frequency detailed information, we illustrate that the phase spectrum is critical information to perceive the spatial saliency of anomalies. The saliency-aware map obtained by phase-only reconstruction (POR) is used to enhance the initial anomaly map, which realizes a significant improvement in background suppression. In addition to the standard Fourier transform (FT), we adopt the quaternion FT (QFT) for conducting multiscale and multifeature processing in a parallel way, to obtain the frequency domain representation of the hyperspectral images (HSIs). This helps with robust detection performance. Experimental results on four real HSIs validate the remarkable detection performance and excellent time efficiency of our proposed approach when compared to some state-of-the-art anomaly detection methods.

Self-Paced Co-Training of Graph Neural Networks for Semi-Supervised Node Classification

Graph neural networks (GNNs) have demonstrated great success in many graph data-based applications. The impressive behavior of GNNs typically relies on the availability of a sufficient amount of labeled data for model training. However, in practice, obtaining a large number of annotations is prohibitively labor-intensive and even impossible. Co-training is a popular semi-supervised learning (SSL) paradigm, which trains multiple models based on a common training set while augmenting the limited amount of labeled data used for training each model via the pseudolabeled data generated from the prediction results of other models. Most of the existing co-training works do not control the quality of pseudolabeled data when using them. Therefore, the inaccurate pseudolabels generated by immature models in the early stage of the training process are likely to cause noticeable errors when they are used for augmenting the training data for other models. To address this issue, we propose a self-paced co-training for the GNN (SPC-GNN) framework for semi-supervised node classification. This framework trains multiple GNNs with the same or different structures on different representations of the same training data. Each GNN carries out SSL by using both the originally available labeled data and the augmented pseudolabeled data generated from other GNNs. To control the quality of pseudolabels, a self-paced label augmentation strategy is designed to make the pseudolabels generated at a higher confidence level to be utilized earlier during training such that the negative impact of inaccurate pseudolabels on training data augmentation, and accordingly, the subsequent training process can be mitigated. Finally, each of the trained GNN is evaluated on a validation set, and the best-performing one is chosen as the output. To improve the training effectiveness of the framework, we devise a pretraining followed by a two-step optimization scheme to train GNNs. Experimental results on the node classification task demonstrate that the proposed framework achieves significant improvement over the state-of-the-art SSL methods.

Continuous Control Strategy of Planar 3-Linkage Underactuated Manipulator Based on Broad Neural Network

For the position control of a planar 3-linkage underactuated manipulator (PTUM) with a passive first linkage, a continuous control strategy is developed in this paper. In particular, a broad neural network (BNN)-based model is first established to accurately describe the motion coupling relationship between the passive linkage and the second linkage. Based on this model, by using the particle swarm optimization algorithm, the target angles of all linkages are calculated combining the start states of all linkages and the target position of the PTUM. Then, the target angles of the active linkages are directly achieved by their respective actuators, and that of the passive linkage is also achieved by the rotation of the second linkage. By carrying out several experiments, the effectiveness of the above strategy is verified.