A pruning feedforward small-world neural network based on Katz centrality for nonlinear system modeling

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling

It is well-documented that cross-layer connections in feedforward small-world neural networks (FSWNNs) enhance the efficient transmission for gradients, thus improving its generalization ability with a fast learning. However, the merits of long-distance cross-layer connections are not fully utilized due to the random rewiring. In this study, aiming to further improve the learning efficiency, a fast FSWNN (FFSWNN) is proposed by taking into account the positive effects of long-distance cross-layer connections, and applied to nonlinear system modeling. First, a novel rewiring rule by giving priority to long-distance cross-layer connections is proposed to increase the gradient transmission efficiency when constructing FFSWNN. Second, an improved ridge regression method is put forward to determine the initial weights with high activation for the sigmoidal neurons in FFSWNN. Finally, to further improve the learning efficiency, an asynchronous learning algorithm is designed to train FFSWNN, with the weights connected to the output layer updated by the ridge regression method and other weights by the gradient descent method. Several experiments are conducted on four benchmark datasets from the University of California Irvine (UCI) machine learning repository and two datasets from real-life problems to evaluate the performance of FFSWNN on nonlinear system modeling. The results show that FFSWNN has significantly faster convergence speed and higher modeling accuracy than the comparative models, and the positive effects of the novel rewiring rule, the improved weight initialization, and the asynchronous learning algorithm on learning efficiency are demonstrated.

Synchronization in fractional-order neural networks by the energy balance strategy

Considering the individual differences between neurons, the fractional-order framework is introduced, and the neurons with various orders denote the individual differences during the cell differentiation. In this paper, the fractional-order FithzHugh-Nagumo (FHN) neural circuit is used to reproduce the firing patterns. In addition, an energy balance strategy is applied to determine the inter-neuronal communication. The neurons with energy imbalance exchange the information whereas the synaptic channels are blocked when energy balance is achieved. Two neurons coupled by this strategy achieve the phase synchronization and phase lock, and it indicates the two neurons generate spiking at the same time or with an interval. Similarly, the synchronization results are also obtained in the chain neuronal network, and the neurons exhibit the same firing patterns since the synchronization factor is closed to 1. Particularly, the neurons with order diversities lead to the heterogeneity and gradient field in the regular network, and the target wave is developed over time. With the wave spreading in the network, the silent states and exciting states appear in the whole network. The formation and diffusion of the target wave reveals the information transmission in neuronal network, and it indicates the individual differences paly an essential role in the collective behavior of neurons.