Multiple and Complete Stability of Recurrent Neural Networks With Sinusoidal Activation Function

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly $(2m + 3)^{n}$ equilibria, where $(m + 2)^{n}$ equilibria are locally exponentially stable, the remainder $(2m + 3)^{n} - (m + 2)^{n}$ equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.

Stability Analysis of Fractional Bidirectional Associative Memory Neural Networks With Multiple Proportional Delays and Distributed Delays

This article investigates the finite-time stability of a class of fractional-order bidirectional associative memory neural networks (FOBAMNNs) with multiple proportional and distributed delays. Different from the existing Gronwall integral inequality with single proportional delay ( ), we establish the Gronwall integral inequality with multiple proportional delays for the first time in the case of . Since the existing fractional-order single-constant delay Gronwall inequality with two different orders cannot be directly applied to the stability analysis of the aforementioned system, initially, we skillfully develop a novel one with generalized fractional multiproportional delays' Gronwall inequalities of different orders. Furthermore, combined with the newly constructed generalized inequality, the stability criteria of FOBAMNNs with fractional orders and under weaker conditions, i.e., at most linear growth and linear growth conditions rather than the global Lipschitz condition, are given respectively. Finally, numerical experiments verify the effectiveness of the proposed method.

Coexistence and local stability of multiple equilibrium points for fractional-order state-dependent switched competitive neural networks with time-varying delays

This paper investigates the coexistence and local stability of multiple equilibrium points for a class of competitive neural networks with sigmoidal activation functions and time-varying delays, in which fractional-order derivative and state-dependent switching are involved at the same time. Some novel criteria are established to ensure that such n-neuron neural networks can have [Formula: see text] total equilibrium points and [Formula: see text] locally stable equilibrium points with m₁+m₂=n, based on the fixed-point theorem, the definition of equilibrium point in the sense of Filippov, the theory of fractional-order differential equation and Lyapunov function method. The investigation implies that the competitive neural networks with switching can possess greater storage capacity than the ones without switching. Moreover, the obtained results include the multistability results of both fractional-order switched Hopfield neural networks and integer-order switched Hopfield neural networks as special cases, thus generalizing and improving some existing works. Finally, four numerical examples are presented to substantiate the effectiveness of the theoretical analysis.

Predefined-time synchronization of coupled neural networks with switching parameters and disturbed by Brownian motion

This article focuses on predefined time synchronization problem for a class of signal switching neural networks with time-varying delays. In the network models, we not only consider the coupling characteristics in the following networks, but also consider the disturbance with standard Brownian motion. In the design of the controller, the control gain is designed as 1ɛ+T_p-t (t∈[T₀,T_p), ɛ is an optional smaller positive number), which avoids the infinite gain (the control gain is designed as 1T_p-t in other reference). In order to get the predefined time control law, a power function is multiplied to the Lyapunov functional, from which it can get an exponential upper bound function via the derivative and mathematical expectation operation. Utilizing the martingale theory and the method of Laplace matrix, some novel predefined time synchronization criteria are obtained for the leader-following neural networks, meanwhile the following networks can maintain the leader network after achieved synchronization. Based on the special network of the main system, five corollaries separately develop the predefined time synchronization results from different perspectives. An example with some simulation figures and computing results fully exhibits the effectiveness of the achieved synchronization scheme. In this case, although the error signal is disturbed by Brownian motion, the trace signal can still stably converge to zero by this control scheme, meanwhile the predefined-time control effect is achieved.

An Overview of the Stability Analysis of Recurrent Neural Networks With Multiple Equilibria

The stability analysis of recurrent neural networks (RNNs) with multiple equilibria has received extensive interest since it is a prerequisite for successful applications of RNNs. With the increasing theoretical results on this topic, it is desirable to review the results for a systematical understanding of the state of the art. This article provides an overview of the stability results of RNNs with multiple equilibria including complete stability and multistability. First, preliminaries on the complete stability and multistability analysis of RNNs are introduced. Second, the complete stability results of RNNs are summarized. Third, the multistability results of various RNNs are reviewed in detail. Finally, future directions in these interesting topics are suggested.

Coarse-to-Fine: Progressive Knowledge Transfer-Based Multitask Convolutional Neural Network for Intelligent Large-Scale Fault Diagnosis

In modern industry, large-scale fault diagnosis of complex systems is emerging and becoming increasingly important. Most deep learning-based methods perform well on small number of fault diagnosis, but cannot converge to satisfactory results when handling large-scale fault diagnosis because the huge number of fault types will lead to the problems of intra/inter-class distance unbalance and poor local minima in neural networks. To address the above problems, a progressive knowledge transfer-based multitask convolutional neural network (PKT-MCNN) is proposed. First, to construct the coarse-to-fine knowledge structure intelligently, a structure learning algorithm is proposed via clustering fault types in different coarse-grained nodes. Thus, the intra/inter-class distance unbalance problem can be mitigated by spreading similar tasks into different nodes. Then, an MCNN architecture is designed to learn the coarse and fine-grained task simultaneously and extract more general fault information, thereby pushing the algorithm away from poor local minima. Last but not least, a PKT algorithm is proposed, which can not only transfer the coarse-grained knowledge to the fine-grained task and further alleviate the intra/inter-class distance unbalance in feature space, but also regulate different learning stages by adjusting the attention weight to each task progressively. To verify the effectiveness of the proposed method, a dataset of a nuclear power system with 66 fault types was collected and analyzed. The results demonstrate that the proposed method can be a promising tool for large-scale fault diagnosis.

Estimation of Mechanical Power Output Employing Deep Learning on Inertial Measurement Data in Roller Ski Skating

The ability to optimize power generation in sports is imperative, both for understanding and balancing training load correctly, and for optimizing competition performance. In this paper, we aim to estimate mechanical power output by employing a time-sequential information-based deep Long Short-Term Memory (LSTM) neural network from multiple inertial measurement units (IMUs). Thirteen athletes conducted roller ski skating trials on a treadmill with varying incline and speed. The acceleration and gyroscope data collected with the IMUs were run through statistical feature processing, before being used by the deep learning model to estimate power output. The model was thereafter used for prediction of power from test data using two approaches. First, a user-dependent case was explored, reaching a power estimation within 3.5% error. Second, a user-independent case was developed, reaching an error of 11.6% for the power estimation. Finally, the LSTM model was compared to two other machine learning models and was found to be superior. In conclusion, the user-dependent model allows for precise estimation of roller skiing power output after training the model on data from each athlete. The user-independent model provides less accurate estimation; however, the accuracy may be sufficient for providing valuable information for recreational skiers.

Multi-periodicity of switched neural networks with time delays and periodic external inputs under stochastic disturbances

This paper presents new theoretical results on the multi-periodicity of recurrent neural networks with time delays evoked by periodic inputs under stochastic disturbances and state-dependent switching. Based on the geometric properties of activation function and switching threshold, the neuronal state space is partitioned into 5ⁿ regions in which 3ⁿ ones are shown to be positively invariant with probability one. Furthermore, by using Itô's formula, Lyapunov functional method, and the contraction mapping theorem, two criteria are proposed to ascertain the existence and mean-square exponential stability of a periodic orbit in every positive invariant set. As a result, the number of mean-square exponentially stable periodic orbits increases to 3ⁿ from 2ⁿ in a neural network without switching. Two illustrative examples are elaborated to substantiate the efficacy and characteristics of the theoretical results.