On equilibria, stability, and instability of Hopfield neural networks

Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization

In this article, we investigate the routing problem of packet networks through multiagent reinforcement learning (RL), which is a very challenging topic in distributed and autonomous networked systems. In specific, the routing problem is modeled as a networked multiagent partially observable Markov decision process (MDP). Since the MDP of a network node is not only affected by its neighboring nodes' policies but also the network traffic demand, it becomes a multitask learning problem. Inspired by recent success of RL and metalearning, we propose two novel model-free multiagent RL algorithms, named multiagent proximal policy optimization (MAPPO) and multiagent metaproximal policy optimization (meta-MAPPO), to optimize the network performances under fixed and time-varying traffic demand, respectively. A practicable distributed implementation framework is designed based on the separability of exploration and exploitation in training MAPPO. Compared with the existing routing optimization policies, our simulation results demonstrate the excellent performances of the proposed algorithms.

Time-delay estimation based computed torque control with robust adaptive RBF neural network compensator for a rehabilitation exoskeleton

A new approach to gait rehabilitation task of a 12 DOF lower limb exoskeleton is proposed combining time-delay estimation (TDE) based computed torque control (CTC) and robust adaptive RBF neural networks. In addition to the conventional advantages of the CTC, TDE technique is integrated to estimate unmodeled dynamics and external disturbance. To realize more accurate tracking, a robust adaptive RBF neural networks compensator is designed to approximate and compensate TDE error. The final asymptotic stability is guaranteed with Lyapunov criteria. To validate the proposed approach, co-simulation experiments are realized using SolidWorks, SimMechanics and MATLAB/Robotics Toolbox. Compared to CTC, sliding mode based CTC and TDE based CTC, the higher performances of the proposed controller are demonstrated by co-simulation.

Multistability and Bifurcation Analysis of Inhibitory Coupled Cyclic Genetic Regulatory Networks With Delays

Many biological systems have the conspicuous property to present more than one stable state and diverse rhythmic behaviors. A closed relationship between these complex dynamic behaviors and cyclic genetic structures has been witnessed by pioneering works. In this paper, a typical structure of inhibitory coupled cyclic genetic networks is introduced to further enlighten this mechanism of stability and biological rhythms of living cells. The coupled networks consist of two identical cyclic genetic subnetworks, which inhibit each other directly. Each subnetwork can be regarded as a genetic unit at the cellular level. Multiple time delays, including both internal and coupling delays, are considered. The existence of positive equilibriums for this kind of coupled systems is proved, and the stability for each equilibrium is analyzed without or with delays. It is shown that the coupled networks with positive cyclic genetic units have an ability to show multistability, while the coupled networks with negative units may present a series of Hopf bifurcations with the variation of time delays. Several numerical simulations are made to prove our results.

The general critical analysis for continuous-time UPPAM recurrent neural networks

The uniformly pseudo-projection-anti-monotone (UPPAM) neural network model, which can be considered as the unified continuous-time neural networks (CNNs), includes almost all of the known CNNs individuals. Recently, studies on the critical dynamics behaviors of CNNs have drawn special attentions due to its importance in both theory and applications. In this paper, we will present the analysis of the UPPAM network under the general critical conditions. It is shown that the UPPAM network possesses the global convergence and asymptotical stability under the general critical conditions if the network satisfies one quasi-symmetric requirement on the connective matrices, which is easy to be verified and applied. The general critical dynamics have rarely been studied before, and this work is an attempt to gain an meaningful assurance of general critical convergence and stability of CNNs. Since UPPAM network is the unified model for CNNs, the results obtained here can generalize and extend the existing critical conclusions for CNNs individuals, let alone those non-critical cases. Moreover, the easily verified conditions for general critical convergence and stability can further promote the applications of CNNs.

A general backpropagation algorithm for feedforward neural networks learning

A general backpropagation algorithm is proposed for feedforward neural network learning with time varying inputs. The Lyapunov function approach is used to rigorously analyze the convergence of weights, with the use of the algorithm, toward minima of the error function. Sufficient conditions to guarantee the convergence of weights for time varying inputs are derived. It is shown that most commonly used backpropagation learning algorithms are special cases of the developed general algorithm.

Estimating the ultimate bound and positively invariant set for a class of Hopfield networks

In this paper, we investigate the ultimate bound and positively invariant set for a class of Hopfield neural networks (HNNs) based on the Lyapunov stability criterion and Lagrange multiplier method. It is shown that a hyperelliptic estimate of the ultimate bound and positively invariant set for the HNNs can be calculated by solving a linear matrix inequality (LMI). Furthermore, the global stability of the unique equilibrium and the instability region of the HNNs are analyzed, respectively. Finally, the most accurate estimate of the ultimate bound and positively invariant set can be derived by solving the corresponding optimization problems involving the LMI constraints. Some numerical examples are given to illustrate the effectiveness of the proposed results.

A comment on "On equilibria, stability, and instability of Hopfield neural networks"

In this letter, it is pointed out that the main analysis results about the existence, uniqueness, and global asymptotic stability of the equilibrium of a continuous-time Hopfield type neural network given in the recent paper are special cases of relevant ones previously obtained in the literature.

A reference model approach to stability analysis of neural networks

In this paper, a novel methodology called a reference model approach to stability analysis of neural networks is proposed. The core of the new approach is to study a neural network model with reference to other related models, so that different modeling approaches can be combinatively used and powerfully cross-fertilized. Focused on two representative neural network modeling approaches (the neuron state modeling approach and the local field modeling approach), we establish a rigorous theoretical basis on the feasibility and efficiency of the reference model approach. The new approach has been used to develop a series of new, generic stability theories for various neural network models. These results have been applied to several typical neural network systems including the Hopfield-type neural networks, the recurrent back-propagation neural networks, the BSB-type neural networks, the bound-constraints optimization neural networks, and the cellular neural networks. The results obtained unify, sharpen or generalize most of the existing stability assertions, and illustrate the feasibility and power of the new method.

Stability and Hopf Bifurcation in a Simplified BAM Neural Network With Two Time Delays

Various local periodic solutions may represent different classes of storage patterns or memory patterns, and arise from the different equilibrium points of neural networks (NNs) by applying Hopf bifurcation technique. In this paper, a bidirectional associative memory NN with four neurons and multiple delays is considered. By applying the normal form theory and the center manifold theorem, analysis of its linear stability and Hopf bifurcation is performed. An algorithm is worked out for determining the direction and stability of the bifurcated periodic solutions. Numerical simulation results supporting the theoretical analysis are also given.

Dynamics of winner-take-all competition in recurrent neural networks with lateral inhibition

This paper studies the behavior of recurrent neural networks with lateral inhibition. Such network architecture is important in biological neural systems. General conditions determining the existence, number, and stability of network equilibria are derived. The manner in which these features depend upon steepness of neuronal activation functions and the strength of lateral inhibition is demonstrated for a broad range of nondecreasing activation functions including the discontinuous threshold function which represents the infinite gain limit. For uniform lateral inhibitory networks, the lateral inhibition is shown to sharpen neuron output patterns by increasing separation of suprathreshold activity levels of competing neurons. This results in the tendency of one neuron's output to dominate those of the others which can afford a "winner-take-all" (WTA) mechanism. Importantly, multiple stable equilibria may exist and shifts in inputs levels may yield network state transitions that exhibit hysteresis. A limitation of using lateral inhibition to implement WTA is further demonstrated. The possible significance of these identified network dynamics to physiology and pathophysiology of the striatum (particularly in Parkinsonian rest tremor) is discussed.

On stability of recurrent neural networks--an approach from volterra integro-differential equations

The uniform asymptotic stability of recurrent neural networks (RNNs) with distributed delay is analyzed by comparing RNNs to linear Volterra integro-differential systems under Lipschitz continuity of activation functions. The stability criteria obtained have unified and extended many existing results on RNNs.

Some generalized sufficient convergence criteria for nonlinear continuous neural networks

A reason for applying the direct method of Lyapunov to artificial neural networks (ANNs) is to design dynamical neural networks so that they exhibit global asymptotic stability. Lyapunov functions that frequently appear in the ANN literature include the quadratic function, the Persidskii function, and the Luré-Postnikov function. This contribution revisits the quadratic function and shows that via Krasovskii-like stability criteria, it is possible to have a very simple and systematic procedure to obtain not only new and generalized results but also well-known sufficient conditions for convergence established recently by non-Lyapunov methods, such as the matrix measure and nonlinear measure.

A comparative study of two modeling approaches in neural networks

The neuron state modeling and the local field modeling provides two fundamental modeling approaches to neural network research, based on which a neural network system can be called either as a static neural network model or as a local field neural network model. These two models are theoretically compared in terms of their trajectory transformation property, equilibrium correspondence property, nontrivial attractive manifold property, global convergence as well as stability in many different senses. The comparison reveals an important stability invariance property of the two models in the sense that the stability (in any sense) of the static model is equivalent to that of a subsystem deduced from the local field model when restricted to a specific manifold. Such stability invariance property lays a sound theoretical foundation of validity of a useful, cross-fertilization type stability analysis methodology for various neural network models.

Dynamics of periodic delayed neural networks

This paper formulates and studies a model of periodic delayed neural networks. This model can well describe many practical architectures of delayed neural networks, which is generalization of some additive delayed neural networks such as delayed Hopfield neural networks and delayed cellular neural networks, under a time-varying environment, particularly when the network parameters and input stimuli are varied periodically with time. Without assuming the smoothness, monotonicity and boundedness of the activation functions, the two functional issues on neuronal dynamics of this periodic networks, i.e. the existence and global exponential stability of its periodic solutions, are investigated. Some explicit and conclusive results are established, which are natural extension and generalization of the corresponding results existing in the literature. Furthermore, some examples and simulations are presented to illustrate the practical nature of the new results.

Exponential stability of Cohen-Grossberg neural networks

Exponential stabilities of the Cohen-Grossberg neural network with and without delays are analyzed. By Liapunov functions/functionals, sufficient conditions are obtained for general exponential stability, while by using a comparison result from the theory of monotone dynamical systems, componentwise exponential stability is also discussed. All results are established without assuming any symmetry of the connection matrix, and the differentiability and monotonicity of the activation functions.