Multistability and Instability of Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions

Multistability of State-Dependent Switched Fractional-Order Hopfield Neural Networks With Mexican-Hat Activation Function and Its Application in Associative Memories

The multistability and its application in associative memories are investigated in this article for state-dependent switched fractional-order Hopfield neural networks (FOHNNs) with Mexican-hat activation function (AF). Based on the Brouwer's fixed point theorem, the contraction mapping principle and the theory of fractional-order differential equations, some sufficient conditions are established to ensure the existence, exact existence and local stability of multiple equilibrium points (EPs) in the sense of Filippov, in which the positively invariant sets are also estimated. In particular, the analysis concerning the existence and stability of EPs is quite different from those in the literature because the considered system involves both fractional-order derivative and state-dependent switching. It should be pointed out that, compared with the results in the literature, the total number of EPs and stable EPs increases from and to and , respectively, where with being the system dimension. Besides, a new method is designed to realize associative memories for grayscale and color images by introducing a deviation vector, which, in comparison with the existing works, not only improves the utilization efficiency of EPs, but also reduces the system dimension and computational burden. Finally, the effectiveness of the theoretical results is illustrated by four numerical simulations.

Design of continuous-time recurrent neural networks with piecewise-linear activation function for generation of prescribed sequences of bipolar vectors

A recurrent neural network (RNN) can generate a sequence of patterns as the temporal evolution of the output vector. This paper focuses on a continuous-time RNN model with a piecewise-linear activation function that has neither external inputs nor hidden neurons, and studies the problem of finding the parameters of the model so that it generates a given sequence of bipolar vectors. First, a sufficient condition for the model to generate the desired sequence is derived, which is expressed as a system of linear inequalities in the parameters. Next, three approaches to finding solutions of the system of linear inequalities are proposed: One is formulated as a convex quadratic programming problem and others are linear programming problems. Then, two types of sequences of bipolar vectors that can be generated by the model are presented. Finally, the case where the model generates a periodic sequence of bipolar vectors is considered, and a sufficient condition for the trajectory of the state vector to converge to a limit cycle is provided.

Coexistence and local stability of multiple equilibrium points for fractional-order state-dependent switched competitive neural networks with time-varying delays

This paper investigates the coexistence and local stability of multiple equilibrium points for a class of competitive neural networks with sigmoidal activation functions and time-varying delays, in which fractional-order derivative and state-dependent switching are involved at the same time. Some novel criteria are established to ensure that such n-neuron neural networks can have [Formula: see text] total equilibrium points and [Formula: see text] locally stable equilibrium points with m₁+m₂=n, based on the fixed-point theorem, the definition of equilibrium point in the sense of Filippov, the theory of fractional-order differential equation and Lyapunov function method. The investigation implies that the competitive neural networks with switching can possess greater storage capacity than the ones without switching. Moreover, the obtained results include the multistability results of both fractional-order switched Hopfield neural networks and integer-order switched Hopfield neural networks as special cases, thus generalizing and improving some existing works. Finally, four numerical examples are presented to substantiate the effectiveness of the theoretical analysis.

An Overview of the Stability Analysis of Recurrent Neural Networks With Multiple Equilibria

The stability analysis of recurrent neural networks (RNNs) with multiple equilibria has received extensive interest since it is a prerequisite for successful applications of RNNs. With the increasing theoretical results on this topic, it is desirable to review the results for a systematical understanding of the state of the art. This article provides an overview of the stability results of RNNs with multiple equilibria including complete stability and multistability. First, preliminaries on the complete stability and multistability analysis of RNNs are introduced. Second, the complete stability results of RNNs are summarized. Third, the multistability results of various RNNs are reviewed in detail. Finally, future directions in these interesting topics are suggested.

Multistability of Dynamic Memristor Delayed Cellular Neural Networks With Application to Associative Memories

Recently, dynamic memristor (DM)-cellular neural networks (CNNs) have received widespread attention due to their advantage of low power consumption. The previous works showed that DM-CNNs have at most 318 equilibrium points (EPs) with n=16 cells. Since time delay is unavoidable during the process of information transmission, the goal of this article is to research the multistability of DM-CNNs with time delay, and, meanwhile, to increase the storage capacity of DM-delay (D)CNNs. Depending on the different constitutive relations of memristors, two cases of the multistability for DM-DCNNs are discussed. After determining the constitutive relations, the number of EPs of DM-DCNNs is increased to 3ⁿ with n cells by means of the appropriate state-space decomposition and the Brouwer's fixed point theorem. Furthermore, the enlarged attraction domains of EPs can be obtained, and 2ⁿ of these EPs are locally exponentially stable in two cases. Compared with standard CNNs, the dynamic behavior of DM-DCNNs shows an outstanding merit. That is, the value of voltage and current approach to zero when the system becomes stable, and the memristor provides a nonvolatile memory to store the computation results. Finally, two numerical simulations are presented to illustrate the effectiveness of the theoretical results, and the applications of associative memories are shown at the end of this article.

Analysis and Design of Multivalued High-Capacity Associative Memories Based on Delayed Recurrent Neural Networks

This article aims at analyzing and designing the multivalued high-capacity-associative memories based on recurrent neural networks with both asynchronous and distributed delays. In order to increase storage capacities, multivalued activation functions are introduced into associative memories. The stored patterns are retrieved by external input vectors instead of initial conditions, which can guarantee accurate associative memories by avoiding spurious equilibrium points. Some sufficient conditions are proposed to ensure the existence, uniqueness, and global exponential stability of the equilibrium point of neural networks with mixed delays. For neural networks with n neurons, m -dimensional input vectors, and 2k -valued activation functions, the autoassociative memories have (2k)ⁿ storage capacities and heteroassociative memories have min {(2k)ⁿ,(2k)^m} storage capacities. That is, the storage capacities of designed associative memories in this article are obviously higher than the 2ⁿ and min {2ⁿ,2^m} storage capacities of the conventional ones. Three examples are given to support the theoretical results.

Multistability of Switched Neural Networks With Gaussian Activation Functions Under State-Dependent Switching

This article presents theoretical results on the multistability of switched neural networks with Gaussian activation functions under state-dependent switching. It is shown herein that the number and location of the equilibrium points of the switched neural networks can be characterized by making use of the geometrical properties of Gaussian functions and local linearization based on the Brouwer fixed-point theorem. Four sets of sufficient conditions are derived to ascertain the existence of 7^p₁5^p₂3^p₃ equilibrium points, and 4^p₁3^p₂2^p₃ of them are locally stable, wherein p₁ , p₂ , and p₃ are nonnegative integers satisfying 0 ≤ p₁+p₂+p₃ ≤ n and n is the number of neurons. It implies that there exist up to 7ⁿ equilibria, and up to 4ⁿ of them are locally stable when p₁=n . It also implies that properly selecting p₁ , p₂ , and p₃ can engender a desirable number of stable equilibria. Two numerical examples are elaborated to substantiate the theoretical results.

Multistability and Stabilization of Fractional-Order Competitive Neural Networks With Unbounded Time-Varying Delays

This article investigates the multistability and stabilization of fractional-order competitive neural networks (FOCNNs) with unbounded time-varying delays. By utilizing the monotone operator, several sufficient conditions of the coexistence of equilibrium points (EPs) are obtained for FOCNNs with concave-convex activation functions. And then, the multiple μ -stability of delayed FOCNNs is derived by the analytical method. Meanwhile, several comparisons with existing work are shown, which implies that the derived results cover the inverse-power stability and Mittag-Leffler stability as special cases. Moreover, the criteria on the stabilization of FOCNNs with uncertainty are established by designing a controller. Compared with the results of fractional-order neural networks, the obtained results in this article enrich and improve the previous results. Finally, three numerical examples are provided to show the effectiveness of the presented results.

Exact coexistence and locally asymptotic stability of multiple equilibria for fractional-order delayed Hopfield neural networks with Gaussian activation function

This paper explores the multistability issue for fractional-order Hopfield neural networks with Gaussian activation function and multiple time delays. First, several sufficient criteria are presented for ensuring the exact coexistence of 3ⁿ equilibria, based on the geometric characteristics of Gaussian function, the fixed point theorem and the contraction mapping principle. Then, different from the existing methods used in the multistability analysis of fractional-order neural networks without time delays, it is shown that 2ⁿ of 3ⁿ total equilibria are locally asymptotically stable, by applying the theory of fractional-order linear delayed system and constructing suitable Lyapunov function. The obtained results improve and extend some existing multistability works for classical integer-order neural networks and fractional-order neural networks without time delays. Finally, an illustrative example with comprehensive computer simulations is given to demonstrate the theoretical results.

Periodicity and multi-periodicity generated by impulses control in delayed Cohen-Grossberg-type neural networks with discontinuous activations

This paper discusses the periodicity and multi-periodicity in delayed Cohen-Grossberg-type neural networks (CGNNs) possessing impulsive effects, whose activation functions possess discontinuities and are allowed to be unbounded or nonmonotonic. Based on differential inclusion and cone expansion-compression fixed-point theory of set-valued mapping, several improved criteria are given to derive the positive solution with ω-periodicity and ω-multi-periodicity for delayed CGNNs under impulsive control. These ω-periodicity/ω-multi-periodicity orbits are produced by impulses control. The analytical method and theoretical results presented in this paper are of certain significance to the design of neural network models or circuits possessing discontinuous neuron activation and impulsive effects in periodic environment.

Multistability and associative memory of neural networks with Morita-like activation functions

This paper presents the multistability analysis and associative memory of neural networks (NNs) with Morita-like activation functions. In order to seek larger memory capacity, this paper proposes Morita-like activation functions. In a weakened condition, this paper shows that the NNs with n-neurons have (2m+1)ⁿ equilibrium points (Eps) and (m+1)ⁿ of them are locally exponentially stable, where the parameter m depends on the Morita-like activation functions, called Morita parameter. Also the attraction basins are estimated based on the state space partition. Moreover, this paper applies these NNs into associative memories (AMs). Compared with the previous related works, the number of Eps and AM's memory capacity are extensively increased. The simulation results are illustrated and some reliable associative memories examples are shown at the end of this paper.

Multistability of delayed fractional-order competitive neural networks

This paper is concerned with the multistability of fractional-order competitive neural networks (FCNNs) with time-varying delays. Based on the division of state space, the equilibrium points (EPs) of FCNNs are given. Several sufficient conditions and criteria are proposed to ascertain the multiple O(t^-α)-stability of delayed FCNNs. The O(t^-α)-stability is the extension of Mittag-Leffler stability of fractional-order neural networks, which contains monostability and multistability. Moreover, the attraction basins of the stable EPs of FCNNs are estimated, which shows the attraction basins of the stable EPs can be larger than the divided subsets. These conditions and criteria supplement and improve the previous results. Finally, the results are illustrated by the simulation examples.

Multi-periodicity of switched neural networks with time delays and periodic external inputs under stochastic disturbances

This paper presents new theoretical results on the multi-periodicity of recurrent neural networks with time delays evoked by periodic inputs under stochastic disturbances and state-dependent switching. Based on the geometric properties of activation function and switching threshold, the neuronal state space is partitioned into 5ⁿ regions in which 3ⁿ ones are shown to be positively invariant with probability one. Furthermore, by using Itô's formula, Lyapunov functional method, and the contraction mapping theorem, two criteria are proposed to ascertain the existence and mean-square exponential stability of a periodic orbit in every positive invariant set. As a result, the number of mean-square exponentially stable periodic orbits increases to 3ⁿ from 2ⁿ in a neural network without switching. Two illustrative examples are elaborated to substantiate the efficacy and characteristics of the theoretical results.

Multistability of Fractional-Order Neural Networks With Unbounded Time-Varying Delays

This article addresses the multistability and attraction of fractional-order neural networks (FONNs) with unbounded time-varying delays. Several sufficient conditions are given to ensure the coexistence of equilibrium points (EPs) of FONNs with concave-convex activation functions. Moreover, by exploiting the analytical method and the property of the Mittag-Leffler function, it is shown that the multiple Mittag-Leffler stability of delayed FONNs is derived and the obtained criteria do not depend on differentiable time-varying delays. In particular, the criterion of the Mittag-Leffler stability can be simplified to M-matrix. In addition, the estimation of attraction basin of delayed FONNs is studied, which implies that the extension of attraction basin is independent of the magnitude of delays. Finally, three numerical examples are given to show the validity of the theoretical results.

Monostability and Multistability for Almost-Periodic Solutions of Fractional-Order Neural Networks With Unsaturating Piecewise Linear Activation Functions

Since the unsaturating activation function is unbounded, more complex dynamics may exist in neural networks with this kind of activation function. In this article, monostability and multistability results of almost-periodic solutions are developed for fractional-order neural networks with unsaturating piecewise linear activation functions. Some globally Mittag-Leffler attractive sets are given, and the existence of globally Mittag-Leffler stable almost-periodic solution is demonstrated by using Ascoli-Arzela theorem. In particular, some sufficient conditions are provided to ascertain the multistability of almost-periodic solutions based on locally positively invariant set. It shows that there exists an almost-periodic solution in each positively invariant set, and all trajectories converge to this periodic trajectory in that rectangular area. Two illustrative examples are provided to demonstrate the effectiveness of the proposed sufficient criteria.

On Complete Stability of Recurrent Neural Networks With Time-Varying Delays and General Piecewise Linear Activation Functions

This paper addresses the problem of complete stability of delayed recurrent neural networks with a general class of piecewise linear activation functions. By applying an appropriate partition of the state space and iterating the defined bounding functions, some sufficient conditions are obtained to ensure that an n -neuron neural network is completely stable with exactly ∏_i=1ⁿ(2K_i-1) equilibrium points, among which ∏_i=1ⁿK_i equilibrium points are locally exponentially stable and the others are unstable, where K_i (i=1,…,n) are non-negative integers which depend jointly on activation functions and parameters of neural networks. The results of this paper include the existing works on the stability analysis of recurrent neural networks with piecewise linear functions as special cases and hence can be considered as the improvement and extension of the existing stability results in the literature. A numerical example is provided to illustrate the derived theoretical results.

Multistability and attraction basins of discrete-time neural networks with nonmonotonic piecewise linear activation functions

This paper is concerned with multistability and attraction basins of discrete-time neural networks with nonmonotonic piecewise linear activation functions. Under some reasonable conditions, the addressed networks have (2m+1)ⁿ equilibrium points. (m+1)ⁿ of which are locally asymptotically stable, and the others are unstable. The attraction basins of the locally asymptotically stable equilibrium points are given in the form of hyperspherical regions. These results here, which include existence, uniqueness, locally asymptotical stability, instability and attraction basins of the multiple equilibrium points, generalize and improve the earlier publications. Finally, an illustrative example with numerical simulation is given to show the feasibility and the effectiveness of the theoretical results. The theoretical results and illustrative example indicate that the activation functions improve the storage capacity of neural networks significantly.

Multistability of Switched Neural Networks With Piecewise Linear Activation Functions Under State-Dependent Switching

This paper is concerned with the multistability of switched neural networks with piecewise linear activation functions under state-dependent switching. Under some reasonable assumptions on the switching threshold and activation functions, by using the state-space decomposition method, contraction mapping theorem, and strictly diagonally dominant matrix theory, we can characterize the number of equilibria as well as analyze the stability/instability of the equilibria. More interesting, we can find that the switching threshold plays an important role for stable equilibria in the unsaturation regions of activation functions, and the number of stable equilibria of an n -neuron switched neural network with state-dependent parameters increases to 3ⁿ from 2ⁿ in the conventional one. Furthermore, for two-neuron switched neural networks, the precise attraction basin of each stable equilibrium point can be figured out, and its boundary is composed of the stable manifolds of unstable equilibrium points and the switching lines. Two simulation examples are discussed in detail to substantiate the effectiveness of the theoretical analysis.

Multistability of Delayed Hybrid Impulsive Neural Networks With Application to Associative Memories

The important topic of multistability of continuous-and discrete-time neural network (NN) models has been investigated rather extensively. Concerning the design of associative memories, multistability of delayed hybrid NNs is studied in this paper with an emphasis on the impulse effects. Arising from the spiking phenomenon in biological networks, impulsive NNs provide an efficient model for synaptic interconnections among neurons. Using state-space decomposition, the coexistence of multiple equilibria of hybrid impulsive NNs is analyzed. Multistability criteria are then established regrading delayed hybrid impulsive neurodynamics, for which both the impulse effects on the convergence rate and the basins of attraction of the equilibria are discussed. Illustrative examples are given to verify the theoretical results and demonstrate an application to the design of associative memories. It is shown by an experimental example that delayed hybrid impulsive NNs have the advantages of high storage capacity and high fault tolerance when used for associative memories.

Multistability of Recurrent Neural Networks With Nonmonotonic Activation Functions and Unbounded Time-Varying Delays

This paper is concerned with the coexistence of multiple equilibrium points and dynamical behaviors of recurrent neural networks with nonmonotonic activation functions and unbounded time-varying delays. Based on a state space partition by using the geometrical properties of the activation functions, it is revealed that an -neuron neural network can exhibit equilibrium points with . In particular, several sufficient criteria are proposed to ascertain the asymptotical stability of equilibrium points for recurrent neural networks. These theoretical results cover both monostability and multistability. Furthermore, the attraction basins of asymptotically stable equilibrium points are estimated. It is shown that the attraction basins of the stable equilibrium points can be larger than their originally partitioned subsets. Finally, the results are illustrated by using the simulation results of four examples.

Multistability of Delayed Recurrent Neural Networks with Mexican Hat Activation Functions

This letter studies the multistability analysis of delayed recurrent neural networks with Mexican hat activation function. Some sufficient conditions are obtained to ensure that an [Formula: see text]-dimensional recurrent neural network can have [Formula: see text] equilibrium points with [Formula: see text], and [Formula: see text] of them are locally exponentially stable. Furthermore, the attraction basins of these stable equilibrium points are estimated. We show that the attraction basins of these stable equilibrium points can be larger than their originally partitioned subsets. The results of this letter improve and extend the existing stability results in the literature. Finally, a numerical example containing different cases is given to illustrate the theoretical results.