Multistability of Recurrent Neural Networks With Time-varying Delays and the Piecewise Linear Activation Function

Finite-time H∞ output synchronization for DCRDNNs with multiple delayed and adaptive output couplings

This work concentrates on solving the finite-time H∞ output synchronization (FTHOS) issue of directed coupled reaction-diffusion neural networks (DCRDNNs) with multiple delayed and adaptive output couplings in the presence of external disturbances. Based on the output information, an adaptive law to adjust output coupling weights and a controller are respectively developed to ensure that the DCRDNNs achieve FTHOS. Then, in the special case of no external disturbances, a corollary on the finite-time output synchronization (FTOS) of the DCRDNNs with multiple delayed and adaptive output couplings is provided. In addition, a novel adaptive scheme to update output coupling weights is devised to ensure H∞ output synchronization (HOS) in the DCRDNNs with multiple delayed output couplings. Finally, the relevant simulation graphs are provided to certify the validity of several synchronization criteria.

Iterative neural networks for improving memory capacity

In recent years, the problem of the multistability of neural networks has been studied extensively. From the research results obtained, the number of stable equilibrium points depends only on a power form of the network dimension. However, in practical applications, the number of stable equilibrium points needed is often not expressed in power form. Therefore, can we determine an appropriate activation function so that the neural network has exactly the required number of stable equilibrium points? This paper provides a new way to study this problem by means of an iteration method. The necessary activation function is constructed by an appropriate iteration method, and the neural network model is established. Based on the mathematical theories of matrix analysis and functional analysis and on the inequality method, the number and distribution of the network equilibrium points are determined by dividing the state space reasonably, and some multistability criteria that are related to the number of iterations and are independent of the network dimension are established.

Interval Bipartite Synchronization of Multiple Neural Networks in Signed Graphs

Interval bipartite consensus of multiagents described by signed graphs has received extensive concern recently, and the rooted cycles play a critical role in stabilization, while the structurally balanced graphs are essential to achieve bipartite consensus. However, the gauge transformation used in the linear system is no longer feasible in the nonlinear case. This article addresses interval bipartite synchronization of multiple neural networks (NNs) in a signed graph via a Lyapunov-based approach, extending the existing work to a more practical but complicated case. A general matrix M in signed graphs is introduced to construct the novel Lyapunov functions, and sufficient conditions are obtained. We find that the rooted cycles and the structurally balanced graphs are essential to stabilize and achieve bipartite synchronization. More importantly, we discover that the nonrooted cycles are crucial in reaching interval bipartite synchronization, not previously mentioned. Several examples are presented to illustrate interval bipartite synchronization of multiple NNs with signed graphs.

Analysis and Design of Multivalued High-Capacity Associative Memories Based on Delayed Recurrent Neural Networks

This article aims at analyzing and designing the multivalued high-capacity-associative memories based on recurrent neural networks with both asynchronous and distributed delays. In order to increase storage capacities, multivalued activation functions are introduced into associative memories. The stored patterns are retrieved by external input vectors instead of initial conditions, which can guarantee accurate associative memories by avoiding spurious equilibrium points. Some sufficient conditions are proposed to ensure the existence, uniqueness, and global exponential stability of the equilibrium point of neural networks with mixed delays. For neural networks with n neurons, m -dimensional input vectors, and 2k -valued activation functions, the autoassociative memories have (2k)ⁿ storage capacities and heteroassociative memories have min {(2k)ⁿ,(2k)^m} storage capacities. That is, the storage capacities of designed associative memories in this article are obviously higher than the 2ⁿ and min {2ⁿ,2^m} storage capacities of the conventional ones. Three examples are given to support the theoretical results.

Multimode function multistability for Cohen-Grossberg neural networks with mixed time delays

In this paper, we are concerned with the multimode function multistability for Cohen-Grossberg neural networks (CGNNs) with mixed time delays. It is introduced the multimode function multistability as well as its specific mathematical expression, which is a generalization of multiple exponential stability, multiple polynomial stability, multiple logarithmic stability, and asymptotic stability. Also, according to the neural network (NN) model and the maximum and minimum values of activation functions, n pairs of upper and lower boundary functions are obtained. Via the locations of the zeros of the n pairs of upper and lower boundary functions, the state space is divided into ∏_i=1ⁿ(2H_i+1) parts correspondingly. By virtue of the reduction to absurdity, continuity of function, Brouwer's fixed point theorem and Lyapunov stability theorem, the criteria for multimode function multistability are acquired. Multiple types of multistability, including multiple exponential stability, multiple polynomial stability, multiple logarithmic stability, and multiple asymptotic stability, can be achieved by selecting different types of function P(t). Two numerical examples are offered to substantiate the generality of the obtained criteria over the existing results.

Distributed Adaptive Containment Control for Coupled Reaction-Diffusion Neural Networks With Directed Topology

In this article, we consider the problem of distributed adaptive leader-follower coordination of partial differential systems (i.e., reaction-diffusion neural networks, RDNNs) with directed communication topology in the case of multiple leaders. Different from the dynamical networks with ordinary differential dynamics, the design of adaptive protocols is more difficult due to the existence of spatial variables and nonlinear terms in the model. Under directed networks, a novel adaptive control protocol is proposed to solve the containment control problem of RDNNs. By constructing proper Lyapunov functional and adopting some important prior knowledge, the stability of containment for coupled RDNNs is theoretically proved. Furthermore, a corollary about the leader-follower synchronization with a leader for coupled RDNNs with directed communication topology is given. In the end, two numerical examples are provided to illustrate the obtained theoretical results.

Local Lagrange Exponential Stability Analysis of Quaternion-Valued Neural Networks with Time Delays

This study on the local stability of quaternion-valued neural networks is of great significance to the application of associative memory and pattern recognition. In the research, we study local Lagrange exponential stability of quaternion-valued neural networks with time delays. By separating the quaternion-valued neural networks into a real part and three imaginary parts, separating the quaternion field into 34n subregions, and using the intermediate value theorem, sufficient conditions are proposed to ensure quaternion-valued neural networks have 34n equilibrium points. According to the Halanay inequality, the conditions for the existence of 24n local Lagrange exponentially stable equilibria of quaternion-valued neural networks are established. The obtained stability results improve and extend the existing ones. Under the same conditions, quaternion-valued neural networks have more stable equilibrium points than complex-valued neural networks and real-valued neural networks. The validity of the theoretical results were verified by an example.

Discrete analogue of impulsive recurrent neural networks with both discrete and finite distributive asynchronous time-varying delays

This paper studies the dynamical characteristics of discrete analogue of impulsive recurrent neural networks with both discrete and finite distributed asynchronous time-varying delays. Firstly, the discrete impulsive system of the corresponding continuous-time model is reformed by impulsive maps and semi-discrete method. Secondly, by employing a famous delay impulsive differential inequality, several novel sufficient conditions are derived to ensure the uniqueness of equilibrium point and its global exponential stability in Lagrange sense for the discussed discrete-time impulsive system. Meanwhile, it is illustrated that the discrete-time analogue retains the uniqueness of equilibrium point of the corresponding continuous-time model, and some corollaries follow. Finally, one example is given to demonstrate the validity of our obtained results.

Quasisynchronization of Heterogeneous Neural Networks With Time-Varying Delays via Event-Triggered Impulsive Controls

Time delays are unavoidable since they are ubiquitous and may have a great impact on the performance of neural networks. Resources efficiency is a common concern in many networked systems with limited resources. This article investigates quasisynchronization of the heterogeneous neural networks with time-varying delays via event-triggered impulsive controls which combine the impulsive control and the event-triggered technique. The centralized and distributed event-triggered impulsive controls are, respectively, presented. The suitable Lyapunov functions are constructed, and the triggering functions are derived, which guarantee that not only are the synchronization errors less than a non-negative bound but also the Zeno behaviors can be eliminated. It is suggested that the distributed one has great superiority in taking up fewer resources compared with the time-triggered impulsive control. Numerical examples are proposed to verify the validity of the centralized and distributed control methods.

Stimulus-Driven and Spontaneous Dynamics in Excitatory-Inhibitory Recurrent Neural Networks for Sequence Representation

Recurrent neural networks (RNNs) have been widely used to model sequential neural dynamics ("neural sequences") of cortical circuits in cognitive and motor tasks. Efforts to incorporate biological constraints and Dale's principle will help elucidate the neural representations and mechanisms of underlying circuits. We trained an excitatory-inhibitory RNN to learn neural sequences in a supervised manner and studied the representations and dynamic attractors of the trained network. The trained RNN was robust to trigger the sequence in response to various input signals and interpolated a time-warped input for sequence representation. Interestingly, a learned sequence can repeat periodically when the RNN evolved beyond the duration of a single sequence. The eigenspectrum of the learned recurrent connectivity matrix with growing or damping modes, together with the RNN's nonlinearity, were adequate to generate a limit cycle attractor. We further examined the stability of dynamic attractors while training the RNN to learn two sequences. Together, our results provide a general framework for understanding neural sequence representation in the excitatory-inhibitory RNN.

On Complete Stability of Recurrent Neural Networks With Time-Varying Delays and General Piecewise Linear Activation Functions

This paper addresses the problem of complete stability of delayed recurrent neural networks with a general class of piecewise linear activation functions. By applying an appropriate partition of the state space and iterating the defined bounding functions, some sufficient conditions are obtained to ensure that an n -neuron neural network is completely stable with exactly ∏_i=1ⁿ(2K_i-1) equilibrium points, among which ∏_i=1ⁿK_i equilibrium points are locally exponentially stable and the others are unstable, where K_i (i=1,…,n) are non-negative integers which depend jointly on activation functions and parameters of neural networks. The results of this paper include the existing works on the stability analysis of recurrent neural networks with piecewise linear functions as special cases and hence can be considered as the improvement and extension of the existing stability results in the literature. A numerical example is provided to illustrate the derived theoretical results.

Non periodic oscillations, bistability, coexistence of chaos and hyperchaos in the simplest resistorless Op-Amp based Colpitts oscillator

In the framework of a project on simple circuits with unexpected high degrees of freedom, we report an autonomous microwave oscillator made of a CLC linear resonator of Colpitts type and a single general purpose operational amplifier (Op-Amp). The resonator is in a parallel coupling with the Op-Amp to build the necessary feedback loop of the oscillator. Unlike the general topology of Op-Amp-based oscillators found in the literature including almost always the presence of a negative resistance to justify the nonlinear oscillatory behavior of such circuits, our zero resistor circuit exhibits chaotic and hyperchaotic signals in GHz frequency domain, as well as many other features of complex dynamic systems, including bistability. This simplest form of Colpitts oscillator is adequate to be used as didactic model for the study of complex systems at undergraduate level. Analog and experimental results are proposed.

Effects of Subsystem and Coupling on Synchronization of Multiple Neural Networks With Delays via Impulsive Coupling

This paper from new perspectives discusses the global synchronization of multiple recurrent neural networks (MNNs) with time delays via impulsive coupling. A new concept (coupling strength) is introduced, it is a variable parameter and plays a key role on synchronization. The selection of coupling strength can bring more convenience to the design of the impulsive coupling controller. Four results are presented for the synchronization of MNNs with time delays by using impulsive coupling with the coupling gain and variable topology, where two results are dependent on topology and other two results are independent on topological connectivity. In our results, the effects of each NN, coupling topology, and coupling strength can be positive or negative role on synchronization. In addition, three examples are presented to test our results in the theory analysis.

Synchronization of Multiple Reaction-Diffusion Neural Networks With Heterogeneous and Unbounded Time-Varying Delays

The synchronization problem of multiple/coupled reaction-diffusion neural networks with time-varying delays is investigated. Differing from the existing considerations, state delays among distinct neurons and coupling delays among different subnetworks are included in the proposed model, the assumptions posed on the arisen delays are very weak, time-varying, heterogeneous, even unbounded delays are permitted. To overcome the difficulties from this kind of delay as well as diffusion effects, a comparison-based approach is applied to this model and a series of algebraic criteria are successfully obtained to verify the global asymptotical synchronization. By specifying the existing delays, some M -matrix-based criteria are derived to justify the power-rate synchronization and exponential synchronization. In addition, new criterion on synchronization of general connected neural networks without diffusion effects is also given. Finally, two simulation examples are given to verify the effectiveness of the obtained theoretical results and provide a comparison with the existing criterion.

Distributed Adaptive Tracking Synchronization for Coupled Reaction-Diffusion Neural Network

This paper considers the tracking synchronization problem for a class of coupled reaction-diffusion neural networks (CRDNNs) with undirected topology. For the case where the tracking trajectory has identical individual dynamic as that of the network nodes, the edge-based and vertex-based adaptive strategies on coupling strengths as well as adaptive controllers, which demand merely the local neighbor information, are proposed to synchronize the CRDNNs to the tracking trajectory. To reduce the control costs, an adaptive pinning control technique is employed. For the case where the tracking trajectory has different individual dynamic from that of the network nodes, the vertex-based adaptive strategy is proposed to drive the synchronization error to a relatively small area, which is adjustable according to the parameters of the adaptive strategy. This kind of adaptive design can enhance the robustness of the network against the external disturbance posed on the tracking trajectory. The obtained theoretical results are verified by two representative examples.

Algorithm Design for Edge Detection of High-Speed Moving Target Image under Noisy Environment

For some measurement and detection applications based on video (sequence images), if the exposure time of camera is not suitable with the motion speed of the photographed target, fuzzy edges will be produced in the image, and some poor lighting condition will aggravate this edge blur phenomena. Especially, the existence of noise in industrial field environment makes the extraction of fuzzy edges become a more difficult problem when analyzing the posture of a high-speed moving target. Because noise and edge are always both the kind of high-frequency information, it is difficult to make trade-offs only by frequency bands. In this paper, a noise-tolerant edge detection method based on the correlation relationship between layers of wavelet transform coefficients is proposed. The goal of the paper is not to recover a clean image from a noisy observation, but to make a trade-off judgment for noise and edge signal directly according to the characteristics of wavelet transform coefficients, to realize the extraction of edge information from a noisy image directly. According to the wavelet coefficients tree and the Lipschitz exponent property of noise, the idea of neural network activation function is adopted to design the activation judgment method of wavelet coefficients. Then the significant wavelet coefficients can be retained. At the same time, the non-significant coefficients were removed according to the method of judgment of isolated coefficients. On the other hand, based on the design of Daubechies orthogonal compactly-supported wavelet filter, rational coefficients wavelet filters can be designed by increasing free variables. By reducing the vanishing moments of wavelet filters, more high-frequency information can be retained in the wavelet transform fields, which is benefit to the application of edge detection. For a noisy image of high-speed moving targets with fuzzy edges, by using the length 8-4 rational coefficients biorthogonal wavelet filters and the algorithm proposed in this paper, edge information could be detected clearly. Results of multiple groups of comparative experiments have shown that the edge detection effect of the proposed algorithm in this paper has the obvious superiority.

Stability Analysis of Continuous-Time and Discrete-Time Quaternion-Valued Neural Networks With Linear Threshold Neurons

This paper addresses the problem of stability for continuous-time and discrete-time quaternion-valued neural networks (QVNNs) with linear threshold neurons. Applying the semidiscretization technique to the continuous-time QVNNs, the discrete-time analogs are obtained, which preserve the dynamical characteristics of their continuous-time counterparts. Via the plural decomposition method of quaternion, homeomorphic mapping theorem, as well as Lyapunov theorem, some sufficient conditions on the existence, uniqueness, and global asymptotical stability of the equilibrium point are derived for the continuous-time QVNNs and their discrete-time analogs, respectively. Furthermore, a uniform sufficient condition on the existence, uniqueness, and global asymptotical stability of the equilibrium point is obtained for both continuous-time QVNNs and their discrete-time version. Finally, two numerical examples are provided to substantiate the effectiveness of the proposed results.

New Conditions for Global Asymptotic Stability of Memristor Neural Networks

Recent papers in the literature introduced a class of neural networks (NNs) with memristors, named dynamic-memristor (DM) NNs, such that the analog processing takes place in the charge-flux domain, instead of the typical current-voltage domain as it happens for Hopfield NNs and standard cellular NNs. One key advantage is that, when a steady state is reached, all currents, voltages, and power of a DM-NN drop off, whereas the memristors act as nonvolatile memories that store the processing result. Previous work in the literature addressed multistability of DM-NNs, i.e., convergence of solutions in the presence of multiple asymptotically stable equilibrium points (EPs). The goal of this paper is to study a basically different dynamical property of DM-NNs, namely, to thoroughly investigate the fundamental issue of global asymptotic stability (GAS) of the unique EP of a DM-NN in the general case of nonsymmetric neuron interconnections. A basic result on GAS of DM-NNs is established using Lyapunov method and the concept of Lyapunov diagonally stable matrices. On this basis, some relevant classes of nonsymmetric DM-NNs enjoying the property of GAS are highlighted.

Global Asymptotic Stability and Stabilization of Neural Networks With General Noise

Neural networks (NNs) in the stochastic environment were widely modeled as stochastic differential equations, which were driven by white noise, such as Brown or Wiener process in the existing papers. However, they are not necessarily the best models to describe dynamic characters of NNs disturbed by nonwhite noise in some specific situations. In this paper, general noise disturbance, which may be nonwhite, is introduced to NNs. Since NNs with nonwhite noise cannot be described by Itô integral equation, a novel modeling method of stochastic NNs is utilized. By a framework in light of random field approach and Lyapunov theory, the global asymptotic stability and stabilization in probability or in the mean square of NNs with general noise are analyzed, respectively. Criteria for the concerned systems based on linear matrix inequality are proposed. Some examples are given to illustrate the effectiveness of the obtained results.

Multistability and instability analysis of recurrent neural networks with time-varying delays

This paper provides new theoretical results on the multistability and instability analysis of recurrent neural networks with time-varying delays. It is shown that such n-neuronal recurrent neural networks have exactly [Formula: see text] equilibria, [Formula: see text] of which are locally exponentially stable and the others are unstable, where k₀ is a nonnegative integer such that k₀≤n. By using the combination method of two different divisions, recurrent neural networks can possess more dynamic properties. This method improves and extends the existing results in the literature. Finally, one numerical example is provided to show the superiority and effectiveness of the presented results.

Impulsive Multisynchronization of Coupled Multistable Neural Networks With Time-Varying Delay

This paper studies the synchronization problem of coupled delayed multistable neural networks (NNs) with directed topology. To begin with, several sufficient conditions are developed in terms of algebraic inequalities such that every subnetwork has multiple locally exponentially stable periodic orbits or equilibrium points. Then two new concepts named dynamical multisynchronization (DMS) and static multisynchronization (SMS) are introduced to describe the two novel kinds of synchronization manifolds. Using the impulsive control strategy and the Razumikhin-type technique, some sufficient conditions for both the DMS and the SMS of the controlled coupled delayed multistable NNs with fixed and switching topologies are derived, respectively. Simulation examples are presented to illustrate the effectiveness of the proposed results.

Global Synchronization of Multiple Recurrent Neural Networks With Time Delays via Impulsive Interactions

In this paper, new results on the global synchronization of multiple recurrent neural networks (NNs) with time delays via impulsive interactions are presented. Impulsive interaction means that a number of NNs communicate with each other at impulse instants only, while they are independent at the remaining time. The communication topology among NNs is not required to be always connected and can switch ON and OFF at different impulse instants. By using the concept of sequential connectivity and the properties of stochastic matrices, a set of sufficient conditions depending on time delays is derived to ascertain global synchronization of multiple continuous-time recurrent NNs. In addition, a counterpart on the global synchronization of multiple discrete-time NNs is also discussed. Finally, two examples are presented to illustrate the results.

Multistability of Delayed Recurrent Neural Networks with Mexican Hat Activation Functions

This letter studies the multistability analysis of delayed recurrent neural networks with Mexican hat activation function. Some sufficient conditions are obtained to ensure that an [Formula: see text]-dimensional recurrent neural network can have [Formula: see text] equilibrium points with [Formula: see text], and [Formula: see text] of them are locally exponentially stable. Furthermore, the attraction basins of these stable equilibrium points are estimated. We show that the attraction basins of these stable equilibrium points can be larger than their originally partitioned subsets. The results of this letter improve and extend the existing stability results in the literature. Finally, a numerical example containing different cases is given to illustrate the theoretical results.

Stability Analysis of Neural Networks With Two Delay Components Based on Dynamic Delay Interval Method

In this paper, a dynamic delay interval (DDI) method is proposed to deal with the stability problem of neural networks with two delay components. This method extends the fixed interval of a time-varying delay to a dynamic one, which relaxes the restriction on upper and lower bounds of the delay intervals. Combining the reciprocally convex combination technique and Wirtinger integral inequality, the DDI method leads to some much less conservative delay-dependent stability criteria based on a linear matrix inequality for neural networks with two delay components. Furthermore, the criteria for the system with a single time-varying delay are provided. Some examples are given to illustrate the effectiveness of the obtained results.

Dynamical Behaviors of Multiple Equilibria in Competitive Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions

This paper addresses the problem of coexistence and dynamical behaviors of multiple equilibria for competitive neural networks. First, a general class of discontinuous nonmonotonic piecewise linear activation functions is introduced for competitive neural networks. Then based on the fixed point theorem and theory of strict diagonal dominance matrix, it is shown that under some conditions, such n -neuron competitive neural networks can have 5(n) equilibria, among which 3(n) equilibria are locally stable and the others are unstable. More importantly, it is revealed that the neural networks with the discontinuous activation functions introduced in this paper can have both more total equilibria and locally stable equilibria than the ones with other activation functions, such as the continuous Mexican-hat-type activation function and discontinuous two-level activation function. Furthermore, the 3(n) locally stable equilibria given in this paper are located in not only saturated regions, but also unsaturated regions, which is different from the existing results on multistability of neural networks with multiple level activation functions. A simulation example is provided to illustrate and validate the theoretical findings.

Principal Component 2-D Long Short-Term Memory for Font Recognition on Single Chinese Characters

Chinese character font recognition (CCFR) has received increasing attention as the intelligent applications based on optical character recognition becomes popular. However, traditional CCFR systems do not handle noisy data effectively. By analyzing in detail the basic strokes of Chinese characters, we propose that font recognition on a single Chinese character is a sequence classification problem, which can be effectively solved by recurrent neural networks. For robust CCFR, we integrate a principal component convolution layer with the 2-D long short-term memory (2DLSTM) and develop principal component 2DLSTM (PC-2DLSTM) algorithm. PC-2DLSTM considers two aspects: 1) the principal component layer convolution operation helps remove the noise and get a rational and complete font information and 2) simultaneously, 2DLSTM deals with the long-range contextual processing along scan directions that can contribute to capture the contrast between character trajectory and background. Experiments using the frequently used CCFR dataset suggest the effectiveness of PC-2DLSTM compared with other state-of-the-art font recognition methods.

A Combined Adaptive Neural Network and Nonlinear Model Predictive Control for Multirate Networked Industrial Process Control

This paper investigates the multirate networked industrial process control problem in double-layer architecture. First, the output tracking problem for sampled-data nonlinear plant at device layer with sampling period T(d) is investigated using adaptive neural network (NN) control, and it is shown that the outputs of subsystems at device layer can track the decomposed setpoints. Then, the outputs and inputs of the device layer subsystems are sampled with sampling period T(u) at operation layer to form the index prediction, which is used to predict the overall performance index at lower frequency. Radial basis function NN is utilized as the prediction function due to its approximation ability. Then, considering the dynamics of the overall closed-loop system, nonlinear model predictive control method is proposed to guarantee the system stability and compensate the network-induced delays and packet dropouts. Finally, a continuous stirred tank reactor system is given in the simulation part to demonstrate the effectiveness of the proposed method.

Neural Controller Design-Based Adaptive Control for Nonlinear MIMO Systems With Unknown Hysteresis Inputs

This paper studies an adaptive neural control for nonlinear multiple-input multiple-output systems in interconnected form. The studied systems are composed of N subsystems in pure feedback structure and the interconnection terms are contained in every equation of each subsystem. Moreover, the studied systems consider the effects of Prandtl-Ishlinskii (PI) hysteresis model. It is for the first time to study the control problem for such a class of systems. In addition, the proposed scheme removes an important assumption imposed on the previous works that the bounds of the parameters in PI hysteresis are known. The radial basis functions neural networks are employed to approximate unknown functions. The adaptation laws and the controllers are designed by employing the backstepping technique. The closed-loop system can be proven to be stable by using Lyapunov theorem. A simulation example is studied to validate the effectiveness of the scheme.