Adaptive Optimal Control for a Class of Nonlinear Systems: The Online Policy Iteration Approach

This paper studies the online adaptive optimal controller design for a class of nonlinear systems through a novel policy iteration (PI) algorithm. By using the technique of neural network linear differential inclusion (LDI) to linearize the nonlinear terms in each iteration, the optimal law for controller design can be solved through the relevant algebraic Riccati equation (ARE) without using the system internal parameters. Based on PI approach, the adaptive optimal control algorithm is developed with the online linearization and the two-step iteration, i.e., policy evaluation and policy improvement. The convergence of the proposed PI algorithm is also proved. Finally, two numerical examples are given to illustrate the effectiveness and applicability of the proposed method.

Standard representation and unified stability analysis for dynamic artificial neural network models

An overview is provided of dynamic artificial neural network models (DANNs) for nonlinear dynamical system identification and control problems, and convex stability conditions are proposed that are less conservative than past results. The three most popular classes of dynamic artificial neural network models are described, with their mathematical representations and architectures followed by transformations based on their block diagrams that are convenient for stability and performance analyses. Classes of nonlinear dynamical systems that are universally approximated by such models are characterized, which include rigorous upper bounds on the approximation errors. A unified framework and linear matrix inequality-based stability conditions are described for different classes of dynamic artificial neural network models that take additional information into account such as local slope restrictions and whether the nonlinearities within the DANNs are odd. A theoretical example shows reduced conservatism obtained by the conditions.

Feedback control by online learning an inverse model

A model, predictor, or error estimator is often used by a feedback controller to control a plant. Creating such a model is difficult when the plant exhibits nonlinear behavior. In this paper, a novel online learning control framework is proposed that does not require explicit knowledge about the plant. This framework uses two learning modules, one for creating an inverse model, and the other for actually controlling the plant. Except for their inputs, they are identical. The inverse model learns by the exploration performed by the not yet fully trained controller, while the actual controller is based on the currently learned model. The proposed framework allows fast online learning of an accurate controller. The controller can be applied on a broad range of tasks with different dynamic characteristics. We validate this claim by applying our control framework on several control tasks: 1) the heating tank problem (slow nonlinear dynamics); 2) flight pitch control (slow linear dynamics); and 3) the balancing problem of a double inverted pendulum (fast linear and nonlinear dynamics). The results of these experiments show that fast learning and accurate control can be achieved. Furthermore, a comparison is made with some classical control approaches, and observations concerning convergence and stability are made.

IDENTIFICATION AND CONTROL OF DISCRETE-TIME NONLINEAR SYSTEMS USING AFFINE SUPPORT VECTOR MACHINES

Support vector machine (SVM) is a universal learning method. In this paper, an affine support vector machine (ASVM) for regression is presented for identification and control of input-affine nonlinear models. ASVM is a variant of SVM and so inherits its merits. The solution to ASVM is cast into a convex quadratic programming (QP). Hence ASVM has a unique global solution. In addition, the curse of dimensionality is avoided because ASVM is insensitive to the dimensionality of data. A commonly used model for a nonlinear system is a nonlinear autoregressive exogenous (NARX) model. ASVM could get good performance in both identification and control if a NARX model can be well represented by an input-affine nonlinear model. The experimental results validate the efficiency of ASVM in identification and control of discrete-time nonlinear systems.

Uniformly stable backpropagation algorithm to train a feedforward neural network

Neural networks (NNs) have numerous applications to online processes, but the problem of stability is rarely discussed. This is an extremely important issue because, if the stability of a solution is not guaranteed, the equipment that is being used can be damaged, which can also cause serious accidents. It is true that in some research papers this problem has been considered, but this concerns continuous-time NN only. At the same time, there are many systems that are better described in the discrete time domain such as population of animals, the annual expenses in an industry, the interest earned by a bank, or the prediction of the distribution of loads stored every hour in a warehouse. Therefore, it is of paramount importance to consider the stability of the discrete-time NN. This paper makes several important contributions. 1) A theorem is stated and proven which guarantees uniform stability of a general discrete-time system. 2) It is proven that the backpropagation (BP) algorithm with a new time-varying rate is uniformly stable for online identification and the identification error converges to a small zone bounded by the uncertainty. 3) It is proven that the weights' error is bounded by the initial weights' error, i.e., overfitting is eliminated in the proposed algorithm. 4) The BP algorithm is applied to predict the distribution of loads that a transelevator receives from a trailer and places in the deposits in a warehouse every hour, so that the deposits in the warehouse are reserved in advance using the prediction results. 5) The BP algorithm is compared with the recursive least square (RLS) algorithm and with the Takagi-Sugeno type fuzzy inference system in the problem of predicting the distribution of loads in a warehouse, giving that the first and the second are stable and the third is unstable. 6) The BP algorithm is compared with the RLS algorithm and with the Kalman filter algorithm in a synthetic example.

Stability Analysis and the Stabilization of a Class of Discrete-Time Dynamic Neural Networks

This paper deals with problems of stability and the stabilization of discrete-time neural networks. Neural structures under consideration belong to the class of the so-called locally recurrent globally feedforward networks. The single processing unit possesses dynamic behavior. It is realized by introducing into the neuron structure a linear dynamic system in the form of an infinite impulse response filter. In this way, a dynamic neural network is obtained. It is well known that the crucial problem with neural networks of the dynamic type is stability as well as stabilization in learning problems. The paper formulates stability conditions for the analyzed class of neural networks. Moreover, a stabilization problem is defined and solved as a constrained optimization task. In order to tackle this problem two methods are proposed. The first one is based on a gradient projection (GP) and the second one on a minimum distance projection (MDP). It is worth noting that these methods can be easily introduced into the existing learning algorithm as an additional step, and suitable convergence conditions can be developed for them. The efficiency and usefulness of the proposed approaches are justified by using a number of experiments including numerical complexity analysis, stabilization effectiveness, and the identification of an industrial process.

Some new stability properties of dynamic neural networks with different time-scales

Dynamic neural networks with different time-scales include the aspects of fast and slow phenomenons. Some applications require that the equilibrium points of these networks to be stable. The main contribution of the paper is that Lyapunov function and singularly perturbed technique are combined to access several new stable properties of different time-scales neural networks. Exponential stability and asymptotic stability are obtained by sector and bound conditions. Compared to other papers, these conditions are simpler. Numerical examples are given to demonstrate the effectiveness of the theoretical results.

Global Exponential Stability of Multitime Scale Competitive Neural Networks With Nonsmooth Functions

In this paper, we study the global exponential stability of a multitime scale competitive neural network model with nonsmooth functions, which models a literally inhibited neural network with unsupervised Hebbian learning. The network has two types of state variables, one corresponds to the fast neural activity and another to the slow unsupervised modification of connection weights. Based on the nonsmooth analysis techniques, we prove the existence and uniqueness of equilibrium for the system and establish some new theoretical conditions ensuring global exponential stability of the unique equilibrium of the neural network. Numerical simulations are conducted to illustrate the effectiveness of the derived conditions in characterizing stability regions of the neural network.

Global asymptotic stability of a class of dynamical neural networks

The dynamics of cortical cognitive maps developed by self-organization must include the aspects of long and short-term memory. The behavior of the network is such characterized by an equation of neural activity as a fast phenomenon and an equation of synaptic modification as a slow part of the neural biologically relevant system. We present new stability conditions for analyzing the dynamics of a biological relevant system with different time scales based on the theory of flow invariance. We prove the existence and uniqueness of the equilibrium, and give a quadratic-type Lyapunov function for the flow of a competitive neural system with fast and slow dynamic variables and thus prove the global stability of the equilibrium point.