Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs

Approximate tracking control for nonlinear multi-player systems with deferred asymmetric time-varying full-state constraints

This paper proposes a set of Nash equilibrium tracking control strategies based on mixed zero-sum (MZS) game for the continuous-time nonlinear multi-player systems with deferred asymmetric time-varying (DATV) full-state constraints and unknown initial state. Firstly, an improved shift transformation is used to modify the original constrained system with an unknown initial state into a barrier transformable constrained system. Then, based on the barrier transformable constrained system and predefined reference trajectory, an unconstrained augmented system is formed through the application of the barrier function (BF) transformation. Furthermore, the MZS game Nash equilibrium tracking control strategies are derived by establishing the tracking error related quadratic cost functions and corresponding HJ functions for different players. On this basis, a critic-only structure is established to approximate the control strategy of every player online. By employing Lyapunov theory, it is proven that the neural network weights and tracking error are uniformly ultimately bounded (UUB) within DATV full-state constraints. Simulation experiments of a three-player nonlinear system demonstrate that our algorithm successfully handles deferred state constraints and unknown initial conditions, ensuring that the system states follow the desired reference trajectories. Simulation results further validate the uniform ultimate boundedness of neural network weights and tracking errors.

Multi-H∞ Controls for Unknown Input-Interference Nonlinear System With Reinforcement Learning

This article studies the multi- [Formula: see text] controls for the input-interference nonlinear systems via adaptive dynamic programming (ADP) method, which allows for multiple inputs to have the individual selfish component of the strategy to resist weighted interference. In this line, the ADP scheme is used to learn the Nash-optimization solutions of the input-interference nonlinear system such that multiple [Formula: see text] performance indices can reach the defined Nash equilibrium. First, the input-interference nonlinear system is given and the Nash equilibrium is defined. An adaptive neural network (NN) observer is introduced to identify the input-interference nonlinear dynamics. Then, the critic NNs are used to learn the multiple [Formula: see text] performance indices. A novel adaptive law is designed to update the critic NN weights by minimizing the Hamiltonian-Jacobi-Isaacs (HJI) equation, which can be used to directly calculate the multi- [Formula: see text] controls effectively by using input-output data such that the actor structure is avoided. Moreover, the control system stability and updated parameter convergence are proved. Finally, two numerical examples are simulated to verify the proposed ADP scheme for the input-interference nonlinear system.

Model-Free Adaptive Optimal Control for Unknown Nonlinear Multiplayer Nonzero-Sum Game

In this article, an online adaptive optimal control algorithm based on adaptive dynamic programming is developed to solve the multiplayer nonzero-sum game (MP-NZSG) for discrete-time unknown nonlinear systems. First, a model-free coupled globalized dual-heuristic dynamic programming (GDHP) structure is designed to solve the MP-NZSG problem, in which there is no model network or identifier. Second, in order to relax the requirement of systems dynamics, an online adaptive learning algorithm is developed to solve the Hamilton-Jacobi equation using the system states of two adjacent time steps. Third, a series of critic networks and action networks are used to approximate value functions and optimal policies for all players. All the neural network (NN) weights are updated online based on real-time system states. Fourth, the uniformly ultimate boundedness analysis of the NN approximation errors is proved based on the Lyapunov approach. Finally, simulation results are given to demonstrate the effectiveness of the developed scheme.

Online Nash-optimization tracking control of multi-motor driven load system with simplified RL scheme

Although the optimal tracking control problem (OTCP) has been addressed recently, only the single-input system is considered in the recent literature. In this paper, the OTCP of unknown multi-motor driven load systems (MMDLS) is addressed based on a simplified reinforcement learning (RL) structure, where all the motor inputs with different dynamics will be obtained as a Nash equilibrium. Thus, the performance indexes associated with each input can be optimized as an outcome of a Nash equilibrium. Firstly, we use an identifier to reconstruct MMDLS dynamics, such that the accurate model required in the general control design is avoided. We use the identified dynamics to drive Nash-optimization inputs, which include the steady-state controls and the RL-based controls. The steady-state controls are designed with the identified system model. The RL-based controls are designed using the optimization method with the simplified RL-based critic NN schemes. We use the simplified RL structures to approximate the cost function of each motor input in the optimal control design. The NN weights of both the identified algorithm and simplified RL-based structure are approximated by using a novel adaptation algorithm, where the learning gains can be optimized adaptively. The weight convergences and the Nash-optimization MMDLS stability are all proved. Finally, numerical MMDLS simulations are implemented to show the correctness and the improved performance of the proposed methods.