Reinforcement Learning-Based Nearly Optimal Control for Constrained-Input Partially Unknown Systems Using Differentiator

Reinforcement Learning for Robust Dynamic Event-Driven Constrained Control

We consider a robust dynamic event-driven control (EDC) problem of nonlinear systems having both unmatched perturbations and unknown styles of constraints. Specifically, the constraints imposed on the nonlinear systems' input could be symmetric or asymmetric. Initially, to tackle such constraints, we construct a novel nonquadratic cost function for the constrained auxiliary system. Then, we propose a dynamic event-triggering mechanism relied on the time-based variable and the system states simultaneously for cutting down the computational load. Meanwhile, we show that the robust dynamic EDC of original nonlinear-constrained systems could be acquired by solving the event-driven optimal control problem of the constrained auxiliary system. After that, we develop the corresponding event-driven Hamilton-Jacobi-Bellman equation, and then solve it through a unique critic neural network (CNN) in the reinforcement learning framework. To relax the persistence of excitation condition in tuning CNN's weights, we incorporate experience replay into the gradient descent method. With the aid of Lyapunov's approach, we prove that the closed-loop auxiliary system and the weight estimation error are uniformly ultimately bounded stable. Finally, two examples, including a nonlinear plant and the pendulum system, are utilized to validate the theoretical claims.

Combined control algorithm based on synchronous reinforcement learning for a self-balancing bicycle robot

In this paper, balance control of a bicycle robot is studied without either a trail or a mechanical regulator when the robot moves in an approximately rectilinear motion. Based on the principle of moment balance, an input nonaffine nonlinear dynamics model of the bicycle robot is established. A driving velocity condition is proposed to maintain the robot balance. The nonaffine nonlinear system is transformed into an affine nonlinear system by defining the equivalent control. Subsequently, a feedback linearization controller is designed for the equivalent control. We design a combined control algorithm of synchronous policy iteration based on the actor-critic architecture. The actor neural network (NN) is designed based on the feedback linearization control law. Weight tuning laws for the critic and actor NNs are proposed. The system closed-loop stability and convergence of the NN weights are guaranteed based on the Lyapunov analysis. The optimality of the equivalent control policy is guaranteed. To satisfy the driving velocity condition, the values of the steering angle and driving velocity are determined based on the optimal equivalent control. The effectiveness of the proposed algorithm is verified through simulations and real experiments.

Event-Triggered Approximate Optimal Path-Following Control for Unmanned Surface Vehicles With State Constraints

This article investigates the problem of path following for the underactuated unmanned surface vehicles (USVs) subject to state constraints. A useful control algorithm is proposed by combining the backstepping technique, adaptive dynamic programming (ADP), and the event-triggered mechanism. The presented approach consists of three modules: guidance law, dynamic controller, and event triggering. First, to deal with the "singularity" problem, the guidance-based path-following (GBPF) principle is introduced in the guidance law loop. In contrast to the traditional barrier Lyapunov function (BLF) method, this article converts the USV's constraint model to a class of nonlinear systems without state constraints by introducing a nonlinear mapping. The control signal generated by the dynamic controller module consists of a backstepping-based feedforward control signal and an ADP-based approximate optimal feedback control signal. Therefore, the presented scheme can guarantee the approximate optimal performance. To approximate the cost function and its partial derivative, a critic neural network (NN) is constructed. By considering the event-triggered condition, the dynamic controller is further improved. Compared with traditional time-triggered control methods, the proposed approach can greatly reduce communication and computational burdens. This article proves that the closed-loop system is stable, and the simulation results and experimental validation are given to illustrate the effectiveness of the proposed approach.