Event-trigger-based robust control for nonlinear constrained-input systems using reinforcement learning method

Event-Triggered Constrained Optimal Control for Organic Rankine Cycle Systems via Safe Reinforcement Learning

The organic Rankine cycle (ORC) is an effective application for converting low-grade heat sources into power and is crucial for environmentally friendly production and energy recovery. However, the inherent complexity of the mechanism, its strong and unidentified nonlinearity, and the presence of control constraints severely impair the design of its optimal controller. To solve these issues, this study provides a novel event-triggered (ET) constrained optimal control approach for the ORC systems based on a safe reinforcement learning technique to find the optimal control law. Instead of employing the usual non-quadratic integral form to solve the control-limited optimal control problems, a constraint handling strategy based on a relaxed weighted barrier function (BF) technique is proposed. By adding the BF terms to the original value function, a modified value iteration algorithm is developed to make the control input solutions that tend to violate the constraints be pushed back and maintained in their safe sets. In addition, the ET mechanism proposed in this article is critically required for the ORC systems, and it can significantly reduce the computational load. The combination of these two techniques allows the ORC systems to achieve set-point tracking control and satisfy the control restrictions. The proposed approach is conducted based on a heuristic dynamic programming framework with three neural networks (NNs) involved. The safety and convergence of the proposed approach and the stability of the closed-loop system are analyzed. Simulation results and comparisons are presented to demonstrate its effectiveness.

Fixed-time adaptive dynamic event-triggered control of flexible-joint robots with prescribed performance and time delays

In this study, a dynamic event-triggered control strategy was proposed for n-link flexible-joint robots with prescribed tracking performance and time delays. First, an adaptive fixed-time filter was designed to prevent "differential explosion", and a given-time prescribed performance method was introduced. Then, an auxiliary system and Lyapunov-Krasovskii functionals were designed to compensate for input and full-state delays. After that, neural networks were introduced to handle the unknown dynamics and a dynamic event-triggered controller was designed. The closed-loop system was demonstrated fixed-time stability without Zeno behaviors. Finally, simulations were presented to confirm the effectiveness of the proposed scheme.

Event-Triggered Control of Nonlinear Discrete-Time System With Unknown Dynamics Based on HDP(λ)

The heuristic dynamic programming (HDP) ( λ )-based optimal control strategy, which takes a long-term prediction parameter λ into account using an iterative manner, accelerates the learning rate obviously. The computation complexity caused by the state-associated extra variable in λ -return value computing of the traditional value-gradient learning method can be reduced. However, as the iteration number increases, calculation costs have grown dramatically that bring huge challenge for the optimal control process with limited bandwidth and computational units. In this article, we propose an event-triggered HDP (ETHDP) ( λ ) optimal control strategy for nonlinear discrete-time (NDT) systems with unknown dynamics. The iterative relation for λ -return of the final target value is derived first. The event-triggered condition ensuring system stability is designed to reduce the computation and communication requirements. Next, we build a model-actor-critic neural network (NN) structure, in which the model NN evaluates the system state for getting λ -return of the current time target value, which is used to obtain the critic NN real-time update errors. The event-triggered optimal control signal and one-step-return value are approximated by actor and critic NN, respectively. Then, the event trigger-based uniformly ultimately bounded (UUB) stability of the system state and NN weight errors are demonstrated by applying the Lyapunov technology. Finally, we illustrate the effectiveness of our proposed ETHDP ( λ ) strategy by two cases.

Adaptive Finite-Time Neural Network Control of Nonlinear Systems With Multiple Objective Constraints and Application to Electromechanical System

This article investigates an adaptive finite-time neural control for a class of strict feedback nonlinear systems with multiple objective constraints. In order to solve the main challenges brought by the state constraints and the emergence of finite-time stability, a new barrier Lyapunov function is proposed for the first time, not only can it solve multiobjective constraints effectively but also ensure that all states are always within the constraint intervals. Second, by combining the command filter method and backstepping control, the adaptive controller is designed. What is more, the proposed controller has the ability to avoid the "singularity" problem. The compensation mechanism is introduced to neutralize the error appearing in the filtering process. Furthermore, the neural network is used to approximate the unknown function in the design process. It is shown that the proposed finite-time neural adaptive control scheme achieves a good tracking effect. And each objective function does not violate the constraint bound. Finally, a simulation example of electromechanical dynamic system is given to prove the effectiveness of the proposed finite-time control strategy.

Event-Driven H∞-Constrained Control Using Adaptive Critic Learning

This article considers an event-driven H_∞ control problem of continuous-time nonlinear systems with asymmetric input constraints. Initially, the H_∞ -constrained control problem is converted into a two-person zero-sum game with the discounted nonquadratic cost function. Then, we present the event-driven Hamilton-Jacobi-Isaacs equation (HJIE) associated with the two-person zero-sum game. Meanwhile, we develop a novel event-triggering condition making Zeno behavior excluded. The present event-triggering condition differs from the existing literature in that it can make the triggering threshold non-negative without the requirement of properly selecting the prescribed level of disturbance attenuation. After that, under the framework of adaptive critic learning, we use a single critic network to solve the event-driven HJIE and tune its weight parameters by using historical and instantaneous state data simultaneously. Based on the Lyapunov approach, we demonstrate that the uniform ultimate boundedness of all the signals in the closed-loop system is guaranteed. Finally, simulations of a nonlinear plant are presented to validate the developed event-driven H_∞ control strategy.

Observer-based event-triggered control for zero-sum games of input constrained multi-player nonlinear systems

In this paper, an event-triggered control (ETC) method is investigated to solve zero-sum game (ZSG) problems of unknown multi-player continuous-time nonlinear systems with input constraints by using adaptive dynamic programming (ADP). To relax the requirement of system dynamics, a neural network (NN) observer is constructed to identify the dynamics of multi-player system via the input and output data. Then, the event-triggered Hamilton-Jacobi-Isaacs (HJI) equation of the ZSG can be solved by constructing a critic NN, and the approximated optimal control law and the worst disturbance law can be obtained directly. A triggering scheme which determines the updating time instants of the control law and the disturbance law is developed. Thus, the proposed ADP-based ETC method cannot only reduce the computational burden, but also save communication resource and bandwidths. Furthermore, we prove that the signals of the closed-loop system and the approximate errors of the critic NN weights are uniformly ultimately bounded by using Lyapunov's direct method, and the Zeno behavior is excluded. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed ETC scheme.

Adaptive Critic Learning for Constrained Optimal Event-Triggered Control With Discounted Cost

This article studies an optimal event-triggered control (ETC) problem of nonlinear continuous-time systems subject to asymmetric control constraints. The present nonlinear plant differs from many studied systems in that its equilibrium point is nonzero. First, we introduce a discounted cost for such a system in order to obtain the optimal ETC without making coordinate transformations. Then, we present an event-triggered Hamilton-Jacobi-Bellman equation (ET-HJBE) arising in the discounted-cost constrained optimal ETC problem. After that, we propose an event-triggering condition guaranteeing a positive lower bound for the minimal intersample time. To solve the ET-HJBE, we construct a critic network under the framework of adaptive critic learning. The critic network weight vector is tuned through a modified gradient descent method, which simultaneously uses historical and instantaneous state data. By employing the Lyapunov method, we prove that the uniform ultimate boundedness of all signals in the closed-loop system is guaranteed. Finally, we provide simulations of a pendulum system and an oscillator system to validate the obtained optimal ETC strategy.

Decentralized Event-Triggered Adaptive Control of Discrete-Time Nonzero-Sum Games Over Wireless Sensor-Actuator Networks With Input Constraints

This article studies an event-triggered communication and adaptive dynamic programming (ADP) co-design control method for the multiplayer nonzero-sum (NZS) games of a class of nonlinear discrete-time wireless sensor-actuator network (WSAN) systems subject to input constraints. By virtue of the ADP algorithm, the critic and actor networks are established to attain the approximate Nash equilibrium point solution in the context of the constrained control mechanism. Simultaneously, as the sensors and actuators are physically distributed, a decentralized event-triggered communication protocol is presented, accompanied by a dead-zone operation which avoids the unnecessary events. By predefining the triggering thresholds and compensation values, a novel adaptive triggering condition is derived to guarantee the stability of the event-based closed-loop control system. Then resorting to the Lyapunov theory, the system states and the critic/actor network weight estimation errors are proven to be ultimately bounded. Moreover, an explicit analysis on the nontriviality of the interevent times is also provided. Finally, two numerical examples are conducted to validate the effectiveness of the proposed method.

Event-driven H∞ control with critic learning for nonlinear systems

In this paper, we study an event-driven H_∞ control problem of continuous-time nonlinear systems. Initially, with the introduction of a discounted cost function, we convert the nonlinear H_∞ control problem into an event-driven nonlinear two-player zero-sum game. Then, we develop an event-driven Hamilton-Jacobi-Isaacs equation (HJIE) related to the two-player zero-sum game. After that, we propose a novel event-triggering condition guaranteeing Zeno behavior not to happen. The triggering threshold in the newly proposed event-triggering condition can be kept positive without requiring to properly choose the prescribed level of disturbance attenuation. To solve the event-driven HJIE, we employ an adaptive critic architecture which contains a unique critic neural network (NN). The weight parameters used in the critic NN are tuned via the gradient descent method. After that, we carry out stability analysis of the hybrid closed-loop system based on Lyapunov's direct approach. Finally, we provide two nonlinear plants, including the pendulum system, to validate the proposed event-driven H_∞ control scheme.