1
|
Zhang D, Wang Y, Meng L, Yan J, Qin C. Adaptive critic design for safety-optimal FTC of unknown nonlinear systems with asymmetric constrained-input. ISA TRANSACTIONS 2024; 155:309-318. [PMID: 39306561 DOI: 10.1016/j.isatra.2024.09.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 09/11/2024] [Accepted: 09/11/2024] [Indexed: 12/13/2024]
Abstract
Safe fault tolerant control is one of the key technologies to improve the reliability of dynamic complex nonlinear systems with limited inputs, which is hard to solve and definitely a great challenge to tackle. Thus the paper presents a novel safety-optimal FTC (Fault Tolerant Control) approach for a category of completely unknown nonlinear systems incorporating actuator fault and asymmetric constrained-input, which can guarantee the system's operation within a safe range while showcasing optimal performance. Firstly, a CBF (Control Barrier Function) is incorporated into the cost function to penalize unsafe behaviors, and then we translate the intractable safety-optimal FTC problem into a differential ZSG (Zero-Sum Game) problem by defining the control input and the actuator fault as two opposing sides. Secondly, a neural-network-based identifier is employed to reconstruct system dynamics using system data, and the resolution of handling asymmetric constrained-input with the introduced non-quadratic cost function is achieved through the design of an adaptive critic scheme, aiming to reduce computational expenses accordingly. Finally, through the theoretical stability analysis, it is demonstrated that all signals in the closed-loop system are consistently UUB (Uniformly Ultimately Bounded). Furthermore, the proposed method's effectiveness is also verified in the simulation experiments conducted on a model of a single-link robotic arm system with actuator failure. The result shows that the algorithm can fulfill the safety-optimal demand of fault tolerant control in fault system with asymmetric constrained-input.
Collapse
Affiliation(s)
- Dehua Zhang
- School of Artificial Intelligence, Henan University, Zhengzhou, 450046, China.
| | - Yuchen Wang
- School of Artificial Intelligence, Henan University, Zhengzhou, 450046, China.
| | - Lei Meng
- School of Artificial Intelligence, Henan University, Zhengzhou, 450046, China.
| | - Jiayuan Yan
- School of Artificial Intelligence, Henan University, Zhengzhou, 450046, China.
| | - Chunbin Qin
- School of Artificial Intelligence, Henan University, Zhengzhou, 450046, China.
| |
Collapse
|
2
|
Wu Y, Chen M, Li H, Chadli M. Mixed-Zero-Sum-Game-Based Memory Event-Triggered Cooperative Control of Heterogeneous MASs Against DoS Attacks. IEEE TRANSACTIONS ON CYBERNETICS 2024; 54:5733-5745. [PMID: 38478450 DOI: 10.1109/tcyb.2024.3369975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/20/2024]
Abstract
This article studies the problem of memory event-triggered cooperative adaptive control of heterogeneous nonlinear multiagent systems (MASs) under denial-of-service (DoS) attacks based on the multiplayer mixed zero-sum (ZS) game strategy. First, a neural-network-based reinforcement learning scheme is structured to obtain the Nash equilibrium solution of the proposed multiplayer mixed ZS game scheme. Then, a memory-based event-triggered mechanism considering the historical data is proposed. This effectively avoids incorrect triggering information caused by unknown external factors. Moreover, thanks to the idea of switching topology, the mixed ZS game problem under the influence of node-based DoS attacks is solved efficiently. In accordance with the Lyapunov stability theory, it is proved that all signals of heterogeneous MASs are bounded, all heterogeneous followers can track the trajectory of the leader during the no-attack period, the attacked follower can achieve stabilization control during the attack period, and the remaining nonattacked followers can achieve cooperative control during the attack period. Finally, the effectiveness of the designed memory-event-triggered-based mixed ZS game cooperative control strategy is tested by the given simulation results.
Collapse
|
3
|
Qiao J, Li M, Wang D. Asymmetric Constrained Optimal Tracking Control With Critic Learning of Nonlinear Multiplayer Zero-Sum Games. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:5671-5683. [PMID: 36191112 DOI: 10.1109/tnnls.2022.3208611] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
By utilizing a neural-network-based adaptive critic mechanism, the optimal tracking control problem is investigated for nonlinear continuous-time (CT) multiplayer zero-sum games (ZSGs) with asymmetric constraints. Initially, we build an augmented system with the tracking error system and the reference system. Moreover, a novel nonquadratic function is introduced to address asymmetric constraints. Then, we derive the tracking Hamilton-Jacobi-Isaacs (HJI) equation of the constrained nonlinear multiplayer ZSG. However, it is extremely hard to get the analytical solution to the HJI equation. Hence, an adaptive critic mechanism based on neural networks is established to estimate the optimal cost function, so as to obtain the near-optimal control policy set and the near worst disturbance policy set. In the process of neural critic learning, we only utilize one critic neural network and develop a new weight updating rule. After that, by using the Lyapunov approach, the uniform ultimate boundedness stability of the tracking error in the augmented system and the weight estimation error of the critic network is verified. Finally, two simulation examples are provided to demonstrate the efficacy of the established mechanism.
Collapse
|
4
|
Li B, Chen N, Luo B, Chen J, Yang C, Gui W. ADP-Based Event-Triggered Constrained Optimal Control on Spatiotemporal Process: Application to Temperature Field in Roller Kiln. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:3229-3241. [PMID: 37195852 DOI: 10.1109/tnnls.2023.3267516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
The precise control of the spatiotemporal process in a roller kiln is crucial in the production of Ni-Co-Mn layered cathode material of lithium-ion batteries. Since the product is extremely sensitive to temperature distribution, temperature field control is of great significance. In this article, an event-triggered optimal control (ETOC) method with input constraints for the temperature field is proposed, which takes up an important position in reducing the communication and computation costs. A nonquadratic cost function is adopted to describe the system performance with input constraints. First, we present the problem description of the temperature field event-triggered control, where this field is described by a partial differential equation (PDE). Then, the event-triggered condition is designed according to the information of system states and control inputs. On this basis, a framework of the event-triggered adaptive dynamic programming (ETADP) method that is based on the model reduction technology is proposed for the PDE system. A critic network is used to approach the optimal performance index by a neural network (NN) together with that an actor network is used to optimize the control strategy. Furthermore, an upper bound of the performance index and a lower bound of interexecution times, as well as the stabilities of the impulsive dynamic system and the closed-loop PDE system, are also proved. Simulation verification demonstrates the effectiveness of the proposed method.
Collapse
|
5
|
Zhu L, Guo P, Wei Q. Synergetic learning for unknown nonlinear H ∞ control using neural networks. Neural Netw 2023; 168:287-299. [PMID: 37774514 DOI: 10.1016/j.neunet.2023.09.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Revised: 08/24/2023] [Accepted: 09/15/2023] [Indexed: 10/01/2023]
Abstract
The well-known H∞ control design gives robustness to a controller by rejecting perturbations from the external environment, which is difficult to do for completely unknown affine nonlinear systems. Accordingly, the immediate objective of this paper is to develop an on-line real-time synergetic learning algorithm, so that a data-driven H∞ controller can be received. By converting the H∞ control problem into a two-player zero-sum game, a model-free Hamilton-Jacobi-Isaacs equation (MF-HJIE) is first derived using off-policy reinforcement learning, followed by a proof of equivalence between the MF-HJIE and the conventional HJIE. Next, by applying the temporal difference to the MF-HJIE, a synergetic evolutionary rule with experience replay is designed to learn the optimal value function, the optimal control, and the worst perturbation, that can be performed on-line and in real-time along the system state trajectory. It is proven that the synergistic learning system constructed by the system plant and the evolutionary rule is uniformly ultimately bounded. Finally, simulation results on an F16 aircraft system and a nonlinear system back up the tractability of the proposed method.
Collapse
Affiliation(s)
- Liao Zhu
- International Academic Center of Complex Systems, Beijing Normal University, Zhuhai, 519087, Guangdong, China; School of Systems Science, Beijing Normal University, Beijing, 100875, China.
| | - Ping Guo
- International Academic Center of Complex Systems, Beijing Normal University, Zhuhai, 519087, Guangdong, China; School of Systems Science, Beijing Normal University, Beijing, 100875, China.
| | - Qinglai Wei
- The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China; Institute of Systems Engineering, Macau University of Science and Technology, 999078, Macao Special Administrative Region of China.
| |
Collapse
|
6
|
Zhang H, Ming Z, Yan Y, Wang W. Data-Driven Finite-Horizon H ∞ Tracking Control With Event-Triggered Mechanism for the Continuous-Time Nonlinear Systems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:4687-4701. [PMID: 34633936 DOI: 10.1109/tnnls.2021.3116464] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
In this article, the neural network (NN)-based adaptive dynamic programming (ADP) event-triggered control method is presented to obtain the near-optimal control policy for the model-free finite-horizon H∞ optimal tracking control problem with constrained control input. First, using available input-output data, a data-driven model is established by a recurrent NN (RNN) to reconstruct the unknown system. Then, an augmented system with event-triggered mechanism is obtained by a tracking error system and a command generator. We present a novel event-triggering condition without Zeno behavior. On this basis, the relationship between event-triggered Hamilton-Jacobi-Isaacs (HJI) equation and time-triggered HJI equation is given in Theorem 3. Since the solution of the HJI equation is time-dependent for the augmented system, the time-dependent activation functions of NNs are considered. Moreover, an extra error is incorporated to satisfy the terminal constraints of cost function. This adaptive control pattern finds, in real time, approximations of the optimal value while also ensuring the uniform ultimate boundedness of the closed-loop system. Finally, the effectiveness of the proposed near-optimal control pattern is verified by two simulation examples.
Collapse
|
7
|
Zhao H, Wang H, Niu B, Zhao X, Alharbi KH. Event-triggered fault-tolerant control for input-constrained nonlinear systems with mismatched disturbances via adaptive dynamic programming. Neural Netw 2023; 164:508-520. [PMID: 37201311 DOI: 10.1016/j.neunet.2023.05.001] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 02/26/2023] [Accepted: 05/01/2023] [Indexed: 05/20/2023]
Abstract
In this paper, the issue of event-triggered optimal fault-tolerant control is investigated for input-constrained nonlinear systems with mismatched disturbances. To eliminate the effect of abrupt faults and ensure the optimal performance of general nonlinear dynamics, an adaptive dynamic programming (ADP) algorithm is employed to develop a sliding mode fault-tolerant control strategy. When the system trajectories converge to the sliding-mode surface, the equivalent sliding mode dynamics is transformed into a reformulated auxiliary system with a modified cost function. Then, a single critic neural network (NN) is adopted to solve the modified Hamilton-Jacobi-Bellman (HJB) equation. In order to overcome the difficulty that arises from the persistence of excitation (PE) condition, the experience replay technique is utilized to update the critic weights. In this study, a novel control method is proposed, which can effectively eliminate the effects of abrupt faults while achieving optimal control with the minimum cost under a single network architecture. Furthermore, the closed-loop nonlinear system is proved to be uniformly ultimate boundedness based on Lyapunov stability theory. Finally, three examples are presented to verify the validity of the control strategy.
Collapse
Affiliation(s)
- Heng Zhao
- College of Control Science and Engineering, Bohai University, Jinzhou, Liaoning 121013, China.
| | - Huanqing Wang
- College of Mathematical Science, Bohai University, Jinzhou, Liaoning 121013, China
| | - Ben Niu
- School of Information Science and Engineering, Shandong Normal University, Jinan 250014, China
| | - Xudong Zhao
- College of Control Science and Engineering, Bohai University, Jinzhou, Liaoning 121013, China
| | - Khalid H Alharbi
- Communication Systems and Networks Research Group, Department of Electrical and Computer Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah Saudi Arabia
| |
Collapse
|
8
|
Li M, Wang D, Zhao M, Qiao J. Event-triggered constrained neural critic control of nonlinear continuous-time multiplayer nonzero-sum games. Inf Sci (N Y) 2023. [DOI: 10.1016/j.ins.2023.02.081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2023]
|
9
|
Chen Z, Chen K, Chen SZ, Zhang Y. Event-triggered H∞ consensus for uncertain nonlinear systems using integral sliding mode based adaptive dynamic programming. Neural Netw 2022; 156:258-270. [DOI: 10.1016/j.neunet.2022.09.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Revised: 08/31/2022] [Accepted: 09/26/2022] [Indexed: 11/06/2022]
|
10
|
Tian B, Wang C, Guo L. Composite Antidisturbance Control for Non-Gaussian Stochastic Systems via Information-Theoretic Learning Technique. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:7644-7654. [PMID: 34138721 DOI: 10.1109/tnnls.2021.3086032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
In this article, a novel composite hierarchical antidisturbance control (CHADC) algorithm aided by the information-theoretic learning (ITL) technique is developed for non-Gaussian stochastic systems subject to dynamic disturbances. The whole control process consists of some time-domain intervals called batches. Within each batch, a CHADC scheme is applied to the system, where a disturbance observer (DO) is employed to estimate the dynamic disturbance and a composite control strategy integrating feedforward compensation and feedback control is adopted. The information-theoretic measure (entropy or information potential) is employed to quantify the randomness of the controlled system, based on which the gain matrices of DO and feedback controller are updated between two adjacent batches. In this way, the mean-square stability is guaranteed within each batch, and the system performance is improved along with the progress of batches. The proposed algorithm has enhanced disturbance rejection ability and good applicability to non-Gaussian noise environment, which contributes to extending CHADC theory to the general stochastic case. Finally, simulation examples are included to verify the effectiveness of theoretical results.
Collapse
|
11
|
Neural critic learning for tracking control design of constrained nonlinear multi-person zero-sum games. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.09.103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
12
|
Xia H, Zhao B, Guo P. Synergetic learning structure-based neuro-optimal fault tolerant control for unknown nonlinear systems. Neural Netw 2022; 155:204-214. [DOI: 10.1016/j.neunet.2022.08.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 06/02/2022] [Accepted: 08/08/2022] [Indexed: 10/31/2022]
|
13
|
Xue S, Luo B, Liu D, Gao Y. Neural network-based event-triggered integral reinforcement learning for constrained H∞ tracking control with experience replay. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.09.119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
14
|
Liu X, Xu B, Shou Y, Fan QY, Chen Y. Event-Triggered Adaptive Control of Uncertain Nonlinear Systems With Composite Condition. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:6030-6037. [PMID: 33961566 DOI: 10.1109/tnnls.2021.3072107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
This article concentrates on the event-based collaborative design for strict-feedback systems with uncertain nonlinearities. The controller is designed based on neural network (NN) weights adaptive law. The controller and NN weights adaptive law are only updated at the triggering instants determined by a novel composite triggering threshold. Considering the conservativeness of event condition, the state-model error is integrated into constructing the composite condition and NN weights adaptive law. In the context of the proposed mechanism, the requirements of system information and the allowable range of event-triggering error are relaxed. The number of triggering instants is greatly reduced without deteriorating the system performance. Moreover, the stability of the closed-loop is proved by the Lyapunov method following time-interval and sampling instants. Simulation results show the effectiveness of the scheme proposed in this article.
Collapse
|
15
|
Tan H, Wang Y, Wu M, Huang Z, Miao Z. Distributed Group Coordination of Multiagent Systems in Cloud Computing Systems Using a Model-Free Adaptive Predictive Control Strategy. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:3461-3473. [PMID: 33531307 DOI: 10.1109/tnnls.2021.3053016] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
This article studies the group coordinated control problem for distributed nonlinear multiagent systems (MASs) with unknown dynamics. Cloud computing systems are employed to divide agents into groups and establish networked distributed multigroup-agent systems (ND-MGASs). To achieve the coordination of all agents and actively compensate for communication network delays, a novel networked model-free adaptive predictive control (NMFAPC) strategy combining networked predictive control theory with model-free adaptive control method is proposed. In the NMFAPC strategy, each nonlinear agent is described as a time-varying data model, which only relies on the system measurement data for adaptive learning. To analyze the system performance, a simultaneous analysis method for stability and consensus of ND-MGASs is presented. Finally, the effectiveness and practicability of the proposed NMFAPC strategy are verified by numerical simulations and experimental examples. The achievement also provides a solution for the coordination of large-scale nonlinear MASs.
Collapse
|
16
|
Zhao Q, Si J, Sun J. Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: From Time-Driven to Event-Driven. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:4139-4144. [PMID: 33534714 DOI: 10.1109/tnnls.2021.3053037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
In this work, time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives. Among existing approximate dynamic programming (ADP) and reinforcement learning (RL) algorithms, the direct heuristic dynamic programming (dHDP) has been shown an effective tool as demonstrated in solving several complex learning control problems. It continuously updates the control policy and the critic as system states continuously evolve. It is therefore desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise. Toward this goal, we propose a new event-driven dHDP. By constructing a Lyapunov function candidate, we prove the uniformly ultimately boundedness (UUB) of the system states and the weights in the critic and the control policy networks. Consequently, we show the approximate control and cost-to-go function approaching Bellman optimality within a finite bound. We also illustrate how the event-driven dHDP algorithm works in comparison to the original time-driven dHDP.
Collapse
|
17
|
Yang X, Xu M, Wei Q. Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; PP:923-937. [PMID: 35666792 DOI: 10.1109/tnnls.2022.3178017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.
Collapse
|
18
|
Fu Y, Hong C, Fu J, Chai T. Approximate Optimal Tracking Control of Nondifferentiable Signals for a Class of Continuous-Time Nonlinear Systems. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:4441-4450. [PMID: 33141675 DOI: 10.1109/tcyb.2020.3027344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
In this article, for a class of continuous-time nonlinear nonaffine systems with unknown dynamics, a robust approximate optimal tracking controller (RAOTC) is proposed in the framework of adaptive dynamic programming (ADP). The distinguishing contribution of this article is that a new Lyapunov function is constructed, by using which the derivative information of tracking errors is not required in computing its time derivative along with the solution of the closed-loop system. Thus, the proposed method can make the system states follow nondifferentiable reference signals, which removes the common assumption that the reference signals have to be continuous for tracking control of continuous-time nonlinear systems in the literature. The theoretical analysis, simulation, and application results well illustrate the effectiveness and superiority of the proposed method.
Collapse
|
19
|
Zhao Q, Sun J, Wang G, Chen J. Event-Triggered ADP for Nonzero-Sum Games of Unknown Nonlinear Systems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:1905-1913. [PMID: 33882002 DOI: 10.1109/tnnls.2021.3071545] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
For nonzero-sum (NZS) games of nonlinear systems, reinforcement learning (RL) or adaptive dynamic programming (ADP) has shown its capability of approximating the desired index performance and the optimal input policy iteratively. In this article, an event-triggered ADP is proposed for NZS games of continuous-time nonlinear systems with completely unknown system dynamics. To achieve the Nash equilibrium solution approximately, the critic neural networks and actor neural networks are utilized to estimate the value functions and the control policies, respectively. Compared with the traditional time-triggered mechanism, the proposed algorithm updates the neural network weights as well as the inputs of players only when a state-based event-triggered condition is violated. It is shown that the system stability and the weights' convergence are still guaranteed under mild assumptions, while occupation of communication and computation resources is considerably reduced. Meanwhile, the infamous Zeno behavior is excluded by proving the existence of a minimum inter-event time (MIET) to ensure the feasibility of the closed-loop event-triggered continuous-time system. Finally, a numerical example is simulated to illustrate the effectiveness of the proposed approach.
Collapse
|
20
|
Zhang K, Su R, Zhang H, Tian Y. Adaptive Resilient Event-Triggered Control Design of Autonomous Vehicles With an Iterative Single Critic Learning Framework. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021; 32:5502-5511. [PMID: 33534717 DOI: 10.1109/tnnls.2021.3053269] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
This article investigates the adaptive resilient event-triggered control for rear-wheel-drive autonomous (RWDA) vehicles based on an iterative single critic learning framework, which can effectively balance the frequency/changes in adjusting the vehicle's control during the running process. According to the kinematic equation of RWDA vehicles and the desired trajectory, the tracking error system during the autonomous driving process is first built, where the denial-of-service (DoS) attacking signals are injected into the networked communication and transmission. Combining the event-triggered sampling mechanism and iterative single critic learning framework, a new event-triggered condition is developed for the adaptive resilient control algorithm, and the novel utility function design is considered for driving the autonomous vehicle, where the control input can be guaranteed into an applicable saturated bound. Finally, we apply the new adaptive resilient control scheme to a case of driving the RWDA vehicles, and the simulation results illustrate the effectiveness and practicality successfully.
Collapse
|
21
|
Yang X, He H. Event-Driven H ∞-Constrained Control Using Adaptive Critic Learning. IEEE TRANSACTIONS ON CYBERNETICS 2021; 51:4860-4872. [PMID: 32112694 DOI: 10.1109/tcyb.2020.2972748] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
This article considers an event-driven H∞ control problem of continuous-time nonlinear systems with asymmetric input constraints. Initially, the H∞ -constrained control problem is converted into a two-person zero-sum game with the discounted nonquadratic cost function. Then, we present the event-driven Hamilton-Jacobi-Isaacs equation (HJIE) associated with the two-person zero-sum game. Meanwhile, we develop a novel event-triggering condition making Zeno behavior excluded. The present event-triggering condition differs from the existing literature in that it can make the triggering threshold non-negative without the requirement of properly selecting the prescribed level of disturbance attenuation. After that, under the framework of adaptive critic learning, we use a single critic network to solve the event-driven HJIE and tune its weight parameters by using historical and instantaneous state data simultaneously. Based on the Lyapunov approach, we demonstrate that the uniform ultimate boundedness of all the signals in the closed-loop system is guaranteed. Finally, simulations of a nonlinear plant are presented to validate the developed event-driven H∞ control strategy.
Collapse
|
22
|
Online event-based adaptive critic design with experience replay to solve partially unknown multi-player nonzero-sum games. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.05.087] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
23
|
Zhang S, Zhao B, Liu D, Zhang Y. Observer-based event-triggered control for zero-sum games of input constrained multi-player nonlinear systems. Neural Netw 2021; 144:101-112. [PMID: 34478940 DOI: 10.1016/j.neunet.2021.08.012] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Revised: 07/18/2021] [Accepted: 08/09/2021] [Indexed: 11/18/2022]
Abstract
In this paper, an event-triggered control (ETC) method is investigated to solve zero-sum game (ZSG) problems of unknown multi-player continuous-time nonlinear systems with input constraints by using adaptive dynamic programming (ADP). To relax the requirement of system dynamics, a neural network (NN) observer is constructed to identify the dynamics of multi-player system via the input and output data. Then, the event-triggered Hamilton-Jacobi-Isaacs (HJI) equation of the ZSG can be solved by constructing a critic NN, and the approximated optimal control law and the worst disturbance law can be obtained directly. A triggering scheme which determines the updating time instants of the control law and the disturbance law is developed. Thus, the proposed ADP-based ETC method cannot only reduce the computational burden, but also save communication resource and bandwidths. Furthermore, we prove that the signals of the closed-loop system and the approximate errors of the critic NN weights are uniformly ultimately bounded by using Lyapunov's direct method, and the Zeno behavior is excluded. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed ETC scheme.
Collapse
Affiliation(s)
- Shunchao Zhang
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China.
| | - Bo Zhao
- School of Systems Science, Beijing Normal University, Beijing 100875, China.
| | - Derong Liu
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China.
| | - Yongwei Zhang
- School of Automation, Guangdong University of Technology, Guangzhou 510006, China.
| |
Collapse
|
24
|
Zhang S, Zhao B, Zhang Y. Event-triggered control for input constrained non-affine nonlinear systems based on neuro-dynamic programming. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.01.116] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
25
|
Zhang Y, Zhao B, Liu D. Event-triggered adaptive dynamic programming for multi-player zero-sum games with unknown dynamics. Soft comput 2021. [DOI: 10.1007/s00500-020-05293-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
26
|
Liang Y, Zhang H, Duan J, Sun S. Event-triggered reinforcement learning H∞control design for constrained-input nonlinear systems subject to actuator failures. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2020.07.055] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
27
|
Su H, Zhang H, Liang X, Liu C. Decentralized Event-Triggered Online Adaptive Control of Unknown Large-Scale Systems Over Wireless Communication Networks. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:4907-4919. [PMID: 31940563 DOI: 10.1109/tnnls.2019.2959005] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
In this article, a novel online decentralized event-triggered control scheme is proposed for a class of nonlinear interconnected large-scale systems subject to unknown internal system dynamics and interconnected terms. First, by designing a neural network-based identifier, the unknown internal dynamics of the interconnected systems is reconstructed. Then, the adaptive critic design method is used to learn the approximate optimal control policies in the context of event-triggered mechanism. Specifically, the event-based control processes of different subsystems are independent, asynchronous, and decentralized. That is, the decentralized event-triggering conditions and the controllers only rely on the local state information of the corresponding subsystems, which avoids the transmissions of the state information between the subsystems over the wireless communication networks. Then, with the help of Lyapunov's theorem, the states of the developed closed-loop control system and the critic weight estimation errors are proved to be uniformly ultimately bounded. Finally, the effectiveness and applicability of the event-based control method are verified by an illustrative numerical example and a practical example.
Collapse
|
28
|
Su H, Zhang H, Jiang H, Wen Y. Decentralized Event-Triggered Adaptive Control of Discrete-Time Nonzero-Sum Games Over Wireless Sensor-Actuator Networks With Input Constraints. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:4254-4266. [PMID: 31940556 DOI: 10.1109/tnnls.2019.2953613] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
This article studies an event-triggered communication and adaptive dynamic programming (ADP) co-design control method for the multiplayer nonzero-sum (NZS) games of a class of nonlinear discrete-time wireless sensor-actuator network (WSAN) systems subject to input constraints. By virtue of the ADP algorithm, the critic and actor networks are established to attain the approximate Nash equilibrium point solution in the context of the constrained control mechanism. Simultaneously, as the sensors and actuators are physically distributed, a decentralized event-triggered communication protocol is presented, accompanied by a dead-zone operation which avoids the unnecessary events. By predefining the triggering thresholds and compensation values, a novel adaptive triggering condition is derived to guarantee the stability of the event-based closed-loop control system. Then resorting to the Lyapunov theory, the system states and the critic/actor network weight estimation errors are proven to be ultimately bounded. Moreover, an explicit analysis on the nontriviality of the interevent times is also provided. Finally, two numerical examples are conducted to validate the effectiveness of the proposed method.
Collapse
|
29
|
Adaptive dynamic programming based event-triggered control for unknown continuous-time nonlinear systems with input constraints. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2018.09.097] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
30
|
Xiao X, Fu D, Wang G, Liao S, Qi Y, Huang H, Jin L. Two neural dynamics approaches for computing system of time-varying nonlinear equations. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.02.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
31
|
Su H, Zhang H, Sun S, Cai Y. Integral reinforcement learning-based online adaptive event-triggered control for non-zero-sum games of partially unknown nonlinear systems. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.09.088] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
32
|
Niu H, Bhowmick C, Jagannathan S. Attack Detection and Approximation in Nonlinear Networked Control Systems Using Neural Networks. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:235-245. [PMID: 30892252 DOI: 10.1109/tnnls.2019.2900430] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
In networked control systems (NCS), a certain class of attacks on the communication network is known to raise traffic flows causing delays and packet losses to increase. This paper presents a novel neural network (NN)-based attack detection and estimation scheme that captures the abnormal traffic flow due to a class of attacks on the communication links within the feedback loop of an NCS. By modeling the unknown network flow as a nonlinear function at the bottleneck node and using a NN observer, the network attack detection residual is defined and utilized to determine the onset of an attack in the communication network when the residual exceeds a predefined threshold. Upon detection, another NN is used to estimate the flow injected by the attack. For the physical system, we develop an attack detection scheme by using an adaptive dynamic programming-based optimal event-triggered NN controller in the presence of network delays and packet losses. Attacks on the network as well as on the sensors of the physical system can be detected and estimated with the proposed scheme. The simulation results confirm theoretical conclusions.
Collapse
|
33
|
Luo B, Yang Y, Liu D, Wu HN. Event-Triggered Optimal Control With Performance Guarantees Using Adaptive Dynamic Programming. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020; 31:76-88. [PMID: 30892242 DOI: 10.1109/tnnls.2019.2899594] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
This paper studies the problem of event-triggered optimal control (ETOC) for continuous-time nonlinear systems and proposes a novel event-triggering condition that enables designing ETOC methods directly based on the solution of the Hamilton-Jacobi-Bellman (HJB) equation. We provide formal performance guarantees by proving a predetermined upper bound. Moreover, we also prove the existence of a lower bound for interexecution time. For implementation purposes, an adaptive dynamic programming (ADP) method is developed to realize the ETOC using a critic neural network (NN) to approximate the value function of the HJB equation. Subsequently, we prove that semiglobal uniform ultimate boundedness can be guaranteed for states and NN weight errors with the ADP-based ETOC. Simulation results demonstrate the effectiveness of the developed ADP-based ETOC method.
Collapse
|
34
|
Online event-triggered adaptive critic design for non-zero-sum games of partially unknown networked systems. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.07.029] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
35
|
Event-triggered H∞ optimal control for continuous-time nonlinear systems using neurodynamic programming. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.06.090] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
36
|
Yang X, He H. Adaptive Critic Designs for Event-Triggered Robust Control of Nonlinear Systems With Unknown Dynamics. IEEE TRANSACTIONS ON CYBERNETICS 2019; 49:2255-2267. [PMID: 29993650 DOI: 10.1109/tcyb.2018.2823199] [Citation(s) in RCA: 48] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
This paper develops a novel event-triggered robust control strategy for continuous-time nonlinear systems with unknown dynamics. To begin with, the event-triggered robust nonlinear control problem is transformed into an event-triggered nonlinear optimal control problem by introducing an infinite-horizon integral cost for the nominal system. Then, a recurrent neural network (RNN) and adaptive critic designs (ACDs) are employed to solve the derived event-triggered nonlinear optimal control problem. The RNN is applied to reconstruct the system dynamics based on collected system data. After acquiring the knowledge of system dynamics, a unique critic network is proposed to obtain the approximate solution of the event-triggered Hamilton-Jacobi-Bellman equation within the framework of ACDs. The critic network is updated by using simultaneously historical and instantaneous state data. An advantage of the present critic network update law is that it can relax the persistence of excitation condition. Meanwhile, under a newly developed event-triggering condition, the proposed critic network tuning rule not only guarantees the critic network weights to converge to optimums but also ensures nominal system states to be uniformly ultimately bounded. Moreover, by using Lyapunov method, it is proved that the derived optimal event-triggered control (ETC) guarantees uniform ultimate boundedness of all the signals in the original system. Finally, a nonlinear oscillator and an unstable power system are provided to validate the developed robust ETC scheme.
Collapse
|
37
|
Xu B, Shou Y, Luo J, Pu H, Shi Z. Neural Learning Control of Strict-Feedback Systems Using Disturbance Observer. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019; 30:1296-1307. [PMID: 30222586 DOI: 10.1109/tnnls.2018.2862907] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
This paper studies the compound learning control of disturbed uncertain strict-feedback systems. The design is using the dynamic surface control equipped with a novel learning scheme. This paper integrates the recently developed online recorded data-based neural learning with the nonlinear disturbance observer (DOB) to achieve good "understanding" of the system uncertainty including unknown dynamics and time-varying disturbance. With the proposed method to show how the neural networks and DOB are cooperating with each other, one indicator is constructed and included into the update law. The closed-loop system stability analysis is rigorously presented. Different kinds of disturbances are considered in a third-order system as simulation examples and the results confirm that the proposed method achieves higher tracking accuracy while the compound estimation is much more precise. The design is applied to the flexible hypersonic flight dynamics and a better tracking performance is obtained.
Collapse
|
38
|
Wan X, Wang Z, Wu M, Liu X. H ∞ State Estimation for Discrete-Time Nonlinear Singularly Perturbed Complex Networks Under the Round-Robin Protocol. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019; 30:415-426. [PMID: 29994721 DOI: 10.1109/tnnls.2018.2839020] [Citation(s) in RCA: 48] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
This paper investigates the H∞ state estimation problem for a class of discrete-time nonlinear singularly perturbed complex networks (SPCNs) under the Round-Robin (RR) protocol. A discrete-time nonlinear SPCN model is first devised on two time scales with their discrepancies reflected by a singular perturbation parameter (SPP). The network measurement outputs are transmitted via a communication network where the data transmissions are scheduled by the RR protocol with hope to avoid the undesired data collision. The error dynamics of the state estimation is governed by a switched system with a periodic switching parameter. A novel Lyapunov function is constructed that is dependent on both the transmission order and the SPP. By establishing a key lemma specifically tackling the SPP, sufficient conditions are obtained such that, for any SPP less than or equal to a predefined upper bound, the error dynamics of the state estimation is asymptotically stable and satisfies a prescribed H∞ performance requirement. Furthermore, the explicit parameterization of the desired state estimator is given by means of the solution to a set of matrix inequalities, and the upper bound of the SPP is then evaluated in the feasibility of these matrix inequalities. Moreover, the corresponding results for linear discrete-time SPCNs are derived as corollaries. A numerical example is given to illustrate the effectiveness of the proposed state estimator design scheme.
Collapse
|
39
|
Lyu W, Cheng X, Wang J. An Improved Adaptive Compensation H∞ Filtering Method for the SINS' Transfer Alignment Under a Complex Dynamic Environment. SENSORS 2019; 19:s19020401. [PMID: 30669475 PMCID: PMC6359554 DOI: 10.3390/s19020401] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/19/2018] [Revised: 01/15/2019] [Accepted: 01/16/2019] [Indexed: 11/29/2022]
Abstract
Transfer alignment on a moving base under a complex dynamic environment is one of the toughest challenges in a strapdown inertial navigation system (SINS). With the aim of improving rapidity and accuracy, velocity plus attitude matching is applied in the transfer alignment model. Meanwhile, the error compensation model is established to calibrate and compensate the errors of inertial sensors online. To suppress the filtering divergence during the process of transfer alignment, this paper proposes an improved adaptive compensation H∞ filtering method. The cause of filtering divergence has been analyzed carefully and the corresponding adjustment and optimization have been made in the proposed adaptive compensation H∞ filter. In order to balance accuracy and robustness of the transfer alignment system, the robustness factor of the adaptive compensation H∞ filter can be dynamically adjusted according to the complex external environment. The aerial transfer alignment experiments illustrate that the adaptive compensation H∞ filter can effectively improve the transfer alignment accuracy and the pure inertial navigation accuracy under a complex dynamic environment, which verifies the advantage of the proposed method.
Collapse
Affiliation(s)
- Weiwei Lyu
- School of Instrument Science & Engineering, Southeast University, Nanjing 210096, China.
- Key Laboratory of Micro-Inertial Instrument and Advanced Navigation Technology, Ministry of Education, Southeast University, Nanjing 210096, China.
| | - Xianghong Cheng
- School of Instrument Science & Engineering, Southeast University, Nanjing 210096, China.
- Key Laboratory of Micro-Inertial Instrument and Advanced Navigation Technology, Ministry of Education, Southeast University, Nanjing 210096, China.
| | - Jinling Wang
- School of Civil and Environmental Engineering, University of New South Wales, Sydney, NSW 2052, Australia.
| |
Collapse
|