Leader-Follower Bipartite Output Synchronization on Signed Digraphs Under Adversarial Factors via Data-Based Reinforcement Learning

ADP-based fault-tolerant consensus control for multiagent systems with irregular state constraints

This paper investigates the consensus control issue for nonlinear multiagent systems (MASs) subject to irregular state constraints and actuator faults using an adaptive dynamic programming (ADP) algorithm. Unlike the regular state constraints considered in previous studies, this paper addresses irregular state constraints that may exhibit asymmetry, time variation, and can emerge or disappear during operation. By developing a system transformation method based on one-to-one state mapping, equivalent unconstrained MASs can be obtained. Subsequently, a finite-time distributed observer is designed to estimate the state information of the leader, and the consensus control problem is transformed into the tracking control problem for each agent to ensure that actuator faults of any agent cannot affect its neighboring agents. Then, a critic-only ADP-based fault tolerant control strategy, which consists of the optimal control policy for nominal system and online fault compensation for time-varying addictive faults, is proposed to achieve optimal tracking control. To enhance the learning efficiency of critic neural networks (NNs), an improved weight learning law utilizing stored historical data is employed, ensuring the convergence of critic NN weights towards ideal values under a finite excitation condition. Finally, a practical example of multiple manipulator systems is presented to demonstrate the effectiveness of the developed control method.

Data-Driven Optimal Bipartite Consensus Control for Second-Order Multiagent Systems via Policy Gradient Reinforcement Learning

This article investigates the optimal bipartite consensus control (OBCC) problem for unknown second-order discrete-time multiagent systems (MASs). First, the coopetition network is constructed to describe the cooperative and competitive relationships between agents, and the OBCC problem is proposed by the tracking error and related performance index function. Based on the distributed policy gradient reinforcement learning (RL) theory, a data-driven distributed optimal control strategy is obtained to guarantee the bipartite consensus of all agents' position and velocity states. In addition, the offline data sets ensure the learning efficiency of the system. These data sets are generated by running the system in real time. Besides, the designed algorithm is an asynchronous version, which is essential to solve the challenge caused by the computational ability difference between nodes in MASs. Then, by means of the functional analysis and Lyapunov theory, the stability of the proposed MASs and the convergence of the learning process are analyzed. Furthermore, an actor-critic structure containing two neural networks is used to implement the proposed methods. Finally, a numerical simulation shows the effectiveness and validity of the results.

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS With Unidentified Exosystem Dynamics

In this article, a data-efficient off-policy reinforcement learning (RL) approach is proposed for distributed output tracking control of heterogeneous multiagent systems (HMASs) using approximate dynamic programming (ADP). Different from existing results that the kinematic model of the exosystem is addressable to partial or all agents, the dynamics of the exosystem are assumed to be completely unknown for all agents in this article. To solve this difficulty, an identifiable algorithm using the experience-replay method is designed for each agent to identify the system matrices of the novel reference model instead of the original exosystem. Then, an output-based distributed adaptive output observer is proposed to provide the estimations of the leader, and the proposed observer not only has a low dimension and less data transmission among agents but also is implemented in a fully distributed way. Besides, a data-efficient RL algorithm is given to design the optimal controller offline along with the system trajectories without solving output regulator equations. An ADP approach is developed to iteratively solve game algebraic Riccati equations (GAREs) using online information of state and input in an online way, which relaxes the requirement of knowing prior knowledge of agents' system matrices in an offline way. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Data-Based Output Synchronization of Multi-Agent Systems With Actuator Faults

In this brief, the output synchronization of multi-agent systems (MAS) with actuator faults is studied. To detect the faults, a backward input-driven fault detection mechanism (BIFDM) is presented for MAS. Different from previous works, the system operation can be monitored without system model by the proposed BIFDM. Then to tolerate the faults, a novel fault-tolerant controller (FTC) based on reinforcement learning (RL) and backward information (BI) is proposed. Particularly, by the combination of BI, the design of additional parameters for faults is avoided. Furthermore, the proposed FTC overcomes the shortcoming that the previous FTCs cannot be applied to heterogeneous MAS. Finally, two simulation examples are given to verify the effectiveness of the proposed methods.

Underwater robot coordination using a bio-inspired electrocommunication system

Due to the challenging communication and control systems, few underwater multi-robot coordination systems are currently developed. In nature, weakly electric fish can organize their collective activities using electrocommunication in turbid water. Inspired by this communication mechanism, we developed an artificial electrocommunication system for underwater robots in our previous work. In this study, we coordinate a group of underwater robots using this bio-inspired electrocommunication. We first design a time division multiple access (TDMA) network protocol for electrocommunication to avoid communication conflicts during multi-robot coordination. Then, we revise a distributed controller to coordinate a group of underwater robots. The distributed controller on each robot generates the required controls based on adjacent states obtained through electrocommunication. A central pattern generator (CPG) controller is designed to adjust the speed of individuals according to distributed control law. Simulations and experimental results show that a group of underwater robots is able to achieve coordination with the developed electrocommunication and control systems.

Distributed optimized dynamic event-triggered control for unknown heterogeneous nonlinear MASs with input-constrained

The distributed optimized dynamic event-triggered controller is investigated for completely unknown heterogeneous nonlinear multi-agent systems (MASs) on a directed graph subject to input-constrained. First, the distributed observer is designed to estimate the information of the leader for each follower, and a network of the augmented system is constructed by employing the dynamics of the followers and the observers. An identifier with a compensator is designed to approximate the unknown augmented system (agent) with an arbitrarily small identifier error. Then, consider that the input-constrained optimal controller, along with Hamilton-Jacobi-Bellman (HJB) equation, is under pressure to execute in certain systems associated with bottlenecks such as communication and computing burdens. A critic-actor-based optimized dynamic event-triggered controller, which tunes the parameters of critic-actor neural networks (NNs) by the dynamic triggering mechanism, is leveraged to determine the rule of aperiodic sampling and maintain the desired synchronization service. In addition, the existence of a positive minimum inter-event time (MIET) between consecutive events is also proved. Finally, the applications in non-identical nonlinear MAS and 2-DOF robots illustrate the availability of the proposed theoretical results.

Distributed Controller Design and Analysis of Second-Order Signed Networks With Communication Delays

This article concentrates on dealing with distributed control problems for second-order signed networks subject to not only cooperative but also antagonistic interactions. A distributed control protocol is proposed based on the nearest neighbor rules, with which necessary and sufficient conditions are developed for consensus of second-order signed networks whose communication topologies are described by strongly connected signed digraphs. Besides, another distributed control protocol in the presence of a communication delay is designed, for which a time margin of the delay can be determined simultaneously. It is shown that under the delay margin condition, necessary and sufficient consensus results can be derived even though second-order signed networks with a communication delay are considered. Simulation examples are included to illustrate the validity of our established consensus results of second-order signed networks.