Cooperative Adaptive Output Regulation for Lower Triangular Nonlinear Multi-Agent Systems Subject to Jointly Connected Switching Networks

Optimal Consensus Control for Continuous-Time Linear Multiagent Systems: A Dynamic Event-Triggered Approach

This article investigates the optimal consensus problem for general linear multiagent systems (MASs) via a dynamic event-triggered approach. First, a modified interaction-related cost function is proposed. Second, a dynamic event-triggered approach is developed by constructing a new distributed dynamic triggering function and a new distributed event-triggered consensus protocol. Consequently, the modified interaction-related cost function can be minimized by applying the distributed control laws, which overcomes the difficulty in the optimal consensus problem that seeking the interaction-related cost function needs all agents' information. Then, some sufficient conditions are obtained to guarantee optimality. It is shown that the developed optimal consensus gain matrices are only related to the designed triggering parameters and the desirable modified interaction-related cost function, relaxing the constraint that the controller design requires the knowledge of system dynamics, initial states, and network scale. Meanwhile, the tradeoff between optimal consensus performance and event-triggered behavior is also considered. Finally, a simulation example is provided to verify the validity of the designed distributed event-triggered optimal controller.

Cooperative Containment Control for Multiagent Systems With Reduced-Order Protocols

This article addresses the problem of containment control for continuous-time multiagent systems. A containment error is first given to show the coordination between the outputs of leaders and followers. Then, an observer is designed based on the neighbor observable convex hull state. Under the assumption that the designed reduced-order observer is subject to external disturbances, a reduced-order protocol is designed to realize the containment coordination. In order to ensure the designed control protocol can achieve the effect of the main theories, a corresponding Sylvester equation is given with a novel approach which proves that the Sylvester equation is solvable. Finally, a numerical example is given to verify the validity of the main results.

Consensus of a novel heuristic nonlinear multi-agent system in DOS attack network environment via saturation impulse control mechanism

This paper mainly studies the consensus control strategy for a novel heuristic nonlinear multi-agent system. Compared with most existing related researches, firstly, the novel heuristic nonlinear multi-agent system has the ability to construct its communication network topology heuristically, and can withstand long-term DOS(Denial of Service) attacks, with the advantages of high practicality and security. Secondly, in order to control the multi-agent system, a control protocol based on both saturation effect and impulse control mechanism is studied, which has the advantages of high efficiency, low cost and wide applicability. Thirdly, for the multi-agent system, its dynamic model is constructed and analyzed by Lyapunov stability theory and matrix measure theory, and some sufficient conditions for achieving consensus are obtained. Finally, through two simulation experiments and some corresponding comparative analysis, the correctness, efficiency, and superiority of the theories proposed in this paper were verified.

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS With Unidentified Exosystem Dynamics

In this article, a data-efficient off-policy reinforcement learning (RL) approach is proposed for distributed output tracking control of heterogeneous multiagent systems (HMASs) using approximate dynamic programming (ADP). Different from existing results that the kinematic model of the exosystem is addressable to partial or all agents, the dynamics of the exosystem are assumed to be completely unknown for all agents in this article. To solve this difficulty, an identifiable algorithm using the experience-replay method is designed for each agent to identify the system matrices of the novel reference model instead of the original exosystem. Then, an output-based distributed adaptive output observer is proposed to provide the estimations of the leader, and the proposed observer not only has a low dimension and less data transmission among agents but also is implemented in a fully distributed way. Besides, a data-efficient RL algorithm is given to design the optimal controller offline along with the system trajectories without solving output regulator equations. An ADP approach is developed to iteratively solve game algebraic Riccati equations (GAREs) using online information of state and input in an online way, which relaxes the requirement of knowing prior knowledge of agents' system matrices in an offline way. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Robust cooperative output regulation for heterogeneous nonlinear multi-agent systems with an unknown exosystem subject to jointly connected switching networks

This paper investigates the robust cooperative output regulation problem for heterogeneous lower triangular nonlinear multi-agent systems with an unknown exosystem over jointly connected switching networks. The problem has been studied for the exactly known exosystem over switching networks. However, the existing result for the unknown exosystem is still limited to the static networks. To ensure that all followers acquire the reference trajectory generated by the unknown exosystem through the jointly connected switching networks, by combining a set of auxiliary filtering variables and fixed-time stability theory, an adaptive distributed observer is designed. On the basis of the adaptive distributed observer and the distributed internal model approach, we propose a distributed controller under several standard assumptions to solve the problem. Compared with the similar work subject to the static networks, the controller in this paper is applicable to the more general communication network while weakening the assumptions of the controlled system.

SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multiagent Reinforcement Learning

Learning a stable and generalizable centralized value function (CVF) is a crucial but challenging task in multiagent reinforcement learning (MARL), as it has to deal with the issue that the joint action space increases exponentially with the number of agents in such scenarios. This article proposes an approach, named SMIX( λ ), that uses an OFF-policy training to achieve this by avoiding the greedy assumption commonly made in CVF learning. As importance sampling for such OFF-policy training is both computationally costly and numerically unstable, we proposed to use the λ -return as a proxy to compute the temporal difference (TD) error. With this new loss function objective, we adopt a modified QMIX network structure as the base to train our model. By further connecting it with the Q(λ) approach from a unified expectation correction viewpoint, we show that the proposed SMIX( λ ) is equivalent to Q(λ) and hence shares its convergence properties, while without being suffered from the aforementioned curse of dimensionality problem inherent in MARL. Experiments on the StarCraft Multiagent Challenge (SMAC) benchmark demonstrate that our approach not only outperforms several state-of-the-art MARL methods by a large margin but also can be used as a general tool to improve the overall performance of other centralized training with decentralized execution (CTDE)-type algorithms by enhancing their CVFs.

Distributed Observer Design for Linear Systems to Achieve Omniscience Asymptotically Under Jointly Connected Switching Networks

The distributed observer problem is motivated by the case where the output information of the system is decentralized in different subsystems. In this scene, all the subsystems form an observer network, and each of them has access to only a part of output information and the information exchanged via the given communication networks. Due to the limitation of communication conditions among subsystems, the communication network is often time varying and disconnected. However, the existing research about the aforementioned scene is still not enough to solve this problem. To this end, this article is concerned with the challenge of the distributed observer design for linear systems under time-variant disconnected communication networks. The design method is successfully established by fixing both completely decentralized output information and incompletely decentralized output information into account. Our work overcomes the limitation of the existing results that the distributed observer can only reconstruct the full states of the underlying systems by means of fast switching. In the case of completely decentralized output information, a group of sufficient conditions is put forward for the system matrix, and it is proved that the asymptotical omniscience of the distributed observer could be achieved as long as anyone of the developed conditions is satisfied. Furthermore, unlike similar problems in multiagent systems, the systems that can meet the proposed conditions are not only stable and marginally stable systems but also some unstable systems. As for the case where the output information is not completely decentralized, the results show with the observable decomposition and states reorganization technology that the distributed observer could achieve omniscience asymptotically without any constraints on the system matrix. The validity of the proposed design method is emphasized in two numerical simulations.

Couple-Group Consensus of Cooperative-Competitive Heterogeneous Multiagent Systems: A Fully Distributed Event-Triggered and Pinning Control Method

This article discusses the couple-group consensus for heterogeneous multiagent systems via event-triggered and pinning control methods. Considering cooperative-competitive interaction among the agents, a novel group consensus protocol is designed. As inducing the time-correlation threshold function, a class of fully distributed event-triggered conditions without depending on any global information is proposed. Utilizing the Lyapunov stability theory, some sufficient conditions are obtained. Under hybrid event triggered and pinning control, pinning control strategies are first introduced. It is shown that under the proposed strategies, all agents can asymptotically achieve pinning couple-group consensus with discontinuous communication in a fully distributed way. Furthermore, the Zeno behavior for each agent is overcome. Finally, the reduction of the systems' controller update frequency and the correctness of our conclusions are illustrated by some simulations.

Adaptive Neural Control for a Class of Nonlinear Multiagent Systems

This article studies the adaptive neural controller design for a class of uncertain multiagent systems described by ordinary differential equations (ODEs) and beams. Three kinds of agent models are considered in this study, i.e., beams, nonlinear ODEs, and coupled ODE and beams. Both beams and ODEs contain completely unknown nonlinearities. Moreover, the control signals are assumed to suffer from a class of generalized backlash nonlinearities. First, neural networks (NNs) are adopted to approximate the completely unknown nonlinearities. New barrier Lyapunov functions are constructed to guarantee the compact set conditions of the NNs. Second, new adaptive neural proportional integral (PI)-type controllers are proposed for the networked ODEs and beams. The parameters of the PI controllers are adaptively tuned by NNs, which can make the system output remain in a prescribed time-varying constraint. Two illustrative examples are presented to demonstrate the advantages of the obtained results.