Game-Based Backstepping Design for Strict-Feedback Nonlinear Multi-Agent Systems Based on Reinforcement Learning

Multiagent Continual Coordination via Progressive Task Contextualization

Cooperative multiagent reinforcement learning (MARL) has attracted significant attention and has the potential for many real-world applications. Previous arts mainly focus on facilitating the coordination ability from different aspects (e.g., nonstationarity and credit assignment) in single-task or multitask scenarios, ignoring the stream of tasks that appear in a continual manner. This ignorance makes the continual coordination an unexplored territory, neither in problem formulation nor efficient algorithms designed. Toward tackling the mentioned issue, this article proposes an approach, multiagent continual coordination via progressive task contextualization (MACPro). The key point lies in obtaining a factorized policy, using shared feature extraction layers but separated independent task heads, each specializing in a specific class of tasks. The task heads can be progressively expanded based on the learned task contextualization. Moreover, to cater to the popular centralized training with decentralized execution (CTDE) paradigm in MARL, each agent learns to predict and adopt the most relevant policy head based on local information in a decentralized manner. We show in multiple multiagent benchmarks that existing continual learning methods fail, while MACPro is able to achieve close-to-optimal performance. More results also disclose the effectiveness of MACPro from multiple aspects, such as high generalization ability.

An adaptive neural network approach for resilient leader-following consensus control of multi-agent systems under cyber-attacks

This paper addresses the dynamic neural networks (DNNs) based resilient leader-following consensus control of multi-agent systems (MASs) under unidentified false data injection (FDI) attacks. We have examined generic linear leader-following agents in the context of stochastic FDI attacks on the network topology. When information is sent from one agent to another, it is altered as a result of the attacks. In this study, we have introduced a new method to identify FDI attacks using DNNs. The DNNs adapt by adjusting their weights based on system errors, allowing them to approximate the nonlinear dynamics of these attacks using a state translation method for the receiving agent, as we do not have any estimate or the information of the states of the sending agent. The attacks considered in this study are network attacks, which are easier to initiate but harder to counter compared to the traditional input-output attacks. The unknown FDI attacks are estimated with the help of DNNs, which allow the evaluation and isolation of large amplitude attack signals. Unlike previous methods, this approach handles probabilistic stochastic FDI attacks and negates attack estimations from the system dynamics, enhancing the controller resilience. Additionally, the paper extends resilient consensus control to the output feedback methodology, providing a feasible consensus method for MASs under stochastic FDI attacks. Simple design constraints for the consensus control are introduced, and the approach is validated through simulations with six unmanned ground vehicles (UGVs).