Leader Selection via Supermodular Game for Formation Control in Multiagent Systems

Event-Triggered Adaptive Neural Network Tracking Control for Uncertain Systems With Unknown Input Saturation Based on Command Filters

This brief presents a modified event-triggered command filter backstepping tracking control scheme for a class of uncertain nonlinear systems with unknown input saturation based on the adaptive neural network (NN) technique. First, the virtual control functions are reconstructed to address the uncertainties in subsystems by using command filters. A piecewise continuous function is employed to deal with the unknown input saturation problem. Next, an event-triggered tracking controller is developed by utilizing the adaptive NN technique. Compared with standard NN control schemes based on multiple-function-approximators, our controller only requires a single NN. The closed-loop system stability is analyzed based on the Lyapunov stability theorem, and it is shown that the Zeno behavior is also avoided under the designed event-triggering mechanism. Simulation studies are performed to validate the effectiveness of our controller.

Adaptive Neural Consensus for Fractional-Order Multi-Agent Systems With Faults and Delays

This article investigates the consensus control for a class of fractional-order (FO) nonlinear multi-agent systems (MASs). Severe sensor/actuator faults and time-varying delays are both considered in the FO MASs. The severe faults may cause unknown control directions in MASs. A new adaptive controller, which is composed of a distributed FO Nussbaum gain, an FO filter, and an auxiliary function, is presented to deal with the severe faults. To cope with the time-varying delays, two different methods are proposed based on barrier Lyapunov function and Lyapunov-Krasovskii function, respectively. Meanwhile, the radial basis function neural network (RBF NN) is applied to approximate the unknown nonlinear functions during the design procedures. This can result in a low-complexity controller. Finally, two simulation examples are used to verify the validity of the proposed schemes.

Improvement of Reinforcement Learning With Supermodularity

Reinforcement learning (RL) is a promising approach to tackling learning and decision-making problems in a dynamic environment. Most studies on RL focus on the improvement of state evaluation or action evaluation. In this article, we investigate how to reduce action space by using supermodularity. We consider the decision tasks in the multistage decision process as a collection of parameterized optimization problems, where state parameters dynamically vary along with the time or stage. The optimal solutions of these parameterized optimization problems correspond to the optimal actions in RL. For a given Markov decision process (MDP) with supermodularity, the monotonicity of the optimal action set and the optimal selection with respect to state parameters can be obtained by using the monotone comparative statics. Accordingly, we propose a monotonicity cut to remove unpromising actions from the action space. Taking bin packing problem (BPP) as an example, we show how the supermodularity and monotonicity cut work in RL. Finally, we evaluate the monotonicity cut on the benchmark datasets reported in the literature and compare the proposed RL with some popular baseline algorithms. The results show that the monotonicity cut can effectively improve the performance of RL.

Jamming attack against remote state estimation over multiple wireless channels: A reinforcement learning based game theoretical approach

In this paper, we design a potential game to investigate the interplay between sensors and attackers in wireless remote estimation systems (RESs), where we consider both closed-loop and open-loop scenarios that depend on whether the sensor's actions are known to the attacker or not. As a typical application of Cyber-Physical Systems (CPSs), wireless RESs have been showing vulnerable to channel jamming attacks. As a countermeasure, the wireless sensor in the estimation system can switch to another independent wireless channel to transmit sensor data. However, a smart attacker can also switch the target channel to launch a jamming attack, resulting in a game between the sensor and attacker. Moreover, the reinforcement learning based methods are proposed to obtain solutions of the designed game models in two scenarios. We also provide extensive simulation results to validate the effectiveness of the proposed methods.

Reinforcement-Learning-Based Asynchronous Formation Control Scheme for Multiple Unmanned Surface Vehicles

The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.

Finite-Time Consensus of Second-Order Switched Nonlinear Multi-Agent Systems

In this brief, the practical finite-time consensus (FTC) problem is investigated for the second-order heterogeneous switched nonlinear multi-agent systems (MASs), where the subsystems and the switching signal for each agent are different. Mainly due to that agents' dynamics are switched and the unknown nonlinearities in the systems are more general, the practical FTC problem of the MASs is rather difficult to be solved by existing methods. As such, a new protocol design framework for the FTC problem is developed. Then, a novel adaptive protocol is proposed for the switched nonlinear MASs based on the developed design framework and the neural network method. The sufficient conditions for the practical FTC of nonlinear MASs under arbitrary switching are given. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed control scheme.