Learning-Based Robust Tracking Control of Quadrotor With Time-Varying and Coupling Uncertainties

Concurrent Learning-Based Adaptive Control of Underactuated Robotic Systems With Guaranteed Transient Performance for Both Actuated and Unactuated Motions

With the wide applications of underactuated robotic systems, more complex tasks and higher safety demands are put forward. However, it is still an open issue to utilize "fewer" control inputs to satisfy control accuracy and transient performance with theoretical and practical guarantee, especially for unactuated variables. To this end, for underactuated robotic systems, this article designs an adaptive tracking controller to realize exponential convergence results, rather than only asymptotic stability or boundedness; meanwhile, unactuated states exponentially converge to a small enough bound, which is adjustable by control gains. The maximum motion ranges and convergence speed of all variables both exhibit satisfactory performance with higher safety and efficiency. Here, a data-driven concurrent learning (CL) method is proposed to compensate for unknown dynamics/disturbances and improve the estimate accuracy of parameters/weights, without the need for persistency of excitation or linear parametrization (LP) conditions. Then, a disturbance judgment mechanism is utilized to eliminate the detrimental impacts of external disturbances. As far as we know, for general underactuated systems with uncertainties/disturbances, it is the first time to theoretically and practically ensure transient performance and exponential convergence speed for unactuated states, and simultaneously obtain the exponential tracking result of actuated motions. Both theoretical analysis and hardware experiment results illustrate the effectiveness of the designed controller.

Neural Network-Based Robust Guaranteed Cost Control for Image-Based Visual Servoing of Quadrotor

In this article, a neural network (NN)-based robust guaranteed cost control design is proposed for image-based visual servoing (IBVS) control of quadrotors. According to the dynamics of three subsystems (yaw, height, and lateral subsystems) derived from the quadrotor IBVS dynamic model, the main control design is to solve the robust control problem for the time-varying lateral subsystem with angle constraints and uncertain disturbances. Considering the system dynamics, a two-loop structure is conducted. The outer loop uses the linear quadratic regulator to solve the Riccati equation for the lateral image feature system, and the inner loop adopts the optimal robust guaranteed cost control to solve the lateral velocity system. For the lateral velocity system, the optimal robust control problem is transformed to solve the modified Hamilton-Jacobi-Bellman equation of the corresponding optimal control problem utilizing adaptive dynamic programming. The implementation is accomplished with the time-varying NN and the designed estimated weight update law. In addition, the stability and effectiveness are proved by the theoretic proof and simulations.

Event-Triggered Adaptive Containment Control for Heterogeneous Stochastic Nonlinear Multiagent Systems

This article investigates the event-triggered adaptive containment control problem for a class of stochastic nonlinear multiagent systems with unmeasurable states. A stochastic system with unknown heterogeneous dynamics is established to describe the agents in a random vibration environment. Besides, the uncertain nonlinear dynamics are approximated by radial basis function neural networks (NNs), and the unmeasured states are estimated by constructing the NN-based observer. In addition, the switching-threshold-based event-triggered control method is adopted with the hope of reducing communication consumption and balancing system performance and network constraints. Moreover, we develop the novel distributed containment controller by utilizing the adaptive backstepping control strategy and the dynamic surface control (DSC) approach such that the output of each follower converges to the convex hull spanned by multiple leaders, and all signals of the closed-loop system are cooperatively semi-globally uniformly ultimately bounded in mean square. Finally, we verify the efficiency of the proposed controller by the simulation examples.

Advanced Optimal Tracking Control With Stability Guarantee via Novel Value Learning Formulation

In this article, to solve the optimal tracking control problem (OTCP) for discrete-time (DT) nonlinear systems, general value iteration (GVI) scheme and online value iteration (VI) algorithms with novel value function are discussed. First, the disadvantage of the traditional value function for the OTCP is presented and the novel value function is introduced. Second, we analyze the monotonicity and convergence of GVI and establish the admissibility condition of GVI to evaluate the admissibility of the current iterative control. Note that a novel approach is introduced to analyze the admissibility. Third, based on the attraction domain, improved control policies with online VI can be obtained by judging the location of the current tracking error and reference point. Finally, the stability of the online VI-based control system is guaranteed. Besides, we provide two simulation examples to show the performance of the proposed methods.

Distributed Optimal Attitude Synchronization Control of Multiple QUAVs via Adaptive Dynamic Programming

This article proposes a distributed optimal attitude synchronization control strategy for multiple quadrotor unmanned aerial vehicles (QUAVs) through the adaptive dynamic programming (ADP) algorithm. The attitude systems of QUAVs are modeled as affine nominal systems subject to parameter uncertainties and external disturbances. Considering attitude constraints in complex flying environments, a one-to-one mapping technique is utilized to transform the constrained systems into equivalent unconstrained systems. An improved nonquadratic cost function is constructed for each QUAV, which reflects the requirements of robustness and the constraints of control input simultaneously. To overcome the issue that the persistence of excitation (PE) condition is difficult to meet, a novel tuning rule of critic neural network (NN) weights is developed via the concurrent learning (CL) technique. In terms of the Lyapunov stability theorem, the stability of the closed-loop system and the convergence of critic NN weights are proved. Finally, simulation results on multiple QUAVs show the effectiveness of the proposed control strategy.

Guaranteeing Global Stability for Neuro-Adaptive Control of Unknown Pure-Feedback Nonaffine Systems via Barrier Functions

Most existing approximation-based adaptive control (AAC) approaches for unknown pure-feedback nonaffine systems retain a dilemma that all closed-loop signals are semiglobally uniformly bounded (SGUB) rather than globally uniformly bounded (GUB). To achieve the GUB stability result, this article presents a neuro-adaptive backstepping control approach by blending the mean value theorem (MVT), the barrier Lyapunov functions (BLFs), and the technique of neural approximation. Specifically, we first resort the MVT to acquire the intermediate and actual control inputs from the nonaffine structures directly. Then, neural networks (NNs) are adopted to approximate the unknown nonlinear functions, in which the compact sets for maintaining the approximation capabilities of NNs are predetermined actively through the BLFs. It is shown that, with the developed neuro-adaptive control scheme, global stability of the resulting closed-loop system is ensured. Simulations are conducted to verify and clarify the developed approach.

Nonfragile Output Feedback Tracking Control for Markov Jump Fuzzy Systems Based on Integral Reinforcement Learning Scheme

In this article, a novel integral reinforcement learning (RL)-based nonfragile output feedback tracking control algorithm is proposed for uncertain Markov jump nonlinear systems presented by the Takagi-Sugeno fuzzy model. The problem of nonfragile control is converted into solving the zero-sum games, where the control input and uncertain disturbance input can be regarded as two rival players. Based on the RL architecture, an offline parallel output feedback tracking learning algorithm is first designed to solve fuzzy stochastic coupled algebraic Riccati equations for Markov jump fuzzy systems. Furthermore, to overcome the requirement of a precise system information and transition probability, an online parallel integral RL-based algorithm is designed. Besides, the tracking object is achieved and the stochastically asymptotic stability, and expected H_∞ performance for considered systems is ensured via the Lyapunov stability theory and stochastic analysis method. Furthermore, the effectiveness of the proposed control algorithm is verified by a robot arm system.

Neural Adaptive Self-Triggered Control for Uncertain Nonlinear Systems With Input Hysteresis

The issue of neural adaptive self-triggered tracking control for uncertain nonlinear systems with input hysteresis is considered. Combining radial basis function neural networks (RBFNNs) and adaptive backstepping technique, an adaptive self-triggered tracking control approach is developed, where the next trigger instant is determined by the current information. Compared with the event-triggered control mechanism, its biggest advantage is that it does not need to continuously monitor the trigger condition of the system, which is convenient for physical realization. By the proposed controller, the hysteresis's effect can be compensated effectively and the tracking error can be bounded by an explicit function of design parameters. Simultaneously, all other signals in the closed-loop system can be remaining bounded. Finally, two examples are presented to verify the effectiveness of the proposed method.

Adaptive Control and Application for Nonlinear Systems With Input Nonlinearities and Unknown Virtual Control Coefficients

This article is devoted to an adaptive tracking control problem for nonlinear systems with input deadzone and saturation, whose virtual control coefficients include the known and unknown terms. A novel smooth function is first introduced to approximate the input nonlinearities. By utilizing an auxiliary variable and the Nussbaum gain technique, an improved real control signal is constructed to handle the uncertainties of the virtual control coefficients and input nonlinearities. Furthermore, an adaptive tracking controller is constructed and applied to the attitude control of a quadrotor, which guarantees the boundedness of all the signals in the resulting closed-loop system. Finally, both stability analysis and simulation results validate the effectiveness of the developed control strategy.

Cooperative Adaptive Model-Free Control With Model-Free Estimation and Online Gain Tuning

In this article, a distributed adaptive model-free control algorithm is proposed for consensus and formation-tracking problems in a network of agents with completely unknown nonlinear dynamic systems. The specification of the communication graph in the network is incorporated in the adaptive laws for estimation of the unknown linear and nonlinear terms, and in the online updating of the elements in the main controller gain matrix. The decentralized control signal at each agent in the network requires information about the states of the leader agent, as well as the desired formation variables of the agents in a local coordinate frame. These two sets of variables are provided at each agent by utilizing two recently proposed distributed observers. It is shown that only a spanning-tree rooted at the leader agent is enough for the convergence and stability of the proposed cooperative control and observer algorithms. Two simulation studies are provided to evaluate the performance of the proposed algorithm in comparison with two state-of-the-art distributed model-free control algorithms. With lower control effort as well as fewer offline gain tuning, the same level of consensus errors is achieved. Finally, the application of the proposed solution is studied in the formation-tracking control of a team of autonomous aerial mobile robots via simulation results.

Data-Driven Optimal Formation Control for Quadrotor Team With Unknown Dynamics

In this article, the data-driven optimal formation control problem is addressed for a heterogeneous quadrotor team with a virtual leader. Each quadrotor is considered as a highly nonlinear system with six degrees of freedom and the accurate dynamic information of the quadrotor is difficult to obtain in practical applications. An optimal cascade formation controller, including a position controller and an attitude controller, is proposed to track a virtual leader and form a predesigned formation. By using the reinforcement learning (RL) approach, the optimal formation controller is learned from the quadrotor system data without any knowledge of dynamic information of the quadrotors. Simulation results of a heterogeneous multiquadrotor system in a formation flight are given to show the effectiveness of the proposed controllers.

Observer-Based Adaptive Synchronization of Multiagent Systems With Unknown Parameters Under Attacks

This article studies the observer-based adaptive synchronization of multiagent systems (MASs) with unknown parameters under attacks. First, to estimate the state of agents, the observer for MAS is introduced. When disturbance, nonlinear function, and system model uncertainty are not considered, the nominal controller is proposed to achieve synchronization and state estimation. Then, in order to eliminate the effect of unknown parameters in the disturbance, nonlinear function, and system model uncertainty, the adaptive controller with switching term is introduced. However, the attack will lead to the destruction of the network topology so as the destruction of the nominal controller. By constructing an appropriate Lyapunov function, we analyze the effect caused by attacks, and the security control law is given to make sure the synchronization of the MASs under attacks. Finally, a numerical simulation is given to verify the validness of the obtained theorem.

Fast control parameterization optimal control with improved Polak-Ribière-Polyak conjugate gradient implementation for industrial dynamic processes

This paper proposes a fast control parameterization optimal control algorithm for industrial dynamic process with constraints. Derived from the frame of control variable parameterization (CVP) technique, the proposed method combines an efficient gradient computation strategy with an improved nonlinear optimization computation approach to overcome the challenge of computation efficiency caused by gradients and bounds in optimal control problems. Firstly, a fast gradient computation method based on the costate system of Hamiltonian function is developed to decrease the computational expense of gradients by employing approximate treatments and numerical integration strategy. Then, a trigonometric function transformation scheme is presented to tackle the boundary constraints so that the original optimal control problem is further converted into an unconstrained one. On this basis, an improved restricted Polak-Ribière-Polyak (PRP) conjugate gradient approach is introduced to solve the nonlinear optimization problem by using conjugate gradient iterations and strong Wolfe line search. Meanwhile, to enhance the convergence, a restricting condition is imposed in strong Wolfe line search to create iteration step-length. Finally, the proposed algorithm is implemented on three dynamic processes. The detailed comparison among the classical CVP method, literature results and the proposed method are carried out. Simulation studies show that the proposed fast approach averagely saves more than 90% computation time in contrast to the classical CVP method, demonstrating the effectiveness of the proposed fast optimal control approach.

Online Nash Solution in Networked Multirobot Formation Using Stochastic Near-Optimal Control Under Dynamic Events

This article proposes an online stochastic dynamic event-based near-optimal controller for formation in the networked multirobot system. The system is prone to network uncertainties, such as packet loss and transmission delay, that introduce stochasticity in the system. The multirobot formation problem poses a nonzero-sum game scenario. The near-optimal control inputs/policies based on proposed event-based methodology attain a Nash equilibrium achieving the desired formation in the system. These policies are generated online only at events using actor-critic neural network architecture whose weights are updated too at the same instants. The approach ensures system stability by deriving the ultimate boundedness of estimation errors of actor-critic weights and the event-based closed-loop formation error. The efficacy of the proposed approach has been validated in real-time using three Pioneer P3-Dx mobile robots in a multirobot framework. The control update instants are minimized to as low as 20% and 18% for the two follower robots.

Dual Heuristic Programming for Optimal Control of Continuous-Time Nonlinear Systems Using Single Echo State Network

This article presents an improved online adaptive dynamic programming (ADP) algorithm to solve the optimal control problem of continuous-time nonlinear systems with infinite horizon cost. The Hamilton-Jacobi-Bellman (HJB) equation is iteratively approximated by a novel critic-only structure which is constructed using the single echo state network (ESN). Inspired by the dual heuristic programming (DHP) technique, ESN is designed to approximate the costate function, then to derive the optimal controller. As the ESN is characterized by the echo state property (ESP), it is proved that the ESN can successfully approximate the solution to the HJB equation. Besides, to eliminate the requirement for the initial admissible control, a new weight tuning law is designed by adding an alternative condition. The stability of the closed-loop optimal control system and the convergence of the out weights of the ESN are guaranteed by using the Lyapunov theorem in the sense of uniformly ultimately bounded (UUB). Two simulation examples, including linear system and nonlinear system, are given to illustrate the availability and effectiveness of the proposed approach by comparing it with the polynomial neural-network scheme.

Output Feedback Control of Micromechanical Gyroscopes Using Neural Networks and Disturbance Observer

This article addresses the output feedback control of micromechanical (MEMS) gyroscopes using neural networks (NNs) and disturbance observer (DOB). For the unmeasured system states, the state observer and the high gain observer are constructed. The adaptive NNs are investigated to approximate the nonlinear dynamics, including the known nominal terms and the system uncertainties caused by environmental fluctuations. For the time-varying disturbances, the DOB is utilized. The sliding mode control is employed to enhance the robustness. Through simulation verification, the output feedback control using NNs and DOB can adapt to the dynamics of MEMS gyroscope with unmeasured system speed, while an expected effective tracking performance is obtained in the presence of unknown system nonlinearities and external disturbances.

A High-Efficiency Fatigued Speech Feature Selection Method for Air Traffic Controllers Based on Improved Compressed Sensing

Air traffic controller fatigue has recently received considerable attention from researchers because it is one of the main causes of air traffic incidents. Numerous research studies have been conducted to extract speech features related to fatigue, and their practical utilization has achieved some positive detection results. However, there are still challenges associated with the applied speech features usually being of high dimension, which leads to computational complexity and inefficient fatigue detection. This situation makes it meaningful to reduce the dimensionality and select only a few efficient features. This paper addresses these problems by proposing a high-efficiency fatigued speech selection method based on improved compressed sensing. For adapting a method to the specific field of fatigued speech, we propose an improved compressed sensing construction algorithm to decrease the reconstruction error and achieve superior sparse coding. The proposed feature selection method is then applied to optimize the high-dimension fatigued speech features based on the fractal dimension. Finally, a support vector machine classifier is applied to a series of comparative experiments using the Civil Aviation Administration of China radiotelephony corpus to demonstrate that the proposed method provides a significant improvement in the precision of fatigue detection compared with current state-of-the-art approaches.

Output-Feedback Robust Control of Uncertain Systems via Online Data-Driven Learning

Although robust control has been studied for decades, the output-feedback robust control design is still challenging in the control field. This article proposes a new approach to address the output-feedback robust control for continuous-time uncertain systems. First, we transform the robust control problem into an optimal control problem of the nominal linear system with a constructive cost function, which allows simplifying the control design. Then, a modified algebraic Riccati equation (MARE) is constructed by further investigating the corresponding relationship with the state-feedback optimal control. To solve the derived MARE online, the vectorization operation and Kronecker's product are applied to reformulate the output Lyapunov function, and then, a new online data-driven learning method is suggested to learn its solution. Consequently, only the measurable system input and output are used to derive the solution of the MARE. In this case, the output-feedback robust control gain can be obtained without using the unknown system states. The control system stability and convergence of the derived solution are rigorously proved. Two simulation examples are provided to demonstrate the efficacy of the suggested methods.

CED: A Distance for Complex Mass Functions

Evidence theory is an effective methodology for modeling and processing uncertainty that has been widely applied in various fields. In evidence theory, a number of distance measures have been presented, which play an important role in representing the degree of difference between pieces of evidence. However, the existing evidential distances focus on traditional basic belief assignments (BBAs) modeled in terms of real numbers and are not compatible with complex BBAs (CBBAs) extended to the complex plane. Therefore, in this article, a generalized evidential distance measure called the complex evidential distance (CED) is proposed, which can measure the difference or dissimilarity between CBBAs in complex evidence theory. This is the first work to consider distance measures for CBBAs, and it provides a promising way to measure the differences between pieces of evidence in a more general framework of complex plane space. Furthermore, the CED is a strict distance metric with the properties of nonnegativity, nondegeneracy, symmetry, and triangle inequality that satisfies the axioms of a distance. In particular, when the CBBAs degenerate into classical BBAs, the CED will degenerate into Jousselme et al.'s distance. Therefore, the proposed CED is a generalization of the traditional evidential distance, but it has a greater ability to measure the difference or dissimilarity between pieces of evidence. Finally, a decision-making algorithm for pattern recognition is devised based on the CED and is applied to a medical diagnosis problem to illustrate its practicability.

Optimized neural network based sliding mode control for quadrotors with disturbances

In this paper, optimized radial basis function neural networks (RBFNNs) are employed to construct a sliding mode control (SMC) strategy for quadrotors with unknown disturbances. At first, the dynamics model of the controlled quadrotor is built, where some unknown external disturbances are considered explicitly. Then SMC is carried out for the position and the attitude control of the quadrotor. However, there are unknown disturbances in the obtained controllers, so RBFNNs are employed to approximate the unknown parts of the controllers. Furtherly, Particle Swarm optimization algorithm (PSO) based on minimizing the absolute approximation errors is used to improve the performance of the controllers. Besides, the convergence of the state tracking errors of the quadrotor is proved. In order to exposit the superiority of the proposed control strategy, some comparisons are made between the RBFNN based SMC with and without PSO. The results show that the strategy with PSO achieves quicker and smoother trajectory tracking, which verifies the effectiveness of the proposed control strategy.

Event-triggered adaptive finite-time tracking control for full state constraints nonlinear systems with parameter uncertainties and given transient performance

This paper investigates event-triggered finite-time tracking control problem for full state constraints nonlinear systems with uncertain parameters. Considering a class of full state constraints nonlinear systems, a new finite-time barrier Lyapunov function (FTBLF) is constructed, and it is utilized to achieve finite-time tracking control while each state constraints are not violated. Further, to reduce communication resource burden, a time-varying threshold event-triggered mechanism is proposed. Meanwhile, by integrating prescribed exponential function into FTBLF, the transient performance can be guaranteed and free from influences of event-triggered control input. Finally, on the basic of backstepping design, an event-triggered adaptive finite-time tracking control method is developed. The proposed method guarantees that tracking error tends to a small adjustable set and its trajectory is within specified bound, while full state constraints are never violated. Two examples are given to demonstrate the control effect.

Joint Message-Passing and Convex Optimization Framework for Energy-Efficient Surveillance UAV Scheduling

In modern surveillance systems, the use of unmanned aerial vehicles (UAVs) has been actively discussed in order to extend target monitoring areas, even for an extreme circumstances. This paper proposes an energy-efficient UAV-based surveillance system that operates from two different sequential methods. First, the proposed algorithm pursues energy-efficient operations by deactivating selected surveillance cameras on the UAVs located in overlapping areas. For this objective, a message-passing based algorithm is used because the overlapping situations can be formulated using a max-weight independent set. Next, the unscheduled UAVs based on the message-passing fly to the charging towers to be charged. This algorithm computes the optimal matching between the UAVs and charging towers and the amount of energy allocation for the scheduled UAV-tower pairs. This joint optimization is initially formulated as non-convex, and it is then reformulated to be convex, which can guarantee optimal solutions. The proposed framework achieves the desired performance, as presented in the performance evaluation.

Model-Free Optimized Tracking Control Heuristic

Many tracking control solutions proposed in the literature rely on various forms of tracking error signals at the expense of possibly overlooking other dynamic criteria, such as optimizing the control effort, overshoot, and settling time, for example. In this article, a model-free control architectural framework is presented to track reference signals while optimizing other criteria as per the designer’s preference. The control architecture is model-free in the sense that the plant’s dynamics do not have to be known in advance. To this end, we propose and compare four tracking control algorithms which synergistically integrate a few machine learning tools to compromise between tracking a reference signal and optimizing a user-defined dynamic cost function. This is accomplished via two orchestrated control loops, one for tracking and one for optimization. Two control algorithms are designed and compared for the tracking loop. The first is based on reinforcement learning while the second is based on nonlinear threshold accepting technique. The optimization control loop is implemented using an artificial neural network. Each controller is trained offline before being integrated in the aggregate control system. Simulation results of three scenarios with various complexities demonstrated the effectiveness of the proposed control schemes in forcing the tracking error to converge while minimizing a pre-defined system-wide objective function.

Risk-Aware Identification of Highly Suspected COVID-19 Cases in Social IoT: A Joint Graph Theory and Reinforcement Learning Approach

The recent outbreak of the coronavirus disease 2019 (COVID-19) has rapidly become a pandemic, which calls for prompt action in identifying suspected cases at an early stage through risk prediction. To suppress its further spread, we exploit the social relationships between mobile devices in the Social Internet of Things (SIoT) to help control its propagation by allocating the limited protective resources to the influential so-called high-degree individuals to stem the tide of precipitated spreading. By exploiting the so-called differential contact intensity and the infectious rate in susceptible-exposed-infected-removed (SEIR) epidemic model, the resultant optimization problem can be transformed into the minimum weight vertex cover (MWVC) problem of graph theory. To solve this problem in a high-dynamic random network topology, we propose an adaptive scheme by relying on the graph embedding technique during the state representation and reinforcement learning in the training phase. By relying on a pair of real-life datasets, the results demonstrate that our scheme can beneficially reduce the epidemiological reproduction rate of the infection. This technique has the potential of assisting in the early identification of COVID-19 cases.

Learning Output Reference Model Tracking for Higher-Order Nonlinear Systems with Unknown Dynamics

This work suggests a solution for the output reference model (ORM) tracking control problem, based on approximate dynamic programming. General nonlinear systems are included in a control system (CS) and subjected to state feedback. By linear ORM selection, indirect CS feedback linearization is obtained, leading to favorable linear behavior of the CS. The Value Iteration (VI) algorithm ensures model-free nonlinear state feedback controller learning, without relying on the process dynamics. From linear to nonlinear parameterizations, a reliable approximate VI implementation in continuous state-action spaces depends on several key parameters such as problem dimension, exploration of the state-action space, the state-transitions dataset size, and a suitable selection of the function approximators. Herein, we find that, given a transition sample dataset and a general linear parameterization of the Q-function, the ORM tracking performance obtained with an approximate VI scheme can reach the performance level of a more general implementation using neural networks (NNs). Although the NN-based implementation takes more time to learn due to its higher complexity (more parameters), it is less sensitive to exploration settings, number of transition samples, and to the selected hyper-parameters, hence it is recommending as the de facto practical implementation. Contributions of this work include the following: VI convergence is guaranteed under general function approximators; a case study for a low-order linear system in order to generalize the more complex ORM tracking validation on a real-world nonlinear multivariable aerodynamic process; comparisons with an offline deep deterministic policy gradient solution; implementation details and further discussions on the obtained results.