Neural network-based optimal adaptive output feedback control of a helicopter UAV

Applying neural networks as direct controllers in position and trajectory tracking algorithms for holonomic UAVs

This study compares different neural networks as standalone control algorithms for position and trajectory tracking in holonomic UAVs, specifically quadcopters. The research's novelty lies in applying these algorithms directly for control. A position-tracking algorithm based on the artificial potential field method generated extensive training and validation datasets, simulating the tracked point's diverse trajectory shapes and velocities. The most popular neural network architectures were evaluated on the basis of their trajectory tracking accuracy and computational performance, i.e. single-layer regression networks and double-layer perceptron regression networks, deep neural networks, and residual networks. The results highlight that DNNs achieved the highest trajectory tracking accuracy, as measured by root mean squared errors (1.0830) and correlation coefficients (0.9624 given as Pearson's correlation) while providing satisfactory results and stable flight across untrained scenarios, in opposite to other neural networks. However, simpler architectures, such as single-layer perceptrons, exhibit significantly lower latency, making them suitable for real-time applications despite slightly reduced accuracy. In contrast, ResNet architectures underperformed in terms of accuracy and latency, emphasizing the importance of selecting architectures on the basis of specific control objectives. This study demonstrates that deep neural networks can directly control quadcopters, eliminating the need for conventional control algorithms for UAV position-tracking applications, provided sufficient learning data is available. The proposed approach ensures accurate trajectory tracking, effectively handling sudden turns while maintaining stable flight. These findings highlight the potential of neural networks for UAV control, balancing computational efficiency with high precision and reliability.

Dual-loop control and state prediction analysis of QUAV trajectory tracking based on biological swarm intelligent optimization algorithm

Quadrotor unmanned aerial vehicles (QUAVs) have attracted significant research focus due to their outstanding Vertical Take-Off and Landing (VTOL) capabilities. This research addresses the challenge of maintaining precise trajectory tracking in QUAV systems when faced with external disturbances by introducing a robust, two-tier control system based on sliding mode technology. For position control, this approach utilizes a virtual sliding mode control signal to enhance tracking precision and includes adaptive mechanisms to adjust for changes in mass and external disruptions. In controlling the attitude subsystem, the method employs a sliding mode control framework that secures system stability and compliance with intermediate commands, eliminating the reliance on precise models of the inertia matrix. Furthermore, this study incorporates a deep learning approach that combines Particle Swarm Optimization (PSO) with the Long Short-Term Memory (LSTM) network to foresee and mitigate trajectory tracking errors, thereby significantly enhancing the reliability and safety of mission operations. The robustness and effectiveness of this innovative control strategy are validated through comprehensive numerical simulations.

Reinforcement Learning-Based Fractional-Order Adaptive Fault-Tolerant Formation Control of Networked Fixed-Wing UAVs With Prescribed Performance

This article investigates the fault-tolerant formation control (FTFC) problem for networked fixed-wing unmanned aerial vehicles (UAVs) against faults. To constrain the distributed tracking errors of follower UAVs with respect to neighboring UAVs in the presence of faults, finite-time prescribed performance functions (PPFs) are developed to transform the distributed tracking errors into a new set of errors by incorporating user-specified transient and steady-state requirements. Then, the critic neural networks (NNs) are developed to learn the long-term performance indices, which are used to evaluate the distributed tracking performance. Based on the generated critic NNs, actor NNs are designed to learn the unknown nonlinear terms. Moreover, to compensate for the reinforcement learning errors of actor-critic NNs, nonlinear disturbance observers (DOs) with skillfully constructed auxiliary learning errors are developed to facilitate the FTFC design. Furthermore, by using the Lyapunov stability analysis, it is shown that all follower UAVs can track the leader UAV with predesigned offsets, and the distributed tracking errors are finite-time convergent. Finally, comparative simulation results are presented to show the effectiveness of the proposed control scheme.

Adaptive Neural Network Control of an Uncertain 2-DOF Helicopter With Unknown Backlash-Like Hysteresis and Output Constraints

An adaptive neural network (NN) control is proposed for an unknown two-degree of freedom (2-DOF) helicopter system with unknown backlash-like hysteresis and output constraint in this study. A radial basis function NN is adopted to estimate the unknown dynamics model of the helicopter, adaptive variables are employed to eliminate the effect of unknown backlash-like hysteresis present in the system, and a barrier Lyapunov function is designed to deal with the output constraint. Through the Lyapunov stability analysis, the closed-loop system is proven to be semiglobally and uniformly bounded, and the asymptotic attitude adjustment and tracking of the desired set point and trajectory are achieved. Finally, numerical simulation and experiments on a Quanser's experimental platform verify that the control method is appropriate and effective.

AMITE: A Novel Polynomial Expansion for Analyzing Neural Network Nonlinearities

Polynomial expansions are important in the analysis of neural network nonlinearities. They have been applied thereto addressing well-known difficulties in verification, explainability, and security. Existing approaches span classical Taylor and Chebyshev methods, asymptotics, and many numerical approaches. We find that, while these have useful properties individually, such as exact error formulas, adjustable domain, and robustness to undefined derivatives, there are no approaches that provide a consistent method, yielding an expansion with all these properties. To address this, we develop an analytically modified integral transform expansion (AMITE), a novel expansion via integral transforms modified using derived criteria for convergence. We show the general expansion and then demonstrate an application for two popular activation functions: hyperbolic tangent and rectified linear units. Compared with existing expansions (i.e., Chebyshev, Taylor, and numerical) employed to this end, AMITE is the first to provide six previously mutually exclusive desired expansion properties, such as exact formulas for the coefficients and exact expansion errors. We demonstrate the effectiveness of AMITE in two case studies. First, a multivariate polynomial form is efficiently extracted from a single hidden layer black-box multilayer perceptron (MLP) to facilitate equivalence testing from noisy stimulus-response pairs. Second, a variety of feedforward neural network (FFNN) architectures having between three and seven layers are range bounded using Taylor models improved by the AMITE polynomials and error formulas. AMITE presents a new dimension of expansion methods suitable for the analysis/approximation of nonlinearities in neural networks, opening new directions and opportunities for the theoretical analysis and systematic testing of neural networks.

Robust Admittance Control of Optimized Robot-Environment Interaction Using Reference Adaptation

In this article, a robust control scheme is proposed for robots to achieve an optimal performance in the process of interacting with external forces from environments. The environmental dynamics are defined as a linear model, and the interaction performance is evaluated by a defined cost function, which is composed of trajectory errors and force regulation. Based on admittance control, the reference adaptation method is used to minimize the cost function and achieve the optimal interaction performance. To make the trajectory tracking controller robust to the unknown disturbance of internal system dynamics, an auxiliary system is defined and the approximation optimal controller is designed. Experiments on the Baxter robot are conducted to verify the effectiveness of the proposed method.

Robust Adaptive Control for a Small Unmanned Helicopter Using Reinforcement Learning

This article presents a novel adaptive controller for a small-size unmanned helicopter using the reinforcement learning (RL) control methodology. The helicopter is subject to system uncertainties and unknown external disturbances. The dynamic unmodeling uncertainties of the system are estimated online by the actor network, and the tracking performance function is optimized via the critic network. The estimation error of the actor-critic network and the external unknown disturbances are compensated via the nonlinear robust component based on the sliding mode control method. The stability of the closed-loop system and the asymptotic convergence of the attitude tracking error are proved via the Lyapunov-based stability analysis. Finally, real-time experiments are performed on a helicopter control testbed. The experimental results show that the proposed controller achieves good control performance.

Asynchronous learning for actor-critic neural networks and synchronous triggering for multiplayer system

In this paper, based on actor-critic neural network structure and reinforcement learning scheme, a novel asynchronous learning algorithm with event communication is developed, so as to solve Nash equilibrium of multiplayer nonzero-sum differential game in an adaptive fashion. From the point of optimal control view, each player or local controller wants to minimize the individual infinite-time cost function by finding an optimal policy. In this novel learning framework, each player consists of one critic and one actor, and implements distributed asynchronous policy iteration to optimize decision-making process. In addition, communication burden between the system and players is effectively reduced by setting up a central event generator. Critic network executes fast updates by gradient-descent adaption while actor network gives event-induced updates using the gradient projection. The closed-loop asymptotic stability is ensured along with uniform ultimate convergence. Then, the effectiveness of the proposed algorithm is substantiated on a four-player nonlinear system, revealing that it can significantly reduce sampling numbers without impairing learning accuracy. Finally, by leveraging nonzero-sum game idea, the proposed learning scheme is also applied to solve the lateral-directional stability of a linear aircraft system, and is further extended to a nonlinear vehicle system for achieving adaptive cruise control.

Trajectory Tracking of Underactuated Unmanned Surface Vessels: Non-Singular Terminal Sliding Control with Nonlinear Disturbance Observer

An underactuated unmanned surface vessel (USV) is a nonholonomic system, in which trajectory tracking is a challenging problem that has drawn more and more attention from researchers recently. The control of trajectory tracking is of critical importance since it determines whether the task can be carried out successfully. In this paper, a non-singular terminal sliding model (NTSM) controller is proposed for the trajectory tracking control of the underactuated USV, with a nonlinear disturbance observer which is designed to measure complex environmental disturbances such as wind, waves, and currents. Exploratory simulations were carried out and the results show that the proposed controller is effective and robust for the trajectory tracking of underactuated USVs in the presence of environmental disturbances.

Convergence and Robustness Analysis of Novel Adaptive Multilayer Neural Dynamics-Based Controllers of Multirotor UAVs

Because of the simple structure and strong flexibility, multirotor unmanned aerial vehicles (UAVs) have attracted considerable attention among scientific researches and engineering fields during the past decades. In this paper, a novel adaptive multilayer neural dynamic (AMND)-based controllers design method is proposed for designing the attitude angle (the roll angle ϕ , the pitch angle θ , and the yaw angle ψ ), height ( z ), and position ( x and y ) controllers of a general multirotor UAV model. Global convergence and strong robustness of the proposed AMND-based method and controllers are analyzed and proved theoretically. By incorporating the adaptive control method into the general multilayer neural dynamic-based controllers design method, multirotor UAVs with unknown disturbances can complete time-varying trajectory tracking tasks. AMND-based controllers with the self-tuning rates can estimate the unknown disturbances and solve the model uncertainty problems. Both the theoretical theorems and simulation results illustrate that the proposed design method and its controllers with strong anti-interference property can achieve the time-varying trajectory tracking control stably, reliably, and effectively. Moreover, a practical experiment by using a mini multirotor UAV illustrates the practicability of the AMND-based method.

Robust Neurooptimal Control for a Robot via Adaptive Dynamic Programming

We aim at the optimization of the tracking control of a robot to improve the robustness, under the effect of unknown nonlinear perturbations. First, an auxiliary system is introduced, and optimal control of the auxiliary system can be seen as an approximate optimal control of the robot. Then, neural networks (NNs) are employed to approximate the solution of the Hamilton-Jacobi-Isaacs equation under the frame of adaptive dynamic programming. Next, based on the standard gradient attenuation algorithm and adaptive critic design, NNs are trained depending on the designed updating law with relaxing the requirement of initial stabilizing control. In light of the Lyapunov stability theory, all the error signals can be proved to be uniformly ultimately bounded. A series of simulation studies are carried out to show the effectiveness of the proposed control.

Single Neural Adaptive PID Control for Small UAV Micro-Turbojet Engine

The micro-turbojet engine (MTE) is especially suitable for unmanned aerial vehicles (UAVs). Because the rotor speed is proportional to the thrust force, the accurate speed tracking control is indispensable for MTE. Thanks to its simplicity, the proportional–integral–derivative (PID) controller is commonly used for rotor speed regulation. However, the PID controller cannot guarantee superior performance over the entire operation range due to the time-variance and strong nonlinearity of MTE. The gain scheduling approach using a family of linear controllers is recognized as an efficient alternative, but such a solution heavily relies on the model sets and pre-knowledge. To tackle such challenges, a single neural adaptive PID (SNA-PID) controller is proposed herein for rotor speed control. The new controller featuring with a single-neuron network is able to adaptively tune the gains (weights) online. The simple structure of the controller reduces the computational load and facilitates the algorithm implementation on low-cost hardware. Finally, the proposed controller is validated by numerical simulations and experiments on the MTE in laboratory conditions, and the results show that the proposed controller achieves remarkable effectiveness for speed tracking control. In comparison with the PID controller, the proposed controller yields 54% and 66% reductions on static tracking error under two typical cases.

Event-Triggered Optimal Control With Performance Guarantees Using Adaptive Dynamic Programming

This paper studies the problem of event-triggered optimal control (ETOC) for continuous-time nonlinear systems and proposes a novel event-triggering condition that enables designing ETOC methods directly based on the solution of the Hamilton-Jacobi-Bellman (HJB) equation. We provide formal performance guarantees by proving a predetermined upper bound. Moreover, we also prove the existence of a lower bound for interexecution time. For implementation purposes, an adaptive dynamic programming (ADP) method is developed to realize the ETOC using a critic neural network (NN) to approximate the value function of the HJB equation. Subsequently, we prove that semiglobal uniform ultimate boundedness can be guaranteed for states and NN weight errors with the ADP-based ETOC. Simulation results demonstrate the effectiveness of the developed ADP-based ETOC method.

Adaptive critic designs for optimal control of uncertain nonlinear systems with unmatched interconnections

In this paper, we develop a novel optimal control strategy for a class of uncertain nonlinear systems with unmatched interconnections. To begin with, we present a stabilizing feedback controller for the interconnected nonlinear systems by modifying an array of optimal control laws of auxiliary subsystems. We also prove that this feedback controller ensures a specified cost function to achieve optimality. Then, under the framework of adaptive critic designs, we use critic networks to solve the Hamilton-Jacobi-Bellman equations associated with auxiliary subsystem optimal control laws. The critic network weights are tuned through the gradient descent method combined with an additional stabilizing term. By using the newly established weight tuning rules, we no longer need the initial admissible control condition. In addition, we demonstrate that all signals in the closed-loop auxiliary subsystems are stable in the sense of uniform ultimate boundedness by using classic Lyapunov techniques. Finally, we provide an interconnected nonlinear plant to validate the present control scheme.

Pattern-Based NN Control of a Class of Uncertain Nonlinear Systems

This paper presents a pattern-based neural network (NN) control approach for a class of uncertain nonlinear systems. The approach consists of two phases of identification and another two phases of recognition and control. First, in the phase (i) of identification, adaptive NN controllers are designed to achieve closed-loop stability and tracking performance of nonlinear systems for different control situations, and the corresponding closed-loop control system dynamics are identified via deterministic learning. The identified control system dynamics are stored in constant radial basis function (RBF) NNs, and a set of constant NN controllers are constructed by using the obtained constant RBF networks. Second, in the phase (ii) of identification, when the plant is operated under different or abnormal conditions, the system dynamics under normal control are identified via deterministic learning. A bank of dynamical estimators is constructed for all the abnormal conditions and the learned knowledge is embedded in the estimators. Third, in the phase of recognition, when one identified control situation recurs, by using the constructed estimators, the recurred control situation will be rapidly recognized. Finally, in the phase of pattern-based control, based on the rapid recognition, the constant NN controller corresponding to the current control situation is selected, and both closed-loop stability and improved control performance can be achieved. The results presented show that the pattern-based control realizes a humanlike control process, and will provide a new framework for fast decision and control in dynamic environments. A simulation example is included to demonstrate the effectiveness of the approach.

Adaptive Critic Nonlinear Robust Control: A Survey

Adaptive dynamic programming (ADP) and reinforcement learning are quite relevant to each other when performing intelligent optimization. They are both regarded as promising methods involving important components of evaluation and improvement, at the background of information technology, such as artificial intelligence, big data, and deep learning. Although great progresses have been achieved and surveyed when addressing nonlinear optimal control problems, the research on robustness of ADP-based control strategies under uncertain environment has not been fully summarized. Hence, this survey reviews the recent main results of adaptive-critic-based robust control design of continuous-time nonlinear systems. The ADP-based nonlinear optimal regulation is reviewed, followed by robust stabilization of nonlinear systems with matched uncertainties, guaranteed cost control design of unmatched plants, and decentralized stabilization of interconnected systems. Additionally, further comprehensive discussions are presented, including event-based robust control design, improvement of the critic learning rule, nonlinear H_∞ control design, and several notes on future perspectives. By applying the ADP-based optimal and robust control methods to a practical power system and an overhead crane plant, two typical examples are provided to verify the effectiveness of theoretical results. Overall, this survey is beneficial to promote the development of adaptive critic control methods with robustness guarantee and the construction of higher level intelligent systems.

Adaptive fuzzy tracking control for non-affine nonlinear yaw channel of unmanned aerial vehicle helicopter

The article deals with the problem of the yaw control of the unmanned aerial vehicle helicopter which is non-affine nonlinear by use of a novel projection-based adaptive fuzzy control approach. First, principle model and control design model of the yaw channel of the unmanned aerial vehicle helicopter are described. Then, a dynamic approximation technique is introduced to approach the non-affine model of the unmanned aerial vehicle helicopter into an affine model with variable parameters, which is applied to facilitate the design of nonlinear control scheme. Next, in the proposed controller, fuzzy controller is designed to deal with the unknown uncertainties and disturbances. Meanwhile, the projection-based adaptive law applied in fuzzy controller bounds the parameter estimation function and can also guarantee the robustness of the designed control scheme against uncertain disturbances. Moreover, the convergence and stability of the designed controller are proved by Lyapunov stability theory. Finally, the simulation results of the yaw channel of an unmanned aerial vehicle helicopter are performed to illustrate that the designed controller has good tracking performance, stability, and robustness under the condition of the time-vary uncertain disturbances.

Adaptive Position/Attitude Tracking Control of Aerial Robot With Unknown Inertial Matrix Based on a New Robust Neural Identifier

This paper presents a novel adaptive controller for controlling an autonomous helicopter with unknown inertial matrix to asymptotically track the desired trajectory. To identify the unknown inertial matrix included in the attitude dynamic model, this paper proposes a new structural identifier that differs from those previously proposed in that it additionally contains a neural networks (NNs) mechanism and a robust adaptive mechanism, respectively. Using the NNs to compensate the unknown aerodynamic forces online and the robust adaptive mechanism to cancel the combination of the overlarge NNs compensation error and the external disturbances, the new robust neural identifier exhibits a better identification performance in the complex flight environment. Moreover, an optimized algorithm is included in the NNs mechanism to alleviate the burdensome online computation. By the strict Lyapunov argument, the asymptotic convergence of the inertial matrix identification error, position tracking error, and attitude tracking error to arbitrarily small neighborhood of the origin is proved. The simulation and implementation results are provided to evaluate the performance of the proposed controller.

Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints

The design of stabilizing controller for uncertain nonlinear systems with control constraints is a challenging problem. The constrained-input coupled with the inability to identify accurately the uncertainties motivates the design of stabilizing controller based on reinforcement-learning (RL) methods. In this paper, a novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints. The robust control problem is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system. Distinct from typical action-critic dual networks employed in RL, only one critic neural network (NN) is constructed to derive the approximate optimal control. Meanwhile, unlike initial stabilizing control often indispensable in RL, there is no special requirement imposed on the initial control. By utilizing Lyapunov's direct method, the closed-loop optimal control system and the estimated weights of the critic NN are proved to be uniformly ultimately bounded. In addition, the derived approximate optimal control is verified to guarantee the uncertain nonlinear system to be stable in the sense of uniform ultimate boundedness. Two simulation examples are provided to illustrate the effectiveness and applicability of the present approach.

Adaptive neural control of nonlinear MIMO systems with time-varying output constraints

In this paper, adaptive neural control is investigated for a class of unknown multiple-input multiple-output nonlinear systems with time-varying asymmetric output constraints. To ensure constraint satisfaction, we employ a system transformation technique to transform the original constrained (in the sense of the output restrictions) system into an equivalent unconstrained one, whose stability is sufficient to solve the output constraint problem. It is shown that output tracking is achieved without violation of the output constraint. More specifically, we can shape the system performance arbitrarily on transient and steady-state stages with the output evolving in predefined time-varying boundaries all the time. A single neural network, whose weights are tuned online, is used in our design to approximate the unknown functions in the system dynamics, while the singularity problem of the control coefficient matrix is avoided without assumption on the prior knowledge of control input's bound. All the signals in the closed-loop system are proved to be semiglobally uniformly ultimately bounded via Lyapunov synthesis. Finally, the merits of the proposed controller are verified in the simulation environment.

Neural-network-based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems

In this paper, the infinite horizon optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems is investigated using neural-network-based online solution of Hamilton-Jacobi-Bellman (HJB) equation. By establishing an appropriate bounded function and defining a modified cost function, the optimal robust guaranteed cost control problem is transformed into an optimal control problem. It can be observed that the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to facilitate the solution of the modified HJB equation corresponding to the nominal system. More importantly, an additional stabilizing term is introduced for helping to verify the stability, which reinforces the updating process of the weight vector and reduces the requirement of an initial stabilizing control. The uniform ultimate boundedness of the closed-loop system is analyzed by using the Lyapunov approach as well. Two simulation examples are provided to verify the effectiveness of the present control approach.

H∞ output tracking control of discrete-time nonlinear systems via standard neural network models

This brief proposes an output tracking control for a class of discrete-time nonlinear systems with disturbances. A standard neural network model is used to represent discrete-time nonlinear systems whose nonlinearity satisfies the sector conditions. H∞ control performance for the closed-loop system including the standard neural network model, the reference model, and state feedback controller is analyzed using Lyapunov-Krasovskii stability theorem and linear matrix inequality (LMI) approach. The H∞ controller, of which the parameters are obtained by solving LMIs, guarantees that the output of the closed-loop system closely tracks the output of a given reference model well, and reduces the influence of disturbances on the tracking error. Three numerical examples are provided to show the effectiveness of the proposed H∞ output tracking design approach.

Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems

This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming (ADP) method for solving the infinite horizon optimal control problem of nonlinear systems. The idea is to use an iterative ADP technique to obtain the iterative control law, which optimizes the iterative performance index function. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time. It shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation. It is also proven that any of the iterative control laws can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control law, respectively, for facilitating the implementation of the iterative ADP algorithm, where the convergence of the weight matrices is analyzed. Finally, the numerical results and analysis are presented to illustrate the performance of the developed method.