NN Reinforcement Learning Adaptive Control for a Class of Nonstrict-Feedback Discrete-Time Systems

Adaptive Neural Zeta-Backstepping With Predefined Damping Ratio. Application to DC Motors

This brief presents an adaptive neural zeta-backstepping control strategy for a class of uncertain nonlinear systems, which allows these systems to be practically stabilized with predefined damping ratios. By introducing the zeta-backstepping technique, system damping ratios can be predetermined based on specific parameter selection rules. To reduce the impact of unknown nonlinearities, neural networks (NNs) with gradient descent training are applied to compensate such nonlinearities online. A new filter, called dynamic command filter, is used to construct the gradient of the NNs. By resorting to second-order Lyapunov stability criteria, it is proved that the closed-loop system is practically stable and has predefined damping ratio. Finally, experiments on a perturbed direct current (DC) motor system demonstrate the advantages of the proposed method.

Event-based adaptive fixed-time optimal control for saturated fault-tolerant nonlinear multiagent systems via reinforcement learning algorithm

This article investigates the problem of adaptive fixed-time optimal consensus tracking control for nonlinear multiagent systems (MASs) affected by actuator faults and input saturation. To achieve optimal control, reinforcement learning (RL) algorithm which is implemented based on neural network (NN) is employed. Under the actor-critic structure, an innovative simple positive definite function is constructed to obtain the upper bound of the estimation error of the actor-critic NN updating law, which is crucial for analyzing fixed-time stabilization. Furthermore, auxiliary functions and estimation laws are designed to eliminate the coupling effects resulting from actuator faults and input saturation. Meanwhile, a novel event-triggered mechanism (ETM) that incorporates the consensus tracking errors into the threshold is proposed, thereby effectively conserving communication resources. Based on this, a fixed-time event-triggered control scheme grounded in RL is proposed through the integration of the backstepping technique and fixed-time theory. It is demonstrated that the consensus tracking errors converge to a specified range in a fixed time and all signals within the closed-loop systems are bounded. Finally, simulation results are provided to verify the effectiveness of the proposed control strategy.

Fuzzy reinforcement learning based control of linear systems with input saturation

This research introduces an innovative approach to optimal control for a class of linear systems with input saturation. It leverages the synergy of Takagi-Sugeno (T-S) fuzzy models and reinforcement learning (RL) techniques. To enhance interpretability and analytical accessibility, our approach applies T-S models to approximate the value function and generate optimal control laws while incorporating prior knowledge. By addressing the challenge of limited interpretability associated with conventional neural network utilization in RL, our approach utilizes segmented functions for saturation derivative characteristics approximation, effectively handling non-differentiability issues at saturation boundaries. Furthermore, our research presents a novel gradient identification method to overcome the impractical reliance on next-time-step State variables in RL for current-time-step policy improvements. This enables the derivation of optimal control laws corresponding to each fuzzy rule, ensuring practical applicability in the control field. The proposed methodology is rigorously evaluated through computer simulations, confirming its effectiveness, optimality, and convergence properties. This research contributes valuable insights and practical solutions to input-saturation control systems, offering a versatile and robust framework for real-world applications.

Highly valued subgoal generation for efficient goal-conditioned reinforcement learning

Goal-conditioned reinforcement learning is widely used in robot control, manipulating the robot to accomplish specific tasks by maximizing accumulated rewards. However, the useful reward signal is only received when the desired goal is reached, leading to the issue of sparse rewards and affecting the efficiency of policy learning. In this paper, we propose a method to generate highly valued subgoals for efficient goal-conditioned policy learning, enabling the development of smart home robots or automatic pilots in our daily life. The highly valued subgoals are conditioned on the context of the specific tasks and characterized by suitable complexity for efficient goal-conditioned action value learning. The context variable captures the latent representation of the particular tasks, allowing for efficient subgoal generation. Additionally, the goal-conditioned action values regularized by the self-adaptive ranges generate subgoals with suitable complexity. Compared to Hindsight Experience Replay that uniformly samples subgoals from visited trajectories, our method generates the subgoals based on the context of tasks with suitable difficulty for efficient policy training. Experimental results show that our method achieves stable performance in robotic environments compared to baseline methods.

Reinforcement learning-based optimal tracking control for uncertain multi-agent systems with uncertain topological networks

Recent decades, extensive applications exemplified in intelligent connected vehicles (ICVs) and unmanned aerial vehicles (UAVs) have emerged with the rapidly development of multi-agent systems (MASs). Inspired by these applications, the optimal tracking control problem for uncertain MASs under uncertain topological networks is addressed based on the theory of observer design and reinforcement learning (RL). Thus, an adaptive extended observer based on concurrent learning (CL) technique is designed to simultaneously estimate system states and unknown parameters, where unknown parameters estimated convergence is guaranteed in a relaxed persistence of excitation condition. Moreover, a Luenberger observer is designed to estimate the state of the leader under uncertain topological networks, which acts as the information compensation of the leader. Via the proposed observers, an optimal tracking control algorithm is devised leveraging actor-critic (AC)-neural network (NN), which does not require the state derivative information. Lastly, a numerical simulation is performed to demonstrate the validity of the scheme in question.

Self-Powered Wearable Displacement Sensor for Continuous Respiratory Monitoring and Human-Machine Synchronous Control

Flexible wearable electronic devices play a vital role in daily monitoring, medical diagnosis, and human-computer interaction, and such devices have a great demand for portability, integration, comfort, and self-power. In this study, a triboelectric nanogenerator integrated into a flexible chest belt is proposed as a displacement sensor to monitor the displacement and frequency of thoracic expansion. Based on three parallel interpolation electrode structures with phase differences, the Triboelectric Nanogenerators's(TENG) output signal pulse number can characterize the sliding displacement, with a resolution of more than 1 mm and a durability of more than 700,000 cycles. Based on the flexible printed circuit processing technology, the volume of the sensor is less than 8.5 cm3, and the weight is less than 3.2 g, which improves the portability of the device. Based on wireless radio frequency technology, the collected signals are transmitted to the upper computer, and then the monitoring of respiratory physiological signals and the human-machine synchronous control of the ventilator are achieved within the overshoot of 1.5% and the control error of 5% through a simulation machine. This work provides a sensing method for daily and medical respiratory monitoring and demonstrates the enormous potential of frictional electric sensors in intelligent medical applications.

Event-Triggered Adaptive Containment Control for Heterogeneous Stochastic Nonlinear Multiagent Systems

This article investigates the event-triggered adaptive containment control problem for a class of stochastic nonlinear multiagent systems with unmeasurable states. A stochastic system with unknown heterogeneous dynamics is established to describe the agents in a random vibration environment. Besides, the uncertain nonlinear dynamics are approximated by radial basis function neural networks (NNs), and the unmeasured states are estimated by constructing the NN-based observer. In addition, the switching-threshold-based event-triggered control method is adopted with the hope of reducing communication consumption and balancing system performance and network constraints. Moreover, we develop the novel distributed containment controller by utilizing the adaptive backstepping control strategy and the dynamic surface control (DSC) approach such that the output of each follower converges to the convex hull spanned by multiple leaders, and all signals of the closed-loop system are cooperatively semi-globally uniformly ultimately bounded in mean square. Finally, we verify the efficiency of the proposed controller by the simulation examples.

Time-varying gain extended state observer-based adaptive optimal control for disturbed unmanned helicopter

In this paper, the robust adaptive optimal tracking control problem is addressed for the disturbed unmanned helicopter based on the time-varying gain extended state observer (TVGESO) and adaptive dynamic programming (ADP) methods. Firstly, a novel TVGESO is developed to tackle the unknown disturbance, which can overcome the drawback of initial peaking phenomenon in the traditional linear ESO method. Meanwhile, compared with the nonlinear ESO, the proposed TVGESO possesses easier and rigorous stability analysis process. Subsequently, the optimal tracking control issue for the original unmanned helicopter system is transformed into an optimization stabilization problem. By means of the ADP and neural network techniques, the feedforward controller and optimal feedback controller are skillfully designed. Compared with the conventional backstepping approach, the designed anti-disturbance optimal controller can make the unmanned helicopter accomplish the tracking task with less energy. Finally, simulation comparisons demonstrate the validity of the developed control scheme.

Optimized Adaptive Fuzzy Security Control of Nonlinear Systems With Prescribed Tracking Performance

This article studies the optimized fuzzy prescribed performance control problem for nonlinear nonstrict-feedback systems under denial-of-service (DoS) attacks. A fuzzy estimator is delicately designed to model the immeasurable system states in the presence of DoS attacks. To achieve the preset tracking performance, a simper prescribed performance error transformation is constructed considering the characteristics of DoS attacks, which helps obtain a novel Hamilton-Jacobi-Bellman equation to derive the optimized prescribed performance controller. Furthermore, the fuzzy-logic system, combined with the reinforcement learning (RL) technique, is employed to approximate the unknown nonlinearity existing in the prescribed performance controller design process. An optimized adaptive fuzzy security control law is then proposed for the considered nonlinear nonstrict-feedback systems subject to DoS attacks. Through the Lyapunov stability analysis, the tracking error is proved to approach the predefined region by the preset finite time, even in the presence of DoS attacks. Meanwhile, the consumed control resources are minimized due to the RL-based optimized algorithm. Finally, an actual example with comparisons verifies the effectiveness of the proposed control algorithm.

Efficient Online Globalized Dual Heuristic Programming With an Associated Dual Network

Globalized dual heuristic programming (GDHP) is the most comprehensive adaptive critic design, which employs its critic to minimize the error with respect to both the cost-to-go and its derivatives simultaneously. Its implementation, however, confronts a dilemma of either introducing more computational load by explicitly calculating the second partial derivative term or sacrificing the accuracy by loosening the association between the cost-to-go and its derivatives. This article aims at increasing the online learning efficiency of GDHP while retaining its analytical accuracy by introducing a novel GDHP design based on a critic network and an associated dual network. This associated dual network is derived from the critic network explicitly and precisely, and its structure is in the same level of complexity as dual heuristic programming critics. Three simulation experiments are conducted to validate the learning ability, efficiency, and feasibility of the proposed GDHP critic design.

Reinforcement Learning-Based Model-Free Controller for Feedback Stabilization of Robotic Systems

This article presents a reinforcement learning (RL) algorithm for achieving model-free control of robotic applications. The RL functions are adapted with the least-square temporal difference (LSTD) learning algorithms to develop a model-free state feedback controller by establishing linear quadratic regulator (LQR) as a baseline controller. The classical least-square policy iteration technique is adapted to establish the boundary conditions for complexities incurred by the learning algorithm. Furthermore, the use of exact and approximate policy iterations estimates the parameters of the learning functions for a feedback policy. To assess the operation of the proposed controller, the trajectory tracking and balancing control problems of unmanned helicopters and balancer robotic applications are solved for real-time experiment. The results showed the robustness of the proposed approach in achieving trajectory tracking and balancing control.

Event-triggered adaptive optimal tracking control for nonlinear stochastic systems with dynamic state constraints

This paper investigates the issue of event-triggered adaptive optimal tracking control for uncertain nonlinear systems with stochastic disturbances and dynamic state constraints. To handle the dynamic state constraints, a novel unified tangent-type nonlinear mapping function is proposed. A neural networks (NNs)-based identifier is designed to cope with the stochastic disturbances. By utilizing adaptive dynamic programming (ADP) of identifier-actor-critic architecture and event triggering mechanism, the adaptive optimized event-triggered control (ETC) approach for the nonlinear stochastic system is first proposed. It is proven that the designed optimized ETC approach guarantees the robustness of the stochastic systems and the semi-globally uniformly ultimately bounded in the mean square of the NNs adaptive estimation error, and the Zeno behavior can be avoided. Simulations are offered to illustrate the effectiveness of the proposed control approach.

Dynamic event-triggered controller design for nonlinear systems: Reinforcement learning strategy

The current investigation aims at the optimal control problem for discrete-time nonstrict-feedback nonlinear systems by invoking the reinforcement learning-based backstepping technique and neural networks. The dynamic-event-triggered control strategy introduced in this paper can alleviate the communication frequency between the actuator and controller. Based on the reinforcement learning strategy, actor-critic neural networks are employed to implement the n-order backstepping framework. Then, a neural network weight-updated algorithm is developed to minimize the computational burden and avoid the local optimal problem. Furthermore, a novel dynamic-event-triggered strategy is introduced, which can remarkably outperform the previously studied static-event-triggered strategy. Moreover, combined with the Lyapunov stability theory, all signals in the closed-loop system are strictly proven to be semiglobal uniformly ultimately bounded. Finally, the practicality of the offered control algorithms is further elucidated by the numerical simulation examples.

Fault-tolerant control for nonlinear systems with a dead zone: Reinforcement learning approach

This paper focuses on the adaptive reinforcement learning-based optimal control problem for standard nonstrict-feedback nonlinear systems with the actuator fault and an unknown dead zone. To simultaneously reduce the computational complexity and eliminate the local optimal problem, a novel neural network weight updated algorithm is presented to replace the classic gradient descent method. By utilizing the backstepping technique, the actor critic-based reinforcement learning control strategy is developed for high-order nonlinear nonstrict-feedback systems. In addition, two auxiliary parameters are presented to deal with the input dead zone and actuator fault respectively. All signals in the system are proven to be semi-globally uniformly ultimately bounded by Lyapunov theory analysis. At the end of the paper, some simulation results are shown to illustrate the remarkable effect of the proposed approach.

Adaptive Multigradient Recursive Reinforcement Learning Event-Triggered Tracking Control for Multiagent Systems

This article proposes a fault-tolerant adaptive multigradient recursive reinforcement learning (RL) event-triggered tracking control scheme for strict-feedback discrete-time multiagent systems. The multigradient recursive RL algorithm is used to avoid the local optimal problem that may exist in the gradient descent scheme. Different from the existing event-triggered control results, a new lemma about the relative threshold event-triggered control strategy is proposed to handle the compensation error, which can improve the utilization of communication resources and weaken the negative impact on tracking accuracy and closed-loop system stability. To overcome the difficulty caused by sensor fault, a distributed control method is introduced by adopting the adaptive compensation technique, which can effectively decrease the number of online estimation parameters. Furthermore, by using the multigradient recursive RL algorithm with less learning parameters, the online estimation time can be effectively reduced. The stability of closed-loop multiagent systems is proved by using the Lyapunov stability theorem, and it is verified that all signals are semiglobally uniformly ultimately bounded. Finally, two simulation examples are given to show the availability of the presented control scheme.

Event-Triggered Multigradient Recursive Reinforcement Learning Tracking Control for Multiagent Systems

In this article, the tracking control problem of event-triggered multigradient recursive reinforcement learning is investigated for nonlinear multiagent systems (MASs). Attention is focused on the distributed reinforcement learning approach for MASs. The critic neural network (NN) is applied to estimate the long-term strategic utility function, and the actor NN is designed to approximate the uncertain dynamics in MASs. The multigradient recursive (MGR) strategy is tailored to learn the weight vector in NN, which eliminates the local optimal problem inherent in gradient descent method and decreases the dependence of initial value. Furthermore, reinforcement learning and event-triggered mechanism can improve the energy conservation of MASs by decreasing the amplitude of the controller signal and the controller update frequency, respectively. It is proved that all signals in MASs are semiglobal uniformly ultimately bounded (SGUUB) according to the Lyapunov theory. Simulation results are given to demonstrate the effectiveness of the proposed strategy.

Observer-Based Adaptive Optimized Control for Stochastic Nonlinear Systems With Input and State Constraints

In this work, an adaptive neural network (NN) optimized output-feedback control problem is studied for a class of stochastic nonlinear systems with unknown nonlinear dynamics, input saturation, and state constraints. A nonlinear state observer is designed to estimate the unmeasured states, and the NNs are used to approximate the unknown nonlinear functions. Under the framework of the backstepping technique, the virtual and actual optimal controllers are developed by employing the actor-critic architecture. Meanwhile, the tan-type Barrier optimal performance index functions are developed to prevent the nonlinear systems from the state constraints, and all the states are confined within the preselected compact sets all the time. It is worth mentioning that the proposed optimized control is clearly simple since the reinforcement learning (RL) algorithm is derived based on the negative gradient of a simple positive function. Furthermore, the proposed optimal control strategy ensures that all the signals in the closed-loop system are bounded. Finally, a practical simulation example is carried out to further illustrate the effectiveness of the proposed optimal control method.

Disturbance observer-based adaptive reinforcement learning for perturbed uncertain surface vessels

This article considers a problem of tracking, convergence of disturbance observer (DO) based optimal control design for uncertain surface vessels (SVs) with external disturbance. The advantage of proposed optimal control using adaptive/approximate reinforcement learning (ARL) is that consideration for whole SVs with only one dynamic equation and without conventional separation technique. Additionally, thanks to appropriate disturbance observer, the attraction region of tracking error is remarkably reduced. On the other hand, the particular case of optimal control problem is presented by directly solving for the purpose of choosing the suitable activation functions of ARL. Furthermore, the proposed ARL based optimal control also deals with non-autonomous property of closed tracking error SV model by considering the equivalent system. Based on the Lyapunov function candidate using optimal function and quadratic form of estimated error of actor/critic weight, the stability and convergence of the closed system are proven. Some examples are given to verify and demonstrate the effectiveness of the new control strategy.

Reinforcement Learning-Based Cooperative Optimal Output Regulation via Distributed Adaptive Internal Model

In this article, a data-driven distributed control method is proposed to solve the cooperative optimal output regulation problem of leader-follower multiagent systems. Different from traditional studies on cooperative output regulation, a distributed adaptive internal model is originally developed, which includes a distributed internal model and a distributed observer to estimate the leader's dynamics. Without relying on the dynamics of multiagent systems, we have proposed two reinforcement learning algorithms, policy iteration and value iteration, to learn the optimal controller through online input and state data, and estimated values of the leader's state. By combining these methods, we have established a basis for connecting data-distributed control methods with adaptive dynamic programming approaches in general since these are the theoretical foundation from which they are built.

Adaptive Observation-Based Efficient Reinforcement Learning for Uncertain Systems

This article develops an adaptive observation-based efficient reinforcement learning (RL) approach for systems with uncertain drift dynamics. A novel concurrent learning adaptive extended observer (CL-AEO) is first designed to jointly estimate the system state and parameter. This observer has a two-time-scale structure and does not require any additional numerical techniques to calculate the state derivative information. The idea of concurrent learning (CL) is leveraged to use the recorded data, which leads to a relaxed verifiable excitation condition for the convergence of parameter estimation. Based on the estimated state and parameter provided by the CL-AEO, a simulation of experience-based RL scheme is developed to online approximate the optimal control policy. Rigorous theoretical analysis is given to show that the practical convergence of the system state to the origin and the developed policy to the ideal optimal policy can be achieved without the persistence of excitation (PE) condition. Finally, the effectiveness and superiority of the developed methodology are demonstrated via comparative simulations.

Adaptive Optimized Backstepping Control-Based RL Algorithm for Stochastic Nonlinear Systems With State Constraints and Its Application

This article investigates the adaptive neural-network (NN) tracking optimal control problem for stochastic nonlinear systems, which contain state constraints and uncertain dynamics. First, to avoid the violation of state constraints in achieving optimal control, the novel barrier optimal performance index functions for subsystems are developed. Second, under the framework of the identifier-actor-critic, the virtual and actual optimal controllers are presented based on the backstepping technique, in which the unknown nonlinear dynamics are learned by the NN approximators. Moreover, the quartic barrier Lyapunov functions are constructed instead of square ones to cope with the Hessian term to ensure the stability of the systems with stochastic disturbance. The proposed optimal control strategy can guarantee the boundedness of closed-loop signals, and the output can follow the given reference signal. Meanwhile, the system states are restricted within some preselected compact sets all the while. Finally, both numerical and practical systems are carried out to further illustrate the validity of the proposed optimal control approach.

Backstepping Control for a Class of Nonlinear Discrete-Time Systems Subject to Multisource Disturbances and Actuator Saturation

In this article, the backstepping control scheme is designed for a class of systems with multisource disturbances, actuator saturation, and nonlinearities in the domain of discrete time. To address the multisource disturbances, we put forward a novel discrete-time hybrid observer, which can deal with both modeled and unmodeled disturbances. In virtue of the radial basis function neural networks, the unknown nonlinearities are approximated. In addition, the anti-windup technique is adopted to cope with the actuator saturation phenomenon, which is pervasive in engineering practice. Bearing all the adopted mechanisms in mind, the composite control strategy is designed in a backstepping manner. Sufficient conditions are established to guarantee that the states of the system ultimately converge to a small range with linear matrix inequalities. Finally, the effectiveness of the presented methodology is verified for the spacecraft attitude system.

Rotating consensus for double-integrator multi-agent systems with communication delay

In this paper, the rotating consensus problem for the multi-agent systems of double-integrator dynamic is mainly considered with and without communication delay. A fully distributed control strategy is given on the more general complex plane. For the case without communication delay, we design a distributed control protocol with the help of local relative information and obtain the sufficient and necessary condition with the lower bounded control parameter for the directed communication topology. Furthermore, with taking communication delay into consideration, the sufficient and necessary condition with the upper bounded communication delay is obtained with the help of frequency domain analysis method. Compared with some existing results, the communication topology considered is a more general directed graph. Finally, some simulation examples are presented to clarify the correctness of our results.

Optimal fractional-order PID controller based on fractional-order actor-critic algorithm

In this paper, an online optimization approach of a fractional-order PID controller based on a fractional-order actor-critic algorithm (FOPID-FOAC) is proposed. The proposed FOPID-FOAC scheme exploits the advantages of the FOPID controller and FOAC approaches to improve the performance of nonlinear systems. The proposed FOAC is built by developing a FO-based learning approach for the actor-critic neural network with adaptive learning rates. Moreover, a FO rectified linear unit (RLU) is introduced to enable the AC neural network to define and optimize its own activation function. By the means of the Lyapunov theorem, the convergence and the stability analysis of the proposed algorithm are investigated. The FO operators for the FOAC learning algorithm are obtained using the gray wolf optimization (GWO) algorithm. The effectiveness of the proposed approach is proven by extensive simulations based on the tracking problem of the two degrees of freedom (2-DOF) helicopter system and the stabilization issue of the inverted pendulum (IP) system. Moreover, the performance of the proposed algorithm is compared against optimized FOPID control approaches in different system conditions, namely when the system is subjected to parameter uncertainties and external disturbances. The performance comparison is conducted in terms of two types of performance indices, the error performance indices, and the time response performance indices. The first one includes the integral absolute error (IAE), and the integral squared error (ISE), whereas the second type involves the rising time, the maximum overshoot (Max. OS), and the settling time. The simulation results explicitly indicate the high effectiveness of the proposed FOPID-FOAC controller in terms of the two types of performance measurements under different scenarios compared with the other control algorithms.

Reduced-Order Filters-Based Adaptive Backstepping Control for Perturbed Nonlinear Systems

In this article, a robust adaptive output-feedback control approach is presented for a class of nonlinear output-feedback systems with parameter uncertainties and time-varying bounded disturbances. A reduced-order filter driven by control input is proposed to reconstruct unmeasured states. The state estimation error is shown to be bounded by dynamic signals driven by system output. The bound estimation technique is employed to estimate the unknown disturbance bound. Based on the backstepping design with three sets of tuning functions, an adaptive output-feedback control scheme with the flat-zone modification is proposed. It is shown that all the signals in the resulting closed-loop adaptive control systems are bounded, and the output tracking error converges to a prespecified small neighborhood of the origin. Two simulation examples are provided to illustrate the effectiveness and validity of the proposed approach.

Distributed Cooperative Compound Tracking Control for a Platoon of Vehicles With Adaptive NN

This article focuses on the distributed cooperative compound tracking issue of the vehicular platoon. First, a definition, called compound tracking control, is proposed, which means that the practical finite-time stability and asymptotical convergence can be simultaneously satisfied. Then, a modified performance function, named finite-time performance function, is designed, which possesses the faster convergence rate compared to the existing ones. Moreover, the adaptive neural network (NN), prescribed performance technique, and backstepping method are utilized to design a distributed cooperative regulation protocol. It is worth noting that the convergence time of the proposed algorithm does not depend on the initial values and design parameters. Finally, simulation experiments are given to further verify the effectiveness of the presented theoretical findings.

Machine-Made Synthesis of Stabilization System by Modified Cartesian Genetic Programming

A numerical solution of the problem of the general synthesis of a stabilization system by a symbolic regression method is considered. The goal is to automatically find a feedback control function using a computer so that the control object can reach a given terminal position from anywhere in a given region of the initial conditions with an optimal value of the quality criterion. Usually, the control synthesis problem is solved analytically or technically taking into account the specific properties of the mathematical model. We suppose that modern numerical approaches of symbolic regression can be applied to find a solution without reference to specific model equations. It is proposed to use the numerical method of Cartesian genetic programming (CGP). It was developed for automatic writing of programs but has never been used to solve the synthesis problem. In the present work, the method was modified with the principle of small variations in order to reduce the search area and increase the rate of convergence. To apply the general principle of small variations to CGP, we developed special types of variations and coding. The modified CGP searches for the mathematical expression of the feedback control function in the form of a code and, at the same time, the optimal value of the parametric vector which is also a new feature-simultaneous tuning of the parameters inside the search process. This approach enables working with objects and functions of any type, which is not always possible with analytical methods. The need to use the received solution on the onboard processor of the control object imposes certain restrictions on the used basic set of elementary functions. This article proposes the theoretical foundations of the study of these functions, and the concept of the space of machine-made functions is introduced. The capabilities of the approach are demonstrated on the numerical solution of the control system synthesis problems for a mobile robot and a Duffing model.

Anti-Saturation-Based Adaptive Sliding-Mode Control for Active Suspension Systems With Time-Varying Vertical Displacement and Speed Constraints

In this article, an adaptive sliding-mode control scheme is developed for a class of uncertain quarter vehicle active suspension systems with time-varying vertical displacement and speed constraints, in which the input saturation is considered. The integral terminal SMC is adopted to improve convergence accuracy and avoid singular problems. In addition, neural networks are used to model unknown terms in the system and the backstepping technique is taken into account to design the actual controller. To guarantee that the time-varying state constraints are not violated, the corresponding Barrier Lyapunov functions are constructed. At the same time, a continuous differentiable asymmetric saturation model is developed to improve the stability of the system. Then, the Lyapunov stability theory is used to verify that all signals of the resulting system are semi globally uniformly ultimately bounded, time-varying state constraints are not violated, and error variables can converge to the small neighborhood of 0. Finally, results of the simulation of the designed control strategy are given to further prove the effectiveness.

Game-Based Backstepping Design for Strict-Feedback Nonlinear Multi-Agent Systems Based on Reinforcement Learning

In this article, the game-based backstepping control method is proposed for the high-order nonlinear multi-agent system with unknown dynamic and input saturation. Reinforcement learning (RL) is employed to get the saddle point solution of the tracking game between each agent and the reference signal for achieving robust control. Specifically, the approximate optimal solution of the established Hamilton-Jacobi-Isaacs (HJI) equation is obtained by policy iteration for each subsystem, and the single network adaptive critic (SNAC) architecture is used to reduce the computational burden. In addition, based on the separation operation of the error term from the derivative of the value function, we achieve the different proportions of the two agents in the game to realize the regulation of the final equilibrium point. Different from the general use of the neural network for system identification, the unknown nonlinear dynamic term is approximated based on the state difference obtained by the command filter. Furthermore, a sufficient condition is established to guarantee that the whole system and each subsystem included are uniformly ultimately bounded. Finally, simulation results are given to show the effectiveness of the proposed method.

Aerodynamic Heating Ground Simulation of Hypersonic Vehicles Based on Model-Free Control Using Super Twisting Nonlinear Fractional Order Sliding Mode

In this article, a model-free control (MFC) using super twisting nonlinear fractional order sliding mode for aerodynamic heating ground simulation of hypersonic vehicles (AHGSHV) is proposed. Firstly, the mathematical model of AHGSHV is built up. To reduce order and simplify the dynamic model of AHGSHV, an ultra-local model of MFC is taken into consideration. Then, time delay estimation can be used to estimate systematic uncertainties and external unknown disturbances. On the basis of the original fractional order sliding mode surface, the nonlinear function fal is introduced to design the nonlinear fractional order sliding mode surface, which can guarantee stability, increase convergence rate, and reduce static error and saturation error. In addition, the super twisting reaching law is used to improve the control performance of the reaching phase, resulting from the existence of sign function in the integral term, and it can effectively reduce the high-frequency chattering. Moreover, the Lyapunov function is used to prove the stability of the whole system. Finally, several numerical simulations show that the designed controller has more advantages than others.

Self-Learning Robust Control Synthesis and Trajectory Tracking of Uncertain Dynamics

In this article, we investigate the self-learning robust control synthesis and tracking design of general uncertain dynamical systems. Based on the adaptive critic learning, the robust stabilization method is developed with the help of conducting problem transformation. In addition, by considering the optimal control solution with a discounted cost function, the established method is extended to address the robust trajectory tracking design problem. The Lyapunov stability analysis is also conducted for proving the robustness of the related control plants. Finally, the simulation verification with the three case studies is provided in terms of robust stabilization and trajectory tracking, respectively.

Artificial Intelligence Methodologies for Data Management

This study analyses the main challenges, trends, technological approaches, and artificial intelligence methods developed by new researchers and professionals in the field of machine learning, with an emphasis on the most outstanding and relevant works to date. This literature review evaluates the main methodological contributions of artificial intelligence through machine learning. The methodology used to study the documents was content analysis; the basic terminology of the study corresponds to machine learning, artificial intelligence, and big data between the years 2017 and 2021. For this study, we selected 181 references, of which 120 are part of the literature review. The conceptual framework includes 12 categories, four groups, and eight subgroups. The study of data management using AI methodologies presents symmetry in the four machine learning groups: supervised learning, unsupervised learning, semi-supervised learning, and reinforced learning. Furthermore, the artificial intelligence methods with more symmetry in all groups are artificial neural networks, Support Vector Machines, K-means, and Bayesian Methods. Finally, five research avenues are presented to improve the prediction of machine learning.

Dynamic Event-Triggered Adaptive Tracking Control for a Class of Unknown Stochastic Nonlinear Strict-Feedback Systems

In this paper, the dynamic event-triggered tracking control issue is studied for a class of unknown stochastic nonlinear systems with strict-feedback form. At first, neural networks (NNs) are used to approximate the unknown nonlinear functions. Then, a dynamic event-triggered controller (DETC) is designed through the adaptive backstepping method. Especially, the triggered threshold is dynamically adjusted. Compared with its corresponding static event-triggered mechanism (SETM), the dynamic event-triggered mechanism (DETM) can generate a larger execution interval and further save resources. Moreover, it is verified by two simulation examples that show that the closed-loop stochastic system signals are ultimately fourth moment semi-globally uniformly bounded (SGUUB).

Adaptive synchronization of marine surface ships using disturbance rejection without leader velocity

This work realizes the adaptive neural disturbance rejection for the leader-follower cooperative synchronization of surface ships with model perturbations and ocean disturbances without leader velocity measurements. The virtual ship alleviates the requirements on leader ship's velocities such that the information requirements are only position and heading on the leader ship. The adaptive neural networks approximate model perturbations. The robustifying term attenuates neural network approximation errors. The adaptive neural network-based disturbance observer achieves the disturbance rejection which is integrated with the dynamic surface control technique. The supply ship synchronization control system is ensured to be practical stable. The synchronization control realizes the ship's cooperative synchronization navigation. Simulations with comparisons validate the synchronization scheme.

Quantized Adaptive Finite-Time Bipartite NN Tracking Control for Stochastic Multiagent Systems

This article investigates the quantized adaptive finite-time bipartite tracking control problem for high-order stochastic pure-feedback nonlinear multiagent systems with sensor faults and Prandtl-Ishlinskii (PI) hysteresis. Different from the existing finite-time control results, the nonlinearity of each agent is totally unknown in this article. To overcome the difficulties caused by asymmetric hysteresis quantization and PI hysteresis, a new distributed control method is proposed by adopting the adaptive compensation technique without estimating the lower bounds of parameters. Radial basis function neural networks are employed to estimate unknown nonlinear functions and solve the problem of algebraic loop caused by the pure-feedback nonlinear systems. Then, an adaptive neural-network compensation control approach is proposed to tackle the problem of sensor faults. The problem of the "explosion of complexity" caused by repeated differentiations of the virtual controller is solved by using the dynamic surface control technique. Based on the Lyapunov stability theorem, it is proved that all signals of the closed-loop systems are semiglobal practical finite-time stable in probability, and the bipartite tracking control performance is achieved. Finally, the effectiveness of the proposed control strategy is verified by some simulation results.

Sliding mode-based online fault compensation control for modular reconfigurable robots through adaptive dynamic programming

In this paper, a sliding mode (SM)-based online fault compensation control scheme is investigated for modular reconfigurable robots (MRRs) with actuator failures via adaptive dynamic programming. It consists of a SM-based iterative controller, an adaptive robust term and an online fault compensator. For fault-free MRR systems, the SM surface-based Hamilton–Jacobi–Bellman equation is solved by online policy iteration algorithm. The adaptive robust term is added to guarantee the reachable condition of SM surface. For faulty MRR systems, the actuator failure is compensated online to avoid the fault detection and isolation mechanism. The closed-loop MRR system is guaranteed to be asymptotically stable under the developed fault compensation control scheme. Simulation results verify the effectiveness of the present fault compensation control approach.