Spiking Adaptive Dynamic Programming Based on Poisson Process for Discrete-Time Nonlinear Systems

Fault-Tolerant Attitude Tracking Control Driven by Spiking NNs for Unmanned Aerial Vehicles

In this article, we proposed a novel fault-tolerant control scheme for quadrotor unmanned aerial vehicles (UAVs) based on spiking neural networks (SNNs), which leverages the inherent features of neural network computing to significantly enhance the reliability and robustness of UAV flight control. Traditional control methods are known to be inadequate in dealing with complex and real-time sensor data, which results in poor performance and reduced robustness in fault-tolerant control. In contrast, the temporal processing, parallelism, and nonlinear capacity of SNNs enable the fault-tolerant control scheme to process vast amounts of sensory data with the ability to accurately identify and respond to faults. Furthermore, SNNs can learn and adjust to new environments and fault conditions, providing effective and adaptive flight control. The proposed SNN-based fault-tolerant control scheme demonstrates significant improvements in control accuracy and robustness compared with conventional methods, indicating its potential applicability and suitability for a range of UAV flight control scenarios.

Barrier-critic-disturbance approximate optimal control of nonzero-sum differential games for modular robot manipulators

In this paper, for addressing the safe control problem of modular robot manipulators (MRMs) system with uncertain disturbances, an approximate optimal control scheme of nonzero-sum (NZS) differential games is proposed based on the control barrier function (CBF). The dynamic model of the manipulator system integrates joint subsystems through the utilization of joint torque feedback (JTF) technique, incorporating interconnected dynamic coupling (IDC) effects. By integrating the cost functions relevant to each player with the CBF, the evolution of system states is ensured to remain within the safe region. Subsequently, the optimal tracking control problem for the MRM system is reformulated as an NZS game involving multiple joint subsystems. Based on the adaptive dynamic programming (ADP) algorithm, a cost function approximator for solving Hamilton-Jacobi (HJ) equation using only critic neural networks (NN) is established, which promotes the feasible derivation of the approximate optimal control strategy. The Lyapunov theory is utilized to demonstrate that the tracking error is uniformly ultimately bounded (UUB). Utilizing the CBF's state constraint mechanism prevents the robot from deviating from the safe region, and the application of the NZS game approach ensures that the subsystems of the MRM reach a Nash equilibrium. The proposed control method effectively addresses the problem of safe and approximate optimal control of MRM system under uncertain disturbances. Finally, the effectiveness and superiority of the proposed method are verified through simulations and experiments.

Time-varying gain extended state observer-based adaptive optimal control for disturbed unmanned helicopter

In this paper, the robust adaptive optimal tracking control problem is addressed for the disturbed unmanned helicopter based on the time-varying gain extended state observer (TVGESO) and adaptive dynamic programming (ADP) methods. Firstly, a novel TVGESO is developed to tackle the unknown disturbance, which can overcome the drawback of initial peaking phenomenon in the traditional linear ESO method. Meanwhile, compared with the nonlinear ESO, the proposed TVGESO possesses easier and rigorous stability analysis process. Subsequently, the optimal tracking control issue for the original unmanned helicopter system is transformed into an optimization stabilization problem. By means of the ADP and neural network techniques, the feedforward controller and optimal feedback controller are skillfully designed. Compared with the conventional backstepping approach, the designed anti-disturbance optimal controller can make the unmanned helicopter accomplish the tracking task with less energy. Finally, simulation comparisons demonstrate the validity of the developed control scheme.

Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing

The utilization of lower extremity exoskeletons has witnessed a growing presence across diverse domains such as the military, medical treatment, and rehabilitation. This paper introduces a novel design of a lower extremity exoskeleton specifically tailored for individuals engaged in heavy object carrying tasks. The exoskeleton incorporates an impressive 12 degrees of freedom (DOF), with four of them being effectively controlled through hydraulic cylinders. To achieve optimal control of this intricate lower extremity exoskeleton system, the authors propose an adaptive dynamic programming (ADP) algorithm. Several crucial components are established to implement this control scheme. These include the formulation of the state equation for the lower extremity exoskeleton system, which is well-suited for the ADP algorithm. Additionally, a corresponding performance index function based on the tracking error is devised, along with the game algebraic Riccati equation. By employing the value iteration ADP scheme, the lower extremity exoskeleton demonstrates highly effective tracking control. This research not only highlights the potential of the proposed control approach but also showcases its ability to enhance the overall performance and functionality of lower extremity exoskeletons, particularly in scenarios involving heavy object carrying. Overall, this study contributes to the advancement of lower extremity exoskeleton technology and offers valuable insights into the application of ADP algorithms for achieving precise and efficient control in demanding tasks.

Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.