Robust Neurooptimal Control for a Robot via Adaptive Dynamic Programming

Dynamic Event-Triggered Strategy-Based Optimal Control of Modular Robot Manipulator: A Multiplayer Nonzero-Sum Game Perspective

Due to the limited computing and processing ability of modular robot manipulator (MRM) components, such as sensors and controllers, event-triggered mechanisms are considered a crucial communication paradigm shift in resource constrained applications. Dynamic event-triggered mechanism is developing into a new technology by reason of its higher resource utilization efficiency and more flexible system design requirements than traditional event-triggered. Therefore, an optimal control scheme of multiplayer nonzero-sum game based on dynamic event-triggered is developed for MRM systems with uncertain disturbances. First, dynamic model of the MRM is established according to joint torque feedback technique and model uncertainty is estimated by data-driven-based neural network identifier. In the framework of differential game, the tracking control problem of MRM system is transformed into the optimal control problem for multiplayer nonzero-sum game with the control input of each joint module as the player. Then, the static event-triggered control problem of MRM system is studied based on adaptive dynamic programming algorithm. On this basis, the internal dynamic variable describing the previous state of the system is introduced, and the characteristics of dynamic trigger rule and its relationship with static rule are revealed theoretically. By designing an exponential attenuation signal, the minimum sampling interval of the system is always positive, so that Zeno behavior is excluded. Lyapunov theory proves that the system is asymptotically stable and the experimental results verify the validity of the proposed method.

Observer-Based Neural Control of N-Link Flexible-Joint Robots

This article concentrates on the adaptive neural control approach of n -link flexible-joint electrically driven robots. The presented control method only needs to know the position and armature current information of the flexible-joint manipulator. An adaptive observer is designed to estimate the velocities of links and motors, and radial basis function neural networks are applied to approximate the unknown nonlinearities. Based on the backstepping technique and the Lyapunov stability theory, the observer-based neural control issue is addressed by relying on uplink-event-triggered states only. It is demonstrated that all signals are semi-globally ultimately uniformly bounded and the tracking errors can converge to a small neighborhood of zero. Finally, simulation results are shown to validate the designed event-triggered control strategy.

Robust Actor-Critic With Relative Entropy Regulating Actor

The accurate estimation of Q-function and the enhancement of agent's exploration ability have always been challenges of off-policy actor-critic algorithms. To address the two concerns, a novel robust actor-critic (RAC) is developed in this article. We first derive a robust policy improvement mechanism (RPIM) by using the local optimal policy about the current estimated Q-function to guide policy improvement. By constraining the relative entropy between the new policy and the previous one in policy improvement, the proposed RPIM can enhance the stability of the policy update process. The theoretical analysis shows that the incentive to increase the policy entropy is endowed when the policy is updated, which is conducive to enhancing the exploration ability of agents. Then, RAC is developed by applying the proposed RPIM to regulate the actor improvement process. The developed RAC is proven to be convergent. Finally, the proposed RAC is evaluated on some continuous-action control tasks in the MuJoCo platform and the experimental results show that RAC outperforms several state-of-the-art reinforcement learning algorithms.

Robust Admittance Control of Optimized Robot-Environment Interaction Using Reference Adaptation

In this article, a robust control scheme is proposed for robots to achieve an optimal performance in the process of interacting with external forces from environments. The environmental dynamics are defined as a linear model, and the interaction performance is evaluated by a defined cost function, which is composed of trajectory errors and force regulation. Based on admittance control, the reference adaptation method is used to minimize the cost function and achieve the optimal interaction performance. To make the trajectory tracking controller robust to the unknown disturbance of internal system dynamics, an auxiliary system is defined and the approximation optimal controller is designed. Experiments on the Baxter robot are conducted to verify the effectiveness of the proposed method.

Industrial Robot Trajectory Tracking Control Using Multi-Layer Neural Networks Trained by Iterative Learning Control

Fast and precise robot motion is needed in many industrial applications. Most industrial robot motion controllers allow externally commanded motion profiles, but the trajectory tracking performance is affected by the robot dynamics and joint servo controllers, to which users have no direct access and about which they have little information. The performance is further compromised by time delays in transmitting the external command as a setpoint to the inner control loop. This paper presents an approach for combining neural networks and iterative learning controls to improve the trajectory tracking performance for a multi-axis articulated industrial robot. For a given desired trajectory, the external command is iteratively refined using a high-fidelity dynamical simulator to compensate for the robot inner-loop dynamics. These desired trajectories and the corresponding refined input trajectories are then used to train multi-layer neural networks to emulate the dynamical inverse of the nonlinear inner-loop dynamics. We show that with a sufficiently rich training set, the trained neural networks generalize well to trajectories beyond the training set as tested in the simulator. In applying the trained neural networks to a physical robot, the tracking performance still improves but not as much as in the simulator. We show that transfer learning effectively bridges the gap between simulation and the physical robot. Finally, we test the trained neural networks on other robot models in simulation and demonstrate the possibility of a general purpose network. Development and evaluation of this methodology are based on the ABB IRB6640-180 industrial robot and ABB RobotStudio software packages.