Robot Learning System Based on Adaptive Neural Control and Dynamic Movement Primitives

NN-based visual servoing compensation control of a Gough-Stewart platform with uncertain load

This paper addresses the trajectory tracking control of a Gough-Stewart platform (GS platform) with an uncertain load. The uncertainty of this load leads to external disturbance to the parallel robot, which affects the dynamic coupling among the six degrees of freedom (DOF) and the tracking performance. Even though many researchers focus on improving the system robustness and tracking accuracy, there still exist two main problems: the system's internal uncertainties, including the modeling, manufacturing, and assembly errors of the parallel robot affect the control accuracy; the uncertain external disturbance varies in an extensive range and reduces the stability and tracking accuracy of the system. Therefore, we propose a novel control methodology: the dynamic Image-based visual servoing (IBVS) Radial basis function neural network (RBFNN) real-time compensation controller. This control considers an acceleration model of visual servoing and performs real-time compensation for the enormous uncertain disturbance from the load with RBFNN. The stability of the proposed controller is fully investigated with the Lyapunov method. Simulations are performed on a GS platform with an uncertain load to test the controller's performance. It turns out that this controller provides good tracking accuracy and robustness simultaneously.

Learning to Perform Trajectory Generation From Low-Quality Demonstrations

Human-robot skill transfer is an important means for robots to learn skills and has received more and more attention and research in recent years. Typically, to ensure effective skill transfer, a skill is demonstrated several times by a human, from which a robot learns the features contained in the demonstrations and reproduces the skill in a new environment. However, it is necessary to consider the cases such as errors in human demonstrations and sensor issues, resulting in imperfect demonstrations, unrelated data, information loss, and variations in the lengths and amplitudes of the demonstrations. Therefore, this brief proposes a new trajectory alignment and filtering method for extracting relatively useful information from multiple demonstrations. This method can be used in conjunction with most probabilistic movement learning methods (this brief uses probabilistic movement primitives (ProMPs) as an example) for learning from demonstrations (LfDs), so that the robot can eventually learn and generate trajectories for completing skills from multiple demonstrations of varying quality. The effectiveness of the proposed method is verified by simulation results.

Learning an Autonomous Dynamic System to Encode Periodic Human Motion Skills

Learning an autonomous dynamic system (ADS) encoding human motion rules has been shown as an effective way for human motion skills transfer. However, most existing approaches focus on goal-directed motion skills transfer, and the study on periodic motion skills transfer is rare. One popular approach for periodic motion skills transfer is learning periodic dynamic movement primitive (DMP); however, periodic DMP is sensitive to spatial disturbances due to the introduction of the phase parameters. To solve this issue, this brief presents a novel approach to learn an ADS with a stable limit cycle without introducing phase parameters. First, a data-driven Lyapunov function (energy function) is learned, such that one of its level surfaces is consistent with periodic human demonstration trajectories. Then, an ADS is learned by sequentially solving energy function-related constrained optimization problems. With a proper design of constraint functions, we can ensure that the trajectory generated by the ADS will converge to an energy function-level surface, of which the shape is similar to periodic human demonstration trajectories. Experiments are conducted to show the effectiveness of the proposed approach (PA).

A novel trajectory learning method for robotic arms based on Gaussian Mixture Model and k-value selection algorithm

In the field of robotic arm trajectory imitation learning, Gaussian Mixture Models are widely used for their ability to capture the characteristics of complex trajectories. However, one major challenge in utilizing these models lies in the initialization process, particularly in determining the number of Gaussian kernels, or the k-value. The choice of the k-value significantly impacts the model's performance, and traditional methods, such as random selection or selection based on empirical knowledge, often lead to suboptimal outcomes. To address this challenge, this paper proposes a novel trajectory learning method for robotic arms that combines Gaussian Mixture Model with a k-value selection algorithm. The proposed approach leverages the principles of the elbow method along with the properties of exponential functions, correction terms, and weight adjustments to accurately determine the optimal k-value. Next, k-means clustering is applied with the optimal k-value to initialize the parameters of the Gaussian Mixture Model, which are then refined and trained through the Expectation-Maximization algorithm. The resulting model parameters are then employed in Gaussian Mixture Regression to generate the robotic arm trajectories. The effectiveness of the proposed method is validated through both simulation experiments with two-dimensional theoretical nonlinear dynamic systems and physical experiments with actual robotic arm data. Experimental results demonstrate that, compared to the traditional Gaussian Mixture Model approach, the proposed method improves trajectory accuracy by more than 15%, as shown by reductions in both the Mean Absolute Error and the Root Mean Square Error. These results highlight that the proposed method significantly enhances the accuracy and efficiency of robotic arm trajectory generation, providing a promising solution for improving robotic manipulation tasks.

Relative Entropy Regularized Sample-Efficient Reinforcement Learning With Continuous Actions

In this article, a novel reinforcement learning (RL) approach, continuous dynamic policy programming (CDPP), is proposed to tackle the issues of both learning stability and sample efficiency in the current RL methods with continuous actions. The proposed method naturally extends the relative entropy regularization from the value function-based framework to the actor-critic (AC) framework of deep deterministic policy gradient (DDPG) to stabilize the learning process in continuous action space. It tackles the intractable softmax operation over continuous actions in the critic by Monte Carlo estimation and explores the practical advantages of the Mellowmax operator. A Boltzmann sampling policy is proposed to guide the exploration of actor following the relative entropy regularized critic for superior learning capability, exploration efficiency, and robustness. Evaluated by several benchmark and real-robot-based simulation tasks, the proposed method illustrates the positive impact of the relative entropy regularization including efficient exploration behavior and stable policy update in RL with continuous action space and successfully outperforms the related baseline approaches in both sample efficiency and learning stability.

Category-Level 6-D Object Pose Estimation With Shape Deformation for Robotic Grasp Detection

Category-level 6-D object pose estimation plays a crucial role in achieving reliable robotic grasp detection. However, the disparity between synthetic and real datasets hinders the direct transfer of models trained on synthetic data to real-world scenarios, leading to ineffective results. Additionally, creating large-scale real datasets is a time-consuming and labor-intensive task. To overcome these challenges, we propose CatDeform, a novel category-level object pose estimation network trained on synthetic data but capable of delivering good performance on real datasets. In our approach, we introduce a transformer-based fusion module that enables the network to leverage multiple sources of information and enhance prediction accuracy through feature fusion. To ensure proper deformation of the prior point cloud to align with scene objects, we propose a transformer-based attention module that deforms the prior point cloud from both geometric and feature perspectives. Building upon CatDeform, we design a two-branch network for supervised learning, bridging the gap between synthetic and real datasets and achieving high-precision pose estimation in real-world scenes using predominantly synthetic data supplemented with a small amount of real data. To minimize reliance on large-scale real datasets, we train the network in a self-supervised manner by estimating object poses in real scenes based on the synthetic dataset without manual annotation. We conduct training and testing on CAMERA25 and REAL275 datasets, and our experimental results demonstrate that the proposed method outperforms state-of-the-art (SOTA) techniques in both self-supervised and supervised training paradigms. Finally, we apply CatDeform to object pose estimation and robotic grasp experiments in real-world scenarios, showcasing a higher grasp success rate.

Neuro-Adaptive-Based Fixed-Time Composite Learning Control for Manipulators With Given Transient Performance

This article investigates an adaptive neural network (NN) control technique with fixed-time tracking capabilities, employing composite learning, for manipulators under constrained position error. The first step involves integrating the composite learning method into the NN to address the dynamic uncertainties that inevitably arise in manipulators. A composite adaptive updating law of NN weights is formulated, requiring adherence solely to the relaxed interval excitation (IE) conditions. In addition, for the output error, instead of knowing the initial conditions, this article integrates the error transfer function and asymmetric barrier function to achieve the specific performance for position error in both steady and transient states. Furthermore, the fixed-time control methodology and Lyapunov stability criterion are synergistically employed in order to guarantee the convergence of all signals in the manipulators to a compact neighborhood around the origin within a fixed-time. Finally, numerical simulation and experiments with the Baxter robot results both determine the capability of the NN composite learning technique and fixed-time control strategy.

Influence of Imperfections on the Operational Correctness of DNN-kWTA Model

The dual neural network (DNN)-based k -winner-take-all (WTA) model is able to identify the k largest numbers from its m input numbers. When there are imperfections, such as non-ideal step function and Gaussian input noise, in the realization, the model may not output the correct result. This brief analyzes the influence of the imperfections on the operational correctness of the model. Due to the imperfections, it is not efficient to use the original DNN- k WTA dynamics for analyzing the influence. In this regard, this brief first derives an equivalent model to describe the dynamics of the model under the imperfections. From the equivalent model, we derive a sufficient condition for which the model outputs the correct result. Thus, we apply the sufficient condition to design an efficiently estimation method for the probability of the model outputting the correct result. Furthermore, for the inputs with uniform distribution, a closed form expression for the probability value is derived. Finally, we extend our analysis for handling non-Gaussian input noise. Simulation results are provided to validate our theoretical results.

Finite-Time Neural Network-Based Hierarchical Sliding Mode Antiswing Control for Underactuated Dual Ship-Mounted Cranes With Unmatched Sea Wave Disturbances Suppression

As typical mechanical transportation equipment, cooperative dual ship-mounted cranes are widely used to transport large goods or containers in the marine environment. However, the control problem of the dual ship-mounted crane system is much more complex due to its underactuated characteristic and persistent unmatched disturbances. To solve these problems, we propose a novel neural network (NN)-based hierarchical sliding mode adaptive (HSMA) control method in this article. More specifically, an appropriate hierarchical sliding mode surface is first designed to connect the actuated and underactuated system state variables effectively. At the same time, the NNs are constructed to compensate for the unmatched interference of ship motions induced by sea waves simultaneously. Not only can the booms and the rope lengths reach their desired positions in finite time, but also the synchronous swing angles of the payload can be effectively eliminated. The asymptotic convergence of the closed-loop system's equilibrium points is achieved through rigorous mathematical proofs. Furthermore, the stability of each sliding mode surface is also analyzed utilizing the Lyapunov technique and Barbalat's lemma. Finally, numerous groups of compared numerical simulation results are investigated to further show the effectiveness and strong robustness of the proposed NN-based HSMA controller.

A Robust Coevolutionary Neural-Based Optimization Algorithm for Constrained Nonconvex Optimization

For nonconvex optimization problems, a routine is to assume that there is no perturbation when executing the solution task. Nevertheless, dealing with the perturbation in advance may increase the burden on the system and take up extra time. To remedy this weakness, we propose a robust coevolutionary neural-based optimization algorithm with inherent robustness based on the hybridization between the particle swarm optimization and a class of robust neural dynamics (RND). In this framework, every neural agent guided by the RND supersedes the place of the particle, mutually searches for the optimal solution, and stabilizes itself from different perturbations. The theoretical analysis ensures that the proposed algorithm is globally convergent with probability one. Besides, the effectiveness and robustness of the proposed approach are illustrated by illustrative examples compared with the existing methods. We further apply this proposed algorithm to the source localization and manipulability optimization of the redundant manipulator, simultaneously disposing of perturbation from the internal and exogenous system with satisfactory performance.

A Deep Learning Framework for Start-End Frame Pair-Driven Motion Synthesis

A start-end frame pair and a motion pattern-based motion synthesis scheme can provide more control to the synthesis process and produce content-various motion sequences. However, the data preparation for the motion training is intractable, and concatenating feature spaces of the start-end frame pair and the motion pattern lacks theoretical rationality in previous works. In this article, we propose a deep learning framework that completes automatic data preparation and learns the nonlinear mapping from start-end frame pairs to motion patterns. The proposed model consists of three modules: action detection, motion extraction, and motion synthesis networks. The action detection network extends the deep subspace learning framework to a supervised version, i.e., uses the local self-expression (LSE) of the motion data to supervise feature learning and complement the classification error. A long short-term memory (LSTM)-based network is used to efficiently extract the motion patterns to address the speed deficiency reflected in the previous optimization-based method. A motion synthesis network consists of a group of LSTM-based blocks, where each of them is to learn the nonlinear relation between the start-end frame pairs and the motion patterns of a certain joint. The superior performances in action detection accuracy, motion pattern extraction efficiency, and motion synthesis quality show the effectiveness of each module in the proposed framework.

Observer-Based Neural Control of N-Link Flexible-Joint Robots

This article concentrates on the adaptive neural control approach of n -link flexible-joint electrically driven robots. The presented control method only needs to know the position and armature current information of the flexible-joint manipulator. An adaptive observer is designed to estimate the velocities of links and motors, and radial basis function neural networks are applied to approximate the unknown nonlinearities. Based on the backstepping technique and the Lyapunov stability theory, the observer-based neural control issue is addressed by relying on uplink-event-triggered states only. It is demonstrated that all signals are semi-globally ultimately uniformly bounded and the tracking errors can converge to a small neighborhood of zero. Finally, simulation results are shown to validate the designed event-triggered control strategy.

A data-driven acceleration-level scheme for image-based visual servoing of manipulators with unknown structure

The research on acceleration-level visual servoing of manipulators is crucial yet insufficient, which restricts the potential application range of visual servoing. To address this issue, this paper proposes a quadratic programming-based acceleration-level image-based visual servoing (AIVS) scheme, which considers joint constraints. Besides, aiming to address the unknown problems in visual servoing systems, a data-driven learning algorithm is proposed to facilitate estimating structural information. Building upon this foundation, a data-driven acceleration-level image-based visual servoing (DAIVS) scheme is proposed, integrating learning and control capabilities. Subsequently, a recurrent neural network (RNN) is developed to tackle the DAIVS scheme, followed by theoretical analyses substantiating its stability. Afterwards, simulations and experiments on a Franka Emika Panda manipulator with eye-in-hand structure and comparisons among the existing methods are provided. The obtained results demonstrate the feasibility and practicality of the proposed schemes and highlight the superior learning and control ability of the proposed RNN. This method is particularly well-suited for visual servoing applications of manipulators with unknown structure.

Research on Robot Screwing Skill Method Based on Demonstration Learning

A robot screwing skill learning framework based on teaching-learning is proposed to improve the generalization ability of robots for different scenarios and objects, combined with the experience of a human operation. This framework includes task-based teaching, learning, and summarization. We teach a robot to twist and gather the operation's trajectories, define the obstacles with potential functions, and counter the twisting of the robot using a skill-learning-based dynamic movement primitive (DMP) and Gaussian mixture model-Gaussian mixture regression (GMM-GMR). The hole-finding and screwing stages of the process are modeled. In order to verify the effectiveness of the robot tightening skill learning model and its adaptability to different tightening scenarios, obstacle avoidance trends and tightening experiments were conducted. Obstacle avoidance and tightening experiments were conducted on the robot tightening platform for bolts, plastic bottle caps, and faucets. The robot successfully avoided obstacles and completed the twisting task, verifying the effectiveness of the robot tightening skill learning model and its adaptability to different tightening scenarios.

Predefined-Time Hierarchical Coordinated Neural Control for Hypersonic Reentry Vehicle

This paper investigates the predefined-time hierarchical coordinated adaptive control on the hypersonic reentry vehicle in presence of low actuator efficiency. In order to compensate for the deficiency of rudder deflection in advantage of channel coupling, the hierarchical design is proposed for coordination of the elevator deflection and aileron deflection. Under the control scheme, the equivalent control law and switching control law are constructed with the predefined-time technology. For the dynamics uncertainty approximation, the composite learning using the tracking error and the prediction error is constructed by designing the serial-parallel estimation model. The closed-loop system stability is analyzed via the Lyapunov approach and the tracking errors are guaranteed to be uniformly ultimately bounded in a predefined time. The tracking performance and the learning accuracy of the proposed algorithm are verified via simulation tests.

Assessment System for Child Head Injury from Falls Based on Neural Network Learning

Toddlers face serious health hazards if they fall from relatively high places at home during everyday activities and are not swiftly rescued. Still, few effective, precise, and exhaustive solutions exist for such a task. This research aims to create a real-time assessment system for head injury from falls. Two phases are involved in processing the framework: In phase I, the data of joints is obtained by processing surveillance video with Open Pose. The long short-term memory (LSTM) network and 3D transform model are then used to integrate key spots' frame space and time information. In phase II, the head acceleration is derived and inserted into the HIC value calculation, and a classification model is developed to assess the injury. We collected 200 RGB-captured daily films of 13- to 30-month-old toddlers playing near furniture edges, guardrails, and upside-down falls. Five hundred video clips extracted from these are divided in an 8:2 ratio into a training and validation set. We prepared an additional collection of 300 video clips (test set) of toddlers' daily falling at home from their parents to evaluate the framework's performance. The experimental findings revealed a classification accuracy of 96.67%. The feasibility of a real-time AI technique for assessing head injuries in falls through monitoring was proven.

A Robot Motion Learning Method Using Broad Learning System Verified by Small-Scale Fish-Like Robot

The widespread application of learning-based methods in robotics has allowed significant simplifications to controller design and parameter adjustment. In this article, robot motion is controlled with learning-based methods. A control policy using a broad learning system (BLS) for robot point-reaching motion is developed. A sample application based on a magnetic small-scale robotic system is designed without detailed mathematical modeling of the dynamic systems. The parameter constraints of the nodes in the BLS-based controller are derived based on Lyapunov theory. The design and control training processes for a small-scale magnetic fish motion are presented. Finally, the effectiveness of the proposed method is demonstrated by convergence of the artificial magnetic fish motion to the targeted area with the BLS trajectory, successfully avoiding obstacles.

Data-Based Predictive Control via Multistep Policy Gradient Reinforcement Learning

In this article, a model-free predictive control algorithm for the real-time system is presented. The algorithm is data driven and is able to improve system performance based on multistep policy gradient reinforcement learning. By learning from the offline dataset and real-time data, the knowledge of system dynamics is avoided in algorithm design and application. Cooperative games of the multiplayer in time horizon are presented to model the predictive control as optimization problems of multiagent and guarantee the optimality of the predictive control policy. In order to implement the algorithm, neural networks are used to approximate the action-state value function and predictive control policy, respectively. The weights are determined by using the methods of weighted residual. Numerical results show the effectiveness of the proposed algorithm.

A Comprehensive Study of 3-D Vision-Based Robot Manipulation

Robot manipulation, for example, pick-and-place manipulation, is broadly used for intelligent manufacturing with industrial robots, ocean engineering with underwater robots, service robots, or even healthcare with medical robots. Most traditional robot manipulations adopt 2-D vision systems with plane hypotheses and can only generate 3-DOF (degrees of freedom) pose accordingly. To mimic human intelligence and endow the robot with more flexible working capabilities, 3-D vision-based robot manipulation has been studied. However, this task is still challenging in the open world especially for general object recognition and pose estimation with occlusion in cluttered backgrounds and human-like flexible manipulation. In this article, we propose a comprehensive analysis of recent progress about the 3-D vision for robot manipulation, including 3-D data acquisition and representation, robot-vision calibration, 3-D object detection/recognition, 6-DOF pose estimation, grasping estimation, and motion planning. We then present some public datasets, evaluation criteria, comparisons, and challenges. Finally, the related application domains of robot manipulation are given, and some future directions and open problems are studied as well.

Force Distribution and Estimation for Cooperative Transportation Control on Multiple Unmanned Ground Vehicles

This article presents an effective design of omnidirectional four-mecanum-wheeled vehicles to transport an object and track a predefined trajectory cooperatively. Furthermore, a novel design of the rotary platform is presented for multiple unmanned ground vehicles (m-UGVs) to load objects and provide better maneuverability in confined spaces during cooperative transportation. The number of unmanned ground vehicles (UGVs) is adjustable according to the object's weight and size in the proposed framework because transportation is accomplished without physical grippers. Moreover, to minimize the complexity in dealing with the interactive force between the object and UGVs, no force/torque sensor is used in the design of the control algorithm. Instead, an adaptive sliding-mode controller is formulated to cope with the dynamic uncertainties and smoothly transport an object along a desired trajectory. Thus, three external force analyses-gradient projection method, adaptive force estimation, and radial basis function neural network force estimation-are proposed for m-UGVs. In addition, the stability and the performance tracking of the m-UGV system in the presence of dynamic uncertainties using the proposed force estimation are investigated by employing the Lyapunov theory. Finally, experiments on cooperative transportation are presented to demonstrate the efficiency and efficacy of the m-UGV system.

A multimodal human-robot sign language interaction framework applied in social robots

Deaf-mutes face many difficulties in daily interactions with hearing people through spoken language. Sign language is an important way of expression and communication for deaf-mutes. Therefore, breaking the communication barrier between the deaf-mute and hearing communities is significant for facilitating their integration into society. To help them integrate into social life better, we propose a multimodal Chinese sign language (CSL) gesture interaction framework based on social robots. The CSL gesture information including both static and dynamic gestures is captured from two different modal sensors. A wearable Myo armband and a Leap Motion sensor are used to collect human arm surface electromyography (sEMG) signals and hand 3D vectors, respectively. Two modalities of gesture datasets are preprocessed and fused to improve the recognition accuracy and to reduce the processing time cost of the network before sending it to the classifier. Since the input datasets of the proposed framework are temporal sequence gestures, the long-short term memory recurrent neural network is used to classify these input sequences. Comparative experiments are performed on an NAO robot to test our method. Moreover, our method can effectively improve CSL gesture recognition accuracy, which has potential applications in a variety of gesture interaction scenarios not only in social robots.

Tongue Image Texture Classification Based on Image Inpainting and Convolutional Neural Network

Tongue texture analysis is of importance to inspection diagnosis in traditional Chinese medicine (TCM), which has great application and irreplaceable value. The tough and tender classification for tongue image relies mainly on image texture of tongue body. However, texture discontinuity adversely affects the classification of the tough and tender tongue classification. In order to promote the accuracy and robustness of tongue texture analysis, a novel tongue image texture classification method based on image inpainting and convolutional neural network is proposed. Firstly, Gaussian mixture model is applied to separate the tongue coating and body. In order to exclude the interference of tongue coating on tough and tender tongue classification, a tongue body image inpainting model is built based on generative image inpainting with contextual attention to realize the inpainting of the tongue body image to ensure the continuity of texture and color change of tongue body image. Finally, the classification model of the tough and tender tongue inpainting image based on ResNet101 residual network is used to train and test. The experimental results show that the proposed method achieves better classification results compared with the existing methods of texture classification of tongue image and provides a new idea for tough and tender tongue classification.

A Learning-Based Stable Servo Control Strategy Using Broad Learning System Applied for Microrobotic Control

As the controller parameter adjustment process is simplified significantly by using learning algorithms, the studies about learning-based control attract a lot of interest in recent years. This article focuses on the intelligent servo control problem using learning from desired demonstrations. Compared with the previous studies about the learning-based servo control, a control policy using the broad learning system (BLS) is developed and first applied to a microrobotic system, since the advantages of the BLS, such as simple structure and no-requirement for retraining when new demos' data is provided. Then, the Lyapunov theory is skillfully combined with the complex learning algorithm to derive the controller parameters' constraints. Thus, the final control policy not only can obtain the movement skills of the desired demonstrations but also have the strong ability of generalization and error convergence. Finally, simulation and experimental examples verify the effectiveness of the proposed strategy using MATLAB and a microswimmer trajectory tracking system.

A Hybrid State/Disturbance Observer-Based Feedback Control of Robot with Multiple Constraints

Controlling the manipulator is a big challenge due to its hysteresis, deadzone, saturation, and the disturbances of actuators. This study proposes a hybrid state/disturbance observer-based multiple-constraint control mechanism to address this difficulty. It first proposes a hybrid state/disturbance observer to simultaneously estimate the unmeasurable states and external disturbances. Based on this, a barrier Lyapunov function is proposed and implemented to handle output saturation constraints, and a back-stepping control method is developed to provide sufficient control performance under multiple constraints. Furthermore, the stability of the proposed controller is analyzed and proved. Finally, simulations and experiments are carried out on a 2-DOF and 6-DOF robot, respectively. The results show that the proposed control method can effectively achieve the desired control performance. Compared with several commonly used control methods and intelligent control methods, the proposed method shows superiority. Experiments on a 6-DOF robot verify that the proposed method has good tracking performance for all joints and does not violate constraints.

Finite-Time Robust Intelligent Control of Strict-Feedback Nonlinear Systems With Flight Dynamics Application

The tracking control is investigated for a class of uncertain strict-feedback systems with robust design and learning systems. Using the switching mechanism, the states will be driven back by the robust design when they run out of the region of adaptive control. The adaptive design is working to achieve precise adaptation and higher tracking precision in the neural working domain, while the finite-time robust design is developed to make the system stable outside. To achieve good tracking performance, the novel prediction error-based adaptive law is constructed by considering the estimation performance. Furthermore, the output constraint is achieved by imbedding the barrier Lyapunov function-based design. The finite-time convergence and the uniformly ultimate boundedness of the system signal can be guaranteed. Simulation studies show that the proposed approach presents robustness and adaptation to system uncertainty.

Neural Networks Enhanced Optimal Admittance Control of Robot-Environment Interaction Using Reinforcement Learning

In this paper, an adaptive admittance control scheme is developed for robots to interact with time-varying environments. Admittance control is adopted to achieve a compliant physical robot-environment interaction, and the uncertain environment with time-varying dynamics is defined as a linear system. A critic learning method is used to obtain the desired admittance parameters based on the cost function composed of interaction force and trajectory tracking without the knowledge of the environmental dynamics. To deal with dynamic uncertainties in the control system, a neural-network (NN)-based adaptive controller with a dynamic learning framework is developed to guarantee the trajectory tracking performance. Experiments are conducted and the results have verified the effectiveness of the proposed method.

DMPs-based skill learning for redundant dual-arm robotic synchronized cooperative manipulation

Dual-arm robot manipulation is applicable to many domains, such as industrial, medical, and home service scenes. Learning from demonstrations is a highly effective paradigm for robotic learning, where a robot learns from human actions directly and can be used autonomously for new tasks, avoiding the complicated analytical calculation for motion programming. However, the learned skills are not easy to generalize to new cases where special constraints such as varying relative distance limitation of robotic end effectors for human-like cooperative manipulations exist. In this paper, we propose a dynamic movement primitives (DMPs) based skills learning framework for redundant dual-arm robots. The method, with a coupling acceleration term to the DMPs function, is inspired by the transient performance control of Barrier Lyapunov Functions. The additional coupling acceleration term is calculated based on the constant joint distance and varying relative distance limitations of end effectors for object-approaching actions. In addition, we integrate the generated actions in joint space and the solution for a redundant dual-arm robot to complete a human-like manipulation. Simulations undertaken in Matlab and Gazebo environments certify the effectiveness of the proposed method.

Adaptive Neural Network Control for Full-State Constrained Robotic Manipulator With Actuator Saturation and Time-Varying Delays

This article proposes an adaptive neural network (NN) control method for an n -link constrained robotic manipulator. Driven by actual demands, manipulator and actuator dynamics, state and input constraints, and unknown time-varying delays are taken into account simultaneously. NNs are employed to approximate unknown nonlinearities. Time-varying barrier Lyapunov functions are utilized to cope with full-state constraints. By resorting to saturation function and Lyapunov-Krasovskii functionals, the effects of actuator saturation and time delays are eliminated. It is proved that all the closed-loop signals are semiglobally uniformly ultimately bounded, full-state constraints and actuator saturation are not violated, and error signals remain within compact sets around zero. Simulation studies are given to demonstrate the validity and advantages of this control scheme.

Local Self-Expression Subspace Learning Network for Motion Capture Data

Deep subspace learning is an important branch of self-supervised learning and has been a hot research topic in recent years, but current methods do not fully consider the individualities of temporal data and related tasks. In this paper, by transforming the individualities of motion capture data and segmentation task as the supervision, we propose the local self-expression subspace learning network. Specifically, considering the temporality of motion data, we use the temporal convolution module to extract temporal features. To implement the local validity of self-expression in temporal tasks, we design the local self-expression layer which only maintains the representation relations with temporally adjacent motion frames. To simulate the interpolatability of motion data in the feature space, we impose a group sparseness constraint on the local self-expression layer to impel the representations only using selected keyframes. Besides, based on the subspace assumption, we propose the subspace projection loss, which is induced from distances of each frame projected to the fitted subspaces, to penalize the potential clustering errors. The superior performances of the proposed model on the segmentation task of synthetic data and three tasks of real motion capture data demonstrate the feature learning ability of our model.

Trajectory Tracking within a Hierarchical Primitive-Based Learning Approach

A hierarchical learning control framework (HLF) has been validated on two affordable control laboratories: an active temperature control system (ATCS) and an electrical rheostatic braking system (EBS). The proposed HLF is data-driven and model-free, while being applicable on general control tracking tasks which are omnipresent. At the lowermost level, L1, virtual state-feedback control is learned from input–output data, using a recently proposed virtual state-feedback reference tuning (VSFRT) principle. L1 ensures a linear reference model tracking (or matching) and thus, indirect closed-loop control system (CLCS) linearization. On top of L1, an experiment-driven model-free iterative learning control (EDMFILC) is then applied for learning reference input–controlled outputs pairs, coined as primitives. The primitives’ signals at the L2 level encode the CLCS dynamics, which are not explicitly used in the learning phase. Data reusability is applied to derive monotonic and safely guaranteed learning convergence. The learning primitives in the L2 level are finally used in the uppermost and final L3 level, where a decomposition/recomposition operation enables prediction of the optimal reference input assuring optimal tracking of a previously unseen trajectory, without relearning by repetitions, as it was in level L2. Hence, the HLF enables control systems to generalize their tracking behavior to new scenarios by extrapolating their current knowledge base. The proposed HLF framework endows the CLCSs with learning, memorization and generalization features which are specific to intelligent organisms. This may be considered as an advancement towards intelligent, generalizable and adaptive control systems.

Adaptive Intelligent Sliding Mode Control of a Dynamic System with a Long Short-Term Memory Structure

In this work, a novel fuzzy neural network (NFNN) with a long short-term memory (LSTM) structure was derived and an adaptive sliding mode controller, using NFNN (ASMC-NFNN), was developed for a class of nonlinear systems. Aimed at the unknown uncertainties in nonlinear systems, an NFNN was designed to estimate unknown uncertainties, which combined the advantages of fuzzy systems and neural networks, and also introduced a special LSTM recursive structure. The special three gating units in the LSTM structure enabled it to have selective forgetting and memory mechanisms, which could make full use of historical information, and have a stronger ability to learn and estimate unknown uncertainties than general recurrent neural networks. The Lyapunov stability rule guaranteed the parameter convergence of the neural network and system stability. Finally, research into a simulation of an active power filter system showed that the proposed new algorithm had better static and dynamic properties and robustness compared with a sliding controller that uses a recurrent fuzzy neural network (RFNN).

Composite-Learning-Based Adaptive Neural Control for Dual-Arm Robots With Relative Motion

This article presents an adaptive control method for dual-arm robot systems to perform bimanual tasks under modeling uncertainties. Different from the traditional symmetric bimanual robot control, we study the dual-arm robot control with relative motions between robotic arms and a grasped object. The robot system is first divided into two subsystems: a settled manipulator system and a tool-used manipulator system. Then, a command filtered control technique is developed for trajectory tracking and contact force control. In addition, to deal with the inevitable dynamic uncertainties, a radial basis function neural network (RBFNN) is employed for the robot, with a novel composite learning law to update the NN weights. The composite learning is mainly based on an integration of the historic data of NN regression such that information of the estimate error can be utilized to improve the convergence. Moreover, a partial persistent excitation condition is employed to ensure estimation convergence. The stability analysis is performed by using the Lyapunov theorem. Numerical simulation results demonstrate the validity of the proposed control and learning algorithm.

Recurrent Neural Dynamics Models for Perturbed Nonstationary Quadratic Programs: A Control-Theoretical Perspective

Recent decades have witnessed a trend that control-theoretical techniques are widely leveraged in various areas, e.g., design and analysis of computational models. Computational methods can be modeled as a controller and searching the equilibrium point of a dynamical system is identical to solving an algebraic equation. Thus, absorbing mature technologies in control theory and integrating it with neural dynamics models can lead to new achievements. This work makes progress along this direction by applying control-theoretical techniques to construct new recurrent neural dynamics for manipulating a perturbed nonstationary quadratic program (QP) with time-varying parameters considered. Specifically, to break the limitations of existing continuous-time models in handling nonstationary problems, a discrete recurrent neural dynamics model is proposed to robustly deal with noise. This work shows how iterative computational methods for solving nonstationary QP can be revisited, designed, and analyzed in a control framework. A modified Newton iteration model and an improved gradient-based neural dynamics are established by referring to the superior structural technology of the presented recurrent neural dynamics, where the chief breakthrough is their excellent convergence and robustness over the traditional models. Numerical experiments are conducted to show the eminence of the proposed models in solving perturbed nonstationary QP.

A Framework for Composite Layup Skill Learning and Generalizing Through Teleoperation

In this article, an impedance control-based framework for human-robot composite layup skill transfer was developed, and the human-in-the-loop mechanism was investigated to achieve human-robot skill transfer. Although there are some works on human-robot skill transfer, it is still difficult to transfer the manipulation skill to robots through teleoperation efficiently and intuitively. In this article, we developed an impedance-based control architecture of telemanipulation in task space for the human-robot skill transfer through teleoperation. This framework not only achieves human-robot skill transfer but also provides a solution to human-robot collaboration through teleoperation. The variable impedance control system enables the compliant interaction between the robot and the environment, smooth transition between different stages. Dynamic movement primitives based learning from demonstration (LfD) is employed to model the human manipulation skills, and the learned skill can be generalized to different tasks and environments, such as the different shapes of components and different orientations of components. The performance of the proposed approach is evaluated on a 7 DoF Franka Panda through the robot-assisted composite layup on different shapes and orientations of the components.

RNN for Repetitive Motion Generation of Redundant Robot Manipulators: An Orthogonal Projection-Based Scheme

For the existing repetitive motion generation (RMG) schemes for kinematic control of redundant manipulators, the position error always exists and fluctuates. This article gives an answer to this phenomenon and presents the theoretical analyses to reveal that the existing RMG schemes exist a theoretical position error related to the joint angle error. To remedy this weakness of existing solutions, an orthogonal projection RMG (OPRMG) scheme is proposed in this article by introducing an orthogonal projection method with the position error eliminated theoretically, which decouples the joint space error and Cartesian space error with joint constraints considered. The corresponding new recurrent neural networks (NRNNs) are structured by exploiting the gradient descent method with the assistance of velocity compensation with theoretical analyses provided to embody the stability and feasibility. In addition, simulation results on a fixed-based redundant manipulator, a mobile manipulator, and a multirobot system synthesized by the existing RMG schemes and the proposed one are presented to verify the superiority and precise performance of the OPRMG scheme for kinematic control of redundant manipulators. Moreover, via adjusting the coefficient, simulations on the position error and joint drift of the redundant manipulator are conducted for comparison to prove the high performance of the OPRMG scheme. To bring out the crucial point, different controllers for the redundancy resolution of redundant manipulators are compared to highlight the superiority and advantage of the proposed NRNN. This work greatly improves the existing RMG solutions in theoretically eliminating the position error and joint drift, which is of significant contributions to increasing the accuracy and efficiency of high-precision instruments in manufacturing production.

Efficient Iterative Regularization Method for Total Variation-Based Image Restoration

Total variation (TV) regularization has received much attention in image restoration applications because of its advantages in denoising and preserving details. A common approach to address TV-based image restoration is to design a specific algorithm for solving typical cost function, which consists of conventional ℓ2 fidelity term and TV regularization. In this work, a novel objective function and an efficient algorithm are proposed. Firstly, a pseudoinverse transform-based fidelity term is imposed on TV regularization, and a closely-related optimization problem is established. Then, the split Bregman framework is used to decouple the complex inverse problem into subproblems to reduce computational complexity. Finally, numerical experiments show that the proposed method can obtain satisfactory restoration results with fewer iterations. Combined with the restoration effect and efficiency, this method is superior to the competitive algorithm. Significantly, the proposed method has the advantage of a simple solving structure, which can be easily extended to other image processing applications.

Model Reference Tracking Control Solutions for a Visual Servo System Based on a Virtual State from Unknown Dynamics

This paper focuses on validating a model-free Value Iteration Reinforcement Learning (MFVI-RL) control solution on a visual servo tracking system in a comprehensive manner starting from theoretical convergence analysis to detailed hardware and software implementation. Learning is based on a virtual state representation reconstructed from input-output (I/O) system samples under nonlinear observability and unknown dynamics assumptions, while the goal is to ensure linear output reference model (ORM) tracking. Secondary, a competitive model-free Virtual State-Feedback Reference Tuning (VSFRT) is learned from the same I/O data using the same virtual state representation, demonstrating the framework’s learning capability. A model-based two degrees-of-freedom (2DOF) output feedback controller serving as a comparisons baseline is designed and tuned using an identified system model. With similar complexity and linear controller structure, MFVI-RL is shown to be superior, confirming that the model-based design issue of poor identified system model and control performance degradation can be solved in a direct data-driven style. Apart from establishing a formal connection between output feedback control, state feedback control and also between classical control and artificial intelligence methods, the results also point out several practical trade-offs, such as I/O data exploration quality and control performance leverage with data volume, control goal and controller complexity.

Soft Array Surface-Changing Compound Eye

The field-of-view (FOV) of compound eyes is an important index for performance evaluation. Most artificial compound eyes are optical, fabricated by imitating insect compound eyes with a fixed FOV that is difficult to adjust over a wide range. The compound eye is of great significance in the field of tracking high-speed moving objects. However, the tracking ability of a compound eye is often limited by its own FOV size and the reaction speed of the rudder unit matched with the compound eye, so that the compound eye cannot better adapt to tracking high-speed moving objects. Inspired by the eyes of many organisms, we propose a soft-array, surface-changing compound eye (SASCE). Taking soft aerodynamic models (SAM) as the carrier and an infrared sensor as the load, the basic model of the variable structure infrared compound eye (VSICE) is established using an array of infrared sensors on the carrier. The VSICE model is driven by air pressure to change the array surface of the infrared sensor. Then, the spatial position of each sensor and its viewing area are changed and, finally, the FOV of the compound eye is changed. Simultaneously, to validate the theory, we measured the air pressure, spatial sensor position, and the FOV of the compound eye. When compared with the current compound eye, the proposed one has a wider adjustable FOV.

Visual Navigation With Multiple Goals Based on Deep Reinforcement Learning

Learning to adapt to a series of different goals in visual navigation is challenging. In this work, we present a model-embedded actor-critic architecture for the multigoal visual navigation task. To enhance the task cooperation in multigoal learning, we introduce two new designs to the reinforcement learning scheme: inverse dynamics model (InvDM) and multigoal colearning (MgCl). Specifically, InvDM is proposed to capture the navigation-relevant association between state and goal and provide additional training signals to relieve the sparse reward issue. MgCl aims at improving the sample efficiency and supports the agent to learn from unintentional positive experiences. Besides, to further improve the scene generalization capability of the agent, we present an enhanced navigation model that consists of two self-supervised auxiliary task modules. The first module, which is named path closed-loop detection, helps to understand whether the state has been experienced. The second one, namely the state-target matching module, tries to figure out the difference between state and goal. Extensive results on the interactive platform AI2-THOR demonstrate that the agent trained with the proposed method converges faster than state-of-the-art methods while owning good generalization capability. The video demonstration is available at https://vsislab.github.io/mgvn.

Adaptive Critic Learning-Based Robust Control of Systems with Uncertain Dynamics

Model uncertainties are usually unavoidable in the control systems, which are caused by imperfect system modeling, disturbances, and nonsmooth dynamics. This paper presents a novel method to address the robust control problem for uncertain systems. The original robust control problem of the uncertain system is first transformed into an optimal control of nominal system via selecting the appropriate cost function. Then, we develop an adaptive critic leaning algorithm to learn online the optimal control solution, where only the critic neural network (NN) is used, and the actor NN widely used in the existing methods is removed. Finally, the feasibility analysis of the control algorithm is given in the paper. Simulation results are given to show the availability of the presented control method.

Hierarchical Cognitive Control for Unknown Dynamic Systems Tracking

A general control system tracking learning framework is proposed, by which an optimal learned tracking behavior called ‘primitive’ is extrapolated to new unseen trajectories without requiring relearning. This is considered intelligent behavior and strongly related to the neuro-motor cognitive control of biological (human-like) systems that deliver suboptimal executions for tasks outside of their current knowledge base, by using previously memorized experience. However, biological systems do not solve explicit mathematical equations for solving learning and prediction tasks. This stimulates the proposed hierarchical cognitive-like learning framework, based on state-of-the-art model-free control: (1) at the low-level L1, an approximated iterative Value Iteration for linearizing the closed-loop system (CLS) behavior by a linear reference model output tracking is first employed; (2) an experiment-driven Iterative Learning Control (EDILC) applied to the CLS from the reference input to the controlled output learns simple tracking tasks called ‘primitives’ in the secondary L2 level, and (3) the tertiary level L3 extrapolates the primitives’ optimal tracking behavior to new tracking tasks without trial-based relearning. The learning framework relies only on input-output system data to build a virtual state space representation of the underlying controlled system that is assumed to be observable. It has been shown to be effective by experimental validation on a representative, coupled, nonlinear, multivariable real-world system. Able to cope with new unseen scenarios in an optimal fashion, the hierarchical learning framework is an advance toward cognitive control systems.