Reinforcement learning-based shared control for walking-aid robot and its experimental verification

A Safe and Compliant Noncontact Interactive Approach for Wheeled Walking Aid Robot

Aiming at promptly and accurately detecting falls and drag-to gaits induced by asynchronous human-robot movement speed during assisted walking, a noncontact interactive approach with generality, compliance and safety is proposed in this paper, and is applied to a wheeled walking aid robot. Firstly, the structure and the functions of the wheeled walking aid robot, including gait rehabilitation robot (GRR) and walking aid robot (WAR) are illustrated, and the characteristic futures of falls and the drag-to gait are shown by experiments. To obtain gait information, a multichannel proximity sensor array is developed, and a two-dimensional gait information detection system is established by combining four proximity sensors groups which are installed in the robot chassis. Additionally, a node-iterative fuzzy Petri net algorithm for abnormal gait recognition is proposed by generating the network trigger mechanism using the fuzzy membership function. It integrates the walking intention direction vector by taking gait deviation, frequency, and torso angle as input parameters of the system. Finally, to improve the compliance of the robot during human-robot interaction, a PID_SC controller is designed by integrating the gait speed compensation, which enables the WAR to track human gait closely. Abnormal gait recognition and assisted walking experiments are carried out respectively. Experimental results show that the proposed algorithm can accurately identify abnormal gaits of different groups of users with different walking habits, and the recognition rate of abnormal gait reaches 91.2%. Results also show that the developed method can guarantee safety in human robot interaction because of user gate follow-up accuracy and compliant movements. The noncontact interactive approach can be applied to robots with similar structure for usage in walking assistance and gait rehabilitation.

Multi-fuzzy Sarsa learning-based sit-to-stand motion control for walking-support assistive robot

Sit-to-stand transfer is a very common and critical movement of daily life in elderly individuals, especially independent elderly individuals. However, most assistive robots do not have a sit-to-stand transfer function. In this article, a multi-fuzzy Sarsa learning-based sit-to-stand motion control method for walking-support assistive robot was proposed. First, the mechanical design of walking-support assistive and sit-to-stand transfer motion control problems were introduced. Then, the fuzzy Sarsa learning method, which is a model-free algorithm, was used to design the motion control algorithm for the human–robot system. To realize natural and intuitive sit-to-stand transfer movement for a human–robot system, the interactive force between the robot and human and the error position between the real-time center of mass and reference center of mass were state variables of the proposed fuzzy Sarsa learning-based sit-to-stand motion control algorithm. Considering the computing efficiency of the controller, a multi-fuzzy Sarsa learning -based motion control algorithm was developed to realize natural sit-to-stand transfer motion. Finally, the experimental results verify the effectiveness of the proposed algorithm.

Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain-Machine Interface

BACKGROUND

For the nonstationarity of neural recordings in intracortical brain-machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders.

METHODS

To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures.

RESULTS

The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times.

CONCLUSIONS

This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.

Shared Control of an Electric Wheelchair Considering Physical Functions and Driving Motivation

Individuals with severe physical impairments have difficulties operating electric wheelchairs (EWs), especially in situations where fine steering abilities are required. Automatic driving partly solves the problem, although excessive reliance on automatic driving is not conducive to maintaining their residual physical functions and may cause more serious diseases in the future. The objective of this study was to develop a shared control system that can be adapted to different environments by completely utilizing the operating ability of the user while maintaining the motivation of the user to drive. The operating characteristics of individuals with severe physical impairments were first analyzed to understand their difficulties when operating EWs. Subsequently, a novel reinforcement learning-based shared control method was proposed to adjust the control weight between the user and the machine to meet the requirements of fully exploiting the operating abilities of the users while assisting them when necessary. Experimental results showed that the proposed shared control system gradually adjusted the control weights between the user and the machine, providing safe operation of the EW while ensuring full use of the control signals from the user. It was also found that the shared control results were deeply affected by the types of users.

Continuous shared control in prosthetic hand grasp tasks by Deep Deterministic Policy Gradient with Hindsight Experience Replay

Grasp using a prosthetic hand in real life can be a difficult task. The amputee users are often capable of planning the reaching trajectory and hand grasp location selection, however, failed in precise finger movements, such as adapting the fingers to the surface of the object without excessive force. It is much efficient to leave that part to the machine autonomy. In order to combine the intention and planning ability of users with robotic control, the shared control is introduced in which users’ inputs and robot control methods are combined to achieve a goal. The shared control problem can be formulated as a Partially Observable Markov Decision Process. To find the optimal control policy, we adopt an adaptive dynamic programming and reinforcement learning-based control algorithm-Deep Deterministic Policy Gradient combined with Hindsight Experience Replay. We proposed the algorithm with a prediction layer using the reparameterization technique. The system was tested in a modified simulation environment for the ability to follow the user’s intention and keep the contact force in boundary for safety.

A Novel Coordinated Motion Fusion-Based Walking-Aid Robot System

Human locomotion is a coordinated motion between the upper and lower limbs, which should be considered in terms of both the user’s normal walking state and abnormal walking state for a walking-aid robot system. Therefore, a novel coordinated motion fusion-based walking-aid robot system was proposed. To develop the accurate human motion intention (HMI) of such robots when the user is in normal walking state, force-sensing resistor (FSR) sensors and a laser range finder (LRF) are used to detect the two HMIs expressed by the user’s upper and lower limbs. Then, a fuzzy logic control (FLC)-Kalman filter (LF)-based coordinated motion fusion algorithm is proposed to synthesize these two segmental HMIs to obtain an accurate HMI. A support vector machine (SVM)-based fall detection algorithm is used to detect whether the user is going to fall and to distinguish the user’s falling mode when he/she is in an abnormal walking state. The experimental results verify the effectiveness of the proposed algorithms.

Hierarchical Shared Control of Cane-Type Walking-Aid Robot

A hierarchical shared-control method of the walking-aid robot for both human motion intention recognition and the obstacle emergency-avoidance method based on artificial potential field (APF) is proposed in this paper. The human motion intention is obtained from the interaction force measurements of the sensory system composed of 4 force-sensing registers (FSR) and a torque sensor. Meanwhile, a laser-range finder (LRF) forward is applied to detect the obstacles and try to guide the operator based on the repulsion force calculated by artificial potential field. An obstacle emergency-avoidance method which comprises different control strategies is also assumed according to the different states of obstacles or emergency cases. To ensure the user's safety, the hierarchical shared-control method combines the intention recognition method with the obstacle emergency-avoidance method based on the distance between the walking-aid robot and the obstacles. At last, experiments validate the effectiveness of the proposed hierarchical shared-control method.

Motion control for a walking companion robot with a novel human–robot interface

A walking companion robot is presented for rehabilitation from dyskinesia of lower limbs in this article. A new human–robot interface (HRI) is designed which adopts one-axis force sensor and potentiometer connector to detect the motion of the user. To accompany in displacement and angle between the user and the robot precisely in real time, the common motions are classified into two elemental motion states. With distinction method of motion states, a classification scheme of motion control is adopted. The mathematical model-based control method is first introduced and the corresponding control systems are built. Due to the unavoidable deviation of the mathematical model-based control method, a force control method is proposed and the corresponding control systems are built. The corresponding simulations demonstrate that the efficiency of the two proposed control methods. The experimental data and paths of robot verify the two control methods and indicate that the force control method can better satisfy the user’s requirements.