Learning from demonstration: Teaching a myoelectric prosthesis with an intact limb via reinforcement learning

Deep Reinforcement Learning for Physics-Based Musculoskeletal Simulations of Healthy Subjects and Transfemoral Prostheses' Users During Normal Walking

This paper proposes to use deep reinforcement learning for the simulation of physics-based musculoskeletal models of both healthy subjects and transfemoral prostheses' users during normal level-ground walking. The deep reinforcement learning algorithm is based on the proximal policy optimization approach in combination with imitation learning to guarantee a natural walking gait while reducing the computational time of the training. Firstly, the optimization algorithm is implemented for the OpenSim model of a healthy subject and validated with experimental data from a public data-set. Afterwards, the optimization algorithm is implemented for the OpenSim model of a generic transfemoral prosthesis' user, which has been obtained by reducing the number of muscles around the knee and ankle joints and, specifically, by keeping only the uniarticular ones. The model of the transfemoral prosthesis' user shows a stable gait, with a forward dynamic comparable to the healthy subject's, yet using higher muscles' forces. Even though the computed muscles' forces could not be directly used as control inputs for muscle-like linear actuators due to their pattern, this study paves the way for using deep reinforcement learning for the design of the control architecture of transfemoral prostheses.

Supporting Play by Applying Haptic Guidance Along a Surface Learnt from Single Motion Trajectories

Haptic-enabled teleoperated robots can help children with physical disabilities to reach toys by applying haptic guidance towards their toys, thus compensating for their limitations in reaching and manipulating objects. In this article we preliminarily tested a learning from demonstration (LfD) approach, where a robotic system learnt the surface that best approximated to all motion trajectories demonstrated by the participants while playing a whack-a-mole game. The end-goal of the system is for therapists or parents to demonstrate to it how to play a game, and then be used by children with physical disabilities. In this study, four adults without disabilities participated, to identify aspects that will be necessary to improve before conducting trials with children. During the demonstration phase, participants played the game in normal teleoperation, assuming the role of the therapist/parent. Then, the surface was modeled using a neural network. Participants played the game without and with the haptic guidance. The movements of the robotic system were mirrored to induce errors in movements, and thus require the guidance. Participants spent more time, moved the robot longer distances, and had jerkier movements when they played the game with the guidance than without it. Possible reasons were discussed, and several solutions were proposed to improve the system. The main contribution of this paper was the learning of a surface instead of learning a single motion trajectory.

A High-Level Control Algorithm Based on sEMG Signalling for an Elbow Joint SMA Exoskeleton

A high-level control algorithm capable of generating position and torque references from surface electromyography signals (sEMG) was designed. It was applied to a shape memory alloy (SMA)-actuated exoskeleton used in active rehabilitation therapies for elbow joints. The sEMG signals are filtered and normalized according to data collected online during the first seconds of a therapy session. The control algorithm uses the sEMG signals to promote active participation of patients during the therapy session. In order to generate the reference position pattern with good precision, the sEMG normalized signal is compared with a pressure sensor signal to detect the intention of each movement. The algorithm was tested in simulations and with healthy people for control of an elbow exoskeleton in flexion–extension movements. The results indicate that sEMG signals from elbow muscles, in combination with pressure sensors that measure arm–exoskeleton interaction, can be used as inputs for the control algorithm, which adapts the reference for exoskeleton movements according to a patient’s intention.