Advanced Myoelectric Control for Robotic Hand-Assisted Training: Outcome from a Stroke Patient

Hand gesture recognition with deep residual network using Semg signal

OBJECTIVES

To design and develop a classifier, named Sewing Driving Training based Optimization-Deep Residual Network (SDTO_DRN) for hand gesture recognition.

METHODS

The electrical activity of forearm muscles generates the signals that can be captured with Surface Electromyography (sEMG) sensors and includes meaningful data for decoding both muscle actions and hand movement. This research develops an efficacious scheme for hand gesture recognition using SDTO_DRN. Here, signal pre-processing is done through Gaussian filtering. Thereafter, desired and appropriate features are extracted. Following that, effective features are chosen using SDTO. At last, hand gesture identification is accomplished based on DRN and this network is effectively fine-tuned by SDTO, which is a combination of Sewing Training Based Optimization (STBO) and Driving Training Based Optimization (DTBO). The datasets employed for the implementation of this work are MyoUP Dataset and putEMG: sEMG Gesture and Force Recognition Dataset.

RESULTS

The designed SDTO_DRN model has gained superior performance with magnificent results by delivering a maximum accuracy of 0.943, True Positive Rate (TPR) of 0.929, True Negative Rate (TNR) of 0.919, Positive Predictive Value (PPV) of 0.924, and Negative Predictive Value (NPV) of 0.924.

CONCLUSIONS

The hand gesture recognition using the proposed model is accurate and improves the effectiveness of the recognition.

A review on EMG/EEG based control scheme of upper limb rehabilitation robots for stroke patients

Stroke is a common worldwide health problem and a crucial contributor to gained disability. The abilities of people, who are subjected to stroke, to live independently are significantly affected since affected upper limbs' functions are essential for our daily life. This review article focuses on emerging trends in BCI-controlled rehabilitation techniques based on EMG, EEG, or EGM + EEG signals in the last few years. Working on developing rehabilitation robotics, is considered a wealthy scientific area for researchers in the last period. There is a significant advantage that the human acquires from the interaction between the machine and his body, rehabilitation for a patient's limb is very important to get the body limb recovery, and this is what is provided mostly by applying robotic devices.

Bilateral mode exoskeleton for hand rehabilitation with wireless control using 3D printing technology based on IMU sensor

This research aimed to develop an open-source exoskeleton for hand rehabilitation (EHR) device that can be controlled wirelessly in bilateral mode. This design has the advantage of being light and being controlled easily using WiFi-based wireless communication by non-paretic hands. This open-source EHR composed of two parts, namely the master and slave parts, each of which uses a mini ESP32 microcontroller, IMU sensor, and 3D printing. The mean RMSE obtained for all exoskeleton fingers is 9.04°. Since the EHR design is open source, the researchers can create and develop rehabilitation device for the therapeutic process of patients who are paralyzed or partially paralyzed independently using healthy hand.

Dynamic Analysis and Experimental Study of Lasso Transmission for Hand Rehabilitation Robot

Lasso transmission is a method for realizing long-distance flexible transmission and lightweight robots. However, there are transmission characteristic losses of velocity, force, and displacement during the motion of lasso transmission. Therefore, the analysis of transmission characteristic losses of lasso transmission has become the focus of research. For this study, at first, we developed a new flexible hand rehabilitation robot with a lasso transmission method. Second, the theoretical analysis and simulation analysis of the dynamics of the lasso transmission in the flexible hand rehabilitation robot were carried out to calculate the force, velocity, and displacement losses of the lasso transmission. Finally, the mechanism and transmission models were established for experimental studies to measure the effects of different curvatures and speeds on the lasso transmission torque. The experimental data and image analysis results show torque loss in the process of lasso transmission and an increase in torque loss with the increase in the lasso curvature radius and transmission speed. The study of the lasso transmission characteristics is important for the design and control of hand functional rehabilitation robots, providing an important reference for the design of flexible rehabilitation robots and also guiding the research on the lasso regarding the compensation method for transmission losses.

Short and long-term effects of robot-assisted therapy on upper limb motor function and activity of daily living in patients post-stroke: a meta-analysis of randomized controlled trials

Objective

To investigate the effect of robot-assisted therapy (RAT) on upper limb motor control and activity function in poststroke patients compared with that of non-robotic therapy.

Methods

We searched PubMed, EMBASE, Cochrane Library, Google Scholar and Scopus. Randomized controlled trials published from 2010 to nowadays comparing the effect of RAT and control treatment on upper limb function of poststroke patients aged 18 or older were included. Researchers extracted all relevant data from the included studies, assessed the heterogeneity with inconsistency statistics (I² statistics), evaluated the risk of bias of individual studies and performed data analysis.

Result

Forty-six studies were included. Meta-analysis showed that the outcome of the Fugl-Meyer Upper Extremity assessment (FM-UE) (SMD = 0.20, P = 0.001) and activity function post intervention was significantly higher (SMD = 0.32, P < 0.001) in the RAT group than in the control group. Differences in outcomes of the FM-UE and activity function between the RAT group and control group were observed at the end of treatment and were not found at the follow-up. Additionally, the outcomes of the FM-UE (SMD = 0.15, P = 0.005) and activity function (SMD = 0.32, P = 0.002) were significantly different between the RAT and control groups only with a total training time of more than 15 h. Moreover, the differences in outcomes of FM-UE and activity post intervention were not significant when the arm robots were applied to patients with severe impairments (FM-UE: SMD = 0.14, P = 0.08; activity: SMD = 0.21, P = 0.06) or when patients were provided with patient-passive training (FM-UE: SMD = − 0.09, P = 0.85; activity: SMD = 0.70, P = 0.16).

Conclusion

RAT has the significant immediate benefits for motor control and activity function of hemiparetic upper limb in patients after stroke compared with controls, but there is no evidence to support its long-term additional benefits. The superiority of RAT in improving motor control and activity function is limited by the amount of training time and the patients' active participation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01058-8.

Individuals with moderate to severe hand impairments may struggle to use EMG control for assistive devices

Neurological trauma, such as stroke, traumatic brain injury (TBI), spinal cord injury, and cerebral palsy can cause mild to severe upper limb impairments. Hand impairment makes it difficult for individuals to complete activities of daily living, especially bimanual tasks. A robotic hand orthosis or hand exoskeleton can be used to restore partial function of an intact but impaired hand. It is common for upper extremity prostheses and orthoses to use electromyography (EMG) sensing as a method for the user to control their device. However some individuals with an intact but impaired hand may struggle to use a myoelectrically controlled device due to potentially confounding muscle activity. This study was conducted to evaluate the application of conventional EMG control techniques as a robotic orthosis/exoskeleton user input method for individuals with mild to severe hand impairments. Nine impaired subjects and ten healthy subjects were asked to perform repeated contractions of muscles in their forearm and then onset analysis and feature classification were used to determine the accuracy of the employed EMG techniques. The average accuracy for contraction identification across employed EMG techniques was 95.4% ± 4.9 for the healthy subjects and 73.9% ± 13.1 for the impaired subjects with a range of 47.0% ± 19.1 - 91.6% ± 8.5. These preliminary results suggest that the conventional EMG control technologies employed in this paper may be difficult for some impaired individuals to use due to their unreliable muscle control.

Hand Rehabilitation Devices: A Comprehensive Systematic Review

A cerebrovascular accident, or a stroke, can cause significant neurological damage, inflicting the patient with loss of motor function in their hands. Standard rehabilitation therapy for the hand increases demands on clinics, creating an avenue for powered hand rehabilitation devices. Hand rehabilitation devices (HRDs) are devices designed to provide the hand with passive, active, and active-assisted rehabilitation therapy; however, HRDs do not have any standards in terms of development or design. Although the categorization of an injury’s severity can guide a patient into seeking proper assistance, rehabilitation devices do not have a set standard to provide a solution from the beginning to the end stages of recovery. In this paper, HRDs are defined and compared by their mechanical designs, actuation mechanisms, control systems, and therapeutic strategies. Furthermore, devices with conducted clinical trials are used to determine the future development of HRDs. After evaluating the abilities of 35 devices, it is inferred that standard characteristics for HRDs should include an exoskeleton design, the incorporation of challenge-based and coaching therapeutic strategies, and the implementation of surface electromyogram signals (sEMG) based control.

Pattern Recognition of EMG Signals by Machine Learning for the Control of a Manipulator Robot

Human Machine Interfaces (HMI) principles are for the development of interfaces for assistance or support systems in physiotherapy or rehabilitation processes. One of the main problems is the degree of customization when applying some rehabilitation therapy or when adapting an assistance system to the individual characteristics of the users. To solve this inconvenience, it is proposed to implement a database of surface Electromyography (sEMG) of a channel in healthy individuals for pattern recognition through Neural Networks of contraction in the muscular region of the biceps brachii. Each movement is labeled using the One-Hot Encoding technique, which activates a state machine to control the position of an anthropomorphic manipulator robot and validate the response time of the designed HMI. Preliminary results show that the learning curve decreases when customizing the interface. The developed system uses muscle contraction to direct the position of the end effector of a virtual robot. The classification of Electromyography (EMG) signals is obtained to generate trajectories in real time by designing a test platform in LabVIEW.

Hand Exoskeleton Development Based on Voice Recognition Using Embedded Machine Learning on Raspberry Pi

The choice of using speech to control the exoskeleton is based on the number of exoskeletons that are controlled using the EMG signal, where the EMG signal itself has the weakness of the complexity of the signal which is influenced by the position of the electrodes, as well as muscle fatigue. The purpose of this research is to develop an exoskeleton device using voice control based on embedded machine learning on a Raspberry Pi minicomputer. In this study, two feature extraction types namely mel-frequency cepstral coefficient (MFCC) and zero-crossing (ZC), and two machine learning algorithms, namely K-nearest Neighbor (K-NN) and Decision Tree (DT) were evaluated. The hand exoskeleton development consists of 3D hand design, microphone, Raspberry Pi 4B+, PCA9685 servo driver, and servo motor. Microphone was used to record voice commands given. After model evaluation, it was found that the MFCC extraction combined with the K-NN algorithm and the best accuracy (96±7.0%). In the implementation, we found that the accuracy is 79±14.46% and 90±14.14% for open and close commands.

A Review: Sensory System, Data Processing, Actuator Type on a Hand Exoskeleton Design

A rehabilitation device for a post-stroke is essential because stroke attacks can cause disable to part or half of the human body. An exoskeleton could be a vital device for rehabilitation for a post-stroke patient. Several studies have proposed the exoskeleton design for rehabilitation purposes to a human limb disorder. This study aims to review the state-of-the-art of hand exoskeleton devices based on myoelectric or any other sensors. This paper is expected to contribute to design a hand exoskeleton device using both myoelectric and force sensors. This was achieved by reviewing several articles related to the development of the exoskeleton, especially in the sensor system, data processing, and actuator system. The results show that the use of Ag electrode disposable Ag (AgCl) is still commonly found to detect the movement of the fingers on the hand because this sensor can reduce the artifact noise. The use of myo-armband is also found in several studies because it has wireless properties so that it is easy to use. In terms of processors, Arduino microcontrollers are more widely used than others. In order to activate the hand exoskeleton, servo motors are more widely used to actuate the finger joints, which is more precise than other actuators. In a further development, integration between exoskeleton systems and information systems will be an expected challenge. Furthermore, hopefully, the development of this exoskeleton can be applied as a rehabilitation device for patients with malfunction or hand paralysis.

Introduction of a sEMG Sensor System for Autonomous Use by Inexperienced Users

Wearable devices play an increasing role in the rehabilitation of patients with movement disorders. Although information about muscular activation is highly interesting, no approach exists that allows reliable collection of this information when the sensor is applied autonomously by the patient. This paper aims to demonstrate the proof-of-principle of an innovative sEMG sensor system, which can be used intuitively by patients while detecting their muscular activation with sufficient accuracy. The sEMG sensor system utilizes a multichannel approach based on 16 sEMG leads arranged circularly around the limb. Its design enables a stable contact between the skin surface and the system's dry electrodes, fulfills the SENIAM recommendations regarding the electrode size and inter-electrode distance and facilitates a high temporal resolution. The proof-of-principle was demonstrated by elbow flexion/extension movements of 10 subjects, proving that it has root mean square values and a signal-to-noise ratio comparable to commercial systems based on pre-gelled electrodes. Furthermore, it can be easily placed and removed by patients with reduced arm function and without detailed knowledge about the exact positioning of the sEMG electrodes. With its features, the demonstration of the sEMG sensor system's proof-of-principle positions it as a wearable device that has the potential to monitor muscular activation in home and community settings.

Investigating the performance of an amplitude-independent algorithm for detecting the hand muscle activity of stroke survivors

To make robotic hand devices controlled by surface electromyography (sEMG) signals feasible and practical tools for assisting patients with hand impairments, the problems that prevent these devices from being widely used have to be overcome. The most significant problem is the involuntary amplitude variation of the sEMG signals due to the movement of electrodes during forearm motion. Moreover, for patients who have had a stroke or another neurological disease, the muscle activity of the impaired hand is weak and has a low signal-to-noise ratio (SNR). Thus, muscle activity detection methods intended for controlling robotic hand devices should not depend mainly on the amplitude characteristics of the sEMG signal in the detection process, and they need to be more reliable for sEMG signals that have a low SNR. Since amplitude-independent muscle activity detection methods meet these requirements, this paper investigates the performance of such a method on people who have had a stroke in terms of the detection of weak muscle activity and resistance to false alarms caused by the involuntary amplitude variation of sEMG signals; these two parameters are very important for achieving the reliable control of robotic hand devices intended for people with disabilities. A comparison between the performance of an amplitude-independent muscle activity detection algorithm and three amplitude-dependent algorithms was conducted by using sEMG signals recorded from six hemiparesis stroke survivors and from six healthy subjects. The results showed that the amplitude-independent algorithm performed better in terms of detecting weak muscle activity and resisting false alarms.

Alternative Motor Task-Based Pattern Training With a Digital Mirror Therapy System Enhances Sensorimotor Signal Rhythms Post-stroke

Mirror therapy (MT) facilitates motor learning and induces cortical reorganization and motor recovery from stroke. We applied the new digital mirror therapy (DMT) system to compare the cortical activation under the three visual feedback conditions: (1) no mirror visual feedback (NoMVF), (2) bilateral synchronized task-based mirror visual feedback training (BMVF), and (3) reciprocal task-based mirror visual feedback training (RMVF). During DMT, EEG recordings, including time-dependent event-related desynchronization (ERD) signal amplitude in both mu and beta bands, were obtained from the standard C3 (ispilesional hemisphere, IH), C4 (contralesional hemisphere, CH), and Cz scalp sites (supplementary motor area, SMA). The entire ERD curve was separated into three time-phases: P0 (-2 to 0 s), P1 (0 to 2 s), and P2 (2 to 4 s). Four-way and subsequent repeated-measures analyses of variance were used to examine the effects of group (stroke vs. control group), test condition (NoMVF, BMVF, and RMVF), time-phase (P0, P1, and P2), and brain area (IH, CH, SMA) on the ERD areas (%) in mu and beta bands. For the mu band, generally, ERD areas (%) were larger in the control than in the stroke group. The ERD areas (%) were largest under the RMVF condition, followed by BMVF and NoMVF conditions. Similar results were found in the beta bands. The main effects of group, time-phase, and test condition on the ERD areas (%) were significant for the three brain areas, except the main effect of group in the SMA (Cz) and CH (C4) brain area. The ERD areas (%) were larger in the control than in the stroke group. The ERD area (%) was significantly larger during P1 than during P0 and P2 (ps < 0.02), and during P2 than during P0 (ps < 0.01). The ERD area (%) under the RMVF condition was significantly larger than that under the BMVF condition and NoMVF condition (ps < 0.05). The present study suggests that cortical activation particularly in the SMA (Cz) of the brain increases in the RMVF condition in both healthy subjects and stroke patients. This result supports the hypothesis that stroke patients may benefit from RMVF training.

Characterizing Intent Changes in Exoskeleton-Assisted Walking Through Onboard Sensors

Robotic exoskeletons are a promising technology for rehabilitation and locomotion following musculoskeletal injury, but their adoption outside the physical therapy clinic has been limited by relatively primitive methods for identifying and incorporating the user's gait intentions. Various intent detection approaches have been demonstrated using electromyography and electroencephalography signals. These technologies sense the human directly but introduce complications for donning/doffing the device and in measurement consistency. By contrast, sensors onboard the exoskeleton avoid these complications but sense the human indirectly via the human-robot interface. This pilot study examines if onboard sensors alone may enable identification of user intent. Joint positions and commanded motor currents are compared prior to and after changes in the user's intended gait speed. Preliminary experimental results confirm that these measures are significantly different following intent changes for both able-bodied and non-able-bodied users. The findings suggest that intent detection is possible with onboard sensors alone, but the intent signals depend on exoskeleton control settings, user ability, and temporal considerations.

Multi-connection load compensation and load information calculation for an upper-limb exoskeleton based on a six-axis force/torque sensor

In this article, a method of multi-connection load compensation and load information calculation for an upper-limb exoskeleton is proposed based on a six-axis force/torque sensor installed between the exoskeleton and the end effector. The proposed load compensation method uses a mounted sensor to measure the force and torque between the exoskeleton and load of different connections and adds a compensator to the controller to compensate the component caused by the load in the human–robot interaction force, so that the human–robot interaction force is only used to operate the exoskeleton. Therefore, the operator can manipulate the exoskeleton with the same interaction force to lift loads of different weights with a passive or fixed connection, and the human–robot interaction force is minimized. Moreover, the proposed load information calculation method can calculate the weight of the load and the position of its center of gravity relative to the exoskeleton and end effector accurately, which is necessary for acquiring the upper-limb exoskeleton center of gravity and stability control of whole-body exoskeleton. In order to verify the effectiveness of the proposed method, we performed load handling and operational stability experiments. The experimental results showed that the proposed method realized the expected function.

Motor Unit Properties of the First Dorsal Interosseous in Chronic Stroke Subjects: Concentric Needle and Single Fiber EMG Analysis

The purpose of this study was to better understand changes in motor unit electrophysiological properties in people with chronic stroke based on concentric needle electromyography (EMG) and single fiber EMG recordings. The first dorsal interosseous (FDI) muscle was studied bilaterally in eleven hemiparetic stroke subjects. A significant increase in mean fiber density (FD) was found in the paretic muscle compared with the contralateral side based on single fiber EMG (1.6 ± 0.2 vs. 1.3 ± 0.1, respectively, P = 0.003). There was no statistically significant difference between the paretic and contralateral sides in most concentric needle motor unit action potential (MUAP) parameters, such as amplitude (768.7 ± 441.7 vs. 855.0 ± 289.9 μV), duration (8.9 ± 1.8 vs. 8.68 ± 0.9 ms) and size index (1.2 ± 0.5 vs. 1.1 ± 0.3) (P > 0.18), nor was there a significant difference in single fiber EMG recorded jitter (37.0 ± 9.6 vs. 39.9 ± 10.6 μs, P = 0.45). The increase in FD suggests motor units of the paretic FDI have enlarged due to collateral reinnervation. However, sprouting might be insufficient to result in a statistically significant change in the concentric needle MUAP parameters. Single fiber EMG appears more sensitive than concentric needle EMG to reflect electrophysiological changes in motor units after stroke. Both single fiber and concentric needle EMG recordings may be necessary to better understand muscle changes after stroke, which is important for development of appropriate rehabilitation strategies. The results provide further evidence that motor units are remodeled after stroke, possibly in response to a loss of motoneurons.

Patient-Active Control of a Powered Exoskeleton Targeting Upper Limb Rehabilitation Training

Robot-assisted therapy affords effective advantages to the rehabilitation training of patients with motion impairment problems. To meet the challenge of integrating the active participation of a patient in robotic training, this study presents an admittance-based patient-active control scheme for real-time intention-driven control of a powered upper limb exoskeleton. A comprehensive overview is proposed to introduce the major mechanical structure and the real-time control system of the developed therapeutic robot, which provides seven actuated degrees of freedom and achieves the natural ranges of human arm movement. Moreover, the dynamic characteristics of the human-exoskeleton system are studied via a Lagrangian method. The patient-active control strategy consisting of an admittance module and a virtual environment module is developed to regulate the robot configurations and interaction forces during rehabilitation training. An audiovisual game-like interface is integrated into the therapeutic system to encourage the voluntary efforts of the patient and recover the neural plasticity of the brain. Further experimental investigation, involving a position tracking experiment, a free arm training experiment, and a virtual airplane-game operation experiment, is conducted with three healthy subjects and eight hemiplegic patients with different motor abilities. Experimental results validate the feasibility of the proposed scheme in providing patient-active rehabilitation training.

Myoelectric Pattern Recognition for Controlling a Robotic Hand: A Feasibility Study in Stroke

OBJECTIVE

Myoelectric pattern recognition has been successfully applied as a human-machine interface to control robotic devices such as prostheses and exoskeletons, significantly improving the dexterity of myoelectric control. This study investigates the feasibility of applying myoelectric pattern recognition for controlling a robotic hand in stroke patients.

METHODS

Myoelectric pattern recognition of six hand motion patterns was performed using forearm electromyogram signals in paretic side of eight stroke subjects. Both the random cross validation (RCV) and the chronological handout validation (CHV) were applied to assess the offline myoelectric pattern recognition performance. Experiments on real-time myoelectric pattern recognition control of an exoskeleton robotic hand were also performed.

RESULTS

An average classification accuracy of 84.1% (the mean value from two different classifiers) and individual subject differences were observed in the offline myoelectric pattern recognition analysis using the RCV, while the accuracy decreased to 65.7% when the CHV was used. The stroke subjects achieved an average accuracy of 61.3 ± 20.9% for controlling the robotic hand. However, our study did not reveal a clear correlation between the real-time control accuracy and the offline myoelectric pattern recognition performance, or any specific characteristics of the stroke subjects.

CONCLUSION

The findings suggest that it is feasible to apply myoelectric pattern recognition to control the robotic hand in some but not all of the stroke patients. Each stroke subject should be individually online tested for the feasibility of applying myoelectric pattern recognition control for robot-assisted rehabilitation.