Simultaneous Force Regression and Movement Classification of Fingers via Surface EMG within a Unified Bayesian Framework

Teleoperation of an Anthropomorphic Robot Hand with a Metamorphic Palm and Tunable-Stiffness Soft Fingers

Teleoperation in soft robotics can endow soft robots with the ability to perform complex tasks through human-robot interaction. In this study, we propose a teleoperated anthropomorphic soft robot hand with variable degrees of freedom (DOFs) and a metamorphic palm. The soft robot hand consists of four pneumatic-actuated fingers, which can be heated to tune stiffness. A metamorphic mechanism was actuated to morph the hand palm by servo motors. The human fingers' DOF, gesture, and muscle stiffness were collected and mapped to the soft robotic hand through the sensory feedback from surface electromyography devices on the jib. The results show that the proposed soft robot hand can generate a variety of anthropomorphic configurations and can be remotely controlled to perform complex tasks such as primitively operating the cell phone and placing the building blocks. We also show that the soft hand can grasp a target through the slit by varying the DOFs and stiffness in a trail.

Hand Force Estimation from Acoustic Myography Using Deep Wavelet Scattering Transform and Long Short-Term Memory

The ability to estimate user intention from surface electromyogram (sEMG) signals is a crucial aspect in the design of powered prosthetics. Recently, researchers have been using regression techniques to connect the user's intent, as expressed through sEMG signals, to the force applied at the fingertips in order to achieve a natural and accurate form of control. However, there are still challenges associated with processing sEMG signals that need to be overcome to allow for widespread and clinical implementation of upper limb prostheses. As a result, alternative modalities functioning as promising control signals have been proposed as source of control input rather than the sEMG, such as Acoustic Myography (AMG). In this study, six high sensitivity array microphones were used to acquire AMG signals, with custom-built 3D printed microphone housing. To tackle the challenge of extracting the relevant information from AMG signals, the Wavelet Scattering Transform (WST) was utilized. alongside a Long Short-Term Memory (LSTM) neural network model for predicting the force from the AMG. The subjects were asked to use a hand dynamometer to measure the changes in force and correlate that to the force predicted by using the AMG features. Seven subjects were recruited for data collection in this study, using hardware designed by the research team. the performance results showed that the WST-LSTM model can be robustly utilized across varying window sizes and testing schemes, to achieve average NRMSE results of approximately 8%. These pioneering results suggest that AMG signals can be utilized to reliably estimate the force levels that the muscles are applying.Clinical Relevance- This research presents a new method for controlling upper limb prostheses using Acoustic Myography (AMG) signals. A novel method mapping the AMG signals to force applied by the corresponding muscles is developed. The presented findings have the potential to lead to the development of more natural and accurate control of human-machine interfaces.

Progressive Rehabilitation Based on EMG Gesture Classification and an MPC-Driven Exoskeleton

Stroke is a leading cause of disability and death worldwide, with a prevalence of 200 millions of cases worldwide. Motor disability is presented in 80% of patients. In this context, physical rehabilitation plays a fundamental role for gradually recovery of mobility. In this work, we designed a robotic hand exoskeleton to support rehabilitation of patients after a stroke episode. The system acquires electromyographic (EMG) signals in the forearm, and automatically estimates the movement intention for five gestures. Subsequently, we developed a predictive adaptive control of the exoskeleton to compensate for three different levels of muscle fatigue during the rehabilitation therapy exercises. The proposed system could be used to assist the rehabilitation therapy of the patients by providing a repetitive, intense, and adaptive assistance.

A CW-CNN regression model-based real-time system for virtual hand control

For upper limb amputees, wearing a myoelectric prosthetic hand is the only way for them to continue normal life. Even until now, the proposal of a high-precision and natural performance real-time control system based on surface electromyography (sEMG) signals is still challenging. Researchers have proposed many strategies for motion classification or regression prediction tasks based on sEMG signals. However, most of them have been limited to offline analysis only. There are even few papers on real-time control based on deep learning models, almost all of which are about motion classification. Rare studies tried to use deep learning-based regression models in real-time control systems for multi-joint angle estimation via sEMG signals. This paper proposed a CW-CNN regression model-based real-time control system for virtual hand control. We designed an Adaptive Kalman Filter to smooth the joint angles output before sending them as control commands to control a virtual hand. Eight healthy participants were invited, and three sessions experiments were conducted on two different days for all of them. During the real-time experiment, we analyzed the joint angles estimation accuracy and computational latency. Moreover, target achievement control (TAC) test was applied to emphasize motion regression in real-time. The experimental results show that the proposed control system has high precision for 3-DOFs motion regression in simultaneously, and the system remains stable and low computational latency. In the future, the proposed real-time control system can be applied to actual prosthetic hand.

Continuous Estimation of Finger and Wrist Joint Angles Using a Muscle Synergy Based Musculoskeletal Model

Recently, many muscle synergy-based human motion prediction models and algorithms have been proposed. In this study, the muscle synergies extracted from electromyography (EMG) data were used to construct a musculoskeletal model (MSM) to predict the joint angles of the wrist, thumb, index finger, and middle finger. EMG signals were analyzed using independent component analysis to reduce signal noise and task-irrelevant artifacts. The weights of each independent component (IC) were converted into a heat map related to the motion pattern and compared with human anatomy to find a different number of ICs matching the motion pattern. Based on the properties of the MSM, non-negative matrix factorization was used to extract muscle synergies from selected ICs that represent the extensor and flexor muscle groups. The effects of these choices on the prediction accuracy was also evaluated. The performance of the model was evaluated using the correlation coefficient (CC) and normalized root-mean-square error (NRMSE). The proposed method has a higher prediction accuracy than those of traditional methods, with an average CC of 92.0% and an average NRMSE of 10.7%.

Hammerstein-Wiener Multimodel Approach for Fast and Efficient Muscle Force Estimation from EMG Signals

This paper develops a novel approach to characterise muscle force from electromyography (EMG) signals, which are the electric activities generated by muscles. Based on the nonlinear Hammerstein-Wiener model, the first part of this study outlines the estimation of different sub-models to mimic diverse force profiles. The second part fixes the appropriate sub-models of a multimodel library and computes the contribution of sub-models to estimate the desired force. Based on a pre-existing dataset, the obtained results show the effectiveness of the proposed approach to estimate muscle force from EMG signals with reasonable accuracy. The coefficient of determination ranges from 0.6568 to 0.9754 using the proposed method compared with a range of 0.5060 to 0.9329 using an artificial neural network (ANN), generating significantly different accuracy (p < 0.03). Results imply that the use of multimodel approach can improve the accuracy in proportional control of prostheses.

Dynamic Gesture Recognition Using Surface EMG Signals Based on Multi-Stream Residual Network

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

Optimal strategy of sEMG feature and measurement position for grasp force estimation

Grasp force estimation based on surface electromyography (sEMG) is essential for the dexterous control of a prosthetic hand. Nowadays, although increasing the number of sEMG measurement positions and extracting more features are common methods to increase the accuracy of grasp force estimation, it will increase the computational burden. In this paper, an approach based on analysis of variance (ANOVA) and generalized regression neural network (GRNN) for optimal measurement positions and features is proposed, with the purpose of using fewer measurement positions or features to achieve higher estimation accuracy. Firstly, we captured six channels of sEMG from subjects’ forearm and grasp force synchronously. Then, four kinds of features in time domain are extracted from each channel of sEMG. By combining different measurement position sets (MPSs) and feature set (FSs), we construct 945 data sets. These data sets are fed to GRNN to realize grasp force estimation. Normalized root mean square error (NRMS), normalized mean of absolute error (NMAE), and correlation coefficient (CC) between estimated grasp force and actual force are introduced to evaluate the performance of grasp force estimation. Finally, ANOVA and Tukey HSD testing are introduced to analyze grasp force estimation results so as to obtain the optimal measurement positions and features. We obtain the optimal MPSs for grasp force estimation when different FSs are employed, and the optimal FSs when different MPSs are utilized.

Online Grasp Force Estimation From the Transient EMG

Myoelectric upper limb prostheses are controlled using information from the electrical activity of residual muscles (i.e. the electromyogram, EMG). EMG patterns at the onset of a contraction (transient phase) have shown predictive information about upcoming grasps. However, decoding this information for the estimation of the grasp force was so far overlooked. In a previous offline study, we proved that the transient phase of the EMG indeed contains information about the grasp force and determined the best algorithm to extract this information. Here we translated those findings into an online platform to be tested with both non-amputees and amputees. The platform was tested during a pick and lift task (tri-digital grasp) with light objects (200 g - 1 kg), for which fine control of the grasp force is more important. Results show that, during this task, it is possible to estimate the target grasp force with an absolute error of 2.06 (1.32) % and 2.04 (0.49) % the maximum voluntary force for non-amputee and amputees, respectively, using information from the transient phase of the EMG. This approach would allow for a biomimetic regulation of the grasp force of a prosthetic hand. Indeed, the users could contract their muscles only once before the grasp begins with no need to modulate the grasp force for the whole duration of the grasp, as required with continuous classifiers. These results pave the way to fast, intuitive and robust myoelectric controllers of limb prostheses.

SEMG Feature Extraction Based on StockwellTransform Improves Hand MovementRecognition Accuracy

Feature extraction, as an important method for extracting useful information from surface electromyography (SEMG), can significantly improve pattern recognition accuracy. Time and frequency analysis methods have been widely used for feature extraction, but these methods analyze SEMG signals only from the time or frequency domain. Recent studies have shown that feature extraction based on time-frequency analysis methods can extract more useful information from SEMG signals. This paper proposes a novel time-frequency analysis method based on the Stockwell transform (S-transform) to improve hand movement recognition accuracy from forearm SEMG signals. First, the time-frequency analysis method, S-transform, is used for extracting a feature vector from forearm SEMG signals. Second, to reduce the amount of calculations and improve the running speed of the classifier, principal component analysis (PCA) is used for dimensionality reduction of the feature vector. Finally, an artificial neural network (ANN)-based multilayer perceptron (MLP) is used for recognizing hand movements. Experimental results show that the proposed feature extraction based on the S-transform analysis method can improve the class separability and hand movement recognition accuracy compared with wavelet transform and power spectral density methods.

SEEDS, simultaneous recordings of high-density EMG and finger joint angles during multiple hand movements

We present the SurfacE Electromyographic with hanD kinematicS (SEEDS) database. It contains electromyographic (EMG) signals and hand kinematics recorded from the forearm muscles of 25 non-disabled subjects while performing 13 different movements at normal and slow-paced speeds. EMG signals were recorded with a high-density 126-channel array centered on the extrinsic flexors of the fingers and 8 further electrodes placed on the extrinsic extensor muscles. A data-glove was used to record 18 angles from the joints of the wrist and fingers. The correct synchronisation of the data-glove and the EMG was ascertained and the resulting data were further validated by implementing a simple classification of the movements. These data can be used to test experimental hypotheses regarding EMG and hand kinematics. Our database allows for the extraction of the neural drive as well as performing electrode selection from the high-density EMG signals. Moreover, the hand kinematic signals allow the development of proportional methods of control of the hand in addition to the more traditional movement classification approaches.

Measurement(s)	muscle electrophysiology trait
Technology Type(s)	electromyography
Factor Type(s)	age • sex • dominant hand
Sample Characteristic - Organism	Homo sapiens

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.9867962

Identification of forearm skin zones with similar muscle activation patterns during activities of daily living

Background

A deeper knowledge of the activity of the forearm muscles during activities of daily living (ADL) could help to better understand the role of those muscles and allow clinicians to treat motor dysfunctions more effectively and thus improve patients’ ability to perform activities of daily living.

Methods

In this work, we recorded sEMG activity from 30 spots distributed over the skin of the whole forearm of six subjects during the performance of 21 representative simulated ADL from the Sollerman Hand Function Test. Functional principal component analysis and hierarchical cluster analysis (HCA) were used to identify forearm spots with similar muscle activation patterns.

Results

The best classification of spots with similar activity in simulated ADL consisted in seven muscular-anatomically coherent groups: (1) wrist flexion and ulnar deviation; (2) wrist flexion and radial deviation; (3) digit flexion; (4) thumb extension and abduction/adduction; (5) finger extension; (6) wrist extension and ulnar deviation; and (7) wrist extension and radial deviation.

Conclusion

The number of sEMG sensors could be reduced from 30 to 7 without losing any relevant information, using them as representative spots of the muscular activity of the forearm in simulated ADL. This may help to assess muscle function in rehabilitation while also simplifying the complexity of prosthesis control.