Classification of finger extension and flexion of EMG and Cyberglove data with modified ICA weight matrix

Review of sEMG for Exoskeleton Robots: Motion Intention Recognition Techniques and Applications

The global aging trend is becoming increasingly severe, and the demand for life assistance and medical rehabilitation for frail and disabled elderly people is growing. As the best solution for assisting limb movement, guiding limb rehabilitation, and enhancing limb strength, exoskeleton robots are becoming the focus of attention from all walks of life. This paper reviews the progress of research on upper limb exoskeleton robots, sEMG technology, and intention recognition technology. It analyzes the literature using keyword clustering analysis and comprehensively discusses the application of sEMG technology, deep learning methods, and machine learning methods in the process of human movement intention recognition by exoskeleton robots. It is proposed that the focus of current research is to find algorithms with strong adaptability and high classification accuracy. Finally, traditional machine learning and deep learning algorithms are discussed, and future research directions are proposed, such as using a deep learning algorithm based on multi-information fusion to fuse EEG signals, electromyographic signals, and basic reference signals. A model with stronger generalization ability is obtained after training, thereby improving the accuracy of human movement intention recognition based on sEMG technology, which provides important support for the realization of human-machine fusion-embodied intelligence of exoskeleton robots.

sEMG-Based Gesture Recognition via Multi-Feature Fusion Network

The sparse surface electromyography-based gesture recognition suffers from the problems of feature information not richness and poor generalization to small sample data. Therefore, a multi-feature fusion network (MFF-Net) model is proposed in this paper. This network incorporates long short-term memory (LSTM) and the attention mechanism into the model, and three sub-networks are constructed for enhancement of features in the time, frequency and spatial domains, respectively. The introduced attention mechanism enhances useful features and weakens useless ones. Then, the processed features are spliced and stacked, which strengthens the information between time and channel to enrich features in sparse sEMG, improved model performance for feature processing. To further validate that the proposed model is effective in improving gesture recognition accuracy. We selected 18 gesture recognition tasks from the NinaPro DB3 and DB7 datasets for experimental evaluation. Among them, ablation experiments were conducted on intact subjects data in DB7. The experimental results show that the proposed model reaches the current optimal in gesture recognition, with 92.47% classification accuracy. Moreover, the model can be transferred to gesture recognition for small sample amputees data, which is also effective in solving insufficient data problem. Two amputees (in DB7) recognition rate have significantly improved from 60.35% to 84.93%, while eleven amputees (in DB3) recognition rate have significantly improved from 71.84% to 82.00%. It is demonstrated the applicability and generalization of the proposed model transfer learning to the amputees gesture recognition task.

Validity and Impact of Methods for Collecting Training Data for Myoelectric Prosthetic Control Algorithms

Intuitive regression control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time regression performance, but accurately labeling intended hand kinematics after hand amputation is challenging. In this study, we quantified the accuracy and precision of labeling hand kinematics using two common training paradigms: 1) mimic training, where participants mimic predetermined motions of a prosthesis, and 2) mirror training, where participants mirror their contralateral intact hand during synchronized bilateral movements. We first explored this question in healthy non-amputee individuals where the ground-truth kinematics could be readily determined using motion capture. Kinematic data showed that mimic training fails to account for biomechanical coupling and temporal changes in hand posture. Additionally, mirror training exhibited significantly higher accuracy and precision in labeling hand kinematics. These findings suggest that the mirror training approach generates a more faithful, albeit more complex, dataset. Accordingly, mirror training resulted in significantly better offline regression performance when using a large amount of training data and a non-linear neural network. Next, we explored these different training paradigms online, with a cohort of unilateral transradial amputees actively controlling a prosthesis in real-time to complete a functional task. Overall, we found that mirror training resulted in significantly faster task completion speeds and similar subjective workload. These results demonstrate that mirror training can potentially provide more dexterous control through the utilization of task-specific, user-selected training data. Consequently, these findings serve as a valuable guide for the next generation of myoelectric and neuroprostheses leveraging machine learning to provide more dexterous and intuitive control.

A Multilayer and Multimodal-Fusion Architecture for Simultaneous Recognition of Endovascular Manipulations and Assessment of Technical Skills

The clinical success of the percutaneous coronary intervention (PCI) is highly dependent on endovascular manipulation skills and dexterous manipulation strategies of interventionalists. However, the analysis of endovascular manipulations and related discussion for technical skill assessment are limited. In this study, a multilayer and multimodal-fusion architecture is proposed to recognize six typical endovascular manipulations. The synchronously acquired multimodal motion signals from ten subjects are used as the inputs of the architecture independently. Six classification-based and two rule-based fusion algorithms are evaluated for performance comparisons. The recognition metrics under the determined architecture are further used to assess technical skills. The experimental results indicate that the proposed architecture can achieve the overall accuracy of 96.41%, much higher than that of a single-layer recognition architecture (92.85%). In addition, the multimodal fusion brings significant performance improvement in comparison with single-modal schemes. Furthermore, the K -means-based skill assessment can obtain an accuracy of 95% to cluster the attempts made by different skill-level groups. These hopeful results indicate the great possibility of the architecture to facilitate clinical skill assessment and skill learning.

A Knitted Sensing Glove for Human Hand Postures Pattern Recognition

In recent years, flexible sensors for data gloves have been developed that aim to achieve excellent wearability, but they are associated with difficulties due to the complicated manufacturing and embedding into the glove. This study proposes a knitted glove integrated with strain sensors for pattern recognition of hand postures. The proposed sensing glove is fabricated at all once by a knitting technique without sewing and bonding, which is composed of strain sensors knitted with conductive yarn and a glove body with non-conductive yarn. To verify the performance of the developed glove, electrical resistance variations were measured according to the flexed angle and speed. These data showed different values depending on the speed or angle of movements. We carried out experiments on hand postures pattern recognition for the practicability verification of the knitted sensing glove. For this purpose, 10 able-bodied subjects participated in the recognition experiments on 10 target hand postures. The average classification accuracy of 10 subjects reached 94.17% when their own data were used. The accuracy of up to 97.1% was achieved in the case of grasp posture among 10 target postures. When all mixed data from 10 subjects were utilized for pattern recognition, the average classification expressed by the confusion matrix arrived at 89.5%. Therefore, the comprehensive experimental results demonstrated the effectiveness of the knitted sensing gloves. In addition, it is expected to reduce the cost through a simple manufacturing process of the knitted sensing glove.

Effect on manual skills of wearing instrumented gloves during manipulation

Instrumented gloves are motion capture systems that are widely used due to the simplicity of the setup required and the absence of occlusion problems when manipulating objects. Nevertheless, the effect of their use on manipulation capabilities has not been studied to date. Therefore, the aim of this work is to quantify the effect of wearing CyberGlove instrumented gloves on these capabilities when different levels of precision are required. Thirty healthy subjects were asked to perform three standardised dexterity tests twice: bare-handed and wearing instrumented gloves. The tests were the Sollerman Hand Function Test (to evaluate capability of performing activities of daily living), the Box and Block Test (to evaluate gross motor skills) and the Purdue Pegboard Test (to evaluate fine motor skills). Scores obtained in the test evaluating fine motor skills decreased by an average of 29% when wearing gloves, while scores obtained on those evaluating gross motor skills and capability to perform activities of daily living were reduced by an average of 8% and 3%, respectively. The use of instrumented gloves to record hand kinematics is only recommended when performing tasks requiring medium and gross motor skills.

A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition

The surface electromyography (sEMG)-based gesture recognition with deep learning approach plays an increasingly important role in human-computer interaction. Existing deep learning architectures are mainly based on Convolutional Neural Network (CNN) architecture which captures spatial information of electromyogram signal. Motivated by the sequential nature of electromyogram signal, we propose an attention-based hybrid CNN and RNN (CNN-RNN) architecture to better capture temporal properties of electromyogram signal for gesture recognition problem. Moreover, we present a new sEMG image representation method based on a traditional feature vector which enables deep learning architectures to extract implicit correlations between different channels for sparse multi-channel electromyogram signal. Extensive experiments on five sEMG benchmark databases show that the proposed method outperforms all reported state-of-the-art methods on both sparse multi-channel and high-density sEMG databases. To compare with the existing works, we set the window length to 200ms for NinaProDB1 and NinaProDB2, and 150ms for BioPatRec sub-database, CapgMyo sub-database, and csl-hdemg databases. The recognition accuracies of the aforementioned benchmark databases are 87.0%, 82.2%, 94.1%, 99.7% and 94.5%, which are 9.2%, 3.5%, 1.2%, 0.2% and 5.2% higher than the state-of-the-art performance, respectively.

Wireless sEMG-Based Body-Machine Interface for Assistive Technology Devices

Assistive technology (AT) tools and appliances are being more and more widely used and developed worldwide to improve the autonomy of people living with disabilities and ease the interaction with their environment. This paper describes an intuitive and wireless surface electromyography (sEMG) based body-machine interface for AT tools. Spinal cord injuries at C5-C8 levels affect patients' arms, forearms, hands, and fingers control. Thus, using classical AT control interfaces (keypads, joysticks, etc.) is often difficult or impossible. The proposed system reads the AT users' residual functional capacities through their sEMG activity, and converts them into appropriate commands using a threshold-based control algorithm. It has proven to be suitable as a control alternative for assistive devices and has been tested with the JACO arm, an articulated assistive device of which the vocation is to help people living with upper-body disabilities in their daily life activities. The wireless prototype, the architecture of which is based on a 3-channel sEMG measurement system and a 915-MHz wireless transceiver built around a low-power microcontroller, uses low-cost off-the-shelf commercial components. The embedded controller is compared with JACO's regular joystick-based interface, using combinations of forearm, pectoral, masseter, and trapeze muscles. The measured index of performance values is 0.88, 0.51, and 0.41 bits/s, respectively, for correlation coefficients with the Fitt's model of 0.75, 0.85, and 0.67. These results demonstrate that the proposed controller offers an attractive alternative to conventional interfaces, such as joystick devices, for upper-body disabled people using ATs such as JACO.

A sparse Bayesian learning based scheme for multi-movement recognition using sEMG

This paper proposed a feature extraction scheme based on sparse representation considering the non-stationary property of surface electromyography (sEMG). Sparse Bayesian learning was introduced to extract the feature with optimal class separability to improve recognition accuracy of multi-movement patterns. The extracted feature, sparse representation coefficients (SRC), represented time-varying characteristics of sEMG effectively because of the compressibility (or weak sparsity) of the signal in some transformed domains. We investigated the effect of the proposed feature by comparing with other fourteen individual features in offline recognition. The results demonstrated the proposed feature revealed important dynamic information in the sEMG signals. The multi-feature sets formed by the SRC and other single feature yielded more superior performance on recognition accuracy, compared with the single features. The best average recognition accuracy of 94.33% was gained by using SVM classifier with the multi-feature set combining the feature SRC, Williston amplitude (WAMP), wavelength (WL) and the coefficients of the fourth order autoregressive model (ARC4) via multiple kernel learning framework. The proposed feature extraction scheme (known as SRC + WAMP + WL + ARC4) is a promising method for multi-movement recognition with high accuracy.

Three-way analysis of spectrospatial electromyography data: classification and interpretation

Classifying multivariate electromyography (EMG) data is an important problem in prosthesis control as well as in neurophysiological studies and diagnosis. With modern high-density EMG sensor technology, it is possible to capture the rich spectrospatial structure of the myoelectric activity. We hypothesize that multi-way machine learning methods can efficiently utilize this structure in classification as well as reveal interesting patterns in it. To this end, we investigate the suitability of existing three-way classification methods to EMG-based hand movement classification in spectrospatial domain, as well as extend these methods by sparsification and regularization. We propose to use Fourier-domain independent component analysis as preprocessing to improve classification and interpretability of the results. In high-density EMG experiments on hand movements across 10 subjects, three-way classification yielded higher average performance compared with state-of-the art classification based on temporal features, suggesting that the three-way analysis approach can efficiently utilize detailed spectrospatial information of high-density EMG. Phase and amplitude patterns of features selected by the classifier in finger-movement data were found to be consistent with known physiology. Thus, our approach can accurately resolve hand and finger movements on the basis of detailed spectrospatial information, and at the same time allows for physiological interpretation of the results.