A convolutional neural network to identify motor units from high-density surface electromyography signals in real time

Prediction of Dexterous Finger Forces With Forearm Rotation Using Motoneuron Discharges

Motor unit (MU) discharge information obtained via electromyogram (EMG) decomposition can be used to decode dexterous multi-finger movement intention for neural-machine interfaces (NMI). However, the variation of the motor unit action potential (MUAP) shape resulted from forearm rotation leads to the decreased performance of EMG decomposition, especially under the real-time condition and then the degradation of motion decoding accuracy. The object of this study was to develop a method to realize the accurate extraction of MU discharge information across forearm pronated/supinated positions in the real-time condition for dexterous multi-finger force prediction. The FastICA-based EMG decomposition technique was used and the proposed method obtained multiple separation vectors for each MU at different forearm positions in the initialization phase. Under the real-time condition, the MU discharge information was extracted adaptively using the separation vector extracted at the nearest forearm position. As comparison, the previous method that utilized a single constant separation vector to extract MU discharges across forearm positions and the conventional method that utilized the EMG amplitude information were also performed. The results showed that the proposed method obtained a significantly better performance compared with the other two methods, manifested in a larger coefficient of determination ( [Formula: see text] and a smaller root mean squared error (RMSE) between the predicted and recorded force. Our results demonstrated the feasibility and the effectiveness of the proposed method to extract MU discharge information during forearm rotation for dexterous force prediction under the real-time conditions. Further development of the proposed method could potentially promote the application of the EMG decomposition technique for continuous dexterous motion decoding in a realistic NMI application scenario.

An early force prediction control scheme using multimodal sensing of electromyography and digit force signals

Different grasping gestures result in the change of muscular activity of the forearm muscles. Similarly, the muscular activity changes with a change in grip force while grasping the object. This change in muscular activity, measured by a technique called Electromyography (EMG) is used in the upper limb bionic devices to select the grasping gesture. Previous research studies have shown gesture classification using pattern recognition control schemes. However, the use of EMG signals for force manipulation is less focused, especially during precision grasping. In this study, an early predictive control scheme is designed for the efficient determination of grip force using EMG signals from forearm muscles and digit force signals. The optimal pattern recognition (PR) control schemes are investigated using three different inputs of two signals: EMG signals, digit force signals and a combination of EMG and digit force signals. The features extracted from EMG signals included Slope Sign Change, Willison Amplitude, Auto Regressive Coefficient and Waveform Length. The classifiers used to predict force levels are Random Forest, Gradient Boosting, Linear Discriminant Analysis, Support Vector Machines, k-nearest Neighbors and Decision Tree. The two-fold objectives of early prediction and high classification accuracy of grip force level were obtained using EMG signals and digit force signals as inputs and Random Forest as a classifier. The earliest prediction was possible at 1000 ms from the onset of the gripping of the object with a mean classification accuracy of 90 % for different grasping gestures. Using this approach to study, an early prediction will result in the determination of force level before the object is lifted from the surface. This approach will also result in better biomimetic regulation of the grip force during precision grasp, especially for a population facing vision deficiency.

Online prediction of sustained muscle force from individual motor unit activities using adaptive surface EMG decomposition

Decoding movement intentions from motor unit (MU) activities to represent neural drive information plays a central role in establishing neural interfaces, but there remains a great challenge for obtaining precise MU activities during sustained muscle contractions. In this paper, we presented an online muscle force prediction method driven by individual MU activities that were decomposed from prolonged surface electromyogram (SEMG) signals in real time. In the training stage of the proposed method, a set of separation vectors was initialized for decomposing MU activities. After transferring each decomposed MU activity into a twitch force train according to its action potential waveform, a neural network was designed and trained for predicting muscle force. In the subsequent online stage, a practical double-thread-parallel algorithm was developed. One frontend thread predicted the muscle force in real time utilizing the trained network and the other backend thread simultaneously updated the separation vectors. To assess the performance of the proposed method, SEMG signals were recorded from the abductor pollicis brevis muscles of eight subjects and the contraction force was simultaneously collected. With the update procedure in the backend thread, the force prediction performance of the proposed method was significantly improved in terms of lower root mean square deviation (RMSD) of around 10% and higher fitness (R2) of around 0.90, outperforming two conventional methods. This study provides a promising technique for real-time myoelectric applications in movement control and health.

Influence of Input Features and EMG Type on Ankle Joint Torque Prediction With Support Vector Regression

Reliable and accurate EMG-driven prediction of joint torques are instrumental in the control of wearable robotic systems. This study investigates how different EMG input features affect the machine learning algorithm-based prediction of ankle joint torque in isometric and dynamic conditions. High-density electromyography (HD-EMG) of five lower leg muscles were recorded during isometric contractions and dynamic tasks. Four datasets (HD-EMG, HD-EMG with reduced dimensionality, features extracted from HD-EMG with Convolutional Neural Network, and bipolar EMG) were created and used alone or in combination with joint kinematic information for the prediction of ankle joint torque using Support Vector Regression. The performance was evaluated under intra-session, inter-subject, and inter-session cases. All HD-EMG-derived datasets led to significantly more accurate isometric ankle torque prediction than the bipolar EMG datasets. The highest torque prediction accuracy for the dynamic tasks was achieved using bipolar EMG or HD-EMG with reduced dimensionality in combination with kinematic features. The findings of this study contribute to the knowledge allowing an informed selection of appropriate features for EMG-driven torque prediction.

Optimal Motor Unit Subset Selection for Accurate Motor Intention Decoding: Towards Dexterous Real-Time Interfacing

OBJECTIVE

Motor unit (MU) discharge timings encode human motor intentions to the finest degree. Whilst tapping into such information can bring significant gains to a range of applications, current approaches to MU decoding from surface signals do not scale well with the demands of dexterous human-machine interfacing (HMI). To optimize the forward estimation accuracy and time-efficiency of such systems, we propose the inclusion of task-wise initialization and MU subset selection.

METHODS

Offline analyses were conducted on data recorded from 11 non-disabled subjects. Task-wise decomposition was applied to identify MUs from high-density surface electromyography (HD-sEMG) pertaining to 18 wrist/forearm motor tasks. The activities of a selected subset of MUs were extracted from test data and used for forward estimation of intended motor tasks and joint kinematics. To that end, various combinations of subset selection and estimation algorithms (both regression and classification-based) were tested for a range of subset sizes.

RESULTS

The mutual information-based minimum Redundancy Maximum Relevance (mRMR-MI) criterion retained MUs with the highest predicative power. When the portion of tracked MUs was reduced down to 25%, the regression performance decreased only by 3% (R2=0.79) while classification accuracy dropped by 2.7% (accuracy = 74%) when kernel-based estimators were considered.

CONCLUSION AND SIGNIFICANCE

Careful selection of tracked MUs can optimize the efficiency of MU-driven interfacing. In particular, prioritization of MUs exhibiting strong nonlinear relationships with target motions is best leveraged by kernel-based estimators. Hence, this frees resources for more robust and adaptive MU decoding techniques to be implemented in future.

Modeling Human Suboptimal Control: A Review

This review paper provides an overview of the approaches to model neuromuscular control, focusing on methods to identify nonoptimal control strategies typical of populations with neuromuscular disorders or children. Where possible, the authors tightened the description of the methods to the mechanisms behind the underlying biomechanical and physiological rationale. They start by describing the first and most simplified approach, the reductionist approach, which splits the role of the nervous and musculoskeletal systems. Static optimization and dynamic optimization methods and electromyography-based approaches are summarized to highlight their limitations and understand (the need for) their developments over time. Then, the authors look at the more recent stochastic approach, introduced to explore the space of plausible neural solutions, thus implementing the uncontrolled manifold theory, according to which the central nervous system only controls specific motions and tasks to limit energy consumption while allowing for some degree of adaptability to perturbations. Finally, they explore the literature covering the explicit modeling of the coupling between the nervous system (acting as controller) and the musculoskeletal system (the actuator), which may be employed to overcome the split characterizing the reductionist approach.

Modelling enzyme inhibition toxicity of ionic liquid from molecular structure via convolutional neural network model

Deep learning (DL) methods further promote the development of quantitative structure-activity/property relationship (QSAR/QSPR) models by dealing with complex relationships between data. An acetylcholinesterase inhibitory toxicity model of ionic liquids (ILs) was established using a convolution neural network (CNN) combined with support vector machine (SVM), random forest (RF) and multilayer perceptron (MLP). A CNN model was proposed for feature self-learning and extraction of ILs. By comparing with the model results through feature engineering (FE), the model regression results based on the CNN model for feature extraction have been substantially improved. The results showed that all six models (FE-SVM, FE-RF, FE-MLP, CNN-SVM, CNN-RF, and CNN-MLP) had good prediction accuracy, but the results based on the CNN model were better. The hyperparameters of six models were optimized by grid search and the 10-fold cross validation. Compared with the existing models in the literature, the model performance has been further improved. The model could be used as an intelligent tool to guide the design or screening of low-toxicity ILs.

Manipulator Control System Based on Flexible Sensor Technology

The research on the remote control of manipulators based on flexible sensor technology is gradually extensive. In order to achieve stable, accurate, and efficient control of the manipulator, it is necessary to reasonably design the structure of the sensor with excellent tensile strength and flexibility. The acquisition of manual information by high-performance sensors is the basis of manipulator control. This paper starts with the manufacturing of materials of the flexible sensor for the manipulator, introduces the substrate, sensor, and flexible electrode materials, respectively, and summarizes the performance of different flexible sensors. From the perspective of manufacturing, it introduces their basic principles and compares their advantages and disadvantages. Then, according to the different ways of wearing, the two control methods of data glove control and surface EMG control are respectively introduced, the principle, control process, and detection accuracy are summarized, and the problems of material microstructure, reducing the cost, optimizing the circuit design and so on are emphasized in this field. Finally, the commercial application in this field is explained and the future research direction is proposed from two aspects: how to ensure real-time control and better receive the feedback signal from the manipulator.

Deep learning-based framework for real-time upper limb motion intention classification using combined bio-signals

This research study proposes a unique framework that takes input from a surface electromyogram (sEMG) and functional near-infrared spectroscopy (fNIRS) bio-signals. These signals are trained using convolutional neural networks (CNN). The framework entails a real-time neuro-machine interface to decode the human intention of upper limb motions. The bio-signals from the two modalities are recorded for eight movements simultaneously for prosthetic arm functions focusing on trans-humeral amputees. The fNIRS signals are acquired from the human motor cortex, while sEMG is recorded from the human bicep muscles. The selected classification and command generation features are the peak, minimum, and mean ΔHbO and ΔHbR values within a 2-s moving window. In the case of sEMG, wavelength, peak, and mean were extracted with a 150-ms moving window. It was found that this scheme generates eight motions with an enhanced average accuracy of 94.5%. The obtained results validate the adopted research methodology and potential for future real-time neural-machine interfaces to control prosthetic arms.

Improving simultaneous and proportional control from EMG signals based on a Two-Stage Regression Structure

A two-stage regression structure is proposed in this paper to improve simultaneous and proportional control based on electromyography (EMG) signals. Instead of considering the conventional approach with a regression model per degree of freedom (DoF), the proposed method applies a regression model per direction of each DoF: the 1st stage detects DoFs and then is used to select the direction models of the 2nd stage. By using linear regression on the data from one healthy experienced participant for 2 DoFs of his wrist, the proposed structure was evaluated offline with cross-evaluation and online with a real-time control of a cursor to hit some targets. The evaluation results showed a clear improvement of the proposed method in terms of accuracy, ability to reach the boundaries, reaction speed, accurate control and reactive control compared to the conventional approach. The potential of this structure also lies in the fact that it can use different regression methods, work for 3 DoFs and more and use the 1st stage knowledge to improve performance of the 2nd stage.

Consensus for experimental design in electromyography (CEDE) project: Single motor unit matrix

The analysis of single motor unit (SMU) activity provides the foundation from which information about the neural strategies underlying the control of muscle force can be identified, due to the one-to-one association between the action potentials generated by an alpha motor neuron and those received by the innervated muscle fibers. Such a powerful assessment has been conventionally performed with invasive electrodes (i.e., intramuscular electromyography (EMG)), however, recent advances in signal processing techniques have enabled the identification of single motor unit (SMU) activity in high-density surface electromyography (HDsEMG) recordings. This matrix, developed by the Consensus for Experimental Design in Electromyography (CEDE) project, provides recommendations for the recording and analysis of SMU activity with both invasive (needle and fine-wire EMG) and non-invasive (HDsEMG) SMU identification methods, summarizing their advantages and disadvantages when used during different testing conditions. Recommendations for the analysis and reporting of discharge rate and peripheral (i.e., muscle fiber conduction velocity) SMU properties are also provided. The results of the Delphi process to reach consensus are contained in an appendix. This matrix is intended to help researchers to collect, report, and interpret SMU data in the context of both research and clinical applications.

Toward a generalizable deep CNN for neural drive estimation across muscles and participants

Objective.High-density electromyography (HD-EMG) decomposition algorithms are used to identify individual motor unit (MU) spike trains, which collectively constitute the neural code of movements, to predict motor intent. This approach has advanced from offline to online decomposition, from isometric to dynamic contractions, leading to a wide range of neural-machine interface applications. However, current online methods need offline retraining when applied to the same muscle on a different day or to a different person, which limits their applications in a real-time neural-machine interface. We proposed a deep convolutional neural network (CNN) framework for neural drive estimation, which takes in frames of HD-EMG signals as input, extracts general spatiotemporal properties of MU action potentials, and outputs the number of spikes in each frame. The deep CNN can generalize its application without retraining to HD-EMG data recorded in separate sessions, muscles, or participants.Approach.We recorded HD-EMG signals from the vastus medialis and vastus lateralis muscles from five participants while they performed isometric contractions during two sessions separated by ∼20 months. We identified MU spike trains from HD-EMG signals using a convolutive blind source separation (BSS) method, and then used the cumulative spike train (CST) of these MUs and the HD-EMG signals to train and validate the deep CNN.Main results.On average, the correlation coefficients between CST from the BSS and that from deep CNN were0.983±0.006for leave-one-out across-sessions-and-muscles validation and0.989±0.002for leave-one-out across-participants validation. When trained with more than four datasets, the performance of deep CNN saturated at0.984±0.001for cross validations across muscles, sessions, and participants.Significance.We can conclude that the deep CNN is generalizable across the aforementioned conditions without retraining. We could potentially generate a robust deep CNN to estimate neural drive to muscles for neural-machine interfaces.

Compressed spike-triggered averaging in iterative decomposition of surface EMG

BACKGROUND AND OBJECTIVE

Analysis of motor unit activity is important for assessing and treating diseases or injuries affecting natural movement. State-of-the-art decomposition translates high-density surface electromyography (HDsEMG) into motor unit activity. However, current decomposition methods offer far from complete separation of all motor units.

METHODS

This paper proposes a peel-off approach to automatic decomposition of HDsEMG into motor unit action potential (MUAP) trains, based on the Fast Independent Component Analysis algorithm (FastICA). The novel steps include utilizing compression by means of Principal Component Analysis and spike-triggered averaging, to estimate surface MUAP distributions with less noise, which are iteratively subtracted from the HDsEMG dataset. Furthermore, motor unit spike trains are estimated by high-dimensional density-based clustering of peaks in the FastICA source output. And finally, a new reliability measure is used to discard poor motor unit estimates by comparing the variance of the FastICA source output before and after the peel-off step. The method was validated using reconstructed synthetic data at three different signal-to-noise levels and was compared to an established deflationary FastICA approach.

RESULTS

Both algorithms had very high recall and precision, over 90%, for spikes from matching motor units, referred to as matched performance. However, the peel-off algorithm correctly identified more motor units for all noise levels. When accounting for unidentified motor units, total recall was up to 33 percentage points higher; and when accounting for duplicate estimates, total precision was up to 24 percentage points higher, compared to the state-of-the-art reference. In addition, a comparison was done using experimental data where the proposed algorithm had a matched recall of 97% and precision of 85% with respect to the reference algorithm.

CONCLUSION

These results show a substantial performance increase for decomposition of simulated HDsEMG data and serve to validate the proposed approach. This performance increase is an important step towards complete decomposition and extraction of information of motor unit activity.

Segment-Wise Decomposition of Surface Electromyography to Identify Discharges Across Motor Neuron Populations

OBJECTIVE

The surface electromyography (EMG) decomposition techniques have shown promising results in neurophysiologic investigations, clinical diagnosis, and human-machine interfacing. However, current decomposition methods could only decode a limited number of motor units (MUs) because of the local convergence. The number of identified MUs remains similar even though more muscles or movements are involved, where multiple motor neuron populations are activated. The objective of this study was to develop a segment-wise decomposition strategy to increase the number of MU decoded from multiple motor neuron populations.

METHODS

The EMG signals were divided into several segments depending on the number of involved movements. The motor neurons, activated during each movement, were regarded as a population. The convolution kernel compensation (CKC) method was applied individually for each segment to decode the motor unit discharges from each motor neuron population. The MU filters were obtained in each segment and filtrated to estimate the MU spike trains (MUSTs) from the global EMG signals. The decomposition performance was validated on synthetic and experimental EMG signals.

MAIN RESULTS

From synthetic EMG signals generated by two motor neuron populations, the proposed segment-wise CKC (swCKC) decoded significantly more MUs during low and medium excitation levels, with an increased rate of 16.3% to 75.4% compared with the conventional CKC. From experimental signals recorded during ten motor tasks, 133±24 MUs with the pulse-to-noise ratio of 36.6±6.5 dB were identified for each subject by swCKC, whereas the conventional CKC identified only 43±12 MUs.

CONCLUSION AND SIGNIFICANCE

These results indicate the feasibility and superiority of the proposed swCKC to decode MU activities across motor neuron populations, extending the potential applications of EMG decomposition for neural decoding during multiple motor tasks.

Blind source separation of inspiration and expiration in respiratory sEMG signals

OBJECTIVE

Surface electromyography (sEMG) is a noninvasive option for monitoring respiratory effort in ventilated patients. However, respiratory sEMG signals are affected by crosstalk and cardiac activity. This work addresses the blind source separation (BSS) of inspiratory and expiratory electrical activity in single- or two-channel recordings. The main contribution of the presented methodology is its applicability to the addressed muscles and the number of available channels.

APPROACH

We propose a two-step procedure consisting of a single-channel cardiac artifact removal algorithm, followed by a single- or multi-channel BSS stage. First, cardiac components are removed in the wavelet domain. Subsequently, a nonnegative matrix factorization (NMF) algorithm is applied to the envelopes of the resulting wavelet bands. The NMF is initialized based on simultaneous standard pneumatic measurements of the ventilated patient.

MAIN RESULTS

The proposed estimation scheme is applied to twelve clinical datasets and simulated sEMG signals of the respiratory system. The results on the clinical datasets are validated based on expert annotations using invasive pneumatic measurements. In the simulation, three measures evaluate the separation success: The distortion and the correlation to the known ground truth and the inspiratory-to-expiratory signal power ratio. We find an improvement across all SNRs, recruitment patterns, and channel configurations. Moreover, our results indicate that the initialization strategy replaces the manual matching of sources after the BSS.

SIGNIFICANCE

The proposed separation algorithm facilitates the interpretation of respiratory sEMG signals. In crosstalk affected measurements, the developed method may help clinicians distinguish between inspiratory effort and other muscle activities using only noninvasive measurements.

Improved online decomposition of non-stationary electromyogram via signal enhancement using a neuron resonance model: a simulation study

Objective. Motor unit (MU) discharge information obtained via the online electromyogram (EMG) decomposition has shown promising prospects in multiple applications. However, the nonstationarity of EMG signals caused by the rotation (recruitment-derecruitment) of MUs and the variation of MU action potentials (MUAP) can significantly degrade the online decomposition performance. This study aimed to develop an independent component analysis-based online decomposition method that can accommodate the nonstationarity of EMG signals.Approach. The EMG nonstationarity can make the separation vectors obtained beforehand inaccurate, resulting in the reduced amplitudes of the peaks corresponding to firing events in the source signal (independent component) and then the decreased accuracy of firing events. Therefore, we utilized the FitzHugh-Nagumo (FHN) resonance model to enhance the firing peaks in the source signal in order to improve the decomposition accuracy. A two-session approach was used with the offline session to extract the separation vectors and train the FHN models. In the online session, the source signal was estimated and further processed using the FHN model before detecting the firing events in a real-time manner. The proposed method was tested on simulated EMG signals, in which MU rotation and MUAP variation were involved to mimic the nonstationarity of EMG recordings.Main results. Compared with the conventional method, the proposed method can improve the decomposition accuracy significantly (88.70% ± 4.17% vs. 92.43% ± 2.79%) by enhancing the firing peaks, and more importantly, the improvement was more prominent when the EMG signal had stronger background noises (87.00% ± 3.70% vs. 91.66% ± 2.63%).Conclusions. Our results demonstrated the effectiveness of the proposed method to utilize the FHN model to improve the online decomposition performance on the nonstationary EMG signals. Further development of our method has the potential to improve the performance of the neural decoding system that utilizes the MU discharge information and promote its application in the neural-machine interface.

fNIRS-Based Upper Limb Motion Intention Recognition Using an Artificial Neural Network for Transhumeral Amputees

Prosthetic arms are designed to assist amputated individuals in the performance of the activities of daily life. Brain machine interfaces are currently employed to enhance the accuracy as well as number of control commands for upper limb prostheses. However, the motion prediction for prosthetic arms and the rehabilitation of amputees suffering from transhumeral amputations is limited. In this paper, functional near-infrared spectroscopy (fNIRS)-based approach for the recognition of human intention for six upper limb motions is proposed. The data were extracted from the study of fifteen healthy subjects and three transhumeral amputees for elbow extension, elbow flexion, wrist pronation, wrist supination, hand open, and hand close. The fNIRS signals were acquired from the motor cortex region of the brain by the commercial NIRSport device. The acquired data samples were filtered using finite impulse response (FIR) filter. Furthermore, signal mean, signal peak and minimum values were computed as feature set. An artificial neural network (ANN) was applied to these data samples. The results show the likelihood of classifying the six arm actions with an accuracy of 78%. The attained results have not yet been reported in any identical study. These achieved fNIRS results for intention detection are promising and suggest that they can be applied for the real-time control of the transhumeral prosthesis.