Muscle Activity Map Reconstruction from High Density Surface EMG Signals With Missing Channels Using Image Inpainting and Surface Reconstruction Methods

Recurrent Neural Network Enabled Continuous Motion Estimation of Lower Limb Joints From Incomplete sEMG Signals

Decoding continuous human motion from surface electromyography (sEMG) in advance is crucial for improving the intelligence of exoskeleton robots. However, incomplete sEMG signals are prevalent on account of unstable data transmission, sensor malfunction, and electrode sheet detachment. These non-ideal factors severely compromise the accuracy of continuous motion recognition and the reliability of clinical applications. To tackle this challenge, this paper develops a multi-task parallel learning framework for continuous motion estimation with incomplete sEMG signals. Concretely, a residual network is incorporated into a recurrent neural network to integrate the information flow of hidden states and reconstruct random and consecutive missing sEMG signals. The attention mechanism is applied for redistributing the distribution of weights. A jointly optimized loss function is devised to enable training the model for simultaneously dealing with signal anomalies/absences and multi-joint continuous motion estimation. The proposed model is implemented for estimating hip, knee, and ankle joint angles of physically competent individuals and patients during diverse exercises. Experimental results indicate that the estimation root-mean-square errors with 60% missing sEMG signals steadily converges to below 5 degrees. Even with multi-channel electrode sheet shedding, our model still demonstrates cutting-edge estimation performance, errors only marginally increase 1 degree.

Biased instantaneous regional muscle activation maps: Embedded fuzzy topology and image feature analysis

The instantaneous spatial representation of electrical propagation produced by muscle contraction may introduce bias in surface electromyographical (sEMG) activation maps. Here, we described the effect of instantaneous spatial representation (sEMG segmentation) on embedded fuzzy topological polyhedrons and image features extracted from sEMG activation maps. We analyzed 73,008 topographic sEMG activation maps from seven healthy participants (age 21.4 ± 1.5 years and body mass 74.5 ± 8.5 kg) who performed submaximal isometric plantar flexions with 64 surface electrodes placed over the medial gastrocnemius muscle. Window lengths of 50, 100, 150, 250, 500, and 1,000 ms and overlap of 0, 25, 50, 75, and 90% to change sEMG map generation were tested in a factorial design (grid search). The Shannon entropy and volume of global embedded tri-dimensional geometries (polyhedron projections), and the Shannon entropy, location of the center (LoC), and image moments of maps were analyzed. The polyhedron volume increased when the overlap was <25% and >75%. Entropy decreased when the overlap was <25% and >75% and when the window length was <100 ms and >500 ms. The LoC in the x-axis, entropy, and the histogram moments of maps showed effects for overlap (p < 0.001), while the LoC in the y-axis and entropy showed effects for both overlap and window length (p < 0.001). In conclusion, the instantaneous sEMG maps are first affected by outer parameters of the overlap, followed by the length of the window. Thus, choosing the window length and overlap parameters can introduce bias in sEMG activation maps, resulting in distorted regional muscle activation.

Recognizing Missing Electromyography Signal by Data Split Reorganization Strategy and Weight-Based Multiple Neural Network Voting Method

Surface electromyography (sEMG) signals have been applied widely in prosthetic hand controlling. In the sEMG signal acquisition, wireless devices bring convenience, but also introduce signal missing due to interference or failure during data transmission. The missing signal may only last for tens of milliseconds, but have a great impact on the recognition. Researchers have employed various methods to complete missing sEMG data, but the completed signal may not totally fit the origins, and more extra calculation time will be spent. When recognizing hand gestures by sEMG from few sensors, to recognize the slightly or not serious signal missing, this study proposed a data split reorganization (DSR) strategy and a weight-based multiple neural network voting (WMV) method. To validate the proposed methods, controllable missing sEMG signals are generated artificially. Three time domain features are extracted based on non-overlapping sliding windows. The DSR is employed to make full use of the features, and then the WMV is utilized to recognize them. Nine subjects participated in the experiments, and the results indicate that the accuracy of the proposed methods is higher. For 5%, 10%, and 15% data missing ratios, the accuracy is 93.66%, 92.55%, and 91.19%, respectively. The Wilcoxon signed-rank test also demonstrates that these results are significantly superior to the situations in which the proposed methods are not applied. In the future, we will optimize the proposed methods to recognize the seriously missing sEMG signal.

High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network

High-density surface electromyography (HD-sEMG) and deep learning technology are becoming increasingly used in gesture recognition. Based on electrode grid data, information can be extracted in the form of images that are generated with instant values of multi-channel sEMG signals. In previous studies, image-based, two-dimensional convolutional neural networks (2D CNNs) have been applied in order to recognize patterns in the electrical activity of muscles from an instantaneous image. However, 2D CNNs with 2D kernels are unable to handle a sequence of images that carry information concerning how the instantaneous image evolves with time. This paper presents a 3D CNN with 3D kernels to capture both spatial and temporal structures from sequential sEMG images and investigates its performance on HD-sEMG-based gesture recognition in comparison to the 2D CNN. Extensive experiments were carried out on two benchmark datasets (i.e., CapgMyo DB-a and CSL-HDEMG). The results show that, where the same network architecture is used, 3D CNN can achieve a better performance than 2D CNN, especially for CSL-HDEMG, which contains the dynamic part of finger movement. For CapgMyo DB-a, the accuracy of 3D CNN was 1% higher than 2D CNN when the recognition window length was equal to 40 ms, and was 1.5% higher when equal to 150 ms. For CSL-HDEMG, the accuracies of 3D CNN were 15.3% and 18.6% higher than 2D CNN when the window length was equal to 40 ms and 150 ms, respectively. Furthermore, 3D CNN achieves a competitive performance in comparison to the baseline methods.

Non-invasive Decoding of the Motoneurons: A Guided Source Separation Method Based on Convolution Kernel Compensation With Clustered Initial Points

Despite the progress in understanding of neural codes, the studies of the cortico-muscular coupling still largely rely on interferential electromyographic (EMG) signal or its rectification for the assessment of motor neuron pool behavior. This assessment is non-trivial and should be used with precaution. Direct analysis of neural codes by decomposing the EMG, also known as neural decoding, is an alternative to EMG amplitude estimation. In this study, we propose a fully-deterministic hybrid surface EMG (sEMG) decomposition approach that combines the advantages of both template-based and Blind Source Separation (BSS) decomposition approaches, a.k.a. guided source separation (GSS), to identify motor unit (MU) firing patterns. We use the single-pass density-based clustering algorithm to identify possible cluster representatives in different sEMG channels. These cluster representatives are then used as initial points of modified gradient Convolution Kernel Compensation (gCKC) algorithm. Afterwards, we use the Kalman filter to reduce the noise impact and increase convergence rate of MU filter identification by gCKC. Moreover, we designed an adaptive soft-thresholding method to identify MU firing times out of estimated MU spike trains. We tested the proposed algorithm on a set of synthetic sEMG signals with known MU firing patterns. A grid of 9 × 10 monopolar surface electrodes with 5-mm inter-electrode distances in both directions was simulated. Muscle excitation was set to 10, 30, and 50%. Colored Gaussian zero-mean noise with the signal-to-noise ratio (SNR) of 10, 20, and 30 dB, respectively, was added to 16 s long sEMG signals that were sampled at 4,096 Hz. Overall, 45 simulated signals were analyzed. Our decomposition approach was compared with gCKC algorithm. Overall, in our algorithm, the average numbers of identified MUs and Rate-of-Agreement (RoA) were 16.41 ± 4.18 MUs and 84.00 ± 0.06%, respectively, whereas the gCKC identified 12.10 ± 2.32 MUs with the average RoA of 90.78 ± 0.08%. Therefore, the proposed GSS method identified more MUs than the gCKC, with comparable performance. Its performance was dependent on the signal quality but not the signal complexity at different force levels. The proposed algorithm is a promising new offline tool in clinical neurophysiology.

Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery

During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.

Detection of Multiple Innervation Zones from Multi-Channel Surface EMG Recordings with Low Signal-to-Noise Ratio Using Graph-Cut Segmentation

Knowledge of the location of muscle Innervation Zones (IZs) is important in many applications, e.g. for minimizing the quantity of injected botulinum toxin for the treatment of spasticity or for deciding on the type of episiotomy during child delivery. Surface EMG (sEMG) can be noninvasively recorded to assess physiological and morphological characteristics of contracting muscles. However, it is not often possible to record signals of high quality. Moreover, muscles could have multiple IZs, which should all be identified. We designed a fully-automatic algorithm based on the enhanced image Graph-Cut segmentation and morphological image processing methods to identify up to five IZs in 60-ms intervals of very-low to moderate quality sEMG signal detected with multi-channel electrodes (20 bipolar channels with Inter Electrode Distance (IED) of 5 mm). An anisotropic multilayered cylinder model was used to simulate 750 sEMG signals with signal-to-noise ratio ranging from -5 to 15 dB (using Gaussian noise) and in each 60-ms signal frame, 1 to 5 IZs were included. The micro- and macro- averaged performance indices were then reported for the proposed IZ detection algorithm. In the micro-averaging procedure, the number of True Positives, False Positives and False Negatives in each frame were summed up to generate cumulative measures. In the macro-averaging, on the other hand, precision and recall were calculated for each frame and their averages are used to determine F1-score. Overall, the micro (macro)-averaged sensitivity, precision and F1-score of the algorithm for IZ channel identification were 82.7% (87.5%), 92.9% (94.0%) and 87.5% (90.6%), respectively. For the correctly identified IZ locations, the average bias error was of 0.02±0.10 IED ratio. Also, the average absolute conduction velocity estimation error was 0.41±0.40 m/s for such frames. The sensitivity analysis including increasing IED and reducing interpolation coefficient for time samples was performed. Meanwhile, the effect of adding power-line interference and using other image interpolation methods on the deterioration of the performance of the proposed algorithm was investigated. The average running time of the proposed algorithm on each 60-ms sEMG frame was 25.5±8.9 (s) on an Intel dual-core 1.83 GHz CPU with 2 GB of RAM. The proposed algorithm correctly and precisely identified multiple IZs in each signal epoch in a wide range of signal quality and is thus a promising new offline tool for electrophysiological studies.