Simultaneous estimation of joint angle and interaction force towards sEMG-driven human-robot interaction during constrained tasks

Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals

BACKGROUND

Simultaneous and proportional control (SPC) based on surface electromyographic (sEMG) signals has emerged as a research hotspot in the field of human-machine interaction (HMI). However, the existing continuous motion estimation methods mostly have an average Pearson coefficient (CC) of less than 0.85, while high-precision methods suffer from the problem of long inference time (> 200 ms) and can only estimate SPC of less than 15 hand movements, which limits their applications in HMI. To overcome these problems, we propose a smooth Multi-scale Attention Patching Encoder Network (sMAPEN).

METHODS

The sMAPEN consists of three modules, the Multi-scale Attention Fusion (MAF) module, the Patching Encoder (PE) module, and a smoothing layer. The MAF module adaptively captures the local spatiotemporal features at multiple scales, the PE module acquires the global spatiotemporal features of sEMG, and the smoothing layer further improves prediction stability.

RESULTS

To evaluate the performance of the model, we conducted continuous estimation of 40 subjects performing over 40 different hand movements on the Ninapro DB2. The results show that the average Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), coefficient of determination (R2), and smoothness (SMOOTH) of the sMAPEN model are 0.9082, 0.0646°, 0.8163, and - 0.0017, respectively, which significantly outperforms that of the state-of-the-art methods in all metrics (p < 0.01). Furthermore, we tested the deployment performance of sMAPEN on the portable device, with a delay of only 97.93 ms.

CONCLUSIONS

Our model can predict up to 40 hand movements while achieving the highest predicting accuracy compared with other methods. Besides, the lightweight design strategy brings an improvement in inference speed, which enables the model to be deployed on wearable devices. All these promotions imply that sMAPEN holds great potential in HMI.

Teleoperation of an Anthropomorphic Robot Hand with a Metamorphic Palm and Tunable-Stiffness Soft Fingers

Teleoperation in soft robotics can endow soft robots with the ability to perform complex tasks through human-robot interaction. In this study, we propose a teleoperated anthropomorphic soft robot hand with variable degrees of freedom (DOFs) and a metamorphic palm. The soft robot hand consists of four pneumatic-actuated fingers, which can be heated to tune stiffness. A metamorphic mechanism was actuated to morph the hand palm by servo motors. The human fingers' DOF, gesture, and muscle stiffness were collected and mapped to the soft robotic hand through the sensory feedback from surface electromyography devices on the jib. The results show that the proposed soft robot hand can generate a variety of anthropomorphic configurations and can be remotely controlled to perform complex tasks such as primitively operating the cell phone and placing the building blocks. We also show that the soft hand can grasp a target through the slit by varying the DOFs and stiffness in a trail.

Ensemble Learning Method for the Continuous Decoding of Hand Joint Angles

Human-machine interface technology is fundamentally constrained by the dexterity of motion decoding. Simultaneous and proportional control can greatly improve the flexibility and dexterity of smart prostheses. In this research, a new model using ensemble learning to solve the angle decoding problem is proposed. Ultimately, seven models for angle decoding from surface electromyography (sEMG) signals are designed. The kinematics of five angles of the metacarpophalangeal (MCP) joints are estimated using the sEMG recorded during functional tasks. The estimation performance was evaluated through the Pearson correlation coefficient (CC). In this research, the comprehensive model, which combines CatBoost and LightGBM, is the best model for this task, whose average CC value and RMSE are 0.897 and 7.09. The mean of the CC and the mean of the RMSE for all the test scenarios of the subjects' dataset outperform the results of the Gaussian process model, with significant differences. Moreover, the research proposed a whole pipeline that uses ensemble learning to build a high-performance angle decoding system for the hand motion recognition task. Researchers or engineers in this field can quickly find the most suitable ensemble learning model for angle decoding through this process, with fewer parameters and fewer training data requirements than traditional deep learning models. In conclusion, the proposed ensemble learning approach has the potential for simultaneous and proportional control (SPC) of future hand prostheses.

Estimation of Knee Extension Force Using Mechanomyography Signals Based on GRA and ICS-SVR

During lower-extremity rehabilitation training, muscle activity status needs to be monitored in real time to adjust the assisted force appropriately, but it is a challenging task to obtain muscle force noninvasively. Mechanomyography (MMG) signals offer unparalleled advantages over sEMG, reflecting the intention of human movement while being noninvasive. Therefore, in this paper, based on MMG, a combined scheme of gray relational analysis (GRA) and support vector regression optimized by an improved cuckoo search algorithm (ICS-SVR) is proposed to estimate the knee joint extension force. Firstly, the features reflecting muscle activity comprehensively, such as time-domain features, frequency-domain features, time–frequency-domain features, and nonlinear dynamics features, were extracted from MMG signals, and the relational degree was calculated using the GRA method to obtain the correlation features with high relatedness to the knee joint extension force sequence. Then, a combination of correlated features with high relational degree was input into the designed ICS-SVR model for muscle force estimation. The experimental results show that the evaluation indices of the knee joint extension force estimation obtained by the combined scheme of GRA and ICS-SVR were superior to other regression models and could estimate the muscle force with higher estimation accuracy. It is further demonstrated that the proposed scheme can meet the need of muscle force estimation required for rehabilitation devices, powered prostheses, etc.