Muscle-Specific High-Density Electromyography Arrays for Hand Gesture Classification

An Information-Rich and Highly Wearable Soft Sensor System Based on Displacement Myography for Practical Hand Gesture Interfaces

Wearable sensors for hand gesture recognition have demonstrated significant potential for creating non-invasive human-machine interfaces. Nonetheless, the trade-off between wearability, practicality and performance constrains their applicability in real-world scenarios. This paper introduces MyoLog, a wearable soft sensor system that utilises forearm muscle deformations for accurate hand gesture recognition. Muscle displacements are captured using an array of magnets and tri-axis magnetometers (displacement myography), integrated into soft and flexible structures that conform to and deform with the shape of forearm muscles. The high signal-to-noise ratio and sensitivity of the sensor modules in MyoLog produce information-rich signals, enabling the detection and differentiation of a wide spectrum of hand gestures. The study used the results of 9 participants performing 44 diverse gestures with MyoLog to investigate its performance in terms of number of gestures and achieved classification accuracy. The average performance achieved by participants was 97.7%, 91.5%, and 89.1% accuracy in executing 13, 22, and 28 gestures, respectively. To demonstrate the capabilities of MyoLog in practical settings, we explored two potential applications in virtual reality training for laparoscopic surgery and prosthetic hand control. The high wearability of MyoLog without compromising the performance paves the way for more practical human-machine interactions in diverse applications.

Body Surface Potential Mapping: A Perspective on High-Density Cutaneous Electrophysiology

The electrophysiological signals recorded by cutaneous electrodes, known as body surface potentials (BSPs), are widely employed biomarkers in medical diagnosis. Despite their widespread application and success in detecting various conditions, the poor spatial resolution of traditional BSP measurements poses a limit to their diagnostic potential. Advancements in the field of bioelectronics have facilitated the creation of compact, high-quality, high-density recording arrays for cutaneous electrophysiology, allowing detailed spatial information acquisition as BSP maps (BSPMs). Currently, the design of electrode arrays for BSP mapping lacks a standardized framework, leading to customizations for each clinical study, limiting comparability, reproducibility, and transferability. This perspective proposes preliminary design guidelines, drawn from existing literature, rooted solely in the physical properties of electrophysiological signals and mathematical principles of signal processing. These guidelines aim to simplify and generalize the optimization process for electrode array design, fostering more effective and applicable clinical research. Moreover, the increased spatial information obtained from BSPMs introduces interpretation challenges. To mitigate this, two strategies are outlined: observational transformations that reconstruct signal sources for intuitive comprehension, and machine learning-driven diagnostics. BSP mapping offers significant advantages in cutaneous electrophysiology with respect to classic electrophysiological recordings and is expected to expand into broader clinical domains in the future.

Comparing online wrist and forearm EMG-based control using a rhythm game-inspired evaluation environment

Objective.The use of electromyogram (EMG) signals recorded from the wrist is emerging as a desirable input modality for human-machine interaction (HMI). Although forearm-based EMG has been used for decades in prosthetics, there has been comparatively little prior work evaluating the performance of wrist-based control, especially in online, user-in-the-loop studies. Furthermore, despite different motivating use cases for wrist-based control, research has mostly adopted legacy prosthesis control evaluation frameworks.Approach.Gaining inspiration from rhythm games and the Schmidt's law speed-accuracy tradeoff, this work proposes a new temporally constrained evaluation environment with a linearly increasing difficulty to compare the online usability of wrist and forearm EMG. Compared to the more commonly used Fitts' Law-style testing, the proposed environment may offer different insights for emerging use cases of EMG as it decouples the machine learning algorithm's performance from proportional control, is easily generalizable to different gesture sets, and enables the extraction of a wide set of usability metrics that describe a users ability to successfully accomplish a task at a certain time with different levels of induced stress.Main results.The results suggest that wrist EMG-based control is comparable to that of forearm EMG when using traditional prosthesis control gestures and can even be better when using fine finger gestures. Additionally, the results suggest that as the difficulty of the environment increased, the online metrics and their correlation to the offline metrics decreased, highlighting the importance of evaluating myoelectric control in real-time evaluations over a range of difficulties.Significance.This work provides valuable insights into the future design and evaluation of myoelectric control systems for emerging HMI applications.

Unveiling EMG semantics: a prototype-learning approach to generalizable gesture classification

Objective.Upper limb loss can profoundly impact an individual's quality of life, posing challenges to both physical capabilities and emotional well-being. To restore limb function by decoding electromyography (EMG) signals, in this paper, we present a novel deep prototype learning method for accurate and generalizable EMG-based gesture classification. Existing methods suffer from limitations in generalization across subjects due to the diverse nature of individual muscle responses, impeding seamless applicability in broader populations.Approach.By leveraging deep prototype learning, we introduce a method that goes beyond direct output prediction. Instead, it matches new EMG inputs to a set of learned prototypes and predicts the corresponding labels.Main results.This novel methodology significantly enhances the model's classification performance and generalizability by discriminating subtle differences between gestures, making it more reliable and precise in real-world applications. Our experiments on four Ninapro datasets suggest that our deep prototype learning classifier outperforms state-of-the-art methods in terms of intra-subject and inter-subject classification accuracy in gesture prediction.Significance.The results from our experiments validate the effectiveness of the proposed method and pave the way for future advancements in the field of EMG gesture classification for upper limb prosthetics.

Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement

Most wearable robots such as exoskeletons and prostheses can operate with dexterity, while wearers do not perceive them as part of their bodies. In this perspective, we contend that integrating environmental, physiological, and physical information through multi-modal fusion, incorporating human-in-the-loop control, utilizing neuromuscular interface, employing flexible electronics, and acquiring and processing human-robot information with biomechatronic chips, should all be leveraged towards building the next generation of wearable robots. These technologies could improve the embodiment of wearable robots. With optimizations in mechanical structure and clinical training, the next generation of wearable robots should better facilitate human motor and sensory reconstruction and enhancement.

Extracting Individual Muscle Drive and Activity From High-Density Surface Electromyography Signals Based on the Center of Gravity of Motor Unit

Neural interfacing has played an essential role in advancing our understanding of fundamental movement neurophysiology and the development of human-machine interface. However, direct neural interfaces from brain and nerve recording are currently limited in clinical areas for their invasiveness and high selectivity. Here, we applied the surface electromyogram (EMG) in studying the neural control of movement and proposed a new non-invasive way of extracting neural drive to individual muscles. Sixteen subjects performed isometric contractions to complete six hand tasks. High-density surface EMG signals (256 channels in total) recorded from the forearm muscles were decomposed into motor unit firing trains. The location of each decomposed motor unit was represented by its center of gravity and was put into clustering for distinct muscle regions. All the motor units in the same cluster served as a muscle-specific motor pool from which individual muscle drive could be extracted directly. Moreover, we cross-validated the self-clustered muscle regions by magnetic resonance imaging (MRI) recorded from the subjects' forearms. All motor units that fall within the MRI region are considered correctly clustered. We achieved a clustering accuracy of 95.72% ± 4.01% for all subjects. We provided a new framework for collecting experimental muscle-specific drives and generalized the way of surface electrode placement without prior knowledge of the targeting muscle architecture.

Wearable bioelectronics fabricated in situ on skins

In recent years, wearable bioelectronics has rapidly expanded for diagnosing, monitoring, and treating various pathological conditions from the skin surface. Although the devices are typically prefabricated as soft patches for general usage, there is a growing need for devices that are customized in situ to provide accurate data and precise treatment. In this perspective, the state-of-the-art in situ fabricated wearable bioelectronics are summarized, focusing primarily on Drawn-on-Skin (DoS) bioelectronics and other in situ fabrication methods. The advantages and limitations of these technologies are evaluated and potential future directions are suggested for the widespread adoption of these technologies in everyday life.

Soft Robotics Enables Neuroprosthetic Hand Design

Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.

Customizable, reconfigurable, and anatomically coordinated large-area, high-density electromyography from drawn-on-skin electrode arrays

Accurate anatomical matching for patient-specific electromyographic (EMG) mapping is crucial yet technically challenging in various medical disciplines. The fixed electrode construction of multielectrode arrays (MEAs) makes it nearly impossible to match an individual's unique muscle anatomy. This mismatch between the MEAs and target muscles leads to missing relevant muscle activity, highly redundant data, complicated electrode placement optimization, and inaccuracies in classification algorithms. Here, we present customizable and reconfigurable drawn-on-skin (DoS) MEAs as the first demonstration of high-density EMG mapping from in situ-fabricated electrodes with tunable configurations adapted to subject-specific muscle anatomy. The DoS MEAs show uniform electrical properties and can map EMG activity with high fidelity under skin deformation-induced motion, which stems from the unique and robust skin-electrode interface. They can be used to localize innervation zones (IZs), detect motor unit propagation, and capture EMG signals with consistent quality during large muscle movements. Reconfiguring the electrode arrangement of DoS MEAs to match and extend the coverage of the forearm flexors enables localization of the muscle activity and prevents missed information such as IZs. In addition, DoS MEAs customized to the specific anatomy of subjects produce highly informative data, leading to accurate finger gesture detection and prosthetic control compared with conventional technology.

Towards Robust, Adaptive and Reliable Upper-limb Motion Estimation Using Machine Learning and Deep Learning--A Survey in Myoelectric Control

To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. \textcolor{red}{Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.