On lightmyography based muscle-machine interfaces for the efficient decoding of human gestures and forces

How Positioning Wearable Haptic Interfaces on Limbs Influences Virtual Embodiment

With increasing use of computer applications and robotic devices in our everyday life, and with the advent of metaverse, there is an urgent need of developing new types of interfaces that facilitate a more intuitive interaction in physical and virtual space. In this work, we investigate the influence of the location of haptic feedback devices on embodiment of virtual hands and user load during an interactive pick-and-place task. To do this, we conducted a user study with a 3x2 repeated measure experiment design: feedback position is varied between the distal phalanx of the index finger and the thumb, the proximal phalanx of the index finger and the thumb, and the wrist. These conditions of feedback are tested with the stimuli applied synchronously to the participant in one case, and with an additional delay of 350 ms in the second case. The results show that the location of the haptic feedback device does not affect embodiment, whereas the delay, i.e., whether the feedback is applied synchronously or asynchronously, affects embodiment. This suggests that for pick-and-place tasks, haptic feedback devices can be placed on the user's wrist without compromising performance making the hands to remain free, allowing unobstructed hand visibility for precise motion tracking, thereby improving accuracy.

Multi-Grasp Classification for the Control of Robot Hands Employing Transformers and Lightmyography Signals

The increasing use of smart technical devices in our everyday lives has necessitated the use of muscle-machine interfaces (MuMI) that are intuitive and that can facilitate immersive interactions with these devices. The most common method to develop MuMIs is using Electromyography (EMG) based signals. However, due to several drawbacks of EMG-based interfaces, alternative methods to develop MuMI are being explored. In our previous work, we presented a new MuMI called Lightmyography (LMG), which achieved outstanding results compared to a classic EMG-based interface in a five-gesture classification task. In this study, we extend our previous work experimentally validating the efficiency of the LMG armband in classifying thirty-two different gestures from six participants using a deep learning technique called Temporal Multi-Channel Vision Transformers (TMC-ViT). The efficiency of the proposed model was assessed using accuracy. Moreover, two different undersampling techniques are compared. The proposed thirty-two-gesture classifiers achieve accuracies as high as 92%. Finally, we employ the LMG interface in the real-time control of a robotic hand using ten different gestures, successfully reproducing several grasp types from taxonomy grasps presented in the literature.