Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future Directions

Application of EMG feedback for hand prosthesis control in high-level amputation: a case study

EMG feedback improves force control of a myoelectric hand prosthesis by conveying the magnitude of the myoelectric signal back to the users via tactile stimulation. The present study aimed to test if this method can be used by a participant with a high-level amputation, and whose muscle used for prosthesis control (pectoralis major) was not intuitively related to hand function. Vibrotactile feedback was delivered to the participant's torso, while the control was tested using EMG from three different muscles. The participant completed four experimental sessions of a force-matching task with a prosthesis. The performance was evaluated by computing the target force success rate. The results of session 1 showed that the participant could effectively employ EMG feedback after only brief training. Session 2 demonstrated that EMG feedback benefited force control, increasing the success rate by approx. 30%. Finally, after proper training (sessions 3 and 4), the participant's performance when using the muscle on the amputated side was similar to that achieved with the muscles on the contralateral side. Overall, the study results indicate that EMG feedback can be used in high-level amputations, despite the extent of the injury and non-intuitive control.

A neurocognitive pathway for engineering artificial touch

Artificial haptics has the potential to revolutionize the way we integrate physical and virtual technologies in our daily lives, with implications for teleoperation, motor skill acquisition, rehabilitation, gaming, interpersonal communication, and beyond. Here, we delve into the intricate interplay between the somatosensory system and engineered haptic inputs for perception and action. We critically examine the sensory feedback's fidelity and the cognitive demands of interfacing with these systems. We examine how artificial touch interfaces could be redesigned to better align with human sensory, motor, and cognitive systems, emphasizing the dynamic and context-dependent nature of sensory integration. We consider the various learning processes involved in adapting to artificial haptics, highlighting the need for interfaces that support both explicit and implicit learning mechanisms. We emphasize the need for technologies that are not only physiologically biomimetic but also behaviorally and cognitively congruent with the user, affording a range of alternative solutions to users' needs.

Wrist posture unpredictably affects perception of targeted transcutaneous electrical nerve stimulation with wrist-placed electrodes

Objective

Targeted transcutaneous electrical nerve stimulation (tTENS) is a non-invasive neural stimulation technique that involves activating sensory nerve fibers to elicit tactile sensations in a distal, or referred, location. Though tTENS is a promising approach for delivering haptic feedback in virtual reality or for use by those with somatosensory deficits, it was not known how the perception of tTENS might be influenced by changing wrist position during sensorimotor tasks.

Approach

We worked with 12 able-bodied individuals and delivered tTENS by placing electrodes on the wrist, thus targeting the ulnar, median, and radial nerves, and eliciting tactile sensations in the hand. We recorded perceptual data across three wrist postures: neutral, 45° extension, and 45° flexion. For each posture, the participants drew where they perceived the elicited percepts on a map of the hand. They verbally reported the quality of the percepts in their own words. We also varied the pulse amplitude and width of the stimulation to generate a strength-duration curve, from which we extracted the rheobase current and chronaxie time. Linear mixed models were run on the slope and intercept of the linear fit between pulse width and pulse amplitude to investigate effects of gender, posture, and electrode placement.

Main results

As wrist posture changed, sensation quality was modulated for half of the participants, and percept location changed for 11/12 participants. The rheobase, chronaxie, and percept sizes were influenced by wrist posture, but the direction of these changes varied by participant and therefore the effect was not systematic. The statistical models indicated interactions between posture and electrode placement, as well as an effect of gender.

Significance

If using tTENS with electrodes placed on the wrist to convey haptic feedback during sensorimotor tasks, in which wrist posture will likely change, it may be important to characterize perception on an individual basis.

Flexible and Transparent Electrovibration-Based Haptic Display with Low Driving Voltage

Electrovibration haptic technology, which provides tactile feedback to users by swiping the surface with a finger via electroadhesion, shows promise as a haptic feedback platform for displays owing to its simple structure, ease of integration with existing displays, and simple driving mechanism. However, without electrical grounding on a user's body, the frequent requirement of a high driving voltage near 50 V limits the use of electrovibration haptic technology in practical display applications. This study introduces materials and fabrication strategies that considerably reduce the driving voltage. We used a transparent poly(vinylidene fluoride) (PVDF) thin film deposited on transparent conductive polymers through a simple spin-coating process, thereby enabling easy integration with existing display technologies. The high dielectric constant characteristics of PVDF enabled the production of tactile cues at low voltages (approximately 15 V), which are within the safety limits of common electronics. We verified the feasibility of our electrovibration haptic feedback system on the basis of the absolute threshold voltage through two-alternative forced choice psychological tests. The results revealed that the PVDF dielectric layer exhibited a relatively lower absolute threshold than commonly used polymer films, which possess a relatively lower dielectric constant. To validate the tactile attributes, a Likert five-point scale survey was conducted, considering flat, concave, and convex curvatures. The results indicated that our haptic device can render diverse surface textures, such as "hairy" and "groovy", on the fingertips through the control of applied pulse width modulated voltage signals.

Modulating the Fidelity and Spatial Extent of Electrotactile Stimulation to Elicit the Embodiment of a Virtual Hand

Restoring tactile feedback in virtual reality can improve user experience and facilitate embodiment. Electrotactile stimulation is an attractive technology in this context because it is compact and allows for high-resolution spatially distributed stimulation. In this study, a 32-channel tactile glove worn on the fingertips was used to provide tactile sensations during a virtual version of a rubber hand illusion experiment. To assess the benefits of multichannel stimulation, we modulated the spatial extent and fidelity of feedback. Thirty-six participants performed the experiment in two conditions, where stimulation was delivered to a single finger or all fingers, and three tactile stimulation types within each condition: no tactile feedback, simple single-point stimulation, and complex sliding stimulation mimicking the brush movements. Following each trial, the participants answered a multi-item embodiment questionnaire and reported the proprioceptive drift. The results confirmed that modulating the spatial extent of stimulation, from a single finger to all fingers, was indeed a successful strategy. When stimulating all fingers, tactile feedback significantly improved all subjective measures compared to receiving no tactile stimulation. However, unexpectedly, the second strategy, that of modulating the fidelity of feedback, was not successful since there was no difference between the simple and complex tactile feedback in any of the measures. The results, therefore, imply that the effects of tactile feedback are better expressed in a more dynamic scenario (i.e., making/breaking contact and stimulating different body locations), while it should be investigated if further improvements of the complex feedback can make it more effective than the simple approach.

Interference haptic stimulation and consistent quantitative tactility in transparent electrotactile screen with pressure-sensitive transistors

Integrating tactile feedback through haptic interfaces enhances experiences in virtual and augmented reality. However, electrotactile systems, which stimulate mechanoreceptors directly, often yield inconsistent tactile results due to variations in pressure between the device and the finger. In this study, we present the integration of a transparent electrotactile screen with pressure-sensitive transistors, ensuring highly consistent quantitative haptic sensations. These transistors effectively calibrate tactile variations caused by touch pressure. Additionally, we explore remote-distance tactile stimulations achieved through the interference of electromagnetic waves. We validated tactile perception using somatosensory evoked potentials, monitoring the somatosensory cortex response. Our haptic screen can stimulate diverse electrotactile sensations and demonstrate various tactile patterns, including Morse code and Braille, when integrated with portable smart devices, delivering a more immersive experience. Furthermore, interference of electric fields allows haptic stimulation to facilitate diverse stimulus positioning at lower current densities, extending the reach beyond direct contact with electrodes of our screen.

Modulation of Ultrasonic Stimulation Parameters towards Evoking Fine Tactile Sensations

An increasing number of recent studies prove that the low-intensity focused ultrasound stimulation (LIFUS) is a promising neuromodulation technique. Its feasibility of eliciting multiple fine tactile sensations, as well as the modulatory effect of ultrasonic stimulation parameters have not been investigated by far. Therefore, in this work, we enrolled eight healthy volunteers to participate into two sets of psychophysical experiment, wherein the ultrasonic stimuli were delivered onto their index fingertips under varying pulse repetition frequency (PRF) and acoustic intensity (AI). Our results showed that ultrasonic stimuli at lower PRF and lower AI were perceived more sensitively in term of Just Noticeable Difference (JND) and Weber fractions (WF) in experiment 1. In experiment 2, six types of fine tactile sensation, including tickling, vibrating, buzzing, pressure, petting, electrical, plus no sensation, could be distinguished under 18 different PRF-AI combinations. Lower PRFs tended to induce rhythmic tap-like sensations, while higher PRFs tended to evoke electrical-like sensations. A synergistic effect of PRF and AI on modulating fine tactile sensations was observed as well. These findings may shed light on modulation of ultrasonic stimulation parameters, and facilitate the application of LIFUS into various human-machine interaction settings.

Conductive block copolymer elastomers and psychophysical thresholding for accurate haptic effects

Electrotactile stimulus is a form of sensory substitution in which an electrical signal is perceived as a mechanical sensation. The electrotactile effect could, in principle, recapitulate a range of tactile experience by selective activation of nerve endings. However, the method has been plagued by inconsistency, galvanic reactions, pain and desensitization, and unwanted stimulation of nontactile nerves. Here, we describe how a soft conductive block copolymer, a stretchable layout, and concentric electrodes, along with psychophysical thresholding, can circumvent these shortcomings. These purpose-designed materials, device layouts, and calibration techniques make it possible to generate accurate and reproducible sensations across a cohort of 10 human participants and to do so at ultralow currents (≥6 microamperes) without pain or desensitization. This material, form factor, and psychophysical approach could be useful for haptic devices and as a tool for activation of the peripheral nervous system.

Encoding contact size using static and dynamic electrotactile finger stimulation: natural decoding vs. trained cues

Electrotactile stimulation through matrix electrodes is a promising technology to restore high-resolution tactile feedback in extended reality applications. One of the fundamental tactile effects that should be simulated is the change in the size of the contact between the finger and a virtual object. The present study investigated how participants perceive the increase of stimulation area when stimulating the index finger using static or dynamic (moving) stimuli produced by activating 1 to 6 electrode pads. To assess the ability to interpret the stimulation from the natural cues (natural decoding), without any prior training, the participants were instructed to draw the size of the stimulated area and identify the size difference when comparing two consecutive stimulations. To investigate if other "non-natural" cues can improve the size estimation, the participants were asked to enumerate the number of active pads following a training protocol. The results demonstrated that participants could perceive the change in size without prior training (e.g., the estimated area correlated with the stimulated area, p < 0.001; ≥ two-pad difference recognized with > 80% success rate). However, natural decoding was also challenging, as the response area changed gradually and sometimes in complex patterns when increasing the number of active pads (e.g., four extra pads needed for the statistically significant difference). Nevertheless, by training the participants to utilize additional cues the limitations of natural perception could be compensated. After the training, the mismatch in the activated and estimated number of pads was less than one pad regardless of the stimulus size. Finally, introducing the movement of the stimulus substantially improved discrimination (e.g., 100% median success rate to recognize ≥ one-pad difference). The present study, therefore, provides insights into stimulation size perception, and practical guidelines on how to modulate pad activation to change the perceived size in static and dynamic scenarios.

Visuo-Haptic Rendering of the Hand during 3D Manipulation in Augmented Reality

Manipulating virtual objects with bare hands is a key interaction in Augmented Reality (AR) applications. However, there are still several limitations that affect the manipulation, including the lack of mutual visual occlusion between virtual and real content as well as the lack of haptic sensations. To address the two abovementioned matters, the role of the visuo-haptic rendering of the hand as sensory feedback is investigated. The first experiment explores the effect of showing the hand of the user as seen by the AR system through an avatar, comparing six visual hand rendering. The second experiment explores the effect of the visuo-haptic hand rendering by comparing two vibrotactile contact techniques provided at four delocalized positions on the hand and combined with the two most representative visual hand renderings from the first experiment. Results show that delocalized vibrotactile haptic hand rendering improved perceived effectiveness, realism, and usefulness when provided close to the contact point. However, the farthest rendering position, i.e., on the contralateral hand, gave the best performance even though it was largely disliked. The visual hand rendering was perceived as less necessary for manipulation when the haptic hand rendering was available, but still provided useful feedback on the hand tracking.

Haptic Feedback, Performance and Arousal: A Comparison Study in an Immersive VR Motor Skill Training Task

This article investigates the relationship between fine motor skill training in VR, haptic feedback, and physiological arousal. To do so, we present the design and development of a motor skill task (buzzwire), along with a custom vibrotactile feedback attachment for the Geomagic Touch haptic device. A controlled experiment following a between-subjects design was conducted with 73 participants, studying the role of three feedback conditions - visual/kinesthetic, visual/vibrotactile and visual only - on the learning and performance of the considered task and the arousal levels of the participants. Results indicate that performance improved in all three feedback conditions after the considered training session. However, participants reported no change in self-efficacy and in terms of presence and task load (NASA-TLX). All three feedback conditions also showed similar arousal levels. Further analysis revealed that positive changes in performance were linked to higher arousal levels. These results suggest the potential of haptic feedback to affect arousal levels and encourage further research into using this relationship to improve motor skill training in VR.

Haptic Sensing and Feedback Techniques toward Virtual Reality

Haptic interactions between human and machines are essential for information acquisition and object manipulation. In virtual reality (VR) system, the haptic sensing device can gather information to construct virtual elements, while the haptic feedback part can transfer feedbacks to human with virtual tactile sensation. Therefore, exploring high-performance haptic sensing and feedback interface imparts closed-loop haptic interaction to VR system. This review summarizes state-of-the-art VR-related haptic sensing and feedback techniques based on the hardware parts. For the haptic sensor, we focus on mechanism scope (piezoresistive, capacitive, piezoelectric, and triboelectric) and introduce force sensor, gesture translation, and touch identification in the functional view. In terms of the haptic feedbacks, methodologies including mechanical, electrical, and elastic actuators are surveyed. In addition, the interactive application of virtual control, immersive entertainment, and medical rehabilitation is also summarized. The challenges of virtual haptic interactions are given including the accuracy, durability, and technical conflicts of the sensing devices, bottlenecks of various feedbacks, as well as the closed-loop interaction system. Besides, the prospects are outlined in artificial intelligence of things, wise information technology of medicine, and multimedia VR areas.

Soft Sensors and Actuators for Wearable Human-Machine Interfaces

Haptic human-machine interfaces (HHMIs) combine tactile sensation and haptic feedback to allow humans to interact closely with machines and robots, providing immersive experiences and convenient lifestyles. Significant progress has been made in developing wearable sensors that accurately detect physical and electrophysiological stimuli with improved softness, functionality, reliability, and selectivity. In addition, soft actuating systems have been developed to provide high-quality haptic feedback by precisely controlling force, displacement, frequency, and spatial resolution. In this Review, we discuss the latest technological advances of soft sensors and actuators for the demonstration of wearable HHMIs. We particularly focus on highlighting material and structural approaches that enable desired sensing and feedback properties necessary for effective wearable HHMIs. Furthermore, promising practical applications of current HHMI technology in various areas such as the metaverse, robotics, and user-interactive devices are discussed in detail. Finally, this Review further concludes by discussing the outlook for next-generation HHMI technology.

Toward the Development of User-Centered Neurointegrated Lower Limb Prostheses

The last few years witnessed radical improvements in lower-limb prostheses. Researchers have presented innovative solutions to overcome the limits of the first generation of prostheses, refining specific aspects which could be implemented in future prostheses designs. Each aspect of lower-limb prostheses has been upgraded, but despite these advances, a number of deficiencies remain and the most capable limb prostheses fall far short of the capabilities of the healthy limb. This article describes the current state of prosthesis technology; identifies a number of deficiencies across the spectrum of lower limb prosthetic components with respect to users' needs; and discusses research opportunities in design and control that would substantially improve functionality concerning each deficiency. In doing so, the authors present a roadmap of patients related issues that should be addressed in order to fulfill the vision of a next-generation, neurally-integrated, highly-functional lower limb prosthesis.

Study of the Effectiveness of a Wearable Haptic Interface With Cutaneous and Vibrotactile Feedback for VR-Based Teleoperation

The control principle of teleoperated robotic systems is often not very intuitive, resulting in substantial cognitive load for the operator. Besides, the teleoperation of robotic systems requires stable and convenient visual feedback for the operator. Better spatio-temporal robot coordination can be achieved by providing the operator with high-fidelity multimodal haptic feedback. Using the advantages of virtual reality (VR) and haptic technology, we can improve the quality of teleoperation and increase the level of user involvement when operating the system. This article focuses on studying the effectiveness of the developed wearable haptic interface, vDeltaGlove, that provides multimodal haptic feedback when controlling a robotic arm through a virtual environment (VE). We evaluated the performance of vDeltaGlove in the series of teleoperation tasks. vDeltaGlove interface showed comparable characteristics with the Omega.7 desktop haptic device in terms of accuracy and outreached the performance of operation with VIVE controller. Besides, the developed interface showed the superior evaluation in terms of user experience. The proposed technology suggests a novel way of human-robot interaction (HRI) to achieve intuitive and immersive robot control.

Exploring Perceptual Intensity Properties Using Electrotactile Stimulation on Fingertips

Understanding electrotactile parametric properties is a crucial milestone in achieving intuitive haptics. Perceptual intensity is a primary property, but its exploration remains challenging due to subjectivity. To address this problem, this study conducted two experiments on fingertips and proposed two metrics based on significant findings. Experiment 1 found a significant linear relationship (R 2 = 0.981) between pulse amplitude (PA) and pulse width (PW) in the logarithmic plane, and proposed a metric of parameter intensity (PI) to estimate the intensity of parameters. In Experiment 2, subjective intensity (SI) was defined and measured using a scale of 0 to 10. A metric model of SI (SI model) was derived based on the linear relationship (R 0.78) between PI and measured SI. A calibration method was proposed and its prediction accuracy has been verified. An average RMSE of 11.2 % indicated an accuracy close to the subjective judgment error of 8.7 %. Results are consistent across subjects and four different electrode-skin conditions (ESC). The findings of this study provide theoretical support for SI prediction and regulation, which is significant for electrotactile feedback.

Action-Specific Perception &amp; Performance on a Fitts's Law Task in Virtual Reality: The Role of Haptic Feedback

While user's perception and performance are predominantly examined independently in virtual reality, the Action-Specific Perception (ASP) theory postulates that the performance of an individual on a task modulates this individual's spatial and time perception pertinent to the task's components and procedures. This paper examines the association between performance and perception and the potential effects that tactile feedback modalities could generate. This paper reports a user study (N=24), in which participants performed a standardized Fitts's law target acquisition task by using three feedback modalities: visual, visuo-electrotactile, and visuo-vibrotactile. The users completed 3 Target Sizes × 2 Distances × 3 feedback modalities = 18 trials. The size perception, distance perception, and (movement) time perception were assessed at the end of each trial. Performance-wise, the results showed that electrotactile feedback facilitates a significantly better accuracy compared to vibrotactile and visual feedback, while vibrotactile provided the worst accuracy. Electrotactile and visual feedback enabled a comparable reaction time, while the vibrotactile offered a substantially slower reaction time than visual feedback. Although amongst feedback types the pattern of differences in perceptual aspects were comparable to performance differences, none of them was statistically significant. However, performance indeed modulated perception. Significant action-specific effects on spatial and time perception were detected. Changes in accuracy modulate both size perception and time perception, while changes in movement speed modulate distance perception. Also, the index of difficulty was found to modulate all three perceptual aspects. However, individual differences appear to affect the magnitude of action-specific effects. These outcomes highlighted the importance of haptic feedback on performance, and importantly the significance of action-specific effects on spatial and time perception in VR, which should be considered in future VR studies.

Electrotactile Communication via Matrix Electrode Placed on the Torso Using Fast Calibration, and Static vs. Dynamic Encoding

Electrotactile stimulation is a technology that reproducibly elicits tactile sensations and can be used as an alternative channel to communicate information to the user. The presented work is a part of an effort to develop this technology into an unobtrusive communication tool for first responders. In this study, the aim was to compare the success rate (SR) between discriminating stimulation at six spatial locations (static encoding) and recognizing six spatio-temporal patterns where pads are activated sequentially in a predetermined order (dynamic encoding). Additionally, a procedure for a fast amplitude calibration, that includes a semi-automated initialization and an optional manual adjustment, was employed and evaluated. Twenty subjects, including twelve first responders, participated in the study. The electrode comprising the 3 × 2 matrix of pads was placed on the lateral torso. The results showed that high SRs could be achieved for both types of message encoding after a short learning phase; however, the dynamic approach led to a statistically significant improvement in messages recognition (SR of 93.3%), compared to static stimulation (SR of 83.3%). The proposed calibration procedure was also effective since in 83.8% of the cases the subjects did not need to adjust the stimulation amplitude manually.