Closed-loop Control using Electrotactile Feedback Encoded in Frequency and Pulse Width

The Interplay Between Muscular Activity and Pattern Recognition of Electro-Stimulated Haptic Cues During Normal Walking: A Pilot Study

This pilot study explored how muscle activation influences the pattern recognition of tactile cues delivered using electrical stimulation (ES) during each 10% window interval of the normal walking gait cycle (GC). Three healthy adults participated in the experiment. After identifying the appropriate threshold, ES as the haptic cue was applied to the gastrocnemius lateralis (GL) and biceps brachii (BB) of participants walking on a treadmill. Findings revealed variable recognition patterns across participants, with the BB showing more variability during walking due to its minimal activity compared to the actively engaged GL. Dynamic time warping (DTW) was used to assess the similarity between muscle activation and electro-stimulated haptic perception. The DTW distance between electromyography (EMG) signals and muscle recognition patterns was significantly smaller for the GL (4.87 ± 0.21, mean ± SD) than the BB (8.65 ± 1.36, mean ± SD), showing a 78.6% relative difference, indicating that higher muscle activation was generally associated with more consistent haptic perception. However, individual differences and variations in recognition patterns were observed, suggesting personal variability influenced the perception outcomes. The study underscores the complexity of human neuromuscular responses to artificial sensory stimuli and suggests a potential link between muscle activity and haptic perception.

Real-time adaptive cancellation of TENS feedback artifact on sEMG for prosthesis closed-loop control

Introduction

The prosthetic hand has been aimed to restore hand functions by estimating the user's intention via bio-signal and providing sensory feedback. Surface electromyogram (sEMG) is a widely used signal, and transcutaneous electrical nerve stimulation (TENS) is a promising method for sensory feedback. However, TENS currents can transmit through the skin and interfere as noise with the sEMG signals, referred to as "Artifact," which degrades the performance of intention estimation.

Method

In this paper, we proposed an adaptive artifact removal method that can cancel artifacts separately across different frequencies and pulse widths of TENS. The modified least-mean-square adaptive filter uses the mean of previous artifacts as reference signals, and compensate using prior information of TENS system. Also temporal separation for artifact discrimination is applied to achieve high artifact removal efficiency. Four sEMG signals-two from flexor digitorum superficialis, flexor carpi ulnaris, extensor carpi ulnaris-was collected to validate signals both offline and online experiments.

Results and Discussion

We validated the filtering performance with twelve participants performing two experiments: artifact cancellation under variable conditions and a real-time hand control simulation called the target reaching experiment (TRE). The result showed that the Signal-to-Noise Ratio (SNR) increased by an average of 10.3dB, and the performance of four TRE indices recovered to the levels similar to those without TENS. The proposed method can significantly improve signal quality via artifact removal in the context of sensory feedback through TENS in prosthetic systems.

A hybrid sensory feedback system for thermal nociceptive warning and protection in prosthetic hand

Background

Advanced prosthetic hands may embed nanosensors and microelectronics in their cosmetic skin. Heat influx may cause damage to these delicate structures. Protecting the integrity of the prosthetic hand becomes critical and necessary to ensure sustainable function. This study aims to mimic the sensorimotor control strategy of the human hand in perceiving nociceptive stimuli and triggering self-protective mechanisms and to investigate how similar neuromorphic mechanisms implemented in prosthetic hand can allow amputees to both volitionally release a hot object upon a nociceptive warning and achieve reinforced release via a bionic withdrawal reflex.

Methods

A steady-state temperature prediction algorithm was proposed to shorten the long response time of a thermosensitive temperature sensor. A hybrid sensory strategy for transmitting force and a nociceptive temperature warning using transcutaneous electrical nerve stimulation based on evoked tactile sensations was designed to reconstruct the nociceptive sensory loop for amputees. A bionic withdrawal reflex using neuromorphic muscle control technology was used so that the prosthetic hand reflexively opened when a harmful temperature was detected. Four able-bodied subjects and two forearm amputees randomly grasped a tube at the different temperatures based on these strategies.

Results

The average prediction error of temperature prediction algorithm was 8.30 ± 6.00%. The average success rate of six subjects in perceiving force and nociceptive temperature warnings was 86.90 and 94.30%, respectively. Under the reinforcement control mode in Test 2, the median reaction time of all subjects was 1.39 s, which was significantly faster than the median reaction time of 1.93 s in Test 1, in which two able-bodied subjects and two amputees participated. Results demonstrated the effectiveness of the integration of nociceptive sensory strategy and withdrawal reflex control strategy in a closed loop and also showed that amputees restored the warning of nociceptive sensation while also being able to withdraw from thermal danger through both voluntary and reflexive protection.

Conclusion

This study demonstrated that it is feasible to restore the sensorimotor ability of amputees to warn and react against thermal nociceptive stimuli. Results further showed that the voluntary release and withdrawal reflex can work together to reinforce heat protection. Nevertheless, fusing voluntary and reflex functions for prosthetic performance in activities of daily living awaits a more cogent strategy in sensorimotor control.

A portable, programmable, multichannel stimulator with high compliance voltage for noninvasive neural stimulation of motor and sensory nerves in humans

Most neural stimulators do not have a high enough compliance voltage to pass current through the skin. The few stimulators that meet the high compliance voltage necessary for transcutaneous stimulation are typically large benchtop units that are not portable, and the stimulation waveforms cannot be readily customized. To address this, we present the design and validation of a portable, programmable, multichannel, noninvasive neural stimulator that can generate three custom bipolar waveforms at ± 150 V with microsecond temporal resolution. The design is low-cost, open-source, and validated on the benchtop and with a healthy population to demonstrate its functionality for sensory and motor stimulation. Sensory stimulation included electrocutaneous stimulation targeting cutaneous mechanoreceptors at the surface of the skin and transcutaneous nerve stimulation targeting the median nerve at the wrist. Both electrocutaneous stimulation on the hand and transcutaneous stimulation at the wrist can elicit isolated tactile percepts on the hand but changes in pulse frequency are more discriminable for electrocutaneous stimulation. Also, neuromuscular electrical stimulation of the flexor digiti profundus is evoked by applying electrical stimulation directly above the muscle in the forearm and to the median and ulnar nerves in the upper arm. Muscle and nerve stimulation evoked similar grip forces and force rise times, but nerve stimulation had a significantly slower fatigue rate. The development and validation of this noninvasive stimulator and direct comparison of common sensory and motor stimulation targets in a human population constitute an important step towards more widespread use and accessibility of neural stimulation for education and research.

Continuous Transition Impairs Discrimination of Electrotactile Frequencies

Just-noticeable difference (JND), indicating the ability to accurately identify small differences in stimulation parameters, can be used to choose more sensitive stimulation methods as well as to calibrate tactile feedback in closed-loop human-machine interfacing. The JND is typically estimated using a forced-choice-discrimination task, in which two stimuli with different intensities are delivered separated by a brief pause. In the applications of tactile feedback, however, the stimulation parameters are typically modulated continuously. It is unclear if the discriminability of stimuli separated in time characterizes the ability to distinguish continuous changes in stimulation intensity. The present study compared the JND when pairs of frequency-modulated electrotactile stimuli were separated in time and presented continuously at two different baseline frequencies (20 and 60 Hz). The results showed that the JND was significantly smaller with time-separation between stimuli, but that the JND obtained with different types of transitions were in most cases linearly associated. In conclusion, the discriminability of time-separated stimuli is systematically better compared to that of the stimuli presented continuously. This can have an impact when calibrating the tactile feedback where the conventional method of the JND assessment might lead to an overly optimistic estimate of detectable changes.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Evaluation of multiple perceptual qualities of transcutaneous electrical nerve stimulation for evoked tactile sensation in forearm amputees

OBJECTIVE

Evoked tactile sensation (ETS) elicited by transcutaneous electrical nerve stimulation (TENS) is promising to convey digit-specific sensory information to amputees naturally and non-invasively. Fitting ETS-based sensory feedback to amputees entails customizing coding of multiple sensory information for each stimulation site. This study was to elucidate the consistency of percepts and qualities by TENS at multiple stimulation sites in amputees retaining ETS.

APPROACH

Five transradial amputees with ETS and fourteen able-bodied subjects participated in this study. Surface electrodes with small size (10 mm in diameter) were adopted to fit the restricted projected finger map on the forearm stump of amputees. Effects of stimulus frequency on sensory types were assessed, and the map of perceptual threshold for each sensation was characterized. Sensitivity for vibration and buzz sensations was measured using distinguishable difference in stimulus pulse width. Rapid assessments for modulation ranges of pulse width at fixed amplitude and frequency were developed for coding sensory information. Buzz sensation was demonstrated for location discrimination relating to prosthetic fingers.

MAIN RESULTS

Vibration and buzz sensations were consistently evoked at 20 Hz and 50 Hz as dominant sensation types in all amputees and able-bodied subjects. Perceptual thresholds of different sensations followed a similar strength-duration curve relating stimulus amplitude to pulse width. The averaged distinguishable difference in pulse width was 12.84 ± 7.23 μs for vibration and 15.21 ± 6.47 μs for buzz in able-bodied subjects, and 14.91 ± 10.54 μs for vibration and 11.30 ± 3.42 μs for buzz in amputees. Buzz coding strategy enabled five amputees to discriminate contact of individual fingers with an overall accuracy of 77.85%.

SIGNIFICANCE

The consistency in perceptual qualities of dominant sensations can be exploited for coding multi-modality sensory feedback. A fast protocol of sensory coding is possible for fitting ETS-based, non-invasive sensory feedback to amputees.

Electrotactile and Vibrotactile Feedback Enable Similar Performance in Psychometric Tests and Closed-Loop Control

Electro- and vibro-tactile stimulation are commonly employed for feedback in closed-loop human-machine interfacing. Although these feedback systems have been extensively investigated individually, they are rarely objectively compared. In this study, two state-of-the-art stimulation units (concentric electrode and C2-tactor) similar in shape and size were compared in psychometric and online control tests. The just noticeable difference and number of discriminable levels for intensity and frequency modulation were determined across values of carrier frequency and intensity, respectively. Next, subjects performed a compensatory tracking task, in which the feedback encoded the momentary tracking error. In the psychometric tests, intensity modulation outperformed frequency modulation and electrotactile stimulation enabled significantly higher resolution than vibrotactile stimulation, for the same carrier frequency. However, for the best-case settings (eletro-tactile: 100 Hz; vibro-tactile: 200 Hz), the two stimulation modalities were equivalent in the psychometric tests and in the online control tests, where the two stimulation methods resulted in similar correlation and deviation between the target and the generated trajectory. Time delay was slightly but significantly lower for the vibrotactile modality. Overall, the present assessment shows that despite psychometric differences between the two stimulation methods, they enable similar online control performance when parameters are optimally selected for each modality.

Electrotactile feedback outweighs natural feedback in sensory integration during control of grasp force

Objective.The nervous system subconsciously estimates the state of the body as a weighted average of the information from various sensory sources, where the weights reflect the perceived reliability of each source. Loss of motor functions can be partially compensated using assistive systems (e.g. prostheses), which may also restore somatosensory feedback through tactile stimulation. Whether such artificial feedback is integrated in the neural state estimation process is not known.Approach.In this study, able-bodied subjects performed a grasp force matching task with supplementary non-invasive electrotactile stimulation with a frequency proportional to grasp force magnitude. Before the task, a brief training session taught the subjects to associate the sensation of electrotactile stimulation with the generated grasp force. In some trials, the force-frequency mapping was biased to introduce an unnoticeable mismatch between natural and electrotactile force feedback, thereby provoking the subject to subconsciously estimate the force as a compromise between the two sources of information.Main results.The outcome of this compromise revealed the weights assigned to each feedback type. The grasp forces were significantly affected by the biased mappings, as indicated by the average estimated relative weights (electrotactile: 0.69 ± 0.29; natural: 0.31 ± 0.29). Across subjects, this weight was correlatedr2=0.75) with the improvement in force matching precision when adding the unbiased electrotactile feedback to the natural force feedback, as predicted by maximum likelihood estimation. This shows that even after minimal training the nervous system adopts electrotactile stimulation as a highly reliable source of information that can improve the precision in the estimation of the grip force.Significance.This result has important implications for the restoration of sensory feedback in upper limb prostheses as it indicates that even non-invasive stimulation can be integrated naturally (i.e. subconsciously and effectively) in the motor controlloop.

Online Closed-Loop Control Using Tactile Feedback Delivered Through Surface and Subdermal Electrotactile Stimulation

Aim

Limb loss is a dramatic event with a devastating impact on a person’s quality of life. Prostheses have been used to restore lost motor abilities and cosmetic appearance. Closing the loop between the prosthesis and the amputee by providing somatosensory feedback to the user might improve the performance, confidence of the amputee, and embodiment of the prosthesis. Recently, a minimally invasive method, in which the electrodes are placed subdermally, was presented and psychometrically evaluated. The present study aimed to assess the quality of online control with subdermal stimulation and compare it to that achieved using surface stimulation (common benchmark) as well as to investigate the impact of training on the two modalities.

Methods

Ten able-bodied subjects performed a PC-based compensatory tracking task. The subjects employed a joystick to track a predefined pseudorandom trajectory using feedback on the momentary tracking error, which was conveyed via surface and subdermal electrotactile stimulation. The tracking performance was evaluated using the correlation coefficient (CORR), root mean square error (RMSE), and time delay between reference and generated trajectories.

Results

Both stimulation modalities resulted in good closed-loop control, and surface stimulation outperformed the subdermal approach. There was significant difference in CORR (86 vs 77%) and RMSE (0.23 vs 0.31) between surface and subdermal stimulation (all p < 0.05). The RMSE of the subdermal stimulation decreased significantly in the first few trials.

Conclusion

Subdermal stimulation is a viable method to provide tactile feedback. The quality of online control is, however, somewhat worse compared to that achieved using surface stimulation. Nevertheless, due to minimal invasiveness, compactness, and power efficiency, the subdermal interface could be an attractive solution for the functional application in sensate prostheses.