Evoked Haptic Sensation in the Hand With Concurrent Non-Invasive Nerve Stimulation

Unsupervised separation of nonlinearly mixed event-related potentials using manifold clustering and non-negative matrix factorization

Event-related potentials (ERPs) can quantify brain responses to reveal the neural mechanisms of sensory perception. However, ERPs often reflect nonlinear mixture responses to multiple sources of sensory stimuli, and an accurate separation of the response to each stimulus remains a challenge. This study aimed to separate the ERP into nonlinearly mixed source components specific to individual stimuli. We developed an unsupervised learning method based on clustering of manifold structures of mixture signals combined with channel optimization for signal source reconstruction using non-negative matrix factorization (NMF). Specifically, we first implemented manifold learning based on Local Tangent Space Alignment (LTSA) to extract the spatial manifold structure of multi-resolution sub-signals separated via wavelet packet transform. We then used fuzzy entropy to extract the dynamical process of the manifold structures and performed a k-means clustering to separate different sources. Lastly, we used NMF to obtain the optimal contributions of multiple channels to ensure accurate source reconstructions. We evaluated our developed approach using a simulated ERP dataset with known ground truth of two components of ERP mixture signals. Our results show that the correlation coefficient between the reconstructed source signal and the true source signal was 92.8 % and that the separation accuracy in ERP amplitude was 91.6 %. The results show that our unsupervised separation approach can accurately separate ERP signals from nonlinear mixture source components. The outcomes provide a promising way to isolate brain responses to multiple stimulus sources during multisensory perception.

Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

A Sensory Feedback Neural Stimulator Prototype for Both Implantable and Wearable Applications

The restoration of sensory feedback is one of the current challenges in the field of prosthetics. This work, following the analysis of the various types of sensory feedback, aims to present a prototype device that could be used both for implantable applications to perform PNS and for wearable applications, performing TENS, to restore sensory feedback. The two systems are composed of three electronic boards that are presented in detail, as well as the bench tests carried out. To the authors' best knowledge, this work presents the first device that can be used in a dual scenario for restoring sensory feedback. Both the implantable and wearable versions respected the expected values regarding the stimulation parameters. In its implantable version, the proposed system allows simultaneous and independent stimulation of 30 channels. Furthermore, the capacity of the wearable version to elicit somatic sensations was evaluated on healthy participants demonstrating performance comparable with commercial solutions.

Distally-referred surface electrical nerve stimulation (DR-SENS) for haptic feedback

Objective.This study's objective is to understand distally-referred surface electrical nerve stimulation (DR-SENS) and evaluates the effects of electrode placement, polarity, and stimulation intensity on the location of elicited sensations in non-disabled individuals.Approach.A two-phased human experiment was used to characterize DR-SENS. In Experiment One, we explored 182 electrode combinations to identify a subset of electrode position combinations that would be most likely to elicit distally-referred sensations isolated to the index finger without discomfort. In Experiment Two, we further examined this subset of electrode combinations to determine the effect of stimulation intensity and electrode position on perceived sensation location. Stimulation thresholds were evaluated using parameter estimation by sequential testing and sensation locations were characterized using psychometric intensity tests.Main Results.We found that electrode positions distal to the wrist can consistently evoke distally referred sensations with no significant polarity dependency. The finger-palm combination had the most occurrences of distal sensations, and the different variations of this combination did not have a significant effect on sensation location. Increasing stimulation intensity significantly expanded the area of the sensation, moved the most distal sensation distally, and moved the vertical centroid proximally. Also, a large anodic-leading electrode at the elbow mitigated all sensation at the anodic-leading electrode site while using symmetric stimulation waveforms. Furthermore, this study showed that the most intense sensation for a given percept can be distally referred. Lastly, for each participant, at least one of the finger-palm combinations evaluated in this study worked at both perception threshold and maximum comfortable stimulation intensities.Significance.These findings show that a non-invasive surface electrical stimulation charge modulated haptic interface can be used to elicit distally-referred sensations on non-disabled users. Furthermore, these results inform the design of novel haptic interfaces and other applications of surface electrical stimulation based haptic feedback on electrodes positioned distally from the wrist.

Eliciting tactile sensations in the hand through non-invasive proximal nerve stimulation: a feasibility study

Recently, non-invasive proximal nerve stimulation has been widely investigated to restore tactile sensations. It has been demonstrated that tactile sensations in the hand could be elicited by nerve stimulation on the upper arm. However, it is still unknown whether tactile sensations could be elicited by stimulation at a proximal location close to the neck. In this study, non-invasive proximal nerve stimulation tests were performed to elicit tactile sensations in the hand of subjects. Six Ag/AgCl gel electrodes (2 × 3) were placed on the supraclavicular fossa where the proximal parts of the brachial plexus nerves were located. Then, fifteen potential electrode pairs were tested to explore whether tactile sensations could be elicited by non-invasive proximal nerve stimulation. Eight able-bodied subjects (male) were recruited to participate in the test. The stimulated sensation regions in the hand and the sensory intensity were reported and recorded during the experiment. The results demonstrated that the tactile sensations in various regions in the hand could be elicited through non-invasive nerve stimulation at the proximal location close to the neck.

Identifying Oscillations under Multi-site Sensory Stimulation for High-level Peripheral Nerve Injured Patients：A Pilot Study

OBJECTIVE

For high-level peripheral nerve injured (PNI) patients with severe sensory dysfunction of upper extremities, identifying the multi-site tactile stimulation is of great importance to provide neurorehabilitation with sensory feedback. In this pilot study, we showed the feasibility of identifying multi-site and multi-intensity tactile stimulation in terms of electroencephalography (EEG).

APPROACH

Three high-level PNI patients and eight non-PNI participants were recruited in this study. Four different sites over the upper arm, forearm, thumb finger and little finger were randomly stimulated at two intensities (both sensory-level) based on the transcutaneous electrical nerve stimulation (TENS). Meanwhile, 64-channel EEG signals were recorded during the passive tactile sense stimulation on each side.

MAIN RESULTS

The spatial-spectral distribution of brain oscillations underlying multi-site sensory stimulation showed dominant power attenuation over the somatosensory and prefrontal cortices in both alpha-band (8-12 Hz) and beta-band (13-30 Hz). But there was no significant difference among different stimulation sites in terms of the averaged power spectral density over the region of interest (ROI). By further identifying different stimulation sites using temporal-spectral features, we found the classification accuracies were all above 89% for the affected arm of PNI patients, comparable to that from their intact side and that from the non-PNI group. When the stimulation site-intensity combinations were treated as eight separate classes, the classification accuracies were ranging from 88.89% to 99.30% for the affected side of PNI subjects, similar to that from their non-affected side and that from the non-PNI group. Other performance metrics, including Specificity, Precision, and F1-Score, also showed a sound identification performance for both PNI patients and non-PNI subjects.

SIGNIFICANCE

These results suggest that reliable brain oscillations could be evoked and identified well, even though induced tactile sense could not be discerned by the PNI patients. This study have implication for facilitating bidirectional neurorehabilitation systems with sensory feedback.

Evaluation of multiple perceptual qualities of transcutaneous electrical nerve stimulation for evoked tactile sensation in forearm amputees

OBJECTIVE

Evoked tactile sensation (ETS) elicited by transcutaneous electrical nerve stimulation (TENS) is promising to convey digit-specific sensory information to amputees naturally and non-invasively. Fitting ETS-based sensory feedback to amputees entails customizing coding of multiple sensory information for each stimulation site. This study was to elucidate the consistency of percepts and qualities by TENS at multiple stimulation sites in amputees retaining ETS.

APPROACH

Five transradial amputees with ETS and fourteen able-bodied subjects participated in this study. Surface electrodes with small size (10 mm in diameter) were adopted to fit the restricted projected finger map on the forearm stump of amputees. Effects of stimulus frequency on sensory types were assessed, and the map of perceptual threshold for each sensation was characterized. Sensitivity for vibration and buzz sensations was measured using distinguishable difference in stimulus pulse width. Rapid assessments for modulation ranges of pulse width at fixed amplitude and frequency were developed for coding sensory information. Buzz sensation was demonstrated for location discrimination relating to prosthetic fingers.

MAIN RESULTS

Vibration and buzz sensations were consistently evoked at 20 Hz and 50 Hz as dominant sensation types in all amputees and able-bodied subjects. Perceptual thresholds of different sensations followed a similar strength-duration curve relating stimulus amplitude to pulse width. The averaged distinguishable difference in pulse width was 12.84 ± 7.23 μs for vibration and 15.21 ± 6.47 μs for buzz in able-bodied subjects, and 14.91 ± 10.54 μs for vibration and 11.30 ± 3.42 μs for buzz in amputees. Buzz coding strategy enabled five amputees to discriminate contact of individual fingers with an overall accuracy of 77.85%.

SIGNIFICANCE

The consistency in perceptual qualities of dominant sensations can be exploited for coding multi-modality sensory feedback. A fast protocol of sensory coding is possible for fitting ETS-based, non-invasive sensory feedback to amputees.

Object Recognition via Evoked Sensory Feedback during Control of a Prosthetic Hand

Haptic and proprioceptive feedback is critical for sensorimotor integration when we use our hand to perform daily tasks. Here, we evaluated how externally evoked haptic and proprioceptive feedback and myoelectric control strategies affected the recognition of object properties when participants controlled a prosthetic hand. Fingertip haptic sensation was elicited using a transcutaneous nerve stimulation grid to encode the prosthetic's fingertip forces. An array of tactors elicited patterned vibratory stimuli to encode tactile-proprioceptive kinematic information of the prosthetic finger joint. Myoelectric signals of the finger flexor and extensor were used to control the position or velocity of joint angles of the prosthesis. Participants were asked to perform object property (stiffness and size) recognition, by controlling the prosthetic hand with concurrent haptic and tactile-proprioceptive feedback. With the evoked feedback, intact and amputee participants recognized the object stiffness and size at success rates ranging from 50% to 100% in both position and velocity control with no significant difference across control schemes. Our findings show that evoked somatosensory feedback in a non-invasive manner can facilitate closed-loop control of the prosthetic hand and allowed for simultaneous recognition of different object properties. The outcomes can facilitate our understanding on the role of sensory feedback during bidirectional human-machine interactions, which can potentially promote user experience in object interactions using prosthetic hands.

Closed-loop control of a prosthetic finger via evoked proprioceptive information

Objective.Proprioceptive information plays an important role for recognizing and coordinating our limb's static and dynamic states relative to our body or the environment. In this study, we determined how artificially evoked proprioceptive feedback affected the continuous control of a prosthetic finger.Approach.We elicited proprioceptive information regarding the joint static position and dynamic movement of a prosthetic finger via a vibrotactor array placed around the subject's upper arm. Myoelectric signals of the finger flexor and extensor muscles were used to control the prosthesis, with or without the evoked proprioceptive feedback. Two control modes were evaluated: the myoelectric signal amplitudes were continuously mapped to either the position or the velocity of the prosthetic joint.Main results.Our results showed that the evoked proprioceptive information improved the control accuracy of the joint angle, with comparable performance in the position- and velocity-control conditions. However, greater angle variability was prominent during position-control than velocity-control. Without the proprioceptive feedback, the position-control tended to show a smaller angle error than the velocity-control condition.Significance.Our findings suggest that closed-loop control of a prosthetic device can potentially be achieved using non-invasive evoked proprioceptive feedback delivered to intact participants. Moreover, the evoked sensory information was integrated during myoelectric control effectively for both control strategies. The outcomes can facilitate our understanding of the sensorimotor integration process during human-machine interactions, which can potentially promote fine control of prosthetic hands.

Static and dynamic proprioceptive recognition through vibrotactile stimulation

Objective.Proprioceptive information provides individuals with a sense of our limb's static position and dynamic movement. Impaired or a lack of such feedback can diminish our ability to perform dexterous motions with our biological limbs or assistive devices. Here we seek to determine whether both static and dynamic components of proprioception can be recognized using variation of the spatial and temporal components of vibrotactile feedback.Approach.An array of five vibrotactors was placed on the forearm of each subject. Each tactor was encoded to represent one of the five forearm postures. Vibratory stimulus was elicited to convey the static position and movement of the forearm. Four experimental blocks were performed to test each subject's recognition of a forearm's simulated static position, rotational amplitude, rotational amplitude and direction, and rotational speed.Main results.Our results showed that the subjects were able to perform proprioceptive recognition based on the delivered vibrotactile information. Specifically, rotational amplitude recognition resulted in the highest level of accuracy (99.0%), while the recognition accuracy of the static position and the rotational amplitude-direction was the lowest (91.7% and 90.8%, respectively). Nevertheless, all proprioceptive properties were perceived with >90% accuracy, indicating that the implemented vibrotactile encoding scheme could effectively provide proprioceptive information to the users.Significance.The outcomes suggest that information pertaining to static and dynamic aspects of proprioception can be accurately delivered using an array of vibrotactors. This feedback approach could be used to potentially evaluate the sensorimotor integration processes during human-machine interactions, and to improve sensory feedback in clinical populations with somatosensory impairments.

Stiffness Perception using Transcutaneous Electrical Stimulation during Active and Passive Prosthetic Control

Haptic feedback allows an individual to identify various object properties. In this preliminary study, we determined the performance of stiffness recognition using transcutaneous nerve stimulation when a prosthetic hand was moved passively or was controlled actively by the subjects. Using a 2x8 electrode grid placed along the subject's upper arm, electrical stimulation was delivered to evoke somatotopic sensation along their index finger. Stimulation intensity, i.e. sensation strength, was modulated using the fingertip forces from a sensorized prosthetic hand. Object stiffness was encoded based on the rate of change of the evoked sensation as the prosthesis grasped one of three objects of different stiffness levels. During active control, sensation was modulated in real time as recorded forces were converted to stimulation amplitudes. During passive control, prerecorded force traces were randomly selected from a pool. Our results showed that the accuracy of object stiffness recognition was similar in both active and passive conditions. A slightly lower accuracy was observed during active control in one subject, which indicated that the sensorimotor integration processes could affect haptic perception for some users.

Evoking Apparent Moving Sensation in the Hand via Transcutaneous Electrical Nerve Stimulation

The restoration of sensory feedback in amputees plays a fundamental role in the prosthesis control and in the communication on the afferent channel between hand and brain. The literature shows that transcutaneous electrical nerve stimulation (TENS) can be a promising non-invasive technique to elicit sensory feedback in amputees, especially in the lower limb through the phenomenon of apparent moving sensation (AMS). It consists of delivering a sensation that moves along a specific part of the body. This study proposes to use TENS to elicit tactile sensations and adopt AMS to reproduce moving sensations on the hand, such as those related to an object moving in the hand or slipping upward or downward. To this purpose, the developed experimental protocol consists of two phases: (i) the mapping of the evoked sensations and (ii) the generation of the AMS. In the latter phase, the pulse amplitude variation (PAV), the pulse width variation (PWV), and the interstimulus delay modulation (ISDM) methods were compared. For the comparative analysis, the Wilcoxon-Mann-Whitney test with Bonferroni correction (P < 0.016) was carried out on the success rate and on the ranking of methods expressed by the subjects. Results from the mapping protocol show that the delivered sensations were mostly described by the subjects as almost natural and superficial tingling. Results from the AMS protocol show that, for each movement direction, the success rate of ISDM method is higher than that of PWV and PAV and significantly higher than that of PAV for the ulnar-median direction. It recreates an AMS in the hand that effectively allows discriminating the type of sensation and distinguishing the movement direction. Moreover, ISDM was ranked by the subjects as the favorite method for recreating a well-defined and comfortable moving sensation only in the median-ulnar direction. For the ranking results, there was not a statistically significant difference among the methods. The experiments confirmed the good potential of recreating an AMS in the hand through TENS. This encourages to push forward this study on amputees and integrate it in the closed-loop control of a prosthetic system, in order to enable full control of grasp stability and prevent the objects from slippage.

Evoking haptic sensations in the foot through high-density transcutaneous electrical nerve stimulations

OBJECTIVE

Evoking haptic sensation on upper limb amputees via peripheral nerve stimulation has been investigated intensively in the past decade, but related studies involving lower limb amputees are limited. This study aimed to evaluate the feasibility of using non-invasive transcutaneous electrical nerve stimulation to evoke haptic sensation along the phantom limb of the amputated foot of transtibial amputees.

APPROACH

A high-density electrode grid (4 × 4) was placed over the skin surface above the distal branching of the sciatic, tibial, and common peroneal nerves. We hypothesized that electrical stimulation delivered to distinct electrode pairs created unique electric fields, which can activate selective sets of sensory axons innervating different skin regions of the foot. Five transtibial amputee subjects (three unilateral and two bilateral) and one able-bodied subject were tested by scanning all possible electrode pair combinations.

MAIN RESULTS

All subjects reported various haptic percepts at distinct regions along the foot with each corresponding to specific electrode pairs. These results demonstrated the capability of our non-invasive nerve stimulation method to evoke haptic sensations in the foot of transtibial amputees and the able-bodied subject.

SIGNIFICANCE

The outcomes contribute important knowledge and evidence regarding missing tactile sensation in the foot of lower limb amputees and might also facilitate future development of strategies to manage phantom pain and enhance embodiment of prosthetic legs in the future.

Object Shape and Surface Topology Recognition Using Tactile Feedback Evoked through Transcutaneous Nerve Stimulation

Tactile feedback is critical for distinguishing different object properties. In this article, we determined if tactile feedback evoked by transcutaneous nerve stimulation can be used to detect objects of different shape and surface topology. To evoke tactile sensation at different fingers, a 2x8 electrode grid was placed along the subject's upper arm, and two concurrent electrical stimulation trains targeted the median and ulnar nerve bundles, which evoked individually modulated sensations at different fingers. Fingertip forces of the prosthetic hand were transformed to stimulation current amplitude. Object shape was encoded based on finger-object contact timing. Surface topology represented by ridge height and spacing was encoded through current amplitude and stimulation time interval, respectively. The elicited sensation allowed subjects to determine object shape with success rates >84%. Surface topology recognition resulted in success rates >81%. Our findings suggest that tactile feedback evoked from transcutaneous nerve stimulation allows the recognition of object shape and surface topology. The ability to recognize these properties may help improve object manipulation and promote fine control of a prosthetic hand.

Object stiffness recognition using haptic feedback delivered through transcutaneous proximal nerve stimulation

OBJECTIVE

Haptic feedback is crucial when we manipulate objects. Information pertaining to an object's stiffness in particular can help facilitate fine motor control. In this study, we seek to determine whether objects of different stiffness levels can be recognized using haptic feedback provided by transcutaneous electrical stimulation of peripheral nerves.

APPROACH

Using a stimulation electrode grid placed along the medial side of the upper arm, the median and ulnar nerve bundles were targeted to evoke haptic sensation on the palmar side of the hand. Stimulation current amplitude was modulated in real-time with the fingertip force recorded from a sensorized prosthetic hand. In order to evaluate which stimulation pattern was more critical, object stiffness was encoded either by the rate of change of the stimulus amplitude or the level of peak stimulus amplitude, as the prosthesis grasped the objects.

MAIN RESULTS

Both encoding methods allowed the subjects to differentiate objects of different stiffness levels with  >90% accuracy. No significant difference was observed between the two encoding methods, which indicated that both the rate of change of the stimulation amplitude and the peak stimulation amplitude could effectively provide stiffness information of the objects.

SIGNIFICANCE

The outcomes suggest that it is possible to elicit haptic sensations describing various object stiffness levels using transcutaneous nerve stimulation. The haptic feedback associated with object stiffness can facilitate object manipulation/interactions. It may also improve user experience during human-machine interactions, when object stiffness information is incorporated.