Design and Integration of an Inexpensive Wearable Mechanotactile Feedback System for Myoelectric Prostheses

Assessment of TENS-Evoked Tactile Sensations for Transradial Amputees via EEG Investigation

Most of current prostheses can offer motor function restoration for limb amputees but usually lack natural and intuitive sensory feedback. Many studies have demonstrated that Transcutaneous Electrical Nerve Stimulation (TENS) is promising in non-invasive sensation evoking for amputees. However, the objective evaluation and mechanism analysis on sensation feedback are still limited. This work utilized multi-channel TENS with diverse stimulus patterns to evoke sensations on four non-disabled subjects and two transradial amputees. Meanwhile, electroencephalogram (EEG) was collected to objectively assess the evoked sensations, where event-related potentials (ERPs), brain electrical activity maps (BEAMs), and functional connectivity (FC) were computed. The results show that various sensations could be successfully evoked for both amputees and non-disabled subjects by customizing stimulus parameters. The ERP confirmed the sensation and revealed the sensory-processing-related components like N100 and P200; the BEAMs confirmed the corresponding regions of somatosensory cortex were activated by stimulation; the FC indicated an increase of interactions between the regions of sensorimotor cortex. This study may shed light on how the brain responds to external stimulation as sensory feedback and serve as a pilot for further bidirectional closed-loop prosthetic control.

Soft Robotics Enables Neuroprosthetic Hand Design

Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.

A Design of Biomimetic Prosthetic Hand

Many patients with upper limb defects desire myoelectric prosthetic hands, but they are still not used for some reasons. One of the most significant reasons is its external appearance, which has the discomfort caused by the structural difference between a human hand and a robotic link. The structure must be based on human anatomy to create a more natural-looking prosthesis. This study designed a biomimetic prosthetic hand with bones, ligaments, tendons, and multiple muscles based on the human musculoskeletal system. We verified the proposed prosthetic hand using the viscoelastic angle sensor to determine whether it works like a human hand. We also compared the finger force of the prosthetic hand with that of a human finger. It could be capable of controlling the angle and the stiffness of the joint by multiple extensor and flexor muscles, like humans.

Preliminary Evaluation of the Effect of Mechanotactile Feedback Location on Myoelectric Prosthesis Performance Using a Sensorized Prosthetic Hand

A commonly cited reason for the high abandonment rate of myoelectric prostheses is a lack of grip force sensory feedback. Researchers have attempted to restore grip force sensory feedback by stimulating the residual limb’s skin surface in response to the prosthetic hand’s measured grip force. Recent work has focused on restoring natural feedback to the missing digits directly through invasive surgical procedures. However, the functional benefit of utilizing somatotopically matching feedback has not been evaluated. In this paper, we propose an experimental protocol centered on a fragile object grasp and lift task using a sensorized myoelectric prosthesis to evaluate sensory feedback techniques. We formalized a suite of outcome measures related to task success, timing, and strategy. A pilot study (n = 3) evaluating the effect of utilizing a somatotopically accurate feedback stimulation location in able-bodied participants was conducted to evaluate the feasibility of the standardized platform, and to inform future studies on the role of feedback stimulation location in prosthesis use. Large between-participant effect sizes were observed in all outcome measures, indicating that the feedback location likely plays a role in myoelectric prosthesis performance. The success rate decreased, and task timing and task focus metrics increased, when using somatotopically-matched feedback compared to non-somatotopically-matched feedback. These results were used to conduct a power analysis, revealing that a sample size of n = 8 would be sufficient to achieve significance in all outcome measures.

A Haptic Sleeve as a Method of Mechanotactile Feedback Restoration for Myoelectric Hand Prosthesis Users

Current myoelectric upper limb prostheses do not restore sensory feedback, impairing fine motor control. Mechanotactile feedback restoration with a haptic sleeve may rectify this problem. This randomised crossover within-participant controlled study aimed to assess a prototype haptic sleeve's effect on routine grasping tasks performed by eight able-bodied participants. Each participant completed 15 repetitions of the three tasks: Task 1—normal grasp, Task 2—strong grasp and Task 3—weak grasp, using visual, haptic, or combined feedback All data were collected in April 2021 in the Scottish Microelectronics Centre, Edinburgh, UK. Combined feedback correlated with significantly higher grasp success rates compared to the vision alone in Task 1 (p < 0.0001), Task 2 (p = 0.0057), and Task 3 (p = 0.0170). Similarly, haptic feedback was associated with significantly higher grasp success rates compared to vision in Task 1 (p < 0.0001) and Task 2 (p = 0.0015). Combined feedback correlated with significantly lower energy expenditure compared to visual feedback in Task 1 (p < 0.0001) and Task 3 (p = 0.0003). Likewise, haptic feedback was associated with significantly lower energy expenditure compared to the visual feedback in Task 1 (p < 0.0001), Task 2 (p < 0.0001), and Task 3 (p < 0.0001). These results suggest that mechanotactile feedback provided by the haptic sleeve effectively augments grasping and reduces its energy expenditure.

Wearable Haptic Device Presenting Sensations of Fingertips to the Forearm

Several existing haptic displays used in virtual reality (VR) environments present haptic sensations generated by the fingertips in the VR to actual fingertips. However, these devices face certain challenges, such as physical interference between the devices, particularly when multi-degree-of-freedom (DOF) force needs to be presented to multiple fingers. To address this issue, we propose a haptic presentation method that transmits haptic sensations generated by the fingertips in the VR, including the direction of the force, to the forearm. We previously　proposed a method to present both magnitude and direction of the force applied to the index finger using a five-bar linkage mechanism, which transmits the force sensation with two DOF to the forearm. In this study, the forces in the downward and left-right directions were obtained from the kinematics of a five-bar linkage mechanism for accurate force presentation. Additionally, we conducted a user study evaluating user grasping an object in the VR and performing task. The results verified the haptic sensation of the force transmitted by the proposed prototype to the user's forearm provides a sufficient comfort level. Furthermore, the task execution time and comfort level were comparable to those of a vibrotactile presentation presented directly to the fingertips.

Embedded Electrotactile Feedback System for Hand Prostheses Using Matrix Electrode and Electronic Skin

As the technology moves towards more human-like bionic limbs, it is necessary to develop a feedback system that provides active touch feedback to a user of a prosthetic hand. Most of the contemporary sensory substitution methods comprise simple position and force sensors combined with few discrete stimulation units, and hence they are characterized with a limited amount of information that can be transmitted by the feedback. The present study describes a novel system for tactile feedback integrating advanced multipoint sensing (electronic skin) and stimulation (matrix electrodes). The system comprises a flexible sensing array (16 sensors) integrated on the index finger of a Michelangelo prosthetic hand mockup, embedded interface electronics and multichannel stimulator connected to a flexible matrix electrode (24 pads). The developed system conveys contact information (binary detections) to the user. To demonstrate the feasibility, the system was tested in six able-bodied subjects who were asked to recognize static patterns (contact position) with two different spatial resolutions and dynamic movement patterns (i.e., sliding along and/or across the finger) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into electrotactile profiles, which the subjects could recognize with good performance. The success rates (mean ± standard deviation) for the static patterns were 91 ± 4% and 58 ± 10% for low and high spatial resolution, respectively, while the success rate for sliding touch was 94 ± 4%. These results demonstrate that the developed system is an important step towards a new generation of tactile feedback interfaces that can provide high-bandwidth connection between the user and his/her bionic limb. Such systems would allow mimicking spatially distributed natural feedback, thereby facilitating the control and embodiment of the artificial device into the user body scheme.

Recent advances in wearable biosensing gloves and sensory feedback biosystems for enhancing rehabilitation, prostheses, healthcare, and virtual reality

Wearable sensing gloves and sensory feedback devices that record and enhance the sensations of the hand are used in healthcare, prosthetics, robotics, and virtual reality. Recent technological advancements in soft actuators, flexible bioelectronics, and wireless data acquisition systems have enabled the development of ergonomic, lightweight, and low-cost wearable devices. This review article includes the most up-to-date materials, sensors, actuators, and system-packaging technologies to develop wearable sensing gloves and sensory feedback devices. Furthermore, this review contemplates the use of wearable sensing gloves and sensory feedback devices together to advance their capabilities as assistive devices for people with prostheses and sensory impaired limbs. This review is divided into two sections: one detailing the technologies used to develop strain, pressure, and temperature sensors integrated with a multifunctional wearable sensing glove, and the other reviewing the devices and methods used for wearable sensory displays. We discuss the limitations of the current methods and technologies along with the future direction of the field. Overall, this paper presents an all-inclusive review of the technologies used to develop wearable sensing gloves and sensory feedback devices.

Online Closed-Loop Control Using Tactile Feedback Delivered Through Surface and Subdermal Electrotactile Stimulation

Aim

Limb loss is a dramatic event with a devastating impact on a person’s quality of life. Prostheses have been used to restore lost motor abilities and cosmetic appearance. Closing the loop between the prosthesis and the amputee by providing somatosensory feedback to the user might improve the performance, confidence of the amputee, and embodiment of the prosthesis. Recently, a minimally invasive method, in which the electrodes are placed subdermally, was presented and psychometrically evaluated. The present study aimed to assess the quality of online control with subdermal stimulation and compare it to that achieved using surface stimulation (common benchmark) as well as to investigate the impact of training on the two modalities.

Methods

Ten able-bodied subjects performed a PC-based compensatory tracking task. The subjects employed a joystick to track a predefined pseudorandom trajectory using feedback on the momentary tracking error, which was conveyed via surface and subdermal electrotactile stimulation. The tracking performance was evaluated using the correlation coefficient (CORR), root mean square error (RMSE), and time delay between reference and generated trajectories.

Results

Both stimulation modalities resulted in good closed-loop control, and surface stimulation outperformed the subdermal approach. There was significant difference in CORR (86 vs 77%) and RMSE (0.23 vs 0.31) between surface and subdermal stimulation (all p < 0.05). The RMSE of the subdermal stimulation decreased significantly in the first few trials.

Conclusion

Subdermal stimulation is a viable method to provide tactile feedback. The quality of online control is, however, somewhat worse compared to that achieved using surface stimulation. Nevertheless, due to minimal invasiveness, compactness, and power efficiency, the subdermal interface could be an attractive solution for the functional application in sensate prostheses.

A low-cost transradial prosthesis controlled by the intention of muscular contraction

Persons with upper-limb amputations face severe problems due to a reduction in their ability to perform the activities of daily living. The prosthesis controlled by electromyography (EMG) or other signals from sensors, switches, accelerometers, etc., can somewhat regain the lost capability of such individuals. However, there are several issues with these prostheses, such as expensive cost, limited functionality, unnatural control, slow operating speed, complexity, heavyweight, large size, etc. This paper proposes an affordable transradial prosthesis, controlled by the muscular contractions from user intention. A surface EMG sensor was explicitly fabricated for capturing the muscle contraction information from the residual forearm of subjects with amputation. An under actuated 3D printed hand was developed with a prosthetic socket assembly to attach the remaining upper-limb of such subjects. The hand integrates an intuitive closed-loop control system that receives reference input from the designed sensor and feedback input from a force sensor installed at the thumb tip. The performance of the EMG sensor was compared with that of a traditional sensor in detecting muscle contractions from the subjects. The designed sensor showed a good correlation (r > 0.93) and a better signal-to-noise ratio (SNR) feature to the conventional sensor. Further, a successful trial of the developed hand prosthesis was made on five different subjects with transradial amputation. The users wearing the hand prototype were able to perform faster and delicate grasping of various objects. The implemented control system allowed the prosthesis users to control the grasp force of hand fingers with their intention of muscular contractions.

Recent Developments in Prosthesis Sensors, Texture Recognition, and Sensory Stimulation for Upper Limb Prostheses

Current developments being made in upper limb prostheses are focused on replacing lost sensory information to the amputees. Providing sensory stimulation from the prosthesis can directly improve control over the prosthetic and provide a sense of body ownership. The focus of this review article is on recent developments while including foundational knowledge for some of the critical concepts in neural prostheses. Reported concepts follow the flow of information from sensors to signal processing, with emphasis on texture recognition, and then to sensory stimulation strategies that reestablish the lost sensory feedback loop. Prosthetic sensors are used to detect the physical environment, converting pressure, force, and position into electrical signals. The electrical signals can then be processed in an effort to identify the surrounding environment using distinctive characteristics such as stiffness and texture. In order for the amputee to use this information in a natural manner, there must be real-time sensory stimulation, perception, and motor control of the prosthesis. Although truly complete sensory replacement has not yet been realized, some basic percepts can be partially restored, allowing progress towards a more realistic prosthesis with natural sensations.

Closed-loop Control using Electrotactile Feedback Encoded in Frequency and Pulse Width

Sensory substitution by electrotactile stimulation has been widely investigated for improving the functionality of human-machine interfaces. Few studies, however, have objectively compared different ways in which such systems can be implemented. In this study, we compare encoding of a feedback variable in stimulation pulse width or stimulation frequency during a closed-loop control task. Specifically, participants were asked to track a predefined pseudorandom trajectory using a joystick with electrotactile feedback as the only indication of the tracking error. Each participant performed eight 90 s trials per encoding scheme. Tracking performance using frequency modulation enabled lower tracking error (RMSE: Frequency modulation: 0.27 ± 0.03; Pulse width modulation: 0.31 ± 0.05; p < 0.05) and a higher correlation with the target trajectory (Frequency modulation: 83.4 ± 4.1%; Pulse width modulation: 79.8 ± 5.2%; p < 0.05). There was no significant improvement in performance over the eight trials. Furthermore, frequency-domain analysis revealed that frequency modulation was characterized with a higher gain at lower error frequencies. In summary, the results indicate that encoding of feedback variables in the frequency of pulses enables better control than pulse width modulation in closed-loop dynamic tasks.

Sensory Feedback in Hand Prostheses: A Prospective Study of Everyday Use

Introduction

Sensory feedback in hand prostheses is lacking but wished for. Many amputees experience a phantom hand map on their residual forearm. When the phantom hand map is touched, it is experienced as touch on the amputated hand. A non-invasive sensory feedback system, applicable to existing hand prostheses, can transfer somatotopical sensory information via phantom hand map. The aim was to evaluate how forearm amputees experienced a non-invasive sensory feedback system used in daily life over a 4-week period.

Methods

This longitudinal cohort study included seven forearm amputees. A non-invasive sensory feedback system was used over 4 weeks. For analysis, a mixed method was used, including quantitative tests (ACMC, proprioceptive pointing task, questionnaire) and interviews. A directed content analysis with predefined categories sensory feedback from the prosthesis, agency, body ownership, performance in activity, and suggestions for improvements was applied.

Results

The results from interviews showed that sensory feedback was experienced as a feeling of touch which contributed to an experience of completeness. However, the results from the questionnaire showed that the sense of agency and performance remained unchanged or deteriorated. The ability to feel and manipulate small objects was difficult and a stronger feedback was wished for. Phantom pain was alleviated in four out of five patients.

Conclusion

This is the first time a non-invasive sensory feedback system for hand prostheses was implemented in the home environment. The qualitative and quantitative results diverged. The sensory feedback was experienced as a feeling of touch which contributed to a feeling of completeness, linked to body ownership. The qualitative result was not verified in the quantitative measurements.

Clinical Trial Registration

Name: Evaluation of a Non-invasive Sensory Feedback System in Hand Prostheses. Date of registration: March 15, 2019. Date the first participant was enrolled: April 1, 2015. ClinicalTrials.gov Identifier: NCT03876405 ORCID ID: https://orcid.org/0000-0002-4140-7478.

A Modular Adjustable Transhumeral Prosthetic Socket for Evaluating Myoelectric Control

Novel myoelectric control strategies may yield more robust, capable prostheses which improve quality of life for those affected by upper-limb loss; however, the development and translation of such strategies from an experimental setting towards daily use by persons with limb loss is a slow and costly process. Since prosthesis functionality is highly dependent on the physical interface between the user’s prosthetic socket and residual limb, assessment of such controllers under realistic (noisy) environmental conditions, integrated into prosthetic sockets, and with participants with amputation is essential for obtaining representative results. Unfortunately, this step is particularly difficult as participant- and control strategy-specific prosthetic sockets must be custom-designed and manufactured. There is thus a need for a system to reduce these burdens and facilitate this crucial phase of the development pipeline. This study aims to address this gap through the design and assessment of an inexpensive and easy-to-use 3D-printed Modular-Adjustable transhumeral Prosthetic Socket (MAPS). This 3D-printed, open-source socket was developed in consultation with prosthetists and compared with a participant-specific suction socket in a single-participant case-study. We conducted mechanical and functional assessments to ensure that the developed socket enabled similar performance compared to participant-specific sockets. Both socket systems yielded similar results in mechanical and functional assessments, as well as in self-reported user feedback. The MAPS system shows promise as a research tool which catalyzes the development and deployment of novel myoelectric control strategies by better-enabling comprehensive assessment involving participants with amputations.

Design of a portable anthropomimetic upper limb rehabilitation device for patients suffering from neuromuscular disability

An upper limb anthropomimetic rehabilitation device has been designed for patients suffering from a neuromuscular disability. The developed device has been designed as a wearable device and attempts to supplement all known functions of the human hand and fingers. The actuation of individual joints of the hand and wrist has been implemented by using DC motors interfaced to a control system. A pulley system was adopted to ensure a low device profile with the aim of maximising functionality in the affected hand. Both actuators and the electronic assembly are sited in the forearm assembly for this purpose. The device is designed to fulfill multiple roles. At its simplest instance, it is designed as a device for providing resistance training in patients suffering from reversible neuromuscular weakness. The device also aims to provide support as an exoskeleton device in patients suffering from partial but permanent neuromuscular weakness. The measurement of finger and wrist bending in axial and radial directions were investigated by an array of potentiometers mounted around the wearable device covering different joints of the fingers and wrist, and were further analyzed to characterize the range of the device. The system is a composite device with diverse functions fulfilling all the requirements of an upperlimb orthotic device. The device is planned to be part of a comprehensive exoskeleton device for quadriparetic patients in the future.

Mechanotactile Sensory Feedback Improves Embodiment of a Prosthetic Hand During Active Use

There have been several advancements in the field of myoelectric prostheses to improve dexterity and restore hand grasp patterns for persons with upper limb loss, including robust control strategies, novel sensory feedback, and multifunction prosthetic terminal devices. Although these advancements have shown to improve prosthesis performance, a key element that may further improve acceptance is often overlooked. Embodiment, which encompasses the feeling of owning, controlling and locating the device without the need to constantly look at it, has been shown to be affected by sensory feedback. However, the specific aspects of embodiment that are influenced are not clearly understood, particularly when a prosthesis is actively controlled. In this work, we used a sensorized simulated prosthesis in able-bodied participants to investigate the contribution of sensory feedback, active motor control, and the combination of both to the components of embodiment; using a common methodology in the literature, namely the rubber hand illusion (RHI). Our results indicate that (1) the sensorized simulated prosthesis may be embodied by able-bodied users in a similar fashion as prosthetic devices embodied by persons with upper limb amputation, and (2) mechanotactile sensory feedback might not only be useful for improving certain aspects of embodiment, i.e., ownership and location, but also may have a modulating effect on other aspects, namely sense of agency, when provided asynchronously during active motor control tasks. This work may allow us to further investigate and manipulate factors contributing to the complex phenomenon of embodiment in relation to active motor control of a device, enabling future study of more precise quantitative measures of embodiment that do not rely as much on subjective perception.

Psychometric characterization of incidental feedback sources during grasping with a hand prosthesis

Background

A prosthetic system should ideally reinstate the bidirectional communication between the user’s brain and its end effector by restoring both motor and sensory functions lost after an amputation. However, current commercial prostheses generally do not incorporate somatosensory feedback. Even without explicit feedback, grasping using a prosthesis partly relies on sensory information. Indeed, the prosthesis operation is characterized by visual and sound cues that could be exploited by the user to estimate the prosthesis state. However, the quality of this incidental feedback has not been objectively evaluated.

Methods

In this study, the psychometric properties of the auditory and visual feedback of prosthesis motion were assessed and compared to that of a vibro-tactile interface. Twelve able-bodied subjects passively observed prosthesis closing and grasping an object, and they were asked to discriminate (experiment I) or estimate (experiment II) the closing velocity of the prosthesis using visual (VIS), acoustic (SND), or combined (VIS + SND) feedback. In experiment II, the subjects performed the task also with a vibrotactile stimulus (VIB) delivered using a single tactor. The outcome measures for the discrimination and estimation experiments were just noticeable difference (JND) and median absolute estimation error (MAE), respectively.

Results

The results demonstrated that the incidental sources provided a remarkably good discrimination and estimation of the closing velocity, significantly outperforming the vibrotactile feedback. Using incidental sources, the subjects could discriminate almost the minimum possible increment/decrement in velocity that could be commanded to the prosthesis (median JND < 2% for SND and VIS + SND). Similarly, the median MAE in estimating the prosthesis velocity randomly commanded from the full working range was also low, i.e., approximately 5% in SND and VIS + SND.

Conclusions

Since the closing velocity is proportional to grasping force in state-of-the-art myoelectric prostheses, the results of the present study imply that the incidental feedback, when available, could be usefully exploited for grasping force control. Therefore, the impact of incidental feedback needs to be considered when designing a feedback interface in prosthetics, especially since the quality of estimation using supplemental sources (e.g., vibration) can be worse compared to that of the intrinsic cues.