1
|
Tchimino J, Dideriksen JL, Dosen S. EMG feedback improves grasping of compliant objects using a myoelectric prosthesis. J Neuroeng Rehabil 2023; 20:119. [PMID: 37705008 PMCID: PMC10500847 DOI: 10.1186/s12984-023-01237-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 08/24/2023] [Indexed: 09/15/2023] Open
Abstract
BACKGROUND Closing the control loop in myoelectric prostheses by providing artificial somatosensory feedback is recognized as an important goal. However, designing a feedback interface that is effective in realistic conditions is still a challenge. Namely, in some situations, feedback can be redundant, as the information it provides can be readily obtained through hearing or vision (e.g., grasping force estimated from the deformation of a compliant object). EMG feedback is a non-invasive method wherein the tactile stimulation conveys to the user the level of their own myoelectric signal, hence a measurement intrinsic to the interface, which cannot be accessed incidentally. METHODS The present study investigated the efficacy of EMG feedback in prosthesis force control when 10 able-bodied participants and a person with transradial amputation used a myoelectric prosthesis to grasp compliant objects of different stiffness values. The performance with feedback was compared to that achieved when the participants relied solely on incidental cues. RESULTS The main outcome measures were the task success rate and completion time. EMG feedback resulted in significantly higher success rates regardless of pin stiffness, indicating that the feedback enhanced the accuracy of force application despite the abundance of incidental cues. Contrary to expectations, there was no difference in the completion time between the two feedback conditions. Additionally, the data revealed that the participants could produce smoother control signals when they received EMG feedback as well as more consistent commands across trials, signifying better control of the system by the participants. CONCLUSIONS The results presented in this study further support the efficacy of EMG feedback when closing the prosthesis control loop by demonstrating its benefits in particularly challenging conditions which maximized the utility of intrinsic feedback sources.
Collapse
Affiliation(s)
- Jack Tchimino
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Jakob Lund Dideriksen
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Strahinja Dosen
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
2
|
Su S, Chai G, Xu W, Meng J, Sheng X, Mouraux A, Zhu X. Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis. J Neural Eng 2023; 20. [PMID: 36595235 DOI: 10.1088/1741-2552/acab32] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 12/13/2022] [Indexed: 12/15/2022]
Abstract
Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.
Collapse
Affiliation(s)
- Shiyong Su
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Guohong Chai
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Wei Xu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Jianjun Meng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - André Mouraux
- Institute of Neuroscience (IoNS), Université catholique de Louvain, Brussels, Belgium
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
3
|
Testing silicone digit extensions as a way to suppress natural sensation to evaluate supplementary tactile feedback. PLoS One 2021; 16:e0256753. [PMID: 34469470 PMCID: PMC8410127 DOI: 10.1371/journal.pone.0256753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 08/13/2021] [Indexed: 11/19/2022] Open
Abstract
Dexterous use of the hands depends critically on sensory feedback, so it is generally agreed that functional supplementary feedback would greatly improve the use of hand prostheses. Much research still focuses on improving non-invasive feedback that could potentially become available to all prosthesis users. However, few studies on supplementary tactile feedback for hand prostheses demonstrated a functional benefit. We suggest that confounding factors impede accurate assessment of feedback, e.g., testing non-amputee participants that inevitably focus intently on learning EMG control, the EMG’s susceptibility to noise and delays, and the limited dexterity of hand prostheses. In an attempt to assess the effect of feedback free from these constraints, we used silicone digit extensions to suppress natural tactile feedback from the fingertips and thus used the tactile feedback-deprived human hand as an approximation of an ideal feed-forward tool. Our non-amputee participants wore the extensions and performed a simple pick-and-lift task with known weight, followed by a more difficult pick-and-lift task with changing weight. They then repeated these tasks with one of three kinds of audio feedback. The tests were repeated over three days. We also conducted a similar experiment on a person with severe sensory neuropathy to test the feedback without the extensions. Furthermore, we used a questionnaire based on the NASA Task Load Index to gauge the subjective experience. Unexpectedly, we did not find any meaningful differences between the feedback groups, neither in the objective nor the subjective measurements. It is possible that the digit extensions did not fully suppress sensation, but since the participant with impaired sensation also did not improve with the supplementary feedback, we conclude that the feedback failed to provide relevant grasping information in our experiments. The study highlights the complex interaction between task, feedback variable, feedback delivery, and control, which seemingly rendered even rich, high-bandwidth acoustic feedback redundant, despite substantial sensory impairment.
Collapse
|
4
|
Sensinger JW, Dosen S. A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control. Front Neurosci 2020; 14:345. [PMID: 32655344 PMCID: PMC7324654 DOI: 10.3389/fnins.2020.00345] [Citation(s) in RCA: 60] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2019] [Accepted: 03/23/2020] [Indexed: 12/22/2022] Open
Abstract
This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.
Collapse
Affiliation(s)
- Jonathon W. Sensinger
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| | - Strahinja Dosen
- Department of Health Science and Technology, The Faculty of Medicine, Integrative Neuroscience, Aalborg University, Aalborg, Denmark
| |
Collapse
|
5
|
Chai G, Briand J, Su S, Sheng X, Zhu X. Electrotactile Feedback with Spatial and Mixed Coding for Object Identification and Closed-loop Control of Grasping Force in Myoelectric Prostheses. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:1805-1808. [PMID: 31946247 DOI: 10.1109/embc.2019.8856508] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Providing high-quality somatosensory feedback from myoelectric prostheses to an upper-limb amputee user is a long-standing challenge. Various approaches have been investigated for tactile feedback, ranging from direct neural stimulation to noninvasive sensory substitution methods. However, only a few of studies evaluated the closed-loop performance, and real-time movement information of active prostheses still could not be transferred in the form of proprioceptive feedback so far. In current study, an integrated closed-loop prosthesis system consisted of two types of sensors, programmable electrical stimulator and multichannel array electrodes was presented. The grasping angle and corresponding grasping force of the single-freedom myoelectric prosthesis were simultaneously coded with spatial and mixed (spatial and intensity of sensation) coding scheme and tested in 15 able-bodied subjects. The experimental results demonstrated that the subjects were able to discriminate 4 types of object sizes, 3 kinds of different softness and 4 levels of grasping forces in relatively high correct identification rates (CIRs) (size: 87.5%, Softness: 94%, grasping force: 73.8%). The study outcomes and specific conclusions provide valuable guidance for the design of closed-loop myoelectric prostheses equipped with electrotactile feedback.
Collapse
|
6
|
Myocontrol is closed-loop control: incidental feedback is sufficient for scaling the prosthesis force in routine grasping. J Neuroeng Rehabil 2018; 15:81. [PMID: 30176929 PMCID: PMC6122439 DOI: 10.1186/s12984-018-0422-7] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2018] [Accepted: 08/22/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Sensory feedback is critical for grasping in able-bodied subjects. Consequently, closing the loop in upper-limb prosthetics by providing artificial sensory feedback to the amputee is expected to improve the prosthesis utility. Nevertheless, even though amputees rate the prospect of sensory feedback high, its benefits in daily life are still very much debated. We argue that in order to measure the potential functional benefit of artificial sensory feedback, the baseline open-loop performance needs to be established. METHODS The myoelectric control of naïve able-bodied subjects was evaluated during modulation of electromyographic signals (EMG task), and grasping with a prosthesis (Prosthesis task). The subjects needed to activate the wrist flexor muscles and close the prosthesis to reach a randomly selected target level (routine grasping). To assess the baseline performance, the tasks were performed with a different extent of implicit feedback (proprioception, prosthesis motion and sound). Finally, the prosthesis task was repeated with explicit visual force feedback. The subjects' ability to scale the prosthesis command/force was assessed by testing for a statistically significant increase in the median of the generated commands/forces between neighboring levels. The quality of control was evaluated by computing the median absolute error (MAE) with respect to the target. RESULTS The subjects could successfully scale their motor commands and generated prosthesis forces across target levels in all tasks, even with the least amount of implicit feedback (only muscle proprioception, EMG task). In addition, the deviation of the generated commands/forces from the target levels decreased with additional feedback. However, the increase in implicit feedback, from proprioception to prosthesis motion and sound, seemed to have a more substantial effect than the final introduction of explicit feedback. Explicit feedback improved the performance mainly at the higher target-force levels. CONCLUSIONS The study establishes the baseline performance of myoelectric control and prosthesis grasping force. The results demonstrate that even without additional feedback, naïve subjects can effectively modulate force with good accuracy with respect to that achieved when increasing the amount of feedback information.
Collapse
|
7
|
Shehata AW, Engels LF, Controzzi M, Cipriani C, Scheme EJ, Sensinger JW. Improving internal model strength and performance of prosthetic hands using augmented feedback. J Neuroeng Rehabil 2018; 15:70. [PMID: 30064477 PMCID: PMC6069837 DOI: 10.1186/s12984-018-0417-4] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 07/17/2018] [Indexed: 11/24/2022] Open
Abstract
Background The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. Methods Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. Results Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. Conclusions We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.
Collapse
Affiliation(s)
- Ahmed W Shehata
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada. .,Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada. .,Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta, Edmonton, AB, T6G 2E1, Canada.
| | - Leonard F Engels
- Scuola Superiore Sant'Anna, The BioRobotics Institute, V.le R. Piaggio 34, 56025, Pontedera, PI, Italy
| | - Marco Controzzi
- Scuola Superiore Sant'Anna, The BioRobotics Institute, V.le R. Piaggio 34, 56025, Pontedera, PI, Italy
| | - Christian Cipriani
- Scuola Superiore Sant'Anna, The BioRobotics Institute, V.le R. Piaggio 34, 56025, Pontedera, PI, Italy
| | - Erik J Scheme
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada.,Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada
| | - Jonathon W Sensinger
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada.,Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB, E3B 5A3, Canada
| |
Collapse
|
8
|
Markovic M, Schweisfurth MA, Engels LF, Bentz T, Wüstefeld D, Farina D, Dosen S. The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis. J Neuroeng Rehabil 2018; 15:28. [PMID: 29580245 PMCID: PMC5870217 DOI: 10.1186/s12984-018-0371-1] [Citation(s) in RCA: 70] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2017] [Accepted: 02/22/2018] [Indexed: 11/26/2022] Open
Abstract
Background To effectively replace the human hand, a prosthesis should seamlessly respond to user intentions but also convey sensory information back to the user. Restoration of sensory feedback is rated highly by the prosthesis users, and feedback is critical for grasping in able-bodied subjects. Nonetheless, the benefits of feedback in prosthetics are still debated. The lack of consensus is likely due to the complex nature of sensory feedback during prosthesis control, so that its effectiveness depends on multiple factors (e.g., task complexity, user learning). Methods We evaluated the impact of these factors with a longitudinal assessment in six amputee subjects, using a clinical setup (socket, embedded control) and a range of tasks (box and blocks, block turn, clothespin and cups relocation). To provide feedback, we have proposed a novel vibrotactile stimulation scheme capable of transmitting multiple variables from a multifunction prosthesis. The subjects wore a bracelet with four by two uniformly placed vibro-tactors providing information on contact, prosthesis state (active function), and grasping force. The subjects also completed a questionnaire for the subjective evaluation of the feedback. Results The tests demonstrated that feedback was beneficial only in the complex tasks (block turn, clothespin and cups relocation), and that the training had an important, task-dependent impact. In the clothespin relocation and block turn tasks, training allowed the subjects to establish successful feedforward control, and therefore, the feedback became redundant. In the cups relocation task, however, the subjects needed some training to learn how to properly exploit the feedback. The subjective evaluation of the feedback was consistently positive, regardless of the objective benefits. These results underline the multifaceted nature of closed-loop prosthesis control as, depending on the context, the same feedback interface can have different impact on performance. Finally, even if the closed-loop control does not improve the performance, it could be beneficial as it seems to improve the subjective experience. Conclusions Therefore, in this study we demonstrate, for the first time, the relevance of an advanced, multi-variable feedback interface for dexterous, multi-functional prosthesis control in a clinically relevant setting. Electronic supplementary material The online version of this article (10.1186/s12984-018-0371-1) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Marko Markovic
- Department for Trauma Surgery, Orthopedics and Plastic Surgery, University Medical Center Göttingen, Von-Siebold-Str. 3, 37075, Göttingen, Germany.
| | - Meike A Schweisfurth
- Department for Trauma Surgery, Orthopedics and Plastic Surgery, University Medical Center Göttingen, Von-Siebold-Str. 3, 37075, Göttingen, Germany.,Faculty of Life Sciences, University of Applied Sciences (HAW), Ulmenliet 20, 21033, Hamburg, Germany
| | - Leonard F Engels
- Department for Trauma Surgery, Orthopedics and Plastic Surgery, University Medical Center Göttingen, Von-Siebold-Str. 3, 37075, Göttingen, Germany.,Biorobotics Institute, Scuola Superiore Sant'Anna, Viale R. Piaggio, 34, 56025, Pontedera (PI), Italy
| | | | - Daniela Wüstefeld
- Otto Bock Competence Center, Otto Bock HealthCare GmbH, 37115, Duderstadt, Germany
| | - Dario Farina
- Department for Trauma Surgery, Orthopedics and Plastic Surgery, University Medical Center Göttingen, Von-Siebold-Str. 3, 37075, Göttingen, Germany.,Department of Bioengineering, Imperial College London, SW7 2AZ, London, UK
| | - Strahinja Dosen
- Department for Trauma Surgery, Orthopedics and Plastic Surgery, University Medical Center Göttingen, Von-Siebold-Str. 3, 37075, Göttingen, Germany.,The Faculty of Medicine, Department of Health Science and Technology Center for Sensory-Motor Interaction, Aalborg University, Aalborg, Denmark
| |
Collapse
|
9
|
Godfrey SB, Bianchi M, Bicchi A, Santello M. Influence of force feedback on grasp force modulation in prosthetic applications: a preliminary study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2016:5439-5442. [PMID: 28269488 DOI: 10.1109/embc.2016.7591957] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In typical movement, humans use a combination of feed-forward and feedback motor control strategies to interact with the world around them. However, when sensory input is impaired or absent, as in the case of various neuropathies or amputation, the ability to perform everyday tasks, like modulating grip force to object weight, can be affected. In this study, we show the results of a preliminary study using a pressure cuff-like force feedback device (CUFF) with the SoftHand Pro (SHP) prosthetic hand. Subjects lifted an object of various weights using their own hand, with the SHP without feedback, and the SHP with force feedback. As expected, significant differences were found between the two SHP conditions and the native hand, but surprisingly not between the SHP conditions. A closer look at the data suggests the feedback may help diminish the overall grip force used during grasping even if it does not alter the grip force modulation to object weight. The lack of significance may be due in part to high intra- and inter-subject variability. Additional training with the CUFF and/or customization of the feedback may enhance the effects and warrants further study.
Collapse
|
10
|
Markovic M, Karnal H, Graimann B, Farina D, Dosen S. GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses. J Neural Eng 2017; 14:036007. [PMID: 28355147 DOI: 10.1088/1741-2552/aa620a] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
OBJECTIVE Providing sensory feedback to the user of the prosthesis is an important challenge. The common approach is to use tactile stimulation, which is easy to implement but requires training and has limited information bandwidth. In this study, we propose an alternative approach based on augmented reality. APPROACH We have developed the GLIMPSE, a Google Glass application which connects to the prosthesis via a Bluetooth interface and renders the prosthesis states (EMG signals, aperture, force and contact) using augmented reality (see-through display) and sound (bone conduction transducer). The interface was tested in healthy subjects that used the prosthesis with (FB group) and without (NFB group) feedback during a modified clothespins test that allowed us to vary the difficulty of the task. The outcome measures were the number of unsuccessful trials, the time to accomplish the task, and the subjective ratings of the relevance of the feedback. MAIN RESULTS There was no difference in performance between FB and NFB groups in the case of a simple task (basic, same-color clothespins test), but the feedback significantly improved the performance in a more complex task (pins of different resistances). Importantly, the GLIMPSE feedback did not increase the time to accomplish the task. Therefore, the supplemental feedback might be useful in the tasks which are more demanding, and thereby less likely to benefit from learning and feedforward control. The subjects integrated the supplemental feedback with the intrinsic sources (vision and muscle proprioception), developing their own idiosyncratic strategies to accomplish the task. SIGNIFICANCE The present study demonstrates a novel self-contained, ready-to-deploy, wearable feedback interface. The interface was successfully tested and was proven to be feasible and functionally beneficial. The GLIMPSE can be used as a practical solution but also as a general and flexible instrument to investigate closed-loop prosthesis control.
Collapse
Affiliation(s)
- Marko Markovic
- Institute of Neurorehabilitation Systems, Clinics for Trauma, Orthopedic and Plastic Surgery, University Medical Center Göttingen, Georg-August University, 37075 Göttingen, Germany
| | | | | | | | | |
Collapse
|
11
|
Brown JD, Paek A, Syed M, O'Malley MK, Shewokis PA, Contreras-Vidal JL, Davis AJ, Gillespie RB. An exploration of grip force regulation with a low-impedance myoelectric prosthesis featuring referred haptic feedback. J Neuroeng Rehabil 2015; 12:104. [PMID: 26602538 PMCID: PMC4659194 DOI: 10.1186/s12984-015-0098-1] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2015] [Accepted: 11/16/2015] [Indexed: 11/17/2022] Open
Abstract
Background Haptic display technologies are well suited to relay proprioceptive, force, and contact cues from a prosthetic terminal device back to the residual limb and thereby reduce reliance on visual feedback. The ease with which an amputee interprets these haptic cues, however, likely depends on whether their dynamic signal behavior corresponds to expected behaviors—behaviors consonant with a natural limb coupled to the environment. A highly geared motor in a terminal device along with the associated high back-drive impedance influences dynamic interactions with the environment, creating effects not encountered with a natural limb. Here we explore grasp and lift performance with a backdrivable (low backdrive impedance) terminal device placed under proportional myoelectric position control that features referred haptic feedback. Methods We fabricated a back-drivable terminal device that could be used by amputees and non-amputees alike and drove aperture (or grip force, when a stiff object was in its grasp) in proportion to a myoelectric signal drawn from a single muscle site in the forearm. In randomly ordered trials, we assessed the performance of N=10 participants (7 non-amputee, 3 amputee) attempting to grasp and lift an object using the terminal device under three feedback conditions (no feedback, vibrotactile feedback, and joint torque feedback), and two object weights that were indiscernible by vision. Results Both non-amputee and amputee participants scaled their grip force according to the object weight. Our results showed only minor differences in grip force, grip/load force coordination, and slip as a function of sensory feedback condition, though the grip force at the point of lift-off for the heavier object was significantly greater for amputee participants in the presence of joint torque feedback. An examination of grip/load force phase plots revealed that our amputee participants used larger safety margins and demonstrated less coordination than our non-amputee participants. Conclusions Our results suggest that a backdrivable terminal device may hold advantages over non-backdrivable devices by allowing grip/load force coordination consistent with behaviors observed in the natural limb. Likewise, the inconclusive effect of referred haptic feedback on grasp and lift performance suggests the need for additional testing that includes adequate training for participants.
Collapse
Affiliation(s)
- Jeremy D Brown
- Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA. .,Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA.
| | - Andrew Paek
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA.
| | - Mashaal Syed
- School of Biomedical Engineering, Science and Health Systems (BIOMED), Drexel University, Philadelphia, PA, USA.
| | - Marcia K O'Malley
- Department of Mechanical Engineering, Rice University, Houston, TX, USA.
| | - Patricia A Shewokis
- School of Biomedical Engineering, Science and Health Systems (BIOMED), Drexel University, Philadelphia, PA, USA. .,Nutrition Sciences Department, College of Nursing and Health Professions, Drexel University, Philadelphia, PA, USA. .,Department of Surgery, Drexel University College of Medicine, Philadelphia, PA, USA.
| | - Jose L Contreras-Vidal
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA.
| | - Alicia J Davis
- UM Orthotics and Prosthetics Center, University of Michigan, Ann Arbor, MI, USA.
| | - R Brent Gillespie
- Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
12
|
Schofield JS, Evans KR, Carey JP, Hebert JS. Applications of sensory feedback in motorized upper extremity prosthesis: a review. Expert Rev Med Devices 2014; 11:499-511. [PMID: 24928327 DOI: 10.1586/17434440.2014.929496] [Citation(s) in RCA: 101] [Impact Index Per Article: 10.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Dexterous hand movement is possible due to closed loop control dependent on efferent motor output and afferent sensory feedback. This control strategy is significantly altered in those with upper limb amputation as sensations of touch and movement are inherently lost. For upper limb prosthetic users, the absence of sensory feedback impedes efficient use of the prosthesis and is highlighted as a major factor contributing to user rejection of myoelectric prostheses. Numerous sensory feedback systems have been proposed in literature to address this gap in prosthetic control; however, these systems have yet to be implemented for long term use. Methodologies for communicating prosthetic grasp and touch information are reviewed, including discussion of selected designs and test results. With a focus on clinical and translational challenges, this review highlights and compares techniques employed to provide amputees with sensory feedback. Additionally, promising future directions are discussed and highlighted.
Collapse
Affiliation(s)
- Jonathon S Schofield
- Department of Mechanical Engineering, University of Alberta, 6-23 Mechanical Engineering, Edmonton, AB T6G 2G8, Canada
| | | | | | | |
Collapse
|
13
|
Bouwsema H, van der Sluis CK, Bongers RM. Effect of feedback during virtual training of grip force control with a myoelectric prosthesis. PLoS One 2014; 9:e98301. [PMID: 24865570 PMCID: PMC4035328 DOI: 10.1371/journal.pone.0098301] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2013] [Accepted: 04/30/2014] [Indexed: 01/12/2023] Open
Abstract
The aim of this study was to determine whether virtual training improves grip force control in prosthesis use, and to examine which type of augmented feedback facilitates its learning most. Thirty-two able-bodied participants trained grip force with a virtual ball-throwing game for five sessions in a two-week period, using a myoelectric simulator. They received either feedback on movement outcome or on movement execution. Sixteen controls received training that did not focus on force control. Variability over learning was examined with the Tolerance-Noise-Covariation approach, and the transfer of grip force control was assessed in five test-tasks that assessed different aspects of force control in a pretest, a posttest and a retention test. During training performance increased while the variability in performance was decreased, mainly by reduction in noise. Grip force control only improved in the test-tasks that provided information on performance. Starting the training with a task that required low force production showed no transfer of the learned grip force. Feedback on movement execution was detrimental to grip force control, whereas feedback on movement outcome enhanced transfer of grip force control to tasks other than trained. Clinical implications of these results regarding virtual training of grip force control are discussed.
Collapse
Affiliation(s)
- Hanneke Bouwsema
- University of Groningen, University Medical Center Groningen, Center for Human Movement Sciences, Groningen, The Netherlands
- * E-mail:
| | - Corry K. van der Sluis
- University Medical Center Groningen, Center for Rehabilitation, Groningen, The Netherlands
| | - Raoul M. Bongers
- University of Groningen, University Medical Center Groningen, Center for Human Movement Sciences, Groningen, The Netherlands
| |
Collapse
|
14
|
Virtual grasping: closed-loop force control using electrotactile feedback. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2014; 2014:120357. [PMID: 24516504 PMCID: PMC3909980 DOI: 10.1155/2014/120357] [Citation(s) in RCA: 39] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2013] [Accepted: 11/20/2013] [Indexed: 11/29/2022]
Abstract
Closing the control loop by providing somatosensory feedback to the user of a prosthesis is a well-known, long standing challenge in the field of prosthetics. Various approaches have been investigated for feedback restoration, ranging from direct neural stimulation to noninvasive sensory substitution methods. Although there are many studies presenting closed-loop systems, only a few of them objectively evaluated the closed-loop performance, mostly using vibrotactile stimulation. Importantly, the conclusions about the utility of the feedback were partly contradictory. The goal of the current study was to systematically investigate the capability of human subjects to control grasping force in closed loop using electrotactile feedback. We have developed a realistic experimental setup for virtual grasping, which operated in real time, included a set of real life objects, as well as a graphical and dynamical model of the prosthesis. We have used the setup to test 10 healthy, able bodied subjects to investigate the role of training, feedback and feedforward control, robustness of the closed loop, and the ability of the human subjects to generalize the control to previously “unseen” objects. Overall, the outcomes of this study are very optimistic with regard to the benefits of feedback and reveal various, practically relevant, aspects of closed-loop control.
Collapse
|
15
|
Abstract
BACKGROUND Upper limb amputees have no direct sense of the grip force applied by a prosthetic hand; thus, precise control of the applied grip force is difficult for amputees. Since there is little object deformation when rigid objects are grasped, it is difficult for amputees to visually gauge the applied grip force in this situation. OBJECTIVES To determine if the applied grip force from a prosthetic hand can be visually displayed and used to more efficaciously grasp objects. STUDY DESIGN Experimental controlled trial. METHODS Force feedback is used in the control algorithm for the prosthetic hand and supplied visually to the user through a bicolor LED experimentally mounted to the thumb. Several experiments are performed by able-bodied test subjects to rate the usefulness of the additional visual feedback when manipulating a clearly visible, brittle object that can break if grasped too firmly. A hybrid force-velocity sliding mode controller is used with and without additional visual force feedback supplied to the operators. RESULTS Subjective evaluations and success rates from the test subjects indicate a statistically significant reduction in breaking the grasped object when using the prosthesis with the extra visual feedback. CONCLUSIONS The additional visual force feedback can effectively facilitate the manipulation of brittle objects. Clinical relevance The novel approach of this research is the implementation of a noninvasive, effective and economic technique to visually indicate the grip force applied by a prosthetic hand to upper limb amputees. This technique provides a statistically significant improvement when handling brittle objects.
Collapse
Affiliation(s)
- Erik D Engeberg
- University of Akron, Mechanical Engineering Department, ASEC, Room 113 Akron, OH 44325-3903, USA. engeberg@uakron
| | | |
Collapse
|
16
|
Saunders I, Vijayakumar S. The role of feed-forward and feedback processes for closed-loop prosthesis control. J Neuroeng Rehabil 2011; 8:60. [PMID: 22032545 PMCID: PMC3227590 DOI: 10.1186/1743-0003-8-60] [Citation(s) in RCA: 114] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2011] [Accepted: 10/27/2011] [Indexed: 12/01/2022] Open
Abstract
Background It is widely believed that both feed-forward and feed-back mechanisms are required for successful object manipulation. Open-loop upper-limb prosthesis wearers receive no tactile feedback, which may be the cause of their limited dexterity and compromised grip force control. In this paper we ask whether observed prosthesis control impairments are due to lack of feedback or due to inadequate feed-forward control. Methods Healthy subjects were fitted with a closed-loop robotic hand and instructed to grasp and lift objects of different weights as we recorded trajectories and force profiles. We conducted three experiments under different feed-forward and feed-back configurations to elucidate the role of tactile feedback (i) in ideal conditions, (ii) under sensory deprivation, and (iii) under feed-forward uncertainty. Results (i) We found that subjects formed economical grasps in ideal conditions. (ii) To our surprise, this ability was preserved even when visual and tactile feedback were removed. (iii) When we introduced uncertainty into the hand controller performance degraded significantly in the absence of either visual or tactile feedback. Greatest performance was achieved when both sources of feedback were present. Conclusions We have introduced a novel method to understand the cognitive processes underlying grasping and lifting. We have shown quantitatively that tactile feedback can significantly improve performance in the presence of feed-forward uncertainty. However, our results indicate that feed-forward and feed-back mechanisms serve complementary roles, suggesting that to improve on the state-of-the-art in prosthetic hands we must develop prostheses that empower users to correct for the inevitable uncertainty in their feed-forward control.
Collapse
Affiliation(s)
- Ian Saunders
- Institute of Perception, Action and Behaviour, School of Informatics, University of Edinburgh, UK.
| | | |
Collapse
|
17
|
Effect of biofeedback accompanying occupational therapy and functional electrical stimulation in hemiplegic patients. Int J Rehabil Res 2008; 31:33-41. [DOI: 10.1097/mrr.0b013e3282f4524c] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
18
|
Buma DG, Buitenweg JR, Veltink PH. Intermittent stimulation delays adaptation to electrocutaneous sensory feedback. IEEE Trans Neural Syst Rehabil Eng 2007; 15:435-41. [PMID: 17894276 DOI: 10.1109/tnsre.2007.903942] [Citation(s) in RCA: 40] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Electrotactile displays deliver information to the user by means of electrocutaneous stimulation. If such displays are used in prostheses, the functionality depends on long term stability of this information channel. The perceived sensation, however, decays within 15 min due to central adaptation if the stimulation is applied continuously and at constant strength. In this study, the effects of stimulus amplitude and intermittent stimulation on adaptation were investigated in ten healthy subjects. The perceived sensation was recorded during 15 min of constant stimulation using a visual analog scale (VAS). The sensation level with time thus measured were parameterized by the initial sensation level, the time constant of decay and the end sensation level after fitting of an exponential function through the VAS data. The time constant increased significantly when applying a high stimulation level (at 80% of the range between sensation and pain thresholds) if compared with lower levels of stimulation (20% and 50%) during continuous stimulation. Intermittent stimulation at this high stimulation level significantly increased end sensation level.
Collapse
Affiliation(s)
- Dorindo G Buma
- Biomedical Signals and Systems Group, Institute for Biomedical Technology BMTI/Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE Enschede, The Netherlands
| | | | | |
Collapse
|
19
|
Carpaneto J, Micera S, Zaccone F, Vecchi F, Dario P. A sensorized thumb for force closed-loop control of hand neuroprostheses. IEEE Trans Neural Syst Rehabil Eng 2004; 11:346-53. [PMID: 14960109 DOI: 10.1109/tnsre.2003.819938] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In this paper, we presented a sensorized thumb based on a matrix of piezoresistive force sensors, with an acquisition unit and a special wearing support. The sensor was calibrated and then the device was tested during different tasks simulating activities of daily living performed by seven able-bodied subjects. By means of these experiments, we verified that the device proposed can be used to extract force information during grasp. In fact, the device was able to provide useful force information in the 98% of the trials with a good repeatability during all the different conditions. Moreover, we evaluated the patterns obtained during the different grasping tasks. The palmar grasps were performed in a similar manner, whereas the lateral pinch and the spherical volar grip were more different. This device can provide force information with good performance and acceptability and it can be used for force closed-loop control of hand neuroprostheses.
Collapse
Affiliation(s)
- Jacopo Carpaneto
- Advanced Robotics Technologies and Systems Laboratory, Scuola Superiore Sant'Anna Valdera, 56025 Pisa, Italy
| | | | | | | | | |
Collapse
|
20
|
Humbert SD, Snyder SA, Grill WM. Evaluation of command algorithms for control of upper-extremity neural prostheses. IEEE Trans Neural Syst Rehabil Eng 2002; 10:94-101. [PMID: 12236452 DOI: 10.1109/tnsre.2002.1031977] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Five new command control algorithms were created to enable increased control over grasp force in upper-extremity neural prostheses. Most of these algorithms took advantage of the ability to lock or assign a steady command value to the hand neural prosthesis. Five able-bodied subjects tested the algorithms by using a shoulder controller that controlled a video-simulated hand to repeatedly complete a consistent evaluation task. A generalized estimating equations-based linear model was used to analyze the data. The algorithms were ranked via contrast analyses between the coefficient values from the linear model of the proportional control with lock algorithm, which is the algorithm presently used in neural prostheses, and each of the other algorithms. The algorithms that allowed adjustment of the command value after the hand was locked as well as algorithms that allowed a decrease in controller gain after the hand was locked performed better than the proportional control with lock algorithm. Algorithms that changed command as a function of time performed worse than the proportional control with lock algorithm. Further, the computer-based video simulator proved to be useful as a first-pass evaluation tool for neural prosthesis control.
Collapse
Affiliation(s)
- Scott D Humbert
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106-4912, USA
| | | | | |
Collapse
|