1
|
Campbell E, Eddy E, Bateman S, Côté-Allard U, Scheme E. Context-informed incremental learning improves both the performance and resilience of myoelectric control. J Neuroeng Rehabil 2024; 21:70. [PMID: 38702813 PMCID: PMC11067119 DOI: 10.1186/s12984-024-01355-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Accepted: 04/04/2024] [Indexed: 05/06/2024] Open
Abstract
Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.
Collapse
Affiliation(s)
- Evan Campbell
- Institute of Biomedical Engineering, University of new Brunswick, Dineen Dr., Fredericton, NB, E3B 5A3, Canada.
| | - Ethan Eddy
- Institute of Biomedical Engineering, University of new Brunswick, Dineen Dr., Fredericton, NB, E3B 5A3, Canada
- Spectral Lab, University of New Brunswick, Peter Kelly Dr, Fredericton, NB, E3B 5A1, Canada
| | - Scott Bateman
- Spectral Lab, University of New Brunswick, Peter Kelly Dr, Fredericton, NB, E3B 5A1, Canada
| | - Ulysse Côté-Allard
- Department of Technology Systems, University of Oslo, Gunnar Randers vei, Kjeller, P.O Box 70, Norway
| | - Erik Scheme
- Institute of Biomedical Engineering, University of new Brunswick, Dineen Dr., Fredericton, NB, E3B 5A3, Canada
| |
Collapse
|
2
|
Castañeda TS, Connan M, Capsi-Morales P, Beckerle P, Castellini C, Piazza C. Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses. J Neuroeng Rehabil 2024; 21:57. [PMID: 38627772 PMCID: PMC11020298 DOI: 10.1186/s12984-024-01352-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 04/03/2024] [Indexed: 04/19/2024] Open
Abstract
INTRODUCTION Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users' own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users' experience regarding embodiment and their interaction with different functional levels. METHODS To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey. RESULTS Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented. CONCLUSIONS Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.
Collapse
Affiliation(s)
| | - Mathilde Connan
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Oberpfaffenhofen, Germany
| | - Patricia Capsi-Morales
- Department of Computer Engineering, Technical University of Munich (TUM), Garching bei Munich, Germany.
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich (TUM), Munich, Germany.
| | - Philipp Beckerle
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
- Department of Electrical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Claudio Castellini
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Oberpfaffenhofen, Germany
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Cristina Piazza
- Department of Computer Engineering, Technical University of Munich (TUM), Garching bei Munich, Germany
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich (TUM), Munich, Germany
| |
Collapse
|
3
|
Maas B, Van Der Sluis CK, Bongers RM. Assessing the effectiveness of serious game training designed to assist in upper limb prothesis rehabilitation. FRONTIERS IN REHABILITATION SCIENCES 2024; 5:1353077. [PMID: 38348457 PMCID: PMC10859406 DOI: 10.3389/fresc.2024.1353077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/09/2023] [Accepted: 01/16/2024] [Indexed: 02/15/2024]
Abstract
Introduction Controlling a myoelectric upper limb prosthesis is difficult, therefore training is required. Since training with serious games showed promising results, the current paper focuses on game design and its effectivity for transfer between in-game skill to actual prosthesis use for proportional control of hand opening and control of switching between grips. We also examined training duration and individual differences. Method Thirty-six participants were randomly assigned to one of three groups: a task-specific serious game training group, a non-task-specific serious game training group and a control group. Each group performed a pre-test, mid-test and a post-test with five training sessions between each test moment. Test sessions assessed proportional control using the Cylinder test, a test designed to measure scaling of hand aperture during grabbing actions, and the combined use of proportional and switch control using the Clothespin Relocation Test, part of the Southampton Hand Assessment Procedure and Tray Test. Switch control was assessed during training by measuring amplitude difference and phasing of co-contraction triggers. Results Differences between groups over test sessions were observed for proportional control tasks, however there was lack of structure in these findings. Maximum aperture changed with test moment and some participants adjusted maximum aperture for smaller objects. For proportional and switch control tasks no differences between groups were observed. The effect of test moment suggests a testing effect. For learning switch control, an overall improvement across groups was found in phasing of the co-contraction peaks. Importantly, individual differences were found in all analyses. Conclusion As improvements over test sessions were found, but no relevant differences between groups were revealed, we conclude that transfer effects from game training to actual prosthesis use did not take place. Task specificity nor training duration had effects on outcomes. Our results imply testing effects instead of transfer effects, in which individual differences played a significant role. How transfer from serious game training in upper limb prosthesis use can be enhanced, needs further attention.
Collapse
Affiliation(s)
- Bart Maas
- Department of Human Movement Sciences, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
| | - Corry K. Van Der Sluis
- Department of Rehabilitation Medicine, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
| | - Raoul M. Bongers
- Department of Human Movement Sciences, University Medical Center Groningen, University of Groningen, Groningen, Netherlands
| |
Collapse
|
4
|
Li W, Shi P, Li S, Yu H. Current status and clinical perspectives of extended reality for myoelectric prostheses: review. Front Bioeng Biotechnol 2024; 11:1334771. [PMID: 38260728 PMCID: PMC10800532 DOI: 10.3389/fbioe.2023.1334771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 12/13/2023] [Indexed: 01/24/2024] Open
Abstract
Training with "Extended Reality" or X-Reality (XR) systems can undoubtedly enhance the control of the myoelectric prostheses. However, there is no consensus on which factors improve the efficiency of skill transfer from virtual training to actual prosthesis abilities. This review examines the current status and clinical applications of XR in the field of myoelectric prosthesis training and analyses possible influences on skill migration. We have conducted a thorough search on databases in the field of prostheses using keywords such as extended reality, virtual reality and serious gaming. Our scoping review encompassed relevant applications, control methods, performance evaluation and assessment metrics. Our findings indicate that the implementation of XR technology for myoelectric rehabilitative training on prostheses provides considerable benefits. Additionally, there are numerous standardised methods available for evaluating training effectiveness. Recently, there has been a surge in the number of XR-based training tools for myoelectric prostheses, with an emphasis on user engagement and virtual training evaluation. Insufficient attention has been paid to significant limitations in the behaviour, functionality, and usage patterns of XR and myoelectric prostheses, potentially obstructing the transfer of skills and prospects for clinical application. Improvements are recommended in four critical areas: activities of daily living, training strategies, feedback, and the alignment of the virtual environment with the physical devices.
Collapse
Affiliation(s)
- Wei Li
- Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Assistive Devices, Shanghai, China
- Key Laboratory of Neural-Functional Information and Rehabilitation Engineering of the Ministry of Civil Affairs, Shanghai, China
| | - Ping Shi
- Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Assistive Devices, Shanghai, China
- Key Laboratory of Neural-Functional Information and Rehabilitation Engineering of the Ministry of Civil Affairs, Shanghai, China
| | - Sujiao Li
- Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Assistive Devices, Shanghai, China
- Key Laboratory of Neural-Functional Information and Rehabilitation Engineering of the Ministry of Civil Affairs, Shanghai, China
| | - Hongliu Yu
- Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Assistive Devices, Shanghai, China
- Key Laboratory of Neural-Functional Information and Rehabilitation Engineering of the Ministry of Civil Affairs, Shanghai, China
| |
Collapse
|
5
|
Egle F, Di Domenico D, Marinelli A, Boccardo N, Canepa M, Laffranchi M, De Michieli L, Castellini C. Preliminary Assessment of Two Simultaneous and Proportional Myocontrol Methods for 3-DoFs Prostheses Using Incremental Learning. IEEE Int Conf Rehabil Robot 2023; 2023:1-6. [PMID: 37941277 DOI: 10.1109/icorr58425.2023.10304813] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2023]
Abstract
Despite progressive developments over the last decades, current upper limb prostheses still lack a suitable control able to fully restore the functionalities of the lost arm. Traditional control approaches for prostheses fail when simultaneously actuating multiple Degrees of Freedom (DoFs), thus limiting their usability in daily-life scenarios. Machine learning, on the one hand, offers a solution to this issue through a promising approach for decoding user intentions but fails when input signals change. Incremental learning, on the other hand, reduces sources of error by quickly updating the model on new data rather than training the control model from scratch. In this study, we present an initial evaluation of a position and a velocity control strategy for simultaneous and proportional control over 3-DoFs based on incremental learning. The proposed controls are tested using a virtual Hannes prosthesis on two healthy participants. The performances are evaluated over eight sessions by performing the Target Achievement Control test and administering SUS and NASA-TLX questionnaires. Overall, this preliminary study demonstrates that both control strategies are promising approaches for prosthetic control, offering the potential to improve the usability of prostheses for individuals with limb loss. Further research extended to a wider population of both healthy subjects and amputees will be essential to thoroughly assess these control paradigms.
Collapse
|
6
|
Hashim NA, Abd Razak NA, Shanmuganathan T, Jaladin RA, Gholizadeh H, Abu Osman NA. On the use of virtual reality for individuals with upper limb loss: a systematic scoping review. Eur J Phys Rehabil Med 2022; 58:612-620. [PMID: 35044131 PMCID: PMC9987328 DOI: 10.23736/s1973-9087.22.06794-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2021] [Revised: 09/02/2021] [Accepted: 01/03/2022] [Indexed: 11/08/2022]
Abstract
INTRODUCTION Virtual reality has recently become a popular application for rehabilitation and motor control research. This technology has emerged as a valid addition to conventional therapy and promises a successful rehabilitation. This study describes recent research related to the use of virtual reality applications in the rehabilitation of individuals with upper limb loss and to see whether this technology has enough proof of its applicability. EVIDENCE ACQUISITION Searches were conducted with the Web of Science, Google Scholar, IEEE Xplore, and PubMed databases from inception up to September 2020. Articles that employed virtual reality in the rehabilitation of individual with upper limb loss were included in the research if it is written in English, the keyword exists in the title and abstract; it uses visual feedback in nonimmersive, semi-immersive, or fully immersive virtual environments. Data extraction was carried out by two independent researchers. The study was drafted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocols (PRISMA). EVIDENCE SYNTHESIS A total of 38 articles met the inclusion criteria. Most studies were published between 2010 and 2020. Thirty-nine percent of the studies (N.=15), originates from North America; 55% of the studies (N.=21), were publicly funded; 61% of the studies (N.=24), was without disclosure of conflict of interest; 82% of the studies (N.=31), were cited in other studies. All the studies were published in journals and conference proceedings. Sixty-six percent of the studies (N.=25) has come out with positive outcome. The design studies were mostly case reports, case series, and poorly designed cohort studies that made up 55% (N.=21) of all the studies cited here. CONCLUSIONS The research conducted on the use of virtual reality in individual with upper limb loss rehabilitation is of very low quality. The improvements to the research protocol are much needed. It is not necessary to develop new devices, but rather to assess existing devices with well-conducted randomized controlled trials.
Collapse
Affiliation(s)
- Nur A Hashim
- Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia
| | - Nasrul A Abd Razak
- Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia -
| | | | - Rafidah A Jaladin
- Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia
| | | | - Noor A Abu Osman
- Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia
| |
Collapse
|
7
|
Virtual/Augmented Reality for Rehabilitation Applications Using Electromyography as Control/Biofeedback: Systematic Literature Review. ELECTRONICS 2022. [DOI: 10.3390/electronics11142271] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Virtual reality (VR) and augmented reality (AR) are engaging interfaces that can be of benefit for rehabilitation therapy. However, they are still not widely used, and the use of surface electromyography (sEMG) signals is not established for them. Our goal is to explore whether there is a standardized protocol towards therapeutic applications since there are not many methodological reviews that focus on sEMG control/feedback. A systematic literature review using the PRISMA (preferred reporting items for systematic reviews and meta-analyses) methodology is conducted. A Boolean search in databases was performed applying inclusion/exclusion criteria; articles older than 5 years and repeated were excluded. A total of 393 articles were selected for screening, of which 66.15% were excluded, 131 records were eligible, 69.46% use neither VR/AR interfaces nor sEMG control; 40 articles remained. Categories are, application: neurological motor rehabilitation (70%), prosthesis training (30%); processing algorithm: artificial intelligence (40%), direct control (20%); hardware: Myo Armband (22.5%), Delsys (10%), proprietary (17.5%); VR/AR interface: training scene model (25%), videogame (47.5%), first-person (20%). Finally, applications are focused on motor neurorehabilitation after stroke/amputation; however, there is no consensus regarding signal processing or classification criteria. Future work should deal with proposing guidelines to standardize these technologies for their adoption in clinical practice.
Collapse
|
8
|
Garcia GJ, Alepuz A, Balastegui G, Bernat L, Mortes J, Sanchez S, Vera E, Jara CA, Morell V, Pomares J, Ramon JL, Ubeda A. ARMIA: A Sensorized Arm Wearable for Motor Rehabilitation. BIOSENSORS 2022; 12:bios12070469. [PMID: 35884272 PMCID: PMC9313425 DOI: 10.3390/bios12070469] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 06/22/2022] [Accepted: 06/27/2022] [Indexed: 01/20/2023]
Abstract
In this paper, we present ARMIA: a sensorized arm wearable that includes a combination of inertial and sEMG sensors to interact with serious games in telerehabilitation setups. This device reduces the cost of robotic assistance technologies to be affordable for end-users at home and at rehabilitation centers. Hardware and acquisition software specifications are described together with potential applications of ARMIA in real-life rehabilitation scenarios. A detailed comparison with similar medical technologies is provided, with a specific focus on wearable devices and virtual and augmented reality approaches. The potential advantages of the proposed device are also described showing that ARMIA could provide similar, if not better, the effectivity of physical therapy as well as giving the possibility of home-based rehabilitation.
Collapse
|
9
|
Gaballa A, Cavalcante RS, Lamounier E, Soares A, Cabibihan JJ. Extended Reality "X-Reality" for Prosthesis Training of Upper-Limb Amputees: A Review on Current and Future Clinical Potential. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1652-1663. [PMID: 35635835 DOI: 10.1109/tnsre.2022.3179327] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The rejection rates of upper-limb prosthetic devices in adults are high, currently averaging 26% and 23% for body-powered and electric devices, respectively. While many factors influence acceptance, prosthesis training methods relying on novel virtual reality systems have been cited as a critical factor capable of increasing the likelihood of long-term, full-time use. Despite that, these implementations have not yet garnered widespread traction in the clinical setting, and their use remains immaterial. This review aims to explore the reasons behind this situation by identifying trends in existing research that seek to advance Extended Reality "X-Reality" systems for the sake of upper-limb prosthesis rehabilitation and, secondly, analyzing barriers and presenting potential pathways to deployment for successful adoption in the future. The search yielded 42 research papers that were divided into two categories. The first category included articles that focused on the technical aspect of virtual prosthesis training. Articles in the second category utilize user evaluation procedures to ensure applicability in a clinical environment. The review showed that 75% of articles that conducted whole system testing experimented with non-immersive virtual systems. Furthermore, there is a shortage of experiments performed with amputee subjects. From the large-scale studies analyzed, 71% of those recruited solely able-bodied participants. This paper shows that X-Reality technologies for prosthesis rehabilitation of upper-limb amputees carry significant benefits. Nevertheless, much still must be done so that the technology reaches widespread clinical use.
Collapse
|
10
|
Gentile C, Cordella F, Zollo L. Hierarchical Human-Inspired Control Strategies for Prosthetic Hands. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22072521. [PMID: 35408135 PMCID: PMC9003226 DOI: 10.3390/s22072521] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 03/02/2022] [Accepted: 03/23/2022] [Indexed: 05/14/2023]
Abstract
The abilities of the human hand have always fascinated people, and many studies have been devoted to describing and understanding a mechanism so perfect and important for human activities. Hand loss can significantly affect the level of autonomy and the capability of performing the activities of daily life. Although the technological improvements have led to the development of mechanically advanced commercial prostheses, the control strategies are rather simple (proportional or on/off control). The use of these commercial systems is unnatural and not intuitive, and therefore frequently abandoned by amputees. The components of an active prosthetic hand are the mechatronic device, the decoding system of human biological signals into gestures and the control law that translates all the inputs into desired movements. The real challenge is the development of a control law replacing human hand functions. This paper presents a literature review of the control strategies of prosthetics hands with a multiple-layer or hierarchical structure, and points out the main critical aspects of the current solutions, in terms of human's functions replicated with the prosthetic device. The paper finally provides several suggestions for designing a control strategy able to mimic the functions of the human hand.
Collapse
Affiliation(s)
- Cosimo Gentile
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, 00128 Rome, Italy; (F.C.); (L.Z.)
- INAIL Prosthetic Center, Vigorso di Budrio, 40054 Bologna, Italy
- Correspondence:
| | - Francesca Cordella
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, 00128 Rome, Italy; (F.C.); (L.Z.)
| | - Loredana Zollo
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, 00128 Rome, Italy; (F.C.); (L.Z.)
| |
Collapse
|
11
|
Wang H, Huang P, Xu T, Li G, Hu Y. Towards zero retraining for multiday motion recognition via a fully unsupervised adaptive approach and fabric myoelectric armband. IEEE Trans Neural Syst Rehabil Eng 2022; 30:217-225. [PMID: 35041609 DOI: 10.1109/tnsre.2022.3144323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Surface electromyogram pattern recognition (EMG-PR) requires time-consuming training and retraining procedures for long-term use, hindering the usability of myoelectric control. In this paper, we design a fabric myoelectric armband to reduce the electrode shifts. Furthermore, we propose a fully unsupervised adaptive approach called hybrid serial classifier (HSC) to eliminate the burden of retraining over multiply days. We investigated the performance of our approach with a dataset of ten types of forearm motion from ten male subjects over eight weeks (total ten days, including: from day 1 to day 7, day 14, day 28, day 56). The average inter-day classification accuracies of HSC without any new retraining data are 86.61% when trained exclusively with the first day's EMG data, and 94.77% when trained with other nine days' data. We compare our proposed HSC algorithm with linear discriminant analysis (LDA) without recalibration (BLDA) and supervised adaption LDA (ALDA) with just one trial of new retraining data. The inter-day classification accuracy of HSC is significantly higher than that of BLDA and ALDA. These results indicate that our novel armband sEMG device is feasible for long-term use in conjunction with the proposed HSC algorithm.
Collapse
|
12
|
Garske CA, Dyson M, Dupan S, Morgan G, Nazarpour K. Serious Games Are Not Serious Enough for Myoelectric Prosthetics. JMIR Serious Games 2021; 9:e28079. [PMID: 34747715 PMCID: PMC8663510 DOI: 10.2196/28079] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Revised: 06/09/2021] [Accepted: 08/25/2021] [Indexed: 02/07/2023] Open
Abstract
Serious games show a lot of potential for use in movement rehabilitation (eg, after a stroke, injury to the spinal cord, or limb loss). However, the nature of this research leads to diversity both in the background of the researchers and in the approaches of their investigation. Our close examination and categorization of virtual training software for upper limb prosthetic rehabilitation found that researchers typically followed one of two broad approaches: (1) focusing on the game design aspects to increase engagement and muscle training and (2) concentrating on an accurate representation of prosthetic training tasks, to induce task-specific skill transfer. Previous studies indicate muscle training alone does not lead to improved prosthetic control without a transfer-enabling task structure. However, the literature shows a recent surge in the number of game-based prosthetic training tools, which focus on engagement without heeding the importance of skill transfer. This influx appears to have been strongly influenced by the availability of both software and hardware, specifically the launch of a commercially available acquisition device and freely available high-profile game development engines. In this Viewpoint, we share our perspective on the current trends and progress of serious games for prosthetic training.
Collapse
Affiliation(s)
- Christian Alexander Garske
- Intelligent Sensing Laboratory, School of Engineering, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Matthew Dyson
- Intelligent Sensing Laboratory, School of Engineering, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Sigrid Dupan
- Edinburgh Neuroprosthetics Laboratory, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
| | - Graham Morgan
- Networked and Ubiquitous Systems Engineering Group, School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Kianoush Nazarpour
- Edinburgh Neuroprosthetics Laboratory, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
| |
Collapse
|
13
|
Spieker V, Ganguly A, Haddadin S, Piazza C. An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition. SENSORS (BASEL, SWITZERLAND) 2021; 21:7404. [PMID: 34770709 PMCID: PMC8587119 DOI: 10.3390/s21217404] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 10/29/2021] [Accepted: 11/01/2021] [Indexed: 11/16/2022]
Abstract
Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.
Collapse
Affiliation(s)
- Veronika Spieker
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Amartya Ganguly
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Sami Haddadin
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Cristina Piazza
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
- Department of Informatics, Technical University of Munich, 85748 Garching bei München, Germany
| |
Collapse
|
14
|
Karczewski AM, Dingle AM, Poore SO. The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements. Front Neurorobot 2021; 15:711028. [PMID: 34366820 PMCID: PMC8334559 DOI: 10.3389/fnbot.2021.711028] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 06/22/2021] [Indexed: 11/21/2022] Open
Abstract
Over the last few decades there has been a push to enhance the use of advanced prosthetics within the fields of biomedical engineering, neuroscience, and surgery. Through the development of peripheral neural interfaces and invasive electrodes, an individual's own nervous system can be used to control a prosthesis. With novel improvements in neural recording and signal decoding, this intimate communication has paved the way for bidirectional and intuitive control of prostheses. While various collaborations between engineers and surgeons have led to considerable success with motor control and pain management, it has been significantly more challenging to restore sensation. Many of the existing peripheral neural interfaces have demonstrated success in one of these modalities; however, none are currently able to fully restore limb function. Though this is in part due to the complexity of the human somatosensory system and stability of bioelectronics, the fragmentary and as-yet uncoordinated nature of the neuroprosthetic industry further complicates this advancement. In this review, we provide a comprehensive overview of the current field of neuroprosthetics and explore potential strategies to address its unique challenges. These include exploration of electrodes, surgical techniques, control methods, and prosthetic technology. Additionally, we propose a new approach to optimizing prosthetic limb function and facilitating clinical application by capitalizing on available resources. It is incumbent upon academia and industry to encourage collaboration and utilization of different peripheral neural interfaces in combination with each other to create versatile limbs that not only improve function but quality of life. Despite the rapidly evolving technology, if the field continues to work in divided "silos," we will delay achieving the critical, valuable outcome: creating a prosthetic limb that is right for the patient and positively affects their life.
Collapse
Affiliation(s)
| | - Aaron M. Dingle
- Division of Plastic Surgery, Department of Surgery, University of Wisconsin–Madison, Madison, WI, United States
| | | |
Collapse
|
15
|
Pan L, Huang H(H. A robust model-based neural-machine interface across different loading weights applied at distal forearm. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102509] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
16
|
Walsh KA, Sanford SP, Collins BD, Harel NY, Nataraj R. Performance potential of classical machine learning and deep learning classifiers for isometric upper-body myoelectric control of direction in virtual reality with reduced muscle inputs. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
17
|
Boschmann A, Neuhaus D, Vogt S, Kaltschmidt C, Platzner M, Dosen S. Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis. J Neuroeng Rehabil 2021; 18:25. [PMID: 33541376 PMCID: PMC7860185 DOI: 10.1186/s12984-021-00822-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Accepted: 01/12/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training. METHODS In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback. RESULTS The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7). CONCLUSION The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development.
Collapse
Affiliation(s)
- Alexander Boschmann
- Computer Engineering Group, Department of Computer Science, Faculty of Computer Science, Electrical Engineering and Mathematics, Paderborn University, Paderborn, Germany.
| | - Dorothee Neuhaus
- Exercise Science & Neuroscience Unit, Department Exercise and Health, Faculty of Science, Paderborn University, Paderborn, Germany
| | - Sarah Vogt
- Exercise Science & Neuroscience Unit, Department Exercise and Health, Faculty of Science, Paderborn University, Paderborn, Germany
| | - Christian Kaltschmidt
- Exercise Science & Neuroscience Unit, Department Exercise and Health, Faculty of Science, Paderborn University, Paderborn, Germany
| | - Marco Platzner
- Computer Engineering Group, Department of Computer Science, Faculty of Computer Science, Electrical Engineering and Mathematics, Paderborn University, Paderborn, Germany
| | - Strahinja Dosen
- Department of Health Science and Technology, Faculty of Medicine, Aalborg University, Aalborg, Denmark
| |
Collapse
|
18
|
Mick S, Segas E, Dure L, Halgand C, Benois-Pineau J, Loeb GE, Cattaert D, de Rugy A. Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand. J Neuroeng Rehabil 2021; 18:3. [PMID: 33407618 PMCID: PMC7789560 DOI: 10.1186/s12984-020-00793-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Accepted: 12/01/2020] [Indexed: 11/20/2022] Open
Abstract
Background Prosthetic restoration of reach and grasp function after a trans-humeral amputation requires control of multiple distal degrees of freedom in elbow, wrist and fingers. However, such a high level of amputation reduces the amount of available myoelectric and kinematic information from the residual limb. Methods To overcome these limits, we added contextual information about the target’s location and orientation such as can now be extracted from gaze tracking by computer vision tools. For the task of picking and placing a bottle in various positions and orientations in a 3D virtual scene, we trained artificial neural networks to predict postures of an intact subject’s elbow, forearm and wrist (4 degrees of freedom) either solely from shoulder kinematics or with additional knowledge of the movement goal. Subjects then performed the same tasks in the virtual scene with distal joints predicted from the context-aware network. Results Average movement times of 1.22s were only slightly longer than the naturally controlled movements (0.82 s). When using a kinematic-only network, movement times were much longer (2.31s) and compensatory movements from trunk and shoulder were much larger. Integrating contextual information also gave rise to motor synergies closer to natural joint coordination. Conclusions Although notable challenges remain before applying the proposed control scheme to a real-world prosthesis, our study shows that adding contextual information to command signals greatly improves prediction of distal joint angles for prosthetic control.
Collapse
Affiliation(s)
- Sébastien Mick
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France.
| | - Effie Segas
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France
| | - Lucas Dure
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France
| | - Christophe Halgand
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France
| | - Jenny Benois-Pineau
- Laboratoire Bordelais de Recherche en Informatique, UMR 5800, CNRS, Univ. Bordeaux and Bordeaux INP, 351 cours de la Libération, 33405, Talence, France
| | - Gerald E Loeb
- Department of Biomedical Engineering, Univ. Southern California, 1042 Downey Way, Los Angeles, CA, 90089, USA
| | - Daniel Cattaert
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France
| | - Aymar de Rugy
- Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, CNRS and Univ. Bordeaux, 146 rue Léo Saignat, 33076, Bordeaux, France.,Centre for Sensorimotor Performance, School of Human Movement and Nutrition Sciences, Univ. Queensland, Blair Drive, Brisbane, QLD, 4059, Australia
| |
Collapse
|
19
|
Hashim NA, Razak NAA, Osman NAA. Comparison of Conventional and Virtual Reality Box and Blocks Tests in Upper Limb Amputees: A Case-Control Study. IEEE ACCESS 2021; 9:76983-76990. [DOI: 10.1109/access.2021.3072988] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
|
20
|
Gigli A, Brusamento D, Meattini R, Melchiorri C, Castellini C. Feedback-aided data acquisition improves myoelectric control of a prosthetic hand. J Neural Eng 2020; 17:056047. [PMID: 33022665 DOI: 10.1088/1741-2552/abbed0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Pattern-recognition-based myocontrol can be unreliable, which may limit its use in the clinical practice and everyday activities. One cause for this is the poor generalization of the underlying machine learning models to untrained conditions. Acquiring the training data and building the model more interactively can reduce this problem. For example, the user could be encouraged to target the model's instabilities during the data acquisition supported by automatic feedback guidance. Interactivity is an emerging trend in myocontrol of upper-limb electric prostheses: the user should be actively involved throughout the training and usage of the device. APPROACH In this study, 18 non-disabled participants tested two novel feedback-aided acquisition protocols against a standard one that did not provide any guidance. All the protocols acquired data dynamically in multiple arm positions to counteract the limb position effect. During feedback-aided acquisition, an acoustic signal urged the participant to hover with the arm in specific regions of her peri-personal space, de facto acquiring more data where needed. The three protocols were compared on everyday manipulation tasks performed with a prosthetic hand. MAIN RESULTS Our results showed that feedback-aided data acquisition outperformed the acquisition routine without guidance, both objectively and subjectively. SIGNIFICANCE This indicates that the interaction with the user during the data acquisition is fundamental to improve myocontrol.
Collapse
Affiliation(s)
- Andrea Gigli
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | | | | | | | | |
Collapse
|
21
|
Kristoffersen MB, Franzke AW, van der Sluis CK, Murgia A, Bongers RM. Serious gaming to generate separated and consistent EMG patterns in pattern-recognition prosthesis control. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.102140] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
|
22
|
Kanoga S, Kanemura A, Asoh H. Are armband sEMG devices dense enough for long-term use?—Sensor placement shifts cause significant reduction in recognition accuracy. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.101981] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
23
|
Teh Y, Hargrove LJ. Understanding Limb Position and External Load Effects on Real-Time Pattern Recognition Control in Amputees. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1605-1613. [PMID: 32396094 PMCID: PMC7391097 DOI: 10.1109/tnsre.2020.2991643] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Limb position is a factor that negatively affects myoelectric pattern recognition classification accuracy. However, prior studies evaluating impact on real-time control for upper-limb amputees have done so without a physical prosthesis on the residual limb. It remains unclear how limb position affects real-time pattern recognition control in amputees when their residual limb is supporting various weights. We used a virtual reality target achievement control test to evaluate the effects of limb position and external load on real-time pattern recognition control in fourteen intact limb subjects and six major upper limb amputee subjects. We also investigated how these effects changed based on different control system training methods. In a static training method, subjects kept their unloaded arm by their side with the elbow bent whereas in the dynamic training method, subjects moved their arm throughout a workspace while supporting a load. When static training was used, limb position significantly affected real-time control in all subjects. However, amputee subjects were still able to adequately complete tasks in all conditions, even in untrained limb positions. Moreover, increasing external loads decreased controller performance, albeit to a lesser extent in amputee subjects. The effects of limb position did not change as load increased, and vice versa. In intact limb subjects, dynamic training significantly reduced the limb position effect but did not completely remove them. In contrast, in amputee subjects, dynamic training eliminated the limb position effect in three out of four outcome measures. However, it did not reduce the effects of load for either subject population. These findings suggest that results obtained from intact limb subjects may not generalize to amputee subjects and that advanced training methods can substantially improve controller robustness to different limb positions regardless of limb loading.
Collapse
|
24
|
Gigli A, Gijsberts A, Castellini C. The Merits of Dynamic Data Acquisition for Realistic Myocontrol. Front Bioeng Biotechnol 2020; 8:361. [PMID: 32426344 PMCID: PMC7203421 DOI: 10.3389/fbioe.2020.00361] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2019] [Accepted: 03/31/2020] [Indexed: 11/13/2022] Open
Abstract
Natural myocontrol is the intuitive control of a prosthetic limb via the user's voluntary muscular activations. This type of control is usually implemented by means of pattern recognition, which uses a set of training data to create a model that can decipher these muscular activations. A consequence of this approach is that the reliability of a myocontrol system depends on how representative this training data is for all types of signal variability that may be encountered when the amputee puts the prosthesis into real use. Myoelectric signals are indeed known to vary according to the position and orientation of the limb, among other factors, which is why it has become common practice to take this variability into account by acquiring training data in multiple body postures. To shed further light on this problem, we compare two ways of collecting data: while the subjects hold their limb statically in several positions one at a time, which is the traditional way, or while they dynamically move their limb at a constant pace through those same positions. Since our interest is to investigate any differences when controlling an actual prosthetic device, we defined an evaluation protocol that consisted of a series of complex, bimanual daily-living tasks. Fourteen intact participants performed these tasks while wearing prosthetic hands mounted on splints, which were controlled via either a statically or dynamically built myocontrol model. In both cases all subjects managed to complete all tasks and participants without previous experience in myoelectric control manifested a significant learning effect; moreover, there was no significant difference in the task completion times achieved with either model. When evaluated in a simulated scenario with traditional offline performance evaluation, on the other hand, the dynamically-trained system showed significantly better accuracy. Regardless of the setting, the dynamic data acquisition was faster, less tiresome, and better accepted by the users. We conclude that dynamic data acquisition is advantageous and confirm the limited relevance of offline analyses for online myocontrol performance.
Collapse
Affiliation(s)
- Andrea Gigli
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Weßling, Germany
| | - Arjan Gijsberts
- Vandal Laboratory, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Claudio Castellini
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Weßling, Germany
| |
Collapse
|
25
|
Wolf EJ, Cruz TH, Emondi AA, Langhals NB, Naufel S, Peng GCY, Schulz BW, Wolfson M. Advanced technologies for intuitive control and sensation of prosthetics. Biomed Eng Lett 2020; 10:119-128. [PMID: 32175133 PMCID: PMC7046895 DOI: 10.1007/s13534-019-00127-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Accepted: 07/31/2019] [Indexed: 02/06/2023] Open
Abstract
The Department of Defense, Department of Veterans Affairs and National Institutes of Health have invested significantly in advancing prosthetic technologies over the past 25 years, with the overall intent to improve the function, participation and quality of life of Service Members, Veterans, and all United States Citizens living with limb loss. These investments have contributed to substantial advancements in the control and sensory perception of prosthetic devices over the past decade. While control of motorized prosthetic devices through the use of electromyography has been widely available since the 1980s, this technology is not intuitive. Additionally, these systems do not provide stimulation for sensory perception. Recent research has made significant advancement not only in the intuitive use of electromyography for control but also in the ability to provide relevant meaningful perceptions through various stimulation approaches. While much of this previous work has traditionally focused on those with upper extremity amputation, new developments include advanced bidirectional neuroprostheses that are applicable to both the upper and lower limb amputation. The goal of this review is to examine the state-of-the-science in the areas of intuitive control and sensation of prosthetic devices and to discuss areas of exploration for the future. Current research and development efforts in external systems, implanted systems, surgical approaches, and regenerative approaches will be explored.
Collapse
Affiliation(s)
- Erik J. Wolf
- Clinical and Rehabilitative Medicine Research Program, US Army Medical Research and Development Command, Fort Detrick, MD 21702 USA
| | - Theresa H. Cruz
- National Institute of Child Health and Human Development, National Institute of Health, Bethesda, MD 20817 USA
| | - Alfred A. Emondi
- Defense Advanced Research Projects Agency, Arlington, VA 22203 USA
| | - Nicholas B. Langhals
- National Institute of Neurological Disorders and Stroke, National Institute of Health, Bethesda, MD 20892 USA
| | | | - Grace C. Y. Peng
- National Institute of Biomedical Imaging and Bioengineering, National Institute of Health, Bethesda, MD 20817 USA
| | - Brian W. Schulz
- VA Office of Research and Development, Washington, DC 20002 USA
| | - Michael Wolfson
- National Institute of Biomedical Imaging and Bioengineering, National Institute of Health, Bethesda, MD 20817 USA
| |
Collapse
|
26
|
Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation. SENSORS 2019; 19:s19204596. [PMID: 31652616 PMCID: PMC6832440 DOI: 10.3390/s19204596] [Citation(s) in RCA: 100] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/18/2019] [Revised: 10/17/2019] [Accepted: 10/19/2019] [Indexed: 11/17/2022]
Abstract
Upper limb amputation is a condition that significantly restricts the amputees from performing their daily activities. The myoelectric prosthesis, using signals from residual stump muscles, is aimed at restoring the function of such lost limbs seamlessly. Unfortunately, the acquisition and use of such myosignals are cumbersome and complicated. Furthermore, once acquired, it usually requires heavy computational power to turn it into a user control signal. Its transition to a practical prosthesis solution is still being challenged by various factors particularly those related to the fact that each amputee has different mobility, muscle contraction forces, limb positional variations and electrode placements. Thus, a solution that can adapt or otherwise tailor itself to each individual is required for maximum utility across amputees. Modified machine learning schemes for pattern recognition have the potential to significantly reduce the factors (movement of users and contraction of the muscle) affecting the traditional electromyography (EMG)-pattern recognition methods. Although recent developments of intelligent pattern recognition techniques could discriminate multiple degrees of freedom with high-level accuracy, their efficiency level was less accessible and revealed in real-world (amputee) applications. This review paper examined the suitability of upper limb prosthesis (ULP) inventions in the healthcare sector from their technical control perspective. More focus was given to the review of real-world applications and the use of pattern recognition control on amputees. We first reviewed the overall structure of pattern recognition schemes for myo-control prosthetic systems and then discussed their real-time use on amputee upper limbs. Finally, we concluded the paper with a discussion of the existing challenges and future research recommendations.
Collapse
|
27
|
Pan L, Crouch DL, Huang H. Comparing EMG-Based Human-Machine Interfaces for Estimating Continuous, Coordinated Movements. IEEE Trans Neural Syst Rehabil Eng 2019; 27:2145-2154. [PMID: 31478862 DOI: 10.1109/tnsre.2019.2937929] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Electromyography (EMG)-based interfaces are trending toward continuous, simultaneous control with multiple degrees of freedom. Emerging methods range from data-driven approaches to biomechanical model-based methods. However, there has been no direct comparison between these two types of continuous EMG-based interfaces. The aim of this study was to compare a musculoskeletal model (MM) with two data-driven approaches, linear regression (LR) and artificial neural network (ANN), for predicting continuous wrist and hand motions for EMG-based interfaces. Six able-bodied subjects and one transradial amputee subject performed (missing) metacarpophalangeal (MCP) and wrist flexion/extension, simultaneously or independently, while four EMG signals were recorded from forearm muscles. To add variation to the EMG signals, the subjects repeated the MCP and wrist motions at various upper extremity postures. For each subject, the EMG signals collected from the neutral posture were used to build the EMG interfaces; the EMG signals collected from all postures were used to evaluate the interfaces. The performance of the interface was quantified by Pearson's correlation coefficient (r) and the normalized root mean square error (NRMSE) between measured and estimated joint angles. The results demonstrated that the MM predicted movements more accurately, with higher r values and lower NRMSE, than either LR or ANN. Similar results were observed in the transradial amputee. Additionally, the variation in r across postures, an indicator of reliability against posture changes, was significantly lower (better) for the MM than for either LR or ANN. Our findings suggest that incorporating musculoskeletal knowledge into EMG-based human-machine interfaces could improve the estimation of continuous, coordinated motion.
Collapse
|