Portable Take-Home System Enables Proportional Control and High-Resolution Data Logging With a Multi-Degree-of-Freedom Bionic Arm

An efficient surface electromyography-based gesture recognition algorithm based on multiscale fusion convolution and channel attention

In the field of rehabilitation, although deep learning have been widely used in multitype gesture recognition via surface electromyography (sEMG), their higher algorithmic complexity often leads to low computationally inefficient, which compromise their practicality. To achieve more efficient multitype recognition, We propose the Residual-Inception-Efficient (RIE) model, which integrates Inception and efficient channel attention (ECA). The Inception, which is a multiscale fusion convolutional module, is adopted to enhance the ability to extract sEMG features. It uses fast dimensionality reduction, asymmetric convolution decomposition, and pooling to suppress the accumulation of parameters, then reducing the algorithmic complexity; The ECA is adopted to reweight the output features of Inception in different channels, enabling the RIE model to focus on information that is more relevant to gestures. 52-, 49-, and 52-class gesture recognition experiments are conducted on NinaPro DB1, DB3, and DB4 datasets, which contain 14,040, 3234, and 3120 gesture samples, respectively. RIE model proposed in this study achieves accuracies of 88.27%, 69.52%, and 84.55% on the three datasets, exhibiting excellent recognition accuracy and strong generalization ability. Moreover, this method reduces the algorithmic complexity from both spatial and temporal aspects, rendering it smaller in size and faster in computation compared to other lightweight algorithms. Therefore, the proposed RIE model possesses both lightweight computational requirements and reliable performance, providing an efficient deep learning method for gesture recognition based on sEMG.

Internet of Things for beyond-the-laboratory prosthetics research

Research on upper-limb prostheses is typically laboratory-based. Evidence indicates that research has not yet led to prostheses that meet user needs. Inefficient communication loops between users, clinicians and manufacturers limit the amount of quantitative and qualitative data that researchers can use in refining their innovations. This paper offers a first demonstration of an alternative paradigm by which remote, beyond-the-laboratory prosthesis research according to user needs is feasible. Specifically, the proposed Internet of Things setting allows remote data collection, real-time visualization and prosthesis reprogramming through Wi-Fi and a commercial cloud portal. Via a dashboard, the user can adjust the configuration of the device and append contextual information to the prosthetic data. We evaluated this demonstrator in real-time experiments with three able-bodied participants. Results promise the potential of contextual data collection and system update through the internet, which may provide real-life data for algorithm training and reduce the complexity of send-home trials. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Monitoring at-home prosthesis control improvements through real-time data logging

OBJECTIVE

Validating the ability for advanced prostheses to improve function beyond the laboratory remains a critical step in enabling long-term benefits for prosthetic limb users.

APPROACH

A nine week take-home case study was completed with a single participant with upper limb amputation and osseointegration (OI) to better understand how an advanced prosthesis is used during daily activities. The participant was already an expert prosthesis user and used the Modular Prosthetic Limb (MPL) at home during the study. The MPL was controlled using wireless electromyography (EMG) pattern recognition-based movement decoding. Clinical assessments were performed before and after the take-home portion of the study. Data was recorded using an onboard data log in order to measure daily prosthesis usage, sensor data, and EMG data.

MAIN RESULT

The participant's continuous prosthesis usage steadily increased (p = 0.04, max = 5.5 hr) over time and over 30% of the total time was spent actively controlling the prosthesis. The duration of prosthesis usage after each pattern recognition training session also increased over time (p = 0.04), resulting in up to 5.4 hr of usage before retraining the movement decoding algorithm. Pattern recognition control accuracy improved (1.2% per week, p < 0.001) with a maximum number of 10 classes trained at once and the transitions between different degrees of freedom increased as the study progressed, indicating smooth and efficient control of the advanced prosthesis. Variability of decoding accuracy also decreased with prosthesis usage (p < 0.001) and 30% of the time was spent performing a prosthesis movement. During clinical evaluations, Box and Blocks and the Assessment of the Capacity for Myoelectric Control (ACMC) scores increased by 43% and 6.2%, respectively, demonstrating prosthesis functionality and the NASA Task Load Index (NASA-TLX) scores decreased, on average, by 25% across assessments, indicating reduced cognitive workload while using the MPL, over the nine week study.

SIGNIFICANCE

In this case study, we demonstrate that an onboard system to monitor prosthesis usage enables better understanding of how prostheses are incorporated into daily life. That knowledge can support the long-term goal of completely restoring independence and quality of life to individuals living with upper limb amputation.

Towards User-Centred Prosthetics Research Beyond the Laboratory

The purpose of this study was to explore a range of perspectives on how academic research and clinical assessment of upper-limb prosthetics could happen in environments outside of laboratories and clinics, such as within peoples' homes. Two co-creation workshops were held, which included people who use upper limb prosthetic devices (hereafter called users), clinicians, academics, a policy stakeholder, and a representative from the upper-limb prosthetics industry (hereafter called professionals). The discussions during the workshops indicate that research and clinical assessment conducted remotely from a laboratory or clinic could inform future solutions that address user needs. Users were open to the idea of sharing sensor and contextual data from within their homes to external laboratories during research studies. However, this was dependent upon several considerations, such as choice and control over data collection. Regarding clinical assessment, users had reservations of how data may be used to inform future prosthetic prescriptions whilst, clinicians were concerned with resource implications and capacity to process user data. The paper presents findings of the discussions shared by participants during both workshops. The paper concludes with a conjecture that collecting sensor and contextual data from users within their home environment will contribute towards literature within the field, and potentially inform future care policies for upper limb prosthetics. The involvement of users during such studies will be critical and can be enabled via a co-creation approach. In the short term, this may be achieved through academic research studies, which may in the long term inform a framework for clinical in-home trials and clinical remote assessment.

Robust Torque Predictions From Electromyography Across Multiple Levels of Active Exoskeleton Assistance Despite Non-linear Reorganization of Locomotor Output

Robotic exoskeletons can assist humans with walking by providing supplemental torque in proportion to the user's joint torque. Electromyographic (EMG) control algorithms can estimate a user's joint torque directly using real-time EMG recordings from the muscles that generate the torque. However, EMG signals change as a result of supplemental torque from an exoskeleton, resulting in unreliable estimates of the user's joint torque during active exoskeleton assistance. Here, we present an EMG control framework for robotic exoskeletons that provides consistent joint torque predictions across varying levels of assistance. Experiments with three healthy human participants showed that using diverse training data (from different levels of assistance) enables robust torque predictions, and that a convolutional neural network (CNN), but not a Kalman filter (KF), can capture the non-linear transformations in EMG due to exoskeleton assistance. With diverse training, the CNN could reliably predict joint torque from EMG during zero, low, medium, and high levels of exoskeleton assistance [root mean squared error (RMSE) below 0.096 N-m/kg]. In contrast, without diverse training, RMSE of the CNN ranged from 0.106 to 0.144 N-m/kg. RMSE of the KF ranged from 0.137 to 0.182 N-m/kg without diverse training, and did not improve with diverse training. When participant time is limited, training data should emphasize the highest levels of assistance first and utilize at least 35 full gait cycles for the CNN. The results presented here constitute an important step toward adaptive and robust human augmentation via robotic exoskeletons. This work also highlights the non-linear reorganization of locomotor output when using assistive exoskeletons; significant reductions in EMG activity were observed for the soleus and gastrocnemius, and a significant increase in EMG activity was observed for the erector spinae. Control algorithms that can accommodate spatiotemporal changes in muscle activity have broad implications for exoskeleton-based assistance and rehabilitation following neuromuscular injury.

Portable System for Home Use Enables Closed-Loop, Continuous Control of Multi-Degree-of-Freedom Bionic Arm

Commercial prosthetic hands are frequently abandoned due to unintuitive control methods and a lack of sensory feedback from the prosthesis. Advanced neuromyoelectric prostheses can restore intuitive control and sensory feedback to prosthesis users and potentially reduce abandonment. However, not all advanced prosthetic systems are deployable for home use on portable systems with limited computational power. In this work, we use a commercially available portable neural interface processor (the Ripple Neuro Nomad), and a multi-degree-of-freedom bionic arm (the DEKA LUKE Arm) to create a closed-loop neuromyoelectric prosthesis. The system restores intuitive, independent, continuous control over the arm's six-degrees-of-freedom and provides sensory feedback for up to 288 neural and six vibrotactile channels. Additionally, the large storage capacity of the system enables high-resolution logging of EMG, hand positions, prosthesis sensors, and stimulation parameters. We developed two GUIs enabling wireless, real-time adjustments to motor control and feedback parameters: one with nearly full control over motor control and feedback parameters for investigators, and one with restricted capabilities enabling end-user safety. We verified the system's closed-loop function through a fragile egg task with vibrotactile sensory feedback. We tested the neural stimulation with an amplifier capable of eliciting transcutaneous percepts. This neuromyoelectric prosthetic system will be used for an extended take-home trial that could provide strong clinical justification for advanced, closed-loop prostheses.Clinical Relevance- This work establishes an advanced, intuitive, sensorized prosthesis that can be used in home and clinical settings.

A portable, self-contained neuroprosthetic hand with deep learning-based finger control

Objective.Deep learning-based neural decoders have emerged as the prominent approach to enable dexterous and intuitive control of neuroprosthetic hands. Yet few studies have materialized the use of deep learning in clinical settings due to its high computational requirements.Approach.Recent advancements of edge computing devices bring the potential to alleviate this problem. Here we present the implementation of a neuroprosthetic hand with embedded deep learning-based control. The neural decoder is designed based on the recurrent neural network architecture and deployed on the NVIDIA Jetson Nano-a compacted yet powerful edge computing platform for deep learning inference. This enables the implementation of the neuroprosthetic hand as a portable and self-contained unit with real-time control of individual finger movements.Main results.A pilot study with a transradial amputee is conducted to evaluate the proposed system using peripheral nerve signals acquired from implanted intrafascicular microelectrodes. The preliminary experiment results show the system's capabilities of providing robust, high-accuracy (95%-99%) and low-latency (50-120 ms) control of individual finger movements in various laboratory and real-world environments.Conclusion.This work is a technological demonstration of modern edge computing platforms to enable the effective use of deep learning-based neural decoders for neuroprosthesis control as an autonomous system.Significance.The proposed system helps pioneer the deployment of deep neural networks in clinical applications underlying a new class of wearable biomedical devices with embedded artificial intelligence.Clinical trial registration: DExterous Hand Control Through Fascicular Targeting (DEFT). Identifier: NCT02994160.

Acquisition of Surface EMG Using Flexible and Low-Profile Electrodes for Lower Extremity Neuroprosthetic Control

For persons with lower extremity (LE) amputation, acquisition of surface electromyography (sEMG) from within the prosthetic socket remains a significant challenge due to the dynamic loads experienced during the gait cycle. However, these signals are critical for both understanding the clinical effects of LE amputation and determining the desired control trajectories of active LE prostheses. Current solutions for collecting within-socket sEMG are generally (i) incompatible with a subject's prescribed prosthetic socket and liners, (ii) uncomfortable, and (iii) expensive. This study presents an alternative within-socket sEMG acquisition paradigm using a novel flexible and low-profile electrode. First, the practical performance of this Sub-Liner Interface for Prosthetics (SLIP) electrode is compared to that of commercial Ag/AgCl electrodes within a cohort of subjects without amputation. Then, the corresponding SLIP electrode sEMG acquisition paradigm is implemented in a single subject with unilateral transtibial amputation performing unconstrained movements and walking on level ground. Finally, a quantitative questionnaire characterizes subjective comfort for SLIP electrode and commercial Ag/AgCl electrode instrumentation setups. Quantitative analyses suggest comparable signal qualities between SLIP and Ag/AgCl electrodes while qualitative analyses suggest the feasibility of using the SLIP electrode for real-time sEMG data collection from load-bearing, ambulatory subjects with LE amputation.

Co-creation and User Perspectives for Upper Limb Prosthetics

People who either use an upper limb prosthesis and/or have used services provided by a prosthetic rehabilitation centre, experience limitations of currently available prosthetic devices. Collaboration between academia and a broad range of stakeholders, can lead to the development of solutions that address peoples' needs. By doing so, the rate of prosthetic device abandonment can decrease. Co-creation is an approach that can enable collaboration of this nature to occur throughout the research process. We present findings of a co-creation project that gained user perspectives from a user survey, and a subsequent workshop involving: people who use an upper limb prosthesis and/or have experienced care services (users), academics, industry experts, charity executives, and clinicians. The survey invited users to prioritise six themes, which academia, clinicians, and industry should focus on over the next decade. The prioritisation of the themes concluded in the following order, with the first as the most important: function, psychology, aesthetics, clinical service, collaboration, and media. Within five multi-stakeholder groups, the workshop participants discussed challenges and collaborative opportunities for each theme. Workshop groups prioritised the themes based on their discussions, to highlight opportunities for further development. Two groups chose function, one group chose clinical service, one group chose collaboration, and another group chose media. The identified opportunities are presented within the context of the prioritised themes, including the importance of transparent information flow between all stakeholders; user involvement throughout research studies; and routes to informing healthcare policy through collaboration. As the field of upper limb prosthetics moves toward in-home research, we present co-creation as an approach that can facilitate user involvement throughout the duration of such studies.

Co-Creation Facilitates Translational Research on Upper Limb Prosthetics

People who either use an upper limb prosthesis and/or have used services provided by a prosthetic rehabilitation centre, hereafter called users, are yet to benefit from the fast-paced growth in academic knowledge within the field of upper limb prosthetics. Crucially over the past decade, research has acknowledged the limitations of conducting laboratory-based studies for clinical translation. This has led to an increase, albeit rather small, in trials that gather real-world user data. Multi-stakeholder collaboration is critical within such trials, especially between researchers, users, and clinicians, as well as policy makers, charity representatives, and industry specialists. This paper presents a co-creation model that enables researchers to collaborate with multiple stakeholders, including users, throughout the duration of a study. This approach can lead to a transition in defining the roles of stakeholders, such as users, from participants to co-researchers. This presents a scenario whereby the boundaries between research and participation become blurred and ethical considerations may become complex. However, the time and resources that are required to conduct co-creation within academia can lead to greater impact and benefit the people that the research aims to serve.

Arduino-Based Myoelectric Control: Towards Longitudinal Study of Prosthesis Use

Understanding how upper-limb prostheses are used in daily life helps to improve the design and robustness of prosthesis control algorithms and prosthetic components. However, only a very small fraction of published research includes prosthesis use in community settings. The cost, limited battery life, and poor generalisation may be the main reasons limiting the implementation of home-based applications. In this work, we introduce the design of a cost-effective Arduino-based myoelectric control system with wearable electromyogram (EMG) sensors. The design considerations focused on home studies, so the robustness, user-friendly control adjustments, and user supports were the main concerns. Three control algorithms, namely, direct control, abstract control, and linear discriminant analysis (LDA) classification, were implemented in the system. In this paper, we will share our design principles and report the robustness of the system in continuous operation in the laboratory. In addition, we will show a first real-time implementation of the abstract decoder for prosthesis control with an able-bodied participant.

Long-term performance of Utah slanted electrode arrays and intramuscular electromyographic leads implanted chronically in human arm nerves and muscles

OBJECTIVE

We explore the long-term performance and stability of seven percutaneous Utah Slanted Electrode Arrays (USEAs) and intramuscular recording leads (iEMGs) implanted chronically in the residual arm nerves and muscles of three human participants as a means to permanently restore sensorimotor function after transradial amputations.

APPROACH

We quantify the number of functional recording and functional stimulating electrodes over time. We also calculate the signal-to-noise ratio (SNR) of USEA and iEMG recordings and quantify the stimulation current necessary to evoke detectable sensory percepts. Furthermore, we quantify the consistency of the sensory modality, receptive field location, and receptive field size of USEA-evoked percepts.

MAIN RESULTS

In the most recent subject, involving USEAs with technical improvements, neural recordings persisted for 502 d (entire implant duration) and the number of functional recording electrodes for one USEA increased over time. However, for six out of seven USEAs across the three participants, the number of functional recording electrodes decreased within the first 2 months after implantation. The SNR of neural recordings and electromyographic recordings stayed relatively consistent over time. Sensory percepts were consistently evoked over the span of 14 months, were not significantly different in size, and highlighted the nerves' fascicular organization. The percentage of percepts with consistent modality or consistent receptive field location between sessions (∼1 month apart) varied between 0%-86.2% and 9.1%-100%, respectively. Stimulation thresholds and electrode impedances increased initially but then remained relatively stable over time.

SIGNIFICANCE

This work demonstrates improved performance of USEAs, and provides a basis for comparing the longevity and stability of USEAs to that of other neural interfaces. USEAs provide a rich repertoire of neural recordings and sensory percepts. Although their performance still generally declines over time, functionality can persist long-term. Future work should leverage the results presented here to further improve USEA design or to develop adaptive algorithms that can maintain a high level of performance.