Brain-actuated gait trainer with visual and proprioceptive feedback

Boosting lower-limb motor imagery performance through an ensemble method for gait rehabilitation

Lower-limb exoskeletons have been used extensively in many rehabilitation applications to assist disabled people with their therapies. Brain-machine interfaces (BMIs) further provide effective and natural control schemes. However, the limited performance of brain signal decoding from lower-limb kinematics restricts the broad growth of both BMI and rehabilitation industry. To address these challenges, we propose an ensemble method for lower-limb motor imagery (MI) classification. The proposed model employs multiple techniques to boost performance, including deep and shallow parts. Traditional wavelet transformation followed by filter-bank common spatial pattern (CSP) employs neurophysiologically reasonable patterns, while multi-head self-attention (MSA) followed by temporal convolutional network (TCN) extracts deeper encoded generalized patterns. Experimental results in a customized lower-limb exoskeleton on 8 subjects in 3 consecutive sessions showed that the proposed method achieved 60.27% and 64.20% for three (MI of left leg, MI of right leg, and rest) and two classes (lower-limb MI vs. rest), respectively. Besides, the proposed model achieves improvements of up to 4% and 2% accuracy for the subject-specific and subject-independent modes compared to the current state-of-the-art (SOTA) techniques, respectively. Finally, feature analysis was conducted to show discriminative brain patterns in each MI task and sessions with different feedback modalities. The proposed models integrated in the brain-actuated lower-limb exoskeleton established a potential BMI for gait training and neuroprosthesis.

Electroencephalography Signal Analysis for Human Activities Classification: A Solution Based on Machine Learning and Motor Imagery

Electroencephalography (EEG) is a fundamental tool for understanding the brain's electrical activity related to human motor activities. Brain-Computer Interface (BCI) uses such electrical activity to develop assistive technologies, especially those directed at people with physical disabilities. However, extracting signal features and patterns is still complex, sometimes delegated to machine learning (ML) algorithms. Therefore, this work aims to develop a ML based on the Random Forest algorithm to classify EEG signals from subjects performing real and imagery motor activities. The interpretation and correct classification of EEG signals allow the development of tools controlled by cognitive processes. We evaluated our ML Random Forest algorithm using a consumer and a research-grade EEG system. Random Forest efficiently distinguishes imagery and real activities and defines the related body part, even with consumer-grade EEG. However, interpersonal variability of the EEG signals negatively affects the classification process.

Cortical processing during robot and functional electrical stimulation

Introduction

Like alpha rhythm, the somatosensory mu rhythm is suppressed in the presence of somatosensory inputs by implying cortical excitation. Sensorimotor rhythm (SMR) can be classified into two oscillatory frequency components: mu rhythm (8-13 Hz) and beta rhythm (14-25 Hz). The suppressed/enhanced SMR is a neural correlate of cortical activation related to efferent and afferent movement information. Therefore, it would be necessary to understand cortical information processing in diverse movement situations for clinical applications.

Methods

In this work, the EEG of 10 healthy volunteers was recorded while fingers were moved passively under different kinetic and kinematic conditions for proprioceptive stimulation. For the kinetics aspect, afferent brain activity (no simultaneous volition) was compared under two conditions of finger extension: (1) generated by an orthosis and (2) generated by the orthosis simultaneously combined and assisted with functional electrical stimulation (FES) applied at the forearm muscles related to finger extension. For the kinematic aspect, the finger extension was divided into two phases: (1) dynamic extension and (2) static extension (holding the extended position).

Results

In the kinematic aspect, both mu and beta rhythms were more suppressed during a dynamic than a static condition. However, only the mu rhythm showed a significant difference between kinetic conditions (with and without FES) affected by attention to proprioception after transitioning from dynamic to static state, but the beta rhythm was not.

Discussion

Our results indicate that mu rhythm was influenced considerably by muscle kinetics during finger movement produced by external devices, which has relevant implications for the design of neuromodulation and neurorehabilitation interventions.

Deep Convolutional Neural Network for EEG-Based Motor Decoding

Brain-machine interfaces (BMIs) have been applied as a pattern recognition system for neuromodulation and neurorehabilitation. Decoding brain signals (e.g., EEG) with high accuracy is a prerequisite to building a reliable and practical BMI. This study presents a deep convolutional neural network (CNN) for EEG-based motor decoding. Both upper-limb and lower-limb motor imagery were detected from this end-to-end learning with four datasets. An average classification accuracy of 93.36 ± 1.68% was yielded on the four datasets. We compared the proposed approach with two other models, i.e., multilayer perceptron and the state-of-the-art framework with common spatial patterns and support vector machine. We observed that the performance of the CNN-based framework was significantly better than the other two models. Feature visualization was further conducted to evaluate the discriminative channels employed for the decoding. We showed the feasibility of the proposed architecture to decode motor imagery from raw EEG data without manually designed features. With the advances in the fields of computer vision and speech recognition, deep learning can not only boost the EEG decoding performance but also help us gain more insight from the data, which may further broaden the knowledge of neuroscience for brain mapping.

Past, Present, and Future of EEG-Based BCI Applications

An electroencephalography (EEG)-based brain-computer interface (BCI) is a system that provides a pathway between the brain and external devices by interpreting EEG. EEG-based BCI applications have initially been developed for medical purposes, with the aim of facilitating the return of patients to normal life. In addition to the initial aim, EEG-based BCI applications have also gained increasing significance in the non-medical domain, improving the life of healthy people, for instance, by making it more efficient, collaborative and helping develop themselves. The objective of this review is to give a systematic overview of the literature on EEG-based BCI applications from the period of 2009 until 2019. The systematic literature review has been prepared based on three databases PubMed, Web of Science and Scopus. This review was conducted following the PRISMA model. In this review, 202 publications were selected based on specific eligibility criteria. The distribution of the research between the medical and non-medical domain has been analyzed and further categorized into fields of research within the reviewed domains. In this review, the equipment used for gathering EEG data and signal processing methods have also been reviewed. Additionally, current challenges in the field and possibilities for the future have been analyzed.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

A body-weight-supported visual feedback system for gait recovering in stroke patients: A randomized controlled study

OBJECTIVE

The aim of this study was to determine the effectiveness of a novel body-weight-supported (BWS) gait training system with visual feedback, called Copernicus® (Rehalife, Italy). This computerized device provides comfortable, regular and repeatable locomotion in hemiplegic patients. Through visual real-time monitoring of gait parameters, patients are trained to transfer weight loading alternately on both feet.

DESIGN

A single-blind, randomized controlled study. A single center used a computer-generated randomization code to allocate treatments.

SETTING

Intensive rehabilitation unit (IRU) at the Institute S. Anna (Italy).

PARTICIPANTS

63 first-ever stroke patients (39 men, age: 66.1 ± 9.6 years; 61.6 % with left-sided lesion) randomly distributed into three demographically/clinically matched groups.

TREATMENTS

All groups were treated five times a week for 2 -h sessions for six consecutive weeks. The first group ("control") underwent a conventional physical therapy; the second group performed advanced BWS gait training sessions without visual feedback (Experimental VF- group); whereas the third group used BWS with visual feedback stimulation (Experimental VF+ group).

MAIN OUTCOME MEASURES

Absolute changes were recorded using conventional clinical scales and kinematic measurement of static gait balance from baseline to follow-up.

RESULTS

Significant interaction Group*Time effects scales (F_2,126 = 5.1, p-level = 0.005, η²_p = 0.25; F_2,126 = 4.7, p-level = 0.007, η²_p = 0.19; respectively) were detected in the Functional Independence Measure and Tinetti-Balance scales. Post hoc analysis demonstrated that the recovery of motor functioning was greater for the VF + group with respect to other groups (all p's ≤ 0.001). A similar pattern of findings was also obtained with a stabilometric analysis, demonstrating a better clinical improvement in static balance after VF + treatment.

CONCLUSION

The proposed advanced rehabilitation system with visual feedback was more effective in improving gait recovery with respect to conventional and high-tech therapies without a sensor feedback.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

Decoding Imagined 3D Arm Movement Trajectories From EEG to Control Two Virtual Arms-A Pilot Study

Background: Realization of online control of an artificial or virtual arm using information decoded from EEG normally occurs by classifying different activation states or voluntary modulation of the sensorimotor activity linked to different overt actions of the subject. However, using a more natural control scheme, such as decoding the trajectory of imagined 3D arm movements to move a prosthetic, robotic, or virtual arm has been reported in a limited amount of studies, all using offline feed-forward control schemes.

Objective: In this study, we report the first attempt to realize online control of two virtual arms generating movements toward three targets/arm in 3D space. The 3D trajectory of imagined arm movements was decoded from power spectral density of mu, low beta, high beta, and low gamma EEG oscillations using multiple linear regression. The analysis was performed on a dataset recorded from three subjects in seven sessions wherein each session comprised three experimental blocks: an offline calibration block and two online feedback blocks. Target classification accuracy using predicted trajectories of the virtual arms was computed and compared with results of a filter-bank common spatial patterns (FBCSP) based multi-class classification method involving mutual information (MI) selection and linear discriminant analysis (LDA) modules.

Main Results: Target classification accuracy from predicted trajectory of imagined 3D arm movements in the offline runs for two subjects (mean 45%, std 5%) was significantly higher (p < 0.05) than chance level (33.3%). Nevertheless, the accuracy during real-time control of the virtual arms using the trajectory decoded directly from EEG was in the range of chance level (33.3%). However, the results of two subjects show that false-positive feedback may increase the accuracy in closed-loop. The FBCSP based multi-class classification method distinguished imagined movements of left and right arm with reasonable accuracy for two of the three subjects (mean 70%, std 5% compared to 50% chance level). However, classification of the imagined arm movement toward three targets was not successful with the FBCSP classifier as the achieved accuracy (mean 33%, std 5%) was similar to the chance level (33.3%). Sub-optimal components of the multi-session experimental paradigm were identified, and an improved paradigm proposed.

EEG-Based Lower-Limb Movement Onset Decoding: Continuous Classification and Asynchronous Detection

Brain-machine interfaces have been used to incorporate the user intention to trigger robotic devices by decoding movement onset from electroencephalography. Active neural participation is crucial to promote brain plasticity thus to enhance the opportunity of motor recovery. This paper presents the decoding of lower-limb movement-related cortical potentials with continuous classification and asynchronous detection. We executed experiments in a customized gait trainer, where 10 healthy subjects performed self-initiated ankle plantar flexion. We further analyzed the features, evaluated the impact of the limb side, and compared the proposed framework with other typical decoding methods. No significant differences were observed between the left and right legs in terms of neural signatures of movement and classification performance. We obtained a higher true positive rate, lower false positives, and comparable latencies with respect to the existing online detection methods. This paper demonstrates the feasibility of the proposed framework to build a closed-loop gait trainer. Potential applications include gait training neurorehabilitation in clinical trials.

Decoding of Self-paced Lower-Limb Movement Intention: A Case Study on the Influence Factors

Brain-machine interfaces (BMIs) have been applied as new rehabilitation tools for motor disabled individuals. Active involvement of cerebral activity has been shown to enhance neuroplasticity and thus to restore mobility. Various studies have focused on the detection of upper-limb movement intention, while the fewer study has investigated the lower-limb movement intention decoding. This study presents a BMI to decode the self-paced lower-limb movement intention, with 10 healthy subjects participating in the experiment. We varied four influence factors including the movement type (dorsiflexion or plantar flexion), the limb side (left or right leg), the processing method (time-series analysis based on MRCP, i.e., movement-related cortical potential or frequency-domain estimation based on SMR, i.e., sensory motor rhythm) and the frequency band (e.g., delta, theta, mu, beta and MRCP band at [0.1 1] Hz), to estimate both single-trial and sample-based performance. Feature analysis was then conducted to show the discriminant power (DP) and brain modulations. The average detection latency was -0.334 ± 0.216 s in single-trial basis across all conditions. An average area under the curve (AUC) of 91.0 ± 3.5% and 68.2 ± 4.6% was obtained for the MRCP-based and SMR-based method in the classification, respectively. The best performance was yielded from plantar flexion with left leg using time-series analysis on the MRCP band. The feature analysis indicated a cross-subject consistency of DP with the MRCP-based method and subject-specific variance of DP with the SMR-based method. The results presented here might be further exploited in a rehabilitation scenario. The comprehensive factor analysis might be used to shed light on the design of an effective brain switch to trigger external robotic devices.

A brain-controlled lower-limb exoskeleton for human gait training

Brain-computer interfaces have been a novel approach to translate human intentions into movement commands in robotic systems. This paper describes an electroencephalogram-based brain-controlled lower-limb exoskeleton for gait training, as a proof of concept towards rehabilitation with human-in-the-loop. Instead of using conventional single electroencephalography correlates, e.g., evoked P300 or spontaneous motor imagery, we propose a novel framework integrated two asynchronous signal modalities, i.e., sensorimotor rhythms (SMRs) and movement-related cortical potentials (MRCPs). We executed experiments in a biologically inspired and customized lower-limb exoskeleton where subjects (N = 6) actively controlled the robot using their brain signals. Each subject performed three consecutive sessions composed of offline training, online visual feedback testing, and online robot-control recordings. Post hoc evaluations were conducted including mental workload assessment, feature analysis, and statistics test. An average robot-control accuracy of 80.16% ± 5.44% was obtained with the SMR-based method, while estimation using the MRCP-based method yielded an average performance of 68.62% ± 8.55%. The experimental results showed the feasibility of the proposed framework with all subjects successfully controlled the exoskeleton. The current paradigm could be further extended to paraplegic patients in clinical trials.