Decoding the ERD/ERS: influence of afferent input induced by a leg assistive robot

BCI-Based Control for Ankle Exoskeleton T-FLEX: Comparison of Visual and Haptic Stimuli with Stroke Survivors

Brain–computer interface (BCI) remains an emerging tool that seeks to improve the patient interaction with the therapeutic mechanisms and to generate neuroplasticity progressively through neuromotor abilities. Motor imagery (MI) analysis is the most used paradigm based on the motor cortex’s electrical activity to detect movement intention. It has been shown that motor imagery mental practice with movement-associated stimuli may offer an effective strategy to facilitate motor recovery in brain injury patients. In this sense, this study aims to present the BCI associated with visual and haptic stimuli to facilitate MI generation and control the T-FLEX ankle exoskeleton. To achieve this, five post-stroke patients (55–63 years) were subjected to three different strategies using T-FLEX: stationary therapy (ST) without motor imagination, motor imagination with visual stimulation (MIV), and motor imagination with visual-haptic inducement (MIVH). The quantitative characterization of both BCI stimuli strategies was made through the motor imagery accuracy rate, the electroencephalographic (EEG) analysis during the MI active periods, the statistical analysis, and a subjective patient’s perception. The preliminary results demonstrated the viability of the BCI-controlled ankle exoskeleton system with the beta rebound, in terms of patient’s performance during MI active periods and satisfaction outcomes. Accuracy differences employing haptic stimulus were detected with an average of 68% compared with the 50.7% over only visual stimulus. However, the power spectral density (PSD) did not present changes in prominent activation of the MI band but presented significant variations in terms of laterality. In this way, visual and haptic stimuli improved the subject’s MI accuracy but did not generate differential brain activity over the affected hemisphere. Hence, long-term sessions with a more extensive sample and a more robust algorithm should be carried out to evaluate the impact of the proposed system on neuronal and motor evolution after stroke.

A Novel Smart Motor Imagery Intention Human-Computer Interaction Model Using Extreme Learning Machine and EEG Signals

The brain is the central nervous system that governs human activities. However, in modern society, more and more diseases threaten the health of the brain and nerves and spinal cord, making the human brain unable to conduct normal information interaction with the outside world. The rehabilitation training of the brain-computer interface can promote the nerve repair of the sensorimotor cortex in patients with brain diseases. Therefore, the research of brain-computer interface for motor imaging is of great significance for patients with brain diseases to restore motor function. Due to the characteristics of non-stationary, nonlinear, and individual differences of EEG signals, there are still many difficulties in the analysis and classification of EEG signals at this stage. In this study, the Extreme Learning Machine (ELM) model was used to classify motor-imaging EEG signals, identify the user’s intention, and control external devices. Considering that single-modal features cannot represent the core information, this study uses a fusion feature that combines temporal and spatial features as the final feature data. The fusion features are input to the trained ELM classifier, and the final classification result is obtained. Two sets of BCI competition data in the BCI competition public database are used to verify the validity of the model. The experimental results show that the ELM model has achieved a classification accuracy of 0.7832 in the classification task of Data Sets IIb, which is higher than other comparison algorithms, and shows universal applicability among different subjects. In addition, the average recognition rate of this model in the Data Sets IIIa classification task reaches 0.8347, which has obvious advantages compared with the comparative classification algorithm. The classification effect is smaller than the classification effect obtained by the champion algorithm of the same project, which has certain reference value.

Quantification of anticipation of excitement with a three-axial model of emotion with EEG

OBJECTIVE

Multiple facets of human emotion underlie diverse and sparse neural mechanisms. Among the many existing models of emotion, the two-dimensional circumplex model of emotion is an important theory. The use of the circumplex model allows us to model variable aspects of emotion; however, such momentary expressions of one's internal mental state still lacks a notion of the third dimension of time. Here, we report an exploratory attempt to build a three-axis model of human emotion to model our sense of anticipatory excitement, 'Waku-Waku' (in Japanese), in which people predictively code upcoming emotional events.

APPROACH

Electroencephalography (EEG) data were recorded from 28 young adult participants while they mentalized upcoming emotional pictures. Three auditory tones were used as indicative cues, predicting the likelihood of the valence of an upcoming picture: positive, negative, or unknown. While seeing an image, the participants judged its emotional valence during the task and subsequently rated their subjective experiences on valence, arousal, expectation, and Waku-Waku immediately after the experiment. The collected EEG data were then analyzed to identify contributory neural signatures for each of the three axes.

MAIN RESULTS

A three-axis model was built to quantify Waku-Waku. As expected, this model revealed the considerable contribution of the third dimension over the classical two-dimensional model. Distinctive EEG components were identified. Furthermore, a novel brain-emotion interface was proposed and validated within the scope of limitations.

SIGNIFICANCE

The proposed notion may shed new light on the theories of emotion and support multiplex dimensions of emotion. With the introduction of the cognitive domain for a brain-computer interface, we propose a novel brain-emotion interface. Limitations of the study and potential applications of this interface are discussed.

Design of a Database-Driven Kansei Feedback Control System Using a Hydraulic Excavators Simulator

In Japan, the level of happiness is considered low despite the gross domestic product (GDP) being high, and a wide gap separates “products wealth” related to GDP and “mental wealth such as Kansei” related to the level of happiness. To fill this gap, products should be controlled to enhance Kansei according to human feelings. However, it is difficult to obtain the Kansei model because of time-variant and nonlinear system. In this paper, the design of a data-oriented cascade control system based on Kansei is newly proposed. In particular, a database-driven controller is designed for a human based on Kansei. The effectiveness of the proposed scheme is verified by using the electroencephalograph (EEG).

Mu-Beta event-related (de)synchronization and EEG classification of left-right foot dorsiflexion kinaesthetic motor imagery for BCI

The left and right foot representation area is located within the interhemispheric fissure of the sensorimotor cortex and share spatial proximity. This makes it difficult to visualize the cortical lateralization of event-related (de)synchronization (ERD/ERS) during left and right foot motor imageries. The aim of this study is to investigate the possibility of using ERD/ERS in the mu, low beta, and high beta bandwidth, during left and right foot dorsiflexion kinaesthetic motor imageries (KMI), as unilateral control commands for a brain-computer interface (BCI). EEG was recorded from nine healthy participants during cue-based left-right foot dorsiflexion KMI tasks. The features were analysed for common average and bipolar references. With each reference, mu and beta band-power features were analysed using time–frequency (TF) maps, scalp topographies, and average time course for ERD/ERS. The cortical lateralization of ERD/ERS, during left and right foot KMI, was confirmed. Statistically significant features were classified using LDA, SVM, and KNN model, and evaluated using the area under ROC curves. An increase in high beta power following the end of KMI for both tasks was recorded, from right and left hemispheres, respectively, at the vertex. The single trial analysis and classification models resulted in high discrimination accuracies, i.e. maximum 83.4% for beta ERS, 79.1% for beta ERD, and 74.0% for mu ERD. With each model the features performed above the statistical chance level of 2-class discrimination for a BCI. Our findings indicate these features can evoke left-right differences in single EEG trials. This suggests that any BCI employing unilateral foot KMI can attain classification accuracy suitable for practical implementation. Given results stipulate the novel utilization of mu and beta as independent control features for discrimination of bilateral foot KMI in a BCI.

EEG Sensorimotor Correlates of Speed During Forearm Passive Movements

Although passive movement therapy has been widely adopted to recover lost motor functions of impaired body parts, the underlying neural mechanisms are still unclear. In this context, fully understanding how the proprioceptive input modulates the brain activity may provide valuable insights. Specifically, it has not been investigated how the speed of motions, passively guided by a haptic device, affects the sensorimotor rhythms (SMR). On the grounds that faster passive motions elicit larger quantity of afferent input, we hypothesize a proportional relationship between localized SMR features and passive movement speed. To address this hypothesis, we conducted an experiment where healthy subjects received passive forearm oscillations at different speed levels while their electroencephalogram was recorded. The mu and beta event related desynchronization (ERD) and beta rebound of both left and right sensorimotor areas are analyzed by linear mixed-effects models. Results indicate that passive movement speed is correlated with the contralateral beta rebound and ipsilateral mu ERD. The former has been previously linked with the processing of proprioceptive afferent input quantity, while the latter with speed-dependent inhibitory processes. This suggests the existence of functionally-distinct frequency-specific neuronal populations associated with passive movements. In future, our findings may guide the design of novel rehabilitation paradigms.

Markov Switching Model for Quick Detection of Event Related Desynchronization in EEG

Quick detection of motor intentions is critical in order to minimize the time required to activate a neuroprosthesis. We propose a Markov Switching Model (MSM) to achieve quick detection of an event related desynchronization (ERD) elicited by motor imagery (MI) and recorded by electroencephalography (EEG). Conventional brain computer interfaces (BCI) rely on sliding window classifiers in order to perform online continuous classification of the rest vs. MI classes. Based on this approach, the detection of abrupt changes in the sensorimotor power suffers from an intrinsic delay caused by the necessity of computing an estimate of variance across several tenths of a second. Here we propose to avoid explicitly computing the EEG signal variance, and estimate the ERD state directly from the voltage information, in order to reduce the detection latency. This is achieved by using a model suitable in situations characterized by abrupt changes of state, the MSM. In our implementation, the model takes the form of a Gaussian observation model whose variance is governed by two latent discrete states with Markovian dynamics. Its objective is to estimate the brain state (i.e., rest vs. ERD) given the EEG voltage, spatially filtered by common spatial pattern (CSP), as observation. The two variances associated with the two latent states are calibrated using the variance of the CSP projection during rest and MI, respectively. The transition matrix of the latent states is optimized by the “quickest detection” strategy that minimizes a cost function of detection latency and false positive rate. Data collected by a dry EEG system from 50 healthy subjects, was used to assess performance and compare the MSM with several logistic regression classifiers of different sliding window lengths. As a result, the MSM achieves a significantly better tradeoff between latency, false positive and true positive rates. The proposed model could be used to achieve a more reactive and stable control of a neuroprosthesis. This is a desirable property in BCI-based neurorehabilitation, where proprioceptive feedback is provided based on the patient's brain signal. Indeed, it is hypothesized that simultaneous contingent association between brain signals and proprioceptive feedback induces superior associative learning.

Acquisition of a mental strategy to control a virtual tail via brain-computer interface

The objective of the present study was to clarify the variation in and properties of mental images and policies used to regulate specific image selection when learning to control a brain-computer interface. Healthy volunteers performed a reaching task with a virtually generated monkey tail-like object on a computer monitor by regulating event-related desynchronization (ERD) on the buttock area of the sensorimotor cortex as recorded by electroencephalogram (EEG). Participants were instructed to find a free image by which the tail was well controlled. Seven participants frequently returned to specific images that were mostly unrelated to a tail, and returned to these images on the last day of training. The ERD levels were greater during use of those selected images versus when selected images were not employed. Our results suggest that individuals adopted a mental strategy where they imagine what would reduce the prediction error between the predicted and actual BCI actions.

Brain-actuated gait trainer with visual and proprioceptive feedback

OBJECTIVE

Brain-machine interfaces (BMIs) have been proposed in closed-loop applications for neuromodulation and neurorehabilitation. This study describes the impact of different feedback modalities on the performance of an EEG-based BMI that decodes motor imagery (MI) of leg flexion and extension.

APPROACH

We executed experiments in a lower-limb gait trainer (the legoPress) where nine able-bodied subjects participated in three consecutive sessions based on a crossover design. A random forest classifier was trained from the offline session and tested online with visual and proprioceptive feedback, respectively. Post-hoc classification was conducted to assess the impact of feedback modalities and learning effect (an improvement over time) on the simulated trial-based performance. Finally, we performed feature analysis to investigate the discriminant power and brain pattern modulations across the subjects.

MAIN RESULTS

(i) For real-time classification, the average accuracy was [Formula: see text]% and [Formula: see text]% for the two online sessions. The results were significantly higher than chance level, demonstrating the feasibility to distinguish between MI of leg extension and flexion. (ii) For post-hoc classification, the performance with proprioceptive feedback ([Formula: see text]%) was significantly better than with visual feedback ([Formula: see text]%), while there was no significant learning effect. (iii) We reported individual discriminate features and brain patterns associated to each feedback modality, which exhibited differences between the two modalities although no general conclusion can be drawn.

SIGNIFICANCE

The study reported a closed-loop brain-controlled gait trainer, as a proof of concept for neurorehabilitation devices. We reported the feasibility of decoding lower-limb movement in an intuitive and natural way. As far as we know, this is the first online study discussing the role of feedback modalities in lower-limb MI decoding. Our results suggest that proprioceptive feedback has an advantage over visual feedback, which could be used to improve robot-assisted strategies for motor training and functional recovery.

A Synchronous Motor Imagery Based Neural Physiological Paradigm for Brain Computer Interface Speller

Brain Computer Interface (BCI) speller is a typical BCI-based application to help paralyzed patients express their thoughts. This paper proposed a novel motor imagery based BCI speller with Oct-o-spell paradigm for word input. Furthermore, an intelligent input method was used for improving the performance of the BCI speller. For the English word spelling experiment, we compared synchronous control with previous asynchronous control under the same experimental condition. There were no significant differences between these two control methods in the classification accuracy, information transmission rate (ITR) or letters per minute (LPM). And the accuracy rates of over 70% validated the feasibility for these two control strategies. It was indicated that MI-based synchronous control protocol was feasible for BCI speller. And the efficiency of the predictive text entry (PTE) mode was superior to that of the Non-PTE mode.

A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control

To recognize the user’s motion intention, brain-machine interfaces (BMI) usually decode movements from cortical activity to control exoskeletons and neuroprostheses for daily activities. The aim of this paper is to investigate whether self-induced variations of the electroencephalogram (EEG) can be useful as control signals for an upper-limb exoskeleton developed by us. A BMI based on event-related desynchronization/synchronization (ERD/ERS) is proposed. In the decoder-training phase, we investigate the offline classification performance of left versus right hand and left hand versus both feet by using motor execution (ME) or motor imagery (MI). The results indicate that the accuracies of ME sessions are higher than those of MI sessions, and left hand versus both feet paradigm achieves a better classification performance, which would be used in the online-control phase. In the online-control phase, the trained decoder is tested in two scenarios (wearing or without wearing the exoskeleton). The MI and ME sessions wearing the exoskeleton achieve mean classification accuracy of 84.29% ± 2.11% and 87.37% ± 3.06%, respectively. The present study demonstrates that the proposed BMI is effective to control the upper-limb exoskeleton, and provides a practical method by non-invasive EEG signal associated with human natural behavior for clinical applications.

Quantitative EEG Evaluation During Robot-Assisted Foot Movement

Passiveand imagined limbmovements induce changes in cerebral oscillatory activity. Central modulatory effects play a role in plastic changes, and are of uttermost importance in rehabilitation. This has extensively been studied for upper limb, but less is known for lower limb. The aim of this study is to investigate the topographical distribution of event-related desynchronization/synchronization(ERD/ERS) and task-relatedcoherence during a robot-assisted and a motor imagery task of lower limb in healthy subjects to inform rehabilitation paradigms. 32-channels electroencephalogram (EEG) was recorded in twenty-one healthy right footed and handed subjects during a robot-assisted single-joint cyclic right ankle movement performed by the BTS ANYMOV robotic hospital bed. Data were acquired with a block protocol for passive and imagined movement at a frequency of 0.2 Hz. ERD/ERS and task related coherence were calculated in alpha1 (8-10 Hz), alpha2 (10.5-12.5 Hz) and beta (13-30 Hz) frequency ranges. During passive movement, alpha2 rhythm desynchronized overC3 and ipsilateral frontal areas (F4, FC2, FC6); betaERD was detected over the bilateral motor areas (Cz, C3, C4). During motor imagery, a significant desynchronization was evident for alpha1 over contralateral sensorimotor cortex (C3), for alpha2 over bilateral motor areas (C3 and C4), and for beta over central scalp areas. Task-related coherence decreased during passive movement in alpha2 band between contralateral central area (C3, CP5, CP1, P3) and ipsilateral frontal area (F8, FC6, T8); beta band coherence decreased between C3-C4 electrodes, and increased between C3-Cz. These data contribute to the understanding of oscillatory activity and functional neuronal interactions during lower limb robot-assisted motor performance. The final output of this line of research is to inform the design and development of neurorehabilitation protocols.

Creating the brain and interacting with the brain: an integrated approach to understanding the brain

In the past two decades, brain science and robotics have made gigantic advances in their own fields, and their interactions have generated several interdisciplinary research fields. First, in the 'understanding the brain by creating the brain' approach, computational neuroscience models have been applied to many robotics problems. Second, such brain-motivated fields as cognitive robotics and developmental robotics have emerged as interdisciplinary areas among robotics, neuroscience and cognitive science with special emphasis on humanoid robots. Third, in brain-machine interface research, a brain and a robot are mutually connected within a closed loop. In this paper, we review the theoretical backgrounds of these three interdisciplinary fields and their recent progress. Then, we introduce recent efforts to reintegrate these research fields into a coherent perspective and propose a new direction that integrates brain science and robotics where the decoding of information from the brain, robot control based on the decoded information and multimodal feedback to the brain from the robot are carried out in real time and in a closed loop.

EEG Single-Trial Detection of Gait Speed Changes during Treadmill Walk

In this study, we analyse the electroencephalography (EEG) signal associated with gait speed changes (i.e. acceleration or deceleration). For data acquisition, healthy subjects were asked to perform volitional speed changes between 0, 1, and 2 Km/h, during treadmill walk. Simultaneously, the treadmill controller modified the speed of the belt according to the subject’s linear speed. A classifier is trained to distinguish between the EEG signal associated with constant speed gait and with gait speed changes, respectively. Results indicate that the classification performance is fair to good for the majority of the subjects, with accuracies always above chance level, in both batch and pseudo-online approaches. Feature visualisation and equivalent dipole localisation suggest that the information used by the classifier is associated with increased activity in parietal areas, where mu and beta rhythms are suppressed during gait speed changes. Specifically, the parietal cortex may be involved in motor planning and visuomotor transformations throughout the online gait adaptation, which is in agreement with previous research. The findings of this study may help to shed light on the cortical involvement in human gait control, and represent a step towards a BMI for applications in post-stroke gait rehabilitation.

Control of Leg Movements Driven by EMG Activity of Shoulder Muscles

During human walking, there exists a functional neural coupling between arms and legs, and between cervical and lumbosacral pattern generators. Here, we present a novel approach for associating the electromyographic (EMG) activity from upper limb muscles with leg kinematics. Our methodology takes advantage of the high involvement of shoulder muscles in most locomotor-related movements and of the natural co-ordination between arms and legs. Nine healthy subjects were asked to walk at different constant and variable speeds (3–5 km/h), while EMG activity of shoulder (deltoid) muscles and the kinematics of walking were recorded. To ensure a high level of EMG activity in deltoid, the subjects performed slightly larger arm swinging than they usually do. The temporal structure of the burst-like EMG activity was used to predict the spatiotemporal kinematic pattern of the forthcoming step. A comparison of actual and predicted stride leg kinematics showed a high degree of correspondence (r > 0.9). This algorithm has been also implemented in pilot experiments for controlling avatar walking in a virtual reality setup and an exoskeleton during over-ground stepping. The proposed approach may have important implications for the design of human–machine interfaces and neuroprosthetic technologies such as those of assistive lower limb exoskeletons.