Efficacy of kinesthetic motor imagery based brain computer interface combined with tDCS on upper limb function in subacute stroke

This study investigates whether the combined effect of kinesthetic motor imagery-based brain computer interface (KI-BCI) and transcranial direct current stimulation (tDCS) on upper limb function in subacute stroke patients is more effective than using KI-BCI or tDCS alone. Forty-eight subacute stroke survivors were randomized to the KI-BCI, tDCS, or BCI-tDCS group. The KI-BCI group performed 30 min of KI-BCI training. Patients in tDCS group received 30 min of tDCS. Patients in BCI-tDCS group received 15 min of tDCS and 15 min of KI-BCI. The treatment cycle was five times a week, for four weeks. After all intervention, the Fugl-Meyer Assessment-Upper Extremity, Motor Status Scale, and the Modified Barthel Index scores of the KI-BCI group were superior to those of the tDCS group. The BCI-tDCS group was superior to the tDCS group in terms of the Motor Status Scale. Although quantitative EEG showed no significant group differences, the quantitative EEG indices in the tDCS group were significantly lower than before treatment. In conclusion, after treatment, although all intervention strategies improved upper limb motor function and daily living abilities in subacute stroke patients, KI-BCI demonstrated significantly better efficacy than tDCS. Under the same total treatment duration, the combined use of tDCS and KI-BCI did not achieve the hypothesized optimal outcome. Notably, tDCS reduced QEEG indices, possibly indicating favorable future outcomes in future.Trial registry number: ChiCTR2000034730.

Fusion of EEG and EMG signals for detecting pre-movement intention of sitting and standing in healthy individuals and patients with spinal cord injury

Introduction

Rehabilitation devices assist individuals with movement disorders by supporting daily activities and facilitating effective rehabilitation training. Accurate and early motor intention detection is vital for real-time device applications. However, traditional methods of motor intention detection often rely on single-mode signals, such as EEG or EMG alone, which can be limited by low signal quality and reduced stability. This study proposes a multimodal fusion method based on EEG-EMG functional connectivity to detect sitting and standing intentions before movement execution, enabling timely intervention and reducing latency in rehabilitation devices.

Methods

Eight healthy subjects and five spinal cord injury (SCI) patients performed cue-based sit-to-stand and stand-to-sit transition tasks while EEG and EMG data were recorded simultaneously. We constructed EEG-EMG functional connectivity networks using data epochs from the 1.5-s period prior to movement onset. Pairwise spatial filters were then designed to extract discriminative spatial network topologies. Each filter paired with a support vector machine classifier to classify future movements into one of three classes: sit-to-stand, stand-to-sit, or rest. The final prediction was determined using a majority voting scheme.

Results

Among the three functional connectivity methods investigated-coherence, Pearson correlation coefficient and mutual information (MI)-the MI-based EEG-EMG network showed the highest decoding performance (94.33%), outperforming both EEG (73.89%) and EMG (89.16%). The robustness of the fusion method was further validated through a fatigue training experiment with healthy subjects. The fusion method achieved 92.87% accuracy during the post-fatigue stage, with no significant difference compared to the pre-fatigue stage (p > 0.05). Additionally, the proposed method using pre-movement windows achieved accuracy comparable to trans-movement windows (p > 0.05 for both pre- and post-fatigue stages). For the SCI patients, the fusion method showed improved accuracy, achieving 87.54% compared to single- modality methods (EEG: 83.03%, EMG: 84.13%), suggesting that the fusion method could be promising for practical rehabilitation applications.

Conclusion

Our results demonstrated that the proposed multimodal fusion method significantly enhances the performance of detecting human motor intentions. By enabling early detection of sitting and standing intentions, this method holds the potential to offer more accurate and timely interventions within rehabilitation systems.

A temporal-spectral fusion transformer with subject-specific adapter for enhancing RSVP-BCI decoding

The Rapid Serial Visual Presentation (RSVP)-based Brain-Computer Interface (BCI) is an efficient technology for target retrieval using electroencephalography (EEG) signals. The performance improvement of traditional decoding methods relies on a substantial amount of training data from new test subjects, which increases preparation time for BCI systems. Several studies introduce data from existing subjects to reduce the dependence of performance improvement on data from new subjects, but their optimization strategy based on adversarial learning with extensive data increases training time during the preparation procedure. Moreover, most previous methods only focus on the single-view information of EEG signals, but ignore the information from other views which may further improve performance. To enhance decoding performance while reducing preparation time, we propose a Temporal-Spectral fusion transformer with Subject-specific Adapter (TSformer-SA). Specifically, a cross-view interaction module is proposed to facilitate information transfer and extract common representations across two-view features extracted from EEG temporal signals and spectrogram images. Then, an attention-based fusion module fuses the features of two views to obtain comprehensive discriminative features for classification. Furthermore, a multi-view consistency loss is proposed to maximize the feature similarity between two views of the same EEG signal. Finally, we propose a subject-specific adapter to rapidly transfer the knowledge of the model trained on data from existing subjects to decode data from new subjects. Experimental results show that TSformer-SA significantly outperforms comparison methods and achieves outstanding performance with limited training data from new subjects. This facilitates efficient decoding and rapid deployment of BCI systems in practical use.

Electroencephalography-Based Brain-Computer Interfaces in Rehabilitation: A Bibliometric Analysis (2013-2023)

EEG-based Brain-Computer Interfaces (BCIs) have gained significant attention in rehabilitation due to their non-invasive, accessible ability to capture brain activity and restore neurological functions in patients with conditions such as stroke and spinal cord injuries. This study offers a comprehensive bibliometric analysis of global EEG-based BCI research in rehabilitation from 2013 to 2023. It focuses on primary research and review articles addressing technological innovations, effectiveness, and system advancements in clinical rehabilitation. Data were sourced from databases like Web of Science, and bibliometric tools (bibliometrix R) were used to analyze publication trends, geographic distribution, keyword co-occurrences, and collaboration networks. The results reveal a rapid increase in EEG-BCI research, peaking in 2022, with a primary focus on motor and sensory rehabilitation. EEG remains the most commonly used method, with significant contributions from Asia, Europe, and North America. Additionally, there is growing interest in applying BCIs to mental health, as well as integrating artificial intelligence (AI), particularly machine learning, to enhance system accuracy and adaptability. However, challenges remain, such as system inefficiencies and slow learning curves. These could be addressed by incorporating multi-modal approaches and advanced neuroimaging technologies. Further research is needed to validate the applicability of EEG-BCI advancements in both cognitive and motor rehabilitation, especially considering the high global prevalence of cerebrovascular diseases. To advance the field, expanding global participation, particularly in underrepresented regions like Latin America, is essential. Improving system efficiency through multi-modal approaches and AI integration is also critical. Ethical considerations, including data privacy, transparency, and equitable access to BCI technologies, must be prioritized to ensure the inclusive development and use of these technologies across diverse socioeconomic groups.

Design of a brain-machine interface for reducing false activations of a lower-limb exoskeleton based on error related potential

BACKGROUND AND OBJECTIVE

Brain-Machine Interfaces (BMIs) based on a motor imagination paradigm provide an intuitive approach for the exoskeleton control during gait. However, their clinical applicability remains difficulted by accuracy limitations and sensitivity to false activations. A proposed improvement involves integrating the BMI with methods based on detecting Error Related Potentials (ErrP) to self-tune erroneous commands and enhance not only the system accuracy, but also its usability. The aim of the current research is to characterize the ErrP at the beginning of the gait with a lower limb exoskeleton to reduce the false starts in the BMI system. Furthermore, this study is valuable for determining which type of feedback, Tactile, Visual, or Visuo-Tactile, achieves the best performance in evoking and detecting the ErrP.

METHODS

The initial phase of the research concentrates on detecting ErrP at the beginning of gait to improve the efficiency of an asynchronous BMI based on motor imagery (BMI-MI) to control a lower limb exoskeleton. Initially, an experimental protocol is designed to evoke ErrP at the start of gait, employing three different stimuli: Tactile, Visual, and Visuo-Tactile. An iterative selection process is then utilized to characterize ErrP in both time and frequency domains and fine-tune various parameters, including electrode distribution, feature combinations, and classifiers. A generic classifier with 6 subjects is employed to configure an ensemble classification system for detecting ErrP and reducing the false starts.

RESULTS

The ensembled system configured with the selected parameters yields an average correction of false starts of 72.60 % ± 10.23, highlighting its corrective efficacy. Tactile feedback emerges as the most effective stimulus, outperforming Visual and Visuo-Tactile feedback in both training types.

CONCLUSIONS

The results suggest promising prospects for reducing the false starts when integrating ErrP with BMI-MI, employing Tactile feedback. Consequently, the security of the system is enhanced. Subsequent, further research efforts will focus on detecting error potential during movement for gait stop, in order to limit undesired stops.

Electrotactile BCI for Top-Down Somatosensory Training: Clinical Feasibility Trial of Online BCI Control in Subacute Stroke Patients

This study investigates the feasibility of a novel brain-computer interface (BCI) device designed for sensory training following stroke. The BCI system administers electrotactile stimuli to the user's forearm, mirroring classical sensory training interventions. Concurrently, selective attention tasks are employed to modulate electrophysiological brain responses (somatosensory event-related potentials-sERPs), reflecting cortical excitability in related sensorimotor areas. The BCI identifies attention-induced changes in the brain's reactions to stimulation in an online manner. The study protocol assesses the feasibility of online binary classification of selective attention focus in ten subacute stroke patients. Each experimental session includes a BCI training phase for data collection and classifier training, followed by a BCI test phase to evaluate online classification of selective tactile attention based on sERP. During online classification tests, patients complete 20 repetitions of selective attention tasks with feedback on attention focus recognition. Using a single electroencephalographic channel, attention classification accuracy ranges from 70% to 100% across all patients. The significance of this novel BCI paradigm lies in its ability to quantitatively measure selective tactile attention resources throughout the therapy session, introducing a top-down approach to classical sensory training interventions based on repeated neuromuscular electrical stimulation.

Towards Speech Synthesis of Unconstrained Sentences from Speech-related Biosignals

Brain-to-speech systems offer a new means of human communication, enabling the generation of linguistic expression from neural activity. Recent studies on brain-to-speech using non-invasive brain signals have mostly proved the possibility of decoding words or sentences with repeated and predefined classes. In order to facilitate intuitive and natural communication via brain signals, decoding unconstrained speech in the same way as natural human communication is essential. In this study, we demonstrated the potential of speech synthesis for unseen sentences from biosignals by training the generative model to learn phonological features. We focused on constructing a sequence-conscious architecture that learns temporal dependencies and also leveraging the phoneme prediction loss term to extract speech-related features. Therefore, the feasibility of sentence-level neural communication based on non-predefined vocabularies was addressed, particularly training with non-repetitive sentences. Additionally, we conducted neurophysiological and spatio-spectral analysis by comparing brain activity during speech in terms of cortical and functional brain regions. Our results display the potential of unconstrained sentence generation, which may ultimately provide a new form of human interaction mediated by brain signals.

An EEG channel selection method for motor imagery based on Fisher score and local optimization

Objective. Multi-channel electroencephalogram (EEG) technology in brain-computer interface (BCI) research offers the advantage of enhanced spatial resolution and system performance. However, this also implies that more time is needed in the data processing stage, which is not conducive to the rapid response of BCI. Hence, it is a necessary and challenging task to reduce the number of EEG channels while maintaining decoding effectiveness.Approach. In this paper, we propose a local optimization method based on the Fisher score for within-subject EEG channel selection. Initially, we extract the common spatial pattern characteristics of EEG signals in different bands, calculate Fisher scores for each channel based on these characteristics, and rank them accordingly. Subsequently, we employ a local optimization method to finalize the channel selection.Main results. On the BCI Competition IV Dataset IIa, our method selects an average of 11 channels across four bands, achieving an average accuracy of 79.37%. This represents a 6.52% improvement compared to using the full set of 22 channels. On our self-collected dataset, our method similarly achieves a significant improvement of 24.20% with less than half of the channels, resulting in an average accuracy of 76.95%.Significance. This research explores the importance of channel combinations in channel selection tasks and reveals that appropriately combining channels can further enhance the quality of channel selection. The results indicate that the model selected a small number of channels with higher accuracy in two-class motor imagery EEG classification tasks. Additionally, it improves the portability of BCI systems through channel selection and combinations, offering the potential for the development of portable BCI systems.

Characterization of Event Related Desynchronization in Chronic Stroke Using Motor Imagery Based Brain Computer Interface for Upper Limb Rehabilitation

OBJECTIVE

Motor imagery-based brain-computer interface (MI-BCI) is a promising novel mode of stroke rehabilitation. The current study aims to investigate the feasibility of MI-BCI in upper limb rehabilitation of chronic stroke survivors and also to study the early event-related desynchronization after MI-BCI intervention.

METHODS

Changes in the characteristics of sensorimotor rhythm modulations in response to a short brain-computer interface (BCI) intervention for upper limb rehabilitation of stroke-disabled hand and normal hand were examined. The participants were trained to modulate their brain rhythms through motor imagery or execution during calibration, and they played a virtual marble game during the feedback session, where the movement of the marble was controlled by their sensorimotor rhythm.

RESULTS

Ipsilesional and contralesional activities were observed in the brain during the upper limb rehabilitation using BCI intervention. All the participants were able to successfully control the position of the virtual marble using their sensorimotor rhythm.

CONCLUSIONS

The preliminary results support the feasibility of BCI in upper limb rehabilitation and unveil the capability of MI-BCI as a promising medical intervention. This study provides a strong platform for clinicians to build upon new strategies for stroke rehabilitation by integrating MI-BCI with various therapeutic options to induce neural plasticity and recovery.

Transcranial Direct Current Stimulation and Brain-Computer Interfaces for Improving Post-Stroke Recovery: A Systematic Review and Meta-Analysis

OBJECTIVE

This study aimed to evaluate the effectiveness of transcranial direct current stimulation associated with brain-computer interface in stroke patients.

DATA SOURCES

The PubMed, Central, PEDro, Web of Science, SCOPUS, PsycINFO Ovid, CINAHL EBSCO, EMBASE, and ScienceDirect databases were searched from inception to April 2023 for randomized controlled studies reporting the effects of active transcranial direct current stimulation associated with brain-computer interface to a transcranial direct current stimulation sham associated with brain-computer interface condition on the outcome measure (motor performance and functional independence).

REVIEW METHODS

We searched for full-text articles which had investigated the effect of transcranial direct current stimulation associated with brain-computer interface on motor performance in the upper extremities in stroke patients. The standardized mean differences derived from the change in scores between pretreatment and post-treatment were adopted as the effect size measure, with a 95% confidence interval. Possible sources of heterogeneity were analyzed by performing subgroup analyses in order to examine the moderating effects for one variable: the level of injury severity.

RESULTS

Nine studies were included in the qualitative synthesis and the meta-analysis. The findings of the conducted analyses indicated there is not enough evidence to suggest that active transcranial direct current stimulation associated with brain-computer interface is more efficient in motor performance and functional independence when compared to sham transcranial direct current stimulation associated with brain-computer interface or brain-computer interface alone. In addition, the quality of evidence was rated very low. A subgroup analysis was performed for the motor performance outcome considering the injury severity level.

CONCLUSION

We found evidence that transcranial direct current stimulation associated with brain-computer interface was not more beneficial than sham transcranial direct current stimulation associated with brain-computer interface or brain-computer interface alone.

fNIRS-EEG BCIs for Motor Rehabilitation: A Review

Motor impairment has a profound impact on a significant number of individuals, leading to a substantial demand for rehabilitation services. Through brain-computer interfaces (BCIs), people with severe motor disabilities could have improved communication with others and control appropriately designed robotic prosthetics, so as to (at least partially) restore their motor abilities. BCI plays a pivotal role in promoting smoother communication and interactions between individuals with motor impairments and others. Moreover, they enable the direct control of assistive devices through brain signals. In particular, their most significant potential lies in the realm of motor rehabilitation, where BCIs can offer real-time feedback to assist users in their training and continuously monitor the brain's state throughout the entire rehabilitation process. Hybridization of different brain-sensing modalities, especially functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), has shown great potential in the creation of BCIs for rehabilitating the motor-impaired populations. EEG, as a well-established methodology, can be combined with fNIRS to compensate for the inherent disadvantages and achieve higher temporal and spatial resolution. This paper reviews the recent works in hybrid fNIRS-EEG BCIs for motor rehabilitation, emphasizing the methodologies that utilized motor imagery. An overview of the BCI system and its key components was introduced, followed by an introduction to various devices, strengths and weaknesses of different signal processing techniques, and applications in neuroscience and clinical contexts. The review concludes by discussing the possible challenges and opportunities for future development.

Effects of altered functional connectivity on motor imagery brain-computer interfaces based on the laterality of paralysis in hemiplegia patients

Motor imagery (MI)-based brain-computer interfaces are widely employed for improving the rehabilitation of paralyzed people and their quality of life. It has been well documented that brain activity patterns in the primary motor cortex and sensorimotor cortex during MI are similar to those of motor execution/imagery. However, individuals paralyzed owing to various neurological disorders have debilitated activation of the motor control region. Therefore, the differences in brain activation based on the paralysis location should be considered. We analyzed brain activation patterns using the electroencephalogram (EEG) acquired while performing MI on the right upper limb to investigate hemiplegia-related brain activation patterns. Participants with hemiplegia of the right upper limb (n=7) and left upper limb (n=4) performed the MI task within the right upper limb. EEG signals were acquired using 14 channels based on a 10-20 global system, and analyzed for event-related desynchronization (ERD) based on event-related spectral perturbation and functional connectivity, using the weighted phase-lag index of both hemispheres at the location of hemiplegia. Enhanced ERD was found in the ipsilateral region, compared to the contralateral region, after MI of the affected limb. The reduced difference in the centrality of the channels was observed in all subjects, likely reflecting an altered brain network from increased interhemispheric connections. Furthermore, the tendency of distinct network-based features depending on the MI task on the affected limb was diluted between the inter-hemispheres. Analysis of interaction between inter-region using functional connectivity could provide avenues for further investigation of BCI strategy through the brain state of individuals with hemiplegia.

A large EEG database with users' profile information for motor imagery brain-computer interface research

We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users' profiles and their BCI performances, (2) studying how EEG signals properties varies for different users' profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users' profile information into the design of EEG signal classification algorithms.

Feasibility of Using Source-Level Brain Computer Interface for People with Multiple Sclerosis

This work evaluates the feasibility of using a source level Brain-Computer Interface (BCI) for people with Multiple Sclerosis (MS). Data used was previously collected EEG of eight participants (one participant with MS and seven neurotypical participants) who performed imagined movement of the right and left hand. Equivalent current dipole cluster fitting was used to assess related brain activity at the source level and assessed using dipole location and power spectrum analysis. Dipole clusters were resolved within the motor cortices with some notable spatial difference between the MS and control participants. Neural sources that generate motor imagery originated from similar motor areas in the participant with MS compared to the neurotypical participants. Power spectral analysis indicated a reduced level of alpha power in the participant with MS during imagery tasks compared to neurotypical participants. Power in the beta band may be used to distinguish between left and right imagined movement for users with MS in BCI applications.Clinical Relevance- This paper demonstrates the cortical areas activated during imagined BCI-type tasks in a participant with Multiple Sclerosis (MS), and is a proof of concept for translating BCI research to potential users with MS.

Analysis of Error Potentials generated by a lower limb exoskeleton feedback in a BMI for gait control. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083187 DOI: 10.1109/embc40787.2023.10340275]

Brain-machine interfaces (BMIs) based on motor imagery (MI) for controlling lower-limb exoskeletons during the gait have been gaining importance in the rehabilitation field. However, these MI-BMI are not as precise as they should. The detection of error related potentials (ErrP) as a self-tune parameter to prevent wrong commands could be an interesting approach to improve their performance. For this reason, in this investigation ErrP elicited by the movement of a lower-limb exoskeleton against subject's will is analyzed in the time, frequency and time-frequency domain and compared with the cases where the exoskeleton is correctly commanded by motor imagery (MI). The results of the ErrP study indicate that there is statistical significative evidence of a difference between the signals in the erroneous events and the success events. Thus, ErrP could be used to increase the accuracy of BMIs which commands exoskeletons.Clinical Relevance- This investigation has the purpose of improving brain-machine interfaces (BMIs) based on motor imagery (MI) by means of the detection of error potentials. This could promote the adoption of robotic exoskeletons commanded by BMIs in rehabilitation therapies.

Verum versus Sham brain-computer interface on upper limb function recovery after stroke: A systematic review and meta-analysis of randomized controlled trials

BACKGROUND

Previous clinical trials have reported that the brain-computer interface (BCI) is a useful management tool for upper limb function recovery (ULFR) in stroke. However, there is insufficient evidence regarding this topic. Thus, this study aimed to investigate the effectiveness of verum versus sham BCI on the ULFR in stroke patients.

METHODS

We comprehensively searched the Cochrane Library, PUBMED, EMBASE, Web of Science, and China National Knowledge Infrastructure databases from their inception to January 1, 2023. Randomized clinical trials (RCTs) assessing the effectiveness and safety of BCI for ULFR after stroke were included. The outcomes were the Fugl-Meyer Assessment for Upper Extremity, Wolf Motor Function Test, Modified Barthel Index, motor activity log, and Action Research Arm Test. The methodological quality of all the included randomized controlled trials was evaluated using the Cochrane risk-of-bias tool. Statistical analysis was performed using RevMan 5.4 software.

RESULTS

Eleven eligible studies involving 334 patients were included. The results of the meta-analysis showed significant differences in the Fugl-Meyer Assessment for Upper Extremity (mean difference [MD] = 4.78, 95% confidence interval [CI] [1.90, 7.65], I2 = 0%, P = .001) and Modified Barthel Index (MD = 7.37, 95% CI [1.89, 12.84], I2 = 19%, P = .008). However, no significant differences were found on motor activity log (MD = -0.70, 95% CI [-3.17, 1.77]), Action Research Arm Test (MD = 3.05, 95% CI [-8.33, 14.44], I2 = 0%, P = .60), and Wolf Motor Function Test (MD = 4.23, 95% CI [-0.55, 9.01], P = .08).

CONCLUSION

BCI may be an effective management strategy for ULFR in stroke patients. Future studies with larger sample size and strict design are still needed to warrant the current findings.

Stimulation enhancement effect of the combination of exoskeleton-assisted hand rehabilitation and fingertip haptic stimulation

Introduction

Providing stimulation enhancements to existing hand rehabilitation training methods may help stroke survivors achieve better treatment outcomes. This paper presents a comparison study to explore the stimulation enhancement effects of the combination of exoskeleton-assisted hand rehabilitation and fingertip haptic stimulation by analyzing behavioral data and event-related potentials.

Methods

The stimulation effects of the touch sensations created by a water bottle and that created by cutaneous fingertip stimulation with pneumatic actuators are also investigated. Fingertip haptic stimulation was combined with exoskeleton-assisted hand rehabilitation while the haptic stimulation was synchronized with the motion of our hand exoskeleton. In the experiments, three experimental modes, including exoskeleton-assisted grasping motion without haptic stimulation (Mode 1), exoskeleton-assisted grasping motion with haptic stimulation (Mode 2), and exoskeleton-assisted grasping motion with a water bottle (Mode 3), were compared.

Results

The behavioral analysis results showed that the change of experimental modes had no significant effect on the recognition accuracy of stimulation levels (p = 0.658), while regarding the response time, exoskeleton-assisted grasping motion with haptic stimulation was the same as grasping a water bottle (p = 0.441) but significantly different from that without haptic stimulation (p = 0.006). The analysis of event-related potentials showed that the primary motor cortex, premotor cortex, and primary somatosensory areas of the brain were more activated when both the hand motion assistance and fingertip haptic feedback were provided using our proposed method (P300 amplitude 9.46 μV). Compared to only applying exoskeleton-assisted hand motion, the P300 amplitude was significantly improved by providing both exoskeleton-assisted hand motion and fingertip haptic stimulation (p = 0.006), but no significant differences were found between any other two modes (Mode 2 vs. Mode 3: p = 0.227, Mode 1 vs. Mode 3: p = 0.918). Different modes did not significantly affect the P300 latency (p = 0.102). Stimulation intensity had no effect on the P300 amplitude (p = 0.295, 0.414, 0.867) and latency (p = 0.417, 0.197, 0.607).

Discussion

Thus, we conclude that combining exoskeleton-assisted hand motion and fingertip haptic stimulation provided stronger stimulation on the motor cortex and somatosensory cortex of the brain simultaneously; the stimulation effects of the touch sensations created by a water bottle and that created by cutaneous fingertip stimulation with pneumatic actuators are similar.

Robust Motor Imagery Tasks Classification Approach Using Bayesian Neural Network

The development of Brain-Computer Interfaces based on Motor Imagery (MI) tasks is a relevant research topic worldwide. The design of accurate and reliable BCI systems remains a challenge, mainly in terms of increasing performance and usability. Classifiers based on Bayesian Neural Networks are proposed in this work by using the variational inference, aiming to analyze the uncertainty during the MI prediction. An adaptive threshold scheme is proposed here for MI classification with a reject option, and its performance on both datasets 2a and 2b from BCI Competition IV is compared with other approaches based on thresholds. The results using subject-specific and non-subject-specific training strategies are encouraging. From the uncertainty analysis, considerations for reducing computational cost are proposed for future work.

Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces

Objective.Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates.Approach.We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel.Main results.An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this 'ToeplitzLDA' compared to shrinkage regularized LDA (up to 6 AUC points,p < 0.001) and Riemannian classification approaches (up to 2 AUC points,p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features.Significance.The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.

Gamifying rehabilitation: MILORD platform as an upper limb motion rehabilitation service

Motor learning is based on the correct repetition of specific movements for their permanent storage in the central nervous system (CNS). Rehabilitation relies heavily on the repetition of specific movements, and game scenarios are ideal environments to build routines of repetitive exercises that have entertaining characteristics. In this respect, the gamification of the rehabilitation program, through the introduction of game-specific techniques and design concepts, has gained attention as a complementary or alternative to routine rehabilitation programs. A gamified rehabilitation program promises to gain the patient's attention, to reduce the monotony of the process and preserve motivation to attend, and to create virtual incentives through the game, toward maintaining compliance to the “prescribed” program. This is often achieved through goal-oriented tasks and real-time feedback in the form of points and other in-game rewards. This paper describes MILORD rehabilitation platform, an affordable technological solution, which aims to support health professionals and enable remote rehabilitation, while maintaining health service characteristics and monitoring. MILORD is an end-to-end platform that consists of an interactive computer game, utilizing a leap motion sensor, a centralized user management system, an analysis platform that processes the data generated by the game, and an analysis dashboard presenting a set of meaningful features that describe upper limb movement. Our solution facilitates the monitoring of the patients' progress and provides an alternative way to analyze hand movement. The system was tested with normal subjects and patients and experts to record user's experience, receive feedback, identify any problems, and understand the system's value in monitoring and support motion defect and progress. This small-scale study indicated the capacity of the analysis to quantify the movement in a meaningful way and express the differences between normal and pathological movement, and the user experience was positive with both patients and normal subjects.

Poststroke motor, cognitive and speech rehabilitation with brain-computer interface: a perspective review

Brain-computer interface (BCI) technology translates brain activity into meaningful commands to establish a direct connection between the brain and the external world. Neuroscientific research in the past two decades has indicated a tremendous potential of BCI systems for the rehabilitation of patients suffering from poststroke impairments. By promoting the neuronal recovery of the damaged brain networks, BCI systems have achieved promising results for the recovery of poststroke motor, cognitive, and language impairments. Also, several assistive BCI systems that provide alternative means of communication and control to severely paralysed patients have been proposed to enhance patients' quality of life. In this article, we present a perspective review of the recent advances and challenges in the BCI systems used in the poststroke rehabilitation of motor, cognitive, and communication impairments.

Pseudo-online detection and classification for upper-limb movements

Objective. This study analyzed detection (movement vs. non-movement) and classification (different types of movements) to decode upper-limb movement volitions in a pseudo-online fashion.Approach. Nine healthy subjects executed four self-initiated movements: left wrist extension, right wrist extension, left index finger extension, and right index finger extension. For detection, we investigated the performance of three individual classifiers (support vector machine (SVM), EEGNET, and Riemannian geometry featured SVM) on three frequency bands (0.05-5 Hz, 5-40 Hz, 0.05-40 Hz). The best frequency band and the best classifier combinations were constructed to realize an ensemble processing pipeline using majority voting. For classification, we used adaptive boosted Riemannian geometry model to differentiate contra-lateral and ipsilateral movements.Main results. The ensemble model achieved 79.6 ± 8.8% true positive rate and 3.1 ± 1.2 false positives per minute with 75.3 ± 112.6 ms latency on a pseudo-online detection task. The following classification gave around 67% accuracy to differentiate contralateral movements.Significance. The newly proposed ensemble method and pseudo-online testing procedure could provide a robust brain-computer interface design for movement decoding.

Multimodal Neural Response and Effect Assessment During a BCI-Based Neurofeedback Training After Stroke

Stroke caused by cerebral infarction or hemorrhage can lead to motor dysfunction. The recovery of motor function is vital for patients with stroke in daily activities. Traditional rehabilitation of stroke generally depends on physical practice under passive affected limbs movement. Motor imagery-based brain computer interface (MI-BCI) combined with functional electrical stimulation (FES) is a potential active neural rehabilitation technology for patients with stroke recently, which complements traditional passive rehabilitation methods. As the predecessor of BCI technology, neurofeedback training (NFT) is a psychological process that feeds back neural activities online to users for self-regulation. In this work, BCI-based NFT were proposed to promote the active repair and reconstruction of the whole nerve conduction pathway and motor function. We designed and implemented a multimodal, training type motor NFT system (BCI-NFT-FES) by integrating the visual, auditory, and tactile multisensory pathway feedback mode and using the joint detection of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The results indicated that after 4 weeks of training, the clinical scale score, event-related desynchronization (ERD) of EEG patterns, and cerebral oxygen response of patients with stroke were enhanced obviously. This study preliminarily verified the clinical effectiveness of the long-term NFT system and the prospect of motor function rehabilitation.

Corticomuscular integrated representation of voluntary motor effort in robotic control for wrist-hand rehabilitation after stroke

OBJECTIVE

The central-to-peripheral voluntary motor effort (VME) in physical practice of the paretic limb is a dominant force for driving functional neuroplasticity on motor restoration post-stroke. However, current rehabilitation robots isolated the central and peripheral involvements in the control design, resulting in limited rehabilitation effectiveness. The purpose of this study was to design a corticomuscular coherence (CMC) and electromyography (EMG)-driven (CMC-EMG-driven) system with central-and-peripheral integrated representation of VME for wrist-hand rehabilitation after stroke.

APPROACH

The CMC-EMG-driven control was developed in a neuromuscular electrical stimulation (NMES)-robot system, i.e., CMC-EMG-driven NMES-robot system, to instruct and assist the wrist-hand extension and flexion in persons after stroke. A pilot single-group trial of 20 training sessions was conducted with the developed system to assess the feasibility for wrist-hand practice on the chronic strokes (n=16). The rehabilitation effectiveness was evaluated through clinical assessments, CMC, and EMG activation levels.

MAIN RESULTS

The trigger success rate and laterality index (LI) of CMC were significantly increased in wrist-hand extension across training sessions (p<0.05). After the training, significant improvements in the target wrist-hand joints and suppressed compensation from the proximal shoulder-elbow joints were observed through the clinical scores and EMG activation levels (p<0.05). The central-to-peripheral VME distribution across upper extremity (UE) muscles was also significantly improved, as revealed by the CMC values (p<0.05).

SIGNIFICANCE

Precise wrist-hand rehabilitation was achieved by the developed system, presenting suppressed cortical and muscular compensation from the contralesional hemisphere and the proximal UE, and improved distribution of the central-and-peripheral VME on UE muscles.

Effects of a Brain-Computer Interface-Operated Lower Limb Rehabilitation Robot on Motor Function Recovery in Patients with Stroke

Purpose

To observe the effect of a brain-computer interface-operated lower limb rehabilitation robot (BCI-LLRR) on functional recovery from stroke and to explore mechanisms.

Methods

Subacute-phase stroke patients were randomly divided into two groups. In addition to the routine intervention, patients in the treatment group trained on the BCI-LLRR and underwent the lower limb pedal training in the control group, both for the same time (30 min/day). All patients underwent assessment by instruments such as the National Institutes of Health Stroke Scale (NIHSS) and the Fugl-Meyer upper and lower limb motor function and balance tests, at 2 and 4 weeks of treatment and at 3 months after the end of treatment. Patients were also tested before treatment and after 4 weeks by leg motor evoked potential (MEP) and diffusion tensor imaging/tractography (DTI/DTT) of the head.

Results

After 4 weeks, the Fugl-Meyer leg function and NIHSS scores were significantly improved in the treatment group vs. controls (P < 0.01). At 3 months, further significant improvement was observed. The MEP amplitude and latency of the treatment group were significantly improved vs. controls. The effect of treatment on fractional anisotropy values was not significant.

Conclusions

The BCI-LLRR promoted leg functional recovery after stroke and improved activities of daily living, possibly by improving cerebral-cortex excitability and white matter connectivity.

BCI-Based Control for Ankle Exoskeleton T-FLEX: Comparison of Visual and Haptic Stimuli with Stroke Survivors

Brain–computer interface (BCI) remains an emerging tool that seeks to improve the patient interaction with the therapeutic mechanisms and to generate neuroplasticity progressively through neuromotor abilities. Motor imagery (MI) analysis is the most used paradigm based on the motor cortex’s electrical activity to detect movement intention. It has been shown that motor imagery mental practice with movement-associated stimuli may offer an effective strategy to facilitate motor recovery in brain injury patients. In this sense, this study aims to present the BCI associated with visual and haptic stimuli to facilitate MI generation and control the T-FLEX ankle exoskeleton. To achieve this, five post-stroke patients (55–63 years) were subjected to three different strategies using T-FLEX: stationary therapy (ST) without motor imagination, motor imagination with visual stimulation (MIV), and motor imagination with visual-haptic inducement (MIVH). The quantitative characterization of both BCI stimuli strategies was made through the motor imagery accuracy rate, the electroencephalographic (EEG) analysis during the MI active periods, the statistical analysis, and a subjective patient’s perception. The preliminary results demonstrated the viability of the BCI-controlled ankle exoskeleton system with the beta rebound, in terms of patient’s performance during MI active periods and satisfaction outcomes. Accuracy differences employing haptic stimulus were detected with an average of 68% compared with the 50.7% over only visual stimulus. However, the power spectral density (PSD) did not present changes in prominent activation of the MI band but presented significant variations in terms of laterality. In this way, visual and haptic stimuli improved the subject’s MI accuracy but did not generate differential brain activity over the affected hemisphere. Hence, long-term sessions with a more extensive sample and a more robust algorithm should be carried out to evaluate the impact of the proposed system on neuronal and motor evolution after stroke.

Feature Selection Combining Filter and Wrapper Methods for Motor-Imagery Based Brain-Computer Interfaces

Motor imagery (MI) based brain-computer interfaces help patients with movement disorders to regain the ability to control external devices. Common spatial pattern (CSP) is a popular algorithm for feature extraction in decoding MI tasks. However, due to noise and nonstationarity in electroencephalography (EEG), it is not optimal to combine the corresponding features obtained from the traditional CSP algorithm. In this paper, we designed a novel CSP feature selection framework that combines the filter method and the wrapper method. We first evaluated the importance of every CSP feature by the infinite latent feature selection method. Meanwhile, we calculated Wasserstein distance between feature distributions of the same feature under different tasks. Then, we redefined the importance of every CSP feature based on two indicators mentioned above, which eliminates half of CSP features to create a new CSP feature subspace according to the new importance indicator. At last, we designed the improved binary gravitational search algorithm (IBGSA) by rebuilding its transfer function and applied IBGSA on the new CSP feature subspace to find the optimal feature set. To validate the proposed method, we conducted experiments on three public BCI datasets and performed a numerical analysis of the proposed algorithm for MI classification. The accuracies were comparable to those reported in related studies and the presented model outperformed other methods in literature on the same underlying data.

Enhancing Detection of SSMVEP Induced by Action Observation Stimuli Based on Task-Related Component Analysis

Action observation (AO)-based brain-computer interface (BCI) is an important technology in stroke rehabilitation training. It has the advantage of simultaneously inducing steady-state motion visual evoked potential (SSMVEP) and activating sensorimotor rhythm. Moreover, SSMVEP could be utilized to perform classification. However, SSMVEP is composed of complex modulation frequencies. Traditional canonical correlation analysis (CCA) suffers from poor recognition performance in identifying those modulation frequencies at short stimulus duration. To address this issue, task-related component analysis (TRCA) was utilized to deal with SSMVEP for the first time. An interesting phenomenon was found: different modulated frequencies in SSMVEP distributed in different task-related components. On this basis, a multi-component TRCA method was proposed. All the significant task-related components were utilized to construct multiple spatial filters to enhance the detection of SSMVEP. Further, a combination of TRCA and CCA was proposed to utilize both advantages. Results showed that the accuracies using the proposed methods were significant higher than that using CCA at all window lengths and significantly higher than that using ensemble-TRCA at short window lengths (≤2 s). Therefore, the proposed methods further validate the induced modulation frequencies and will speed up the application of the AO-based BCI in rehabilitation.

Cross-correlation based discriminant criterion for channel selection in motor imagery BCI systems

Objective.Many electroencephalogram (EEG)-based brain-computer interface (BCI) systems use a large amount of channels for higher performance, which is time-consuming to set up and inconvenient for practical applications. Finding an optimal subset of channels without compromising the performance is a necessary and challenging task.Approach.In this article, we proposed a cross-correlation based discriminant criterion (XCDC) which assesses the importance of a channel for discriminating the mental states of different motor imagery (MI) tasks. Channels are ranked and selected according to the proposed criterion. The efficacy of XCDC is evaluated on two MI EEG datasets.Main results.On the two datasets, the proposed method reduces the channel number from 71 and 15 to under 18 and 11 respectively without compromising the classification accuracy on unseen data. Under the same constraint of accuracy, the proposed method requires fewer channels than existing channel selection methods based on Pearson's correlation coefficient and common spatial pattern. Visualization of XCDC shows consistent results with neurophysiological principles.Significance.This work proposes a quantitative criterion for assessing and ranking the importance of EEG channels in MI tasks and provides a practical method for selecting the ranked channels in the calibration phase of MI BCI systems, which alleviates the computational complexity and configuration difficulty in the subsequent steps, leading to real-time and more convenient BCI systems.

Decoding Finger Tapping With the Affected Hand in Chronic Stroke Patients During Motor Imagery and Execution

In stroke rehabilitation, motor imagery based on a brain-computer interface is an extremely useful method to control an external device and utilize neurofeedback. Many studies have reported on the classification performance of motor imagery to decode individual fingers in stroke patients compared with healthy controls. However, classification performance for a given limb is still low because the differences between patients owing to brain reorganization after stroke are not considered. We used electroencephalography signals from eleven healthy controls and eleven stroke patients in this study. The subjects performed a finger tapping task during motor execution, and motor imagery was performed with the dominant and affected hands in the healthy controls and stroke patients, respectively. All fingers except for the thumb were classified using the proposed framework based on a voting module. The averaged four-class accuracies during motor execution and motor imagery were 53.16 ± 8.42% and 46.94 ± 5.99% for the healthy controls and 53.17 ± 14.09% and 66.00 ± 14.96% for the stroke patients, respectively. Importantly, the classification accuracies in the stroke patients were statistically higher than those in healthy controls during motor imagery. However, there was no significant difference between the accuracies of motor execution and motor imagery. These findings show the potential for high classification performance for a given limb during motor imagery in stroke patients. These results can also provide insights into controlling an external device on the basis of a brain-computer interface.

Attention Enhancement for Exoskeleton-Assisted Hand Rehabilitation Using Fingertip Haptic Stimulation

Active enrollment in rehabilitation training yields better treatment outcomes. This paper introduces an exoskeleton-assisted hand rehabilitation system. It is the first attempt to combine fingertip cutaneous haptic stimulation with exoskeleton-assisted hand rehabilitation for training participation enhancement. For the first time, soft material 3D printing techniques are adopted to make soft pneumatic fingertip haptic feedback actuators to achieve cheaper and faster iterations of prototype designs with consistent quality. The fingertip haptic stimulation is synchronized with the motion of our hand exoskeleton. The contact force of the fingertips resulted from a virtual interaction with a glass of water was based on data collected from normal hand motions to grasp a glass of water. System characterization experiments were conducted and exoskeleton-assisted hand motion with and without the fingertip cutaneous haptic stimulation were compared in an experiment involving healthy human subjects. Users' attention levels were monitored in the motion control process using a Brainlink EEG-recording device and software. The results of characterization experiments show that our created haptic actuators are lightweight (6.8 ± 0.23 g each with a PLA fixture and Velcro) and their performance is consistent and stable with small hysteresis. The user study experimental results show that participants had significantly higher attention levels with additional haptic stimulations compared to when only the exoskeleton was deployed; heavier stimulated grasping weight (a 300 g glass) was associated with significantly higher attention levels of the participants compared to when lighter stimulated grasping weight (a 150 g glass) was applied. We conclude that haptic stimulations increase the involvement level of human subjects during exoskeleton-assisted hand exercises. Potentially, the proposed exoskeleton-assisted hand rehabilitation with fingertip stimulation may better attract user's attention during treatment.

Influential Factors of an Asynchronous BCI for Movement Intention Detection

In recent years, asynchronous brain computer interface (BCI) systems have been utilized in many domains such as robot controlling, assistive technology, and rehabilitation. In such BCI systems, movement intention detection algorithms are used to detect movement desires. In recent years, movement-related cortical potential (MRCP), an electroencephalogram (EEG) pattern representing voluntary movement intention, attracts wide attention in movement intention detection. Unfortunately, low MRCP detection accuracy makes the asynchronous BCI system impractical for real usage. In order to develop an effective MRCP detection algorithm, EEG data have to be properly preprocessed. In this work, we investigate the relationship and effects of three factors including frequency bands, spatial filters, and classifiers on MRCP classification performance to determine best settings. In particular, we performed a systematic performance investigation on combinations of five frequency bands, five spatial filters, and six classifiers. The EEG data were acquired from subjects performing series of self-paced ankle dorsiflexions. Analysis of variance (ANOVA) statistical test was performed on F1 scores to investigate effects of these three factors. The results show that frequency bands and spatial filters depend on each other. The combinations directly affect the F1 scores, so they have to be chosen carefully. The results can be used as guidelines for BCI researchers to effectively design a preprocessing method for an advanced asynchronous BCI system, which can assist the stroke rehabilitation.

High-wearable EEG-based distraction detection in motor rehabilitation

A method for EEG-based distraction detection during motor-rehabilitation tasks is proposed. A wireless cap guarantees very high wearability with dry electrodes and a low number of channels. Experimental validation is performed on a dataset from 17 volunteers. Different feature extractions from spatial, temporal, and frequency domain and classification strategies were evaluated. The performances of five supervised classifiers in discriminating between attention on pure movement and with distractors were compared. A k-Nearest Neighbors classifier achieved an accuracy of 92.8 ± 1.6%. In this last case, the feature extraction is based on a custom 12 pass-band Filter-Bank (FB) and the Common Spatial Pattern (CSP) algorithm. In particular, the mean Recall of classification (percentage of true positive in distraction detection) is higher than 92% and allows the therapist or an automated system to know when to stimulate the patient's attention for enhancing the therapy effectiveness.

Detection of the Intention of Direction Changes During Gait Through EEG Signals

Brain-Computer Interfaces (BCIs) are becoming an important technological tool for the rehabilitation process of patients with locomotor problems, due to their ability to recover the connection between brain and limbs by promoting neural plasticity. They can be used as assistive devices to improve the mobility of handicapped people. For this reason, current BCIs have to be improved to allow an accurate and natural use of external devices. This work proposes a novel methodology for the detection of the intention to change the direction during gait based on event-related desynchronization (ERD). Frequency and temporal features of the electroencephalographic (EEG) signals are characterized. Then, a selection of the most influential features and electrodes to differentiate the direction change intention from the walking is carried out. Best results are obtained when combining frequency and temporal features with an average accuracy of [Formula: see text]%, which are promising to be applied for future BCIs.

Dual stream neural networks for brain signal classification

Objective. The primary objective of this work is to develop a neural nework classifier for arbitrary collections of functional neuroimaging signals to be used in brain–computer interfaces (BCIs). Approach. We propose a dual stream neural network (DSNN) for the classification problem. The first stream is an end-to-end classifier taking raw time-dependent signals as input and generating feature identification signatures from them. The second stream enhances the identified features from the first stream by adjoining a dynamic functional connectivity matrix aimed at incorporating nuanced multi-channel information during specified BCI tasks. Main results. The proposed DSNN classifier is benchmarked against three publicly available datasets, where the classifier demonstrates performance comparable to, or better than the state-of-art in each instance. An information theoretic examination of the trained network is also performed, utilizing various tools, to demonstrate how to glean interpretive insight into how the hidden layers of the network parse the underlying biological signals. Significance . The resulting DSNN is a subject-independent classifier that works for any collection of 1D functional neuroimaging signals, with the option of integrating domain specific information in the design.

BCI-Based Rehabilitation on the Stroke in Sequela Stage

Background

Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke.

Method

Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function.

Results

The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study.

Conclusions

Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation.

Serious Gaming Technology in Upper Extremity Rehabilitation: Scoping Review

BACKGROUND

Serious gaming has increasingly gained attention as a potential new component in clinical practice. Specifically, its use in the rehabilitation of motor dysfunctions has been intensively researched during the past three decades.

OBJECTIVE

The aim of this scoping review was to evaluate the current role of serious games in upper extremity rehabilitation, and to identify common methods and practice as well as technology patterns. This objective was approached via the exploration of published research efforts over time.

METHODS

The literature search, using the PubMed and Scopus databases, included articles published from 1999 to 2019. The eligibility criteria were (i) any form of game-based arm rehabilitation; (ii) published in a peer-reviewed journal or conference; (iii) introduce a game in an electronic format; (iv) published in English; and (v) not a review, meta-analysis, or conference abstract. The search strategy identified 169 relevant articles.

RESULTS

The results indicated an increasing research trend in the domain of serious gaming deployment in upper extremity rehabilitation. Furthermore, differences regarding the number of publications and the game approach were noted between studies that used commercial devices in their rehabilitation systems and those that proposed a custom-made robotic arm, glove, or other devices for the connection and interaction with the game platform. A particularly relevant observation concerns the evaluation of the introduced systems. Although one-third of the studies evaluated their implementations with patients, in most cases, there is the need for a larger number of participants and better testing of the rehabilitation scheme efficiency over time. Most of the studies that included some form of assessment for the introduced rehabilitation game mentioned user experience as one of the factors considered for evaluation of the system. Besides user experience assessment, the most common evaluation method involving patients was the use of standard medical tests. Finally, a few studies attempted to extract game features to introduce quantitative measurements for the evaluation of patient improvement.

CONCLUSIONS

This paper presents an overview of a significant research topic and highlights the current state of the field. Despite extensive attempts for the development of gamified rehabilitation systems, there is no definite answer as to whether a serious game is a favorable means for upper extremity functionality improvement; however, this certainly constitutes a supplementary means for motivation. The development of a unified performance quantification framework and more extensive experiments could generate richer evidence and contribute toward this direction.

Motor Imagery Hand Movement Direction Decoding Using Brain Computer Interface to Aid Stroke Recovery and Rehabilitation

Motor Imagery (MI)-based Brain Computer Interface (BCI) system is a potential technology for active neurorehabilitation of stroke patients by complementing the conventional passive rehabilitation methods. Research to date mainly focused on classifying left vs. right hand/foot MI of stroke patients. Though a very few studies have reported decoding imagined hand movement directions using electroencephalogram (EEG)-based BCI, the experiments were conducted on healthy subjects. Our work analyzes MI-based brain cortical activity from EEG signals and decodes the imagined hand movement directions in stroke patients. The decoded direction (left vs. right) of hand movement imagination is used to provide control commands to a motorized arm support on which patient's affected (paralyzed) arm is placed. This enables the patient to move his/her stroke-affected hand towards the intended (imagined) direction that aids neuroplasticity in the brain. The synchronization measure called Phase Locking Value (PLV), extracted from EEG, is the neuronal signature used to decode the directional movement of the MI task. Event-related desynchronization/synchronization (ERD/ERS) analysis on Mu and Beta frequency bands of EEG is done to select the time bin corresponding to the MI task. The dissimilarities between the two directions of MI tasks are identified by selecting the most significant channel pairs that provided maximum difference in PLV features. The training protocol has an initial calibration session followed by a feedback session with 50 trials of MI task in each session. The feedback session extracts PLV features corresponding to most significant channel pairs which are identified in the calibration session and is used to predict the direction of MI task in left/right direction. An average MI direction classification accuracy of 74.44% is obtained in performing the training protocol and 68.63% from the prediction protocol during feedback session on 16 stroke patients.

A review of user training methods in brain computer interfaces based on mental tasks

Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.

Sensory Integration in Human Movement: A New Brain-Machine Interface Based on Gamma Band and Attention Level for Controlling a Lower-Limb Exoskeleton

Brain-machine interfaces (BMIs) can improve the control of assistance mobility devices making its use more intuitive and natural. In the case of an exoskeleton, they can also help rehabilitation therapies due to the reinforcement of neuro-plasticity through repetitive motor actions and cognitive engagement of the subject. Therefore, the cognitive implication of the user is a key aspect in BMI applications, and it is important to assure that the mental task correlates with the actual motor action. However, the process of walking is usually an autonomous mental task that requires a minimal conscious effort. Consequently, a brain-machine interface focused on the attention to gait could facilitate sensory integration in individuals with neurological impairment through the analysis of voluntary gait will and its repetitive use. This way the combined use of BMI+exoskeleton turns from assistance to restoration. This paper presents a new brain-machine interface based on the decoding of gamma band activity and attention level during motor imagery mental tasks. This work also shows a case study tested in able-bodied subjects prior to a future clinical study, demonstrating that a BMI based on gamma band and attention-level paradigm allows real-time closed-loop control of a Rex exoskeleton.

BETA: A Large Benchmark Database Toward SSVEP-BCI Application

The brain-computer interface (BCI) provides an alternative means to communicate and it has sparked growing interest in the past two decades. Specifically, for Steady-State Visual Evoked Potential (SSVEP) based BCI, marked improvement has been made in the frequency recognition method and data sharing. However, the number of pubic databases is still limited in this field. Therefore, we present a BEnchmark database Towards BCI Application (BETA) in the study. The BETA database is composed of 64-channel Electroencephalogram (EEG) data of 70 subjects performing a 40-target cued-spelling task. The design and the acquisition of the BETA are in pursuit of meeting the demand from real-world applications and it can be used as a test-bed for these scenarios. We validate the database by a series of analyses and conduct the classification analysis of eleven frequency recognition methods on BETA. We recommend using the metric of wide-band signal-to-noise ratio (SNR) and BCI quotient to characterize the SSVEP at the single-trial and population levels, respectively. The BETA database can be downloaded from the following link http://bci.med.tsinghua.edu.cn/download.html.

Bacomics: a comprehensive cross area originating in the studies of various brain-apparatus conversations

The brain is the most important organ of the human body, and the conversations between the brain and an apparatus can not only reveal a normally functioning or a dysfunctional brain but also can modulate the brain. Here, the apparatus may be a nonbiological instrument, such as a computer, and the consequent brain-computer interface is now a very popular research area with various applications. The apparatus may also be a biological organ or system, such as the gut and muscle, and their efficient conversations with the brain are vital for a healthy life. Are there any common bases that bind these different scenarios? Here, we propose a new comprehensive cross area: Bacomics, which comes from brain-apparatus conversations (BAC) + omics. We take Bacomics to cover at least three situations: (1) The brain is normal, but the conversation channel is disabled, as in amyotrophic lateral sclerosis. The task is to reconstruct or open up new channels to reactivate the brain function. (2) The brain is in disorder, such as in Parkinson's disease, and the work is to utilize existing or open up new channels to intervene, repair and modulate the brain by medications or stimulation. (3) Both the brain and channels are in order, and the goal is to enhance coordinated development between the brain and apparatus. In this paper, we elaborate the connotation of BAC into three aspects according to the information flow: the issue of output to the outside (BAC-1), the issue of input to the brain (BAC-2) and the issue of unity of brain and apparatus (BAC-3). More importantly, there are no less than five principles that may be taken as the cornerstones of Bacomics, such as feedforward and feedback control, brain plasticity, harmony, the unity of opposites and systems principles. Clearly, Bacomics integrates these seemingly disparate domains, but more importantly, opens a much wider door for the research and development of the brain, and the principles further provide the general framework in which to realize or optimize these various conversations.

Low-Cost Robotic Guide Based on a Motor Imagery Brain-Computer Interface for Arm Assisted Rehabilitation

Motor imagery has been suggested as an efficient alternative to improve the rehabilitation process of affected limbs. In this study, a low-cost robotic guide is implemented so that linear position can be controlled via the user’s motor imagination of movement intention. The patient can use this device to move the arm attached to the guide according to their own intentions. The first objective of this study was to check the feasibility and safety of the designed robotic guide controlled via a motor imagery (MI)-based brain–computer interface (MI-BCI) in healthy individuals, with the ultimate aim to apply it to rehabilitation patients. The second objective was to determine which are the most convenient MI strategies to control the different assisted rehabilitation arm movements. The results of this study show a better performance when the BCI task is controlled with an action–action MI strategy versus an action–relaxation one. No statistically significant difference was found between the two action–action MI strategies.

Analysis and Classification for Single-Trial EEG Induced by Sequential Finger Movements

In recent years, motor imagery-based BCIs (MI-BCIs) controlled various external devices successfully, which have great potential in neurological rehabilitation. In this paper, we designed a paradigm of sequential finger movements and utilized spatial filters for feature extraction to classify single-trial electroencephalography (EEG) induced by finger movements of left and right hand. Ten healthy subjects participated the experiment. The analysis of EEG patterns showed significant contralateral dominance. We investigated how data length affected the classification accuracy. The classification accuracy was improved with the increase of the keystrokes in one trial, and the results were 87.42%, 91.21%, 93.08% and 93.59% corresponding to single keystroke, two keystrokes, three keystrokes and four keystrokes. This study would be helpful to improve the decoding efficiency and optimize the encoding method of motor-related EEG information.

HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification

OBJECTIVE

Electroencephalography (EEG) motor imagery classification has been widely used in healthcare applications such as mobile assistive robots and post-stroke rehabilitation. Recently, EEG motor imagery classification methods based on convolutional neural networks (CNNs) have been proposed and have achieved relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when training data is limited.

APPROACH

To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification.

MAIN RESULTS

Compared with several state-of-the-art methods, the proposed method achieves an average classification accuracy of 91.57% and 87.6% on two commonly used datasets, which outperforms several state-of-the-art EEG motor imagery classification methods.

SIGNIFICANCE

The proposed method effectively addresses the issues of existing CNN-based EEG motor imagery classification methods and improves the classification accuracy.

Direct comparison of supervised and semi-supervised retraining approaches for co-adaptive BCIs

For Brain-Computer interfaces (BCIs), system calibration is a lengthy but necessary process for successful operation. Co-adaptive BCIs aim to shorten training and imply positive motivation to users by presenting feedback already at early stages: After just 5 min of gathering calibration data, the systems are able to provide feedback and engage users in a mutual learning process. In this work, we investigate whether the retraining stage of co-adaptive BCIs can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. The aim of the current work was to evaluate whether a semi-supervised co-adaptive BCI could successfully compete with a supervised co-adaptive BCI model. In a supporting two-class (190 trials per condition) BCI study based on motor imagery tasks, we evaluated both approaches in two separate groups of 10 participants online, while we simulated the other approach in each group offline. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. We believe that these findings contribute to developing BCIs for long-term use, where continuous adaptation becomes imperative for maintaining meaningful BCI performance.

Graphical abstract