A systematic evaluation of Euclidean alignment with deep learning for EEG decoding

Electroencephalography (EEG) signals are frequently used for various Brain-Computer Interface (BCI) tasks. While Deep Learning (DL) techniques have shown promising results, they are hindered by the substantial data requirements. By leveraging data from multiple subjects, transfer learning enables more effective training of DL models. A technique that is gaining popularity is Euclidean Alignment (EA) due to its ease of use, low computational complexity, and compatibility with Deep Learning models. However, few studies evaluate its impact on the training performance of shared and individual DL models. In this work, we systematically evaluate the effect of EA combined with DL for decoding BCI signals. We used EA to train shared models with data from multiple subjects and evaluated its transferability to new subjects. Our experimental results show that it improves decoding in the target subject by 4.33% and decreases convergence time by more than 70%. We also trained individual models for each subject to use as a majority-voting ensemble classifier. In this scenario, using EA improved the 3-model ensemble accuracy by 3.7%. However, when compared to the shared model with EA, the ensemble accuracy was 3.62% lower.

Association between lesion location and sensorimotor rhythms in stroke - a systematic review with narrative synthesis

BACKGROUND

Stroke causes alterations in the sensorimotor rhythms (SMRs) of the brain. However, little is known about the influence of lesion location on the SMRs. Understanding this relationship is relevant for the use of SMRs in assistive and rehabilitative therapies, such as Brain-Computer Interfaces (BCIs)..

METHODS

We reviewed current evidence on the association between stroke lesion location and SMRs through systematically searching PubMed and Embase and generated a narrative synthesis of findings.

RESULTS

We included 12 articles reporting on 161 patients. In resting-state studies, cortical and pontine damage were related to an overall decrease in alpha (∼8-12 Hz) and increase in delta (∼1-4 Hz) power. In movement paradigm studies, attenuated alpha and beta (∼15-25 Hz) event-related desynchronization (ERD) was shown in stroke patients during (attempted) paretic hand movement, compared to controls. Stronger reductions in alpha and beta ERD in the ipsilesional, compared to contralesional hemisphere, were observed for cortical lesions. Subcortical stroke was found to affect bilateral ERD and ERS, but results were highly variable.

CONCLUSIONS

Findings suggest a link between stroke lesion location and SMR alterations, but heterogeneity across studies and limited lesion location descriptions precluded a meta-analysis.

SIGNIFICANCE

Future research would benefit from more uniformly defined outcome measures, homogeneous methodologies, and improved lesion location reporting.

Effects of the concentration level, eye fatigue and coffee consumption on the performance of a BCI system based on visual ERP-P300

BACKGROUND

A widely used paradigm for Brain-Computer Interfaces (BCI) is based on detecting P300 Event-Related Potentials (ERPs) in response to stimulation and concentration tasks. An open challenge corresponds to maximizing the performance of a BCI by considering artifacts arising from the user's cognitive and physical conditions during task execution.

NEW METHOD

In this study, an analysis of the performance of a visual BCI-P300 system was performed under the metrics of Sensitivity (Sen), Specificity (Spe), Accuracy (Acc), and Area-Under the ROC Curve (AUC), considering the main reported factors affecting the neurophysiological behavior of the P300 signal: Concentration Level, Eye Fatigue, and Coffee Consumption.

COMPARISON WITH EXISTING METHODS

We compared the performance of three P300 signal detection methods (MA-LDA, CCA-RLR, and MA+CCA-RLR) using a public database (GigaScience) in different groups. Data were segmented according to three factors of interest: high and low levels of concentration, high and low eye fatigue, and coffee consumption at different times.

RESULTS

The results showed a significant improvement between 3% and 6% for the metrics evaluated for identifying the P300 signal in relation to concentration levels and coffee consumption.

CONCLUSION

P300 signal can be influenced by physical and mental factors during the execution of ERPs evocation tasks, which could be controlled to maximize the interface's capacity to detect the individual's intention.

A Transfer Learning Algorithm to Reduce Brain-Computer Interface Calibration Time for Long-Term Users

Current motor imagery-based brain-computer interface (BCI) systems require a long calibration time at the beginning of each session before they can be used with adequate levels of classification accuracy. In particular, this issue can be a significant burden for long term BCI users. This article proposes a novel transfer learning algorithm, called r-KLwDSA, to reduce the BCI calibration time for long-term users. The proposed r-KLwDSA algorithm aligns the user's EEG data collected in previous sessions to the few EEG trials collected in the current session, using a novel linear alignment method. Thereafter, the aligned EEG trials from the previous sessions and the few EEG trials from the current sessions are fused through a weighting mechanism before they are used for calibrating the BCI model. To validate the proposed algorithm, a large dataset containing the EEG data from 11 stroke patients, each performing 18 BCI sessions, was used. The proposed framework demonstrated a significant improvement in the classification accuracy, of over 4% compared to the session-specific algorithm, when there were as few as two trials per class available from the current session. The proposed algorithm was particularly successful in improving the BCI accuracy of the sessions that had initial session-specific accuracy below 60%, with an average improvement of around 10% in the accuracy, leading to more stroke patients having meaningful BCI rehabilitation.

Analytic beamformer transformation for transfer learning in motion-onset visual evoked potential decoding

OBJECTIVE

While decoders of EEG-based event-related potentials (ERPs) are routinely tailored to the individual user to maximize performance, developing them on populations for individual usage has proven much more challenging. We propose the analytic beamformer transformation (ABT) to extract phase and/or magnitude information from spatiotemporal ERPs in response to motion-onset stimulation.

APPROACH

We have tested ABT on 52 motion-onset visual evoked potential (mVEP) datasets from 26 healthy subjects and compared the classification accuracy of support vector machine (SVM), spatiotemporal beamformer (stBF) and stepwise linear discriminant analysis (SWLDA) when trained on individual subjects and on a population thereof.

MAIN RESULTS

When using phase- and combined phase/magnitude information extracted by ABT, we show significant improvements in accuracy of population-trained classifiers applied to individual users (p<0.001). We also show that 450 epochs are needed for a correct functioning of ABT, which corresponds to 2 minutes of paradigm stimulation.

SIGNIFICANCE

We have shown that ABT can be used to create population-trained mVEP classifiers using a limited number of epochs. We expect this to pertain to other ERPs or synchronous stimulation paradigms, allowing for a more effective, population-based training of visual BCIs. Finally, as ABT renders recordings across subjects more structurally invariant, it could be used for transfer learning purposes in view of plug-and-play BCI applications.

Comparative Analysis of Spectral and Temporal Combinations in CSP-based Methods for Decoding Hand Motor Imagery Tasks

BACKGROUND

A widely used paradigm for brain-computer interfaces (BCI) is based on the detection of event-related (des)synchronization (ERD/S) in response to hand motor imagery (MI) tasks. The common spatial pattern (CSP) has been recognized as a powerful algorithm to design spatial filters for ERD/ERS detection. However, a limitation of CSP focus on identification only of discriminative spatial information but not the spectral one.

NEW METHOD

An open problem remains in literature related to extracting the most discriminative brain patterns in MI-based BCIs using an optimal time segment and spectral information that accounts for intersubject variability. In recent years, different variants of CSP-based methods have been proposed to address the problem of decoding motor imagery tasks under the intersubject variability of frequency bands related to ERD/ERS events, including Filter Bank Common Spatial Patterns (FBCSP) and Filter Bank Common Spatio-Spectral Patterns (FBCSSP).

COMPARISON WITH EXISTING METHODS

We performed a comparative study of different combinations of time segments and filter banks for three methods (CSP, FBCSP, and FBCSSP) to decode hand (right and left) motor imagery tasks using two different EEG datasets (Gigascience and BCI IVa competition).

RESULTS

The best configuration corresponds to a filter bank with 3 filters (8-15 Hz, 15-22 Hz and 22-29 Hz) using a time window of 1.5 seconds after the trigger, which provide accuracies of approximately 74% and an estimated ITRs of approximately 7 bits/min.

CONCLUSION

Discriminative information in time and spectral domains could be obtained using a convenient filter bank and a time segment configuration, to enhance the classification rate and ITR for detection of hand motor imagery tasks with CSP-related methods, to be used in the implementation of a real-time BCI system.

Assessment of Motor Cortex in Active, Passive and Imagery Wrist Movement Using Functional MRI

Background:

Functional Magnetic resonance imaging (fMRI) measures the small fluctuation of blood flow happening during task-fMRI in brain regions.

Objective:

This research investigated these active, imagery and passive movements in volunteers design to permit a comparison of their capabilities in activating the brain areas.

Material and Methods:

In this applied research, the activity of the motor cortex during the right-wrist movement was evaluated in 10 normal volunteers under active, passive, and imagery conditions. T2* weighted, three-dimensional functional images were acquired using a BOLD sensitive gradient-echo EPI (echo planar imaging) sequence with echo time (TE) of 30 ms and repetition time (TR) of 2000 ms. The functional data, which included 248 volumes per subject and condition, were acquired using the blocked design paradigm. The images were analyzed by the SPM12 toolbox, MATLAB software.

Results:

The findings determined a significant increase in signal intensity of the motor cortex while performing the test compared to the rest time (p< 0.05). It was also observed that the active areas in hand representation of the motor cortex are different in terms of locations and the number of voxels in different wrist directions. Moreover, the findings showed that the position of active centers in the brain is different in active, passive, and imagery conditions.

Conclusion:

Results confirm that primary motor cortex neurons play an essential role in the processing of complex information and are designed to control the direction of movement. It seems that the findings of this study can be applied for rehabilitation studies.

A New Method to Improve the Performance of Deep Neural Networks in Detecting P300 Signals: Optimizing Curvature of Error Surface Using Genetic Algorithm

Background:

Deep neural networks have been widely used in detection of P300 signal in Brain Machine Interface (BMI) systems which are rely on Event-Related Potentials (ERPs) (i.e. P300 signals). Such networks have high curvature variation in their error surface hampering their favorable performance. Therefore, the variations in curvature of the error surface must be minimized to improve the performance of these networks in detecting P300 signals.

Objective:

The aim of this paper is to introduce a method for minimizing the curvature of the error surface during training Convolutional Neural Network (CNN). The curvature variation of the error surface is highly dependent on model parameters of deep neural network; therefore, we try to minimize this curvature by optimizing the model parameters.

Material and Methods:

In this experimental study an attempt is made to tune the CNN parameters affecting the curvature of its error surface in order to obtain the best possible learning. For achieving this goal, Genetic Algorithm is utilized to optimize the above parameters in order to minimize the curvature variations.

Results:

The performance of the proposed algorithm was evaluated on EPFL dataset. The obtained results demonstrated that the proposed method detected the P300 signals with maximally 98.91% classification accuracy and 98.54% True Positive Ratio (TPR).

Conclusion:

The obtained results showed that using genetic algorithm for minimizing curvature of the error surface in CNN increased its accuracy in parallel with decreasing the variance of the results. Consequently, it may be concluded that the proposed method has considerable potential to be used as P300 detection module in BMI applications.

Comparison of Smoothing Filters' Influence on Quality of Data Recorded with the Emotiv EPOC Flex Brain-Computer Interface Headset during Audio Stimulation

Off-the-shelf, consumer-grade EEG equipment is nowadays becoming the first-choice equipment for many scientists when it comes to recording brain waves for research purposes. On one hand, this is perfectly understandable due to its availability and relatively low cost (especially in comparison to some clinical-level EEG devices), but, on the other hand, quality of the recorded signals is gradually increasing and reaching levels that were offered just a few years ago by much more expensive devices used in medicine for diagnostic purposes. In many cases, a well-designed filter and/or a well-thought signal acquisition method improve the signal quality to the level that it becomes good enough to become subject of further analysis allowing to formulate some valid scientific theories and draw far-fetched conclusions related to human brain operation. In this paper, we propose a smoothing filter based upon the Savitzky-Golay filter for the purpose of EEG signal filtering. Additionally, we provide a summary and comparison of the applied filter to some other approaches to EEG data filtering. All the analyzed signals were acquired from subjects performing visually involving high-concentration tasks with audio stimuli using Emotiv EPOC Flex equipment.

Summary of over Fifty Years with Brain-Computer Interfaces-A Review

Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.

Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from “BCI Competition IV”. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

Trends in research participant categories and descriptions in abstracts from the International BCI Meeting series, 1999 to 2016

Much brain-computer interface (BCI) research is intended to benefit people with disabilities (PWD), but inclusion of these individuals as study participants remains relatively rare. When participants with disabilities are included, they are described with a range of clinical and non-clinical terms with varying degrees of specificity, often leading to difficulty in interpreting or replicating results. This study examined trends in inclusion and description of study participants with disabilities across six International BCI Meetings from 1999 to 2016. Abstracts from each Meeting were analyzed by two trained independent reviewers. Results suggested a decline in participation by PWD across Meetings until the 2016 Meeting. Increased diagnostic specificity was noted at the 2013 and 2016 Meetings. Fifty-eight percent of the abstracts identified PWD as being the target beneficiaries of BCI research, though only twenty-two percent included participants with disabilities, suggesting evidence of a persistent translational gap. Participants with disabilities were most commonly described as having physical and/or communication impairments compared to impairments in other areas. Implementing participatory action research principles and user-centered design strategies continues to be necessary within BCI research to bridge the translational gap and facilitate use of BCI systems within functional environments for PWD.

Improving P300 Spelling Rate using Language Models and Predictive Spelling

The P300 Speller Brain-Computer Interface (BCI) provides a means of communication for those suffering from advanced neuromuscular diseases such as amyotrophic lateral sclerosis (ALS). Recent literature has incorporated language-based modelling, which uses previously chosen characters and the structure of natural language to modify the interface and classifier. Two complementary methods of incorporating language models have previously been independently studied: predictive spelling uses language models to generate suggestions of complete words to allow for the selection of multiple characters simultaneously, and language model-based classifiers have used prior characters to create a prior probability distribution over the characters based on how likely they are to follow. In this study, we propose a combined method which extends a language-based classifier to generate prior probabilities for both individual characters and complete words. In order to gauge the efficiency of this new model, results across 12 healthy subjects were measured. Incorporating predictive spelling increased typing speed using the P300 speller, with an average increase of 15.5% in typing rate across subjects, demonstrating that language models can be effectively utilized to create full word suggestions for predictive spelling. When combining predictive spelling with language model classification, typing speed is significantly improved, resulting in better typing performance.

Decoding Saccadic Directions Using Epidural ECoG in Non-Human Primates

A brain-computer interface (BCI) can be used to restore some communication as an alternative interface for patients suffering from locked-in syndrome. However, most BCI systems are based on SSVEP, P300, or motor imagery, and a diversity of BCI protocols would be needed for various types of patients. In this paper, we trained the choice saccade (CS) task in 2 non-human primate monkeys and recorded the brain signal using an epidural electrocorticogram (eECoG) to predict eye movement direction. We successfully predicted the direction of the upcoming eye movement using a support vector machine (SVM) with the brain signals after the directional cue onset and before the saccade execution. The mean accuracies were 80% for 2 directions and 43% for 4 directions. We also quantified the spatial-spectro-temporal contribution ratio using SVM recursive feature elimination (RFE). The channels over the frontal eye field (FEF), supplementary eye field (SEF), and superior parietal lobule (SPL) area were dominantly used for classification. The α-band in the spectral domain and the time bins just after the directional cue onset and just before the saccadic execution were mainly useful for prediction. A saccade based BCI paradigm can be projected in the 2D space, and will hopefully provide an intuitive and convenient communication platform for users.

SOLICITING BCI USER EXPERIENCE FEEDBACK FROM PEOPLE WITH SEVERE SPEECH AND PHYSICAL IMPAIRMENTS

Brain-computer interface (BCI) researchers have shown increasing interest in soliciting user experience (UX) feedback, but the severe speech and physical impairments (SSPI) of potential users create barriers to effective implementation with existing feedback instruments. This article describes augmentative and alternative communication (AAC)-based techniques for obtaining feedback from this population, and presents results from administration of a modified questionnaire to 12 individuals with SSPI after trials with a BCI spelling system. The proposed techniques facilitated successful questionnaire completion and provision of narrative feedback for all participants. Questionnaire administration required less than five minutes and minimal effort from participants. Results indicated that individual users may have very different reactions to the same system, and that ratings of workload and comfort provide important information not available through objective performance measures. People with SSPI are critical stakeholders in the future development of BCI, and appropriate adaptation of feedback questionnaires and administration techniques allows them to participate in shaping this assistive technology.

Improving BCI performance through co-adaptation: applications to the P300-speller

A well-known neurophysiological marker that can easily be captured with electroencephalography (EEG) is the so-called P300: a positive signal deflection occurring at about 300 ms after a relevant stimulus. This brain response is particularly salient when the target stimulus is rare among a series of distracting stimuli, whatever the type of sensory input. Therefore, it has been proposed and extensively studied as a possible feature for direct brain-computer communication. The most advanced non-invasive BCI application based on this principle is the P300-speller. However, it is still a matter of debate whether this application will prove relevant to any population of patients. In a series of recent theoretical and empirical studies, we have been using this P300-based paradigm to push forward the performance of non-invasive BCI. This paper summarizes the proposed improvements and obtained results. Importantly, those could be generalized to many kinds of BCI, beyond this particular application. Indeed, they relate to most of the key components of a closed-loop BCI, namely: improving the accuracy of the system by trying to detect and correct for errors automatically; optimizing the computer's speed-accuracy trade-off by endowing it with adaptive behavior; but also simplifying the hardware and time for set-up in the aim of routine use in patients. Our results emphasize the importance of the closed-loop interaction and of the ensuing co-adaptation between the user and the machine whenever possible. Most of our evaluations have been conducted in healthy subjects. We conclude with perspectives for clinical applications.