Single trial detection of error-related potentials in brain-machine interfaces: a survey and comparison of methods

Objective.Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain-computer interfaces (BCIs), increase the naturalness of human-machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms).Approach.With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification.Main results.From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination).Significance.This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

Unsupervised adaptation of an ECoG based brain-computer interface using neural correlates of task performance

Brain-computer interfaces (BCIs) translate brain signals into commands to external effectors, and mainly target severely disabled users. The usability of BCIs may be improved by reducing their major constraints, such as the necessity for special training sessions to initially calibrate and later keep up to date the neural signal decoders. In this study, we show that it is possible to train and update BCI decoders during free use of motor BCIs. In addition to the neural signal decoder controlling effectors (control decoder), one more classifier is proposed to detect neural correlates of BCI motor task performances (MTP). MTP decoders reveal whether the actions performed by BCI effectors matched the user's intentions. The combined outputs of MTP and control decoders allow forming training datasets to update the control decoder online and in real time during free use of BCIs. The usability of the proposed auto-adaptive BCI (aaBCI) is demonstrated for two principle BCIs paradigms: with discrete outputs (4 classes BCI, virtual 4-limb exoskeleton), and with continuous outputs (cursor 2D control). The proof of concept was performed in an online simulation study using an ECoG dataset collected from a tetraplegic during a BCI clinical trial. The control decoder reached a multiclass area under the ROC curve of 0.7404 using aaBCI, compared to a chance level of 0.5173 and to 0.8187 for supervised training for the multiclass BCI, and a cosine similarity of 0.1211 using aaBCI, compared to a chance level of 0.0036 and to 0.2002 for supervised training for the continuous BCI.

Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier

For brain-computer interface (BCI) users, the awareness of an error is associated with a cortical signature known as an error-related potential (ErrP). The incorporation of ErrP detection into BCIs can improve their performance.

OBJECTIVE

This work has three main aims. First, we investigate whether an ErrP classifier is transferable from able-bodied participants to participants with a spinal cord injury (SCI). Second, we test this generic ErrP classifier with SCI and control participants, in an online experiment without offline calibration. Third, we investigate the morphology of ErrPs in both groups of participants.

APPROACH

We used previously recorded electroencephalographic data from able-bodied participants to train an ErrP classifier. We tested the classifier asynchronously, in an online experiment with 16 new participants: 8 participants with SCI and 8 able-bodied control participants. The experiment had no offline calibration and participants received feedback regarding the ErrP detections from the start. To increase the fluidity of the experiment, feedback regarding false positive ErrP detections was not presented to the participants, but these detections were taken into account in the evaluation of the classifier. The generic classifier was not trained with the user's brain signals. However, its performance was optimized during the online experiment by the use of personalized decision thresholds. The classifier's performance was evaluated using trial-based metrics, which considered the asynchronous detection of ErrPs during the entire trial's duration.

MAIN RESULTS

Participants with SCI presented a non-homogenous ErrP morphology, and four of them did not present clear ErrP signals. The generic classifier performed better than chance in participants with clear ErrP signals, independently of the SCI (11 out of 16 participants). Three out of the five participants that obtained chance level results with the generic classifier would have not benefitted from the use of a personalized classifier.

SIGNIFICANCE

This work shows the feasibility of transferring an ErrP classifier from able-bodied participants to participants with SCI, for asynchronous detection of ErrPs in an online experiment without offline calibration, which provided immediate feedback to the users.

The value-complexity trade-off for reinforcement learning based brain-computer interfaces

OBJECTIVE

One of the recent developments in the field of brain-computer interfaces (BCI) is the reinforcement learning (RL) based BCI paradigm, which uses neural error responses as the reward feedback on the agent's action. While having several advantages over motor imagery based BCI, the reliability of RL-BCI is critically dependent on the decoding accuracy of noisy neural error signals. A principled method is needed to optimally handle this inherent noise under general conditions.

APPROACH

By determining a trade-off between the expected value and the informational cost of policies, the info-RL (IRL) algorithm provides optimal low-complexity policies, which are robust under noisy reward conditions and achieve the maximal obtainable value. In this work we utilize the IRL algorithm to characterize the maximal obtainable value under different noise levels, which in turn is used to extract the optimal robust policy for each noise level.

MAIN RESULTS

Our simulation results of a setting with Gaussian noise show that the complexity level of the optimal policy is dependent on the reward magnitude but not on the reward variance, whereas the variance determines whether a lower complexity solution is favorable or not. We show how this analysis can be utilized to select optimal robust policies for an RL-BCI and demonstrate its use on EEG data.

SIGNIFICANCE

We propose here a principled method to determine the optimal policy complexity of an RL problem with a noisy reward, which we argue is particularly useful for RL-based BCI paradigms. This framework may be used to minimize initial training time and allow for a more dynamic and robust shared control between the agent and the operator under different conditions.

Adaptive classification to reduce non-stationarity in visual evoked potential brain-computer interfaces

Non-stationarity of electroencephalogram (EEG) signals greatly affect classifier performance in brain-computer interface (BCI). To overcome this problem we propose an adaptive classifier model known as extended multi-class pooled mean linear discriminant analysis (EMPMLDA). Here, we update the average class pair co-variance matrix along with pooled mean values. Evaluation of classifiers are done on visual evoked cortical potential data-sets. We demonstrate that EMPMLDA can significantly outperform other static classifiers such as MLDA and adaptive classifiers (MPMLDA). Furthermore an optimal update coefficient can be achieved using different datasets.

Masked and unmasked error-related potentials during continuous control and feedback

The detection of error-related potentials (ErrPs) in tasks with discrete feedback is well established in the brain-computer interface (BCI) field. However, the decoding of ErrPs in tasks with continuous feedback is still in its early stages.

OBJECTIVE

We developed a task in which subjects have continuous control of a cursor's position by means of a joystick. The cursor's position was shown to the participants in two different modalities of continuous feedback: normal and jittered. The jittered feedback was created to mimic the instability that could exist if participants controlled the trajectory directly with brain signals.

APPROACH

This paper studies the electroencephalographic (EEG)-measurable signatures caused by a loss of control over the cursor's trajectory, causing a target miss.

MAIN RESULTS

In both feedback modalities, time-locked potentials revealed the typical frontal-central components of error-related potentials. Errors occurring during the jittered feedback (masked errors) were delayed in comparison to errors occurring during normal feedback (unmasked errors). Masked errors displayed lower peak amplitudes than unmasked errors. Time-locked classification analysis allowed a good distinction between correct and error classes (average Cohen-[Formula: see text], average TPR  =  81.8% and average TNR  =  96.4%). Time-locked classification analysis between masked error and unmasked error classes revealed results at chance level (average Cohen-[Formula: see text], average TPR  =  60.9% and average TNR  =  58.3%). Afterwards, we performed asynchronous detection of ErrPs, combining both masked and unmasked trials. The asynchronous detection of ErrPs in a simulated online scenario resulted in an average TNR of 84.0% and in an average TPR of 64.9%.

SIGNIFICANCE

The time-locked classification results suggest that the masked and unmasked errors were indistinguishable in terms of classification. The asynchronous classification results suggest that the feedback modality did not hinder the asynchronous detection of ErrPs.

Designing Guiding Systems for Brain-Computer Interfaces

Brain–Computer Interface (BCI) community has focused the majority of its research efforts on signal processing and machine learning, mostly neglecting the human in the loop. Guiding users on how to use a BCI is crucial in order to teach them to produce stable brain patterns. In this work, we explore the instructions and feedback for BCIs in order to provide a systematic taxonomy to describe the BCI guiding systems. The purpose of our work is to give necessary clues to the researchers and designers in Human–Computer Interaction (HCI) in making the fusion between BCIs and HCI more fruitful but also to better understand the possibilities BCIs can provide to them.

Context-aware adaptive spelling in motor imagery BCI

OBJECTIVE

This work presents a first motor imagery-based, adaptive brain-computer interface (BCI) speller, which is able to exploit application-derived context for improved, simultaneous classifier adaptation and spelling. Online spelling experiments with ten able-bodied users evaluate the ability of our scheme, first, to alleviate non-stationarity of brain signals for restoring the subject's performances, second, to guide naive users into BCI control avoiding initial offline BCI calibration and, third, to outperform regular unsupervised adaptation.

APPROACH

Our co-adaptive framework combines the BrainTree speller with smooth-batch linear discriminant analysis adaptation. The latter enjoys contextual assistance through BrainTree's language model to improve online expectation-maximization maximum-likelihood estimation.

MAIN RESULTS

Our results verify the possibility to restore single-sample classification and BCI command accuracy, as well as spelling speed for expert users. Most importantly, context-aware adaptation performs significantly better than its unsupervised equivalent and similar to the supervised one. Although no significant differences are found with respect to the state-of-the-art PMean approach, the proposed algorithm is shown to be advantageous for 30% of the users.

SIGNIFICANCE

We demonstrate the possibility to circumvent supervised BCI recalibration, saving time without compromising the adaptation quality. On the other hand, we show that this type of classifier adaptation is not as efficient for BCI training purposes.

Adaptive Stacked Generalization for Multiclass Motor Imagery-Based Brain Computer Interfaces

Practical motor imagery-based brain computer interface (MI-BCI) applications are limited by the difficult to decode brain signals in a reliable way. In this paper, we propose a processing framework to address non-stationarity, as well as handle spectral, temporal, and spatial characteristics associated with execution of motor tasks. Stacked generalization is used to exploit the power of classifier ensembles for combining information coming from multiple sources and reducing the existing uncertainty in EEG signals. The outputs of several regularized linear discriminant analysis (RLDA) models are combined to account for temporal, spatial, and spectral information. The resultant algorithm is called stacked RLDA (SRLDA). Additionally, an adaptive processing stage is introduced before classification to reduce the harmful effect of intersession non-stationarity. The benefits of the proposed method are evaluated on the BCI Competition IV dataset 2a. We demonstrate its effectiveness in binary and multiclass settings with four different motor imagery tasks: left-hand, right-hand, both feet, and tongue movements. The results show that adaptive SRLDA outperforms the winner of the competition and other approaches tested on this multiclass dataset.

Quantitative analysis of task selection for brain-computer interfaces

OBJECTIVE

To assess quantitatively the impact of task selection in the performance of brain-computer interfaces (BCI).

APPROACH

We consider the task-pairs derived from multi-class BCI imagery movement tasks in three different datasets. We analyze for the first time the benefits of task selection on a large-scale basis (109 users) and evaluate the possibility of transferring task-pair information across days for a given subject.

MAIN RESULTS

Selecting the subject-dependent optimal task-pair among three different imagery movement tasks results in approximately 20% potential increase in the number of users that can be expected to control a binary BCI. The improvement is observed with respect to the best task-pair fixed across subjects. The best task-pair selected for each subject individually during a first day of recordings is generally a good task-pair in subsequent days. In general, task learning from the user side has a positive influence in the generalization of the optimal task-pair, but special attention should be given to inexperienced subjects.

SIGNIFICANCE

These results add significant evidence to existing literature that advocates task selection as a necessary step towards usable BCIs. This contribution motivates further research focused on deriving adaptive methods for task selection on larger sets of mental tasks in practical online scenarios.

Errare machinale est: the use of error-related potentials in brain-machine interfaces

The ability to recognize errors is crucial for efficient behavior. Numerous studies have identified electrophysiological correlates of error recognition in the human brain (error-related potentials, ErrPs). Consequently, it has been proposed to use these signals to improve human-computer interaction (HCI) or brain-machine interfacing (BMI). Here, we present a review of over a decade of developments toward this goal. This body of work provides consistent evidence that ErrPs can be successfully detected on a single-trial basis, and that they can be effectively used in both HCI and BMI applications. We first describe the ErrP phenomenon and follow up with an analysis of different strategies to increase the robustness of a system by incorporating single-trial ErrP recognition, either by correcting the machine's actions or by providing means for its error-based adaptation. These approaches can be applied both when the user employs traditional HCI input devices or in combination with another BMI channel. Finally, we discuss the current challenges that have to be overcome in order to fully integrate ErrPs into practical applications. This includes, in particular, the characterization of such signals during real(istic) applications, as well as the possibility of extracting richer information from them, going beyond the time-locked decoding that dominates current approaches.

Adaptive Multiclass Classification for Brain Computer Interfaces

We consider the problem of multiclass adaptive classification for brain-computer interfaces and propose the use of multiclass pooled mean linear discriminant analysis (MPMLDA), a multiclass generalization of the adaptation rule introduced by Vidaurre, Kawanabe, von Bünau, Blankertz, and Müller ( 2010 ) for the binary class setting. Using publicly available EEG data sets and tangent space mapping (Barachant, Bonnet, Congedo, & Jutten, 2012 ) as a feature extractor, we demonstrate that MPMLDA can significantly outperform state-of-the-art multiclass static and adaptive methods. Furthermore, efficient learning rates can be achieved using data from different subjects.

Brain connectivity in continuous error tasks

Error-related potentials (ErrP) have been recently incorporated in brain-machine interfaces (BMIs) due to its ability to adapt and correct both the output of the BMI or the behavior of the machine. Most of these applications rely on synchronous tasks with different user's evaluations associated to correct and wrong events. Asynchronous detection during the continuous evaluation of the task, however, has to cope with background noise and an increased number of misdetections common in event-related potential detection. This paper studies a different characteristic that may carry additional information to be exploited by asynchronous ErrP detectors: brain connectivity coherence patterns appearing while the user monitors the continuous operation of a device. The results obtained with five subject revealed the presence of an error potential in an asynchronous reaching task an showed an increase in the coherency within the theta band.

Discriminative Learning of Propagation and Spatial Pattern for Motor Imagery EEG Analysis

Effective learning and recovery of relevant source brain activity patterns is a major challenge to brain-computer interface using scalp EEG. Various spatial filtering solutions have been developed. Most current methods estimate an instantaneous demixing with the assumption of uncorrelatedness of the source signals. However, recent evidence in neuroscience suggests that multiple brain regions cooperate, especially during motor imagery, a major modality of brain activity for brain-computer interface. In this sense, methods that assume uncorrelatedness of the sources become inaccurate. Therefore, we are promoting a new methodology that considers both volume conduction effect and signal propagation between multiple brain regions. Specifically, we propose a novel discriminative algorithm for joint learning of propagation and spatial pattern with an iterative optimization solution. To validate the new methodology, we conduct experiments involving 16 healthy subjects and perform numerical analysis of the proposed algorithm for EEG classification in motor imagery brain-computer interface. Results from extensive analysis validate the effectiveness of the new methodology with high statistical significance.