Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces

Influence of pitch modulation on event-related potentials elicited by Dutch word stimuli in a brain-computer interface language rehabilitation task

Objective.Recently, a novel language training using an auditory brain-computer interface (BCI) based on electroencephalogram recordings has been proposed for chronic stroke patients with aphasia. Tested with native German patients, it has shown significant and medium to large effect sizes in improving multiple aspects of language. During the training, the auditory BCI system delivers word stimuli using six spatially arranged loudspeakers. As delivering the word stimuli via headphones reduces spatial cues and makes the attention to target words more difficult, we investigate the influence of added pitch information. While pitch modulations have shown benefits for tone stimuli, they have not yet been investigated in the context of language stimuli.Approach.The study translated the German experimental setup into Dutch. Seventeen native Dutch speakers participated in a single session of an exploratory study. An incomplete Dutch sentence cued them to listen to a target word embedded into a sequence of comparable non-target words while an electroencephalogram was recorded. Four conditions were compared within-subject to investigate the influence of pitch modulation: presenting the words spatially from six loudspeakers without (6D) and with pitch modulation (6D-Pitch), via stereo headphones with simulated spatial cues and pitch modulation (Stereo-Pitch), and via headphones without spatial cues or pitch modulation (Mono).Main results.Comparing the 6D conditions of both language setups, the Dutch setup could be validated. For the Dutch setup, the binary AUC classification score in the 6D and the 6D-Pitch condition were 0.75 and 0.76, respectively, and adding pitch information did not significantly alter the binary classification accuracy of the event-related potential responses. The classification scores in the 6D condition and the Stereo-Pitch condition were on the same level.Significance.The competitive performance of pitch-modulated word stimuli suggests that the complex hardware setup of the 6D condition could be replaced by a headphone condition. If future studies with aphasia patients confirm the effectiveness and higher usability of a headphone-based language rehabilitation training, a simplified setup could be implemented more easily outside of clinics to deliver frequent training sessions to patients in need.

Four-class ASME BCI: investigation of the feasibility and comparison of two strategies for multiclassing

Introduction

The ASME (stands for Auditory Stream segregation Multiclass ERP) paradigm is proposed and used for an auditory brain-computer interface (BCI). In this paradigm, a sequence of sounds that are perceived as multiple auditory streams are presented simultaneously, and each stream is an oddball sequence. The users are requested to focus selectively on deviant stimuli in one of the streams, and the target of the user attention is detected by decoding event-related potentials (ERPs). To achieve multiclass ASME BCI, the number of streams must be increased. However, increasing the number of streams is not easy because of a person's limited audible frequency range. One method to achieve multiclass ASME with a limited number of streams is to increase the target stimuli in a single stream.

Methods

Two approaches for the ASME paradigm, ASME-4stream (four streams with a single target stimulus in each stream) and ASME-2stream (two streams with two target stimuli in each stream) were investigated. Fifteen healthy subjects with no neurological disorders participated in this study. An electroencephalogram was acquired, and ERPs were analyzed. The binary classification and BCI simulation (detecting the target class of the trial out of four) were conducted with the help of linear discriminant analysis, and its performance was evaluated offline. Its usability and workload were also evaluated using a questionnaire.

Results

Discriminative ERPs were elicited in both paradigms. The average accuracies of the BCI simulations were 0.83 (ASME-4stream) and 0.86 (ASME-2stream). In the ASME-2stream paradigm, the latency and the amplitude of P300 were shorter and larger, the average binary classification accuracy was higher, and the average weighted workload was smaller.

Discussion

Both four-class ASME paradigms achieved a sufficiently high accuracy (over 80%). The shorter latency and larger amplitude of P300 and the smaller workload indicated that subjects could perform the task confidently and had high usability in ASME-2stream compared to ASME-4stream paradigm. A paradigm with multiple target stimuli in a single stream could create a multiclass ASME BCI with limited streams while maintaining task difficulty. These findings expand the potential for an ASME BCI multiclass extension, offering practical auditory BCI choices for users.

Review of deep representation learning techniques for brain-computer interfaces

In the field of brain-computer interfaces (BCIs), the potential for leveraging deep learning techniques for representing electroencephalogram (EEG) signals has gained substantial interest.Objective: This review synthesizes empirical findings from a collection of articles using deep representation learning techniques for BCI decoding, to provide a comprehensive analysis of the current state-of-the-art.Approach: Each article was scrutinized based on three criteria: (1) the deep representation learning technique employed, (2) the underlying motivation for its utilization, and (3) the approaches adopted for characterizing the learned representations.Main results: Among the 81 articles finally reviewed in depth, our analysis reveals a predominance of 31 articles using autoencoders. We identified 13 studies employing self-supervised learning (SSL) techniques, among which ten were published in 2022 or later, attesting to the relative youth of the field. However, at the time being, none of these have led to standard foundation models that are picked up by the BCI community. Likewise, only a few studies have introspected their learned representations. We observed that the motivation in most studies for using representation learning techniques is for solving transfer learning tasks, but we also found more specific motivations such as to learn robustness or invariances, as an algorithmic bridge, or finally to uncover the structure of the data.Significance: Given the potential of foundation models to effectively tackle these challenges, we advocate for a continued dedication to the advancement of foundation models specifically designed for EEG signal decoding by using SSL techniques. We also underline the imperative of establishing specialized benchmarks and datasets to facilitate the development and continuous improvement of such foundation models.

Classification of BCI-EEG Based on the Augmented Covariance Matrix

OBJECTIVE

Electroencephalography signals are recorded as multidimensional datasets. We propose a new framework based on the augmented covariance that stems from an autoregressive model to improve motor imagery classification.

METHODS

From the autoregressive model can be derived the Yule-Walker equations, which show the emergence of a symmetric positive definite matrix: the augmented covariance matrix. The state-of the art for classifying covariance matrices is based on Riemannian Geometry. A fairly natural idea is therefore to apply this Riemannian Geometry based approach to these augmented covariance matrices. The methodology for creating the augmented covariance matrix shows a natural connection with the delay embedding theorem proposed by Takens for dynamical systems. Such an embedding method is based on the knowledge of two parameters: the delay and the embedding dimension, respectively related to the lag and the order of the autoregressive model. This approach provides new methods to compute the hyper-parameters in addition to standard grid search.

RESULTS

The augmented covariance matrix performed ACMs better than any state-of-the-art methods. We will test our approach on several datasets and several subjects using the MOABB framework, using both within-session and cross-session evaluation.

CONCLUSION

The improvement in results is due to the fact that the augmented covariance matrix incorporates not only spatial but also temporal information. As such, it contains information on the nonlinear components of the signal through the embedding procedure, which allows the leveraging of dynamical systems algorithms.

SIGNIFICANCE

These results extend the concepts and the results of the Riemannian distance based classification algorithm.

Correcting for ERP latency jitter improves gaze-independent BCI decoding

Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

Competing Visual Cues Revealed by Electroencephalography: Sensitivity to Motion Speed and Direction

Motion speed and direction are two fundamental cues for the mammalian visual system. Neurons in various places of the neocortex show tuning properties in term of firing frequency to both speed and direction. The present study applied a 32-channel electroencephalograph (EEG) system to 13 human subjects while they were observing a single object moving with different speeds in various directions from the center of view to the periphery on a computer monitor. Depending on the experimental condition, the subjects were either required to fix their gaze at the center of the monitor while the object was moving or to track the movement with their gaze; eye-tracking glasses were used to ensure that they followed instructions. In each trial, motion speed and direction varied randomly and independently, forming two competing visual features. EEG signal classification was performed for each cue separately (e.g., 11 speed values or 11 directions), regardless of variations in the other cue. Under the eye-fixed condition, multiple subjects showed distinct preferences to motion direction over speed; however, two outliers showed superb sensitivity to speed. Under the eye-tracking condition, in which the EEG signals presumably contained ocular movement signals, all subjects showed predominantly better classification for motion direction. There was a trend that speed and direction were encoded by different electrode sites. Since EEG is a noninvasive and portable approach suitable for brain-computer interfaces (BCIs), this study provides insights on fundamental knowledge of the visual system as well as BCI applications based on visual stimulation.