A Cross-Session Dataset for Collaborative Brain-Computer Interfaces Based on Rapid Serial Visual Presentation

Attention-ProNet: A Prototype Network with Hybrid Attention Mechanisms Applied to Zero Calibration in Rapid Serial Visual Presentation-Based Brain-Computer Interface

The rapid serial visual presentation-based brain-computer interface (RSVP-BCI) system achieves the recognition of target images by extracting event-related potential (ERP) features from electroencephalogram (EEG) signals and then building target classification models. Currently, how to reduce the training and calibration time for classification models across different subjects is a crucial issue in the practical application of RSVP. To address this issue, a zero-calibration (ZC) method termed Attention-ProNet, which involves meta-learning with a prototype network integrating multiple attention mechanisms, was proposed in this study. In particular, multiscale attention mechanisms were used for efficient EEG feature extraction. Furthermore, a hybrid attention mechanism was introduced to enhance model generalization, and attempts were made to incorporate suitable data augmentation and channel selection methods to develop an innovative and high-performance ZC RSVP-BCI decoding model algorithm. The experimental results demonstrated that our method achieved a balance accuracy (BA) of 86.33% in the decoding task for new subjects. Moreover, appropriate channel selection and data augmentation methods further enhanced the performance of the network by affording an additional 2.3% increase in BA. The model generated by the meta-learning prototype network Attention-ProNet, which incorporates multiple attention mechanisms, allows for the efficient and accurate decoding of new subjects without the need for recalibration or retraining.

Multi-source domain adaptation based tempo-spatial convolution network for cross-subject EEG classification in RSVP task

Objective.Many subject-dependent methods were proposed for electroencephalogram (EEG) classification in rapid serial visual presentation (RSVP) task, which required a large amount of data from new subject and were time-consuming to calibrate system. Cross-subject classification can realize calibration reduction or zero calibration. However, cross-subject classification in RSVP task is still a challenge.Approach.This study proposed a multi-source domain adaptation based tempo-spatial convolution (MDA-TSC) network for cross-subject RSVP classification. The proposed network consisted of three modules. First, the common feature extraction with multi-scale tempo-spatial convolution was constructed to extract domain-invariant features across all subjects, which could improve generalization of the network. Second, the multi-branch domain-specific feature extraction and alignment was conducted to extract and align domain-specific feature distributions of source and target domains in pairs, which could consider feature distribution differences among source domains. Third, the domain-specific classifier was exploited to optimize the network through loss functions and obtain prediction for the target domain.Main results.The proposed network was evaluated on the benchmark RSVP dataset, and the cross-subject classification results showed that the proposed MDA-TSC network outperformed the reference methods. Moreover, the effectiveness of the MDA-TSC network was verified through both ablation studies and visualization.Significance.The proposed network could effectively improve cross-subject classification performance in RSVP task, and was helpful to reduce system calibration time.

Single-Trial EEG Classification Using Spatio-Temporal Weighting and Correlation Analysis for RSVP-Based Collaborative Brain Computer Interface

OBJECTIVE

Since single brain computer interface (BCI) is limited in performance, it is necessary to develop collaborative BCI (cBCI) systems which integrate multi-user electroencephalogram (EEG) information to improve system performance. However, there are still some challenges in cBCI systems, including effective discriminant feature extraction of multi-user EEG data, fusion algorithms, time reduction of system calibration, etc. Methods: This study proposed an event-related potential (ERP) feature extraction and classification algorithm of spatio-temporal weighting and correlation analysis (STC) to improve the performance of cBCI systems. The proposed STC algorithm consisted of three modules. First, source extraction and interval modeling were used to overcome the problem of inter-trial variability. Second, spatio-temporal weighting and temporal projection were utilized to extract effective discriminant features for multi-user information fusion and cross-session transfer. Third, correlation analysis was conducted to match target/non-target templates for classification of multi-user and cross-session datasets.

RESULTS

The collaborative cross-session datasets of rapid serial visual presentation (RSVP) from 14 subjects were used to evaluate the performance of the EEG classification algorithm. For single-user/collaborative EEG classification of within-session and cross-session datasets, STC had significantly higher performance than the existing state-of-the-art machine learning algorithms.

CONCLUSION

It was demonstrated that STC was effective to improve the classification performance of multi-user collaboration and cross-session transfer for RSVP-based BCI systems, and was helpful to reduce the system calibration time.

Single-trial P300 classification algorithm based on centralized multi-person data fusion CNN

Introduction

Currently, it is still a challenge to detect single-trial P300 from electroencephalography (EEG) signals. In this paper, to address the typical problems faced by existing single-trial P300 classification, such as complex, time-consuming and low accuracy processes, a single-trial P300 classification algorithm based on multiplayer data fusion convolutional neural network (CNN) is proposed to construct a centralized collaborative brain-computer interfaces (cBCI) for fast and highly accurate classification of P300 EEG signals.

Methods

In this paper, two multi-person data fusion methods (parallel data fusion and serial data fusion) are used in the data pre-processing stage to fuse multi-person EEG information stimulated by the same task instructions, and then the fused data is fed as input to the CNN for classification. In building the CNN network for single-trial P300 classification, the Conv layer was first used to extract the features of single-trial P300, and then the Maxpooling layer was used to connect the Flatten layer for secondary feature extraction and dimensionality reduction, thereby simplifying the computation. Finally batch normalisation is used to train small batches of data in order to better generalize the network and speed up single-trial P300 signal classification.

Results

In this paper, the above new algorithms were tested on the Kaggle dataset and the Brain-Computer Interface (BCI) Competition III dataset, and by analyzing the P300 waveform features and EEG topography and the four standard evaluation metrics, namely Accuracy, Precision, Recall and F1-score,it was demonstrated that the single-trial P300 classification algorithm after two multi-person data fusion CNNs significantly outperformed other classification algorithms.

Discussion

The results show that the single-trial P300 classification algorithm after two multi-person data fusion CNNs significantly outperformed the single-person model, and that the single-trial P300 classification algorithm with two multi-person data fusion CNNs involves smaller models, fewer training parameters, higher classification accuracy and improves the overall P300-cBCI classification rate and actual performance more effectively with a small amount of sample information compared to other algorithms.

A time-frequency denoising method for single-channel event-related EEG

Introduction

Electroencephalogram (EEG) acquisition is easily affected by various noises, including those from electrocardiogram (ECG), electrooculogram (EOG), and electromyogram (EMG). Because noise interference can significantly limit the study and analysis of brain signals, there is a significant need for the development of improved methods to remove this interference for more accurate measurement of EEG signals.

Methods

Based on the non-linear and non-stationary characteristics of brain signals, a strategy was developed to denoise brain signals using a time-frequency denoising algorithm framework of short-time Fourier transform (STFT), bidimensional empirical mode decomposition (BEMD), and non-local means (NLM). Time-frequency analysis can reveal the signal frequency component and its evolution process, allowing the elimination of noise according to the signal and noise distribution. BEMD can be used to decompose the time-frequency signals into sub-time-frequency signals for noise removal at different scales. NLM relies on structural self-similarity to locally smooth an image to remove noise and restore its main geometric structure, making this method appropriate for time-frequency signal denoising.

Results

The experimental results show that the proposed method can effectively suppress the high-frequency components of brain signals, resulting in a smoother brain signal waveform after denoising. The correlation coefficient of the reference signal, a superposition average of multiple trial signals, and the original single trial signal was determined, and then correlation coefficients were calculated between the reference signal and single trial signals processed by time-frequency denoising, ensemble empirical mode decomposition (EEMD)-independent component analysis (ICA), EEMD-canonical correlation analysis (CCA), and wavelet threshold denoising methods. The correlation coefficient was highest for the signal processed by the time-frequency denoising method and the reference signal, indicating that the single trial signal after time-frequency denoising was most similar to the waveform of the reference signal and suggesting this is a feasible strategy to effectively reduce noise and more accurately determine signals.

Discussion

The proposed time-frequency denoising method exhibits excellent performance with promising potential for practical application.

Winners and losers in brain computer interface competitive gaming: Directional connectivity analysis

OBJECTIVE

to characterize the direction within and between brain connectivity in winning and losing players in a competitive brain-computer interface game.

APPROACH

ten dyads (26.9 ± 4.7 years old, eight females and 12 males) participated in the study. In a competitive game based on neurofeedback, they used their relative alpha (RA) band power from the electrode location Pz, to control a virtual seesaw. The players in each pair were separated into winners (W) and losers (L) based on their scores. Intrabrain connectivity was analyzed using multivariate Granger Causality (GC) and Directed Transfer Function, while interbrain connectivity was analyzed using bivariate GC.

RESULTS

linear regression analysis revealed a significant relationship (p<0.05) between RA and individual scores. During the game, W players maintained a higher RA than L players, although it was not higher than their baseline RA. The analysis of intrabrain GC indicated that both groups engaged in general social interactions, but only the W group succeeded in controlling their brain activity at Pz. Group L applied an inappropriate metal strategy, characterized by strong activity in the left frontal cortex, indicative of collaborative gaming. Interbrain GC showed a larger flow of information from the L to the W group, suggesting a higher capability of the W group to monitor the activity of their opponent.

SIGNIFICANCE

both innate neurological indices and gaming mental strategies contribute to game outcomes. Future studies should investigate whether there is a causal relationship between these two factors.

A Spatio-Temporal Method for Extracting Gamma-Band Features to Enhance Classification in a Rapid Serial Visual Presentation Task

Rapid serial visual presentation (RSVP) is a type of electroencephalogram (EEG) pattern commonly used for target recognition. Besides delta- and theta-band responses already used for classification, RSVP task also evokes gamma-band responses having low amplitude and large individual difference. This paper proposes a filter bank spatio-temporal component analysis (FBSCA) method, extracting spatio-temporal features of the gamma-band responses for the first time, to enhance the RSVP classification performance. Considering the individual difference in time latency and responsive frequency, the proposed FBSCA method decomposes the gamma-band EEG data into sub-components in different time-frequency-space domains and seeks the weight coefficients to optimize the combinations of electrodes, common spatial pattern (CSP) components, time windows and frequency bands. Two state-of-the-art methods, i.e. hierarchical discriminant principal component analysis (HDPCA) and discriminative canonical pattern matching (DCPM), were used for comparison. The performance was evaluated in [Formula: see text] cross validations using a public dataset. Study results showed that the FBSCA method outperformed the other methods regardless of number of training trials. These results suggest that the proposed FBSCA method can enhance the RSVP classification.

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs. The present study proposed a rapid serial multimodal presentation (RSMP) BCI that incorporates artificial facial images and artificial voice stimuli. To clarify the effect of audiovisual stimuli on the RSMP BCI, scrambled images and masked sounds were applied instead of visual and auditory stimuli, respectively. The findings indicated that the audiovisual stimuli improved performance of the RSMP BCI, and that P300 at Pz contributed to classification accuracy. Online accuracy of the BCI reached 85.7 ± 11.5 %. Taken together, these findings may aid in the development of better gaze-independent BCI systems.