An efficient frequency recognition method based on likelihood ratio test for SSVEP-based BCI

A Time-Local Weighted Transformation Recognition Framework for Steady State Visual Evoked Potentials Based Brain-Computer Interfaces

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

A novel untrained SSVEP-EEG feature enhancement method using canonical correlation analysis and underdamped second-order stochastic resonance

Objective

Compared with the light-flashing paradigm, the ring-shaped motion checkerboard patterns avoid uncomfortable flicker or brightness modulation, improving the practical interactivity of brain-computer interface (BCI) applications. However, due to fewer harmonic responses and more concentrated frequency energy elicited by the ring-shaped checkerboard patterns, the mainstream untrained algorithms such as canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA) methods have poor recognition performance and low information transmission rate (ITR).

Methods

To address this issue, a novel untrained SSVEP-EEG feature enhancement method using CCA and underdamped second-order stochastic resonance (USSR) is proposed to extract electroencephalogram (EEG) features.

Results

In contrast to typical unsupervised dimensionality reduction methods such as common average reference (CAR), principal component analysis (PCA), multidimensional scaling (MDS), and locally linear embedding (LLE), CCA exhibits higher adaptability for SSVEP rhythm components.

Conclusion

This study recruits 42 subjects to evaluate the proposed method and experimental results show that the untrained method can achieve higher detection accuracy and robustness.

Significance

This untrained method provides the possibility of applying a nonlinear model from one-dimensional signals to multi-dimensional signals.

An improved cross-subject spatial filter transfer method for SSVEP-based BCI

Steady-state visual evoked potential (SSVEP) training feature recognition algorithms utilize user training data to reduce the interference of spontaneous electroencephalogram (EEG) activities on SSVEP response for improved recognition accuracy. The data collection process can be tedious, increasing the mental fatigue of users and also seriously affecting the practicality of SSVEP-based brain-computer interface (BCI) systems. As an alternative, a cross-subject spatial filter transfer (CSSFT) method to transfer an existing user data model with good SSVEP response to new user test data has been proposed. The CSSFT method uses superposition averages of data for multiple blocks of data as transfer data. However, the amplitude and pattern of brain signals are often significantly different across trials. The goal of this study was to improve superposition averaging for the CSSFT method and propose an Ensemble scheme based on ensemble learning, and an Expansion scheme based on matrix expansion. The feature recognition performance was compared for CSSFT and the proposed improved CSSFT method using two public datasets. The results demonstrated that the improved CSSFT method can significantly improve the recognition accuracy and information transmission rate of existing methods. This strategy avoids a tedious data collection process, and promotes the potential practical application of BCI systems.

An MVMD-CCA Recognition Algorithm in SSVEP-Based BCI and Its Application in Robot Control

This article proposes a novel recognition algorithm for the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) system. By combining the advantages of multivariate variational mode decomposition (MVMD) and canonical correlation analysis (CCA), an MVMD-CCA algorithm is investigated to improve the detection ability of SSVEP electroencephalogram (EEG) signals. In comparison with the classical filter bank canonical correlation analysis (FBCCA), the nonlinear and non-stationary EEG signals are decomposed into a fixed number of sub-bands by MVMD, which can enhance the effect of SSVEP-related sub-bands. The experimental results show that MVMD-CCA can effectively reduce the influence of noise and EEG artifacts and improve the performance of SSVEP-based BCI. The offline experiments show that the average accuracies of MVMD-CCA in the training dataset and testing dataset are improved by 3.08% and 1.67%, respectively. In the SSVEP-based online robotic manipulator grasping experiment, the recognition accuracies of the four subjects are 92.5%, 93.33%, 90.83%, and 91.67%, respectively.

SSVEP-EEG Feature Enhancement Method Using an Image Sharpening Filter

Steady-state visual evoked potential (SSVEP) is widely used in brain computer interface (BCI), medical detection, and neuroscience, so there is significant interest in enhancing SSVEP features via signal processing for better performance. In this study, an image processing method was combined with brain signal analysis and a sharpening filter was used to extract image details and features for the enhancement of SSVEP features. The results demonstrated that sharpening filter could eliminate the SSVEP signal trend term and suppress its low-frequency component. Meanwhile, sharpening filter effectively enhanced the signal-to-noise ratios (SNRs) of the single-channel and multi-channel fused signals. Image sharpening filter also significantly improved the recognition accuracy of canonical correlation analysis (CCA), filter bank canonical correlation analysis (FBCCA), and task-related component analysis (TRCA). The tools developed here effectively enhanced the SSVEP signal features, suggesting that image processing methods can be considered for improved brain signal analysis.

Multivariate synchronization curve: A measure of synchronization in different multivariate signals

As a method to measure the synchronization between two different sets of signals, the multivariate synchronization index (MSI) has played an irreplaceable role in the field of frequency recognition of brain-computer interface since it was proposed. On this basis, we make a generalization of MSI by using the escort distribution to replace the original distribution. In this way, MSI can be converted from a determined value to the multivariate synchronization curve, which will vary as the parameter q of the escort distribution changes. Numerical experiments are carried out on both simulated and real-world data to confirm the effectiveness of this new method. Compared with the case of MSI (i.e., q = 1), the extended form of MSI proposed in this article can obviously capture the relationship between signals more comprehensively, implying that it is a more perfect method to describe the synchronization between them. The results reveal that this method can not only effectively extract the important information contained in different signals, but also has the potential to become a practical synchronization measurement method of multivariate signals.

Multi-scale noise transfer and feature frequency detection in SSVEP based on FitzHugh-Nagumo neuron system

Objective. The steady-state visual evoked potential (SSVEP) is one of the most commonly used control signals for brain-computer interfaces (BCIs) due to its excellent interactive potential, such as high tolerance to noises and robust performance across users. In addition, it has a stable cycle, obvious characteristics and minimal training requirements. However, the SSVEP is extremely weak and companied with strong and multi-scale noise, resulting in a poor signal-to-noise ratio in practice. Common algorithms for classification are based on the principle of template matching and spatial filtering, which cannot obtain satisfied performance of SSVEP detection under the multi-scale noise. Therefore, using linear methods to extract SSVEP with obvious nonlinear and non-stationary characteristics, the useful signal will be attenuated or lost.Approach.To address this issue, two novel frameworks based on a two-dimensional nonlinear FitzHugh-Nagumo (FHN) neuron system are proposed to extract feature frequency of SSVEP.Results.In order to evaluate the effectiveness of the proposed methods, this research recruit 22 subjects to participate the experiment. Experimental results show that nonlinear FHN neuron model can force the energy of noise to be transferred into SSVEP and hence amplifying the amplitude of the target frequency. Compared with the traditional methods, the FHN and FHNCCA methods can achieve higher classification accuracy and faster processing speed, which effectively improves the information transmission rate of SSVEP-based BCI.

Information Bottleneck as Optimisation Method for SSVEP-Based BCI

In this study, the information bottleneck method is proposed as an optimisation method for steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI). The information bottleneck is an information-theoretic optimisation method for solving problems with a trade-off between preserving meaningful information and compression. Its main practical application in machine learning is in representation learning or feature extraction. In this study, we use the information bottleneck to find optimal classification rule for a BCI. This is a novel application for the information bottleneck. This approach is particularly suitable for BCIs since the information bottleneck optimises the amount of information transferred by the BCI. Steady-state visual evoked potential-based BCIs often classify targets using very simple rules like choosing the class corresponding to the largest feature value. We call this classifier the arg max classifier. It is unlikely that this approach is optimal, and in this study, we propose a classification method specifically designed to optimise the performance measure of BCIs. This approach gives an advantage over standard machine learning methods, which aim to optimise different measures. The performance of the proposed algorithm is tested on two publicly available datasets in offline experiments. We use the standard power spectral density analysis (PSDA) and canonical correlation analysis (CCA) feature extraction methods on one dataset and show that the current approach outperforms most of the related studies on this dataset. On the second dataset, we use the task-related component analysis (TRCA) method and demonstrate that the proposed method outperforms the standard argmax classification rule in terms of information transfer rate when using a small number of classes. To our knowledge, this is the first time the information bottleneck is used in the context of SSVEP-based BCIs. The approach is unique in the sense that optimisation is done over the space of classification functions. It potentially improves the performance of BCIs and makes it easier to calibrate the system for different subjects.

SSVEP-EEG Denoising via Image Filtering Methods

Steady-state visual evoked potential (SSVEP) is widely used in electroencephalogram (EEG) control, medical detection, cognitive neuroscience, and other fields. However, successful application requires improving the detection performance of SSVEP signal frequency characteristics. Most strategies to enhance the signal-to-noise ratio of SSVEP utilize application of a spatial filter. Here, we propose a method for image filtering denoising (IFD) of the SSVEP signal from the perspective of image analysis, as a preprocessing step for signal analysis. Arithmetic mean, geometric mean, Gaussian, and non-local means filtering methods were tested, and the experimental results show that image filtering of SSVEP cannot effectively remove the noise. Thus, we proposed a reverse solution in which the SSVEP noise signal was obtained by image filtering, and then the noise was subtracted from the original signal. Comparison of the recognition accuracy of the SSVEP signal before and after denoising was used to evaluate the denoising performance for stimuli of different duration. After IFD processing, SSVEP exhibited higher recognition accuracy, indicating the effectiveness of this proposed denoising method. Application of this method improves the detection performance of SSVEP signal frequency characteristics, combines image processing and brain signal analysis, and expands the research scope of brain signal analysis for widespread application.

Brain-Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance

The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain-computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.

Enhancing detection of steady-state visual evoked potentials using channel ensemble method

OBJECTIVE

This study proposed and evaluated a channel ensemble approach to enhance detection of steady-state visual evoked potentials (SSVEPs).

APPROACH

Collected multi-channel electroencephalogram (EEG) signals were classified into multiple groups of new analysis signals based on correlation analysis, and each group of analysis signals contained signals from a different number of electrode channels. These groups of analysis signals were used as the input of a training-free feature extraction model, and the obtained feature coefficients were converted into feature probability values using the softmax function. The ensemble value of multiple sets of feature probability values was determined and used as the final discrimination coefficient.

MAIN RESULTS

Compared with canonical correlation analysis (CCA), likelihood ratio test (LRT), and multivariate synchronization index (MSI) analysis methods using a standard approach, the recognition accuracies of the methods using a channel ensemble approach were improved by 5.05%, 3.87%, and 3.42%, and the information transfer rates (ITRs) were improved by 6.00%, 4.61%, and 3.71%, respectively. The channel ensemble method also obtained better recognition results than the standard algorithm on the public dataset. This study validated the efficiency of the proposed method to enhance the detection of SSVEPs, demonstrating its potential use in practical brain-computer interface (BCI) systems.

SIGNIFICANCE

A SSVEP-based BCI system using a channel ensemble method could achieve high ITR, indicating great potential of this design for various applications with improved control and interaction.

Filter bank temporally local canonical correlation analysis for short time window SSVEPs classification

Canonical correlation analysis (CCA) method and its extended methods have been widely and successfully applied to the frequency recognition in SSVEP-based BCI systems. As a state-of-the-art extended method, filter bank canonical correlation analysis has higher accuracy and information transmission rate (ITR) than CCA. However, in the CCA method, the temporally local structure of samples has not been well considered. In this correspondence, we proposed termed temporally local canonical correlation analysis (TCCA). In this new method, the original covariance matrix was replaced by the temporally local covariance matrix. Furthermore, we proposed an improved frequency identification method of filter bank based on TCCA, named filter bank temporally local canonical correlation analysis (FBTCCA). In the offline environment, we used a leave-one-subject-out validation strategy on datasets of ten testees to optimize the parameters of TCCA and FBTCCA and evaluate the two algorithms. The experimental results affirm that TCCA markedly outperformed CCA, and FBTCCA obtained the highest accuracy among the four methods. This study corroborates that TCCA-based approaches have great potential for implementing short time window SSVEP-based BCI systems.

A new multivariate empirical mode decomposition method for improving the performance of SSVEP-based brain-computer interface

OBJECTIVE

Accurate and efficient detection of steady-state visual evoked potentials (SSVEP) in electroencephalogram (EEG) is essential for the related brain-computer interface (BCI) applications.

APPROACH

Although the canonical correlation analysis (CCA) has been applied extensively and successfully to SSVEP recognition, the spontaneous EEG activities and artifacts that often occur during data recording can deteriorate the recognition performance. Therefore, it is meaningful to extract a few frequency sub-bands of interest to avoid or reduce the influence of unrelated brain activity and artifacts. This paper presents an improved method to detect the frequency component associated with SSVEP using multivariate empirical mode decomposition (MEMD) and CCA (MEMD-CCA). EEG signals from nine healthy volunteers were recorded to evaluate the performance of the proposed method for SSVEP recognition.

MAIN RESULTS

We compared our method with CCA and temporally local multivariate synchronization index (TMSI). The results suggest that the MEMD-CCA achieved significantly higher accuracy in contrast to standard CCA and TMSI. It gave the improvements of 1.34%, 3.11%, 3.33%, 10.45%, 15.78%, 18.45%, 15.00% and 14.22% on average over CCA at time windows from 0.5 s to 5 s and 0.55%, 1.56%, 7.78%, 14.67%, 13.67%, 7.33% and 7.78% over TMSI from 0.75 s to 5 s. The method outperformed the filter-based decomposition (FB), empirical mode decomposition (EMD) and wavelet decomposition (WT) based CCA for SSVEP recognition.

SIGNIFICANCE

The results demonstrate the ability of our proposed MEMD-CCA to improve the performance of SSVEP-based BCI.

To train or not to train? A survey on training of feature extraction methods for SSVEP-based BCIs

OBJECTIVE

Despite the vast research aimed at improving the performance of steady-state visually evoked potential (SSVEP)-based brain-computer interfaces (BCIs), several limitations exist that restrict the use of such applications for long-term users in the real-world. One of the main challenges has been to reduce training time while maintaining good BCI performance. In view of this challenge, this survey identifies and compares the different training requirements of feature extraction methods for SSVEP-based BCIs.

APPROACH

This paper reviews the various state-of-the-art SSVEP feature extraction methods that have been developed and are most widely used in the literature.

MAIN RESULTS

The main contributions compared to existing reviews are the following: (i) a detailed summary, including a brief mathematical description of each feature extraction algorithm, providing a guide to the basic concepts of the state-of-the-art techniques for SSVEP-based BCIs found in literature; (ii) a categorisation of the training requirements of SSVEP-based methods into three categories, defined as training-free methods, subject-specific and subject-independent training methods; (iii) a comparative review of the training requirements of SSVEP feature extraction methods, providing a reference for future work on SSVEP-based BCIs.

SIGNIFICANCE

This review highlights the strengths and weaknesses of the three categories of SSVEP training methods. Training-free systems are more practical but their performance is limited due to inter-subject variability resulting from the complex EEG activity. Feature extraction methods that incorporate some training data address this issue and in fact have outperformed training-free methods: subject-specific BCIs are tuned to the individual yielding the best performance at the cost of long, tiring training sessions making these methods unsuitable for everyday use; subject-independent BCIs that make use of training data from various subjects offer a good trade-off between training effort and performance, making these BCIs better suited for practical use.

A Novel Multilayer Correlation Maximization Model for Improving CCA-Based Frequency Recognition in SSVEP Brain-Computer Interface

Multiset canonical correlation analysis (MsetCCA) has been successfully applied to optimize the reference signals by extracting common features from multiple sets of electroencephalogram (EEG) for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface application. To avoid extracting the possible noise components as common features, this study proposes a sophisticated extension of MsetCCA, called multilayer correlation maximization (MCM) model for further improving SSVEP recognition accuracy. MCM combines advantages of both CCA and MsetCCA by carrying out three layers of correlation maximization processes. The first layer is to extract the stimulus frequency-related information in using CCA between EEG samples and sine-cosine reference signals. The second layer is to learn reference signals by extracting the common features with MsetCCA. The third layer is to re-optimize the reference signals set in using CCA with sine-cosine reference signals again. Experimental study is implemented to validate effectiveness of the proposed MCM model in comparison with the standard CCA and MsetCCA algorithms. Superior performance of MCM demonstrates its promising potential for the development of an improved SSVEP-based brain-computer interface.

Robust frequency recognition for SSVEP-based BCI with temporally local multivariate synchronization index

Multivariate synchronization index (MSI) has been proved to be an efficient method for frequency recognition in SSVEP-BCI systems. It measures the correlation according to the entropy of the normalized eigenvalues of the covariance matrix of multichannel signals. In the MSI method, the estimation of covariance matrix omits the temporally local structure of samples. In this study, a new spatio-temporal method, termed temporally local MSI (TMSI), was presented. This new method explicitly exploits temporally local information in modelling the covariance matrix. In order to evaluate the performance of the TMSI, we performs a comparison between the two methods on the real SSVEP datasets from eleven subjects. The results show that the TMSI outperforms the standard MSI. TMSI benefits from exploiting the temporally local structure of EEG signals, and could be a potential method for robust performance of SSVEP-based BCI.

A Double-Partial Least-Squares Model for the Detection of Steady-State Visual Evoked Potentials

Establishing a high-accuracy and training-free brain-computer interface (BCI) system is essential for improving BCI practicality. In this study, we propose for the first time a training-free double-partial least-squares (D-PLS) model for steady-state visual evoked potential (SSVEP) detection that consists of double-layer PLS, a PLS spatial filter, and a PLS feature extractor. Electroencephalographic data from 11 healthy volunteers under four different visual stimulation frequencies were used to test the proposed method. Compared with commonly used spatial filters, minimum energy combination and average maximum contrast combination, the classification accuracies could be improved 2-10% by our proposed PLS spatial filter. Furthermore, our proposed PLS feature extractor achieved better performance than current feature extraction methods, namely power spectral density analysis, canonical correlation analysis, and the use of the least absolute shrinkage and selection operator. The average classification accuracy for our proposed D-PLS model exceeded [Formula: see text] when the signal time window was longer than 3.5 s and reached as high as [Formula: see text] when the time window was 5 s. Moreover, the D-PLS model can be easily set without training data, so it can be used widely in SSVEP-based BCI systems.

Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-Based BCI

Many of the most widely accepted methods for reliable detection of steady-state visual evoked potentials (SSVEPs) in the electroencephalogram (EEG) utilize canonical correlation analysis (CCA). CCA uses pure sine and cosine reference templates with frequencies corresponding to the visual stimulation frequencies. These generic reference templates may not optimally reflect the natural SSVEP features obscured by the background EEG. This paper introduces a new approach that utilizes spatio-temporal feature extraction with multivariate linear regression (MLR) to learn discriminative SSVEP features for improving the detection accuracy. MLR is implemented on dimensionality-reduced EEG training data and a constructed label matrix to find optimally discriminative subspaces. Experimental results show that the proposed MLR method significantly outperforms CCA as well as several other competing methods for SSVEP detection, especially for time windows shorter than 1 second. This demonstrates that the MLR method is a promising new approach for achieving improved real-time performance of SSVEP-BCIs.

Evaluate the Feasibility of Using Frontal SSVEP to Implement an SSVEP-Based BCI in Young, Elderly and ALS Groups

This paper studies the amplitude-frequency characteristic of frontal steady-state visual evoked potential (SSVEP) and its feasibility as a control signal for brain computer interface (BCI). SSVEPs induced by different stimulation frequencies, from 13 ~ 31 Hz in 2 Hz steps, were measured in eight young subjects, eight elders and seven ALS patients. Each subject was requested to participate in a calibration study and an application study. The calibration study was designed to find the amplitude-frequency characteristics of SSVEPs recorded from Oz and Fpz positions, while the application study was designed to test the feasibility of using frontal SSVEP to control a two-command SSVEP-based BCI. The SSVEP amplitude was detected by an epoch-average process which enables artifact-contaminated epochs can be removed. The seven ALS patients were severely impaired, and four patients, who were incapable of completing our BCI task, were excluded from calculation of BCI performance. The averaged accuracies, command transfer intervals and information transfer rates in operating frontal SSVEP-based BCI were 96.1%, 3.43 s/command, and 14.42 bits/min in young subjects; 91.8%, 6.22 s/command, and 6.16 bits/min in elders; 81.2%, 12.14 s/command, and 1.51 bits/min in ALS patients, respectively. The frontal SSVEP could be an alternative choice to design SSVEP-based BCI.

A Comparison Study of Canonical Correlation Analysis Based Methods for Detecting Steady-State Visual Evoked Potentials

Canonical correlation analysis (CCA) has been widely used in the detection of the steady-state visual evoked potentials (SSVEPs) in brain-computer interfaces (BCIs). The standard CCA method, which uses sinusoidal signals as reference signals, was first proposed for SSVEP detection without calibration. However, the detection performance can be deteriorated by the interference from the spontaneous EEG activities. Recently, various extended methods have been developed to incorporate individual EEG calibration data in CCA to improve the detection performance. Although advantages of the extended CCA methods have been demonstrated in separate studies, a comprehensive comparison between these methods is still missing. This study performed a comparison of the existing CCA-based SSVEP detection methods using a 12-class SSVEP dataset recorded from 10 subjects in a simulated online BCI experiment. Classification accuracy and information transfer rate (ITR) were used for performance evaluation. The results suggest that individual calibration data can significantly improve the detection performance. Furthermore, the results showed that the combination method based on the standard CCA and the individual template based CCA (IT-CCA) achieved the highest performance.