Brain-Computer Interface Speller System for Alternative Communication: A Review

Improved performance of fNIRS-BCI by stacking of deep learning-derived frequency domain features

The functional near-infrared spectroscopy-based brain-computer interface (fNIRS-BCI) systems recognize patterns in brain signals and generate control commands, thereby enabling individuals with motor disabilities to regain autonomy. In this study hand gripping data is acquired using fNIRS neuroimaging system, preprocessing is performed using nirsLAB and features extraction is performed using deep learning (DL) Algorithms. For feature extraction and classification stack and fft methods are proposed. Convolutional neural networks (CNN), long short-term memory (LSTM), and bidirectional long-short-term memory (Bi-LSTM) are employed to extract features. The stack method classifies these features using a stack model and the fft method enhances features by applying fast Fourier transformation which is followed by classification using a stack model. The proposed methods are applied to fNIRS signals from twenty participants engaged in a two-class hand-gripping motor activity. The classification performance of the proposed methods is compared with conventional CNN, LSTM, and Bi-LSTM algorithms and one another. The proposed fft and stack methods yield 90.11% and 87.00% classification accuracies respectively, which are significantly higher than those achieved by CNN (85.16%), LSTM (79.46%), and Bi-LSTM (81.88%) conventional algorithms. The results show that the proposed stack and fft methods can be effectively used for the classification of the two and three-class problems in fNIRS-BCI applications.

Toward brain-computer interface speller with movement-related cortical potentials as control signals

Brain Computer Interface spellers offer a promising alternative for individuals with Amyotrophic Lateral Sclerosis (ALS) by facilitating communication without relying on muscle activity. This study assessed the feasibility of using movement related cortical potentials (MRCPs) as a control signal for a Brain-Computer Interface speller in an offline setting. Unlike motor imagery-based BCIs, this study focused on executed movements. Fifteen healthy subjects performed three spelling tasks that involved choosing specific letters displayed on a computer screen by performing a ballistic dorsiflexion of the dominant foot. Electroencephalographic signals were recorded from 10 sites centered around Cz. Three conditions were tested to evaluate MRCP performance under varying task demands: a control condition using repeated selections of the letter "O" to isolate movement-related brain activity; a phrase spelling condition with structured text ("HELLO IM FINE") to simulate a meaningful spelling task with moderate cognitive load; and a random condition using a randomized sequence of letters to introduce higher task complexity by removing linguistic or semantic context. The success rate, defined as the presence of an MRCP, was manually determined. It was approximately 69% for both the control and phrase conditions, with a slight decrease in the random condition, likely due to increased task complexity. Significant differences in MRCP features were observed between conditions with Laplacian filtering, whereas no significant differences were found in single-site Cz recordings. These results contribute to the development of MRCP-based BCI spellers by demonstrating their feasibility in a spelling task. However, further research is required to implement and validate real-time applications.

EEG-DG: A Multi-Source Domain Generalization Framework for Motor Imagery EEG Classification

Motorimagery EEG classification plays a crucial role in non-invasive Brain-Computer Interface (BCI) research. However, the performance of classification is affected by the non-stationarity and individual variations of EEG signals. Simply pooling EEG data with different statistical distributions to train a classification model can severely degrade the generalization performance. To address this issue, the existing methods primarily focus on domain adaptation, which requires access to the test data during training. This is unrealistic and impractical in many EEG application scenarios. In this paper, we propose a novel multi-source domain generalization framework called EEG-DG, which leverages multiple source domains with different statistical distributions to build generalizable models on unseen target EEG data. We optimize both the marginal and conditional distributions to ensure the stability of the joint distribution across source domains and extend it to a multi-source domain generalization framework to achieve domain-invariant feature representation, thereby alleviating calibration efforts. Systematic experiments conducted on a simulative dataset, BCI competition IV 2a, 2b, and OpenBMI datasets, demonstrate the superiority and competitive performance of our proposed framework over other state-of-the-art methods. Specifically, EEG-DG achieves average classification accuracies of 81.79% and 87.12% on datasets IV-2a and IV-2b, respectively, and 78.37% and 76.94% for inter-session and inter-subject evaluations on dataset OpenBMI, which even outperforms some domain adaptation methods.

Recent applications of EEG-based brain-computer-interface in the medical field

Brain-computer interfaces (BCIs) represent an emerging technology that facilitates direct communication between the brain and external devices. In recent years, numerous review articles have explored various aspects of BCIs, including their fundamental principles, technical advancements, and applications in specific domains. However, these reviews often focus on signal processing, hardware development, or limited applications such as motor rehabilitation or communication. This paper aims to offer a comprehensive review of recent electroencephalogram (EEG)-based BCI applications in the medical field across 8 critical areas, encompassing rehabilitation, daily communication, epilepsy, cerebral resuscitation, sleep, neurodegenerative diseases, anesthesiology, and emotion recognition. Moreover, the current challenges and future trends of BCIs were also discussed, including personal privacy and ethical concerns, network security vulnerabilities, safety issues, and biocompatibility.

Non-Invasive Brain-Computer Interfaces: State of the Art and Trends

Brain-computer interface (BCI) is a rapidly evolving technology that has the potential to widely influence research, clinical and recreational use. Non-invasive BCI approaches are particularly common as they can impact a large number of participants safely and at a relatively low cost. Where traditional non-invasive BCIs were used for simple computer cursor tasks, it is now increasingly common for these systems to control robotic devices for complex tasks that may be useful in daily life. In this review, we provide an overview of the general BCI framework as well as the various methods that can be used to record neural activity, extract signals of interest, and decode brain states. In this context, we summarize the current state-of-the-art of non-invasive BCI research, focusing on trends in both the application of BCIs for controlling external devices and algorithm development to optimize their use. We also discuss various open-source BCI toolboxes and software, and describe their impact on the field at large.

An Unsupervised Feature Extraction Method based on CLSTM-AE for Accurate P300 Classification in Brain-Computer Interface Systems

Background

The P300 signal, an endogenous component of event-related potentials, is extracted from an electroencephalography signal and employed in Brain-computer Interface (BCI) devices.

Objective

The current study aimed to address challenges in extracting useful features from P300 components and detecting P300 through a hybrid unsupervised manner based on Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM).

Material and Methods

In this cross-sectional study, CNN as a useful method for the P300 classification task emphasizes spatial characteristics of data. However, CNN and LSTM networks are combined to modify the classification system by extracting both spatial and temporal features. Then, the CNN-LSTM network was trained in an unsupervised learning method based on an autoencoder to improve Signal-to-noise Ratio (SNR) by extracting main components from latent space. To deal with imbalanced data, an Adaptive Synthetic Sampling Approach (ADASYN) is used and augmented without any duplication.

Results

The trained model, tested on the BCI competition III dataset, including two normal subjects, with an accuracy of 95% and 94% for subjects A and B in P300 detection, respectively.

Conclusion

CNN-LSTM, was embedded into an autoencoder and introduced to simultaneously extract spatial and temporal features and manage the computational complexity of the method. Further, ADASYN as an augmentation method was proposed to deal with the imbalanced nature of data, which not only maintained feature space as before but also preserved anatomical features of P300. High-quality results highlight the suitable efficiency of the proposed method.

A novel dual-step transfer framework based on domain selection and feature alignment for motor imagery decoding

In brain-computer interfaces (BCIs) based on motor imagery (MI), reducing calibration time is gradually becoming an urgent issue in practical applications. Recently, transfer learning (TL) has demonstrated its effectiveness in reducing calibration time in MI-BCI. However, the different data distribution of subjects greatly affects the application effect of TL in MI-BCI. Therefore, this paper combines data alignment, source domain selection, and feature alignment into the MI-TL. We propose a novel dual-step transfer framework based on source domain selection and feature alignment. First, the source and target domains are aligned using a pre-calibration strategy (PS), and then a sequential reverse selection method is proposed to match the optimal source domain for each target domain with the designed dual model selection strategy. We use filter bank regularization common space pattern (FBRCSP) to obtain more features and introduce manifold embedded distribution alignment (MEDA) to correct the prediction results of the support vector machine (SVM). The experimental results on two competition public datasets (BCI competition IV Dataset 1 and Dataset 2a) and our dataset show that the average classification accuracy of the proposed framework is higher than the baseline method (no domain selection and no feature alignment), which reaches 84.12%, 79.91%, and 78.45%, respectively. And the computational cost is reduced by half compared with the baseline method.

Neural Decoding of Spontaneous Overt and Intended Speech

PURPOSE

The aim of this study was to decode intended and overt speech from neuromagnetic signals while the participants performed spontaneous overt speech tasks without cues or prompts (stimuli).

METHOD

Magnetoencephalography (MEG), a noninvasive neuroimaging technique, was used to collect neural signals from seven healthy adult English speakers performing spontaneous, overt speech tasks. The participants randomly spoke the words yes or no at a self-paced rate without cues. Two machine learning models, namely, linear discriminant analysis (LDA) and one-dimensional convolutional neural network (1D CNN), were employed to classify the two words from the recorded MEG signals.

RESULTS

LDA and 1D CNN achieved average decoding accuracies of 79.02% and 90.40%, respectively, in decoding overt speech, significantly surpassing the chance level (50%). The accuracy for decoding intended speech was 67.19% using 1D CNN.

CONCLUSIONS

This study showcases the possibility of decoding spontaneous overt and intended speech directly from neural signals in the absence of perceptual interference. We believe that these findings make a steady step toward the future spontaneous speech-based brain-computer interface.

Evaluation of Different Visual Feedback Methods for Brain-Computer Interfaces (BCI) Based on Code-Modulated Visual Evoked Potentials (cVEP)

Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices using electroencephalography (EEG) signals. BCIs based on code-modulated visual evoked potentials (cVEPs) are based on visual stimuli, thus appropriate visual feedback on the interface is crucial for an effective BCI system. Many previous studies have demonstrated that implementing visual feedback can improve information transfer rate (ITR) and reduce fatigue. This research compares a dynamic interface, where target boxes change their sizes based on detection certainty, with a threshold bar interface in a three-step cVEP speller. In this study, we found that both interfaces perform well, with slight variations in accuracy, ITR, and output characters per minute (OCM). Notably, some participants showed significant performance improvements with the dynamic interface and found it less distracting compared to the threshold bars. These results suggest that while average performance metrics are similar, the dynamic interface can provide significant benefits for certain users. This study underscores the potential for personalized interface choices to enhance BCI user experience and performance. By improving user friendliness, performance, and reducing distraction, dynamic visual feedback could optimize BCI technology for a broader range of users.

A high-performance general computer cursor control scheme based on a hybrid BCI combining motor imagery and eye-tracking

This study introduces a novel virtual cursor control system designed to empower individuals with neuromuscular disabilities in the digital world. By combining eye-tracking with motor imagery (MI) in a hybrid brain-computer interface (BCI), the system enhances cursor control accuracy and simplicity. Real-time classification accuracy reaches 87.92% (peak of 93.33%), with cursor stability in the gazing state at 96.1%. Integrated into common operating systems, it enables tasks like text entry, online chatting, email, web surfing, and picture dragging, with an average text input rate of 53.2 characters per minute (CPM). This technology facilitates fundamental computing tasks for patients, fostering their integration into the online community and paving the way for future developments in BCI systems.

A method for synchronized use of EEG and eye tracking in fully immersive VR

This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

Dual-Alpha: a large EEG study for dual-frequency SSVEP brain-computer interface

BACKGROUND

The domain of brain-computer interface (BCI) technology has experienced significant expansion in recent years. However, the field continues to face a pivotal challenge due to the dearth of high-quality datasets. This lack of robust datasets serves as a bottleneck, constraining the progression of algorithmic innovations and, by extension, the maturation of the BCI field.

FINDINGS

This study details the acquisition and compilation of electroencephalogram data across 3 distinct dual-frequency steady-state visual evoked potential (SSVEP) paradigms, encompassing over 100 participants. Each experimental condition featured 40 individual targets with 5 repetitions per target, culminating in a comprehensive dataset consisting of 21,000 trials of dual-frequency SSVEP recordings. We performed an exhaustive validation of the dataset through signal-to-noise ratio analyses and task-related component analysis, thereby substantiating its reliability and effectiveness for classification tasks.

CONCLUSIONS

The extensive dataset presented is set to be a catalyst for the accelerated development of BCI technologies. Its significance extends beyond the BCI sphere and holds considerable promise for propelling research in psychology and neuroscience. The dataset is particularly invaluable for discerning the complex dynamics of binocular visual resource distribution.

Fatigue factors and fatigue indices in SSVEP-based brain-computer interfaces: a systematic review and meta-analysis

Background

Fatigue is a serious challenge when applying a steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) in the real world. Many researchers have used quantitative indices to study the effect of visual stimuli on fatigue. According to a wide range of studies in fatigue analysis, there are contradictions and inconsistencies in the behavior of fatigue indicators.

New method

In this study, for the first time, a systematic review and meta-analysis were performed on fatigue indices and fatigue caused by stimulation paradigm. We queried three scientific search engines for studies published between 2000 and 2022. The inclusion criteria were papers investigating mental and visual fatigue from performing a visual task using electroencephalogram (EEG) signals.

Results

Attractiveness and variation are the most effective ways to reduce BCI fatigue. Therefore, zoom motion, Newton's ring motion, and cue patterns reduce fatigue. While the color of the cue could effectively reduce fatigue, its shape and background had no effect on fatigue. Additionally, the questionnaire and quantitative indicators such as frequency indices, signal-to-noise ratio (SNR), SSVEP amplitude, and multiscale entropy were utilized to assess fatigue. Meta-analysis indicated that when a person is fatigued, the spectrum amplitude of alpha, theta, and α + θ / β increase significantly, while SNR and SSVEP amplitude decrease significantly.

Conclusion

The outcomes of this study can be used to design more optimal stimulation protocols that cause less fatigue. Moreover, the level of fatigue can be quantitatively assessed with indicators without the participant's self-reports.

Motor imagery EEG signal classification with a multivariate time series approach

BACKGROUND

Electroencephalogram (EEG) signals record electrical activity on the scalp. Measured signals, especially EEG motor imagery signals, are often inconsistent or distorted, which compromises their classification accuracy. Achieving a reliable classification of motor imagery EEG signals opens the door to possibilities such as the assessment of consciousness, brain computer interfaces or diagnostic tools. We seek a method that works with a reduced number of variables, in order to avoid overfitting and to improve interpretability. This work aims to enhance EEG signal classification accuracy by using methods based on time series analysis. Previous work on this line, usually took a univariate approach, thus losing the possibility to take advantage of the correlation information existing within the time series provided by the different electrodes. To overcome this problem, we propose a multivariate approach that can fully capture the relationships among the different time series included in the EEG data. To perform the multivariate time series analysis, we use a multi-resolution analysis approach based on the discrete wavelet transform, together with a stepwise discriminant that selects the most discriminant variables provided by the discrete wavelet transform analysis RESULTS: Applying this methodology to EEG data to differentiate between the motor imagery tasks of moving either hands or feet has yielded very good classification results, achieving in some cases up to 100% of accuracy for this 2-class pre-processed dataset. Besides, the fact that these results were achieved using a reduced number of variables (55 out of 22,176) can shed light on the relevance and impact of those variables.

CONCLUSIONS

This work has a potentially large impact, as it enables classification of EEG data based on multivariate time series analysis in an interpretable way with high accuracy. The method allows a model with a reduced number of features, facilitating its interpretability and improving overfitting. Future work will extend the application of this classification method to help in diagnosis procedures for detecting brain pathologies and for its use in brain computer interfaces. In addition, the results presented here suggest that this method could be applied to other fields for the successful analysis of multivariate temporal data.

A hybrid P300-SSVEP brain-computer interface speller with a frequency enhanced row and column paradigm

Objective

This study proposes a new hybrid brain-computer interface (BCI) system to improve spelling accuracy and speed by stimulating P300 and steady-state visually evoked potential (SSVEP) in electroencephalography (EEG) signals.

Methods

A frequency enhanced row and column (FERC) paradigm is proposed to incorporate the frequency coding into the row and column (RC) paradigm so that the P300 and SSVEP signals can be evoked simultaneously. A flicker (white-black) with a specific frequency from 6.0 to 11.5 Hz with an interval of 0.5 Hz is assigned to one row or column of a 6 × 6 layout, and the row/column flashes are carried out in a pseudorandom sequence. A wavelet and support vector machine (SVM) combination is adopted for P300 detection, an ensemble task-related component analysis (TRCA) method is used for SSVEP detection, and the two detection possibilities are fused using a weight control approach.

Results

The implemented BCI speller achieved an accuracy of 94.29% and an information transfer rate (ITR) of 28.64 bit/min averaged across 10 subjects during the online tests. An accuracy of 96.86% is obtained during the offline calibration tests, higher than that of only using P300 (75.29%) or SSVEP (89.13%). The SVM in P300 outperformed the previous linear discrimination classifier and its variants (61.90–72.22%), and the ensemble TRCA in SSVEP outperformed the canonical correlation analysis method (73.33%).

Conclusion

The proposed hybrid FERC stimulus paradigm can improve the performance of the speller compared with the classical single stimulus paradigm. The implemented speller can achieve comparable accuracy and ITR to its state-of-the-art counterparts with advanced detection algorithms.

A flexible speller based on time-space frequency conversion SSVEP stimulation paradigm under dry electrode

Introduction

Speller is the best way to express the performance of the brain-computer interface (BCI) paradigm. Due to its advantages of short analysis time and high accuracy, the SSVEP paradigm has been widely used in the BCI speller system based on the wet electrode. It is widely known that the wet electrode operation is cumbersome and that the subjects have a poor experience. In addition, in the asynchronous SSVEP system based on threshold analysis, the system flickers continuously from the beginning to the end of the experiment, which leads to visual fatigue. The dry electrode has a simple operation and provides a comfortable experience for subjects. The EOG signal can avoid the stimulation of SSVEP for a long time, thus reducing fatigue.

Methods

This study first designed the brain-controlled switch based on continuous blinking EOG signal and SSVEP signal to improve the flexibility of the BCI speller. Second, in order to increase the number of speller instructions, we designed the time-space frequency conversion (TSFC) SSVEP stimulus paradigm by constantly changing the time and space frequency of SSVEP sub-stimulus blocks, and designed a speller in a dry electrode environment.

Results

Seven subjects participated and completed the experiments. The results showed that the accuracy of the brain-controlled switch designed in this study was up to 94.64%, and all the subjects could use the speller flexibly. The designed 60-character speller based on the TSFC-SSVEP stimulus paradigm has an accuracy rate of 90.18% and an information transmission rate (ITR) of 117.05 bits/min. All subjects can output the specified characters in a short time.

Discussion

This study designed and implemented a multi-instruction SSVEP speller based on dry electrode. Through the combination of EOG and SSVEP signals, the speller can be flexibly controlled. The frequency of SSVEP stimulation sub-block is recoded in time and space by TSFC-SSVEP stimulation paradigm, which greatly improves the number of output instructions of BCI system in dry electrode environment. This work only uses FBCCA algorithm to test the stimulus paradigm, which requires a long stimulus time. In the future, we will use trained algorithms to study stimulus paradigm to improve its overall performance.

Identifying Thematics in a Brain-Computer Interface Research

This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.