Semi-supervised generative and discriminative adversarial learning for motor imagery-based brain-computer interface

Survey on the research direction of EEG-based signal processing

Electroencephalography (EEG) is increasingly important in Brain-Computer Interface (BCI) systems due to its portability and simplicity. In this paper, we provide a comprehensive review of research on EEG signal processing techniques since 2021, with a focus on preprocessing, feature extraction, and classification methods. We analyzed 61 research articles retrieved from academic search engines, including CNKI, PubMed, Nature, IEEE Xplore, and Science Direct. For preprocessing, we focus on innovatively proposed preprocessing methods, channel selection, and data augmentation. Data augmentation is classified into conventional methods (sliding windows, segmentation and recombination, and noise injection) and deep learning methods [Generative Adversarial Networks (GAN) and Variation AutoEncoder (VAE)]. We also pay attention to the application of deep learning, and multi-method fusion approaches, including both conventional algorithm fusion and fusion between conventional algorithms and deep learning. Our analysis identifies 35 (57.4%), 18 (29.5%), and 37 (60.7%) studies in the directions of preprocessing, feature extraction, and classification, respectively. We find that preprocessing methods have become widely used in EEG classification (96.7% of reviewed papers) and comparative experiments have been conducted in some studies to validate preprocessing. We also discussed the adoption of channel selection and data augmentation and concluded several mentionable matters about data augmentation. Furthermore, deep learning methods have shown great promise in EEG classification, with Convolutional Neural Networks (CNNs) being the main structure of deep neural networks (92.3% of deep learning papers). We summarize and analyze several innovative neural networks, including CNNs and multi-structure fusion. However, we also identified several problems and limitations of current deep learning techniques in EEG classification, including inappropriate input, low cross-subject accuracy, unbalanced between parameters and time costs, and a lack of interpretability. Finally, we highlight the emerging trend of multi-method fusion approaches (49.2% of reviewed papers) and analyze the data and some examples. We also provide insights into some challenges of multi-method fusion. Our review lays a foundation for future studies to improve EEG classification performance.

Closed-loop motor imagery EEG simulation for brain-computer interfaces

In a brain-computer interface (BCI) system, the testing of decoding algorithms, tasks, and their parameters is critical for optimizing performance. However, conducting human experiments can be costly and time-consuming, especially when investigating broad sets of parameters. Attempts to utilize previously collected data in offline analysis lack a co-adaptive feedback loop between the system and the user present online, limiting the applicability of the conclusions obtained to real-world uses of BCI. As such, a number of studies have attempted to address this cost-wise middle ground between offline and live experimentation with real-time neural activity simulators. We present one such system which generates motor imagery electroencephalography (EEG) via forward modeling and novel motor intention encoding models for conducting sensorimotor rhythm (SMR)-based continuous cursor control experiments in a closed-loop setting. We use the proposed simulator with 10 healthy human subjects to test the effect of three decoder and task parameters across 10 different values. Our simulated approach produces similar statistical conclusions to those produced during parallel, paired, online experimentation, but in 55% of the time. Notably, both online and simulated experimentation expressed a positive effect of cursor velocity limit on performance regardless of subject average performance, supporting the idea of relaxing constraints on cursor gain in online continuous cursor control. We demonstrate the merits of our closed-loop motor imagery EEG simulation, and provide an open-source framework to the community for closed-loop SMR-based BCI studies in the future. All code including the simulator have been made available on GitHub.