Improving zero-training brain-computer interfaces by mixing model estimators

Assembling global and local spatial-temporal filters to extract discriminant information of EEG in RSVP task

Objective.Brain-computer interface (BCI) system has developed rapidly in the past decade. And rapid serial visual presentation (RSVP) is an important BCI paradigm to detect the targets in high-speed image streams. For decoding electroencephalography (EEG) in RSVP task, the ensemble-model methods have better performance than the single-model ones.Approach.This study proposed a method based on ensemble learning to extract discriminant information of EEG. An extreme gradient boosting framework was utilized to sequentially generate the sub models, including one global spatial-temporal filter and a group of local ones. EEG was reshaped into a three-dimensional form by remapping the electrode dimension into a 2D array to learn the spatial-temporal features from real local space.Main results.A benchmark RSVP EEG dataset was utilized to evaluate the performance of the proposed method, where EEG data of 63 subjects were analyzed. Compared with several state-of-the-art methods, the spatial-temporal patterns of proposed method were more consistent with P300, and the proposed method can provide significantly better classification performance.Significance.The ensemble model in this study was end-to-end optimized, which can avoid error accumulation. The sub models optimized by gradient boosting theory can extract discriminant information complementarily and non-redundantly.

An Open Source-Based BCI Application for Virtual World Tour and Its Usability Evaluation

Brain-computer interfaces can provide a new communication channel and control functions to people with restricted movements. Recent studies have indicated the effectiveness of brain-computer interface (BCI) applications. Various types of applications have been introduced so far in this field, but the number of those available to the public is still insufficient. Thus, there is a need to expand the usability and accessibility of BCI applications. In this study, we introduce a BCI application for users to experience a virtual world tour. This software was built on three open-source environments and is publicly available through the GitHub repository. For a usability test, 10 healthy subjects participated in an electroencephalography (EEG) experiment and evaluated the system through a questionnaire. As a result, all the participants successfully played the BCI application with 96.6% accuracy with 20 blinks from two sessions and gave opinions on its usability (e.g., controllability, completeness, comfort, and enjoyment) through the questionnaire. We believe that this open-source BCI world tour system can be used in both research and entertainment settings and hopefully contribute to open science in the BCI field.

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration.

Ensemble or pool: A comprehensive study on transfer learning for c-VEP BCI during interpersonal interaction

BACKGROUND

To reduce calibration time of brain-computer interface (BCI) or even implement zero-training BCI, researchers have been studying how to effectively apply transfer learning in the field. In order to thoroughly investigate the performance of transfer learning in BCI and the key factors affecting transfer performance in the field, we carried out a comprehensive study.

NEW METHOD

In general, transferring knowledge in BCI is implemented in two ways: ensemble or pool. In this work, we propose two different transfer approaches. One is to transfer the information of all channels as a whole from the source subjects to a target subject. The second approach is to transfer the information of corresponding channels between the subjects. A subject transfer framework is built by combining the two approaches with ensemble or pool.

RESULTS

We investigated the performances of eight implementations of this framework on a data set acquired by an interpersonal interaction (Chicken Game) experiment based on code-modulated visual evoked potential (c-VEP) BCI. The results show that transfer learning generally provides acceptable classification performance. Additionally, an in-depth analysis reveals that a target subject usually shares different brain signal distribution with different source subjects. In fact, this is a hypothesis usually implied by this kind of research.

CONCLUSIONS

Transfer learning for c-VEP BCI can be qualified for reducing calibration time or starting the recognition of BCI without sufficient subjects' own data. In addition, our finding suggests a solid validity of the hypothesis underlying transferring knowledge in BCI.

Hearing the needs of clinical users

In the past 10 years, brain-computer interfaces (BCIs) for controlling assistive devices have seen tremendous progress with respect to reliability and learnability, and numerous exemplary applications were demonstrated to be controllable by a BCI. Yet, BCI-controlled applications are hardly used for patients with neurologic or neurodegenerative disease. Such patient groups are considered potential end-users of BCI, specifically for replacing or improving lost function. We argue that BCI research and development still faces a translational gap, i.e., the knowledge of how to bring BCIs from the laboratory to the field is insufficient. BCI-controlled applications lack usability and accessibility; both constitute two sides of one coin, which is the key to use in daily life and to prevent nonuse. To increase usability, we suggest rigorously adopting the user-centered design in applied BCI research and development. To provide accessibility, assistive technology (AT) experts, providers, and other stakeholders have to be included in the user-centered process. BCI experts have to ensure the transfer of knowledge to AT professionals, and listen to the needs of primary, secondary, and tertiary end-users of BCI technology. Addressing both, usability and accessibility, in applied BCI research and development will bridge the translational gap and ensure that the needs of clinical end-users are heard, understood, addressed, and fulfilled.

The Potential for a Speech Brain-Computer Interface Using Chronic Electrocorticography

A brain-computer interface (BCI) is a technology that uses neural features to restore or augment the capabilities of its user. A BCI for speech would enable communication in real time via neural correlates of attempted or imagined speech. Such a technology would potentially restore communication and improve quality of life for locked-in patients and other patients with severe communication disorders. There have been many recent developments in neural decoders, neural feature extraction, and brain recording modalities facilitating BCI for the control of prosthetics and in automatic speech recognition (ASR). Indeed, ASR and related fields have developed significantly over the past years, and many lend many insights into the requirements, goals, and strategies for speech BCI. Neural speech decoding is a comparatively new field but has shown much promise with recent studies demonstrating semantic, auditory, and articulatory decoding using electrocorticography (ECoG) and other neural recording modalities. Because the neural representations for speech and language are widely distributed over cortical regions spanning the frontal, parietal, and temporal lobes, the mesoscopic scale of population activity captured by ECoG surface electrode arrays may have distinct advantages for speech BCI, in contrast to the advantages of microelectrode arrays for upper-limb BCI. Nevertheless, there remain many challenges for the translation of speech BCIs to clinical populations. This review discusses and outlines the current state-of-the-art for speech BCI and explores what a speech BCI using chronic ECoG might entail.