Flashing characters with famous faces improves ERP-based brain–computer interface performance

The use of happy faces as visual stimuli improves the performance of the hybrid SSVEP+P300 brain computer interface

BACKGROUND

This study illustrates a hybrid brain-computer interface (BCI) in which steady-state visual evoked potentials (SSVEP) and event-related potentials (P300) are evoked simultaneously. The goal of this study was to improve the performance of the current hybrid SSVEP+P300 BCI systems by incorporating a happy face into visual stimuli.

NEW METHOD

In this study, happy and sad faces were added to a visual stimulus to induce stronger cortical signals in a hybrid SSVEP+P300 BCI. Additionally, we developed a paradigm in which SSVEP responses were triggered by non-face stimuli, whereas P300 responses were triggered by face stimuli. We tested four paradigms: happy face paradigm (HF), sad face paradigm (SF), happy face and flicker paradigm (HFF), and sad face and flicker paradigm (SFF).

RESULTS AND CONCLUSIONS

The results demonstrated that the HFF paradigm elicited more robust cortical responses, which resulted in enhanced system accuracy and information transfer rate (ITR). The HFF paradigm has a system communication rate of 25.9 bits per second and an average accuracy of 96.1%. Compared with other paradigms, the HFF paradigm is the best choice for BCI applications because it has the highest ITR and maximum level of comfort.

Signal alignment for cross-datasets in P300 brain-computer interfaces

Objective.Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment (SA) for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning.Approach.We proposed a linear SA that uses the P300's latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm).Results.Although the standard approach without SA had an average precision (AP) score of 25.5%, the approach demonstrated a 35.8% AP score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets.Significance.We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.

Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.

Promoting Simple and Engaging Brain-Computer Interface Designs for Children by Evaluating Contrasting Motion Techniques

PURPOSE

There is an increasing focus on using motion in augmentative and alternative communication (AAC) systems. In considering brain-computer interface access to AAC (BCI-AAC), motion may provide a simpler or more intuitive avenue for BCI-AAC control. Different motion techniques may be utilized in supporting competency with AAC devices including simple (e.g., zoom) and complex (behaviorally relevant animation) methods. However, how different pictorial symbol animation techniques impact BCI-AAC is unclear.

METHOD

Sixteen healthy children completed two experimental conditions. These conditions included highlighting of pictorial symbols via both functional (complex) and zoom (simple) animation to evaluate the effects of motion techniques on P300-based BCI-AAC signals and offline (predicted) BCI-AAC performance.

RESULTS

Functional (complex) animation significantly increased attentional-related P200/P300 event-related potential (ERP) amplitudes in the parieto-occipital area. Zoom (simple) animation significantly decreased N400 latency. N400 ERP amplitude was significantly greater, and occurred significantly earlier, on the right versus left side for the functional animation condition within the parieto-occipital bin. N200 ERP latency was significantly reduced over the left hemisphere for the zoom condition in the central bin. As hypothesized, elicitation of all targeted ERP components supported offline (predicted) BCI-AAC performance being similar between conditions.

CONCLUSION

Study findings provide continued support for the use of animation in BCI-AAC systems for children and highlight differences in neural and attentional processing between complex and simple animation techniques.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24085623.

Evaluation of Single-Trial Classification to Control a Visual ERP-BCI under a Situation Awareness Scenario

An event-related potential (ERP)-based brain-computer interface (BCI) can be used to monitor a user's cognitive state during a surveillance task in a situational awareness context. The present study explores the use of an ERP-BCI for detecting new planes in an air traffic controller (ATC). Two experiments were conducted to evaluate the impact of different visual factors on target detection. Experiment 1 validated the type of stimulus used and the effect of not knowing its appearance location in an ERP-BCI scenario. Experiment 2 evaluated the effect of the size of the target stimulus appearance area and the stimulus salience in an ATC scenario. The main results demonstrate that the size of the plane appearance area had a negative impact on the detection performance and on the amplitude of the P300 component. Future studies should address this issue to improve the performance of an ATC in stimulus detection using an ERP-BCI.

Comparison of Two Paradigms Based on Stimulation with Images in a Spelling Brain-Computer Interface

A P300-based speller can be used to control a home automation system via brain activity. Evaluation of the visual stimuli used in a P300-based speller is a common topic in the field of brain-computer interfaces (BCIs). The aim of the present work is to compare, using the usability approach, two types of stimuli that have provided high performance in previous studies. Twelve participants controlled a BCI under two conditions, which varied in terms of the type of stimulus employed: a red famous face surrounded by a white rectangle (RFW) and a range of neutral pictures (NPs). The usability approach included variables related to effectiveness (accuracy and information transfer rate), efficiency (stress and fatigue), and satisfaction (pleasantness and System Usability Scale and Affect Grid questionnaires). The results indicated that there were no significant differences in effectiveness, but the system that used NPs was reported as significantly more pleasant. Hence, since satisfaction variables should also be considered in systems that potential users are likely to employ regularly, the use of different NPs may be a more suitable option than the use of a single RFW for the development of a home automation system based on a visual P300-based speller.

Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.

A Symbols Based BCI Paradigm for Intelligent Home Control Using P300 Event-Related Potentials

Brain-Computer Interface (BCI) is a technique that allows the disabled to interact with a computer directly from their brain. P300 Event-Related Potentials (ERP) of the brain have widely been used in several applications of the BCIs such as character spelling, word typing, wheelchair control for the disabled, neurorehabilitation, and smart home control. Most of the work done for smart home control relies on an image flashing paradigm where six images are flashed randomly, and the users can select one of the images to control an object of interest. The shortcoming of such a scheme is that the users have only six commands available in a smart home to control. This article presents a symbol-based P300-BCI paradigm for controlling home appliances. The proposed paradigm comprises of a 12-symbols, from which users can choose one to represent their desired command in a smart home. The proposed paradigm allows users to control multiple home appliances from signals generated by the brain. The proposed paradigm also allows the users to make phone calls in a smart home environment. We put our smart home control system to the test with ten healthy volunteers, and the findings show that the proposed system can effectively operate home appliances through BCI. Using the random forest classifier, our participants had an average accuracy of 92.25 percent in controlling the home devices. As compared to the previous studies on the smart home control BCIs, the proposed paradigm gives the users more degree of freedom, and the users are not only able to control several home appliances but also have an option to dial a phone number and make a call inside the smart home. The proposed symbols-based smart home paradigm, along with the option of making a phone call, can effectively be used for controlling home through signals of the brain, as demonstrated by the results.

Identifying potential training factors in a vibrotactile P300-BCI

Brain-computer interfaces (BCI) often rely on visual stimulation and feedback. Potential end-users with impaired vision, however, cannot use these BCIs efficiently and require a non-visual alternative. Both auditory and tactile paradigms have been developed but are often not sufficiently fast or accurate. Thus, it is particularly relevant to investigate if and how users can train and improve performance. We report data from 29 healthy participants who trained with a 4-choice tactile P300-BCI during five sessions. To identify potential training factors, we pre-post assessed the robustness of the BCI performance against increased workload in a dual task condition and determined the participants' somatosensory sensitivity thresholds with a forced-choice intensity discrimination task. Accuracy (M = 79.2% to 92.0%) and tactually evoked P300 amplitudes increased significantly, confirming successful training. Pre-post somatosensory sensitivity increased, and workload decreased significantly, but results of the dual task condition remained inconclusive. The present study confirmed the previously reported feasibility and trainability of our tactile BCI paradigm within a multi-session design. Importantly, we provide first evidence of improvement in the somatosensory system as a potential mediator for the observed training effects.

Toward P300-brain-computer interface access to contextual scene displays for AAC: An initial exploration of context and asymmetry processing in healthy adults

Brain-computer interfaces for augmentative and alternative communication (BCI-AAC) may help overcome physical barriers to AAC access. Traditionally, visually based P300-BCI-AAC displays utilize a symmetrical grid layout. Contextual scene displays are composed of context-rich images (e.g., photographs) and may support AAC success. However, contextual scene displays contrast starkly with the standard P300-grid approach. Understanding the neurological processes from which BCI-AAC devices function is crucial to human-centered computing for BCI-AAC. Therefore, the aim of this multidisciplinary investigation is to provide an initial exploration of contextual scene use for BCI-AAC.

METHODS

Participants completed three experimental conditions to evaluate the effects of item arrangement asymmetry and context on P300-based BCI-AAC signals and offline BCI-AAC accuracy, including 1) the full contextual scene condition, 2) asymmetrical item arraignment without context condition and 3) the grid condition. Following each condition, participants completed task-evaluation ratings (e.g., engagement). Offline BCI-AAC accuracy for each condition was evaluated using cross-validation.

RESULTS

Display asymmetry significantly decreased P300 latency in the centro-parietal cluster. P300 amplitudes in the frontal cluster were decreased, though nonsignificantly. Display context significantly increased N170 amplitudes in the occipital cluster, and N400 amplitudes in the centro-parietal and occipital clusters. Scenes were rated as more visually appealing and engaging, and offline BCI-AAC performance for the scene condition was not statistically different from the grid standard.

CONCLUSION

Findings support the feasibility of incorporating scene-based displays for P300-BCI-AAC development to help provide communication for individuals with minimal or emerging language and literacy skills.

Spike-Representation of EEG Signals for Performance Enhancement of Brain-Computer Interfaces

Brain-computer interfaces (BCI) relying on electroencephalography (EEG) based neuroimaging mode has shown prospects for real-world usage due to its portability and optional selectivity of fewer channels for compactness. However, noise and artifacts often limit the capacity of BCI systems especially for event-related potentials such as P300 and error-related negativity (ERN), whose biomarkers are present in short time segments at the time-series level. Contrary to EEG, invasive recording is less prone to noise but requires a tedious surgical procedure. But EEG signal is the result of aggregation of neuronal spiking information underneath the scalp surface and transforming the relevant BCI task's EEG signal to spike representation could potentially help improve the BCI performance. In this study, we designed an approach using a spiking neural network (SNN) which is trained using surrogate-gradient descent to generate task-related multi-channel EEG template signals of all classes. The trained model is in turn leveraged to obtain the latent spike representation for each EEG sample. Comparing the classification performance of EEG signal and its spike-representation, the proposed approach enhanced the performance of ERN dataset from 79.22 to 82.27% with naive bayes and for P300 dataset, the accuracy was improved from 67.73 to 69.87% using xGboost. In addition, principal component analysis and correlation metrics were evaluated on both EEG signals and their spike-representation to identify the reason for such improvement.

Neural decoding of semantic concepts: A systematic literature review

Objective Semantic concepts are coherent entities within our minds. They underpin our thought processes and are a part of the basis for our understanding of the world. Modern neuroscience research is increasingly exploring how individual semantic concepts are encoded within our brains and a number of studies are beginning to reveal key patterns of neural activity that underpin specific concepts. Building upon this basic understanding of the process of semantic neural encoding, neural engineers are beginning to explore tools and methods for semantic decoding: identifying which semantic concepts an individual is focused on at a given moment in time from recordings of their neural activity. In this paper we review the current literature on semantic neural decoding. Approach We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. Specifically, we assess the eligibility of published peer-reviewed reports via a search of PubMed and Google Scholar. We identify a total of 74 studies in which semantic neural decoding is used to attempt to identify individual semantic concepts from neural activity. Results Our review reveals how modern neuroscientific tools have been developed to allow decoding of individual concepts from a range of neuroimaging modalities. We discuss specific neuroimaging methods, experimental designs, and machine learning pipelines that are employed to aid the decoding of semantic concepts. We quantify the efficacy of semantic decoders by measuring information transfer rates. We also discuss current challenges presented by this research area and present some possible solutions. Finally, we discuss some possible emerging and speculative future directions for this research area. Significance Semantic decoding is a rapidly growing area of research. However, despite its increasingly widespread popularity and use in neuroscientific research this is the first literature review focusing on this topic across neuroimaging modalities and with a focus on quantifying the efficacy of semantic decoders.

A BCI-Based Study on the Relationship Between the SSVEP and Retinal Eccentricity in Overt and Covert Attention

Covert attention aids us in monitoring the environment and optimizing performance in visual tasks. Past behavioral studies have shown that covert attention can enhance spatial resolution. However, electroencephalography (EEG) activity related to neural processing between central and peripheral vision has not been systematically investigated. Here, we conducted an EEG study with 25 subjects who performed covert attentional tasks at different retinal eccentricities ranging from 0.75° to 13.90°, as well as tasks involving overt attention and no attention. EEG signals were recorded with a single stimulus frequency to evoke steady-state visual evoked potentials (SSVEPs) for attention evaluation. We found that the SSVEP response in fixating at the attended location was generally negatively correlated with stimulus eccentricity as characterized by Euclidean distance or horizontal and vertical distance. Moreover, more pronounced characteristics of SSVEP analysis were also acquired in overt attention than in covert attention. Furthermore, offline classification of overt attention, covert attention, and no attention yielded an average accuracy of 91.42%. This work contributes to our understanding of the SSVEP representation of attention in humans and may also lead to brain-computer interfaces (BCIs) that allow people to communicate with choices simply by shifting their attention to them.

Extending Brain-Computer Interface Access with a Multilingual Language Model in the P300 Speller

Brain-computer interfaces (BCI) such as the P300 speller have the potential to restore communication to advanced-stage neuromuscular disease patients. Research has improved typing speed and accuracy through innovations including the use of language models. While significant advances have been made, implementations have largely been restricted to a single language, primarily English. It is unclear whether these improvements would extend to other languages that present potential technical hurdles due to different alphabets and grammatical structures. Here, we adapt a language model-based classifier designed for English to two other languages, Spanish and Greek, to demonstrate the generalizability of these methods. Online experimental trials with 30 healthy native English, Spanish, and Greek speakers showed no significant difference between performances using the different versions of the system (66.20 vs. 61.97 vs. 60.89 bits/minute). Extending these methods across languages allows for expanding access to BCI systems to other populations, particularly in the developing world.

Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review

Objective.Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain-computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines.Approach.The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc.Main results.The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development.Significance.Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

Artificial Intelligence Algorithms in Visual Evoked Potential-Based Brain-Computer Interfaces for Motor Rehabilitation Applications: Systematic Review and Future Directions

Brain-Computer Interface (BCI) is a technology that uses electroencephalographic (EEG) signals to control external devices, such as Functional Electrical Stimulation (FES). Visual BCI paradigms based on P300 and Steady State Visually Evoked potentials (SSVEP) have shown high potential for clinical purposes. Numerous studies have been published on P300- and SSVEP-based non-invasive BCIs, but many of them present two shortcomings: (1) they are not aimed for motor rehabilitation applications, and (2) they do not report in detail the artificial intelligence (AI) methods used for classification, or their performance metrics. To address this gap, in this paper the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was applied to prepare a systematic literature review (SLR). Papers older than 10 years, repeated or not related to a motor rehabilitation application, were excluded. Of all the studies, 51.02% referred to theoretical analysis of classification algorithms. Of the remaining, 28.48% were for spelling, 12.73% for diverse applications (control of wheelchair or home appliances), and only 7.77% were focused on motor rehabilitation. After the inclusion and exclusion criteria were applied and quality screening was performed, 34 articles were selected. Of them, 26.47% used the P300 and 55.8% the SSVEP signal. Five applications categories were established: Rehabilitation Systems (17.64%), Virtual Reality environments (23.52%), FES (17.64%), Orthosis (29.41%), and Prosthesis (11.76%). Of all the works, only four performed tests with patients. The most reported machine learning (ML) algorithms used for classification were linear discriminant analysis (LDA) (48.64%) and support vector machine (16.21%), while only one study used a deep learning algorithm: a Convolutional Neural Network (CNN). The reported accuracy ranged from 38.02 to 100%, and the Information Transfer Rate from 1.55 to 49.25 bits per minute. While LDA is still the most used AI algorithm, CNN has shown promising results, but due to their high technical implementation requirements, many researchers do not justify its implementation as worthwile. To achieve quick and accurate online BCIs for motor rehabilitation applications, future works on SSVEP-, P300-based and hybrid BCIs should focus on optimizing the visual stimulation module and the training stage of ML and DL algorithms.

Improve P300-speller performance by online tuning stimulus onset asynchrony (SOA)

Objective. The P300-Speller is a classic brain-computer interface paradigm that has been subjected to numerous clinical trials. Some studies have reported that the performance of the P300-Speller is closely related to stimulus onset asynchrony (SOA), but very few studies have attempted to improve the performance of the P300-Speller by optimizing SOA.Approach.In this paper, we designed a P300-Speller system based on a variable SOA and dynamic stop strategy, which can automatically adjust SOA according to real-time operational performance.Main results.The online experiment results of 18 subjects showed that the event-related potential classifier and the dynamic stop algorithm established at 200 ms SOA can maintain the performance at a certain level among 50-300 ms SOA. The system can then reduce the SOA from an initial 200 ms to an average of about 98.5 ms while maintaining letter output accuracy. The average theoretical information transfer rate was significantly improved from 42.4 to 85 bit min^-1(the maximum was 232 bit min^-1).Significance.These results demonstrate that the system established in this paper can automatically optimize the SOA settings, and this personalized SOA adjustment can effectively improve the performance of the P300-Speller.

Evaluation of color modulation in visual P300-speller using new stimulus patterns

Objective The stimulus color of P300-BCI systems has been successfully modified. However, the effects of different color combinations have not been widely investigated. In this study, we designed new stimulus patterns to evaluate the influence of color modulation on the BCI performance and waveforms of the evoked related potential (ERP).Methods Comparison was performed for three new stimulus patterns consisting of red face and colored block-shape, namely, red face with a white rectangle (RFW), red face with a blue rectangle (RFB), and red face with a red rectangle (RFR). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis. Results The RFW pattern obtained the highest average online accuracy with 96.94%, and those of RFR and RFB patterns were 93.61% and of 92.22% respectively. Significant differences in online accuracy and information transfer rate (ITR) were found between RFW and RFR patterns (p < 0.05). Conclusion Compared with RFR and RFB patterns, RFW yielded the best performance in P300-BCI. These new stimulus patterns with different color combinations have considerable importance to BCI applications and user-friendliness.

A novel approach for designing authentication system using a picture based P300 speller

Due to great advances in the field of information technology, the need for a more reliable authentication system has been growing rapidly for protecting the individual or organizational assets from online frauds. In the past, many authentication techniques have been proposed like password and tokens but these techniques suffer from many shortcomings such as offline attacks (guessing) and theft. To overcome these shortcomings, in this paper brain signal based authentication system is proposed. A Brain-Computer Interface (BCI) is a tool that provides direct human-computer interaction by analyzing brain signals. In this study, a person authentication approach that can effectively recognize users by generating unique brain signal features in response to pictures of different objects is presented. This study focuses on a P300 BCI for authentication system design. Also, three classifiers were tested: Quadratic Discriminant Analysis (QDA), K-Nearest Neighbor, and Quadratic Support Vector Machine. The results showed that the proposed visual stimuli with pictures as selection attributes obtained significantly higher classification accuracies (97%) and information transfer rates (37.14 bits/min) as compared to the conventional paradigm. The best performance was observed with the QDA as compare to other classifiers. This method is advantageous for developing brain signal based authentication application as it cannot be forged (like Shoulder surfing) and can still be used for disabled users with a brain in good running condition. The results show that reduced matrix size and modified visual stimulus typically affects the accuracy and communication speed of P300 BCI performance.

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Electroencephalography (EEG) signals elicited by multimodal stimuli can drive brain-computer interfaces (BCIs), and research has demonstrated that visual and auditory stimuli can be employed simultaneously to improve BCI performance. However, no studies have investigated the effect of multimodal stimuli in rapid serial visual presentation (RSVP) BCIs. The present study proposed a rapid serial multimodal presentation (RSMP) BCI that incorporates artificial facial images and artificial voice stimuli. To clarify the effect of audiovisual stimuli on the RSMP BCI, scrambled images and masked sounds were applied instead of visual and auditory stimuli, respectively. The findings indicated that the audiovisual stimuli improved performance of the RSMP BCI, and that P300 at Pz contributed to classification accuracy. Online accuracy of the BCI reached 85.7 ± 11.5 %. Taken together, these findings may aid in the development of better gaze-independent BCI systems.

Enhancement for P300-speller classification using multi-window discriminative canonical pattern matching

Objective.P300s are one of the most studied event-related potentials (ERPs), which have been widely used for brain-computer interfaces (BCIs). Thus, fast and accurate recognition of P300s is an important issue for BCI study. Recently, there emerges a lot of novel classification algorithms for P300-speller. Among them, discriminative canonical pattern matching (DCPM) has been proven to work effectively, in which discriminative spatial pattern (DSP) filter can significantly enhance the spatial features of P300s. However, the pattern of ERPs in space varies with time, which was not taken into consideration in the traditional DCPM algorithm.Approach.In this study, we developed an advanced version of DCPM, i.e. multi-window DCPM, which contained a series of time-dependent DSP filters to fine-tune the extraction of spatial ERP features. To verify its effectiveness, 25 subjects were recruited and they were asked to conduct the typical P300-speller experiment.Main results.As a result, multi-window DCPM achieved the character recognition accuracy of 91.84% with only five training characters, which was significantly better than the traditional DCPM algorithm. Furthermore, it was also compared with eight other popular methods, including SWLDA, SKLDA, STDA, BLDA, xDAWN, HDCA, sHDCA and EEGNet. The results showed multi-window DCPM preformed the best, especially using a small calibration dataset. The proposed algorithm was applied to the BCI Controlled Robot Contest of P300 paradigm in 2019 World Robot Conference, and won the first place.Significance.These results demonstrate that multi-window DCPM is a promising method for improving the performance and enhancing the practicability of P300-speller.

Habituation of P300 in the Use of P300-based Brain-Computer Interface Spellers: Individuals With Amyotrophic Lateral Sclerosis Versus Age-Matched Controls

The P300-based brain-computer interface speller can provide motor independent communication to individuals with amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disorder that affects the motor system. P300 amplitude stability is critical for operation of the P300 speller. The P300 has good long-term stability, but to our knowledge, short-term habituation in the P300 speller has not been studied. In the current study, 15 participants: 8 ALS patients and 7 age-matched healthy volunteers (HVs), used 2 versions of P300 spellers, Face speller and Flash speller, each for 30 minutes. The ALS group performed as well as the HVs in both spellers and HVs did better with the Face speller than Flash speller while the ALS group performed equally well in both spellers. Neither intra-run P300 habituation nor inter-run P300 habituation was found. The P300 speller could be a reliable communication device for individuals with ALS.

Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm

Previous proposals for controlling a P300-based BCI speller have shown an improvement using alternative images instead of letters as target stimuli under a row-column paradigm (RCP). However, the RCP is not suitable for those patients with a lack of gaze control. To solve that, the rapid serial visual presentation (RSVP) paradigm has been proposed in previous studies. The aim of the present work is to assess if a set of alternative pictures that improved performance in RCP could also improve performance in RSVP. Sixteen participants controlled four conditions in calibration and online tasks: letters in RCP, pictures in RCP, letters in RSVP and pictures in RSVP. The effect given by pictures was greater under RCP than under RSVP, both for performance and event-related potential analyses. Indeed, pictures did not show any improvement under RSVP in comparison to letters. In addition, the condition with pictures under RCP was declared the favourite by most users (68.75%), while the condition with pictures under RSVP was not chosen as favourite by any participant. Therefore, this work shows that the improvement related to the use of pictures as alternative flashing stimuli under RCP may not be transferred to RSVP. Graphical abstract.

Performance Analysis With Different Types of Visual Stimuli in a BCI-Based Speller Under an RSVP Paradigm

Brain-Computer Interface (BCI) systems enable an alternative communication channel for severely-motor disabled patients to interact with their environment using no muscular movements. In recent years, the importance of research into non-gaze dependent brain-computer interface paradigms has been increasing, in contrast to the most frequently studied BCI-based speller paradigm (i.e., row-column presentation, RCP). Several visual modifications that have already been validated under the RCP paradigm for communication purposes have not been validated under the most extended non-gaze dependent rapid serial visual presentation (RSVP) paradigm. Thus, in the present study, three different sets of stimuli were assessed under RSVP, with the following communication features: white letters (WL), famous faces (FF), neutral pictures (NP). Eleven healthy subjects participated in this experiment, in which the subjects had to go through a calibration phase, an online phase and, finally, a subjective questionnaire completion phase. The results showed that the FF and NP stimuli promoted better performance in the calibration and online phases, being slightly better in the FF paradigm. Regarding the subjective questionnaires, again both FF and NP were preferred by the participants in contrast to the WL stimuli, but this time the NP stimuli scored slightly higher. These findings suggest that the use of FF and NP for RSVP-based spellers could be beneficial to increase information transfer rate in comparison to the most frequently used letter-based stimuli and could represent a promising communication system for individuals with altered ocular-motor function.

A Spelling Paradigm With an Added Red Dot Improved the P300 Speller System Performance

The traditional P300 speller system uses the flashing row or column spelling paradigm. However, the classification accuracy and information transfer rate of the P300 speller are not adequate for real-world application. To improve the performance of the P300 speller, we devised a new spelling paradigm in which the flashing row or column of a virtual character matrix is covered by a translucent green circle with a red dot in either the upper or lower half (GC-RD spelling paradigm). We compared the event-related potential (ERP) waveforms with a control paradigm (GC spelling paradigm), in which the flashing row or column of a virtual character matrix was covered by a translucent green circle only. Our experimental results showed that the amplitude of P3a at the parietal area and P3b at the frontal–central–parietal areas evoked by the GC-RD paradigm were significantly greater than those induced by the GC paradigm. Higher classification accuracy and information transmission rates were also obtained in the GC-RD system. Our results indicated that the added red dots increased attention and visuospatial information, resulting in an amplitude increase in both P3a and P3b, thereby improving the performance of the P300 speller system.

Single-Option P300-BCI Performance Is Affected by Visual Stimulation Conditions

The P300 paradigm is one of the most promising techniques for its robustness and reliability in Brain-Computer Interface (BCI) applications, but it is not exempt from shortcomings. The present work studied single-trial classification effectiveness in distinguishing between target and non-target responses considering two conditions of visual stimulation and the variation of the number of symbols presented to the user in a single-option visual frame. In addition, we also investigated the relationship between the classification results of target and non-target events when training and testing the machine-learning model with datasets containing different stimulation conditions and different number of symbols. To this end, we designed a P300 experimental protocol considering, as conditions of stimulation: the color highlighting or the superimposing of a cartoon face and from four to nine options. These experiments were carried out with 19 healthy subjects in 3 sessions. The results showed that the Event-Related Potentials (ERP) responses and the classification accuracy are stronger with cartoon faces as stimulus type and similar irrespective of the amount of options. In addition, the classification performance is reduced when using datasets with different type of stimulus, but it is similar when using datasets with different the number of symbols. These results have a special connotation for the design of systems, in which it is intended to elicit higher levels of evoked potentials and, at the same time, optimize training time.

P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis

OBJECTIVE

People with amyotrophic lateral sclerosis (ALS) can benefit from brain-computer interfaces (BCIs). However, users with ALS may experience significant variations in BCI performance and event-related potential (ERP) characteristics. This study investigated latency jitter and its correlates in ALS.

METHODS

Electroencephalographic (EEG) responses were recorded from six people with ALS and nine neurotypical controls. ERP amplitudes and latencies were extracted. Classifier-based latency estimation was used to calculate latency jitter. ERP components and latency jitter were compared between groups using Wilcoxon rank-sum tests. Correlations between latency jitter and each of the clinical measures, ERP features, and performance measures were investigated using Spearman and repeated measures correlations.

RESULTS

Latency jitter was significantly increased in participants with ALS and significantly negatively correlated with BCI performance in both ALS and control participants. ERP amplitudes were significantly attenuated in ALS, and significant correlations between ERP features and latency jitter were observed. There was no significant correlation between latency jitter and clinical measures.

CONCLUSIONS

Latency jitter is increased in ALS and correlates with both BCI performance and ERP features.

SIGNIFICANCE

These results highlight the associations of latency jitter with BCI performance and ERP characteristics and could inform future BCI designs for people with ALS.

Impact of Stimulus Features on the Performance of a Gaze-Independent Brain-Computer Interface Based on Covert Spatial Attention Shifts

Regaining communication abilities in patients who are unable to speak or move is one of the main goals in decoding brain waves for brain-computer interface (BCI) control. Many BCI approaches designed for communication rely on attention to visual stimuli, commonly applying an oddball paradigm, and require both eye movements and adequate visual acuity. These abilities may, however, be absent in patients who depend on BCI communication. We have therefore developed a response-based communication BCI, which is independent of gaze shifts but utilizes covert shifts of attention to the left or right visual field. We recorded the electroencephalogram (EEG) from 29 channels and coregistered the vertical and horizontal electrooculogram. Data-driven decoding of small attention-based differences between the hemispheres, also known as N2pc, was performed using 14 posterior channels, which are expected to reflect correlates of visual spatial attention. Eighteen healthy participants responded to 120 statements by covertly directing attention to one of two colored symbols (green and red crosses for "yes" and "no," respectively), presented in the user's left and right visual field, respectively, while maintaining central gaze fixation. On average across participants, 88.5% (std: 7.8%) of responses were correctly decoded online. In order to investigate the potential influence of stimulus features on accuracy, we presented the symbols with different visual angles, by altering symbol size and eccentricity. The offline analysis revealed that stimulus features have a minimal impact on the controllability of the BCI. Hence, we show with our novel approach that spatial attention to a colored symbol is a robust method with which to control a BCI, which has the potential to support severely paralyzed people with impaired eye movements and low visual acuity in communicating with their environment.

TeleBCI: remote user training, monitoring, and communication with an evoked-potential brain-computer interface

Brain-computer interfaces (BCIs) are a movement-independent form of augmentative and alternative communication (AAC) for individuals with amyotrophic lateral sclerosis (ALS). The rare utilization of such devices in the homes of patients stems from a number of factors, one of which is the complexity of providing training and support for users. This paper describes the teleBCI interface used to train the patient and facilitator in the operation of a virtual keyboard using an evoked potential BCI. Fifteen patients with motor neuron disease and their communication partners were included in the study, participating from their homes while receiving remote support from the research team. Patient/caregiver teams completed 8 sessions each of P300 BCI training virtually with the researcher. As they participated in subsequent training sessions, participant teams required less help to complete physical, computer, and BCI-specific tasks associated with device use. A subset of users experienced improved performance over sessions, progressing to utilize the full functionality of the speller and communicate with a nurse partner over a telemedicine interface. Perceptions of device utility varied with accuracy of the BCI system. In the management of ALS, the integration of telemedicine provides new opportunities for care delivery, including how BCI-AAC are deployed and used.

A Comparison of Classification Techniques to Predict Brain-Computer Interfaces Accuracy Using Classifier-Based Latency Estimation

P300-based Brain-Computer Interface (BCI) performance is vulnerable to latency jitter. To investigate the role of latency jitter on BCI system performance, we proposed the classifier-based latency estimation (CBLE) method. In our previous study, CBLE was based on least-squares (LS) and stepwise linear discriminant analysis (SWLDA) classifiers. Here, we aim to extend the CBLE method using sparse autoencoders (SAE) to compare the SAE-based CBLE method with LS- and SWLDA-based CBLE. The newly-developed SAE-based CBLE and previously used methods are also applied to a newly-collected dataset to reduce the possibility of spurious correlations. Our results showed a significant (p<0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, we also examined the effect of the number of electrodes on each classification technique. Our results showed that on the whole, CBLE worked regardless of the classification method and electrode count; by contrast the effect of the number of electrodes on BCI performance was classifier dependent.

The Effect of the Graphic Structures of Humanoid Robot on N200 and P300 Potentials

Humanoid robots are widely used in brain computer interface (BCI). Using a humanoid robot stimulus could increase the amplitude of event-related potentials (ERPs), which improves BCI performance. Since a humanoid robot contains many human elements, the element that increases the ERPs amplitude is unclear, and how to test the effect of it on the brain is a problem. This study used different graphic structures of an NAO humanoid robot to design three types of robot stimuli: a global robot, its local information, and its topological action. Ten subjects first conducted an odd-ball-based BCI (OD-BCI) by applying these stimuli. Then, they accomplished a delayed matching-to-sample task (DMST) that was used to specialize the encoding and retrieval phases of the OD-BCI task. In the retrieval phase of the DMST, the global stimulus induces the largest N200 and P300 potentials with the shortest latencies in the frontal, central, and occipital areas. This finding is in accordance with the P300 and classification performance of the OD-BCI task. When induced by the local stimulus, the subjects responded faster and more accurately in the retrieval phase of the DMST than in the other two conditions, indicating that the local stimulus improved the subject's responses. These results indicate that the OD-BCI task causes subject's retrieval work when the subject recognizes and outputs the stimulus. The global stimulus that contains topological and local elements could make brain react faster and induce larger ERPs, this finding could be used during the development of visual stimuli to improve BCI performance.

Doubling the Speed of N200 Speller via Dual-Directional Motion Encoding

OBJECTIVE

Motion-onset visual evoked potentials (mVEPs)-based spellers, also known as N200 spellers, have been successfully implemented, avoiding flashing stimuli that are common in visual brain-computer interface (BCI). However, their information transfer rates (ITRs), typically below 50 bits/min, are lower than other visual BCI spellers. In this study, we sought to improve the speed of N200 speller to a level above the well-known P300 spellers.

APPROACH

Based on our finding of the spatio-temporal asymmetry of N200 response elicited by leftward and rightward visual motion, a novel dual-directional N200 speller was implemented. By presenting visual stimuli moving in two different directions simultaneously, the new paradigm reduced the stimuli presentation time by half, while ensuring separable N200 features between two visual motion directions. Furthermore, a probability-based dynamic stopping algorithm was also proposed to shorten the decision time for each output further. Both offline and online tests were conducted to evaluate the performance in ten participants.

MAIN RESULTS

Offline results revealed contralateral dominant temporal and spatial patterns in N200 responses when subjects attended to stimuli moving leftward or rightward. In online experiments, the dual-directional paradigm achieved an average ITR of 79.8 bits/min, with the highest ITR of 124.8 bits/min. Compared with the traditional uni-directional N200 speller, the median gain on the ITR was 202%.

SIGNIFICANCE

The proposed dual-directional paradigm managed to double the speed of the N200 speller. Together with its non-flashing characteristics, this dual-directional N200 speller is promising to be a competent candidate for fast and reliable BCI applications.

BRAIN SIGNATURES PERSPECTIVE FOR HIGH-SECURITY AUTHENTICATION

In the recent past, the web (internet) has emerged as the most interactive authentication system for all of us (i.e. Internet banking passwords, system or building access, and e-payment platforms, etc.) and as a result, traditional authentication systems (like passwords or token-based) are never again more secure i.e. they are vulnerable to attacks. As a result, the security of individual information and safe access to a system winds up prime necessities. Therefore, the EEG-based authentication system has recently become a reasonable key for high-level security. This study centers upon P300 evoked potential-based authentication system designing. In this paper, a new visual stimulus paradigm (i.e. [Formula: see text] P300 speller) using pictures of different objects as stimuli for a person authentication system is designed instead of the conventional character-based paradigm (i.e. [Formula: see text] speller) for increasing the classification accuracy and Information Transfer Rate (ITR). The trial begins by exhibiting a collection of pictures of various objects on four corners of the PC screen comprising of random object pictures (non-target) alongside password pictures (target) that trigger P300 reactions. The P300 reaction’s rightness then checks the identity of the subject concerning the focused pictures (Target). The proposed investigation model achieves higher classification accuracy of 96.78%, along with 0.03075 False Rejection Rate (FRR), 0.03297 False Acceptation Rate (FAR), and ITR of [Formula: see text]. This study has shown that P300-based authentication system has an advantage over conventional methods (Password, Token, etc.) as EEG-based systems cannot be mimicked or forged (like Shoulder surfing in case of password) and can still be used for disabled users with a brain in good running condition. The classification results revealed that the performance of the QDA classifier outperformed other classifiers based on accuracy and ITR.

Independent Low-Rank Matrix Analysis-Based Automatic Artifact Reduction Technique Applied to Three BCI Paradigms

Electroencephalogram (EEG)-based brain-computer interfaces (BCIs) can potentially enable people to non-invasively and directly communicate with others using brain activities. Artifacts generated from body activities (e.g., eyeblinks and teeth clenches) often contaminate EEGs and make EEG-based classification/identification hard. Although independent component analysis (ICA) is the gold-standard technique for attenuating the effects of such contamination, the estimated independent components are still mixed with artifactual and neuronal information because ICA relies only on the independence assumption. The same problem occurs when using independent vector analysis (IVA), an extended ICA method. To solve this problem, we designed an independent low-rank matrix analysis (ILRMA)-based automatic artifact reduction technique that clearly models sources from observations under the independence assumption and a low-rank nature in the frequency domain. For automatic artifact reduction, we combined the signal separation technique with an independent component classifier for EEGs named ICLabel. To assess the comparative efficiency of the proposed method, the discriminabilities of artifact-reduced EEGs using ICA, IVA, and ILRMA were determined using an open-access EEG dataset named OpenBMI, which contains EEG data obtained through three BCI paradigms [motor-imagery (MI), event-related potential (ERP), and steady-state visual evoked potential (SSVEP)]. BCI performances were obtained using these three paradigms after applying artifact reduction techniques, and the results suggested that our proposed method has the potential to achieve higher discriminability than ICA and IVA for BCIs. In addition, artifact reduction using the ILRMA approach clearly improved (by over 70%) the averaged BCI performances using artifact-reduced data sufficiently for most needs of the BCI community. The extension of ICA families to supervised separation that leaves the discriminative ability would further improve the usability of BCIs for real-life environments in which artifacts frequently contaminate EEGs.

30+ years of P300 brain-computer interfaces

Brain-computer interfaces (BCIs) directly measure brain activity with no physical movement and translate the neural signals into messages. BCIs that employ the P300 event-related brain potential often have used the visual modality. The end user is presented with flashing stimuli that indicate selections for communication, control, or both. Counting each flash that corresponds to a specific target selection while ignoring other flashes will elicit P300s to only the target selection. P300 BCIs also have been implemented using auditory or tactile stimuli. P300 BCIs have been used with a variety of applications for severely disabled end users in their homes without frequent expert support. P300 BCI research and development has made substantial progress, but challenges remain before these tools can become practical devices for impaired patients and perhaps healthy people.

Effects of Spatial Stimulus Overlap in a Visual P300-based Brain-computer Interface

The rapid serial visual presentation (RSVP) paradigm seems to be one of the most appropriate for patients using P300-based brain-computer interface (BCI) applications, since non-ocular movements are required. However, according to previous works, the use of different locations for each stimulus may improve performance. Thus, the aim of the present work is to explore how spatial overlap between stimuli influences performance in controlling a visual P300-based BCI. Nineteen participants were tested using four levels of overlap between two stimuli: 100%, 66.7%, 33.3% and 0%. Significant differences in accuracy were found between the 0% overlapped condition and all the other conditions, and between 33.3% and higher overlap (66.7% and 100%). These results can be explained due to a modulation in the non-target stimulus amplitude signal caused by the overlapping factor. In short, the stimulus overlap provokes a modulation in performance using a P300-based BCI; this should be considered in future BCI proposals in which an optimal surface exploitation is convenient and potential users have only residual ocular movement.

A novel hybrid BCI speller based on RSVP and SSVEP paradigm

BACKGROUND AND OBJECTIVE

Steady-state visual evoked potential (SSVEP) and rapid serial visual presentation (RSVP) are useful methods in the brain-computer interface (BCI) systems. Hybrid BCI systems that combine these two approaches can enhance the proficiency of the P300 spellers.

METHODS

In this study, a new hybrid RSVP/SSVEP BCI is proposed to increase the classification accuracy and information transfer rate (ITR) as compared with the other RSVP speller paradigms. In this paradigm, RSVP (eliciting a P300 response) and SSVEP stimulations are presented in such a way that the target group of characters is identified by RSVP stimuli, and the target character is recognized by SSVEP stimuli.

RESULTS

The proposed paradigm achieved accuracy of 93.06%, and ITR of 23.41 bit/min averaged across six subjects.

CONCLUSIONS

The new hybrid system demonstrates that by using SSVEP stimulation in Triple RSVP speller paradigm, we could enhance the performance of the system as compared with the traditional Triple RSVP paradigm. Our work is the first hybrid paradigm in RSVP spellers that could obtain the higher classification accuracy and information transfer rate in comparison with the previous RSVP spellers.

Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System

OBJECTIVE

Brain-computer interface (BCI) based communication remains a challenge for people with later-stage amyotrophic lateral sclerosis (ALS) who lose all voluntary muscle control. Although recent studies have demonstrated the feasibility of functional near-infrared spectroscopy (fNIRS) to successfully control BCIs primarily for healthy cohorts, these systems are yet inefficient for people with severe motor disabilities like ALS. In this study, we developed a new fNIRS-based BCI system in concert with a single-trial Visuo-Mental (VM) paradigm to investigate the feasibility of enhanced communication for ALS patients, particularly those in the later stages of the disease.

METHODS

In the first part of the study, we recorded data from six ALS patients using our proposed protocol (fNIRS-VM) and compared the results with the conventional electroencephalography (EEG)-based multi-trial P3Speller (P3S). In the second part, we recorded longitudinal data from one patient in the late locked-in state (LIS) who had fully lost eye-gaze control. Using statistical parametric mapping (SPM) and correlation analysis, the optimal channels and hemodynamic features were selected and used in linear discriminant analysis (LDA).

RESULTS

Over all the subjects, we obtained an average accuracy of 81.3%±5.7% within comparatively short times (< 4 sec) in the fNIRS-VM protocol relative to an average accuracy of 74.0%±8.9% in the P3S, though not competitive in patients with no substantial visual problems. Our longitudinal analysis showed substantially superior accuracy using the proposed fNIRS-VM protocol (73.2%±2.0%) over the P3S (61.8%±1.5%).

SIGNIFICANCE

Our findings indicate the potential efficacy of our proposed system for communication and control for late-stage ALS patients.

The N400 for brain computer interfacing: complexities and opportunities

The N400 is an event related potential that is evoked in response to conceptually meaningful stimuli. It is for instance more negative in response to incongruent than congruent words in a sentence, and more negative for unrelated than related words following a prime word. This sensitivity to semantic content of a stimulus in relation to the mental context of an individual makes it a signal of interest for Brain Computer Interfaces. A complicating aspect is the number of factors that can affect the N400 amplitude. In this paper, we provide an accessible overview of this range of N400 effects, and survey the three main BCI application areas that currently exploit the N400: (1) exploiting the semantic processing of faces to enhance matrix speller performance, (2) detecting language processing in patients with Disorders of Consciousness, and (3) using semantic stimuli to probe what is on a user's mind. Drawing on studies from these application areas, we illustrate that the N400 can successfully be exploited for BCI purposes, but that the signal-to-noise ratio is a limiting factor, with signal strength also varying strongly across subjects. Furthermore, we put findings in context of the general N400 literature, noting open questions and identifying opportunities for further research.

Effects of Visual Attention on Tactile P300 BCI

Objective. Tactile P300 brain-computer interfaces (BCIs) can be manipulated by users who only need to focus their attention on a single-target stimulus within a stream of tactile stimuli. To date, a multitude of tactile P300 BCIs have been proposed. In this study, our main purpose is to explore and investigate the effects of visual attention on a tactile P300 BCI. Approach. We designed a conventional tactile P300 BCI where vibration stimuli were provided by five stimulators and two of them were fixed on target locations on the participant's left and right wrists. Two conditions (one condition with visual attention and the other condition without visual attention) were tested by eleven healthy participants. Main Results. Our results showed that, when participants visually attended to the location of target stimulus, significantly higher classification accuracies and information transfer rates were obtained (both for p < 0.05). Furthermore, participants reported that visually attending to the stimulus made it easier to identify the target stimulus in random sequences of vibration stimuli. Significance. These findings suggest that visual attention has positive effects on both tactile P300 BCI performance and user-evaluation.

Active inference as a unifying, generic and adaptive framework for a P300-based BCI

OBJECTIVE

Going adaptive is a major challenge for the field of brain-computer interface (BCI). This entails a machine that optimally articulates inference about the user's intentions and its own actions. Adaptation can operate over several dimensions which calls for a generic and flexible framework.

APPROACH

We appeal to one of the most comprehensive computational approach to brain (adaptive) functions: the active inference (AI) framework. It entails an explicit (probabilistic) model of the user that the machine interacts with, here involved in a P300-spelling task. This takes the form of a discrete input-output state-space model establishing the link between the machine's (i) observations-a P300 or error potential for instance, (ii) representations-of the user intentions to spell or pause, and (iii) actions-to flash, spell or switch-off the application.

MAIN RESULTS

Using simulations with real EEG data from 18 subjects, results demonstrate the ability of AI to yield a significant increase in bit rate (17%) over state-of-the-art approaches, such as dynamic stopping.

SIGNIFICANCE

Thanks to its flexibility, this one model enables to implement optimal (dynamic) stopping but also optimal flashing (i.e. active sampling), automated error correction, and switching off when the user does not look at the screen anymore. Importantly, this approach enables the machine to flexibly arbitrate between all these possible actions. We demonstrate AI as a unifying and generic framework to implement a flexible interaction behaviour in a given BCI context.

Comparison of the ERP-Based BCI Performance Among Chromatic (RGB) Semitransparent Face Patterns

Objective

Previous studies have shown that combing with color properties may be used as part of the display presented to BCI users in order to improve performance. Build on this, we explored the effects of combinations of face stimuli with three primary colors (RGB) on BCI performance which is assessed by classification accuracy and information transfer rate (ITR). Furthermore, we analyzed the waveforms of three patterns.

Methods

We compared three patterns in which semitransparent face is overlaid three primary colors as stimuli: red semitransparent face (RSF), green semitransparent face (GSF), and blue semitransparent face (BSF). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. In addition, a Repeated-measures ANOVA (RM-ANOVA) and Bonferroni correction were chosen for statistical analysis.

Results

The results indicated that the RSF pattern achieved the highest online averaged accuracy with 93.89%, followed by the GSF pattern with 87.78%, while the lowest performance was caused by the BSF pattern with an accuracy of 81.39%. Furthermore, significant differences in classification accuracy and ITR were found between RSF and GSF (p < 0.05) and between RSF and BSF patterns (p < 0.05).

Conclusion

The semitransparent faces colored red (RSF) pattern yielded the best performance of the three patterns. The proposed patterns based on ERP-BCI system have a clinically significant impact by increasing communication speed and accuracy of the P300-speller for patients with severe motor impairment.

Extremely Reduced Data Sets Indicate Optimal Stimulation Parameters for Classification in Brain-Computer Interfaces

The time between the onset of subsequent auditory or visual stimuli - also known as stimulus onset asynchrony (SOA) - determines many of the event-related potential characteristics of the resulting evoked brain signals. Specifically, the SOA value influences the performance of an individual subject in brain-computer interface (BCI) applications like spellers. In the past, subject-specific optimization of the SOA was rarely considered in BCI studies. Our research strives to reduce the time requirements of individual BCI stimulus parameter optimization. This work contributes to this goal in two ways. First, we show that even the classification performance on extremely reduced training data subsets reveals the influence of SOA. Second, we show, that these noisy estimates are sufficient to make decisions for individual choices of the SOA that transfer to good classification performance on large training data sets. Thus our work contributes to individually tailored SOA selection procedures for BCI users.

The Self-Face Paradigm Improves the Performance of the P300-Speller System

Objective: Previous studies have shown that the performance of the famous face P300-speller was better than that of the classical row/column flashing P300-speller. Furthermore, in some studies, the brain was more active when responding to one's own face than to a famous face, and a self-face stimulus elicited larger amplitude event-related potentials (ERPs) than did a famous face. Thus, we aimed to study the role of the self-face paradigm on further improving the performance of the P300-speller system with the famous face P300-speller paradigm as the control paradigm.

Methods: We designed two facial P300-speller paradigms based on the self-face and a famous face (Ming Yao, a sports star; the famous face spelling paradigm) with a neutral expression.

Results: ERP amplitudes were significantly greater in the self-face than in the famous face spelling paradigm at the parietal area from 340 to 480 ms (P300), from 480 to 600 ms (P600f), and at the fronto-central area from 700 to 800 ms. Offline and online classification results showed that the self-face spelling paradigm accuracies were significantly higher than those of the famous face spelling paradigm at superposing first two times (P < 0.05). Similar results were found for information transfer rates (P < 0.05).

Conclusions: The self-face spelling paradigm significantly improved the performance of the P300-speller system. This has significant practical applications for brain-computer interfaces (BCIs) and could avoid infringement issues caused by using images of other people's faces.