An adaptive P300-based control system

A novel visual brain-computer interfaces paradigm based on evoked related potentials evoked by weak and small number of stimuli

Introduction

Traditional visual Brain-Computer Interfaces (v-BCIs) usually use large-size stimuli to attract more attention from users and then elicit more distinct and robust EEG responses, which would cause visual fatigue and limit the length of use of the system. On the contrary, small-size stimuli always need multiple and repeated stimulus to code more instructions and increase separability among each code. These common v-BCIs paradigms can cause problems such as redundant coding, long calibration time, and visual fatigue.

Methods

To address these problems, this study presented a novel v-BCI paradigm using weak and small number of stimuli, and realized a nine-instruction v-BCI system that controlled by only three tiny stimuli. Each of these stimuli were located between instructions, occupied area with eccentricities subtended 0.4°, and flashed in the row-column paradigm. The weak stimuli around each instruction would evoke specific evoked related potentials (ERPs), and a template-matching method based on discriminative spatial pattern (DSP) was employed to recognize these ERPs containing the intention of users. Nine subjects participated in the offline and online experiments using this novel paradigm.

Results

The average accuracy of the offline experiment was 93.46% and the online average information transfer rate (ITR) was 120.95 bits/min. Notably, the highest online ITR achieved 177.5 bits/min.

Discussion

These results demonstrate the feasibility of using a weak and small number of stimuli to implement a friendly v-BCI. Furthermore, the proposed novel paradigm achieved higher ITR than traditional ones using ERPs as the controlled signal, which showed its superior performance and may have great potential of being widely used in various fields.

Improvement of brain-computer interface in motor imagery training through the designing of a dynamic experiment and FBCSP

Motor imagery (MI) can produce a specific brain pattern when the subject imagines performing a particular action without any actual body movements. According to related previous research, the improvement of the training of MI brainwaves can be adopted by feedback methods in which the analysis of brainwave characteristics is very important. The aim of this study was to improve the subject's MI and the accuracy of classification. In order to ameliorate the accuracy of the MI of the left and right hand, the present study designed static and dynamic visual stimuli in experiments so as to evaluate which one can improve subjects' imagination training. Additionally, the filter bank common spatial pattern (FBCSP) method was used to divide the frequency band range of the brainwaves into multiple segments, following which linear discriminant analysis (LDA) was adopted for classification. The results revealed that the averaged false positive rate (FPR) under FBCSP-LDA in the dynamic MI experiment was the lowest FPR (23.76%). As such, this study suggested that a combination of the dynamic MI experiment and the FBCSP-LDA method improved the overall prediction error rate and ameliorated the performance of the MI brain-computer interface.

A high-speed hybrid brain-computer interface with more than 200 targets

Objective. Brain-computer interfaces (BCIs) have recently made significant strides in expanding their instruction set, which has attracted wide attention from researchers. The number of targets and commands is a key indicator of how well BCIs can decode the brain's intentions. No studies have reported a BCI system with over 200 targets.Approach. This study developed the first high-speed BCI system with up to 216 targets that were encoded by a combination of electroencephalography features, including P300, motion visual evoked potential (mVEP), and steady-state visual evoked potential (SSVEP). Specifically, the hybrid BCI paradigm used the time-frequency division multiple access strategy to elaborately tag targets with P300 and mVEP of different time windows, along with SSVEP of different frequencies. The hybrid features were then decoded by task-discriminant component analysis and linear discriminant analysis. Ten subjects participated in the offline and online cued-guided spelling experiments. Other ten subjects took part in online free-spelling experiments.Main results.The offline results showed that the mVEP and P300 components were prominent in the central, parietal, and occipital regions, while the most distinct SSVEP feature was in the occipital region. The online cued-guided spelling and free-spelling results showed that the proposed BCI system achieved an average accuracy of 85.37% ± 7.49% and 86.00% ± 5.98% for the 216-target classification, resulting in an average information transfer rate (ITR) of 302.83 ± 39.20 bits min-1and 204.47 ± 37.56 bits min-1, respectively. Notably, the peak ITR could reach up to 367.83 bits min-1.Significance.This study developed the first high-speed BCI system with more than 200 targets, which holds promise for extending BCI's application scenarios.

A novel multiple time-frequency sequential coding strategy for hybrid brain-computer interface

Background

For brain-computer interface (BCI) communication, electroencephalography provides a preferable choice due to its high temporal resolution and portability over other neural recording techniques. However, current BCIs are unable to sufficiently use the information from time and frequency domains simultaneously. Thus, we proposed a novel hybrid time-frequency paradigm to investigate better ways of using the time and frequency information.

Method

We adopt multiple omitted stimulus potential (OSP) and steady-state motion visual evoked potential (SSMVEP) to design the hybrid paradigm. A series of pre-experiments were undertaken to study factors that would influence the feasibility of the hybrid paradigm and the interaction between multiple features. After that, a novel Multiple Time-Frequencies Sequential Coding (MTFSC) strategy was introduced and explored in experiments.

Results

Omissions with multiple short and long durations could effectively elicit time and frequency features, including the multi-OSP, ERP, and SSVEP in this hybrid paradigm. The MTFSC was feasible and efficient. The preliminary online analysis showed that the accuracy and the ITR of the nine-target stimulator over thirteen subjects were 89.04% and 36.37 bits/min.

Significance

This study first combined the SSMVEP and multi-OSP in a hybrid paradigm to produce robust and abundant time features for coding BCI. Meanwhile, the MTFSC proved feasible and showed great potential in improving performance, such as expanding the number of BCI targets by better using time information in specific stimulated frequencies. This study holds promise for designing better BCI systems with a novel coding method.

Instant classification for the spatially-coded BCI

The spatially-coded SSVEP BCI exploits changes in the topography of the steady-state visual evoked response to visual flicker stimulation in the extrafoveal field of view. In contrast to frequency-coded SSVEP BCIs, the operator does not gaze into any flickering lights; therefore, this paradigm can reduce visual fatigue. Other advantages include high classification accuracies and a simplified stimulation setup. Previous studies of the paradigm used stimulation intervals of a fixed duration. For frequency-coded SSVEP BCIs, it has been shown that dynamically adjusting the trial duration can increase the system’s information transfer rate (ITR). We therefore investigated whether a similar increase could be achieved for spatially-coded BCIs by applying dynamic stopping methods. To this end we introduced a new stopping criterion which combines the likelihood of the classification result and its stability across larger data windows. Whereas the BCI achieved an average ITR of 28.4±6.4 bits/min with fixed intervals, dynamic intervals increased the performance to 81.1±44.4 bits/min. Users were able to maintain performance up to 60 minutes of continuous operation. We suggest that the dynamic response time might have worked as a kind of temporal feedback which allowed operators to optimize their brain signals and compensate fatigue.

A Human-Machine Interface Based on an EOG and a Gyroscope for Humanoid Robot Control and Its Application to Home Services

The human-machine interface (HMI) has been studied for robot teleoperation with the aim of empowering people who experience motor disabilities to increase their interaction with the physical environment. The challenge of an HMI for robot control is to rapidly, accurately, and sufficiently produce control commands. In this paper, an asynchronous HMI based on an electrooculogram (EOG) and a gyroscope is proposed using two self-paced and endogenous features, double blink and head rotation. By designing the multilevel graphical user interface (GUI), the user can rotate his head to move the cursor of the GUI and create a double blink to trigger the button in the interface. The proposed HMI is able to supply sufficient commands at the same time with high accuracy (ACC) and low response time (RT). In the trigger task of sixteen healthy subjects, the target was clicked from 20 options with ACC of 99.2% and RT 2.34 s. Furthermore, a continuous strategy that uses motion start and motion stop commands to create a certain robot motion is proposed to control a humanoid robot based on the HMI. It avoids the situation that combines some commands to achieve one motion or converts the certain motion to a command directly. In the home service experiment, all subjects operated a humanoid robot changing the state of a switch, grasping a key, and putting it into a box. The time ratio between HMI control and manual control was 1.22, and the number of commands ratio was 1.18. The results demonstrated that the continuous strategy and proposed HMI can improve performance in humanoid robot control.

Effects of inter-stimulus intervals on concurrent P300 and SSVEP features for hybrid Brain-computer interfaces

BACKGROUND

Recently, we have implemented a high-speed brain-computer interface (BCI) system with a large instruction set using the concurrent P300 and steady-state visual evoked potential (SSVEP) features (also known as hybrid features). However, it remains unclear how to select inter-stimulus interval (ISI) for the proposed BCI system to balance the encoding efficiency and decoding performance.

NEW METHOD

This study developed a 6⁎9 hybrid P300-SSVEP BCI system and investigated a series of ISIs ranged from -175ms to 0ms with a step of 25ms. The influence of ISI on the hybrid features was analyzed from several aspects, including the amplitude of the induced features, classification accuracy, information transfer rate (ITR). Twelve naive subjects were recruited for the experiment.

RESULTS

The results showed the ISI factor had a significant impact on the hybrid features. Specifically, as the values of ISI decreased, the amplitudes of the induced features and accuracies decreased gradually, while the ITRs increased rapidly. It's achieved the highest ITR of 158.50 bits/min when ISI equal to -175ms.

COMPARISON WITH EXISTING METHOD

The optimal ISI in this study achieved superior performance in comparison with the one we used in the previous study.

CONCLUSIONS

The ISI can exert an important influence on the P300-SSVEP BCI system and its optimal value is -175ms in this study, which is significant for developing the high-speed BCI system with larger instruction sets in the future.

Evaluation of color modulation in visual P300-speller using new stimulus patterns

Objective The stimulus color of P300-BCI systems has been successfully modified. However, the effects of different color combinations have not been widely investigated. In this study, we designed new stimulus patterns to evaluate the influence of color modulation on the BCI performance and waveforms of the evoked related potential (ERP).Methods Comparison was performed for three new stimulus patterns consisting of red face and colored block-shape, namely, red face with a white rectangle (RFW), red face with a blue rectangle (RFB), and red face with a red rectangle (RFR). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis. Results The RFW pattern obtained the highest average online accuracy with 96.94%, and those of RFR and RFB patterns were 93.61% and of 92.22% respectively. Significant differences in online accuracy and information transfer rate (ITR) were found between RFW and RFR patterns (p < 0.05). Conclusion Compared with RFR and RFB patterns, RFW yielded the best performance in P300-BCI. These new stimulus patterns with different color combinations have considerable importance to BCI applications and user-friendliness.

A novel approach for designing authentication system using a picture based P300 speller

Due to great advances in the field of information technology, the need for a more reliable authentication system has been growing rapidly for protecting the individual or organizational assets from online frauds. In the past, many authentication techniques have been proposed like password and tokens but these techniques suffer from many shortcomings such as offline attacks (guessing) and theft. To overcome these shortcomings, in this paper brain signal based authentication system is proposed. A Brain-Computer Interface (BCI) is a tool that provides direct human-computer interaction by analyzing brain signals. In this study, a person authentication approach that can effectively recognize users by generating unique brain signal features in response to pictures of different objects is presented. This study focuses on a P300 BCI for authentication system design. Also, three classifiers were tested: Quadratic Discriminant Analysis (QDA), K-Nearest Neighbor, and Quadratic Support Vector Machine. The results showed that the proposed visual stimuli with pictures as selection attributes obtained significantly higher classification accuracies (97%) and information transfer rates (37.14 bits/min) as compared to the conventional paradigm. The best performance was observed with the QDA as compare to other classifiers. This method is advantageous for developing brain signal based authentication application as it cannot be forged (like Shoulder surfing) and can still be used for disabled users with a brain in good running condition. The results show that reduced matrix size and modified visual stimulus typically affects the accuracy and communication speed of P300 BCI performance.

A Single-Trial P300 Detector Based on Symbolized EEG and Autoencoded-(1D)CNN to Improve ITR Performance in BCIs

In this paper, we propose a breakthrough single-trial P300 detector that maximizes the information translate rate (ITR) of the brain–computer interface (BCI), keeping high recognition accuracy performance. The architecture, designed to improve the portability of the algorithm, demonstrated full implementability on a dedicated embedded platform. The proposed P300 detector is based on the combination of a novel pre-processing stage based on the EEG signals symbolization and an autoencoded convolutional neural network (CNN). The proposed system acquires data from only six EEG channels; thus, it treats them with a low-complexity preprocessing stage including baseline correction, windsorizing and symbolization. The symbolized EEG signals are then sent to an autoencoder model to emphasize those temporal features that can be meaningful for the following CNN stage. This latter consists of a seven-layer CNN, including a 1D convolutional layer and three dense ones. Two datasets have been analyzed to assess the algorithm performance: one from a P300 speller application in BCI competition III data and one from self-collected data during a fluid prototype car driving experiment. Experimental results on the P300 speller dataset showed that the proposed method achieves an average ITR (on two subjects) of 16.83 bits/min, outperforming by +5.75 bits/min the state-of-the-art for this parameter. Jointly with the speed increase, the recognition performance returned disruptive results in terms of the harmonic mean of precision and recall (F1-Score), which achieve 51.78 ± 6.24%. The same method used in the prototype car driving led to an ITR of ~33 bit/min with an F1-Score of 70.00% in a single-trial P300 detection context, allowing fluid usage of the BCI for driving purposes. The realized network has been validated on an STM32L4 microcontroller target, for complexity and implementation assessment. The implementation showed an overall resource occupation of 5.57% of the total available ROM, ~3% of the available RAM, requiring less than 3.5 ms to provide the classification outcome.

Enhancement for P300-speller classification using multi-window discriminative canonical pattern matching

Objective.P300s are one of the most studied event-related potentials (ERPs), which have been widely used for brain-computer interfaces (BCIs). Thus, fast and accurate recognition of P300s is an important issue for BCI study. Recently, there emerges a lot of novel classification algorithms for P300-speller. Among them, discriminative canonical pattern matching (DCPM) has been proven to work effectively, in which discriminative spatial pattern (DSP) filter can significantly enhance the spatial features of P300s. However, the pattern of ERPs in space varies with time, which was not taken into consideration in the traditional DCPM algorithm.Approach.In this study, we developed an advanced version of DCPM, i.e. multi-window DCPM, which contained a series of time-dependent DSP filters to fine-tune the extraction of spatial ERP features. To verify its effectiveness, 25 subjects were recruited and they were asked to conduct the typical P300-speller experiment.Main results.As a result, multi-window DCPM achieved the character recognition accuracy of 91.84% with only five training characters, which was significantly better than the traditional DCPM algorithm. Furthermore, it was also compared with eight other popular methods, including SWLDA, SKLDA, STDA, BLDA, xDAWN, HDCA, sHDCA and EEGNet. The results showed multi-window DCPM preformed the best, especially using a small calibration dataset. The proposed algorithm was applied to the BCI Controlled Robot Contest of P300 paradigm in 2019 World Robot Conference, and won the first place.Significance.These results demonstrate that multi-window DCPM is a promising method for improving the performance and enhancing the practicability of P300-speller.

Effects of Skin Friction on Tactile P300 Brain-Computer Interface Performance

Tactile perception, the primary sensing channel of the tactile brain-computer interface (BCI), is a complicated process. Skin friction plays a vital role in tactile perception. This study aimed to examine the effects of skin friction on tactile P300 BCI performance. Two kinds of oddball paradigms were designed, silk-stim paradigm (SSP) and linen-stim paradigm (LSP), in which silk and linen were wrapped on target vibration motors, respectively. In both paradigms, the disturbance vibrators were wrapped in cotton. The experimental results showed that LSP could induce stronger event-related potentials (ERPs) and achieved a higher classification accuracy and information transfer rate (ITR) compared with SSP. The findings indicate that high skin friction can achieve high performance in tactile BCI. This work provides a novel research direction and constitutes a viable basis for the future tactile P300 BCI, which may benefit patients with visual impairments.

The Influence of Visual Attention on The Performance of A Novel Tactile P300 Brain-Computer Interface with Cheeks-Stim Paradigm

Tactile P300 brain-computer interface (BCI) generally has a worse accuracy and information transfer rate (ITR) than the visual-based BCI. It may be due to the fact that human beings have a relatively poor tactile perception. This study investigated the influence of visual attention on the performance of a tactile P300 BCI. We designed our paradigms based on a novel cheeks-stim paradigm which attached the stimulators on the subject's cheeks. Two paradigms were designed as follows: a paradigm with no visual attention and another paradigm with visual attention to the target position. Eleven subjects were invited to perform the two paradigms. We also recorded and analyzed the eyeball movement data during the paradigm with visual attention to explore whether the eyeball movement would have an effect on the BCI classification. The average online accuracy was 89.09% for the paradigm with visual attention, which was significantly higher than that of the paradigm with no visual attention (70.45%). Significant difference in ITR was also found between the two paradigms ([Formula: see text]). The results demonstrated that visual attention was an effective method to improve the performance of tactile P300 BCI. Our findings suggested that it may be feasible to complete an efficient tactile BCI system by adding visual attention.

Novel channel selection method based on position priori weighted permutation entropy and binary gravity search algorithm

Brain-computer interface (BCI) system based on motor imagery (MI) usually adopts multichannel Electroencephalograph (EEG) signal recording method. However, EEG signals recorded in multi-channel mode usually contain many redundant and artifact information. Therefore, selecting a few effective channels from whole channels may be a means to improve the performance of MI-based BCI systems. We proposed a channel evaluation parameter called position priori weight-permutation entropy (PPWPE), which include amplitude information and position information of a channel. According to the order of PPWPE values, we initially selected half of the channels with large PPWPE value from all sampling electrode channels. Then, the binary gravitational search algorithm (BGSA) was used in searching a channel combination that will be used in determining an optimal channel combination. The features were extracted by common spatial pattern (CSP) method from the final selected channels, and the classifier was trained by support vector machine. The PPWPE + BGSA + CSP channel selection method is validated on two data sets. Results showed that the PPWPE + BGSA + CSP method obtained better mean classification accuracy (88.0% vs. 57.5% for Data set 1 and 91.1% vs. 79.4% for Data set 2) than All-C + CSP method. The PPWPE + BGSA + CSP method can achieve higher classification in fewer channels selected. This method has great potential to improve the performance of MI-based BCI systems.

Bacomics: a comprehensive cross area originating in the studies of various brain-apparatus conversations

The brain is the most important organ of the human body, and the conversations between the brain and an apparatus can not only reveal a normally functioning or a dysfunctional brain but also can modulate the brain. Here, the apparatus may be a nonbiological instrument, such as a computer, and the consequent brain-computer interface is now a very popular research area with various applications. The apparatus may also be a biological organ or system, such as the gut and muscle, and their efficient conversations with the brain are vital for a healthy life. Are there any common bases that bind these different scenarios? Here, we propose a new comprehensive cross area: Bacomics, which comes from brain-apparatus conversations (BAC) + omics. We take Bacomics to cover at least three situations: (1) The brain is normal, but the conversation channel is disabled, as in amyotrophic lateral sclerosis. The task is to reconstruct or open up new channels to reactivate the brain function. (2) The brain is in disorder, such as in Parkinson's disease, and the work is to utilize existing or open up new channels to intervene, repair and modulate the brain by medications or stimulation. (3) Both the brain and channels are in order, and the goal is to enhance coordinated development between the brain and apparatus. In this paper, we elaborate the connotation of BAC into three aspects according to the information flow: the issue of output to the outside (BAC-1), the issue of input to the brain (BAC-2) and the issue of unity of brain and apparatus (BAC-3). More importantly, there are no less than five principles that may be taken as the cornerstones of Bacomics, such as feedforward and feedback control, brain plasticity, harmony, the unity of opposites and systems principles. Clearly, Bacomics integrates these seemingly disparate domains, but more importantly, opens a much wider door for the research and development of the brain, and the principles further provide the general framework in which to realize or optimize these various conversations.

Effects of Visual Attention on Tactile P300 BCI

Objective. Tactile P300 brain-computer interfaces (BCIs) can be manipulated by users who only need to focus their attention on a single-target stimulus within a stream of tactile stimuli. To date, a multitude of tactile P300 BCIs have been proposed. In this study, our main purpose is to explore and investigate the effects of visual attention on a tactile P300 BCI. Approach. We designed a conventional tactile P300 BCI where vibration stimuli were provided by five stimulators and two of them were fixed on target locations on the participant's left and right wrists. Two conditions (one condition with visual attention and the other condition without visual attention) were tested by eleven healthy participants. Main Results. Our results showed that, when participants visually attended to the location of target stimulus, significantly higher classification accuracies and information transfer rates were obtained (both for p < 0.05). Furthermore, participants reported that visually attending to the stimulus made it easier to identify the target stimulus in random sequences of vibration stimuli. Significance. These findings suggest that visual attention has positive effects on both tactile P300 BCI performance and user-evaluation.

Comparison of the ERP-Based BCI Performance Among Chromatic (RGB) Semitransparent Face Patterns

Objective

Previous studies have shown that combing with color properties may be used as part of the display presented to BCI users in order to improve performance. Build on this, we explored the effects of combinations of face stimuli with three primary colors (RGB) on BCI performance which is assessed by classification accuracy and information transfer rate (ITR). Furthermore, we analyzed the waveforms of three patterns.

Methods

We compared three patterns in which semitransparent face is overlaid three primary colors as stimuli: red semitransparent face (RSF), green semitransparent face (GSF), and blue semitransparent face (BSF). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. In addition, a Repeated-measures ANOVA (RM-ANOVA) and Bonferroni correction were chosen for statistical analysis.

Results

The results indicated that the RSF pattern achieved the highest online averaged accuracy with 93.89%, followed by the GSF pattern with 87.78%, while the lowest performance was caused by the BSF pattern with an accuracy of 81.39%. Furthermore, significant differences in classification accuracy and ITR were found between RSF and GSF (p < 0.05) and between RSF and BSF patterns (p < 0.05).

Conclusion

The semitransparent faces colored red (RSF) pattern yielded the best performance of the three patterns. The proposed patterns based on ERP-BCI system have a clinically significant impact by increasing communication speed and accuracy of the P300-speller for patients with severe motor impairment.

Adaptive Time Segment Analysis for Steady-State Visual Evoked Potential Based Brain-Computer Interfaces

The research in non-invasive Brain-Computer Interface (BCI) has led to significant improvements in the recent years for potential end users. However, the user experience and the BCI illiteracy problem remains challenging areas to address for obtaining robust and resilient clinical applications. In this study, we address the choice of the time segment for the detection of steady state visual evoked potential (SSVEP) detection. This problem has been widely addressed for the detection of event-related potentials compared to SSVEP based BCIs. The choice of this parameter is typically fixed and has a direct influence on both the detection accuracy and the information transfer rate. We propose to shift the problem of the time segment to the choice of the threshold for determining if a response has been properly detected. We consider two open-datasets for benchmarking the rationale of the approach. The results support the conclusion that an adaptive time segment for each trial, based on the selection of a threshold, can lead to a substantial higher ITR (86.92 bits/min), compared to the time segment chosen at the user (79.56 bits/min) or group level (73.78 bits/min). Finally, the results suggest that the threshold could be determined automatically in relation to the number of classes. Such an approach can leverage the literacy of SSVEP based BCI.

Novel hybrid brain-computer interface system based on motor imagery and P300

Motor imagery (MI) is a mental representation of motor behavior and has been widely used in electroencephalogram based brain-computer interfaces (BCIs). Several studies have demonstrated the efficacy of MI-based BCI-feedback training in post-stroke rehabilitation. However, in the earliest stage of the training, calibration data typically contain insufficient discriminability, resulting in unreliable feedback, which may decrease subjects' motivation and even hinder their training. To improve the performance in the early stages of MI training, a novel hybrid BCI paradigm based on MI and P300 is proposed in this study. In this paradigm, subjects are instructed to imagine writing the Chinese character following the flash order of the desired Chinese character displayed on the screen. The event-related desynchronization/synchronization (ERD/ERS) phenomenon is produced with writing based on one's imagination. Simultaneously, the P300 potential is evoked by the flash of each stroke. Moreover, a fusion method of P300 and MI classification is proposed, in which unreliable P300 classifications are corrected by reliable MI classifications. Twelve healthy naïve MI subjects participated in this study. Results demonstrated that the proposed hybrid BCI paradigm yielded significantly better performance than the single-modality BCI paradigm. The recognition accuracy of the fusion method is significantly higher than that of P300 (p < 0.05) and MI (p < 0.01). Moreover, the training data size can be reduced through fusion of these two modalities.

A New Early Stopping Method for P300 Spellers

In event-related potentials based brain-computer interfaces, the responses evoked by a well defined stimuli sequence are usually averaged to overcome the limitations caused by the intrinsic poor EEG signal-to-noise ratio. This, however, implies that the time necessary to detect the brain signals increases and then that the communication rate can be dramatically reduced. A common approach is then at first to estimate an optimal fixed number of responses to be averaged on a calibration data set and then to use this number on the online/testing dataset. In contrast to this strategy, several early stopping methods have been successfully proposed, aiming at dynamically stopping the stimulation sequence when a certain condition is met. We propose an efficient and easy to implement early stopping method that outperforms the ones proposed in the literature, showing its effectiveness on several publicly available datasets recorded from either healthy subjects or amyotrophic lateral sclerosis patients.

Two-Stage Frequency Recognition Method Based on Correlated Component Analysis for SSVEP-Based BCI

A canonical correlation analysis (CCA) is a state-of-the-art method for frequency recognition in steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems. Various extended methods have been developed, and among such methods, a combination method of CCA and individual-template-based CCA has achieved the best performance. However, the CCA requires the canonical vectors to be orthogonal, which may not be a reasonable assumption for the EEG analysis. In this paper, we propose using the correlated component analysis (CORRCA) rather than CCA to implement frequency recognition. CORRCA can relax the constraint of canonical vectors in CCA and generate the same projection vector for two multichannel EEG signals. Furthermore, we propose a two-stage method based on the basic CORRCA method (termed TSCORRCA). Evaluated on a benchmark data set of 35 subjects, the experimental results demonstrate that CORRCA significantly outperformed CCA, and TSCORRCA obtained the best performance among the compared methods. This paper demonstrates that CORRCA-based methods have a great potential for implementing high-performance SSVEP-based BCI systems.

Effect of a combination of flip and zooming stimuli on the performance of a visual brain-computer interface for spelling

Brain-computer interface (BCI) systems can allow their users to communicate with the external world by recognizing intention directly from their brain activity without the assistance of the peripheral motor nervous system. The P300-speller is one of the most widely used visual BCI applications. In previous studies, a flip stimulus (rotating the background area of the character) that was based on apparent motion, suffered from less refractory effects. However, its performance was not improved significantly. In addition, a presentation paradigm that used a "zooming" action (changing the size of the symbol) has been shown to evoke relatively higher P300 amplitudes and obtain a better BCI performance. To extend this method of stimuli presentation within a BCI and, consequently, to improve BCI performance, we present a new paradigm combining both the flip stimulus with a zooming action. This new presentation modality allowed BCI users to focus their attention more easily. We investigated whether such an action could combine the advantages of both types of stimuli presentation to bring a significant improvement in performance compared to the conventional flip stimulus. The experimental results showed that the proposed paradigm could obtain significantly higher classification accuracies and bit rates than the conventional flip paradigm (p<0.01).

An ERP-based BCI with peripheral stimuli: validation with ALS patients

Many studies reported that ERP-based BCIs can provide communication for some people with amyotrophic lateral sclerosis (ALS). ERP-based BCIs often present characters within a matrix that occupies the center of the visual field. However, several studies have identified some concerns with the matrix-based approach. This approach may lead to fatigue and errors resulting from flashing adjacent stimuli, and is impractical for users who might want to use the BCI in tandem with other software or feedback in the center of the monitor. In this paper, we introduce and validate an alternate ERP-based BCI display approach. By presenting stimuli near the periphery of the display, we reduce the adjacency problem and leave the center of the display available for feedback or other applications. Two ERP-based display approaches were tested on 18 ALS patients to: (1) compare performance between a conventional matrix speller paradigm (Matrix-P, mean visual angle 6°) and a new speller paradigm with peripherally distributed stimuli (Peripheral-P, mean visual angle 8.8°); and (2) assess performance while spelling 42 characters online continuously, without a break. In the Peripheral-P condition, 12 subjects attained higher than 80% feedback accuracy during online performance, and 7 of these subjects obtained higher than 90% accuracy. The experimental results showed that the Peripheral-P condition yielded performance comparable to the conventional Matrix-P condition (p > 0.05) in accuracy and information transfer rate. This paper introduces a new display approach that leaves the center of the monitor open for feedback and/or other display elements, such as movies, games, art, or displays from other AAC software or conventional software tools.

Correlated Component Analysis for Enhancing the Performance of SSVEP-Based Brain-Computer Interface

A new method for steady-state visual evoked potentials (SSVEPs) frequency recognition is proposed to enhance the performance of SSVEP-based brain-computer interface (BCI). Correlated component analysis (CORCA) is introduced, which originally was designed to find linear combinations of electrodes that are consistent across subjects and maximally correlated between them. We propose a CORCA algorithm to learn spatial filters with multiple blocks of individual training data for SSVEP-based BCI scenario. The spatial filters are used to remove background noises by combining the multichannel electroencephalogram signals. We conduct a comparison between the proposed CORCA-based and the task-related component analysis (TRCA) based methods using a 40-class SSVEP benchmark data set recorded from 35 subjects. Our experimental study validates the efficiency of the CORCA-based method, and the extensive comparison results indicate that the CORCA-based method significantly outperforms the TRCA-based method. Superior performance demonstrates that the proposed method holds the promising potential to achieve satisfactory performance for SSVEP-based BCI with a large number of targets.

An Optimized Channel Selection Method Based on Multifrequency CSP-Rank for Motor Imagery-Based BCI System

Background

Due to the redundant information contained in multichannel electroencephalogram (EEG) signals, the classification accuracy of brain-computer interface (BCI) systems may deteriorate to a large extent. Channel selection methods can help to remove task-independent electroencephalogram (EEG) signals and hence improve the performance of BCI systems. However, in different frequency bands, brain areas associated with motor imagery are not exactly the same, which will result in the inability of traditional channel selection methods to extract effective EEG features.

New Method

To address the above problem, this paper proposes a novel method based on common spatial pattern- (CSP-) rank channel selection for multifrequency band EEG (CSP-R-MF). It combines the multiband signal decomposition filtering and the CSP-rank channel selection methods to select significant channels, and then linear discriminant analysis (LDA) was used to calculate the classification accuracy.

Results

The results showed that our proposed CSP-R-MF method could significantly improve the average classification accuracy compared with the CSP-rank channel selection method.

Investigation of Visual Stimulus With Various Colors and the Layout for the Oddball Paradigm in Evoked Related Potential-Based Brain-Computer Interface

Objective: Stimulus visual patterns, such as size, content, color, luminosity, and interval, play key roles for brain–computer interface (BCI) performance. However, the three primary colors to be intercompared as a single variable or factor on the same platform are poorly studied. In this work, we configured the visual stimulus patterns with red, green, and blue operating on a newly designed layout of the flash pattern of BCI to study the waveforms and performance of the evoked related potential (ERP).

Approach: Twelve subjects participated in our experiment, and each subject was required to finish three different color sub-experiments. Four blocks of the interface were presented along the edge of the screen, and the other four were assembled in the center, aiming to investigate the problem of adjacency distraction. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis.

Main results: The averaged online accuracy was 98.44% for the red paradigm, higher than 92.71% for the green paradigm, and 93.23% for the blue paradigm. Furthermore, significant differences in online accuracy (p < 0.05) and information transfer rate (p < 0.05) were found between the red and green paradigms.

Significance: The red stimulus paradigm yielded the best performance. The proposed design of ERP-based BCI was practical and effective for many potential applications.

Subject-Independent ERP-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) are desirable for people to express their thoughts, especially those with profound disabilities in communication. The classification of brain patterns for each different subject requires an extensively time-consuming learning stage specific to that person, in order to reach satisfactory accuracy performance. The training session could also be infeasible for disabled patients as they may not fully understand the training instructions. In this paper, we propose a unified classification scheme based on ensemble classifier, dynamic stopping, and adaptive learning. We apply this scheme on the P300-based BCI, with the subject-independent manner, where no learning session is required for new experimental users. According to our theoretical analysis and empirical results, the harmonized integration of these three methods can significantly boost up the average accuracy from 75.00% to 91.26%, while at the same time reduce the average spelling time from 12.62 to 6.78 iterations, approximately to two-fold faster. The experiments were conducted on a large public dataset which had been used in other related studies. Direct comparisons between our work with the others' are also reported in details.

Classification for Single-Trial N170 During Responding to Facial Picture With Emotion

Whether an event-related potential (ERP), N170, related to facial recognition was modulated by emotion has always been a controversial issue. Some researchers considered the N170 to be independent of emotion, whereas a recent study has shown the opposite view. In the current study, electroencephalogram (EEG) recordings while responding to facial pictures with emotion were utilized to investigate whether the N170 was modulated by emotion. We found that there was a significant difference between ERP trials with positive and negative emotions of around 170 ms at the occipitotemporal electrodes (i.e., N170). Then, we further proposed the application of the single-trial N170 as a feature for the classification of facial emotion, which could avoid the fact that ERPs were obtained by averaging most of the time while ignoring the trial-to-trial variation. In order to find an optimal classifier for emotional classification with single-trial N170 as a feature, three types of classifiers, namely, linear discriminant analysis (LDA), L1-regularized logistic regression (L1LR), and support vector machine with radial basis function (RBF-SVM), were comparatively investigated. The results showed that the single-trial N170 could be used as a classification feature to successfully distinguish positive emotion from negative emotion. L1-regularized logistic regression classifiers showed a good generalization, whereas LDA showed a relatively poor generalization. Moreover, when compared with L1LR, the RBF-SVM required more time to optimize the parameters during the classification, which became an obstacle while applying it to the online operating system of brain-computer interfaces (BCIs). The findings suggested that face-related N170 could be affected by facial expression and that the single-trial N170 could be a biomarker used to monitor the emotional states of subjects for the BCI domain.

A Study of the Effects of Electrode Number and Decoding Algorithm on Online EEG-Based BCI Behavioral Performance

Motor imagery–based brain–computer interface (BCI) using electroencephalography (EEG) has demonstrated promising applications by directly decoding users' movement related mental intention. The selection of control signals, e.g., the channel configuration and decoding algorithm, plays a vital role in the online performance and progressing of BCI control. While several offline analyses report the effect of these factors on BCI accuracy for a single session—performance increases asymptotically by increasing the number of channels, saturates, and then decreases—no online study, to the best of our knowledge, has yet been performed to compare for a single session or across training. The purpose of the current study is to assess, in a group of forty-five subjects, the effect of channel number and decoding method on the progression of BCI performance across multiple training sessions and the corresponding neurophysiological changes. The 45 subjects were divided into three groups using Laplacian Filtering (LAP/S) with nine channels, Common Spatial Pattern (CSP/L) with 40 channels and CSP (CSP/S) with nine channels for online decoding. At the first training session, subjects using CSP/L displayed no significant difference compared to CSP/S but a higher average BCI performance over those using LAP/S. Despite the average performance when using the LAP/S method was initially lower, but LAP/S displayed improvement over first three sessions, whereas the other two groups did not. Additionally, analysis of the recorded EEG during BCI control indicates that the LAP/S produces control signals that are more strongly correlated with the target location and a higher R-square value was shown at the fifth session. In the present study, we found that subjects' average online BCI performance using a large EEG montage does not show significantly better performance after the first session than a smaller montage comprised of a common subset of these electrodes. The LAP/S method with a small EEG montage allowed the subjects to improve their skills across sessions, but no improvement was shown for the CSP method.

"Enheduanna-A Manifesto of Falling" Live Brain-Computer Cinema Performance: Performer and Audience Participation, Cognition and Emotional Engagement Using Multi-Brain BCI Interaction

The fields of neural prosthetic technologies and Brain-Computer Interfaces (BCIs) have witnessed in the past 15 years an unprecedented development, bringing together theories and methods from different scientific fields, digital media, and the arts. More in particular, artists have been amongst the pioneers of the design of relevant applications since their emergence in the 1960s, pushing the boundaries of applications in real-life contexts. With the new research, advancements, and since 2007, the new low-cost commercial-grade wireless devices, there is a new increasing number of computer games, interactive installations, and performances that involve the use of these interfaces, combining scientific, and creative methodologies. The vast majority of these works use the brain-activity of a single participant. However, earlier, as well as recent examples, involve the simultaneous interaction of more than one participants or performers with the use of Electroencephalography (EEG)-based multi-brain BCIs. In this frame, we discuss and evaluate “Enheduanna—A Manifesto of Falling,” a live brain-computer cinema performance that enables for the first time the simultaneous real-time multi-brain interaction of more than two participants, including a performer and members of the audience, using a passive EEG-based BCI system in the context of a mixed-media performance. The performance was realised as a neuroscientific study conducted in a real-life setting. The raw EEG data of seven participants, one performer and two different members of the audience for each performance, were simultaneously recorded during three live events. The results reveal that the majority of the participants were able to successfully identify whether their brain-activity was interacting with the live video projections or not. A correlation has been found between their answers to the questionnaires, the elements of the performance that they identified as most special, and the audience's indicators of attention and emotional engagement. Also, the results obtained from the performer's data analysis are consistent with the recall of working memory representations and the increase of cognitive load. Thus, these results prove the efficiency of the interaction design, as well as the importance of the directing strategy, dramaturgy and narrative structure on the audience's perception, cognitive state, and engagement.

Towards correlation-based time window selection method for motor imagery BCIs

The start of the cue is often used to initiate the feature window used to control motor imagery (MI)-based brain-computer interface (BCI) systems. However, the time latency during an MI period varies between trials for each participant. Fixing the starting time point of MI features can lead to decreased system performance in MI-based BCI systems. To address this issue, we propose a novel correlation-based time window selection (CTWS) algorithm for MI-based BCIs. Specifically, the optimized reference signals for each class were selected based on correlation analysis and performance evaluation. Furthermore, the starting points of time windows for both training and testing samples were adjusted using correlation analysis. Finally, the feature extraction and classification algorithms were used to calculate the classification accuracy. With two datasets, the results demonstrate that the CTWS algorithm significantly improved the system performance when compared to directly using feature extraction approaches. Importantly, the average improvement in accuracy of the CTWS algorithm on the datasets of healthy participants and stroke patients was 16.72% and 5.24%, respectively when compared to traditional common spatial pattern (CSP) algorithm. In addition, the average accuracy increased 7.36% and 9.29%, respectively when the CTWS was used in conjunction with Sub-Alpha-Beta Log-Det Divergences (Sub-ABLD) algorithm. These findings suggest that the proposed CTWS algorithm holds promise as a general feature extraction approach for MI-based BCIs.

An Adaptive Calibration Framework for mVEP-Based Brain-Computer Interface

Electroencephalogram signals and the states of subjects are nonstationary. To track changing states effectively, an adaptive calibration framework is proposed for the brain-computer interface (BCI) with the motion-onset visual evoked potential (mVEP) as the control signal. The core of this framework is to update the training set adaptively for classifier training. The updating procedure consists of two operations, that is, adding new samples to the training set and removing old samples from the training set. In the proposed framework, a support vector machine (SVM) and fuzzy C-mean clustering (fCM) are combined to select the reliable samples for the training set from the blocks close to the current blocks to be classified. Because of the complementary information provided by SVM and fCM, they can guarantee the reliability of information fed into classifier training. The removing procedure will aim to remove those old samples recorded a relatively long time before current new blocks. These two operations could yield a new training set, which could be used to calibrate the classifier to track the changing state of the subjects. Experimental results demonstrate that the adaptive calibration framework is effective and efficient and it could improve the performance of online BCI systems.

Usage of drip drops as stimuli in an auditory P300 BCI paradigm

Recently, many auditory BCIs are using beeps as auditory stimuli, while beeps sound unnatural and unpleasant for some people. It is proved that natural sounds make people feel comfortable, decrease fatigue, and improve the performance of auditory BCI systems. Drip drop is a kind of natural sounds that makes humans feel relaxed and comfortable. In this work, three kinds of drip drops were used as stimuli in an auditory-based BCI system to improve the user-friendness of the system. This study explored whether drip drops could be used as stimuli in the auditory BCI system. The auditory BCI paradigm with drip-drop stimuli, which was called the drip-drop paradigm (DP), was compared with the auditory paradigm with beep stimuli, also known as the beep paradigm (BP), in items of event-related potential amplitudes, online accuracies and scores on the likability and difficulty to demonstrate the advantages of DP. DP obtained significantly higher online accuracy and information transfer rate than the BP (p < 0.05, Wilcoxon signed test; p < 0.05, Wilcoxon signed test). Besides, DP obtained higher scores on the likability with no significant difference on the difficulty (p < 0.05, Wilcoxon signed test). The results showed that the drip drops were reliable acoustic materials as stimuli in an auditory BCI system.

Index finger motor imagery EEG pattern recognition in BCI applications using dictionary cleaned sparse representation-based classification for healthy people

Electroencephalogram (EEG)-based motor imagery (MI) brain-computer interface (BCI) has shown its effectiveness for the control of rehabilitation devices designed for large body parts of the patients with neurologic impairments. In order to validate the feasibility of using EEG to decode the MI of a single index finger and constructing a BCI-enhanced finger rehabilitation system, we collected EEG data during right hand index finger MI and rest state for five healthy subjects and proposed a pattern recognition approach for classifying these two mental states. First, Fisher's linear discriminant criteria and power spectral density analysis were used to analyze the event-related desynchronization patterns. Second, both band power and approximate entropy were extracted as features. Third, aiming to eliminate the abnormal samples in the dictionary and improve the classification performance of the conventional sparse representation-based classification (SRC) method, we proposed a novel dictionary cleaned sparse representation-based classification (DCSRC) method for final classification. The experimental results show that the proposed DCSRC method gives better classification accuracies than SRC and an average classification accuracy of 81.32% is obtained for five subjects. Thus, it is demonstrated that single right hand index finger MI can be decoded from the sensorimotor rhythms, and the feature patterns of index finger MI and rest state can be well recognized for robotic exoskeleton initiation.

An EOG-Based Human-Machine Interface for Wheelchair Control

OBJECTIVE

Nonmanual human-machine interfaces (HMIs) have been studied for wheelchair control with the aim of helping severely paralyzed individuals regain some mobility. The challenge is to rapidly, accurately, and sufficiently produce control commands, such as left and right turns, forward and backward motions, acceleration, deceleration, and stopping. In this paper, a novel electrooculogram (EOG) based HMI is proposed for wheelchair control.

METHODS

A total of 13 flashing buttons, each of which corresponds to a command, are presented in the graphical user interface. These buttons flash on a one-by-one manner in a predefined sequence. The user can select a button by blinking in sync with its flashes. The algorithm detects the eye blinks from a channel of vertical EOG data and determines the user's target button based on the synchronization between the detected blinks and the button's flashes.

RESULTS

For healthy subjects/patients with spinal cord injuries, the proposed HMI achieved an average accuracy of 96.7% / 91.7% and a response time of 3.53 s/3.67 s with 0 false positive rates (FPRs).

CONCLUSION

Using one channel of vertical EOG signals associated with eye blinks, the proposed HMI can accurately provide sufficient commands with a satisfactory response time.

SIGNIFICANCE

The proposed HMI provides a novel nonmanual approach for severely paralyzed individuals to control a wheelchair. Compared with a newly established EOG-based HMI, the proposed HMI can generate more commands with higher accuracy, lower FPR, and fewer electrodes.

Enhancing Classification Performance of Functional Near-Infrared Spectroscopy- Brain-Computer Interface Using Adaptive Estimation of General Linear Model Coefficients

In this paper, a novel methodology for enhanced classification of functional near-infrared spectroscopy (fNIRS) signals utilizable in a two-class [motor imagery (MI) and rest; mental rotation (MR) and rest] brain–computer interface (BCI) is presented. First, fNIRS signals corresponding to MI and MR are acquired from the motor and prefrontal cortex, respectively, afterward, filtered to remove physiological noises. Then, the signals are modeled using the general linear model, the coefficients of which are adaptively estimated using the least squares technique. Subsequently, multiple feature combinations of estimated coefficients were used for classification. The best classification accuracies achieved for five subjects, for MI versus rest are 79.5, 83.7, 82.6, 81.4, and 84.1% whereas those for MR versus rest are 85.5, 85.2, 87.8, 83.7, and 84.8%, respectively, using support vector machine. These results are compared with the best classification accuracies obtained using the conventional hemodynamic response. By means of the proposed methodology, the average classification accuracy obtained was significantly higher (p < 0.05). These results serve to demonstrate the feasibility of developing a high-classification-performance fNIRS-BCI.

A Synchronous Motor Imagery Based Neural Physiological Paradigm for Brain Computer Interface Speller

Brain Computer Interface (BCI) speller is a typical BCI-based application to help paralyzed patients express their thoughts. This paper proposed a novel motor imagery based BCI speller with Oct-o-spell paradigm for word input. Furthermore, an intelligent input method was used for improving the performance of the BCI speller. For the English word spelling experiment, we compared synchronous control with previous asynchronous control under the same experimental condition. There were no significant differences between these two control methods in the classification accuracy, information transmission rate (ITR) or letters per minute (LPM). And the accuracy rates of over 70% validated the feasibility for these two control strategies. It was indicated that MI-based synchronous control protocol was feasible for BCI speller. And the efficiency of the predictive text entry (PTE) mode was superior to that of the Non-PTE mode.

Comparison of the BCI Performance between the Semitransparent Face Pattern and the Traditional Face Pattern

Brain-computer interface (BCI) systems allow users to communicate with the external world by recognizing the brain activity without the assistance of the peripheral motor nervous system. P300-based BCI is one of the most common used BCI systems that can obtain high classification accuracy and information transfer rate (ITR). Face stimuli can result in large event-related potentials and improve the performance of P300-based BCI. However, previous studies on face stimuli focused mainly on the effect of various face types (i.e., face expression, face familiarity, and multifaces) on the BCI performance. Studies on the influence of face transparency differences are scarce. Therefore, we investigated the effect of semitransparent face pattern (STF-P) (the subject could see the target character when the stimuli were flashed) and traditional face pattern (F-P) (the subject could not see the target character when the stimuli were flashed) on the BCI performance from the transparency perspective. Results showed that STF-P obtained significantly higher classification accuracy and ITR than those of F-P (p < 0.05).

Progress in EEG-Based Brain Robot Interaction Systems

The most popular noninvasive Brain Robot Interaction (BRI) technology uses the electroencephalogram- (EEG-) based Brain Computer Interface (BCI), to serve as an additional communication channel, for robot control via brainwaves. This technology is promising for elderly or disabled patient assistance with daily life. The key issue of a BRI system is to identify human mental activities, by decoding brainwaves, acquired with an EEG device. Compared with other BCI applications, such as word speller, the development of these applications may be more challenging since control of robot systems via brainwaves must consider surrounding environment feedback in real-time, robot mechanical kinematics, and dynamics, as well as robot control architecture and behavior. This article reviews the major techniques needed for developing BRI systems. In this review article, we first briefly introduce the background and development of mind-controlled robot technologies. Second, we discuss the EEG-based brain signal models with respect to generating principles, evoking mechanisms, and experimental paradigms. Subsequently, we review in detail commonly used methods for decoding brain signals, namely, preprocessing, feature extraction, and feature classification, and summarize several typical application examples. Next, we describe a few BRI applications, including wheelchairs, manipulators, drones, and humanoid robots with respect to synchronous and asynchronous BCI-based techniques. Finally, we address some existing problems and challenges with future BRI techniques.

Optimized Motor Imagery Paradigm Based on Imagining Chinese Characters Writing Movement

BACKGROUND

motor imagery (MI) is a mental representation of motor behavior. The MI-based brain computer interfaces (BCIs) can provide communication for the physically impaired. The performance of MI-based BCI mainly depends on the subject's ability to self-modulate electroencephalogram signals. Proper training can help naive subjects learn to modulate brain activity proficiently. However, training subjects typically involve abstract motor tasks and are time-consuming.

METHODS

to improve the performance of naive subjects during motor imagery, a novel paradigm was presented that would guide naive subjects to modulate brain activity effectively. In this new paradigm, pictures of the left or right hand were used as cues for subjects to finish the motor imagery task. Fourteen healthy subjects (11 male, aged 22-25 years, and mean 23.6±1.16) participated in this study. The task was to imagine writing a Chinese character. Specifically, subjects could imagine hand movements corresponding to the sequence of writing strokes in the Chinese character. This paradigm was meant to find an effective and familiar action for most Chinese people, to provide them with a specific, extensively practiced task and help them modulate brain activity.

RESULTS

results showed that the writing task paradigm yielded significantly better performance than the traditional arrow paradigm (p < 0.001). Questionnaire replies indicated that most subjects thought that the new paradigm was easier.

CONCLUSION

the proposed new motor imagery paradigm could guide subjects to help them modulate brain activity effectively. Results showed that there were significant improvements using new paradigm, both in classification accuracy and usability.