Pushing the P300-based brain–computer interface beyond 100 bpm: extending performance guided constraints into the temporal domain

Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.

A high-speed hybrid brain-computer interface with more than 200 targets

Objective. Brain-computer interfaces (BCIs) have recently made significant strides in expanding their instruction set, which has attracted wide attention from researchers. The number of targets and commands is a key indicator of how well BCIs can decode the brain's intentions. No studies have reported a BCI system with over 200 targets.Approach. This study developed the first high-speed BCI system with up to 216 targets that were encoded by a combination of electroencephalography features, including P300, motion visual evoked potential (mVEP), and steady-state visual evoked potential (SSVEP). Specifically, the hybrid BCI paradigm used the time-frequency division multiple access strategy to elaborately tag targets with P300 and mVEP of different time windows, along with SSVEP of different frequencies. The hybrid features were then decoded by task-discriminant component analysis and linear discriminant analysis. Ten subjects participated in the offline and online cued-guided spelling experiments. Other ten subjects took part in online free-spelling experiments.Main results.The offline results showed that the mVEP and P300 components were prominent in the central, parietal, and occipital regions, while the most distinct SSVEP feature was in the occipital region. The online cued-guided spelling and free-spelling results showed that the proposed BCI system achieved an average accuracy of 85.37% ± 7.49% and 86.00% ± 5.98% for the 216-target classification, resulting in an average information transfer rate (ITR) of 302.83 ± 39.20 bits min-1and 204.47 ± 37.56 bits min-1, respectively. Notably, the peak ITR could reach up to 367.83 bits min-1.Significance.This study developed the first high-speed BCI system with more than 200 targets, which holds promise for extending BCI's application scenarios.

Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery

Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.

Effects of inter-stimulus intervals on concurrent P300 and SSVEP features for hybrid Brain-computer interfaces

BACKGROUND

Recently, we have implemented a high-speed brain-computer interface (BCI) system with a large instruction set using the concurrent P300 and steady-state visual evoked potential (SSVEP) features (also known as hybrid features). However, it remains unclear how to select inter-stimulus interval (ISI) for the proposed BCI system to balance the encoding efficiency and decoding performance.

NEW METHOD

This study developed a 6⁎9 hybrid P300-SSVEP BCI system and investigated a series of ISIs ranged from -175ms to 0ms with a step of 25ms. The influence of ISI on the hybrid features was analyzed from several aspects, including the amplitude of the induced features, classification accuracy, information transfer rate (ITR). Twelve naive subjects were recruited for the experiment.

RESULTS

The results showed the ISI factor had a significant impact on the hybrid features. Specifically, as the values of ISI decreased, the amplitudes of the induced features and accuracies decreased gradually, while the ITRs increased rapidly. It's achieved the highest ITR of 158.50 bits/min when ISI equal to -175ms.

COMPARISON WITH EXISTING METHOD

The optimal ISI in this study achieved superior performance in comparison with the one we used in the previous study.

CONCLUSIONS

The ISI can exert an important influence on the P300-SSVEP BCI system and its optimal value is -175ms in this study, which is significant for developing the high-speed BCI system with larger instruction sets in the future.

Prediction Deviants with Varying Degrees Induce Separable Error-related EEG Features

Error-related potential (ErrP) usually emerges in the brain when human perceives errors and is believed to be a promising signal for optimizing brain-computer interface (BCI) system. However, most of the ErrP studies only focus on how to distinguish the correct and wrong conditions, which is not enough for the BCI application in real scenarios. Therefore, it is necessary to study the ErrPs induced by the prediction deviants with varying degrees, concurrently test the separability of such EEG features. To this end, electroencephalogram (EEG) data of twelve healthy subjects were recorded when they participated in a direction prediction experiment. There are three prediction -deviant conditions in it, i.e., correct prediction, 90°deviant, 180° deviant. Event-related potential and inter-trial coherence were analyzed. Consequently, the error-related negativity (ERN) and N450 component in FCZ were significantly modulated by the degrees of prediction deviants, especially in the low-frequency band (<13Hz). Moreover, single-trial classification was adopted to test the separability of these features; the averaged accuracies between any two conditions were 87.75%, 85.25%, 64.79%. This study demonstrates the prediction deviants with varying degrees can induce separable ErrP features, which provide a deeper understanding of the ErrP signatures for developing BCIs.

Galvanic Vestibular Stimulation-Based Prediction Error Decoding and Channel Optimization

A significant problem in brain-computer interface (BCI) research is decoding - obtaining required information from very weak noisy electroencephalograph signals and extracting considerable information from limited data. Traditional intention decoding methods, which obtain information from induced or spontaneous brain activity, have shortcomings in terms of performance, computational expense and usage burden. Here, a new methodology called prediction error decoding was used for motor imagery (MI) detection and compared with direct intention decoding. Galvanic vestibular stimulation (GVS) was used to induce subliminal sensory feedback between the forehead and mastoids without any burden. Prediction errors were generated between the GVS-induced sensory feedback and the MI direction. The corresponding prediction error decoding of the front/back MI task was validated. A test decoding accuracy of 77.83-78.86% (median) was achieved during GVS for every 100[Formula: see text]ms interval. A nonzero weight parameter-based channel screening (WPS) method was proposed to select channels individually and commonly during GVS. When the WPS common-selected mode was compared with the WPS individual-selected mode and a classical channel selection method based on correlation coefficients (CCS), a satisfactory decoding performance of the selected channels was observed. The results indicated the positive impact of measuring common specific channels of the BCI.

Brain-Computer Interface (BCI) Control of a Virtual Assistant in a Smartphone to Manage Messaging Applications

Brain-computer interfaces (BCI) are a type of assistive technology that uses the brain signals of users to establish a communication and control channel between them and an external device. BCI systems may be a suitable tool to restore communication skills in severely motor-disabled patients, as BCI do not rely on muscular control. The loss of communication is one of the most negative consequences reported by such patients. This paper presents a BCI system focused on the control of four mainstream messaging applications running in a smartphone: WhatsApp, Telegram, e-mail and short message service (SMS). The control of the BCI is achieved through the well-known visual P300 row-column paradigm (RCP), allowing the user to select control commands as well as spelling characters. For the control of the smartphone, the system sends synthesized voice commands that are interpreted by a virtual assistant running in the smartphone. Four tasks related to the four mentioned messaging services were tested with 15 healthy volunteers, most of whom were able to accomplish the tasks, which included sending free text e-mails to an address proposed by the subjects themselves. The online performance results obtained, as well as the results of subjective questionnaires, support the viability of the proposed system.

Generalizing neural signal-to-text brain-computer interfaces

Objective:Brain-Computer Interfaces (BCI) may help patients with faltering communication abilities due to neurodegenerative diseases produce text or speech by direct neural processing. However, their practical realization has proven difficult due to limitations in speed, accuracy, and generalizability of existing interfaces. The goal of this study is to evaluate the BCI performance of a robust speech decoding system that translates neural signals evoked by speech to a textual output. While previous studies have approached this problem by using neural signals to choose from a limited set of possible words, we employ a more general model that can type any word from a large corpus of English text.Approach:In this study, we create an end-to-end BCI that translates neural signals associated with overt speech into text output. Our decoding system first isolates frequency bands in the input depth-electrode signal encapsulating differential information regarding production of various phonemic classes. These bands form a feature set that then feeds into a Long Short-Term Memory (LSTM) model which discerns at each time point probability distributions across all phonemes uttered by a subject. Finally, a particle filtering algorithm temporally smooths these probabilities by incorporating prior knowledge of the English language to output text corresponding to the decoded word. The generalizability of our decoder is driven by the lack of a vocabulary constraint on this output word.Main result:This method was evaluated using a dataset of 6 neurosurgical patients implanted with intra-cranial depth electrodes to identify seizure foci for potential surgical treatment of epilepsy. We averaged 32% word accuracy and on the phoneme-level obtained 46% precision, 51% recall and 73.32% average phoneme error rate while also achieving significant increases in speed when compared to several other BCI approaches.Significance:Our study employs a more general neural signal-to-text model which could facilitate communication by patients in everyday environments.

'Write' but not 'spell' Chinese characters with a BCI-controlled robot

Visual brain-computer interface (BCI) systems have made tremendous process in recent years. It has been demonstrated to perform well in spelling words. However, different from spelling English words in one-dimension sequences, Chinese characters are often written in a two-dimensional structure. Previous studies had never investigated how to use BCI to 'write' but not 'spell' Chinese characters. This study developed an innovative BCI-controlled robot for writing Chinese characters. The BCI system contained 108 commands displayed in a 9*12 array. A pixel-based writing method was proposed to map the starting point and ending point of each stroke of Chinese characters to the array. Connecting the starting and ending points for each stroke can make up any Chinese character. The large command set was encoded by the hybrid P300 and SSVEP features efficiently, in which each output needed only 1s of EEG data. The task-related component analysis was used to decode the combined features. Five subjects participated in this study and achieved an average accuracy of 87.23% and a maximal accuracy of 100%. The corresponding information transfer rate was 56.85 bits/min and 71.10 bits/min, respectively. The BCI-controlled robotic arm could write a Chinese character '' with 16 strokes within 5.7 seconds for the best subject. The demo video can be found at https://www.youtube.com/watch?v=A1w-e2dBGl0. The study results demonstrated that the proposed BCI-controlled robot is efficient for writing ideogram (e.g. Chinese characters) and phonogram (e.g. English letter), leading to broad prospects for real-world applications of BCIs.

Weighted Subject-Semi-Independent ERP-based Brain-Computer Interface

Subject-independent brain-computer interfaces (SI-BCIs) which require no calibration process, are increasingly affect researchers in BCI field. The efficiencies (accuracies), however, were not satisfying till now. In this paper, we proposed a weighted subject-semi-independent classification method (WSSICM) for ERP based BCI system in which a few blocks data of target subject were used. 47 participants were attended in this study. We compared the accuracies of proposed method with traditional subject-specific classification method（SSCM） which used 15 blocks data of target subject. The averaged accuracies were 95.2% for the WSSICM at 5 blocks and 95.7% for the SSCM at 15 blocks. The accuracies of two method did not show significant difference (p-value=0.652). The method we proposed in this paper which could reduce the calibration time can be used for future BCI systems.

Sensor Modalities for Brain-Computer Interface Technology: A Comprehensive Literature Review

Brain-computer interface (BCI) technology is rapidly developing and changing the paradigm of neurorestoration by linking cortical activity with control of an external effector to provide patients with tangible improvements in their ability to interact with the environment. The sensor component of a BCI circuit dictates the resolution of brain pattern recognition and therefore plays an integral role in the technology. Several sensor modalities are currently in use for BCI applications and are broadly either electrode-based or functional neuroimaging-based. Sensors vary in their inherent spatial and temporal resolutions, as well as in practical aspects such as invasiveness, portability, and maintenance. Hybrid BCI systems with multimodal sensory inputs represent a promising development in the field allowing for complimentary function. Artificial intelligence and deep learning algorithms have been applied to BCI systems to achieve faster and more accurate classifications of sensory input and improve user performance in various tasks. Neurofeedback is an important advancement in the field that has been implemented in several types of BCI systems by showing users a real-time display of their recorded brain activity during a task to facilitate their control over their own cortical activity. In this way, neurofeedback has improved BCI classification and enhanced user control over BCI output. Taken together, BCI systems have progressed significantly in recent years in terms of accuracy, speed, and communication. Understanding the sensory components of a BCI is essential for neurosurgeons and clinicians as they help advance this technology in the clinical setting.

The Effect of the Graphic Structures of Humanoid Robot on N200 and P300 Potentials

Humanoid robots are widely used in brain computer interface (BCI). Using a humanoid robot stimulus could increase the amplitude of event-related potentials (ERPs), which improves BCI performance. Since a humanoid robot contains many human elements, the element that increases the ERPs amplitude is unclear, and how to test the effect of it on the brain is a problem. This study used different graphic structures of an NAO humanoid robot to design three types of robot stimuli: a global robot, its local information, and its topological action. Ten subjects first conducted an odd-ball-based BCI (OD-BCI) by applying these stimuli. Then, they accomplished a delayed matching-to-sample task (DMST) that was used to specialize the encoding and retrieval phases of the OD-BCI task. In the retrieval phase of the DMST, the global stimulus induces the largest N200 and P300 potentials with the shortest latencies in the frontal, central, and occipital areas. This finding is in accordance with the P300 and classification performance of the OD-BCI task. When induced by the local stimulus, the subjects responded faster and more accurately in the retrieval phase of the DMST than in the other two conditions, indicating that the local stimulus improved the subject's responses. These results indicate that the OD-BCI task causes subject's retrieval work when the subject recognizes and outputs the stimulus. The global stimulus that contains topological and local elements could make brain react faster and induce larger ERPs, this finding could be used during the development of visual stimuli to improve BCI performance.

Doubling the Speed of N200 Speller via Dual-Directional Motion Encoding

OBJECTIVE

Motion-onset visual evoked potentials (mVEPs)-based spellers, also known as N200 spellers, have been successfully implemented, avoiding flashing stimuli that are common in visual brain-computer interface (BCI). However, their information transfer rates (ITRs), typically below 50 bits/min, are lower than other visual BCI spellers. In this study, we sought to improve the speed of N200 speller to a level above the well-known P300 spellers.

APPROACH

Based on our finding of the spatio-temporal asymmetry of N200 response elicited by leftward and rightward visual motion, a novel dual-directional N200 speller was implemented. By presenting visual stimuli moving in two different directions simultaneously, the new paradigm reduced the stimuli presentation time by half, while ensuring separable N200 features between two visual motion directions. Furthermore, a probability-based dynamic stopping algorithm was also proposed to shorten the decision time for each output further. Both offline and online tests were conducted to evaluate the performance in ten participants.

MAIN RESULTS

Offline results revealed contralateral dominant temporal and spatial patterns in N200 responses when subjects attended to stimuli moving leftward or rightward. In online experiments, the dual-directional paradigm achieved an average ITR of 79.8 bits/min, with the highest ITR of 124.8 bits/min. Compared with the traditional uni-directional N200 speller, the median gain on the ITR was 202%.

SIGNIFICANCE

The proposed dual-directional paradigm managed to double the speed of the N200 speller. Together with its non-flashing characteristics, this dual-directional N200 speller is promising to be a competent candidate for fast and reliable BCI applications.

Separable EEG Features Induced by Timing Prediction for Active Brain-Computer Interfaces

Brain–computer interfaces (BCI) have witnessed a rapid development in recent years. However, the active BCI paradigm is still underdeveloped with a lack of variety. It is imperative to adapt more voluntary mental activities for the active BCI control, which can induce separable electroencephalography (EEG) features. This study aims to demonstrate the brain function of timing prediction, i.e., the expectation of upcoming time intervals, is accessible for BCIs. Eighteen subjects were selected for this study. They were trained to have a precise idea of two sub-second time intervals, i.e., 400 ms and 600 ms, and were asked to measure a time interval of either 400 ms or 600 ms in mind after a cue onset. The EEG features induced by timing prediction were analyzed and classified using the combined discriminative canonical pattern matching and common spatial pattern. It was found that the ERPs in low-frequency (0~4 Hz) and energy in high-frequency (20~60 Hz) were separable for distinct timing predictions. The accuracy reached the highest of 93.75% with an average of 76.45% for the classification of 400 vs. 600 ms timing. This study first demonstrates that the cognitive EEG features induced by timing prediction are detectable and separable, which is feasible to be used in active BCIs controls and can broaden the category of BCIs.

A Hybrid Asynchronous Brain-Computer Interface Combining SSVEP and EOG Signals

OBJECTIVE

A challenging task for an electroencephalography (EEG)-based asynchronous brain-computer interface (BCI) is to effectively distinguish between the idle state and the control state while maintaining a short response time and a high accuracy when commands are issued in the control state. This study proposes a novel hybrid asynchronous BCI system based on a combination of steady-state visual evoked potentials (SSVEPs) in the EEG signal and blink-related electrooculography (EOG) signals.

METHODS

Twelve buttons corresponding to 12 characters are included in the graphical user interface (GUI). These buttons flicker at different fixed frequencies and phases to evoke SSVEPs and are simultaneously highlighted by changing their sizes. The user can select a character by focusing on its frequency-phase stimulus and simultaneously blinking his/her eyes in accordance with its highlighting as his/her EEG and EOG signals are recorded. A multifrequency band-based canonical correlation analysis (CCA) method is applied to the EEG data to detect the evoked SSVEPs, whereas the EOG data are analyzed to identify the user's blinks. Finally, the target character is identified based on the SSVEP and blink detection results.

RESULTS

Ten healthy subjects participated in our experiments and achieved an average information transfer rate (ITR) of 105.52 bits/min, an average accuracy of 95.42%, an average response time of 1.34 s and an average false-positive rate (FPR) of 0.8%.

CONCLUSION

The proposed BCI generates multiple commands with a high ITR and low FPR.

SIGNIFICANCE

The hybrid asynchronous BCI has great potential for practical applications in communication and control.

A comparison of classification methods for recognizing single-trial P300 in brain-computer interfaces

P300s are one of the most popular and robust control signals for brain-computer interfaces (BCIs). Fast classifying P300s is vital for the good performance of P300-based BCIs. However, due to noisy background electroencephalography (EEG) environments, current P300-based BCI systems need to collect multiple trials for a reliable output, which is inefficient. This study compared a recently developed algorithm, i.e. discriminative canonical pattern matching (DCPM), with five traditional classification methods,i.e. linear discriminant analysis (LDA), stepwise LDA, Bayesian LDA, shrinkage LDA and spatial-temporal discriminant analysis (STDA), for the detection of single-trial P300s. Eight subjects participated in the classical P300-speller experiments. Study results showed that the DCPM significantly outperformed the other traditional methods in single-trial P300 classification even with small training samples, suggesting the DCPM is a promising classification algorithm for the P300-based BCI.

Visual P300 Mind-Speller Brain-Computer Interfaces: A Walk Through the Recent Developments With Special Focus on Classification Algorithms

Brain-computer interfaces are sophisticated signal processing systems, which directly operate on neuronal signals to identify specific human intents. These systems can be applied to overcome certain disabilities or to enhance the natural capabilities of human beings. The visual P300 mind-speller is a prominent one among them, which has opened up tremendous possibilities in movement and communication applications. Today, there exist many state-of-the-art visual P300 mind-speller implementations in the literature as a result of numerous researches in this domain over the past 2 decades. Each of these systems can be evaluated in terms of performance metrics like classification accuracy, information transfer rate, and processing time. Various classification techniques associated with these systems, which include but are not limited to discriminant analysis, support vector machine, neural network, distance-based and ensemble of classifiers, have major roles in determining the overall system performances. The significance of a proper review on the recent developments in visual P300 mind-spellers with proper emphasis on their classification algorithms is the key insight for this work. This article is organized with a brief introduction to P300, concepts of visual P300 mind-spellers, the survey of literature with special focus on classification algorithms, followed by the discussion of various challenges and future directions.

Learning Discriminative Spatiospectral Features of ERPs for Accurate Brain-Computer Interfaces

Constructing accurate predictive models is at the heart of brain-computer interfaces (BCIs) because these models can ultimately translate brain activities into communication and control commands. The majority of the previous work in BCI use spatial, temporal, or spatiotemporal features of event-related potentials (ERPs). In this study, we examined the discriminatory effect of their spatiospectral features to capture the most relevant set of neural activities from electroencephalographic recordings that represent users' mental intent. In this regard, we model ERP waveforms using a sum of sinusoids with unknown amplitudes, frequencies, and phases. The effect of this signal modeling step is to represent high-dimensional ERP waveforms in a substantially lower dimensionality space, which includes their dominant power spectral contents. We found that the most discriminative frequencies for accurate decoding of visual attention modulated ERPs lie in a spectral range less than 6.4 Hz. This was empirically verified by treating dominant frequency contents of ERP waveforms as feature vectors in the state-of-the-art machine learning techniques used herein. The constructed predictive models achieved remarkable performance, which for some subjects was as high as 94% as measured by the area under curve. Using these spectral contents, we further studied the discriminatory effect of each channel and proposed an efficient strategy to choose subject-specific subsets of channels that generally led to classifiers with comparable performance.

EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) provide a direct communication channel between human brain and output devices. Due to advantages such as non-invasiveness, ease of use, and low cost, electroencephalography (EEG) is the most popular method for current BCIs. This chapter gives an overview of the current EEG-based BCIs for the main purpose of communication and control. This chapter first provides a taxonomy of the EEG-based BCI systems by categorizing them into three major groups: (1) BCIs based on event-related potentials (ERPs), (2) BCIs based on sensorimotor rhythms, and (3) hybrid BCIs. Next, this chapter describes challenges and potential solutions in developing practical BCI systems toward high communication speed, convenient system use, and low user variation. Then this chapter briefly reviews both medical and non-medical applications of current BCIs. Finally, this chapter concludes with a summary of current stage and future perspectives of the EEG-based BCI technology.

Cortical control of a tablet computer by people with paralysis

General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

Optimizing the stimulus presentation paradigm design for the P300-based brain-computer interface using performance prediction

OBJECTIVE

The role of a brain-computer interface (BCI) is to discern a user's intended message or action by extracting and decoding relevant information from brain signals. Stimulus-driven BCIs, such as the P300 speller, rely on detecting event-related potentials (ERPs) in response to a user attending to relevant or target stimulus events. However, this process is error-prone because the ERPs are embedded in noisy electroencephalography (EEG) data, representing a fundamental problem in communication of the uncertainty in the information that is received during noisy transmission. A BCI can be modeled as a noisy communication system and an information-theoretic approach can be exploited to design a stimulus presentation paradigm to maximize the information content that is presented to the user. However, previous methods that focused on designing error-correcting codes failed to provide significant performance improvements due to underestimating the effects of psycho-physiological factors on the P300 ERP elicitation process and a limited ability to predict online performance with their proposed methods. Maximizing the information rate favors the selection of stimulus presentation patterns with increased target presentation frequency, which exacerbates refractory effects and negatively impacts performance within the context of an oddball paradigm. An information-theoretic approach that seeks to understand the fundamental trade-off between information rate and reliability is desirable.

APPROACH

We developed a performance-based paradigm (PBP) by tuning specific parameters of the stimulus presentation paradigm to maximize performance while minimizing refractory effects. We used a probabilistic-based performance prediction method as an evaluation criterion to select a final configuration of the PBP.

MAIN RESULTS

With our PBP, we demonstrate statistically significant improvements in online performance, both in accuracy and spelling rate, compared to the conventional row-column paradigm.

SIGNIFICANCE

By accounting for refractory effects, an information-theoretic approach can be exploited to significantly improve BCI performance across a wide range of performance levels.

Comparison of Four Control Methods for a Five-Choice Assistive Technology

Severe motor impairments can affect the ability to communicate. The ability to see has a decisive influence on the augmentative and alternative communication (AAC) systems available to the user. To better understand the initial impressions users have of AAC systems we asked naïve healthy participants to compare two visual (a visual P300 brain-computer interface (BCI) and an eye-tracker) and two non-visual systems (an auditory and a tactile P300 BCI). Eleven healthy participants performed 20 selections in a five choice task with each system. The visual P300 BCI used face stimuli, the auditory P300 BCI used Japanese Hiragana syllables and the tactile P300 BCI used a stimulator on the small left finger, middle left finger, right thumb, middle right finger and small right finger. The eye-tracker required a dwell time of 3 s on the target for selection. We calculated accuracies and information-transfer rates (ITRs) for each control method using the selection time that yielded the highest ITR and an accuracy above 70% for each system. Accuracies of 88% were achieved with the visual P300 BCI (4.8 s selection time, 20.9 bits/min), of 70% with the auditory BCI (19.9 s, 3.3 bits/min), of 71% with the tactile BCI (18 s, 3.4 bits/min) and of 100% with the eye-tracker (5.1 s, 28.2 bits/min). Performance between eye-tracker and visual BCI correlated strongly, correlation between tactile and auditory BCI performance was lower. Our data showed no advantage for either non-visual system in terms of ITR but a lower correlation of performance which suggests that choosing the system which suits a particular user is of higher importance for non-visual systems than visual systems.

Incorporation of dynamic stopping strategy into the high-speed SSVEP-based BCIs

OBJECTIVE

Electroencephalography (EEG) is a non-linear and non-stationary process, as a result, its features are unstable and often vary in quality across trials, which poses significant challenges to brain-computer interfaces (BCIs). One remedy to this problem is to adaptively collect sufficient EEG evidence using dynamic stopping (DS) strategies. The high-speed steady-state visual evoked potential (SSVEP)-based BCI has experienced tremendous progress in recent years. This study aims to further improve the high-speed SSVEP-based BCI by incorporating the DS strategy.

APPROACH

This study involves the development of two different DS strategies for a high-speed SSVEP-based BCI, which were based on the Bayes estimation and the discriminant analysis, respectively. To evaluate their performance, they were compared with the conventional fixed stopping (FS) strategy using simulated online tests on both our collected data and a public dataset. Two most effective SSVEP recognition methods were used for comparison, including the extended canonical correlation analysis (CCA) and the ensemble task-related component analysis (TRCA).

MAIN RESULTS

The DS strategies achieved significantly higher information transfer rates (ITRs) than the FS strategy for both datasets, improving 9.78% for the Bayes-based DS and 6.7% for the discriminant-based DS. Specifically, the discriminant-based DS strategy using ensemble TRCA performed the best for our collected data, reaching an average ITR of 353.3  ±  67.1 bits min^-1 with a peak of 460 bits min^-1. The Bayes-based DS strategy using ensemble TRCA had the highest ITR for the public dataset, reaching an average of 230.2  ±  65.8 bits min^-1 with a peak of 304.1 bits min^-1.

SIGNIFICANCE

This study demonstrates that the proposed dynamic stopping strategies can further improve the performance of a SSVEP-based BCI, and hold promise for practical applications.

Utilizing sensory prediction errors for movement intention decoding: A new methodology

We propose a new methodology for decoding movement intentions of humans. This methodology is motivated by the well-documented ability of the brain to predict sensory outcomes of self-generated and imagined actions using so-called forward models. We propose to subliminally stimulate the sensory modality corresponding to a user's intended movement, and decode a user's movement intention from his electroencephalography (EEG), by decoding for prediction errors-whether the sensory prediction corresponding to a user's intended movement matches the subliminal sensory stimulation we induce. We tested our proposal in a binary wheelchair turning task in which users thought of turning their wheelchair either left or right. We stimulated their vestibular system subliminally, toward either the left or the right direction, using a galvanic vestibular stimulator and show that the decoding for prediction errors from the EEG can radically improve movement intention decoding performance. We observed an 87.2% median single-trial decoding accuracy across tested participants, with zero user training, within 96 ms of the stimulation, and with no additional cognitive load on the users because the stimulation was subliminal.

Brain-Computer Interfaces for Augmentative and Alternative Communication: A Tutorial

PURPOSE

Brain-computer interfaces (BCIs) have the potential to improve communication for people who require but are unable to use traditional augmentative and alternative communication (AAC) devices. As BCIs move toward clinical practice, speech-language pathologists (SLPs) will need to consider their appropriateness for AAC intervention.

METHOD

This tutorial provides a background on BCI approaches to provide AAC specialists foundational knowledge necessary for clinical application of BCI. Tutorial descriptions were generated based on a literature review of BCIs for restoring communication.

RESULTS

The tutorial responses directly address 4 major areas of interest for SLPs who specialize in AAC: (a) the current state of BCI with emphasis on SLP scope of practice (including the subareas: the way in which individuals access AAC with BCI, the efficacy of BCI for AAC, and the effects of fatigue), (b) populations for whom BCI is best suited, (c) the future of BCI as an addition to AAC access strategies, and (d) limitations of BCI.

CONCLUSION

Current BCIs have been designed as access methods for AAC rather than a replacement; therefore, SLPs can use existing knowledge in AAC as a starting point for clinical application. Additional training is recommended to stay updated with rapid advances in BCI.

Key considerations in designing a speech brain-computer interface

Restoring communication in case of aphasia is a key challenge for neurotechnologies. To this end, brain-computer strategies can be envisioned to allow artificial speech synthesis from the continuous decoding of neural signals underlying speech imagination. Such speech brain-computer interfaces do not exist yet and their design should consider three key choices that need to be made: the choice of appropriate brain regions to record neural activity from, the choice of an appropriate recording technique, and the choice of a neural decoding scheme in association with an appropriate speech synthesis method. These key considerations are discussed here in light of (1) the current understanding of the functional neuroanatomy of cortical areas underlying overt and covert speech production, (2) the available literature making use of a variety of brain recording techniques to better characterize and address the challenge of decoding cortical speech signals, and (3) the different speech synthesis approaches that can be considered depending on the level of speech representation (phonetic, acoustic or articulatory) envisioned to be decoded at the core of a speech BCI paradigm.

A Single-Channel EOG-Based Speller

Electrooculography (EOG) signals, which can be used to infer the intentions of a user based on eye movements, are widely used in human-computer interface (HCI) systems. Most existing EOG-based HCI systems incorporate a limited number of commands because they generally associate different commands with a few different types of eye movements, such as looking up, down, left, or right. This paper presents a novel single-channel EOG-based HCI that allows users to spell asynchronously by only blinking. Forty buttons corresponding to 40 characters displayed to the user via a graphical user interface are intensified in a random order. To select a button, the user must blink his/her eyes in synchrony as the target button is flashed. Two data processing procedures, specifically support vector machine (SVM) classification and waveform detection, are combined to detect eye blinks. During detection, we simultaneously feed the feature vectors extracted from the ongoing EOG signal into the SVM classification and waveform detection modules. Decisions are made based on the results of the SVM classification and waveform detection. Three online experiments were conducted with eight healthy subjects. We achieved an average accuracy of 94.4% and a response time of 4.14 s for selecting a character in synchronous mode, as well as an average accuracy of 93.43% and a false positive rate of 0.03/min in the idle state in asynchronous mode. The experimental results, therefore, demonstrated the effectiveness of this single-channel EOG-based speller.

High performance communication by people with paralysis using an intracortical brain-computer interface

Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O'Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4-4.2) and information throughput (by a factor of 2.2-4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.Clinical Trial No: NCT00912041.

BCI in practice

Brain-computer interfaces are systems that use signals recorded from the brain to enable communication and control applications for individuals who have impaired function. This technology has developed to the point that it is now being used by individuals who can actually benefit from it. However, there are several outstanding issues that prevent widespread use. These include the ease of obtaining high-quality recordings by home users, the speed, and accuracy of current devices and adapting applications to the needs of the user. In this chapter, we discuss some of these unsolved issues.