Information transfer rate in a five-classes brain-computer interface

Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications

Brain-computer interfaces (BCIs), a remarkable technological advancement in neurology and neurosurgery, mark a significant leap since the inception of electroencephalography in 1924. These interfaces effectively convert central nervous system signals into commands for external devices, offering revolutionary benefits to patients with severe communication and motor impairments due to a myriad of neurological conditions like stroke, spinal cord injuries, and neurodegenerative disorders. BCIs enable these individuals to communicate and interact with their environment, using their brain signals to operate interfaces for communication and environmental control. This technology is especially crucial for those completely locked in, providing a communication lifeline where other methods fall short. The advantages of BCIs are profound, offering autonomy and an improved quality of life for patients with severe disabilities. They allow for direct interaction with various devices and prostheses, bypassing damaged or nonfunctional neural pathways. However, challenges persist, including the complexity of accurately interpreting brain signals, the need for individual calibration, and ensuring reliable, long-term use. Additionally, ethical considerations arise regarding autonomy, consent, and the potential for dependence on technology. Despite these challenges, BCIs represent a transformative development in neurotechnology, promising enhanced patient outcomes and a deeper understanding of brain-machine interfaces.

A systematic investigation of detectors for low signal-to-noise ratio EMG signals

Background

Active participation of stroke survivors during robot-assisted movement therapy is essential for sensorimotor recovery. Robot-assisted therapy contingent on movement intention is an effective way to encourage patients' active engagement. For severely impaired stroke patients with no residual movements, a surface electromyogram (EMG) has been shown to be a viable option for detecting movement intention. Although numerous algorithms for EMG detection exist, the detector with the highest accuracy and lowest latency for low signal-to-noise ratio (SNR) remains unknown.

Methods

This study, therefore, investigates the performance of 13 existing EMG detection algorithms on simulated low SNR (0dB and -3dB) EMG signals generated using three different EMG signal models: Gaussian, Laplacian, and biophysical model. The detector performance was quantified using the false positive rate (FPR), false negative rate (FNR), and detection latency. Any detector that consistently showed FPR and FNR of no more than 20%, and latency of no more than 50ms, was considered an appropriate detector for use in robot-assisted therapy.

Results

The results indicate that the Modified Hodges detector - a simplified version of the threshold-based Hodges detector introduced in the current study - was the most consistent detector across the different signal models and SNRs. It consistently performed for ~90% and ~40% of the tested trials for 0dB and -3dB SNR, respectively. The two statistical detectors (Gaussian and Laplacian Approximate Generalized Likelihood Ratio) and the Fuzzy Entropy detectors have a slightly lower performance than Modified Hodges.

Conclusions

Overall, the Modified Hodges, Gaussian and Laplacian Approximate Generalized Likelihood Ratio, and the Fuzzy Entropy detectors were identified as the potential candidates that warrant further investigation with real surface EMG data since they had consistent detection performance on low SNR EMG data.

Opportunities, pitfalls and trade-offs in designing protocols for measuring the neural correlates of speech

Decoding speech and speech-related processes directly from the human brain has intensified in studies over recent years as such a decoder has the potential to positively impact people with limited communication capacity due to disease or injury. Additionally, it can present entirely new forms of human-computer interaction and human-machine communication in general and facilitate better neuroscientific understanding of speech processes. Here, we synthesize the literature on neural speech decoding pertaining to how speech decoding experiments have been conducted, coalescing around a necessity for thoughtful experimental design aimed at specific research goals, and robust procedures for evaluating speech decoding paradigms. We examine the use of different modalities for presenting stimuli to participants, methods for construction of paradigms including timings and speech rhythms, and possible linguistic considerations. In addition, novel methods for eliciting naturalistic speech and validating imagined speech task performance in experimental settings are presented based on recent research. We also describe the multitude of terms used to instruct participants on how to produce imagined speech during experiments and propose methods for investigating the effect of these terms on imagined speech decoding. We demonstrate that the range of experimental procedures used in neural speech decoding studies can have unintended consequences which can impact upon the efficacy of the knowledge obtained. The review delineates the strengths and weaknesses of present approaches and poses methodological advances which we anticipate will enhance experimental design, and progress toward the optimal design of movement independent direct speech brain-computer interfaces.

An extended binary subband canonical correlation analysis detection algorithm oriented to the radial contraction-expansion motion steady-state visual evoked paradigm

The radial contraction-expansion motion paradigm is a novel steady-state visual evoked experimental paradigm, and the electroencephalography (EEG) evoked potential is different from the traditional luminance modulation paradigm. The signal energy is concentrated chiefly in the fundamental frequency, while the higher harmonic power is lower. Therefore, the conventional steady-state visual evoked potential recognition algorithms optimizing multiple harmonic response components, such as the extended canonical correlation analysis (eCCA) and task-related component analysis (TRCA) algorithm, have poor recognition performance under the radial contraction-expansion motion paradigm. This paper proposes an extended binary subband canonical correlation analysis (eBSCCA) algorithm for the radial contraction-expansion motion paradigm. For the radial contraction-expansion motion paradigm, binary subband filtering was used to optimize the weighting coefficients of different frequency response signals, thereby improving the recognition performance of EEG signals. The results of offline experiments involving 13 subjects showed that the eBSCCA algorithm exhibits a better performance than the eCCA and TRCA algorithms under the stimulation of the radial contraction-expansion motion paradigm. In the online experiment, the average recognition accuracy of 13 subjects was 88.68% ± 6.33%, and the average information transmission rate (ITR) was 158.77 ± 43.67 bits/min, which proved that the algorithm had good recognition effect signals evoked by the radial contraction-expansion motion paradigm.

Feedback Related Potentials for EEG-Based Typing Systems

Error related potentials (ErrP), which are elicited in the EEG in response to a perceived error, have been used for error correction and adaption in the event related potential (ERP)-based brain computer interfaces designed for typing. In these typing interfaces, ERP evidence is collected in response to a sequence of stimuli presented usually in the visual form and the intended user stimulus is probabilistically inferred (stimulus with highest probability) and presented to the user as the decision. If the inferred stimulus is incorrect, ErrP is expected to be elicited in the EEG. Early approaches to use ErrP in the design of typing interfaces attempt to make hard decisions on the perceived error such that the perceived error is corrected and either the sequence of stimuli are repeated to obtain further ERP evidence, or without further repetition the stimulus with the second highest probability is presented to the user as the decision of the system. Moreover, none of the existing approaches use a language model to increase the performance of typing. In this work, unlike the existing approaches, we study the potential benefits of fusing feedback related potentials (FRP), a form of ErrP, with ERP and context information (language model, LM) in a Bayesian fashion to detect the user intent. We present experimental results based on data from 12 healthy participants using RSVP Keyboard™ to complete a copy-phrase-task. Three paradigms are compared: [P1] uses only ERP/LM Bayesian fusion; [P2] each RSVP sequence is appended with the top candidate in the alphabet according to posterior after ERP evidence fusion; corresponding FRP is then incorporated; and [P3] the top candidate is shown as a prospect to generate FRP evidence only if its posterior exceeds a threshold. Analyses indicate that ERP/LM/FRP evidence fusion during decision making yields significant speed-accuracy benefits for the user.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

A Single-Trial P300 Detector Based on Symbolized EEG and Autoencoded-(1D)CNN to Improve ITR Performance in BCIs

In this paper, we propose a breakthrough single-trial P300 detector that maximizes the information translate rate (ITR) of the brain–computer interface (BCI), keeping high recognition accuracy performance. The architecture, designed to improve the portability of the algorithm, demonstrated full implementability on a dedicated embedded platform. The proposed P300 detector is based on the combination of a novel pre-processing stage based on the EEG signals symbolization and an autoencoded convolutional neural network (CNN). The proposed system acquires data from only six EEG channels; thus, it treats them with a low-complexity preprocessing stage including baseline correction, windsorizing and symbolization. The symbolized EEG signals are then sent to an autoencoder model to emphasize those temporal features that can be meaningful for the following CNN stage. This latter consists of a seven-layer CNN, including a 1D convolutional layer and three dense ones. Two datasets have been analyzed to assess the algorithm performance: one from a P300 speller application in BCI competition III data and one from self-collected data during a fluid prototype car driving experiment. Experimental results on the P300 speller dataset showed that the proposed method achieves an average ITR (on two subjects) of 16.83 bits/min, outperforming by +5.75 bits/min the state-of-the-art for this parameter. Jointly with the speed increase, the recognition performance returned disruptive results in terms of the harmonic mean of precision and recall (F1-Score), which achieve 51.78 ± 6.24%. The same method used in the prototype car driving led to an ITR of ~33 bit/min with an F1-Score of 70.00% in a single-trial P300 detection context, allowing fluid usage of the BCI for driving purposes. The realized network has been validated on an STM32L4 microcontroller target, for complexity and implementation assessment. The implementation showed an overall resource occupation of 5.57% of the total available ROM, ~3% of the available RAM, requiring less than 3.5 ms to provide the classification outcome.

Motor Training Using Mental Workload (MWL) With an Assistive Soft Exoskeleton System: A Functional Near-Infrared Spectroscopy (fNIRS) Study for Brain-Machine Interface (BMI)

Mental workload is a neuroergonomic human factor, which is widely used in planning a system's safety and areas like brain-machine interface (BMI), neurofeedback, and assistive technologies. Robotic prosthetics methodologies are employed for assisting hemiplegic patients in performing routine activities. Assistive technologies' design and operation are required to have an easy interface with the brain with fewer protocols, in an attempt to optimize mobility and autonomy. The possible answer to these design questions may lie in neuroergonomics coupled with BMI systems. In this study, two human factors are addressed: designing a lightweight wearable robotic exoskeleton hand that is used to assist the potential stroke patients with an integrated portable brain interface using mental workload (MWL) signals acquired with portable functional near-infrared spectroscopy (fNIRS) system. The system may generate command signals for operating a wearable robotic exoskeleton hand using two-state MWL signals. The fNIRS system is used to record optical signals in the form of change in concentration of oxy and deoxygenated hemoglobin (HbO and HbR) from the pre-frontal cortex (PFC) region of the brain. Fifteen participants participated in this study and were given hand-grasping tasks. Two-state MWL signals acquired from the PFC region of the participant's brain are segregated using machine learning classifier-support vector machines (SVM) to utilize in operating a robotic exoskeleton hand. The maximum classification accuracy is 91.31%, using a combination of mean-slope features with an average information transfer rate (ITR) of 1.43. These results show the feasibility of a two-state MWL (fNIRS-based) robotic exoskeleton hand (BMI system) for hemiplegic patients assisting in the physical grasping tasks.

Neural Decoding of Imagined Speech and Visual Imagery as Intuitive Paradigms for BCI Communication

Brain-computer interface (BCI) is oriented toward intuitive systems that users can easily operate. Imagined speech and visual imagery are emerging paradigms that can directly convey a user's intention. We investigated the underlying characteristics that affect the decoding performance of these two paradigms. Twenty-two subjects performed imagined speech and visual imagery of twelve words/phrases frequently used for patients' communication. Spectral features were analyzed with thirteen-class classification (including rest class) using EEG filtered in six frequency ranges. In addition, cortical regions relevant to the two paradigms were analyzed by classification using single-channel and pre-defined cortical groups. Furthermore, we analyzed the word properties that affect the decoding performance based on the number of syllables, concrete and abstract concepts, and the correlation between the two paradigms. Finally, we investigated multiclass scalability in both paradigms. The high-frequency band displayed a significantly superior performance to that in the case of any other spectral features in the thirteen-class classification (imagined speech: 39.73 ± 5.64%; visual imagery: 40.14 ± 4.17%). Furthermore, the performance of Broca's and Wernicke's areas and auditory cortex was found to have improved among the cortical regions in both paradigms. As the number of classes increased, the decoding performance decreased moderately. Moreover, every subject exceeded the confidence level performance, implying the strength of the two paradigms in BCI inefficiency. These two intuitive paradigms were found to be highly effective for multiclass communication systems, having considerable similarities between each other. The results could provide crucial information for improving the decoding performance for practical BCI applications.

A review of user training methods in brain computer interfaces based on mental tasks

Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.

A probabilistic approach for calibration time reduction in hybrid EEG-fTCD brain-computer interfaces

Background

Generally, brain–computer interfaces (BCIs) require calibration before usage to ensure efficient performance. Therefore, each BCI user has to attend a certain number of calibration sessions to be able to use the system. However, such calibration requirements may be difficult to fulfill especially for patients with disabilities. In this paper, we introduce a probabilistic transfer learning approach to reduce the calibration requirements of our EEG–fTCD hybrid BCI designed using motor imagery (MI) and flickering mental rotation (MR)/word generation (WG) paradigms. The proposed approach identifies the top similar datasets from previous BCI users to a small training dataset collected from a current BCI user and uses these datasets to augment the training data of the current BCI user. To achieve such an aim, EEG and fTCD feature vectors of each trial were projected into scalar scores using support vector machines. EEG and fTCD class conditional distributions were learnt separately using the scores of each class. Bhattacharyya distance was used to identify similarities between class conditional distributions obtained using training trials of the current BCI user and those obtained using trials of previous users.

Results

Experimental results showed that the performance obtained using the proposed transfer learning approach outperforms the performance obtained without transfer learning for both MI and flickering MR/WG paradigms. In particular, it was found that the calibration requirements can be reduced by at least 60.43% for the MI paradigm, while at most a reduction of 17.31% can be achieved for the MR/WG paradigm.

Conclusions

Data collected using the MI paradigm show better generalization across subjects.

A Novel P300 Classification Algorithm Based on a Principal Component Analysis-Convolutional Neural Network

Aiming at enhancing the classification accuracy of P300 Electroencephalogram signals in a non-invasive brain–computer interface system, a novel P300 electroencephalogram signals classification algorithm is proposed which is based on improved convolutional neural network. In the data preprocessing part, the proposed P300 classification algorithm used the Principal Component Analysis algorithm to not only remove the noise and artifacts in the data, but also increase the data processing speed. Furthermore, the proposed P300 classification algorithm employed the parallel convolution method to improve the traditional convolutional neural network framework, which can increase the network depth and improve the network’s ability to classify P300 electroencephalogram signals. The proposed algorithm was evaluated by two datasets (the dataset from the competition and the dataset from the laboratory). The results show that, in the dataset I, the proposed P300 classification algorithm could obtain accuracy rates higher than 95%, and achieve one of the best performances in four classification algorithms, while, in the dataset II, the proposed P300 classification algorithm can get accuracy rates higher than 90%, and is superior to the other three algorithms in all ten subjects. These demonstrated the effectiveness of the proposed algorithm. The proposed classification algorithm can be applied in the actual brain–computer interface systems to help people with disability in the daily lives.

Optimizing Computer-Brain Interface Parameters for Non-invasive Brain-to-Brain Interface

A non-invasive, brain-to-brain interface (BBI) requires precision neuromodulation and high temporal resolution as well as portability to increase accessibility. A BBI is a combination of the brain-computer interface (BCI) and the computer-brain interface (CBI). The optimization of BCI parameters has been extensively researched, but CBI has not. Parameters taken from the BCI and CBI literature were used to simulate a two-class medical monitoring BBI system under a wide range of conditions. BBI function was assessed using the information transfer rate (ITR), measured in bits per trial and bits per minute. The BBI ITR was a function of classifier accuracy, window update rate, system latency, stimulation failure rate (SFR), and timeout threshold. The BCI parameters, including window length, update rate, and classifier accuracy, were kept constant to investigate the effects of varying the CBI parameters, including system latency, SFR, and timeout threshold. Based on passively monitoring BCI parameters, a base ITR of 1 bit/trial was used. The optimal latency was found to be 100 ms or less, with a threshold no more than twice its value. With the optimal latency and timeout parameters, the system was able to maintain near-maximum efficiency, even with a 25% SFR. When the CBI and BCI parameters are compared, the CBI's system latency and timeout threshold should be reflected in the BCI's update rate. This would maximize the number of trials, even at a high SFR. These findings suggested that a higher number of trials per minute optimizes the ITR of a non-invasive BBI. The delays innate to each BCI protocol and CBI stimulation method must also be accounted for. The high latencies in each are the primary constraints of non-invasive BBI for the foreseeable future.

Electroencephalography

The electroencephalogram (EEG) was invented almost 100 years ago and is still a method of choice for many research questions, even applications-from functional brain imaging in neuroscientific investigations during movement to real-time applications like brain-computer interfacing. This chapter gives some background information on the establishment and properties of the EEG. This chapter starts with a closer look at the sources of EEG at a micro or neuronal level, followed by recording techniques, types of electrodes, and common EEG artifacts. Then an overview on EEG phenomena, namely, spontaneous EEG and event-related potentials build the middle part of this chapter. The last part discusses brain signals, which are used in current BCI research, including short descriptions and examples of applications.

Exploring Cognitive Flexibility With a Noninvasive BCI Using Simultaneous Steady-State Visual Evoked Potentials and Sensorimotor Rhythms

EEG-based brain-computer interface (BCI) technology creates non-biological pathways for conveying a user's mental intent solely through noninvasively measured neural signals. While optimizing the performance of a single task has long been the focus of BCI research, in order to translate this technology into everyday life, realistic situations, in which multiple tasks are performed simultaneously, must be investigated. In this paper, we explore the concept of cognitive flexibility, or multitasking, within the BCI framework by utilizing a 2-D cursor control task, using sensorimotor rhythms (SMRs), and a four-target visual attention task, using steady-state visual evoked potentials (SSVEPs), both individually and simultaneously. We found no significant difference between the accuracy of the tasks when executing them alone (SMR-57.9% ± 15.4% and SSVEP-59.0% ± 14.2%) and simultaneously (SMR-54.9% ± 17.2% and SSVEP-57.5% ± 15.4%). These modest decreases in performance were supported by similar, non-significant changes in the electrophysiology of the SSVEP and SMR signals. In this sense, we report that multiple BCI tasks can be performed simultaneously without a significant deterioration in performance; this finding will help drive these systems toward realistic daily use in which a user's cognition will need to be involved in multiple tasks at once.

Common spatial pattern and wavelet decomposition for motor imagery EEG- fTCD brain-computer interface

BACKGROUND

Recently, hybrid brain-computer interfaces (BCIs) combining more than one modality have been investigated with the aim of boosting the performance of the existing single-modal BCIs in terms of accuracy and information transfer rate (ITR). Previously, we introduced a novel hybrid BCI in which EEG and fTCD modalities are used simultaneously to measure electrical brain activity and cerebral blood velocity during motor imagery (MI) tasks.

NEW METHOD

In this paper, we used multi-scale analysis and common spatial pattern algorithm to extract EEG and fTCD features. Moreover, we proposed probabilistic fusion of EEG and fTCD evidences instead of concatenating EEG and fTCD feature vectors corresponding to each trial. A Bayesian approach was proposed to fuse EEG and fTCD evidences under 3 different assumptions.

RESULTS

Experimental results showed that 93.85%, 93.71%, and 100% average accuracies and 19.89, 26.55, and 40.83 bits/min average ITRs were achieved for right MI vs baseline, left MI versus baseline, and right MI versus left MI respectively.

COMPARISON WITH EXISTING METHODS

These performance measures outperformed the results we obtained before in our preliminary study in which average accuracies of 88.33%, 89.48%, and 82.38% and average ITRs of 4.17, 5.45, and 10.57 bits/min were achieved for right MI versus baseline, left MI versus baseline, and right MI versus left MI respectively. Moreover, in terms of both accuracy and speed, the EEG- fTCD BCI with the proposed analysis techniques outperformed all EEG- fNIRS studies in comparison.

CONCLUSIONS

The proposed system is a more accurate and faster alternative to EEG-fNIRS systems.

EEG-fTCD hybrid brain-computer interface using template matching and wavelet decomposition

OBJECTIVE

We aim at developing a hybrid brain-computer interface that utilizes electroencephalography (EEG) and functional transcranial Doppler (fTCD). In this hybrid BCI, EEG and fTCD are used simultaneously to measure electrical brain activity and cerebral blood velocity respectively in response to flickering mental rotation (MR) and word generation (WG) tasks. In this paper, we improve both the accuracy and information transfer rate (ITR) of this novel hybrid brain computer interface (BCI) we designed in our previous work.

APPROACH

To achieve such aim, we extended our feature extraction approach through using template matching and multi-scale analysis to extract EEG and fTCD features, respectively. In particular, template matching was used to analyze EEG data whereas 5-level wavelet decomposition was applied to fTCD data. Significant EEG and fTCD features were selected using Wilcoxon signed rank test. Support vector machines classifier (SVM) was used to project EEG and fTCD selected features of each trial into scalar SVM scores. Moreover, instead of concatenating EEG and fTCD feature vectors corresponding to each trial, we proposed a Bayesian fusion approach of EEG and fTCD evidences.

MAIN RESULTS

Average accuracy and average ITR of 98.11% and 21.29 bits min^-1 were achieved for WG versus MR classification while MR versus baseline yielded 86.27% average accuracy and 8.95 bit min^-1 average ITR. In addition, average accuracy of 85.29% and average ITR of 8.34 bits min^-1 were obtained for WG versus baseline.

SIGNIFICANCE

The proposed analysis techniques significantly improved the hybrid BCI performance. Specifically, for MR/WG versus baseline problems, we achieved twice of the ITRs obtained in our previous study. Moreover, the ITR of WG versus MR problem is 4-times the ITR we obtained before for the same problem. The current analysis methods boosted the performance of our EEG-fTCD BCI such that it outperformed the existing EEG-fNIRS BCIs in comparison.

Optimal Query Selection Using Multi-Armed Bandits

Query selection for latent variable estimation is conventionally performed by opting for observations with low noise or optimizing information theoretic objectives related to reducing the level of estimated uncertainty based on the current best estimate. In these approaches, typically the system makes a decision by leveraging the current available information about the state. However, trusting the current best estimate results in poor query selection when truth is far from the current estimate, and this negatively impacts the speed and accuracy of the latent variable estimation procedure. We introduce a novel sequential adaptive action value function for query selection using the multi-armed bandit (MAB) framework which allows us to find a tractable solution. For this adaptive-sequential query selection method, we analytically show: (i) performance improvement in the query selection for a dynamical system, (ii) the conditions where the model outperforms competitors. We also present favorable empirical assessments of the performance for this method, compared to alternative methods, both using Monte Carlo simulations and human-in-the-loop experiments with a brain computer interface (BCI) typing system where the language model provides the prior information.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

Three-Dimensional Brain-Computer Interface Control Through Simultaneous Overt Spatial Attentional and Motor Imagery Tasks

OBJECTIVE

While noninvasive electroenceph-alography (EEG) based brain-computer interfacing (BCI) has been successfully demonstrated in two-dimensional (2-D) control tasks, little work has been published regarding its extension to practical three-dimensional (3-D) control.

METHODS

In this study, we developed a new BCI approach for 3-D control by combining a novel form of endogenous visuospatial attentional modulation, defined as overt spatial attention (OSA), and motor imagery (MI).

RESULTS

OSA modulation was shown to provide comparable control to conventional MI modulation in both 1-D and 2-D tasks. Furthermore, this paper provides evidence for the functional independence of traditional MI and OSA, as well as an investigation into the simultaneous use of both. Using this newly proposed BCI paradigm, 16 participants successfully completed a 3-D eight-target control task. Nine of these subjects further demonstrated robust 3-D control in a 12-target task, significantly outperforming the information transfer rate achieved in the 1-D and 2-D control tasks (29.7 ± 1.6 b/min).

CONCLUSION

These results strongly support the hypothesis that noninvasive EEG-based BCI can provide robust 3-D control through endogenous neural modulation in broader populations with limited training.

SIGNIFICANCE

Through the combination of the two strategies (MI and OSA), a substantial portion of the recruited subjects were capable of robustly controlling a virtual cursor in 3-D space. The proposed novel approach could broaden the dimensionality of BCI control and shorten the training time.

Classification of auditory selective attention using spatial coherence and modular attention index

BACKGROUND AND OBJECTIVE

Brain-Computer Interfaces (BCIs) based on auditory selective attention have been receiving much attention because i) they are useful for completely paralyzed users since they do not require muscular effort or gaze and ii) focusing attention is a natural human ability. Several techniques - such as recently developed Spatial Coherence (SC) - have been proposed in order to optimize the BCI procedure. Thus, this work aims at investigating and comparing two strategies based on spatial coherence detection: contralateral and modular classifiers. The latter is a new method using modular attention index. The new classifier was developed to implement an auditory BCI where a volunteer makes binary choices using selective attention under the amplitude-modulated tones stimulation.

METHODS

Contralateral and modular classifiers were applied to the electroencephalogram (EEG) recorded from 144 subjects under the BCI protocol. The best set of parameters (carriers of the stimulus, channels and trials of signal) for this BCI was investigated taking into consideration the hit rate and the information transfer rate.

RESULTS

The best result obtained using the modular classifier was a hit rate of 91.67% and information transfer rate of 6.74 bits/min using 0.5 kHz/4.0 kHz as stimuli and three windows (5.10 sec of EEG signal). These results were obtained with five electrodes (C3, P3, F8, P4, O2) using exhaustive search to identify regions with greater coherence.

CONCLUSION

The modular classifier - using electroencephalogram channels from the central, frontal, occipital and parietal areas - improves the performance of auditory BCIs based on selective attention.

Frequency Specific Cortical Dynamics During Motor Imagery Are Influenced by Prior Physical Activity

Motor imagery is often used inducing changes in electroencephalographic (EEG) signals for imagery-based brain-computer interfacing (BCI). A BCI is a device translating brain signals into control signals providing severely motor-impaired persons with an additional, non-muscular channel for communication and control. In the last years, there is increasing interest using BCIs also for healthy people in terms of enhancement or gaming. Most studies focusing on improving signal processing feature extraction and classification methods, but the performance of a BCI can also be improved by optimizing the user's control strategies, e.g., using more vivid and engaging mental tasks for control. We used multichannel EEG to investigate neural correlates of a sports imagery task (playing tennis) compared to a simple motor imagery task (squeezing a ball). To enhance the vividness of both tasks participants performed a short physical exercise between two imagery sessions. EEG was recorded from 60 closely spaced electrodes placed over frontal, central, and parietal areas of 30 healthy volunteers divided in two groups. Whereas Group 1 (EG) performed a physical exercise between the two imagery sessions, Group 2 (CG) watched a landscape movie without physical activity. Spatiotemporal event-related desynchronization (ERD) and event-related synchronization (ERS) patterns during motor imagery (MI) tasks were evaluated. The results of the EG showed significant stronger ERD patterns in the alpha frequency band (8-13 Hz) during MI of tennis after training. Our results are in evidence with previous findings that MI in combination with motor execution has beneficial effects. We conclude that sports MI combined with an interactive game environment could be a future promising task in motor learning and rehabilitation improving motor functions in late therapy processes or support neuroplasticity.

Developing a Three- to Six-State EEG-Based Brain-Computer Interface for a Virtual Robotic Manipulator Control

OBJECTIVE

We develop an electroencephalography (EEG)-based noninvasive brain-computer interface (BCI) system having short training time (15 min) that can be applied for high-performance control of robotic prosthetic systems.

METHODS

A signal processing system for detecting user's mental intent from EEG data based on up to six-state BCI paradigm is developed and used.

RESULTS

We examine the performance of the developed system on experimental data collected from 12 healthy participants and analyzed offline. Out of 12 participants 3 achieve an accuracy of six-state communication in 80%-90% range, while 2 participants do not achieve a satisfactory accuracy. We further implement an online BCI system for control of a virtual 3 degree-of-freedom (dof) prosthetic manipulator and test it with our three best participants. Two participants are able to successfully complete 100% of the test tasks, demonstrating on average the accuracy rate of 80% and requiring 5-10 s to execute a manipulator move. One participant failed to demonstrate a satisfactory performance in online trials.

CONCLUSION

We show that our offline EEG BCI system can correctly identify different motor imageries in EEG data with high accuracy and our online BCI system can be used for control of a virtual 3 dof prosthetic manipulator.

SIGNIFICANCE

Our results prepare foundation for further development of higher performance EEG BCI-based robotic assistive systems and demonstrate that EEG-based BCI may be feasible for robotic control by paralyzed and immobilized individuals.

Improving EEG-Based Motor Imagery Classification for Real-Time Applications Using the QSA Method

We present an improvement to the quaternion-based signal analysis (QSA) technique to extract electroencephalography (EEG) signal features with a view to developing real-time applications, particularly in motor imagery (IM) cognitive processes. The proposed methodology (iQSA, improved QSA) extracts features such as the average, variance, homogeneity, and contrast of EEG signals related to motor imagery in a more efficient manner (i.e., by reducing the number of samples needed to classify the signal and improving the classification percentage) compared to the original QSA technique. Specifically, we can sample the signal in variable time periods (from 0.5 s to 3 s, in half-a-second intervals) to determine the relationship between the number of samples and their effectiveness in classifying signals. In addition, to strengthen the classification process a number of boosting-technique-based decision trees were implemented. The results show an 82.30% accuracy rate for 0.5 s samples and 73.16% for 3 s samples. This is a significant improvement compared to the original QSA technique that offered results from 33.31% to 40.82% without sampling window and from 33.44% to 41.07% with sampling window, respectively. We can thus conclude that iQSA is better suited to develop real-time applications.

Towards optimal visual presentation design for hybrid EEG-fTCD brain-computer interfaces

OBJECTIVE

In this paper, we introduce a novel hybrid brain-computer interface (BCI) system that measures electrical brain activity as well as cerebral blood velocity using electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD) respectively in response to flickering mental rotation (MR) and flickering word generation (WG) cognitive tasks as well as a fixation cross that represents the baseline. This work extends our previous approach, in which we showed that motor imagery induces simultaneous changes in EEG and fTCD to enable task discrimination; and hence, provides a design approach for a hybrid BCI. Here, we show that instead of using motor imagery, the proposed visual stimulation technique enables the design of an EEG-fTCD based BCI with higher accuracy.

APPROACH

Features based on the power spectrum of EEG and fTCD signals were calculated. Mutual information and support vector machines were used for feature selection and classification purposes.

MAIN RESULTS

EEG-fTCD combination outperformed EEG by 4.05% accuracy for MR versus baseline problem and by 5.81% accuracy for WG versus baseline problem. An average accuracy of 92.38% was achieved for MR versus WG problem using the hybrid combination. Average transmission rates of 4.39, 3.92, and 5.60 bits min^-1 were obtained for MR versus baseline, WG versus baseline, and MR versus WG problems respectively.

SIGNIFICANCE

In terms of accuracy, the current visual presentation outperforms the motor imagery visual presentation we designed before for the EEG-fTCD system by 10% accuracy for task versus task problem. Moreover, the proposed system outperforms the state of the art hybrid EEG-fNIRS BCIs in terms of accuracy and/or information transfer rate. Even though there are still limitations of the proposed system, such promising results show that the proposed hybrid system is a feasible candidate for real-time BCIs.

Is EMG a Viable Alternative to BCI for Detecting Movement Intention in Severe Stroke?

OBJECTIVE

In light of the shortcomings of current restorative brain-computer interfaces (BCI), this study investigated the possibility of using EMG to detect hand/wrist extension movement intention to trigger robot-assisted training in individuals without residual movements.

METHODS

We compared movement intention detection using an EMG detector with a sensorimotor rhythm based EEG-BCI using only ipsilesional activity. This was carried out on data of 30 severely affected chronic stroke patients from a randomized control trial using an EEG-BCI for robot-assisted training.

RESULTS

The results indicate the feasibility of using EMG to detect movement intention in this severely handicapped population; probability of detecting EMG when patients attempted to move was higher (p 0.001) than at rest. Interestingly, 22 out of 30 (or 73%) patients had sufficiently strong EMG in their finger/wrist extensors. Furthermore, in patients with detectable EMG, there was poor agreement between the EEG and EMG intent detectors, which indicates that these modalities may detect different processes.

CONCLUSION

A substantial segment of severely affected stroke patients may benefit from EMG-based assisted therapy. When compared to EEG, a surface EMG interface requires less preparation time, which is easier to don/doff, and is more compact in size.

SIGNIFICANCE

This study shows that a large proportion of severely affected stroke patients have residual EMG, which yields a direct and practical way to trigger robot-assisted training.

Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016

This paper presents a new approach to benchmarking brain-computer interfaces (BCIs) outside the lab. A computer game was created that mimics a real-world application of assistive BCIs, with the main outcome metric being the time needed to complete the game. This approach was used at the Cybathlon 2016, a competition for people with disabilities who use assistive technology to achieve tasks. The paper summarizes the technical challenges of BCIs, describes the design of the benchmarking game, then describes the rules for acceptable hardware, software and inclusion of human pilots in the BCI competition at the Cybathlon. The 11 participating teams, their approaches, and their results at the Cybathlon are presented. Though the benchmarking procedure has some limitations (for instance, we were unable to identify any factors that clearly contribute to BCI performance), it can be successfully used to analyze BCI performance in realistic, less structured conditions. In the future, the parameters of the benchmarking game could be modified to better mimic different applications (e.g., the need to use some commands more frequently than others). Furthermore, the Cybathlon has the potential to showcase such devices to the general public.

Mental Task Evaluation for Hybrid NIRS-EEG Brain-Computer Interfaces

Based on recent electroencephalography (EEG) and near-infrared spectroscopy (NIRS) studies that showed that tasks such as motor imagery and mental arithmetic induce specific neural response patterns, we propose a hybrid brain-computer interface (hBCI) paradigm in which EEG and NIRS data are fused to improve binary classification performance. We recorded simultaneous NIRS-EEG data from nine participants performing seven mental tasks (word generation, mental rotation, subtraction, singing and navigation, and motor and face imagery). Classifiers were trained for each possible pair of tasks using (1) EEG features alone, (2) NIRS features alone, and (3) EEG and NIRS features combined, to identify the best task pairs and assess the usefulness of a multimodal approach. The NIRS-EEG approach led to an average increase in peak kappa of 0.03 when using features extracted from one-second windows (equivalent to an increase of 1.5% in classification accuracy for balanced classes). The increase was much stronger (0.20, corresponding to an 10% accuracy increase) when focusing on time windows of high NIRS performance. The EEG and NIRS analyses further unveiled relevant brain regions and important feature types. This work provides a basis for future NIRS-EEG hBCI studies aiming to improve classification performance toward more efficient and flexible BCIs.

An EOG-Based Human-Machine Interface for Wheelchair Control

OBJECTIVE

Nonmanual human-machine interfaces (HMIs) have been studied for wheelchair control with the aim of helping severely paralyzed individuals regain some mobility. The challenge is to rapidly, accurately, and sufficiently produce control commands, such as left and right turns, forward and backward motions, acceleration, deceleration, and stopping. In this paper, a novel electrooculogram (EOG) based HMI is proposed for wheelchair control.

METHODS

A total of 13 flashing buttons, each of which corresponds to a command, are presented in the graphical user interface. These buttons flash on a one-by-one manner in a predefined sequence. The user can select a button by blinking in sync with its flashes. The algorithm detects the eye blinks from a channel of vertical EOG data and determines the user's target button based on the synchronization between the detected blinks and the button's flashes.

RESULTS

For healthy subjects/patients with spinal cord injuries, the proposed HMI achieved an average accuracy of 96.7% / 91.7% and a response time of 3.53 s/3.67 s with 0 false positive rates (FPRs).

CONCLUSION

Using one channel of vertical EOG signals associated with eye blinks, the proposed HMI can accurately provide sufficient commands with a satisfactory response time.

SIGNIFICANCE

The proposed HMI provides a novel nonmanual approach for severely paralyzed individuals to control a wheelchair. Compared with a newly established EOG-based HMI, the proposed HMI can generate more commands with higher accuracy, lower FPR, and fewer electrodes.

Brain-Computer Interface application: auditory serial interface to control a two-class motor-imagery-based wheelchair

BACKGROUND

Certain diseases affect brain areas that control the movements of the patients' body, thereby limiting their autonomy and communication capacity. Research in the field of Brain-Computer Interfaces aims to provide patients with an alternative communication channel not based on muscular activity, but on the processing of brain signals. Through these systems, subjects can control external devices such as spellers to communicate, robotic prostheses to restore limb movements, or domotic systems. The present work focus on the non-muscular control of a robotic wheelchair.

METHOD

A proposal to control a wheelchair through a Brain-Computer Interface based on the discrimination of only two mental tasks is presented in this study. The wheelchair displacement is performed with discrete movements. The control signals used are sensorimotor rhythms modulated through a right-hand motor imagery task or mental idle state. The peculiarity of the control system is that it is based on a serial auditory interface that provides the user with four navigation commands. The use of two mental tasks to select commands may facilitate control and reduce error rates compared to other endogenous control systems for wheelchairs.

RESULTS

Seventeen subjects initially participated in the study; nine of them completed the three sessions of the proposed protocol. After the first calibration session, seven subjects were discarded due to a low control of their electroencephalographic signals; nine out of ten subjects controlled a virtual wheelchair during the second session; these same nine subjects achieved a medium accuracy level above 0.83 on the real wheelchair control session.

CONCLUSION

The results suggest that more extensive training with the proposed control system can be an effective and safe option that will allow the displacement of a wheelchair in a controlled environment for potential users suffering from some types of motor neuron diseases.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.