Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention

Generative adversarial networks in EEG analysis: an overview

Electroencephalogram (EEG) signals have been utilized in a variety of medical as well as engineering applications. However, one of the challenges associated with recording EEG data is the difficulty of recording large amounts of data. Consequently, data augmentation is a potential solution to overcome this challenge in which the objective is to increase the amount of data. Inspired by the success of Generative Adversarial Networks (GANs) in image processing applications, generating artificial EEG data from the limited recorded data using GANs has seen recent success. This article provides an overview of various techniques and approaches of GANs for augmenting EEG signals. We focus on the utility of GANs in different applications including Brain-Computer Interface (BCI) paradigms such as motor imagery and P300-based systems, in addition to emotion recognition, epileptic seizures detection and prediction, and various other applications. We address in this article how GANs have been used in each study, the impact of using GANs on the model performance, the limitations of each algorithm, and future possibilities for developing new algorithms. We emphasize the utility of GANs in augmenting the limited EEG data typically available in the studied applications.

Long-Range α-Synchronization as Control Signal for BCI: A Feasibility Study

Shifts in spatial attention are associated with variations in α band (α, 8-14 Hz) activity, specifically in interhemispheric imbalance. The underlying mechanism is attributed to local α-synchronization, which regulates local inhibition of neural excitability, and frontoparietal synchronization reflecting long-range communication. The direction-specific nature of this neural correlate brings forward its potential as a control signal in brain-computer interfaces (BCIs). In the present study, we explored whether long-range α-synchronization presents lateralized patterns dependent on voluntary attention orienting and whether these neural patterns can be picked up at a single-trial level to provide a control signal for active BCI. We collected electroencephalography (EEG) data from a cohort of healthy adults (n = 10) while performing a covert visuospatial attention (CVSA) task. The data show a lateralized pattern of α-band phase coupling between frontal and parieto-occipital regions after target presentation, replicating previous findings. This pattern, however, was not evident during the cue-to-target orienting interval, the ideal time window for BCI. Furthermore, decoding the direction of attention trial-by-trial from cue-locked synchronization with support vector machines (SVMs) was at chance level. The present findings suggest EEG may not be capable of detecting long-range α-synchronization in attentional orienting on a single-trial basis and, thus, highlight the limitations of this metric as a reliable signal for BCI control.

Beyond the brain-computer interface: Decoding brain activity as a tool to understand neuronal mechanisms subtending cognition and behavior

One of the major challenges in system neurosciences consists in developing techniques for estimating the cognitive information content in brain activity. This has an enormous potential in different domains spanning from clinical applications, cognitive enhancement to a better understanding of the neural bases of cognition. In this context, the inclusion of machine learning techniques to decode different aspects of human cognition and behavior and its use to develop brain–computer interfaces for applications in neuroprosthetics has supported a genuine revolution in the field. However, while these approaches have been shown quite successful for the study of the motor and sensory functions, success is still far from being reached when it comes to covert cognitive functions such as attention, motivation and decision making. While improvement in this field of BCIs is growing fast, a new research focus has emerged from the development of strategies for decoding neural activity. In this review, we aim at exploring how the advanced in decoding of brain activity is becoming a major neuroscience tool moving forward our understanding of brain functions, providing a robust theoretical framework to test predictions on the relationship between brain activity and cognition and behavior.

Winners and losers in brain computer interface competitive gaming: Directional connectivity analysis

OBJECTIVE

to characterize the direction within and between brain connectivity in winning and losing players in a competitive brain-computer interface game.

APPROACH

ten dyads (26.9 ± 4.7 years old, eight females and 12 males) participated in the study. In a competitive game based on neurofeedback, they used their relative alpha (RA) band power from the electrode location Pz, to control a virtual seesaw. The players in each pair were separated into winners (W) and losers (L) based on their scores. Intrabrain connectivity was analyzed using multivariate Granger Causality (GC) and Directed Transfer Function, while interbrain connectivity was analyzed using bivariate GC.

RESULTS

linear regression analysis revealed a significant relationship (p<0.05) between RA and individual scores. During the game, W players maintained a higher RA than L players, although it was not higher than their baseline RA. The analysis of intrabrain GC indicated that both groups engaged in general social interactions, but only the W group succeeded in controlling their brain activity at Pz. Group L applied an inappropriate metal strategy, characterized by strong activity in the left frontal cortex, indicative of collaborative gaming. Interbrain GC showed a larger flow of information from the L to the W group, suggesting a higher capability of the W group to monitor the activity of their opponent.

SIGNIFICANCE

both innate neurological indices and gaming mental strategies contribute to game outcomes. Future studies should investigate whether there is a causal relationship between these two factors.

Target-Related Alpha Attenuation in a Brain-Computer Interface Rapid Serial Visual Presentation Calibration

This study evaluated the feasibility of using occipitoparietal alpha activity to drive target/non-target classification in a brain-computer interface (BCI) for communication. EEG data were collected from 12 participants who completed BCI Rapid Serial Visual Presentation (RSVP) calibrations at two different presentation rates: 1 and 4 Hz. Attention-related changes in posterior alpha activity were compared to two event-related potentials (ERPs): N200 and P300. Machine learning approaches evaluated target/non-target classification accuracy using alpha activity. Results indicated significant alpha attenuation following target letters at both 1 and 4 Hz presentation rates, though this effect was significantly reduced in the 4 Hz condition. Target-related alpha attenuation was not correlated with coincident N200 or P300 target effects. Classification using posterior alpha activity was above chance and benefitted from individualized tuning procedures. These findings suggest that target-related posterior alpha attenuation is detectable in a BCI RSVP calibration and that this signal could be leveraged in machine learning algorithms used for RSVP or comparable attention-based BCI paradigms.

Attentional load classification in multiple object tracking task using optimized support vector machine classifier: a step towards cognitive brain-computer interface

Cognitive brain-computer interface (cBCI) is an emerging area with applications in neurorehabilitation and performance monitoring. cBCI works on the cognitive brain signal that does not require a person to pay much effort unlike the motor brain-computer interface (BCI) however existing cBCI systems currently offer lower accuracy than the motor BCI. Since attention is one of the cognitive signals that can be used to realise the cBCI, this work uses the multiple object tracking (MOT) task to acquire the desired electroencephalograph (EEG) signal from healthy subjects. The main objective of the paper is to explore the preliminary applications of support vector machine (SVM) classifier to classify the attentional load in multiple object tracking task. Results show that the attentional load can be classified using SVM with sensitivity, specificity, and accuracy of 94.03%, 92.50%, and 93.28%, respectively using the spectral entropy EEG feature. The classification performance promises the potential application of the current approach in the cognitive brain-computer interface for neurorehabilitation.

Bridging M/EEG Source Imaging and Independent Component Analysis Frameworks Using Biologically Inspired Sparsity Priors

Electromagnetic source imaging (ESI) and independent component analysis (ICA) are two popular and apparently dissimilar frameworks for M/EEG analysis. This letter shows that the two frameworks can be linked by choosing biologically inspired source sparsity priors. We demonstrate that ESI carried out by the sparse Bayesian learning (SBL) algorithm yields source configurations composed of a few active regions that are also maximally independent from one another. In addition, we extend the standard SBL approach to source imaging in two important directions. First, we augment the generative model of M/EEG to include artifactual sources. Second, we modify SBL to allow for efficient model inversion with sequential data. We refer to this new algorithm as recursive SBL (RSBL), a source estimation filter with potential for online and offline imaging applications. We use simulated data to verify that RSBL can accurately estimate and demix cortical and artifactual sources under different noise conditions. Finally, we show that on real error-related EEG data, RSBL can yield single-trial source estimates in agreement with the experimental literature. Overall, by demonstrating that ESI can produce maximally independent sources while simultaneously localizing them in cortical space, we bridge the gap between the ESI and ICA frameworks for M/EEG analysis.

Benefits of deep learning classification of continuous noninvasive brain-computer interface control

Objective. Noninvasive brain-computer interfaces (BCIs) assist paralyzed patients by providing access to the world without requiring surgical intervention. Prior work has suggested that EEG motor imagery based BCI can benefit from increased decoding accuracy through the application of deep learning methods, such as convolutional neural networks (CNNs).Approach. Here, we examine whether these improvements can generalize to practical scenarios such as continuous control tasks (as opposed to prior work reporting one classification per trial), whether valuable information remains latent outside of the motor cortex (as no prior work has compared full scalp coverage to motor only electrode montages), and the existing challenges to the practical implementation of deep-learning based continuous BCI control.Main results. We report that: (1) deep learning methods significantly increase offline performance compared to standard methods on an independent, large, and longitudinal online motor imagery BCI dataset with up to 4-classes and continuous 2D feedback; (2) our results suggest that a variety of neural biomarkers for BCI, including those outside the motor cortex, can be detected and used to improve performance through deep learning methods, and (3) tuning neural network output will be an important step in optimizing online BCI control, as we found the CNN models trained with full scalp EEG also significantly reduce the average trial length in a simulated online cursor control environment.Significance. This work demonstrates the benefits of CNNs classification during BCI control while providing evidence that electrode montage selection and the mapping of CNN output to device control will be important design choices in CNN based BCIs.

Behavioral validation of novel high resolution attention decoding method from multi-units & local field potentials

The ability to access brain information in real-time is crucial both for a better understanding of cognitive functions and for the development of therapeutic applications based on brain-machine interfaces. Great success has been achieved in the field of neural motor prosthesis. Progress is still needed in the real-time decoding of higher-order cognitive processes such as covert attention. Recently, we showed that we can track the location of the attentional spotlight using classification methods applied to prefrontal multi-unit activity (MUA) in the non-human primates. Importantly, we demonstrated that the decoded (x,y) attentional spotlight parametrically correlates with the behavior of the monkeys thus validating our decoding of attention. We also demonstrate that this spotlight is extremely dynamic. Here, in order to get closer to non-invasive decoding applications, we extend our previous work to local field potential signals (LFP). Specifically, we achieve, for the first time, high decoding accuracy of the (x,y) location of the attentional spotlight from prefrontal LFP signals, to a degree comparable to that achieved from MUA signals, and we show that this LFP content is predictive of behavior. This LFP attention-related information is maximal in the gamma band (30-250 Hz), peaking between 60 to 120 Hz. In addition, we introduce a novel two-step decoding procedure based on the labelling of maximally attention-informative trials during the decoding procedure. This procedure strongly improves the correlation between our real-time MUA and LFP based decoding and behavioral performance, thus further refining the functional relevance of this real-time decoding of the (x,y) locus of attention. This improvement is more marked for LFP signals than for MUA signals. Overall, this study demonstrates that the attentional spotlight can be accessed from LFP frequency content, in real-time, and can be used to drive high-information content cognitive brain-machine interfaces for the development of new therapeutic strategies.

Impact of Stimulus Features on the Performance of a Gaze-Independent Brain-Computer Interface Based on Covert Spatial Attention Shifts

Regaining communication abilities in patients who are unable to speak or move is one of the main goals in decoding brain waves for brain-computer interface (BCI) control. Many BCI approaches designed for communication rely on attention to visual stimuli, commonly applying an oddball paradigm, and require both eye movements and adequate visual acuity. These abilities may, however, be absent in patients who depend on BCI communication. We have therefore developed a response-based communication BCI, which is independent of gaze shifts but utilizes covert shifts of attention to the left or right visual field. We recorded the electroencephalogram (EEG) from 29 channels and coregistered the vertical and horizontal electrooculogram. Data-driven decoding of small attention-based differences between the hemispheres, also known as N2pc, was performed using 14 posterior channels, which are expected to reflect correlates of visual spatial attention. Eighteen healthy participants responded to 120 statements by covertly directing attention to one of two colored symbols (green and red crosses for "yes" and "no," respectively), presented in the user's left and right visual field, respectively, while maintaining central gaze fixation. On average across participants, 88.5% (std: 7.8%) of responses were correctly decoded online. In order to investigate the potential influence of stimulus features on accuracy, we presented the symbols with different visual angles, by altering symbol size and eccentricity. The offline analysis revealed that stimulus features have a minimal impact on the controllability of the BCI. Hence, we show with our novel approach that spatial attention to a colored symbol is a robust method with which to control a BCI, which has the potential to support severely paralyzed people with impaired eye movements and low visual acuity in communicating with their environment.

Nonspecific Visuospatial Imagery as a Novel Mental Task for Online EEG-Based BCI Control

Brain-computer interfaces (BCIs) can provide a means of communication to individuals with severe motor disorders, such as those presenting as locked-in. Many BCI paradigms rely on motor neural pathways, which are often impaired in these individuals. However, recent findings suggest that visuospatial function may remain intact. This study aimed to determine whether visuospatial imagery, a previously unexplored task, could be used to signify intent in an online electroencephalography (EEG)-based BCI. Eighteen typically developed participants imagined checkerboard arrow stimuli in four quadrants of the visual field in 5-s trials, while signals were collected using 16 dry electrodes over the visual cortex. In online blocks, participants received graded visual feedback based on their performance. An initial BCI pipeline (visuospatial imagery classifier I) attained a mean accuracy of [Formula: see text]% classifying rest against visuospatial imagery in online trials. This BCI pipeline was further improved using restriction to alpha band features (visuospatial imagery classifier II), resulting in a mean pseudo-online accuracy of [Formula: see text]%. Accuracies exceeded the threshold for practical BCIs in 12 participants. This study supports the use of visuospatial imagery as a real-time, binary EEG-BCI control paradigm.

Spatial-temporal aspects of continuous EEG-based neurorobotic control

OBJECTIVE

The goal of this work is to identify the spatio-temporal facets of state-of-the-art electroencephalography (EEG)-based continuous neurorobotics that need to be addressed, prior to deployment in practical applications at home and in the clinic.

APPROACH

Nine healthy human subjects participated in five sessions of one-dimensional (1D) horizontal (LR), 1D vertical (UD) and two-dimensional (2D) neural tracking from EEG. Users controlled a robotic arm and virtual cursor to continuously track a Gaussian random motion target using EEG sensorimotor rhythm modulation via motor imagery (MI) commands. Continuous control quality was analyzed in the temporal and spatial domains separately.

MAIN RESULTS

Axis-specific errors during 2D tasks were significantly larger than during 1D counterparts. Fatigue rates were larger for control tasks with higher cognitive demand (LR, left- and right-hand MI) compared to those with lower cognitive demand (UD, both hands MI and rest). Additionally, robotic arm and virtual cursor control exhibited equal tracking error during all tasks. However, further spatial error analysis of 2D control revealed a significant reduction in tracking quality that was dependent on the visual interference of the physical device. In fact, robotic arm performance was significantly greater than that of virtual cursor control when the users' sightlines were not obstructed.

SIGNIFICANCE

This work emphasizes the need for practical interfaces to be designed around real-world tasks of increased complexity. Here, the dependence of control quality on cognitive task demand emphasizes the need for decoders that facilitate the translation of 1D task mastery to 2D control. When device footprint was accounted for, the introduction of a physical robotic arm improved control quality, likely due to increased user engagement. In general, this work demonstrates the need to consider both the physical footprint of devices, the complexity of training tasks, and the synergy of control strategies during the development of neurorobotic control.

Decoding the Temporal Dynamics of Covert Spatial Attention Using Multivariate EEG Analysis: Contributions of Raw Amplitude and Alpha Power

Attention can be oriented in space covertly without the need of eye movements. We used multivariate pattern classification analyses (MVPA) to investigate whether the time course of the deployment of covert spatial attention leading up to the observer’s perceptual decision can be decoded from both EEG alpha power and raw activity traces. Decoding attention from these signals can help determine whether raw EEG signals and alpha power reflect the same or distinct features of attentional selection. Using a classical cueing task, we showed that the orientation of covert spatial attention can be decoded by both signals. However, raw activity and alpha power may reflect different features of spatial attention, with alpha power more associated with the orientation of covert attention in space and raw activity with the influence of attention on perceptual processes.

Changes of Effective Connectivity in the Alpha Band Characterize Differential Processing of Audiovisual Information in Cross-Modal Selective Attention

Cross-modal selective attention enhances the processing of sensory inputs that are most relevant to the task at hand. Such differential processing could be mediated by a swift network reconfiguration on the macroscopic level, but this remains a poorly understood process. To tackle this issue, we used a behavioral paradigm to introduce a shift of selective attention between the visual and auditory domains, and recorded scalp electroencephalographic signals from eight healthy participants. The changes in effective connectivity caused by the cross-modal attentional shift were delineated by analyzing spectral Granger Causality (GC), a metric of frequency-specific effective connectivity. Using data-driven methods of pattern-classification and feature-analysis, we found that a change in the α band (12 Hz-15 Hz) of GC is a stable feature across different individuals that can be used to decode the attentional shift. Specifically, auditory attention induces more pronounced information flow in the α band, especially from the parietal-occipital areas to the temporal-parietal areas, compared to the case of visual attention, reflecting a reconfiguration of interaction in the macroscopic brain network accompanying different processing. Our results support the role of α oscillation in organizing the information flow across spatially-separated brain areas and, thereby, mediating cross-modal selective attention.

Computer data simulator to assess the accuracy of estimates of visual N2/N2pc event-related potential components

OBJECTIVE

Event-related potentials (ERPs) evoked by visual stimulations comprise several components, with different amplitudes and latencies. Among them, the N2 and N2pc components have been demonstrated to be a measure of subjects' allocation of visual attention to possible targets and to be involved in the suppression of irrelevant items. Unfortunately, the N2 and N2pc components have smaller amplitudes compared with those of the background electroencephalogram (EEG), and their measurement requires employing techniques such as conventional averaging, which in turn necessitates several sweeps to provide acceptable estimates. In visual search studies, the number of sweeps (Nswp) used to extrapolate reliable estimates of N2/N2pc components has always been somehow arbitrary, with studies using 50-500 sweeps. In-silico studies relying on synthetic data providing a close-to-realistic fit to the variability of the visual N2 component and background EEG signals are therefore needed to go beyond arbitrary choices in this context.

APPROACH

In the present work, we sought to take a step in this direction by developing a simulator of ERP variations in the N2 time range based on real experimental data while monitoring variations in the estimation accuracy of N2/N2pc components as a function of two factors, i.e. signal-to-noise ratio (SNR) and number of averaged sweeps.

MAIN RESULTS

The results revealed that both Nswp and SNR had a strong impact on the accuracy of N2/N2pc estimates. Critically, the present simulation showed that, for a given level of SNR, a non-arbitrary Nswp could be parametrically determined, after which no additional significant improvements in noise suppression and N2/N2pc accuracy estimation were observed.

SIGNIFICANCE

The present simulator is thought to provide investigators with quantitative guidelines for designing experimental protocols aimed at improving the detection accuracy of N2/N2pc components. The parameters of the simulator can be tuned, adapted, or integrated to fit other ERP modulations.

The N400 for brain computer interfacing: complexities and opportunities

The N400 is an event related potential that is evoked in response to conceptually meaningful stimuli. It is for instance more negative in response to incongruent than congruent words in a sentence, and more negative for unrelated than related words following a prime word. This sensitivity to semantic content of a stimulus in relation to the mental context of an individual makes it a signal of interest for Brain Computer Interfaces. A complicating aspect is the number of factors that can affect the N400 amplitude. In this paper, we provide an accessible overview of this range of N400 effects, and survey the three main BCI application areas that currently exploit the N400: (1) exploiting the semantic processing of faces to enhance matrix speller performance, (2) detecting language processing in patients with Disorders of Consciousness, and (3) using semantic stimuli to probe what is on a user's mind. Drawing on studies from these application areas, we illustrate that the N400 can successfully be exploited for BCI purposes, but that the signal-to-noise ratio is a limiting factor, with signal strength also varying strongly across subjects. Furthermore, we put findings in context of the general N400 literature, noting open questions and identifying opportunities for further research.

Spontaneous network activity <35 ​Hz accounts for variability in stimulus-induced gamma responses

Gamma activity is thought to serve several cognitive processes, including attention and memory. Even for the simplest stimulus, the occurrence of gamma activity is highly variable, both within and between individuals. The sources of this variability, however, are largely unknown. In this paper, we address one possible cause: the cross-frequency influence of spontaneous, whole-brain network activity on visual stimulus processing. By applying Hidden Markov modelling to MEG data, we reveal that the trial-averaged gamma response to a moving grating depends on the individual network dynamics, inferred from slower brain activity (<35 ​Hz) in the absence of stimulation (resting-state and task baseline). In addition, we demonstrate that modulations of network activity in task baseline influence the gamma response on the level of trials. In summary, our results reveal a cross-frequency and cross-session association between gamma responses induced by visual stimulation and spontaneous network activity. These findings underline the dependency of visual stimulus processing on the individual, functional network architecture.

Uncovering EEG Correlates of Covert Attention in Soccer Goalkeepers: Towards Innovative Sport Training Procedures

Advances in sports sciences and neurosciences offer new opportunities to design efficient and motivating sport training tools. For instance, using NeuroFeedback (NF), athletes can learn to self-regulate specific brain rhythms and consequently improve their performances. Here, we focused on soccer goalkeepers’ Covert Visual Spatial Attention (CVSA) abilities, which are essential for these athletes to reach high performances. We looked for Electroencephalography (EEG) markers of CVSA usable for virtual reality-based NF training procedures, i.e., markers that comply with the following criteria: (1) specific to CVSA, (2) detectable in real-time and (3) related to goalkeepers’ performance/expertise. Our results revealed that the best-known EEG marker of CVSA—increased α-power ipsilateral to the attended hemi-field— was not usable since it did not comply with criteria 2 and 3. Nonetheless, we highlighted a significant positive correlation between athletes’ improvement in CVSA abilities and the increase of their α-power at rest. While the specificity of this marker remains to be demonstrated, it complied with both criteria 2 and 3. This result suggests that it may be possible to design innovative ecological training procedures for goalkeepers, for instance using a combination of NF and cognitive tasks performed in virtual reality.

More than Meets the Mind's Eye? Preliminary Observations Hint at Heterogeneous Alpha Neuromarkers for Visual Attention

With their salient power distribution and privileged timescale for cognition and behavior, brainwaves within the 10 Hz band are special in human waking electroencephalography (EEG). From the inception of electroencephalographic technology, the contribution of alpha rhythm to attention is well-known: Its amplitude increases when visual attention wanes or visual input is removed. However, alpha is not alone in the 10 Hz frequency band. A number of other 10 Hz neuromarkers have function and topography clearly distinct from alpha. In small pilot studies, an activity that we named xi was found over left centroparietal scalp regions when subjects held their attention to spatially peripheral locations while maintaining their gaze centrally ("looking from the corner of the eyes"). I outline several potential functions for xi as a putative neuromarker of covert attention distinct from alpha. I review methodological aids to test and validate their functional role. They emphasize high spectral resolution, sufficient spatial resolution to provide topographical separation, and an acute attention to dynamics that caters to neuromarkers' transiency.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Inter-subject transfer learning with an end-to-end deep convolutional neural network for EEG-based BCI

OBJECTIVE

Despite the effective application of deep learning (DL) in brain-computer interface (BCI) systems, the successful execution of this technique, especially for inter-subject classification, in cognitive BCI has not been accomplished yet. In this paper, we propose a framework based on the deep convolutional neural network (CNN) to detect the attentive mental state from single-channel raw electroencephalography (EEG) data.

APPROACH

We develop an end-to-end deep CNN to decode the attentional information from an EEG time series. We also explore the consequences of input representations on the performance of deep CNN by feeding three different EEG representations into the network. To ensure the practical application of the proposed framework and avoid time-consuming re-training, we perform inter-subject transfer learning techniques as a classification strategy. Eventually, to interpret the learned attentional patterns, we visualize and analyse the network perception of the attention and non-attention classes.

MAIN RESULTS

The average classification accuracy is 79.26%, with only 15.83% of 120 subjects having an accuracy below 70% (a generally accepted threshold for BCI). This is while with the inter-subject approach, it is literally difficult to output high classification accuracy. This end-to-end classification framework surpasses conventional classification methods for attention detection. The visualization results demonstrate that the learned patterns from the raw data are meaningful.

SIGNIFICANCE

This framework significantly improves attention detection accuracy with inter-subject classification. Moreover, this study sheds light on the research on end-to-end learning; the proposed network is capable of learning from raw data with the least amount of pre-processing, which in turn eliminates the extensive computational load of time-consuming data preparation and feature extraction.

Three-Dimensional Brain-Computer Interface Control Through Simultaneous Overt Spatial Attentional and Motor Imagery Tasks

OBJECTIVE

While noninvasive electroenceph-alography (EEG) based brain-computer interfacing (BCI) has been successfully demonstrated in two-dimensional (2-D) control tasks, little work has been published regarding its extension to practical three-dimensional (3-D) control.

METHODS

In this study, we developed a new BCI approach for 3-D control by combining a novel form of endogenous visuospatial attentional modulation, defined as overt spatial attention (OSA), and motor imagery (MI).

RESULTS

OSA modulation was shown to provide comparable control to conventional MI modulation in both 1-D and 2-D tasks. Furthermore, this paper provides evidence for the functional independence of traditional MI and OSA, as well as an investigation into the simultaneous use of both. Using this newly proposed BCI paradigm, 16 participants successfully completed a 3-D eight-target control task. Nine of these subjects further demonstrated robust 3-D control in a 12-target task, significantly outperforming the information transfer rate achieved in the 1-D and 2-D control tasks (29.7 ± 1.6 b/min).

CONCLUSION

These results strongly support the hypothesis that noninvasive EEG-based BCI can provide robust 3-D control through endogenous neural modulation in broader populations with limited training.

SIGNIFICANCE

Through the combination of the two strategies (MI and OSA), a substantial portion of the recruited subjects were capable of robustly controlling a virtual cursor in 3-D space. The proposed novel approach could broaden the dimensionality of BCI control and shorten the training time.

Geometric classification of brain network dynamics via conic derivative discriminants

BACKGROUND

Over the past decade, pattern decoding techniques have granted neuroscientists improved anatomical specificity in mapping neural representations associated with function and cognition. Dynamical patterns are of particular interest, as evidenced by the proliferation and success of frequency domain methods that reveal structured spatiotemporal rhythmic brain activity. One drawback of such approaches, however, is the need to estimate spectral power, which limits the temporal resolution of classification.

NEW METHOD

We propose an alternative method that enables classification of dynamical patterns with high temporal fidelity. The key feature of the method is a conversion of time-series data into temporal derivatives. By doing so, dynamically-coded information may be revealed in terms of geometric patterns in the phase space of the derivative signal.

RESULTS

We derive a geometric classifier for this problem which simplifies into a straightforward calculation in terms of covariances. We demonstrate the relative advantages and disadvantages of the technique with simulated data and benchmark its performance with an EEG dataset of covert spatial attention. We reveal the timecourse of covert spatial attention and, by mapping the classifier weights anatomically, its retinotopic organization.

COMPARISON WITH EXISTING METHOD

We especially highlight the ability of the method to provide strong group-level classification performance compared to existing benchmarks, while providing information that is complementary with classical spectral-based techniques. The robustness and sensitivity of the method to noise is also examined relative to spectral-based techniques.

CONCLUSION

The proposed classification technique enables decoding of dynamic patterns with high temporal resolution, performs favorably to benchmark methods, and facilitates anatomical inference.

Classification of Targets and Distractors Present in Visual Hemifields Using Time-Frequency Domain EEG Features

This paper presents a classification system to classify the cognitive load corresponding to targets and distractors present in opposite visual hemifields. The approach includes the study of EEG (electroencephalogram) signal features acquired in a spatial attention task. The process comprises of EEG feature selection based on the feature distribution, followed by the stepwise discriminant analysis- (SDA-) based channel selection. Repeated measure analysis of variance (rANOVA) is applied to test the statistical significance of the selected features. Classifiers are developed and compared using the selected features to classify the target and distractor present in visual hemifields. The results provide a maximum classification accuracy of 87.2% and 86.1% and an average classification accuracy of 76.5 ± 4% and 76.2 ± 5.3% over the thirteen subjects corresponding to the two task conditions. These correlates present a step towards building a feature-based neurofeedback system for visual attention.

Brain-computer interaction for online enhancement of visuospatial attention performance

OBJECTIVE

this study on real-time decoding of visuospatial attention has two objectives: first, to reliably decode self-directed shifts of attention from electroencephalography (EEG) data, and second, to analyze whether this information can be used to enhance visuospatial performance. Visuospatial performance was measured in a target orientation discrimination task, in terms of reaction time, and error rate.

APPROACH

Our experiment extends the Posner paradigm by introducing a new type of ambiguous cues to indicate upcoming target location. The cues are designed so that their ambiguity is imperceptible to the user. This entails endogenous shifts of attention which are truly self-directed. Two protocols were implemented to exploit the decoding of attention shifts. The first 'adaptive' protocol uses the decoded locus to display the target. In the second 'warning' protocol, the target position is defined in advance, but a warning is flashed when the target mismatches the decoded locus.

MAIN RESULTS

Both protocols were tested in an online experiment involving ten subjects. The reaction time improved in both the adaptive and the warning protocol. The error rate was improved in the adaptive protocol only.

SIGNIFICANCE

This proof of concept study brings evidence that visuospatial brain-computer interfaces (BCIs) can be used to enhance improving human-machine interaction in situations where humans must react to off-center events in the visual field.

Real-time decoding of covert attention in higher-order visual areas

Brain-computer-interfaces (BCI) provide a means of using human brain activations to control devices for communication. Until now this has only been demonstrated in primary motor and sensory brain regions, using surgical implants or non-invasive neuroimaging techniques. Here, we provide proof-of-principle for the use of higher-order brain regions involved in complex cognitive processes such as attention. Using realtime fMRI, we implemented an online 'winner-takes-all approach' with quadrant-specific parameter estimates, to achieve single-block classification of brain activations. These were linked to the covert allocation of attention to real-world images presented at 4-quadrant locations. Accuracies in three target regions were significantly above chance, with individual decoding accuracies reaching upto 70%. By utilising higher order mental processes, 'cognitive BCIs' access varied and therefore more versatile information, potentially providing a platform for communication in patients who are unable to speak or move due to brain injury.

EEG-based neglect assessment: A feasibility study

BACKGROUND

Spatial neglect (SN) is a neuropsychological syndrome that impairs automatic attention orienting to stimuli in the contralesional visual space of stroke patients. SN is commonly assessed using paper and pencil tests. Recently, computerized tests have been proposed to provide a dynamic assessment of SN. However, both paper- and computer-based methods have limitations.

NEW METHOD

Electroencephalography (EEG) shows promise for overcoming the limitations of current assessment methods. The aim of this work is to introduce an objective passive BCI system that records EEG signals in response to visual stimuli appearing in random locations on a screen with a dynamically changing background. Our preliminary experimental studies focused on validating the system using healthy participants with intact brains rather than employing it initially in more complex environments with patients having cortical lesions. Therefore, we designed a version of the test in which we simulated SN by hiding target stimuli appearing on the left side of the screen so that the subject's attention is shifted to the right side.

RESULTS

Results showed that there are statistically significant differences between EEG responses due to right and left side stimuli reflecting different processing and attention levels towards both sides of the screen. The system achieved average accuracy, sensitivity and specificity of 74.24%, 75.17% and 71.36% respectively.

COMPARISON WITH EXISTING METHODS

The proposed test can examine both presence and severity of SN, unlike traditional paper and pencil tests and computer-based methods.

CONCLUSIONS

The proposed test is a promising objective SN evaluation method.

Asymmetrical Electroencephalographic Change of Human Brain During Sleep Onset Period

OBJECTIVE

Human cerebral hemisphere is known to function asymmetrically with daytime left hemisphere superiority in most right-handed persons. It may have relevance to the localization of specific function of the brain. This study attempted to reveal whether the functional cerebral asymmetry in the wakeful state is still maintained throughout the sleep onset period.

METHODS

Thirty-channel EEG was recorded in 61 healthy subjects. The EEG power spectra of each of the seven frequencies were compared between the two kinds of 30-second states; the wakeful stage and the late-sleep stage 1.

RESULTS

The asymmetrical indices of sleep stage 1 at several fronto-central leads were decreased in the delta, theta, alpha-2, and all beta bands. Conversely, at parts of parieto-occipital leads showed an increase in the indices of the theta, alphas, beta-1, and beta-2 bands. Any fronto-central leads did not show an increase in the index, and no parieto-occipital leads showed a decrease.

CONCLUSION

During the sleep onset period, power spectral asymmetry of the brain showed a different pattern from the wakeful stage. This asymmetrical pattern of EEG powers may suggest a reversal of the left hemispheric dominance during sleep.

Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- ) than the NM-RSVP (i.e., 480.4 ms; ). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.

Toward a hybrid brain-computer interface based on repetitive visual stimuli with missing events

Background

Steady-state visually evoked potentials (SSVEPs) can be elicited by repetitive stimuli and extracted in the frequency domain with satisfied performance. However, the temporal information of such stimulus is often ignored. In this study, we utilized repetitive visual stimuli with missing events to present a novel hybrid BCI paradigm based on SSVEP and omitted stimulus potential (OSP).

Methods

Four discs flickering from black to white with missing flickers served as visual stimulators to simultaneously elicit subject’s SSVEPs and OSPs. Key parameters in the new paradigm, including flicker frequency, optimal electrodes, missing flicker duration and intervals of missing events were qualitatively discussed with offline data. Two omitted flicker patterns including missing black/white disc were proposed and compared. Averaging times were optimized with Information Transfer Rate (ITR) in online experiments, where SSVEPs and OSPs were identified using Canonical Correlation Analysis in the frequency domain and Support Vector Machine (SVM)-Bayes fusion in the time domain, respectively.

Results and conclusions

The online accuracy and ITR (mean ± standard deviation) over nine healthy subjects were 79.29 ± 18.14 % and 19.45 ± 11.99 bits/min with missing black disc pattern, and 86.82 ± 12.91 % and 24.06 ± 10.95 bits/min with missing white disc pattern, respectively. The proposed BCI paradigm, for the first time, demonstrated that SSVEPs and OSPs can be simultaneously elicited in single visual stimulus pattern and recognized in real-time with satisfied performance. Besides the frequency features such as SSVEP elicited by repetitive stimuli, we found a new feature (OSP) in the time domain to design a novel hybrid BCI paradigm by adding missing events in repetitive stimuli.

Decoding and Reconstructing the Focus of Spatial Attention from the Topography of Alpha-band Oscillations

Many aspects of perception and cognition are supported by activity in neural populations that are tuned to different stimulus features (e.g., orientation, spatial location, color). Goal-directed behavior, such as sustained attention, requires a mechanism for the selective prioritization of contextually appropriate representations. A candidate mechanism of sustained spatial attention is neural activity in the alpha band (8-13 Hz), whose power in the human EEG covaries with the focus of covert attention. Here, we applied an inverted encoding model to assess whether spatially selective neural responses could be recovered from the topography of alpha-band oscillations during spatial attention. Participants were cued to covertly attend to one of six spatial locations arranged concentrically around fixation while EEG was recorded. A linear classifier applied to EEG data during sustained attention demonstrated successful classification of the attended location from the topography of alpha power, although not from other frequency bands. We next sought to reconstruct the focus of spatial attention over time by applying inverted encoding models to the topography of alpha power and phase. Alpha power, but not phase, allowed for robust reconstructions of the specific attended location beginning around 450 msec postcue, an onset earlier than previous reports. These results demonstrate that posterior alpha-band oscillations can be used to track activity in feature-selective neural populations with high temporal precision during the deployment of covert spatial attention.

Comparison of Athletes with and without Burnout Using the Stroop Color and Word Test

The present study compared brain activity of adolescents with or without burnout during their responses to a computerized version of the Stroop Color and Word Test. The Sport Adaptation of the Maslach Burnout Inventory was administered to 460 Korean high school student athletes. Electroencephalographic data were recorded from frontal, central, parietal, and occipital brain regions while these participants were performing the Stroop Color and Word Test. A 2 (group) × 2 (condition) × 15 (electrodes) three-way analysis of variance was used to analyze the data. Results indicated that the athletes without burnout exhibited significantly higher accuracy than their counterparts with burnout on the Stroop Color and Word Test. The athletes without burnout also showed higher amplitudes for theta, alpha, and beta power in the frontal areas than the athletes with burnout.

Modulation of Posterior Alpha Activity by Spatial Attention Allows for Controlling A Continuous Brain-Computer Interface

Here we report that the modulation of alpha activity by covert attention can be used as a control signal in an online brain-computer interface, that it is reliable, and that it is robust. Subjects were instructed to orient covert visual attention to the left or right hemifield. We decoded the direction of attention from the magnetoencephalogram by a template matching classifier and provided the classification outcome to the subject in real-time using a novel graphical user interface. Training data for the templates were obtained from a Posner-cueing task conducted just before the BCI task. Eleven subjects participated in four sessions each. Eight of the subjects achieved classification rates significantly above chance level. Subjects were able to significantly increase their performance from the first to the second session. Individual patterns of posterior alpha power remained stable throughout the four sessions and did not change with increased performance. We conclude that posterior alpha power can successfully be used as a control signal in brain-computer interfaces. We also discuss several ideas for further improving the setup and propose future research based on solid hypotheses about behavioral consequences of modulating neuronal oscillations by brain computer interfacing.

Selective visual attention to drive cognitive brain-machine interfaces: from concepts to neurofeedback and rehabilitation applications

Brain–machine interfaces (BMIs) using motor cortical activity to drive an external effector like a screen cursor or a robotic arm have seen enormous success and proven their great rehabilitation potential. An emerging parallel effort is now directed to BMIs controlled by endogenous cognitive activity, also called cognitive BMIs. While more challenging, this approach opens new dimensions to the rehabilitation of cognitive disorders. In the present work, we focus on BMIs driven by visuospatial attention signals and we provide a critical review of these studies in the light of the accumulated knowledge about the psychophysics, anatomy, and neurophysiology of visual spatial attention. Importantly, we provide a unique comparative overview of the several studies, ranging from non-invasive to invasive human and non-human primates studies, that decode attention-related information from ongoing neuronal activity. We discuss these studies in the light of the challenges attention-driven cognitive BMIs have to face. In a second part of the review, we discuss past and current attention-based neurofeedback studies, describing both the covert effects of neurofeedback onto neuronal activity and its overt behavioral effects. Importantly, we compare neurofeedback studies based on the amplitude of cortical activity to studies based on the enhancement of cortical information content. Last, we discuss several lines of future research and applications for attention-driven cognitive brain-computer interfaces (BCIs), including the rehabilitation of cognitive deficits, restored communication in locked-in patients, and open-field applications for enhanced cognition in normal subjects. The core motivation of this work is the key idea that the improvement of current cognitive BMIs for therapeutic and open field applications needs to be grounded in a proper interdisciplinary understanding of the physiology of the cognitive function of interest, be it spatial attention, working memory or any other cognitive signal.

Resting alpha activity predicts learning ability in alpha neurofeedback

Individuals differ in their ability to learn how to regulate the brain activity by neurofeedback. This study aimed to investigate whether the resting alpha activity can predict the learning ability in alpha neurofeedback. A total of 25 subjects performed 20 sessions of individualized alpha neurofeedback and the learning ability was assessed by three indices respectively: the training parameter changes between two periods, within a short period and across the whole training time. It was found that the resting alpha amplitude measured before training had significant positive correlations with all learning indices and could be used as a predictor for the learning ability prediction. This finding would help the researchers in not only predicting the training efficacy in individuals but also gaining further insight into the mechanisms of alpha neurofeedback.

Coherent Activity in Bilateral Parieto-Occipital Cortices during P300-BCI Operation

The visual P300 brain–computer interface (BCI), a popular system for electroencephalography (EEG)-based BCI, uses the P300 event-related potential to select an icon arranged in a flicker matrix. In earlier studies, we used green/blue (GB) luminance and chromatic changes in the P300-BCI system and reported that this luminance and chromatic flicker matrix was associated with better performance and greater subject comfort compared with the conventional white/gray (WG) luminance flicker matrix. To highlight areas involved in improved P300-BCI performance, we used simultaneous EEG–fMRI recordings and showed enhanced activities in bilateral and right lateralized parieto-occipital areas. Here, to capture coherent activities of the areas during P300-BCI, we collected whole-head 306-channel magnetoencephalography data. When comparing functional connectivity between the right and left parieto-occipital channels, significantly greater functional connectivity in the alpha band was observed under the GB flicker matrix condition than under the WG flicker matrix condition. Current sources were estimated with a narrow-band adaptive spatial filter, and mean imaginary coherence was computed in the alpha band. Significantly greater coherence was observed in the right posterior parietal cortex under the GB than under the WG condition. Re-analysis of previous EEG-based P300-BCI data showed significant correlations between the power of the coherence of the bilateral parieto-occipital cortices and their performance accuracy. These results suggest that coherent activity in the bilateral parieto-occipital cortices plays a significant role in effectively driving the P300-BCI.

Exploring the impact of target eccentricity and task difficulty on covert visual spatial attention and its implications for brain computer interfacing

Objective

Covert visual spatial attention is a relatively new task used in brain computer interfaces (BCIs) and little is known about the characteristics which may affect performance in BCI tasks. We investigated whether eccentricity and task difficulty affect alpha lateralization and BCI performance.

Approach

We conducted a magnetoencephalography study with 14 participants who performed a covert orientation discrimination task at an easy or difficult stimulus contrast at either a near (3.5°) or far (7°) eccentricity. Task difficulty was manipulated block wise and subjects were aware of the difficulty level of each block.

Main Results

Grand average analyses revealed a significantly larger hemispheric lateralization of posterior alpha power in the difficult condition than in the easy condition, while surprisingly no difference was found for eccentricity. The difference between task difficulty levels was significant in the interval between 1.85 s and 2.25 s after cue onset and originated from a stronger decrease in the contralateral hemisphere. No significant effect of eccentricity was found. Additionally, single-trial classification analysis revealed a higher classification rate in the difficult (65.9%) than in the easy task condition (61.1%). No effect of eccentricity was found in classification rate.

Significance

Our results indicate that manipulating the difficulty of a task gives rise to variations in alpha lateralization and that using a more difficult task improves covert visual spatial attention BCI performance. The variations in the alpha lateralization could be caused by different factors such as an increased mental effort or a higher visual attentional demand. Further research is necessary to discriminate between them. We did not discover any effect of eccentricity in contrast to results of previous research.

The cost of space independence in P300-BCI spellers

Background

Though non-invasive EEG-based Brain Computer Interfaces (BCI) have been researched extensively over the last two decades, most designs require control of spatial attention and/or gaze on the part of the user.

Methods

In healthy adults, we compared the offline performance of a space-independent P300-based BCI for spelling words using Rapid Serial Visual Presentation (RSVP), to the well-known space-dependent Matrix P300 speller.

Results

EEG classifiability with the RSVP speller was as good as with the Matrix speller. While the Matrix speller’s performance was significantly reliant on early, gaze-dependent Visual Evoked Potentials (VEPs), the RSVP speller depended only on the space-independent P300b. However, there was a cost to true spatial independence: the RSVP speller was less efficient in terms of spelling speed.

Conclusions

The advantage of space independence in the RSVP speller was concomitant with a marked reduction in spelling efficiency. Nevertheless, with key improvements to the RSVP design, truly space-independent BCIs could approach efficiencies on par with the Matrix speller. With sufficiently high letter spelling rates fused with predictive language modelling, they would be viable for potential applications with patients unable to direct overt visual gaze or covert attentional focus.