A P300-based EEG-BCI for spatial navigation control

The Current Research of Combining Multi-Modal Brain-Computer Interfaces With Virtual Reality

Combing brain-computer interfaces (BCI) and virtual reality (VR) is a novel technique in the field of medical rehabilitation and game entertainment. However, the limitations of BCI such as a limited number of action commands and low accuracy hinder the widespread use of BCI-VR. Recent studies have used hybrid BCIs that combine multiple BCI paradigms and/or the multi-modal biosensors to alleviate these issues, which may become the mainstream of BCIs in the future. The main purpose of this review is to discuss the current status of multi-modal BCI-VR. This study first reviewed the development of the BCI-VR, and explored the advantages and disadvantages of incorporating eye tracking, motor capture, and myoelectric sensing into the BCI-VR system. Then, this study discussed the development trend of the multi-modal BCI-VR, hoping to provide a pathway for further research in this field.

Riemann-Based Algorithms Assessment for Single- and Multiple-Trial P300 Classification in Non-Optimal Environments

The P300 wave is commonly used in Brain-Computer Interface technology due to its higher bit rates when compared to other BCI paradigms. P300 classification pipelines based on Riemannian Geometry provide accuracies on par with state-of-the-art pipelines, without having the need for spatial filters, and also possess the ability to be calibrated with little data. In this study, five different P300 detection pipelines are compared, with three of them using Riemannian Geometry as either feature extraction or classification algorithms. The goal of this study is to assess the viability of Riemannian Geometry-based methods in non-optimal environments with sudden background noise changes, rather than maximizing classification accuracy values. For fifteen subjects, the average single-trial accuracy obtained for each pipeline was: 56.06% for Linear Discriminant Analysis (LDA), 72.13% for Bayesian Linear Discriminant Analysis (BLDA), 63.56% for Riemannian Minimum Distance to Mean (MDM), 69.22% for Riemannian Tangent Space with Logistic Regression (TS-LogR), and 63.30% for Riemannian Tangent Space with Support Vector Machine (TS-SVM). The results are higher for the pipelines based on BLDA and TS-LogR, suggesting that they could be viable methods for the detection of the P300 component when maximizing the bit rate is needed. For multiple-trial classification, the BLDA pipeline converged faster towards higher average values, closely followed by the TS-LogR pipeline. The two remaining Riemannian methods' accuracy also increases with the number of trials, but towards a lower value compared to the aforementioned ones. Single-stimulus detection metrics revealed that the TS-LogR pipeline can be a viable classification method, as its results are only slightly lower than those obtained with BLDA. P300 waveforms were also analyzed to check for evidence of the component being elicited. Finally, a questionnaire was used to retrieve the most intuitive focusing methods employed by the subjects.

The Role of Audio-Visual Feedback in a Thought-Based Control of a Humanoid Robot: A BCI Study in Healthy and Spinal Cord Injured People

The efficient control of our body and successful interaction with the environment are possible through the integration of multisensory information. Brain-computer interface (BCI) may allow people with sensorimotor disorders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and-place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot's perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous/asynchronous auditory feedback. Healthy participants achieved better placing accuracy when listening to human footstep sounds relative to a computer-generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feedback conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to understand how the contribution of residual sensory channels could improve the reliability of BCI systems.

Audio-visual feedback improves the BCI performance in the navigational control of a humanoid robot

Advancement in brain computer interfaces (BCI) technology allows people to actively interact in the world through surrogates. Controlling real humanoid robots using BCI as intuitively as we control our body represents a challenge for current research in robotics and neuroscience. In order to successfully interact with the environment the brain integrates multiple sensory cues to form a coherent representation of the world. Cognitive neuroscience studies demonstrate that multisensory integration may imply a gain with respect to a single modality and ultimately improve the overall sensorimotor performance. For example, reactivity to simultaneous visual and auditory stimuli may be higher than to the sum of the same stimuli delivered in isolation or in temporal sequence. Yet, knowledge about whether audio-visual integration may improve the control of a surrogate is meager. To explore this issue, we provided human footstep sounds as audio feedback to BCI users while controlling a humanoid robot. Participants were asked to steer their robot surrogate and perform a pick-and-place task through BCI-SSVEPs. We found that audio-visual synchrony between footsteps sound and actual humanoid's walk reduces the time required for steering the robot. Thus, auditory feedback congruent with the humanoid actions may improve motor decisions of the BCI's user and help in the feeling of control over it. Our results shed light on the possibility to increase robot's control through the combination of multisensory feedback to a BCI user.

Detection of attention shift for asynchronous P300-based BCI

Brain-computer interface (BCI) provides patients suffering from severe neuromuscular disorders an alternative way of interacting with the outside world. The P300-based BCI is among the most popular paradigms in the field and most current versions operate in synchronous mode and assume participant engagement throughout operation. In this study, we demonstrate a new approach for assessment of user engagement through a hybrid classification of ERP and band power features of EEG signals that could allow building asynchronous BCIs. EEG signals from nine electrode locations were recorded from nine participants during controlled engagement conditions when subjects were either engaged with the P3speller task or not attending. Statistical analysis of band power showed that there were significant contrasts of attending only for the delta and beta bands as indicators of features for user attendance classification. A hybrid classifier using ERP scores and band power features yielded the best overall performance of 0.98 in terms of the area under the ROC curve (AUC). Results indicate that band powers can provide additional discriminant information to the ERP for user attention detection and this combined approach can be used to assess user engagement for each stimulus sequence during BCI use.