A Novel Dictionary-Driven Mental Spelling Application Based on Code-Modulated Visual Evoked Potentials

A Bayesian dynamic stopping method for evoked response brain-computer interfacing

Introduction

As brain-computer interfacing (BCI) systems transition fromassistive technology to more diverse applications, their speed, reliability, and user experience become increasingly important. Dynamic stopping methods enhance BCI system speed by deciding at any moment whether to output a result or wait for more information. Such approach leverages trial variance, allowing good trials to be detected earlier, thereby speeding up the process without significantly compromising accuracy. Existing dynamic stopping algorithms typically optimize measures such as symbols per minute (SPM) and information transfer rate (ITR). However, these metrics may not accurately reflect system performance for specific applications or user types. Moreover, many methods depend on arbitrary thresholds or parameters that require extensive training data.

Methods

We propose a model-based approach that takes advantage of the analytical knowledge that we have about the underlying classification model. By using a risk minimization approach, our model allows precise control over the types of errors and the balance between precision and speed. This adaptability makes it ideal for customizing BCI systems to meet the diverse needs of various applications.

Results and discussion

We validate our proposed method on a publicly available dataset, comparing it with established static and dynamic stopping methods. Our results demonstrate that our approach offers a broad range of accuracy-speed trade-offs and achieves higher precision than baseline stopping methods.

Burst c-VEP Based BCI: Optimizing stimulus design for enhanced classification with minimal calibration data and improved user experience

The utilization of aperiodic flickering visual stimuli under the form of code-modulated Visual Evoked Potentials (c-VEP) represents a pivotal advancement in the field of reactive Brain-Computer Interface (rBCI). A major advantage of the c-VEP approach is that the training of the model is independent of the number and complexity of targets, which helps reduce calibration time. Nevertheless, the existing designs of c-VEP stimuli can be further improved in terms of visual user experience but also to achieve a higher signal-to-noise ratio, while shortening the selection time and calibration process. In this study, we introduce an innovative variant of code-VEP, referred to as "Burst c-VEP". This original approach involves the presentation of short bursts of aperiodic visual flashes at a deliberately slow rate, typically ranging from two to four flashes per second. The rationale behind this design is to leverage the sensitivity of the primary visual cortex to transient changes in low-level stimuli features to reliably elicit distinctive series of visual evoked potentials. In comparison to other types of faster-paced code sequences, burst c-VEP exhibit favorable properties to achieve high bitwise decoding performance using convolutional neural networks (CNN), which yields potential to attain faster selection time with the need for less calibration data. Furthermore, our investigation focuses on reducing the perceptual saliency of c-VEP through the attenuation of visual stimuli contrast and intensity to significantly improve users' visual comfort. The proposed solutions were tested through an offline 4-classes c-VEP protocol involving 12 participants. Following a factorial design, participants were instructed to focus on c-VEP targets whose pattern (burst and maximum-length sequences) and amplitude (100% or 40% amplitude depth modulations) were manipulated across experimental conditions. Firstly, the full amplitude burst c-VEP sequences exhibited higher accuracy, ranging from 90.5% (with 17.6s of calibration data) to 95.6% (with 52.8s of calibration data), compared to its m-sequence counterpart (71.4% to 85.0%). The mean selection time for both types of codes (1.5 s) compared favorably to reports from previous studies. Secondly, our findings revealed that lowering the intensity of the stimuli only slightly decreased the accuracy of the burst code sequences to 94.2% while leading to substantial improvements in terms of user experience. Taken together, these results demonstrate the high potential of the proposed burst codes to advance reactive BCI both in terms of performance and usability. The collected dataset, along with the proposed CNN architecture implementation, are shared through open-access repositories.

Nonparametric Early Stopping Detection for c-VEP-based Brain-Computer Interfaces: A Pilot Study

Brain-computer interface (BCI) systems based on code-modulated visual evoked potentials (c-VEP) stand out for achieving excellent command selection accuracies with very short calibration times. One of the natural steps to democratize their use in plug-and-play environments is to develop early stopping algorithms. These methods allow real-time detection of the minimum number of code repetitions needed to provide reliable selections. However, such techniques are scarce in the current state-of-the-art for c-VEP-based BCI systems based on the classical circular shifting paradigm. Here, a novel nonparametric early stopping method is proposed, which approximates the distribution of unattended commands to a normal distribution and issues a selection when the correlation of the command is considered an outlier. The proposal has been evaluated offline with 15 healthy users, achieving an average accuracy of 97.08% and a speed of 1.37 s/command. Likewise, the algorithm has also been evaluated with an additional user in an online way, as a proof of concept to validate its technical feasibility, achieving an average accuracy of 96.88% with a speed of 1.67 s/command. These results suggest that the real time application of the proposed algorithm is feasible, significantly reducing the required selection time without compromising accuracy.

Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review

Objective.Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain-computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines.Approach.The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc.Main results.The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development.Significance.Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

Asynchronous c-VEP communication tools-efficiency comparison of low-target, multi-target and dictionary-assisted BCI spellers

Keyboards and smartphones allow users to express their thoughts freely via manual control. Hands-free communication can be realized with brain-computer interfaces (BCIs) based on code-modulated visual evoked potentials (c-VEPs). Various variations of such spellers have been developed: Low-target systems, multi-target systems and systems with dictionary support. In general, it is not clear which kinds of systems are optimal in terms of reliability, speed, cognitive load, and visual load. The presented study investigates the feasibility of different speller variations. 58 users tested a 4-target speller and a 32-target speller with and without dictionary functionality. For classification, multiple individualized spatial filters were generated via canonical correlation analysis (CCA). We used an asynchronous implementation allowing non-control state, thus aiming for high accuracy rather than speed. All users were able to control the tested spellers. Interestingly, no significant differences in accuracy were found: 94.4%, 95.5% and 94.0% for 4-target spelling, 32-target spelling, and dictionary-assisted 32-target spelling. The mean ITRs were highest for the 32-target interface: 45.2, 96.9 and 88.9 bit/min. The output speed in characters per minute, was highest in dictionary-assisted spelling: 8.2, 19.5 and 31.6 characters/min. According to questionnaire results, 86% of the participants preferred the 32-target speller over the 4-target speller.

Towards solving of the Illiteracy phenomenon for VEP-based brain-computer interfaces

Brain-Computer Interface (BCI) systems use brain activity as an input signal and enable communication without requiring bodily movement. This novel technology may help impaired patients and users with disabilities to communicate with their environment. Over the years, researchers investigated the performance of subjects in different BCI paradigms, stating that 15%-30% of BCI users are unable to reach proficiency in using a BCI system and therefore were labelled as BCI illiterates. Recent progress in the BCIs based on the visually evoked potentials (VEPs) necessitates re-considering of this term, as very often all subjects are able to use VEP-based BCI systems. This study examines correlations among BCI performance, personal preferences, and further demographic factors for three different modern visually evoked BCI paradigms: (1) the conventional Steady-State Visual Evoked Potentials (SSVEPs) based on visual stimuli flickering at specific constant frequencies (fVEP), (2) Steady-State motion Visual Evoked Potentials (SSmVEP), and (3) code-modulated Visual Evoked Potentials (cVEP). Demographic parameters, as well as handedness, vision correction, BCI experience, etc., have no significant effect on the performance of VEP-based BCI. Most subjects did not consider the flickering stimuli annoying, only 20 out of a total of 86 participants indicated a change in fatigue during the experiment. 83 subjects were able to successfully finish all spelling tasks with the fVEP speller, with a mean (SD) information transfer rate of 31.87 bit/min (9.83) and an accuracy of 95.28% (5.18), respectively. Compared to that, 80 subjects were able to successfully finish all spelling tasks using SSmVEP, with a mean information transfer rate of 26.44 bit/min (8.04) and an accuracy of 91.10% (6.01), respectively. Finally, all 86 subjects were able to successfully finish all spelling tasks with the cVEP speller, with a mean information transfer rate of 40.23 bit/min (7.63) and an accuracy of 97.83% (3.37).

Five Shades of Grey: Exploring Quintary m-Sequences for More User-Friendly c-VEP-Based BCIs

Responsive EEG-based communication systems have been implemented with brain-computer interfaces (BCIs) based on code-modulated visual evoked potentials (c-VEPs). The BCI targets are typically encoded with binary m-sequences because of their autocorrelation property; the digits one and zero correspond to different target colours (usually black and white), which are updated every frame according to the code. While binary flickering patterns enable high communication speeds, they are perceived as annoying by many users. Quintary (base 5) m-sequences, where the five digits correspond to different shades of grey, may yield a more subtle visual stimulation. This study explores two approaches to reduce the flickering sensation: (1) adjusting the flickering speed via refresh rates and (2) applying quintary codes. In this respect, six flickering modalities are tested using an eight-target spelling application: binary patterns and quintary patterns generated with 60, 120, and 240 Hz refresh rates. This study was conducted with 18 nondisabled participants. For all six flickering modalities, a copy-spelling task was conducted. According to questionnaire results, most users favoured the proposed quintary over the binary pattern while achieving similar performance to it (no statistical differences between the patterns were found). Mean accuracies across participants were above 95%, and information transfer rates were above 55 bits/min for all patterns and flickering speeds.