Training leads to increased auditory brain–computer interface performance of end-users with motor impairments

The challenge of controlling an auditory BCI in the case of severe motor disability

BACKGROUND

The locked-in syndrome (LIS), due to a lesion in the pons, impedes communication. This situation can also be met after some severe brain injury or in advanced Amyotrophic Lateral Sclerosis (ALS). In the most severe condition, the persons cannot communicate at all because of a complete oculomotor paralysis (Complete LIS or CLIS). This even prevents the detection of consciousness. Some studies suggest that auditory brain-computer interface (BCI) could restore a communication through a « yes-no» code.

METHODS

We developed an auditory EEG-based interface which makes use of voluntary modulations of attention, to restore a yes-no communication code in non-responding persons. This binary BCI uses repeated speech sounds (alternating "yes" on the right ear and "no" on the left ear) corresponding to either frequent (short) or rare (long) stimuli. Users are instructed to pay attention to the relevant stimuli only. We tested this BCI with 18 healthy subjects, and 7 people with severe motor disability (3 "classical" persons with locked-in syndrome and 4 persons with ALS).

RESULTS

We report online BCI performance and offline event-related potential analysis. On average in healthy subjects, online BCI accuracy reached 86% based on 50 questions. Only one out of 18 subjects could not perform above chance level. Ten subjects had an accuracy above 90%. However, most patients could not produce online performance above chance level, except for two people with ALS who obtained 100% accuracy. We report individual event-related potentials and their modulation by attention. In addition to the classical P3b, we observed a signature of sustained attention on responses to frequent sounds, but in healthy subjects and patients with good BCI control only.

CONCLUSIONS

Auditory BCI can be very well controlled by healthy subjects, but it is not a guarantee that it can be readily used by the target population of persons in LIS or CLIS. A conclusion that is supported by a few previous findings in BCI and should now trigger research to assess the reasons of such a gap in order to propose new and efficient solutions.

CLINICAL TRIAL REGISTRATIONS

No. NCT02567201 (2015) and NCT03233282 (2013).

Design of auditory P300-based brain-computer interfaces with a single auditory channel and no visual support

Non-invasive brain-computer interfaces (BCIs) based on an event-related potential (ERP) component, P300, elicited via the oddball paradigm, have been extensively developed to enable device control and communication. While most P300-based BCIs employ visual stimuli in the oddball paradigm, auditory P300-based BCIs also need to be developed for users with unreliable gaze control or limited visual processing. Specifically, auditory BCIs without additional visual support or multi-channel sound sources can broaden the application areas of BCIs. This study aimed to design optimal stimuli for auditory BCIs among artificial (e.g., beep) and natural (e.g., human voice and animal sounds) sounds in such circumstances. In addition, it aimed to investigate differences between auditory and visual stimulations for online P300-based BCIs. As a result, natural sounds led to both higher online BCI performance and larger differences in ERP amplitudes between the target and non-target compared to artificial sounds. However, no single type of sound offered the best performance for all subjects; rather, each subject indicated different preferences between the human voice and animal sound. In line with previous reports, visual stimuli yielded higher BCI performance (average 77.56%) than auditory counterparts (average 54.67%). In addition, spatiotemporal patterns of the differences in ERP amplitudes between target and non-target were more dynamic with visual stimuli than with auditory stimuli. The results suggest that selecting a natural auditory stimulus optimal for individual users as well as making differences in ERP amplitudes between target and non-target stimuli more dynamic may further improve auditory P300-based BCIs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09901-3.

Simultaneous multiple-stimulus auditory brain-computer interface with semi-supervised learning and prior probability distribution tuning

Objective.Auditory brain-computer interfaces (BCIs) enable users to select commands based on the brain activity elicited by auditory stimuli. However, existing auditory BCI paradigms cannot increase the number of available commands without decreasing the selection speed, because each stimulus needs to be presented independently and sequentially under the standard oddball paradigm. To solve this problem, we propose a double-stimulus paradigm that simultaneously presents multiple auditory stimuli.Approach.For addition to an existing auditory BCI paradigm, the best discriminable sound was chosen following a subjective assessment. The new sound was located on the right-hand side and presented simultaneously with an existing sound from the left-hand side. A total of six sounds were used for implementing the auditory BCI with a 6 × 6 letter matrix. We employ semi-supervised learning (SSL) and prior probability distribution tuning to improve the accuracy of the paradigm. The SSL method involved updating of the classifier weights, and their prior probability distributions were adjusted using the following three types of distributions: uniform, empirical, and extended empirical (e-empirical). The performance was evaluated based on the BCI accuracy and information transfer rate (ITR).Main results.The double-stimulus paradigm resulted in a BCI accuracy of 67.89 ± 11.46% and an ITR of 2.67 ± 1.09 bits min^-1, in the absence of SSL and with uniform distribution. The proposed combination of SSL with e-empirical distribution improved the BCI accuracy and ITR to 74.59 ± 12.12% and 3.37 ± 1.27 bits min^-1, respectively. The event-related potential analysis revealed that contralateral and right-hemispheric dominances contributed to the BCI performance improvement.Significance.Our study demonstrated that a BCI based on multiple simultaneous auditory stimuli, incorporating SSL and e-empirical prior distribution, can increase the number of commands without sacrificing typing speed beyond the acceptable level of accuracy.

Identifying potential training factors in a vibrotactile P300-BCI

Brain-computer interfaces (BCI) often rely on visual stimulation and feedback. Potential end-users with impaired vision, however, cannot use these BCIs efficiently and require a non-visual alternative. Both auditory and tactile paradigms have been developed but are often not sufficiently fast or accurate. Thus, it is particularly relevant to investigate if and how users can train and improve performance. We report data from 29 healthy participants who trained with a 4-choice tactile P300-BCI during five sessions. To identify potential training factors, we pre-post assessed the robustness of the BCI performance against increased workload in a dual task condition and determined the participants' somatosensory sensitivity thresholds with a forced-choice intensity discrimination task. Accuracy (M = 79.2% to 92.0%) and tactually evoked P300 amplitudes increased significantly, confirming successful training. Pre-post somatosensory sensitivity increased, and workload decreased significantly, but results of the dual task condition remained inconclusive. The present study confirmed the previously reported feasibility and trainability of our tactile BCI paradigm within a multi-session design. Importantly, we provide first evidence of improvement in the somatosensory system as a potential mediator for the observed training effects.

A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities

Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people.

Clinical trial registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.

A P300 Brain-Computer Interface Paradigm Based on Electric and Vibration Simple Command Tactile Stimulation

This paper proposed a novel tactile-stimuli P300 paradigm for Brain-Computer Interface (BCI), which potentially targeted at people with less learning ability or difficulty in maintaining attention. The new paradigm using only two types of stimuli was designed, and different targets were distinguished by frequency and spatial information. The classification algorithm was developed by introducing filters for frequency bands selection and conducting optimization with common spatial pattern (CSP) on the tactile evoked EEG signals. It features a combination of spatial and frequency information, with the spatial information distinguishing the sites of stimuli and frequency information identifying target stimuli and disturbances. We investigated both electrical stimuli and vibration stimuli, in which only one target site was stimulated in each block. The results demonstrated an average accuracy of 94.88% for electrical stimuli and 95.21% for vibration stimuli, respectively.

Impact of Stimulus Features on the Performance of a Gaze-Independent Brain-Computer Interface Based on Covert Spatial Attention Shifts

Regaining communication abilities in patients who are unable to speak or move is one of the main goals in decoding brain waves for brain-computer interface (BCI) control. Many BCI approaches designed for communication rely on attention to visual stimuli, commonly applying an oddball paradigm, and require both eye movements and adequate visual acuity. These abilities may, however, be absent in patients who depend on BCI communication. We have therefore developed a response-based communication BCI, which is independent of gaze shifts but utilizes covert shifts of attention to the left or right visual field. We recorded the electroencephalogram (EEG) from 29 channels and coregistered the vertical and horizontal electrooculogram. Data-driven decoding of small attention-based differences between the hemispheres, also known as N2pc, was performed using 14 posterior channels, which are expected to reflect correlates of visual spatial attention. Eighteen healthy participants responded to 120 statements by covertly directing attention to one of two colored symbols (green and red crosses for "yes" and "no," respectively), presented in the user's left and right visual field, respectively, while maintaining central gaze fixation. On average across participants, 88.5% (std: 7.8%) of responses were correctly decoded online. In order to investigate the potential influence of stimulus features on accuracy, we presented the symbols with different visual angles, by altering symbol size and eccentricity. The offline analysis revealed that stimulus features have a minimal impact on the controllability of the BCI. Hence, we show with our novel approach that spatial attention to a colored symbol is a robust method with which to control a BCI, which has the potential to support severely paralyzed people with impaired eye movements and low visual acuity in communicating with their environment.

A 36-Class Bimodal ERP Brain-Computer Interface Using Location-Congruent Auditory-Tactile Stimuli

To date, traditional visual-based event-related potential brain-computer interface (ERP-BCI) systems continue to dominate the mainstream BCI research. However, these conventional BCIs are unsuitable for the individuals who have partly or completely lost their vision. Considering the poor performance of gaze independent ERP-BCIs, it is necessary to study techniques to improve the performance of these BCI systems. In this paper, we developed a novel 36-class bimodal ERP-BCI system based on tactile and auditory stimuli, in which six-virtual-direction audio files produced via head related transfer functions (HRTF) were delivered through headphones and location-congruent electro-tactile stimuli were simultaneously delivered to the corresponding position using electrodes placed on the abdomen and waist. We selected the eight best channels, trained a Bayesian linear discriminant analysis (BLDA) classifier and acquired the optimal trial number for target selection in online process. The average online information transfer rate (ITR) of the bimodal ERP-BCI reached 11.66 bit/min, improvements of 35.11% and 36.69% compared to the auditory (8.63 bit/min) and tactile approaches (8.53 bit/min), respectively. The results demonstrate the performance of the bimodal system is superior to each unimodal system. These facts indicate that the proposed bimodal system has potential utility as a gaze-independent BCI in future real-world applications.

Stimulus modality influences session-to-session transfer of training effects in auditory and tactile streaming-based P300 brain-computer interfaces

Despite recent successes, patients suffering from locked-in syndrome (LIS) still struggle to communicate using vision-independent brain–computer interfaces (BCIs). In this study, we compared auditory and tactile BCIs, regarding training effects and cross-stimulus-modality transfer effects, when switching between stimulus modalities. We utilized a streaming-based P300 BCI, which was developed as a low workload approach to prevent potential BCI-inefficiency. We randomly assigned 20 healthy participants to two groups. The participants received three sessions of training either using an auditory BCI or using a tactile BCI. In an additional fourth session, BCI versions were switched to explore possible cross-stimulus-modality transfer effects. Both BCI versions could be operated successfully in the first session by the majority of the participants, with the tactile BCI being experienced as more intuitive. Significant training effects were found mostly in the auditory BCI group and strong evidence for a cross-stimulus-modality transfer occurred for the auditory training group that switched to the tactile version but not vice versa. All participants were able to control at least one BCI version, suggesting that the investigated paradigms are generally feasible and merit further research into their applicability with LIS end-users. Individual preferences regarding stimulus modality should be considered.

Wheelchair Control in a Virtual Environment by Healthy Participants Using a P300-BCI Based on Tactile Stimulation: Training Effects and Usability

Tactile stimulation is less frequently used than visual for brain-computer interface (BCI) control, partly because of limitations in speed and accuracy. Non-visual BCI paradigms, however, may be required for patients who struggle with vision dependent BCIs because of a loss of gaze control. With the present study, we attempted to replicate earlier results by Herweg et al. (2016), with several minor adjustments and a focus on training effects and usability. We invited 16 healthy participants and trained them with a 4-class tactile P300-based BCI in five sessions. Their main task was to navigate a virtual wheelchair through a 3D apartment using the BCI. We found significant training effects on information transfer rate (ITR), which increased from a mean of 3.10–9.50 bits/min. Further, both online and offline accuracies significantly increased with training from 65% to 86% and 70% to 95%, respectively. We found only a descriptive increase of P300 amplitudes at Fz and Cz with training. Furthermore, we report subjective data from questionnaires, which indicated a relatively high workload and moderate to high satisfaction. Although our participants have not achieved the same high performance as in the Herweg et al. (2016) study, we provide evidence for training effects on performance with a tactile BCI and confirm the feasibility of the paradigm.

30+ years of P300 brain-computer interfaces

Brain-computer interfaces (BCIs) directly measure brain activity with no physical movement and translate the neural signals into messages. BCIs that employ the P300 event-related brain potential often have used the visual modality. The end user is presented with flashing stimuli that indicate selections for communication, control, or both. Counting each flash that corresponds to a specific target selection while ignoring other flashes will elicit P300s to only the target selection. P300 BCIs also have been implemented using auditory or tactile stimuli. P300 BCIs have been used with a variety of applications for severely disabled end users in their homes without frequent expert support. P300 BCI research and development has made substantial progress, but challenges remain before these tools can become practical devices for impaired patients and perhaps healthy people.

Hearing the needs of clinical users

In the past 10 years, brain-computer interfaces (BCIs) for controlling assistive devices have seen tremendous progress with respect to reliability and learnability, and numerous exemplary applications were demonstrated to be controllable by a BCI. Yet, BCI-controlled applications are hardly used for patients with neurologic or neurodegenerative disease. Such patient groups are considered potential end-users of BCI, specifically for replacing or improving lost function. We argue that BCI research and development still faces a translational gap, i.e., the knowledge of how to bring BCIs from the laboratory to the field is insufficient. BCI-controlled applications lack usability and accessibility; both constitute two sides of one coin, which is the key to use in daily life and to prevent nonuse. To increase usability, we suggest rigorously adopting the user-centered design in applied BCI research and development. To provide accessibility, assistive technology (AT) experts, providers, and other stakeholders have to be included in the user-centered process. BCI experts have to ensure the transfer of knowledge to AT professionals, and listen to the needs of primary, secondary, and tertiary end-users of BCI technology. Addressing both, usability and accessibility, in applied BCI research and development will bridge the translational gap and ensure that the needs of clinical end-users are heard, understood, addressed, and fulfilled.

Analysis of Prefrontal Single-Channel EEG Data for Portable Auditory ERP-Based Brain-Computer Interfaces

An electroencephalogram (EEG)-based brain-computer interface (BCI) is a tool to non-invasively control computers by translating the electrical activity of the brain. This technology has the potential to provide patients who have severe generalized myopathy, such as those suffering from amyotrophic lateral sclerosis (ALS), with the ability to communicate. Recently, auditory oddball paradigms have been developed to implement more practical event-related potential (ERP)-based BCIs because they can operate without ocular activities. These paradigms generally make use of clinical (over 16-channel) EEG devices and natural sound stimuli to maintain the user's motivation during the BCI operation; however, most ALS patients who have taken part in auditory ERP-based BCIs tend to complain about the following factors: (i) total device cost and (ii) setup time. The development of a portable auditory ERP-based BCI could overcome considerable obstacles that prevent the use of this technology in communication in everyday life. To address this issue, we analyzed prefrontal single-channel EEG data acquired from a consumer-grade single-channel EEG device using a natural sound-based auditory oddball paradigm. In our experiments, EEG data was gathered from nine healthy subjects and one ALS patient. The performance of auditory ERP-based BCI was quantified under an offline condition and two online conditions. The offline analysis indicated that our paradigm maintained a high level of detection accuracy (%) and ITR (bits/min) across all subjects through a cross-validation procedure (for five commands: 70.0 ± 16.1 and 1.29 ± 0.93, for four commands: 73.8 ± 14.2 and 1.16 ± 0.78, for three commands: 78.7 ± 11.8 and 0.95 ± 0.61, and for two commands: 85.7 ± 8.6 and 0.63 ± 0.38). Furthermore, the first online analysis demonstrated that our paradigm also achieved high performance for new data in an online data acquisition stream (for three commands: 80.0 ± 19.4 and 1.16 ± 0.83). The second online analysis measured online performances on the different day of offline and first online analyses on a different day (for three commands: 62.5 ± 14.3 and 0.43 ± 0.36). These results indicate that prefrontal single-channel EEGs have the potential to contribute to the development of a user-friendly portable auditory ERP-based BCI.

Algorithmic clustering based on string compression to extract P300 structure in EEG signals

BACKGROUND AND OBJECTIVES

P300 is an Event Related Potential control signal widely used in Brain Computer Interfaces. Using the oddball paradigm, a P300 speller allows a human to spell letters through P300 events produced by his/her brain. One of the most common issues in the detection of this event is that its structure may differ between different subjects and over time for a specific subject. The main purpose of this work is to deal with this inherent variability and identify the main structure of P300 using algorithmic clustering based on string compression.

METHODS

In this work, we make use of the Normalized Compression Distance (NCD) to extract the main structure of the signal regardless of its inherent variability. In order to apply compression distances, we carry out a novel signal-to-ASCII process that transforms and merges different events into suitable objects to be used by a compression algorithm. Once the ASCII objects are created, we use NCD-driven clustering as a tool to analyze if our object creation method suitably represents the information contained in the signals and to explore if compression distances are a valid tool for identifying P300 structure. With the purpose of increasing the level of generalization of our study, we apply two different clustering methods: a hierarchical clustering algorithm based on the minimum quartet tree method and a multidimensional projection method.

RESULTS

Our experimental results show good clustering performance over different experiments, showing the structure extraction capabilities of our procedure. Two datasets with recordings in different scenarios were used to analyze the problem and validate our results, respectively. It has to be pointed out that when the clustering performance over individual electrodes is analyzed, higher P300 activity is found in similar regions to other articles using the same datasets. This suggests that our approach might be used as an electrode-selection criteria.

CONCLUSIONS

The proposed NCD-driven clustering methodology can be used to discover the structural characteristics of EEG and thereby, it is suitable as a complementary methodology for the P300 analysis.

Neural mechanisms of training an auditory event-related potential task in a brain-computer interface context

Effective use of brain-computer interfaces (BCIs) typically requires training. Improved understanding of the neural mechanisms underlying BCI training will facilitate optimisation of BCIs. The current study examined the neural mechanisms related to training for electroencephalography (EEG)-based communication with an auditory event-related potential (ERP) BCI. Neural mechanisms of training in 10 healthy volunteers were assessed with functional magnetic resonance imaging (fMRI) during an auditory ERP-based BCI task before (t1) and after (t5) three ERP-BCI training sessions outside the fMRI scanner (t2, t3, and t4). Attended stimuli were contrasted with ignored stimuli in the first-level fMRI data analysis (t1 and t5); the training effect was verified using the EEG data (t2-t4); and brain activation was contrasted before and after training in the second-level fMRI data analysis (t1 vs. t5). Training increased the communication speed from 2.9 bits/min (t2) to 4 bits/min (t4). Strong activation was found in the putamen, supplementary motor area (SMA), and superior temporal gyrus (STG) associated with attention to the stimuli. Training led to decreased activation in the superior frontal gyrus and stronger haemodynamic rebound in the STG and supramarginal gyrus. The neural mechanisms of ERP-BCI training indicate improved stimulus perception and reduced mental workload. The ERP task used in the current study showed overlapping activations with a motor imagery based BCI task from a previous study on the neural mechanisms of BCI training in the SMA and putamen. This suggests commonalities between the neural mechanisms of training for both BCI paradigms.

Eyes-Closed Increases the Usability of Brain-Computer Interfaces Based on Auditory Event-Related Potentials

Recent research has demonstrated how brain-computer interfaces (BCI) based on auditory stimuli can be used for communication and rehabilitation. In these applications, users are commonly instructed to avoid eye movements while keeping their eyes open. This secondary task can lead to exhaustion and subjects may not succeed in suppressing eye movements. In this work, we investigate the option to use a BCI with eyes-closed. Twelve healthy subjects participated in a single electroencephalography (EEG) session where they were listening to a rapid stream of bisyllabic words while alternatively having their eyes open or closed. In addition, we assessed usability aspects for the two conditions with a questionnaire. Our analysis shows that eyes-closed does not reduce the number of eye artifacts and that event-related potential (ERP) responses and classification accuracies are comparable between both conditions. Importantly, we found that subjects expressed a significant general preference toward the eyes-closed condition and were also less tensed in that condition. Furthermore, switching between eyes-closed and eyes-open and vice versa is possible without a severe drop in classification accuracy. These findings suggest that eyes-closed should be considered as a viable alternative in auditory BCIs that might be especially useful for subjects with limited control over their eye movements.

Comparison of Four Control Methods for a Five-Choice Assistive Technology

Severe motor impairments can affect the ability to communicate. The ability to see has a decisive influence on the augmentative and alternative communication (AAC) systems available to the user. To better understand the initial impressions users have of AAC systems we asked naïve healthy participants to compare two visual (a visual P300 brain-computer interface (BCI) and an eye-tracker) and two non-visual systems (an auditory and a tactile P300 BCI). Eleven healthy participants performed 20 selections in a five choice task with each system. The visual P300 BCI used face stimuli, the auditory P300 BCI used Japanese Hiragana syllables and the tactile P300 BCI used a stimulator on the small left finger, middle left finger, right thumb, middle right finger and small right finger. The eye-tracker required a dwell time of 3 s on the target for selection. We calculated accuracies and information-transfer rates (ITRs) for each control method using the selection time that yielded the highest ITR and an accuracy above 70% for each system. Accuracies of 88% were achieved with the visual P300 BCI (4.8 s selection time, 20.9 bits/min), of 70% with the auditory BCI (19.9 s, 3.3 bits/min), of 71% with the tactile BCI (18 s, 3.4 bits/min) and of 100% with the eye-tracker (5.1 s, 28.2 bits/min). Performance between eye-tracker and visual BCI correlated strongly, correlation between tactile and auditory BCI performance was lower. Our data showed no advantage for either non-visual system in terms of ITR but a lower correlation of performance which suggests that choosing the system which suits a particular user is of higher importance for non-visual systems than visual systems.

Psychological Predictors of Visual and Auditory P300 Brain-Computer Interface Performance

Brain-Computer Interfaces (BCIs) provide communication channels independent from muscular control. In the current study we used two versions of the P300-BCI: one based on visual the other on auditory stimulation. Up to now, data on the impact of psychological variables on P300-BCI control are scarce. Hence, our goal was to identify new predictors with a comprehensive psychological test-battery. A total of N = 40 healthy BCI novices took part in a visual and an auditory BCI session. Psychological variables were measured with an electronic test-battery including clinical, personality, and performance tests. The personality factor "emotional stability" was negatively correlated (Spearman's rho = -0.416; p < 0.01) and an output variable of the non-verbal learning test (NVLT), which can be interpreted as ability to learn, correlated positively (Spearman's rho = 0.412; p < 0.01) with visual P300-BCI performance. In a linear regression analysis both independent variables explained 24% of the variance. "Emotional stability" was also negatively related to auditory P300-BCI performance (Spearman's rho = -0.377; p < 0.05), but failed significance in the regression analysis. Psychological parameters seem to play a moderate role in visual P300-BCI performance. "Emotional stability" was identified as a new predictor, indicating that BCI users who characterize themselves as calm and rational showed worse BCI performance. The positive relation of the ability to learn and BCI performance corroborates the notion that also for P300 based BCIs learning may constitute an important factor. Further studies are needed to consolidate or reject the presented predictors.

Electrophysiological correlates of neurodegeneration in motor and non-motor brain regions in amyotrophic lateral sclerosis-implications for brain-computer interfacing

OBJECTIVE

For patients with amyotrophic lateral sclerosis (ALS) who are suffering from severe communication or motor problems, brain-computer interfaces (BCIs) can improve the quality of life and patient autonomy. However, current BCI systems are not as widely used as their potential and patient demand would let assume. This underutilization is a result of technological as well as user-based limitations but also of the comparatively poor performance of currently existing BCIs in patients with late-stage ALS, particularly in the locked-in state.

APPROACH

Here we review a broad range of electrophysiological studies in ALS patients with the aim to identify electrophysiological correlates of ALS-related neurodegeneration in motor and non-motor brain regions in to better understand potential neurophysiological limitations of current BCI systems for ALS patients. To this end we analyze studies in ALS patients that investigated basic sensory evoked potentials, resting-state and task-based paradigms using electroencephalography or electrocorticography for basic research purposes as well as for brain-computer interfacing. Main results and significance. Our review underscores that, similarly to mounting evidence from neuroimaging and neuropathology, electrophysiological measures too indicate neurodegeneration in non-motor areas in ALS. Furthermore, we identify an unexpected gap of basic and advanced electrophysiological studies in late-stage ALS patients, particularly in the locked-in state. We propose a research strategy on how to fill this gap in order to improve the design and performance of future BCI systems for this patient group.

Brain-Computer Interface Spellers: A Review

A Brain-Computer Interface (BCI) provides a novel non-muscular communication method via brain signals. A BCI-speller can be considered as one of the first published BCI applications and has opened the gate for many advances in the field. Although many BCI-spellers have been developed during the last few decades, to our knowledge, no reviews have described the different spellers proposed and studied in this vital field. The presented speller systems are categorized according to major BCI paradigms: P300, steady-state visual evoked potential (SSVEP), and motor imagery (MI). Different BCI paradigms require specific electroencephalogram (EEG) signal features and lead to the development of appropriate Graphical User Interfaces (GUIs). The purpose of this review is to consolidate the most successful BCI-spellers published since 2010, while mentioning some other older systems which were built explicitly for spelling purposes. We aim to assist researchers and concerned individuals in the field by illustrating the highlights of different spellers and presenting them in one review. It is almost impossible to carry out an objective comparison between different spellers, as each has its variables, parameters, and conditions. However, the gathered information and the provided taxonomy about different BCI-spellers can be helpful, as it could identify suitable systems for first-hand users, as well as opportunities of development and learning from previous studies for BCI researchers.

Usage of drip drops as stimuli in an auditory P300 BCI paradigm

Recently, many auditory BCIs are using beeps as auditory stimuli, while beeps sound unnatural and unpleasant for some people. It is proved that natural sounds make people feel comfortable, decrease fatigue, and improve the performance of auditory BCI systems. Drip drop is a kind of natural sounds that makes humans feel relaxed and comfortable. In this work, three kinds of drip drops were used as stimuli in an auditory-based BCI system to improve the user-friendness of the system. This study explored whether drip drops could be used as stimuli in the auditory BCI system. The auditory BCI paradigm with drip-drop stimuli, which was called the drip-drop paradigm (DP), was compared with the auditory paradigm with beep stimuli, also known as the beep paradigm (BP), in items of event-related potential amplitudes, online accuracies and scores on the likability and difficulty to demonstrate the advantages of DP. DP obtained significantly higher online accuracy and information transfer rate than the BP (p < 0.05, Wilcoxon signed test; p < 0.05, Wilcoxon signed test). Besides, DP obtained higher scores on the likability with no significant difference on the difficulty (p < 0.05, Wilcoxon signed test). The results showed that the drip drops were reliable acoustic materials as stimuli in an auditory BCI system.

Affective Stimuli for an Auditory P300 Brain-Computer Interface

Gaze-independent brain computer interfaces (BCIs) are a potential communication tool for persons with paralysis. This study applies affective auditory stimuli to investigate their effects using a P300 BCI. Fifteen able-bodied participants operated the P300 BCI, with positive and negative affective sounds (PA: a meowing cat sound, NA: a screaming cat sound). Permuted stimuli of the positive and negative affective sounds (permuted-PA, permuted-NA) were also used for comparison. Electroencephalography data was collected, and offline classification accuracies were compared. We used a visual analog scale (VAS) to measure positive and negative affective feelings in the participants. The mean classification accuracies were 84.7% for PA and 67.3% for permuted-PA, while the VAS scores were 58.5 for PA and −12.1 for permuted-PA. The positive affective stimulus showed significantly higher accuracy and VAS scores than the negative affective stimulus. In contrast, mean classification accuracies were 77.3% for NA and 76.0% for permuted-NA, while the VAS scores were −50.0 for NA and −39.2 for permuted NA, which are not significantly different. We determined that a positive affective stimulus with accompanying positive affective feelings significantly improved BCI accuracy. Additionally, an ALS patient achieved 90% online classification accuracy. These results suggest that affective stimuli may be useful for preparing a practical auditory BCI system for patients with disabilities.

A Multifunctional Brain-Computer Interface Intended for Home Use: An Evaluation with Healthy Participants and Potential End Users with Dry and Gel-Based Electrodes

Current brain-computer interface (BCIs) software is often tailored to the needs of scientists and technicians and therefore complex to allow for versatile use. To facilitate home use of BCIs a multifunctional P300 BCI with a graphical user interface intended for non-expert set-up and control was designed and implemented. The system includes applications for spelling, web access, entertainment, artistic expression and environmental control. In addition to new software, it also includes new hardware for the recording of electroencephalogram (EEG) signals. The EEG system consists of a small and wireless amplifier attached to a cap that can be equipped with gel-based or dry contact electrodes. The system was systematically evaluated with a healthy sample, and targeted end users of BCI technology, i.e., people with a varying degree of motor impairment tested the BCI in a series of individual case studies. Usability was assessed in terms of effectiveness, efficiency and satisfaction. Feedback of users was gathered with structured questionnaires. Two groups of healthy participants completed an experimental protocol with the gel-based and the dry contact electrodes (N = 10 each). The results demonstrated that all healthy participants gained control over the system and achieved satisfactory to high accuracies with both gel-based and dry electrodes (average error rates of 6 and 13%). Average satisfaction ratings were high, but certain aspects of the system such as the wearing comfort of the dry electrodes and design of the cap, and speed (in both groups) were criticized by some participants. Six potential end users tested the system during supervised sessions. The achieved accuracies varied greatly from no control to high control with accuracies comparable to that of healthy volunteers. Satisfaction ratings of the two end-users that gained control of the system were lower as compared to healthy participants. The advantages and disadvantages of the BCI and its applications are discussed and suggestions are presented for improvements to pave the way for user friendly BCIs intended to be used as assistive technology by persons with severe paralysis.

Effects of Background Music on Objective and Subjective Performance Measures in an Auditory BCI

Several studies have explored brain computer interface (BCI) systems based on auditory stimuli, which could help patients with visual impairments. Usability and user satisfaction are important considerations in any BCI. Although background music can influence emotion and performance in other task environments, and many users may wish to listen to music while using a BCI, auditory, and other BCIs are typically studied without background music. Some work has explored the possibility of using polyphonic music in auditory BCI systems. However, this approach requires users with good musical skills, and has not been explored in online experiments. Our hypothesis was that an auditory BCI with background music would be preferred by subjects over a similar BCI without background music, without any difference in BCI performance. We introduce a simple paradigm (which does not require musical skill) using percussion instrument sound stimuli and background music, and evaluated it in both offline and online experiments. The result showed that subjects preferred the auditory BCI with background music. Different performance measures did not reveal any significant performance effect when comparing background music vs. no background. Since the addition of background music does not impair BCI performance but is preferred by users, auditory (and perhaps other) BCIs should consider including it. Our study also indicates that auditory BCIs can be effective even if the auditory channel is simultaneously otherwise engaged.

An Evaluation of Training with an Auditory P300 Brain-Computer Interface for the Japanese Hiragana Syllabary

Gaze-independent brain-computer interfaces (BCIs) are a possible communication channel for persons with paralysis. We investigated if it is possible to use auditory stimuli to create a BCI for the Japanese Hiragana syllabary, which has 46 Hiragana characters. Additionally, we investigated if training has an effect on accuracy despite the high amount of different stimuli involved. Able-bodied participants (N = 6) were asked to select 25 syllables (out of fifty possible choices) using a two step procedure: First the consonant (ten choices) and then the vowel (five choices). This was repeated on 3 separate days. Additionally, a person with spinal cord injury (SCI) participated in the experiment. Four out of six healthy participants reached Hiragana syllable accuracies above 70% and the information transfer rate increased from 1.7 bits/min in the first session to 3.2 bits/min in the third session. The accuracy of the participant with SCI increased from 12% (0.2 bits/min) to 56% (2 bits/min) in session three. Reliable selections from a 10 × 5 matrix using auditory stimuli were possible and performance is increased by training. We were able to show that auditory P300 BCIs can be used for communication with up to fifty symbols. This enables the use of the technology of auditory P300 BCIs with a variety of applications.

Evaluation of induced and evoked changes in EEG during selective attention to verbal stimuli

BACKGROUND

Two challenges need to be addressed before bringing non-motor mental tasks for brain-computer interface (BCI) control to persons in a minimally conscious state (MCS), who can be behaviorally unresponsive even when proven to be consciously aware: first, keeping the cognitive demands as low as possible so that they could be fulfilled by persons with MCS. Second, increasing the control of experimental protocol (i.e. type and timing of the task performance).

NEW METHOD

The goal of this study is twofold: first goal is to develop an experimental paradigm that can facilitate the performance of brain-teasers (e.g. mental subtraction and word generation) on the one hand, and can increase the control of experimental protocol on the other hand. The second goal of this study is to exploit the similar findings for mentally attending to someone else's verbal performance of brain-teaser tasks and self-performing the same tasks to setup an online BCI, and to compare it in healthy participants to the current "state-of-the-art" motor imagery (MI, sports).

RESULTS

The response accuracies for the best performing healthy participants indicate that selective attention to verbal performance of mental subtraction (SUB) is a viable alternative to the MI. Time-frequency analysis of the SUB task in one participant with MCS did not reveal any significant (p<0.05) EEG changes, whereas imagined performance of one sport of participants' choice (SPORT) revealed task-related EEG changes over neurophysiological plausible cortical areas.

COMPARISON WITH EXISTING METHODS

We found that mentally attending to someone else's verbal performance of brain-teaser tasks leads to similar results as in self-performing the same tasks.

CONCLUSIONS

In this work we demonstrated that a single auditory selective attention task (i.e. mentally attending to someone else's verbal performance of mental subtraction) can modulate both induced and evoked changes in EEG, and be used for yes/no communication in an auditory scanning paradigm.

The WIN-speller: a new intuitive auditory brain-computer interface spelling application

The objective of this study was to test the usability of a new auditory Brain-Computer Interface (BCI) application for communication. We introduce a word based, intuitive auditory spelling paradigm the WIN-speller. In the WIN-speller letters are grouped by words, such as the word KLANG representing the letters A, G, K, L, and N. Thereby, the decoding step between perceiving a code and translating it to the stimuli it represents becomes superfluous. We tested 11 healthy volunteers and four end-users with motor impairment in the copy spelling mode. Spelling was successful with an average accuracy of 84% in the healthy sample. Three of the end-users communicated with average accuracies of 80% or higher while one user was not able to communicate reliably. Even though further evaluation is required, the WIN-speller represents a potential alternative for BCI based communication in end-users.

Comparison of eye tracking, electrooculography and an auditory brain-computer interface for binary communication: a case study with a participant in the locked-in state

BACKGROUND

In this study, we evaluated electrooculography (EOG), an eye tracker and an auditory brain-computer interface (BCI) as access methods to augmentative and alternative communication (AAC). The participant of the study has been in the locked-in state (LIS) for 6 years due to amyotrophic lateral sclerosis. He was able to communicate with slow residual eye movements, but had no means of partner independent communication. We discuss the usability of all tested access methods and the prospects of using BCIs as an assistive technology.

METHODS

Within four days, we tested whether EOG, eye tracking and a BCI would allow the participant in LIS to make simple selections. We optimized the parameters in an iterative procedure for all systems.

RESULTS

The participant was able to gain control over all three systems. Nonetheless, due to the level of proficiency previously achieved with his low-tech AAC method, he did not consider using any of the tested systems as an additional communication channel. However, he would consider using the BCI once control over his eye muscles would no longer be possible. He rated the ease of use of the BCI as the highest among the tested systems, because no precise eye movements were required; but also as the most tiring, due to the high level of attention needed to operate the BCI.

CONCLUSIONS

In this case study, the partner based communication was possible due to the good care provided and the proficiency achieved by the interlocutors. To ease the transition from a low-tech AAC method to a BCI once control over all muscles is lost, it must be simple to operate. For persons, who rely on AAC and are affected by a progressive neuromuscular disease, we argue that a complementary approach, combining BCIs and standard assistive technology, can prove valuable to achieve partner independent communication and ease the transition to a purely BCI based approach. Finally, we provide further evidence for the importance of a user-centered approach in the design of new assistive devices.