Increasing BCI communication rates with dynamic stopping towards more practical use: an ALS study

Enhancing SSVEP-BCI Performance Under Fatigue State Using Dynamic Stopping Strategy

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have emerged as a prominent technology due to their high information transfer rate, rapid calibration time, and robust signal-to-noise ratio. However, a critical challenge for practical applications is performance degradation caused by user fatigue during prolonged use. This work proposes novel methods to address this challenge by dynamically adjusting data acquisition length and updating detection models based on a fatigue-aware stopping strategy. Two 16-target SSVEP-BCIs were employed, one using low-frequency and the other using high-frequency stimulation. A self-recorded fatigue dataset from 24 subjects was utilized for extensive evaluation. A simulated online experiment demonstrated that the proposed methods outperform the conventional fixed stopping strategy in terms of classification accuracy, information transfer rate, and selection time, irrespective of stimulation frequency. These findings suggest that the proposed approach can significantly improve SSVEP-BCI performance under fatigue conditions, leading to superior performance during extended use.

Brain-Computer Interfaces, Completely Locked-In State in Neurodegenerative Diseases, and End-of-Life Decisions

In the future, policies surrounding end-of-life decisions will be faced with the question of whether competent people in a completely locked-in state should be enabled to make end-of-life decisions via brain-computer interfaces (BCI). This article raises ethical issues with acting through BCIs in the context of these decisions, specifically self-administration requirements within assisted suicide policies. We argue that enabling patients to end their life even once they have entered completely locked-in state might, paradoxically, prolong and uphold their quality of life.

Digital Alternative Communication for Individuals with Amyotrophic Lateral Sclerosis: What We Have

Amyotrophic Lateral Sclerosis is a disease that compromises the motor system and the functional abilities of the person in an irreversible way, causing the progressive loss of the ability to communicate. Tools based on Augmentative and Alternative Communication are essential for promoting autonomy and improving communication, life quality, and survival. This Systematic Literature Review aimed to provide evidence on eye-image-based Human-Computer Interaction approaches for the Augmentative and Alternative Communication of people with Amyotrophic Lateral Sclerosis. The Systematic Literature Review was conducted and guided following a protocol consisting of search questions, inclusion and exclusion criteria, and quality assessment, to select primary studies published between 2010 and 2021 in six repositories: Science Direct, Web of Science, Springer, IEEE Xplore, ACM Digital Library, and PubMed. After the screening, 25 primary studies were evaluated. These studies showcased four low-cost, non-invasive Human-Computer Interaction strategies employed for Augmentative and Alternative Communication in people with Amyotrophic Lateral Sclerosis. The strategies included Eye-Gaze, which featured in 36% of the studies; Eye-Blink and Eye-Tracking, each accounting for 28% of the approaches; and the Hybrid strategy, employed in 8% of the studies. For these approaches, several computational techniques were identified. For a better understanding, a workflow containing the development phases and the respective methods used by each strategy was generated. The results indicate the possibility and feasibility of developing Human-Computer Interaction resources based on eye images for Augmentative and Alternative Communication in a control group. The absence of experimental testing in people with Amyotrophic Lateral Sclerosis reiterates the challenges related to the scalability, efficiency, and usability of these technologies for people with the disease. Although challenges still exist, the findings represent important advances in the fields of health sciences and technology, promoting a promising future with possibilities for better life quality.

Language Model-Guided Classifier Adaptation for Brain-Computer Interfaces for Communication

Brain-computer interfaces (BCIs), such as the P300 speller, can provide a means of communication for individuals with severe neuromuscular limitations. BCIs interpret electroencephalography (EEG) signals in order to translate embedded information about a user's intent into executable commands to control external devices. However, EEG signals are inherently noisy and nonstationary, posing a challenge to extended BCI use. Conventionally, a BCI classifier is trained via supervised learning in an offline calibration session; once trained, the classifier is deployed for online use and is not updated. As the statistics of a user's EEG data change over time, the performance of a static classifier may decline with extended use. It is therefore desirable to automatically adapt the classifier to current data statistics without requiring offline recalibration. In an existing semi-supervised learning approach, the classifier is trained on labeled EEG data and is then updated using incoming unlabeled EEG data and classifier-predicted labels. To reduce the risk of learning from incorrect predictions, a threshold is imposed to exclude unlabeled data with low-confidence label predictions from the expanded training set when retraining the adaptive classifier. In this work, we propose the use of a language model for spelling error correction and disambiguation to provide information about label correctness during semi-supervised learning. Results from simulations with multi-session P300 speller user EEG data demonstrate that our language-guided semi-supervised approach significantly improves spelling accuracy relative to conventional BCI calibration and threshold-based semi-supervised learning.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Artificial Intelligence Algorithms in Visual Evoked Potential-Based Brain-Computer Interfaces for Motor Rehabilitation Applications: Systematic Review and Future Directions

Brain-Computer Interface (BCI) is a technology that uses electroencephalographic (EEG) signals to control external devices, such as Functional Electrical Stimulation (FES). Visual BCI paradigms based on P300 and Steady State Visually Evoked potentials (SSVEP) have shown high potential for clinical purposes. Numerous studies have been published on P300- and SSVEP-based non-invasive BCIs, but many of them present two shortcomings: (1) they are not aimed for motor rehabilitation applications, and (2) they do not report in detail the artificial intelligence (AI) methods used for classification, or their performance metrics. To address this gap, in this paper the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was applied to prepare a systematic literature review (SLR). Papers older than 10 years, repeated or not related to a motor rehabilitation application, were excluded. Of all the studies, 51.02% referred to theoretical analysis of classification algorithms. Of the remaining, 28.48% were for spelling, 12.73% for diverse applications (control of wheelchair or home appliances), and only 7.77% were focused on motor rehabilitation. After the inclusion and exclusion criteria were applied and quality screening was performed, 34 articles were selected. Of them, 26.47% used the P300 and 55.8% the SSVEP signal. Five applications categories were established: Rehabilitation Systems (17.64%), Virtual Reality environments (23.52%), FES (17.64%), Orthosis (29.41%), and Prosthesis (11.76%). Of all the works, only four performed tests with patients. The most reported machine learning (ML) algorithms used for classification were linear discriminant analysis (LDA) (48.64%) and support vector machine (16.21%), while only one study used a deep learning algorithm: a Convolutional Neural Network (CNN). The reported accuracy ranged from 38.02 to 100%, and the Information Transfer Rate from 1.55 to 49.25 bits per minute. While LDA is still the most used AI algorithm, CNN has shown promising results, but due to their high technical implementation requirements, many researchers do not justify its implementation as worthwile. To achieve quick and accurate online BCIs for motor rehabilitation applications, future works on SSVEP-, P300-based and hybrid BCIs should focus on optimizing the visual stimulation module and the training stage of ML and DL algorithms.

A visual brain-computer interface as communication aid for patients with amyotrophic lateral sclerosis

OBJECTIVE

Brain-Computer Interface (BCI) spellers that make use of code-modulated Visual Evoked Potentials (cVEP) may provide a fast and more accurate alternative to existing visual BCI spellers for patients with Amyotrophic Lateral Sclerosis (ALS). However, so far the cVEP speller has only been tested on healthy participants.

METHODS

We assess the brain responses, BCI performance and user experience of the cVEP speller in 20 healthy participants and 10 ALS patients. All participants performed a cued and free spelling task, and a free selection of Yes/No answers.

RESULTS

27 out of 30 participants could perform the cued spelling task with an average accuracy of 79% for ALS patients, 88% for healthy older participants and 94% for healthy young participants. All 30 participants could answer Yes/No questions freely, with an average accuracy of around 90%.

CONCLUSIONS

With ALS patients typing on average 10 characters per minute, the cVEP speller presented in this paper outperforms other visual BCI spellers.

SIGNIFICANCE

These results support a general usability of cVEP signals for ALS patients, which may extend far beyond the tested speller to control e.g. an alarm, automatic door, or TV within a smart home.

Biomedical signals and machine learning in amyotrophic lateral sclerosis: a systematic review

INTRODUCTION

The use of machine learning (ML) techniques in healthcare encompasses an emerging concept that envisages vast contributions to the tackling of rare diseases. In this scenario, amyotrophic lateral sclerosis (ALS) involves complexities that are yet not demystified. In ALS, the biomedical signals present themselves as potential biomarkers that, when used in tandem with smart algorithms, can be useful to applications within the context of the disease.

METHODS

This Systematic Literature Review (SLR) consists of searching for and investigating primary studies that use ML techniques and biomedical signals related to ALS. Following the definition and execution of the SLR protocol, 18 articles met the inclusion, exclusion, and quality assessment criteria, and answered the SLR research questions.

DISCUSSIONS

Based on the results, we identified three classes of ML applications combined with biomedical signals in the context of ALS: diagnosis (72.22%), communication (22.22%), and survival prediction (5.56%).

CONCLUSIONS

Distinct algorithmic models and biomedical signals have been reported and present promising approaches, regardless of their classes. In summary, this SLR provides an overview of the primary studies analyzed as well as directions for the construction and evolution of technology-based research within the scope of ALS.

Operate P300 speller when performing other task

OBJECTIVE

The P300 speller is a classic brain-computer interface (BCI) paradigm that has the potential to restore impaired motor control function. However, previous studies have confirmed that the letter recognition accuracy (LRA) of the P300 speller is a challenge when performing other tasks.

APPROACH

To address this, we implemented a dynamic stopping strategy (DSS) to maintain the P300 speller LRA when performing multiple tasks simultaneously. Multiple tasks with dynamic workload levels were adopted to simulate the brain's other thinking activities while operating P300 speller. A Bayes-based DSS offline model was built in single-task (only P300 speller task) and an online P300 speller system was established to test the DSS algorithm feasibility in dual-task.

MAIN RESULTS

Online experimental results showed that the P300 speller with DSS could achieve a high LRA (96.9%) under dual-task, which was similar to single-task (98.7%, p = 0.126). Under dual-task, DSS dynamically adjusted the discriminant confidence according to the workload levels of the distraction tasks (correlation coefficient r = -0.68). Therefore, DSS can increase the repeated sequences to compensate for the reduction of P300 speller signal-to-noise ratio caused by parallel thinking activities. The average of repeated sequences increased significantly from 4.98 times under single-task to 6.22 times under dual-task (p < 0.005). These results indicated that the P300 speller feature is robust and the DSS model built in single-task maintained the applicability in various dual-tasks.

SIGNIFICANCE

Overall, this study provides a basis for the implementation of laboratory-developed BCI in real-world environments.

Optimizing SSVEP-Based BCI System towards Practical High-Speed Spelling

The brain–computer interface (BCI) spellers based on steady-state visual evoked potentials (SSVEPs) have recently been widely investigated for their high information transfer rates (ITRs). This paper aims to improve the practicability of the SSVEP-BCIs for high-speed spelling. The system acquired the electroencephalogram (EEG) data from a self-developed dedicated EEG device and the stimulation was arranged as a keyboard. The task-related component analysis (TRCA) spatial filter was modified (mTRCA) for target classification and showed significantly higher performance compared with the original TRCA in the offline analysis. In the online system, the dynamic stopping (DS) strategy based on Bayesian posterior probability was utilized to realize alterable stimulating time. In addition, the temporal filtering process and the programs were optimized to facilitate the online DS operation. Notably, the online ITR reached 330.4 ± 45.4 bits/min on average, which is significantly higher than that of fixed stopping (FS) strategy, and the peak value of 420.2 bits/min is the highest online spelling ITR with a SSVEP-BCI up to now. The proposed system with portable EEG acquisition, friendly interaction, and alterable time of command output provides more flexibility for SSVEP-based BCIs and is promising for practical high-speed spelling.

Applying Auto-Regressive Model's Yule-Walker Approach to Amyotrophic Lateral Sclerosis (ALS) patients' Data

OBJECTIVE

The purpose of this study is to identifying time series analysis and mathematical model fitting on electroencephalography channels that are placed on Amyotrophic Lateral Sclerosis (ALS) patients with P300 based brain-computer interface (BCI).

METHODS

Amyotrophic Lateral Sclerosis (ALS) or motor neuron diseases are a rapidly progressing neurological disease that attacks and kills neurons responsible for controlling voluntary muscles. There is no cure and treatment effective to reverse, to halt the disease progression. A Brain- Computer Interface enables disable person to communicate & interact with each other and with the environment. To bypass the important motor difficulties present in ALS patient, BCI is useful. An input for BCI system is patient's brain signals and these signals are converted into external operations, for brain signals detection, Electroencephalography (EEG) is normally used. P300 based BCI is used to record the reading of EEG brain signals with the help of non-invasive placement of channels. In EEG, channel analysis Autoregressive (AR) model is a widely used. In the present study, Yule-Walker approach of AR model has been used for channel data fitting. Model fitting as a form of digitization is majorly required for good understanding of the dataset, and also for data prediction.

RESULTS

Fourth order of the mathematical curve for this dataset is selected. The reason is the high accuracy obtained in the 4th order of Autoregressive model (97.51±0.64).

CONCLUSION

In proposed Auto Regressive (AR) model has been used for Amyotrophic Lateral Sclerosis (ALS) patient data analysis. The 4th order of Yule Walker auto-regressive model is giving best fitting on this problem.

A New Early Stopping Method for P300 Spellers

In event-related potentials based brain-computer interfaces, the responses evoked by a well defined stimuli sequence are usually averaged to overcome the limitations caused by the intrinsic poor EEG signal-to-noise ratio. This, however, implies that the time necessary to detect the brain signals increases and then that the communication rate can be dramatically reduced. A common approach is then at first to estimate an optimal fixed number of responses to be averaged on a calibration data set and then to use this number on the online/testing dataset. In contrast to this strategy, several early stopping methods have been successfully proposed, aiming at dynamically stopping the stimulation sequence when a certain condition is met. We propose an efficient and easy to implement early stopping method that outperforms the ones proposed in the literature, showing its effectiveness on several publicly available datasets recorded from either healthy subjects or amyotrophic lateral sclerosis patients.

An ERP-based BCI with peripheral stimuli: validation with ALS patients

Many studies reported that ERP-based BCIs can provide communication for some people with amyotrophic lateral sclerosis (ALS). ERP-based BCIs often present characters within a matrix that occupies the center of the visual field. However, several studies have identified some concerns with the matrix-based approach. This approach may lead to fatigue and errors resulting from flashing adjacent stimuli, and is impractical for users who might want to use the BCI in tandem with other software or feedback in the center of the monitor. In this paper, we introduce and validate an alternate ERP-based BCI display approach. By presenting stimuli near the periphery of the display, we reduce the adjacency problem and leave the center of the display available for feedback or other applications. Two ERP-based display approaches were tested on 18 ALS patients to: (1) compare performance between a conventional matrix speller paradigm (Matrix-P, mean visual angle 6°) and a new speller paradigm with peripherally distributed stimuli (Peripheral-P, mean visual angle 8.8°); and (2) assess performance while spelling 42 characters online continuously, without a break. In the Peripheral-P condition, 12 subjects attained higher than 80% feedback accuracy during online performance, and 7 of these subjects obtained higher than 90% accuracy. The experimental results showed that the Peripheral-P condition yielded performance comparable to the conventional Matrix-P condition (p > 0.05) in accuracy and information transfer rate. This paper introduces a new display approach that leaves the center of the monitor open for feedback and/or other display elements, such as movies, games, art, or displays from other AAC software or conventional software tools.

Subject-Independent ERP-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) are desirable for people to express their thoughts, especially those with profound disabilities in communication. The classification of brain patterns for each different subject requires an extensively time-consuming learning stage specific to that person, in order to reach satisfactory accuracy performance. The training session could also be infeasible for disabled patients as they may not fully understand the training instructions. In this paper, we propose a unified classification scheme based on ensemble classifier, dynamic stopping, and adaptive learning. We apply this scheme on the P300-based BCI, with the subject-independent manner, where no learning session is required for new experimental users. According to our theoretical analysis and empirical results, the harmonized integration of these three methods can significantly boost up the average accuracy from 75.00% to 91.26%, while at the same time reduce the average spelling time from 12.62 to 6.78 iterations, approximately to two-fold faster. The experiments were conducted on a large public dataset which had been used in other related studies. Direct comparisons between our work with the others' are also reported in details.

Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain–computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user’s environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.

Optimizing the stimulus presentation paradigm design for the P300-based brain-computer interface using performance prediction

OBJECTIVE

The role of a brain-computer interface (BCI) is to discern a user's intended message or action by extracting and decoding relevant information from brain signals. Stimulus-driven BCIs, such as the P300 speller, rely on detecting event-related potentials (ERPs) in response to a user attending to relevant or target stimulus events. However, this process is error-prone because the ERPs are embedded in noisy electroencephalography (EEG) data, representing a fundamental problem in communication of the uncertainty in the information that is received during noisy transmission. A BCI can be modeled as a noisy communication system and an information-theoretic approach can be exploited to design a stimulus presentation paradigm to maximize the information content that is presented to the user. However, previous methods that focused on designing error-correcting codes failed to provide significant performance improvements due to underestimating the effects of psycho-physiological factors on the P300 ERP elicitation process and a limited ability to predict online performance with their proposed methods. Maximizing the information rate favors the selection of stimulus presentation patterns with increased target presentation frequency, which exacerbates refractory effects and negatively impacts performance within the context of an oddball paradigm. An information-theoretic approach that seeks to understand the fundamental trade-off between information rate and reliability is desirable.

APPROACH

We developed a performance-based paradigm (PBP) by tuning specific parameters of the stimulus presentation paradigm to maximize performance while minimizing refractory effects. We used a probabilistic-based performance prediction method as an evaluation criterion to select a final configuration of the PBP.

MAIN RESULTS

With our PBP, we demonstrate statistically significant improvements in online performance, both in accuracy and spelling rate, compared to the conventional row-column paradigm.

SIGNIFICANCE

By accounting for refractory effects, an information-theoretic approach can be exploited to significantly improve BCI performance across a wide range of performance levels.

Real-time classification of auditory sentences using evoked cortical activity in humans

OBJECTIVE

Recent research has characterized the anatomical and functional basis of speech perception in the human auditory cortex. These advances have made it possible to decode speech information from activity in brain regions like the superior temporal gyrus, but no published work has demonstrated this ability in real-time, which is necessary for neuroprosthetic brain-computer interfaces.

APPROACH

Here, we introduce a real-time neural speech recognition (rtNSR) software package, which was used to classify spoken input from high-resolution electrocorticography signals in real-time. We tested the system with two human subjects implanted with electrode arrays over the lateral brain surface. Subjects listened to multiple repetitions of ten sentences, and rtNSR classified what was heard in real-time from neural activity patterns using direct sentence-level and HMM-based phoneme-level classification schemes.

MAIN RESULTS

We observed single-trial sentence classification accuracies of [Formula: see text] or higher for each subject with less than 7 minutes of training data, demonstrating the ability of rtNSR to use cortical recordings to perform accurate real-time speech decoding in a limited vocabulary setting.

SIGNIFICANCE

Further development and testing of the package with different speech paradigms could influence the design of future speech neuroprosthetic applications.

Incorporation of dynamic stopping strategy into the high-speed SSVEP-based BCIs

OBJECTIVE

Electroencephalography (EEG) is a non-linear and non-stationary process, as a result, its features are unstable and often vary in quality across trials, which poses significant challenges to brain-computer interfaces (BCIs). One remedy to this problem is to adaptively collect sufficient EEG evidence using dynamic stopping (DS) strategies. The high-speed steady-state visual evoked potential (SSVEP)-based BCI has experienced tremendous progress in recent years. This study aims to further improve the high-speed SSVEP-based BCI by incorporating the DS strategy.

APPROACH

This study involves the development of two different DS strategies for a high-speed SSVEP-based BCI, which were based on the Bayes estimation and the discriminant analysis, respectively. To evaluate their performance, they were compared with the conventional fixed stopping (FS) strategy using simulated online tests on both our collected data and a public dataset. Two most effective SSVEP recognition methods were used for comparison, including the extended canonical correlation analysis (CCA) and the ensemble task-related component analysis (TRCA).

MAIN RESULTS

The DS strategies achieved significantly higher information transfer rates (ITRs) than the FS strategy for both datasets, improving 9.78% for the Bayes-based DS and 6.7% for the discriminant-based DS. Specifically, the discriminant-based DS strategy using ensemble TRCA performed the best for our collected data, reaching an average ITR of 353.3  ±  67.1 bits min^-1 with a peak of 460 bits min^-1. The Bayes-based DS strategy using ensemble TRCA had the highest ITR for the public dataset, reaching an average of 230.2  ±  65.8 bits min^-1 with a peak of 304.1 bits min^-1.

SIGNIFICANCE

This study demonstrates that the proposed dynamic stopping strategies can further improve the performance of a SSVEP-based BCI, and hold promise for practical applications.

Stable long-term BCI-enabled communication in ALS and locked-in syndrome using LFP signals

Restoring communication for people with locked-in syndrome remains a challenging clinical problem without a reliable solution. Recent studies have shown that people with paralysis can use brain-computer interfaces (BCIs) based on intracortical spiking activity to efficiently type messages. However, due to neuronal signal instability, most intracortical BCIs have required frequent calibration and continuous assistance of skilled engineers to maintain performance. Here, an individual with locked-in syndrome due to brain stem stroke and an individual with tetraplegia secondary to amyotrophic lateral sclerosis (ALS) used a simple communication BCI based on intracortical local field potentials (LFPs) for 76 and 138 days, respectively, without recalibration and without significant loss of performance. BCI spelling rates of 3.07 and 6.88 correct characters/minute allowed the participants to type messages and write emails. Our results indicate that people with locked-in syndrome could soon use a slow but reliable LFP-based BCI for everyday communication without ongoing intervention from a technician or caregiver. NEW & NOTEWORTHY This study demonstrates, for the first time, stable repeated use of an intracortical brain-computer interface by people with tetraplegia over up to four and a half months. The approach uses local field potentials (LFPs), signals that may be more stable than neuronal action potentials, to decode participants' commands. Throughout the several months of evaluation, the decoder remained unchanged; thus no technical interventions were required to maintain consistent brain-computer interface operation.

Evaluating Brain-Computer Interface Performance in an ALS Population: Checkerboard and Color Paradigms

The objective of this study was to investigate the performance of 3 brain-computer interface (BCI) paradigms in an amyotrophic lateral sclerosis (ALS) population (n = 11). Using a repeated-measures design, participants completed 3 BCI conditions: row/column (RCW), checkerboard (CBW), and gray-to-color (CBC). Based on previous studies, it is hypothesized that the CBC and CBW conditions will result in higher accuracy, information transfer rate, waveform amplitude, and user preference over the RCW condition. An offline dynamic stopping simulation will also increase information transfer rate. Higher mean accuracy was observed in the CBC condition (89.7%), followed by the CBW (84.3%) condition, and lowest in the RCW condition (78.7%); however, these differences did not reach statistical significance ( P = .062). Eight of the eleven participants preferred the CBC and the remaining three preferred the CBW conditions. The offline dynamic stopping simulation significantly increased information transfer rate ( P = .005) and decreased accuracy ( P < .000). The findings of this study suggest that color stimuli provide a modest improvement in performance and that participants prefer color stimuli over monochromatic stimuli. Given these findings, BCI paradigms that use color stimuli should be considered for individuals who have ALS.

Brain-Computer Interfaces for Augmentative and Alternative Communication: A Tutorial

PURPOSE

Brain-computer interfaces (BCIs) have the potential to improve communication for people who require but are unable to use traditional augmentative and alternative communication (AAC) devices. As BCIs move toward clinical practice, speech-language pathologists (SLPs) will need to consider their appropriateness for AAC intervention.

METHOD

This tutorial provides a background on BCI approaches to provide AAC specialists foundational knowledge necessary for clinical application of BCI. Tutorial descriptions were generated based on a literature review of BCIs for restoring communication.

RESULTS

The tutorial responses directly address 4 major areas of interest for SLPs who specialize in AAC: (a) the current state of BCI with emphasis on SLP scope of practice (including the subareas: the way in which individuals access AAC with BCI, the efficacy of BCI for AAC, and the effects of fatigue), (b) populations for whom BCI is best suited, (c) the future of BCI as an addition to AAC access strategies, and (d) limitations of BCI.

CONCLUSION

Current BCIs have been designed as access methods for AAC rather than a replacement; therefore, SLPs can use existing knowledge in AAC as a starting point for clinical application. Additional training is recommended to stay updated with rapid advances in BCI.

Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis

OBJECTIVES

Amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disease, restricts patients' communication capacity a few years after onset. A proof-of-concept of brain-computer interface (BCI) has shown promise in ALS and "locked-in" patients, mostly in pre-clinical studies or with only a few patients, but performance was estimated not high enough to support adoption by people with physical limitation of speech. Here, we evaluated a visual BCI device in a clinical study to determine whether disabled people with multiple deficiencies related to ALS would be able to use BCI to communicate in a daily environment.

METHODS

After clinical evaluation of physical, cognitive and language capacities, 20 patients with ALS were included. The P300 speller BCI system consisted of electroencephalography acquisition connected to real-time processing software and separate keyboard-display control software. It was equipped with original features such as optimal stopping of flashes and word prediction. The study consisted of two 3-block sessions (copy spelling, free spelling and free use) with the system in several modes of operation to evaluate its usability in terms of effectiveness, efficiency and satisfaction.

RESULTS

The system was effective in that all participants successfully achieved all spelling tasks and was efficient in that 65% of participants selected more than 95% of the correct symbols. The mean number of correct symbols selected per minute ranged from 3.6 (without word prediction) to 5.04 (with word prediction). Participants expressed satisfaction: the mean score was 8.7 on a 10-point visual analog scale assessing comfort, ease of use and utility. Patients quickly learned how to operate the system, which did not require much learning effort.

CONCLUSION

With its word prediction and optimal stopping of flashes, which improves information transfer rate, the BCI system may be competitive with alternative communication systems such as eye-trackers. Remaining requirements to improve the device for suitable ergonomic use are in progress.

A Synchronous Motor Imagery Based Neural Physiological Paradigm for Brain Computer Interface Speller

Brain Computer Interface (BCI) speller is a typical BCI-based application to help paralyzed patients express their thoughts. This paper proposed a novel motor imagery based BCI speller with Oct-o-spell paradigm for word input. Furthermore, an intelligent input method was used for improving the performance of the BCI speller. For the English word spelling experiment, we compared synchronous control with previous asynchronous control under the same experimental condition. There were no significant differences between these two control methods in the classification accuracy, information transmission rate (ITR) or letters per minute (LPM). And the accuracy rates of over 70% validated the feasibility for these two control strategies. It was indicated that MI-based synchronous control protocol was feasible for BCI speller. And the efficiency of the predictive text entry (PTE) mode was superior to that of the Non-PTE mode.

High performance communication by people with paralysis using an intracortical brain-computer interface

Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O'Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4-4.2) and information throughput (by a factor of 2.2-4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.Clinical Trial No: NCT00912041.

Online BCI Typing using Language Model Classifiers by ALS Patients in their Homes

The P300 speller is a common brain-computer interface system that can provide a means of communication for patients with amyotrophic lateral sclerosis (ALS). Recent studies have shown that incorporating language information in signal classification can improve system performance, but they have largely been tested on healthy volunteers in a laboratory setting. The goal of this study was to demonstrate the functionality of the P300 speller system with language models when used by ALS patients in their homes. Six ALS patients with functional ratings ranging from two to 28 participated in this study. All subjects had improved offline performance when using a language model and five subjects were able to type at least six characters per minute with over 84% accuracy in online sessions. The results of this study indicate that the improvements in performance using language models in the P300 speller translate into the ALS population, which could help to make it a viable assistive device.

Unintended Consequences of Sensor, Signal, and Imaging Informatics: New Problems and New Solutions

OBJECTIVES

This synopsis presents a selection for the IMIA (International Medical Informatics Association) Yearbook 2016 of excellent research in the broad field of Sensor, Signal and Imaging Informatics published in the year 2015, with a focus on Unintended consequences: new problems and new solutions.

METHODS

We performed a systematic initial selection and a double blind peer review process to find the best papers in this domain published in 2015, from the PubMed and Web of Science databases. The set of MesH keywords used was provided by experts.

RESULTS

The constant advances in medical technology allow ever more relevant diagnostic and therapeutic approaches to be designed. Nevertheless, there is a need to acquire expert knowledge of these innovations in order to identify precociously new associated problems for which new solutions need to be designed and developed.

Applying dynamic data collection to improve dry electrode system performance for a P300-based brain-computer interface

OBJECTIVE

Dry electrodes have an advantage over gel-based 'wet' electrodes by providing quicker set-up time for electroencephalography recording; however, the potentially poorer contact can result in noisier recordings. We examine the impact that this may have on brain-computer interface communication and potential approaches for mitigation.

APPROACH

We present a performance comparison of wet and dry electrodes for use with the P300 speller system in both healthy participants and participants with communication disabilities (ALS and PLS), and investigate the potential for a data-driven dynamic data collection algorithm to compensate for the lower signal-to-noise ratio (SNR) in dry systems.

MAIN RESULTS

Performance results from sixteen healthy participants obtained in the standard static data collection environment demonstrate a substantial loss in accuracy with the dry system. Using a dynamic stopping algorithm, performance may have been improved by collecting more data in the dry system for ten healthy participants and eight participants with communication disabilities; however, the algorithm did not fully compensate for the lower SNR of the dry system. An analysis of the wet and dry system recordings revealed that delta and theta frequency band power (0.1-4 Hz and 4-8 Hz, respectively) are consistently higher in dry system recordings across participants, indicating that transient and drift artifacts may be an issue for dry systems.

SIGNIFICANCE

Using dry electrodes is desirable for reduced set-up time; however, this study demonstrates that online performance is significantly poorer than for wet electrodes for users with and without disabilities. We test a new application of dynamic stopping algorithms to compensate for poorer SNR. Dynamic stopping improved dry system performance; however, further signal processing efforts are likely necessary for full mitigation.

Using the detectability index to predict P300 speller performance

OBJECTIVE

The P300 speller is a popular brain-computer interface (BCI) system that has been investigated as a potential communication alternative for individuals with severe neuromuscular limitations. To achieve acceptable accuracy levels for communication, the system requires repeated data measurements in a given signal condition to enhance the signal-to-noise ratio of elicited brain responses. These elicited brain responses, which are used as control signals, are embedded in noisy electroencephalography (EEG) data. The discriminability between target and non-target EEG responses defines a user's performance with the system. A previous P300 speller model has been proposed to estimate system accuracy given a certain amount of data collection. However, the approach was limited to a static stopping algorithm, i.e. averaging over a fixed number of measurements, and the row-column paradigm. A generalized method that is also applicable to dynamic stopping (DS) algorithms and other stimulus paradigms is desirable.

APPROACH

We developed a new probabilistic model-based approach to predicting BCI performance, where performance functions can be derived analytically or via Monte Carlo methods. Within this framework, we introduce a new model for the P300 speller with the Bayesian DS algorithm, by simplifying a multi-hypothesis to a binary hypothesis problem using the likelihood ratio test. Under a normality assumption, the performance functions for the Bayesian algorithm can be parameterized with the detectability index, a measure which quantifies the discriminability between target and non-target EEG responses.

MAIN RESULTS

Simulations with synthetic and empirical data provided initial verification of the proposed method of estimating performance with Bayesian DS using the detectability index. Analysis of results from previous online studies validated the proposed method.

SIGNIFICANCE

The proposed method could serve as a useful tool to initially assess BCI performance without extensive online testing, in order to estimate the amount of data required to achieve a desired accuracy level.

BCI in practice

Brain-computer interfaces are systems that use signals recorded from the brain to enable communication and control applications for individuals who have impaired function. This technology has developed to the point that it is now being used by individuals who can actually benefit from it. However, there are several outstanding issues that prevent widespread use. These include the ease of obtaining high-quality recordings by home users, the speed, and accuracy of current devices and adapting applications to the needs of the user. In this chapter, we discuss some of these unsolved issues.

A P300-Based Threshold-Free Brain Switch and Its Application in Wheelchair Control

The key issue of electroencephalography (EEG)-based brain switches is to detect the control and idle states in an asynchronous manner. Most existing methods rely on a threshold. However, it is often time consuming to select a satisfactory threshold, and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals. This paper presents a new P300-based threshold-free brain switch. Specifically, one target button and three pseudo buttons, which are intensified in a random order to produce P300 potential, are set in the graphical user interface. The user can issue a switch command by focusing on the target button. Two support vector machine (SVM) classifiers, namely, SVM1 and SVM2, are used in the detection algorithm. During detection, we first obtained four SVM scores, corresponding to the four flashing buttons, by applying SVM1 to the ongoing EEG. If the SVM score corresponding to the target button was negative or not at the maximum, then an idle state was determined. Moreover, if the target button had a maximum and positive score, then we fed the four SVM scores as features into SVM2 to further discriminate the control and idle states. As an application, this brain switch was used to produce a start/stop command for an intelligent wheelchair, of which the left, right, forward, backward functions were carried out by an autonomous navigation system. Several experiments were conducted with eight healthy subjects and five patients with spinal cord injuries (SCIs). The experimental results not only demonstrated the effectiveness of our approach but also illustrated the potential application for patients with SCIs.

Integrating language models into classifiers for BCI communication: a review

OBJECTIVE

The present review systematically examines the integration of language models to improve classifier performance in brain-computer interface (BCI) communication systems.

APPROACH

The domain of natural language has been studied extensively in linguistics and has been used in the natural language processing field in applications including information extraction, machine translation, and speech recognition. While these methods have been used for years in traditional augmentative and assistive communication devices, information about the output domain has largely been ignored in BCI communication systems. Over the last few years, BCI communication systems have started to leverage this information through the inclusion of language models.

MAIN RESULTS

Although this movement began only recently, studies have already shown the potential of language integration in BCI communication and it has become a growing field in BCI research. BCI communication systems using language models in their classifiers have progressed down several parallel paths, including: word completion; signal classification; integration of process models; dynamic stopping; unsupervised learning; error correction; and evaluation.

SIGNIFICANCE

Each of these methods have shown significant progress, but have largely been addressed separately. Combining these methods could use the full potential of language model, yielding further performance improvements. This integration should be a priority as the field works to create a BCI system that meets the needs of the amyotrophic lateral sclerosis population.

Adaptive-projection intrinsically transformed multivariate empirical mode decomposition in cooperative brain-computer interface applications

An extension to multivariate empirical mode decomposition (MEMD), termed adaptive-projection intrinsically transformed MEMD (APIT-MEMD), is proposed to cater for power imbalances and inter-channel correlations in real-world multichannel data. It is shown that the APIT-MEMD exhibits similar or better performance than MEMD for a large number of projection vectors, whereas it outperforms MEMD for the critical case of a small number of projection vectors within the sifting algorithm. We also employ the noise-assisted APIT-MEMD within our proposed intrinsic multiscale analysis framework and illustrate the advantages of such an approach in notoriously noise-dominated cooperative brain-computer interface (BCI) based on the steady-state visual evoked potentials and the P300 responses. Finally, we show that for a joint cognitive BCI task, the proposed intrinsic multiscale analysis framework improves system performance in terms of the information transfer rate.

Incorporation of Inter-Subject Information to Improve the Accuracy of Subject-Specific P300 Classifiers

Although the inter-subject information has been demonstrated to be effective for a rapid calibration of the P300-based brain–computer interface (BCI), it has never been comprehensively tested to find if the incorporation of heterogeneous data could enhance the accuracy. This study aims to improve the subject-specific P300 classifier by adding other subject’s data. A classifier calibration strategy, weighted ensemble learning generic information (WELGI), was developed, in which elementary classifiers were constructed by using both the intra- and inter-subject information and then integrated into a strong classifier with a weight assessment. 55 subjects were recruited to spell 20 characters offline using the conventional P300-based BCI, i.e. the P300-speller. Four different metrics, the P300 accuracy and precision, the round accuracy, and the character accuracy, were performed for a comprehensive investigation. The results revealed that the classifier constructed on the training dataset in combination with adding other subject’s data was significantly superior to that without the inter-subject information. Therefore, the WELGI is an effective classifier calibration strategy which uses the inter-subject information to improve the accuracy of subject-specific P300 classifiers, and could also be applied to other BCI paradigms.