Brain-computer interfaces for communication

EEG-based Neurophysiological Indicators in Pronoun Resolution Using Feature Analysis

BACKGROUND

Pronoun resolution is a crucial aspect of language comprehension, yet its underlying neural mechanisms remain poorly understood. While previous studies have explored individual linguistic factors, a systematic analysis of Electroencephalography (EEG)-based neurophysiological indicators across different resolution cues (gender, verb bias, and discourse focus) remains unexplored, limiting our understanding of neural-cognitive processes.

NEW METHOD

We developed an approach combining ReliefF feature selection and Linear Discriminant Analysis (LDA) to analyze EEG data from twenty participants during pronoun resolution tasks. The method examined neural indicators focusing on power spectral density (PSD) and time-domain features, including Zero-Crossing Rate and Peak-to-Peak amplitude.

RESULTS

We identified crucial neural indicators across 14 channels and 4 frequency bands, highlighting PSD features in specific channels (AF3, AF4, FC6, F4, T7, T8, and O2) across theta, beta, and gamma bands. Gender-cue processing exhibited enhanced neural responses in prefrontal and temporal regions with shorter reaction times (748.77 ms) compared to verb bias (903.20 ms) and discourse focus (948.92 ms).

COMPARISON WITH EXISTING METHODS

Unlike previous studies examining individual linguistic factors, our approach simultaneously analyzed multiple resolution cues. The method achieved significant above-chance classification accuracy (49.08% vs. 33.33%) across three linguistic factors. This multi-factor analysis provides a more nuanced understanding of pronoun resolution processes than traditional single-factor studies.

CONCLUSIONS

Our findings suggest a more efficient, feature-driven processing mechanism for gender-cue resolution, contrasting with more complex, reasoning-dependent processing of verb semantics and discourse cues. These insights have implications for developing computational models of language processing and potential clinical applications for language disorders.

Comparing a BCI communication system in a patient with Multiple System Atrophy, with an animal model

Paralysis affects many people worldwide, and the people affected often suffer from impaired communication. We developed a microelectrode-based Brain-Computer Interface (BCI) for enabling communication in patients affected by paralysis, and implanted it in a patient with Multiple System Atrophy (MSA), a neurodegenerative disease that causes widespread neural symptoms including paralysis. To verify the effectiveness of the BCI system, it was also tested by implanting it in a non-human primate (NHP). Data from the human and NHP were used to train binary classifiers two different types of machine learning models: a Linear Discriminant Analysis (LDA) model, and a Long Short-Term Memory (LSTM)-based Artificial Neural Network (ANN). The LDA model performed at up to 72.7 % accuracy for binary decoding in the human patient, however, performance was highly variable and was much lower on most recording days. The BCI system was able to accurately decode movement vs non-movement in the NHP (accuracy using LDA: 82.7 ± 3.3 %, LSTM: 83.7 ± 2.2 %, 95 % confidence intervals), however it was not able to with recordings from the human patient (accuracy using LDA: 47.0 ± 5.1 %, LSTM: 44.6 ± 9.9 %, 95 % confidence intervals). We discuss how neurodegenerative diseases such as MSA can impede BCI-based communication, and postulate on the mechanisms by which this may occur.

Stakeholder's perspective on brain-computer interfaces for children and young adults with cerebral palsy

Communication Brain-Computer Interfaces (cBCIs) are a promising tool for people with motor and speech impairment, in particular for children and young adults with communication impairments, for example due to cerebral palsy (CP). Here we aimed to create a solid basis for the user-centered design of cBCIs for children and young adults with severe CP by investigating the perspectives of their parents/caregivers and health care professionals on communication and cBCIs. We conducted an online survey on 1) current communication problems and usability of used aids, 2) interest in cBCIs, and 3) preference for specific types of cBCIs. A total of 19 parents/caregivers and 36 health care professionals who interacted directly with children and young adults (8-25 years old) with severe CP, corresponding to Gross Motor Function Classification System level IV or V, participated. Both groups of respondents indicated that motor impairment occurred the most frequently and had the greatest impact on communication. The currently used communication aids included mainly no/low-tech aids and high-tech aids. The majority of health care professionals and parents/caregivers reported an interest in cBCIs, with a slight preference for implanted electrodes over non-implanted ones, and no preference for either of the two proposed mental BCI control strategies. Results indicate that cBCIs should be considered for a subpopulation of children and young adults with severe CP, and that in the development of cBCIs for this group both visual stimuli and sensorimotor rhythms, as well as the use of implanted electrodes, should be considered.

A click-based electrocorticographic brain-computer interface enables long-term high-performance switch scan spelling

BACKGROUND

Brain-computer interfaces (BCIs) can restore communication for movement- and/or speech-impaired individuals by enabling neural control of computer typing applications. Single command click detectors provide a basic yet highly functional capability.

METHODS

We sought to test the performance and long-term stability of click decoding using a chronically implanted high density electrocorticographic (ECoG) BCI with coverage of the sensorimotor cortex in a human clinical trial participant (ClinicalTrials.gov, NCT03567213) with amyotrophic lateral sclerosis. We trained the participant's click detector using a small amount of training data (<44 min across 4 days) collected up to 21 days prior to BCI use, and then tested it over a period of 90 days without any retraining or updating.

RESULTS

Using a click detector to navigate a switch scanning speller interface, the study participant can maintain a median spelling rate of 10.2 characters per min. Though a transient reduction in signal power modulation can interrupt usage of a fixed model, a new click detector can achieve comparable performance despite being trained with even less data (<15 min, within 1 day).

CONCLUSIONS

These results demonstrate that a click detector can be trained with a small ECoG dataset while retaining robust performance for extended periods, providing functional text-based communication to BCI users.

The ethical significance of user-control in AI-driven speech-BCIs: a narrative review

AI-driven brain-computed interfaces aimed at restoring speech for individuals living with locked-in-syndrome are paired with ethical implications for user's autonomy, privacy and responsibility. Embedding options for sufficient levels of user-control in speech-BCI design has been proposed to mitigate these ethical challenges. However, how user-control in speech-BCIs is conceptualized and how it relates to these ethical challenges is underdetermined. In this narrative literature review, we aim to clarify and explicate the notion of user-control in speech-BCIs, to better understand in what way user-control could operationalize user's autonomy, privacy and responsibility and explore how such suggestions for increasing user-control can be translated to recommendations for the design or use of speech-BCIs. First, we identified types of user control, including executory control that can protect voluntariness of speech, and guidance control that can contribute to semantic accuracy. Second, we identified potential causes for a loss of user-control, including contributions of predictive language models, a lack of ability for neural control, or signal interference and external control. Such a loss of user control may have implications for semantic accuracy and mental privacy. Third we explored ways to design for user-control. While embedding initiation signals for users may increase executory control, they may conflict with other aims such as speed and continuity of speech. Design mechanisms for guidance control remain largely conceptual, similar trade-offs in design may be expected. We argue that preceding these trade-offs, the overarching aim of speech-BCIs needs to be defined, requiring input from current and potential users. Additionally, conceptual clarification of user-control and other (ethical) concepts in this debate has practical relevance for BCI researchers. For instance, different concepts of inner speech may have distinct ethical implications. Increased clarity of such concepts can improve anticipation of ethical implications of speech-BCIs and may help to steer design decisions.

Endovascular Brain-Computer Interfaces in Poststroke Paralysis

Stroke is a leading cause of paralysis, most frequently affecting the upper limbs and vocal folds. Despite recent advances in care, stroke recovery invariably reaches a plateau, after which there are permanent neurological impairments. Implantable brain-computer interface devices offer the potential to bypass permanent neurological lesions. They function by (1) recording neural activity, (2) decoding the neural signal occurring in response to volitional motor intentions, and (3) generating digital control signals that may be used to control external devices. While brain-computer interface technology has the potential to revolutionize neurological care, clinical translation has been limited. Endovascular arrays present a novel form of minimally invasive brain-computer interface devices that have been deployed in human subjects during early feasibility studies. This article provides an overview of endovascular brain-computer interface devices and critically evaluates the patient with stroke as an implant candidate. Future opportunities are mapped, along with the challenges arising when decoding neural activity following infarction. Limitations arise when considering intracerebral hemorrhage and motor cortex lesions; however, future directions are outlined that aim to address these challenges.

Update on How to Approach a Patient with Locked-In Syndrome and Their Communication Ability

Locked-in syndrome (LIS) is a rare and challenging condition that results in tetraplegia and cranial nerve paralysis while maintaining consciousness and variable cognitive function. Once acute management is completed, it is important to work with the patient on developing a plan to maintain and improve their quality of life (QOL). A key component towards increasing or maintaining QOL within this population involves the establishment of a functional communication system. Evaluating cognition in patients with LIS is vital for evaluating patients' communication needs along with physical rehabilitation to maximize their QOL. In the past decade or so, there has been an increase in research surrounding brain-computer interfaces to improve communication abilities for paralyzed patients. This article provides an update on the available technology and the protocol for finding the best way for patients with this condition to communicate. This article aims to increase knowledge of how to enhance and manage communication among LIS patients.

Multivariate fMRI responses in superior temporal cortex predict visual contributions to, and individual differences in, the intelligibility of noisy speech

Humans have the unique ability to decode the rapid stream of language elements that constitute speech, even when it is contaminated by noise. Two reliable observations about noisy speech perception are that seeing the face of the talker improves intelligibility and the existence of individual differences in the ability to perceive noisy speech. We introduce a multivariate BOLD fMRI measure that explains both observations. In two independent fMRI studies, clear and noisy speech was presented in visual, auditory and audiovisual formats to thirty-seven participants who rated intelligibility. An event-related design was used to sort noisy speech trials by their intelligibility. Individual-differences multidimensional scaling was applied to fMRI response patterns in superior temporal cortex and the dissimilarity between responses to clear speech and noisy (but intelligible) speech was measured. Neural dissimilarity was less for audiovisual speech than auditory-only speech, corresponding to the greater intelligibility of noisy audiovisual speech. Dissimilarity was less in participants with better noisy speech perception, corresponding to individual differences. These relationships held for both single word and entire sentence stimuli, suggesting that they were driven by intelligibility rather than the specific stimuli tested. A neural measure of perceptual intelligibility may aid in the development of strategies for helping those with impaired speech perception.

Use of Invasive Brain-Computer Interfaces in Pediatric Neurosurgery: Technical and Ethical Considerations

Invasive brain-computer interfaces hold promise to alleviate disabilities in individuals with neurologic injury, with fully implantable brain-computer interface systems expected to reach the clinic in the upcoming decade. Children with severe neurologic disabilities, like quadriplegic cerebral palsy or cervical spine trauma, could benefit from this technology. However, they have been excluded from clinical trials of intracortical brain-computer interface to date. In this manuscript, we discuss the ethical considerations related to the use of invasive brain-computer interface in children with severe neurologic disabilities. We first review the technical hardware and software considerations for the application of intracortical brain-computer interface in children. We then discuss ethical issues related to motor brain-computer interface use in pediatric neurosurgery. Finally, based on the input of a multidisciplinary panel of experts in fields related to brain-computer interface (functional and restorative neurosurgery, pediatric neurosurgery, mathematics and artificial intelligence research, neuroengineering, pediatric ethics, and pragmatic ethics), we then formulate initial recommendations regarding the clinical use of invasive brain-computer interfaces in children.

Writing Blindly in Incomplete Locked-In Syndrome with A Custom-Made Switch-Operated Voice-Scanning Communicator—A Case Report

Background: Locked-In Syndrome (LIS) is a rare neurological condition in which patients’ ability to move, interact, and communicate is impaired despite their being conscious and awake. After assessing the patient’s needs, we developed a customized device for an LIS patient, as the commercial augmentative and alternative communication (AAC) devices could not be used. Methods: A 51-year-old woman with incomplete LIS for 15 years came to our laboratory seeking a communication tool. After excluding the available AAC devices, a careful evaluation led to the creation of a customized device (hardware + software). Two years later, we assessed the patient’s satisfaction with the device. Results: A switch-operated voice-scanning communicator, which the patient could control by residual movement of her thumb without seeing the computer screen, was implemented, together with postural strategies. The user and her family were generally satisfied with the customized device, with a top rating for its effectiveness: it fit well the patient’s communication needs. Conclusions: Using customized AAC and strategies provides greater opportunities for patients with LIS to resolve their communication problems. Moreover, listening to the patient’s and family’s needs can help increase the AAC’s potential. The presented switch-operated voice-scanning communicator is available for free on request to the authors.

Methodological Recommendations for Studies on the Daily Life Implementation of Implantable Communication-Brain-Computer Interfaces for Individuals With Locked-in Syndrome

Implantable brain-computer interfaces (BCIs) promise to be a viable means to restore communication in individuals with locked-in syndrome (LIS). In 2016, we presented the world-first fully implantable BCI system that uses subdural electrocorticography electrodes to record brain signals and a subcutaneous amplifier to transmit the signals to the outside world, and that enabled an individual with LIS to communicate via a tablet computer by selecting icons in spelling software. For future clinical implementation of implantable communication-BCIs, however, much work is still needed, for example, to validate these systems in daily life settings with more participants, and to improve the speed of communication. We believe the design and execution of future studies on these and other topics may benefit from the experience we have gained. Therefore, based on relevant literature and our own experiences, we here provide an overview of procedures, as well as recommendations, for recruitment, screening, inclusion, imaging, hospital admission, implantation, training, and support of participants with LIS, for studies on daily life implementation of implantable communication-BCIs. With this article, we not only aim to inform the BCI community about important topics of concern, but also hope to contribute to improved methodological standardization of implantable BCI research.

A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities

Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people.

Clinical trial registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.

Exploring the frontiers in reality-enhanced service communication: from augmented and virtual reality to neuro-enhanced reality

Purpose

The authors explore neuro-enhanced reality (NeR) as a novel approach for enhancing service communication between customers, frontline employees, and service organizations that extends beyond current state-of-the-art approaches based on augmented reality (AR) and virtual reality (VR) technologies.

Design/methodology/approach

The authors first take stock of research on reality-enhanced service communication with AR and VR, then complement these insights with emerging neuroscientific research to conceptualize how NeR enables innovative forms of service communication. On this basis, the authors develop a research agenda to guide the future study and managerial exploitation of NeR.

Findings

AR and VR already offer unique affordances for digital-to-physical communication, but these can be extended with NeR. Specifically, NeR supports neuro-to-digital and digital-to-neuro communication based on neuroimaging (e.g. controlling digital content through thought) and neurostimulation (e.g. eliciting brain responses based on digital content). This provides a basis for outlining possible applications of NeR across service settings.

Originality/value

The authors advance knowledge on reality-enhanced service communication with AR and VR, whilst also demonstrating how neuroscientific research can be extended from understanding brain activity to generating novel service interactions.

Brain-Computer Interfaces Systems for Upper and Lower Limb Rehabilitation: A Systematic Review

In recent years, various studies have demonstrated the potential of electroencephalographic (EEG) signals for the development of brain-computer interfaces (BCIs) in the rehabilitation of human limbs. This article is a systematic review of the state of the art and opportunities in the development of BCIs for the rehabilitation of upper and lower limbs of the human body. The systematic review was conducted in databases considering using EEG signals, interface proposals to rehabilitate upper/lower limbs using motor intention or movement assistance and utilizing virtual environments in feedback. Studies that did not specify which processing system was used were excluded. Analyses of the design processing or reviews were excluded as well. It was identified that 11 corresponded to applications to rehabilitate upper limbs, six to lower limbs, and one to both. Likewise, six combined visual/auditory feedback, two haptic/visual, and two visual/auditory/haptic. In addition, four had fully immersive virtual reality (VR), three semi-immersive VR, and 11 non-immersive VR. In summary, the studies have demonstrated that using EEG signals, and user feedback offer benefits including cost, effectiveness, better training, user motivation and there is a need to continue developing interfaces that are accessible to users, and that integrate feedback techniques.

Brain-Computer Interfaces for Communication: Preferences of Individuals With Locked-in Syndrome

BACKGROUND

Brain-computer interfaces (BCIs) have been proposed as an assistive technology (AT) allowing people with locked-in syndrome (LIS) to use neural signals to communicate. To design a communication BCI (cBCI) that is fully accepted by the users, their opinion should be taken into consideration during the research and development process.

OBJECTIVE

We assessed the preferences of prospective cBCI users regarding (1) the applications they would like to control with a cBCI, (2) the mental strategies they would prefer to use to control the cBCI, and (3) when during their clinical trajectory they would like to be informed about AT and cBCIs. Furthermore, we investigated if individuals diagnosed with progressive and sudden onset (SO) disorders differ in their opinion.

METHODS

We interviewed 28 Dutch individuals with LIS during a 3-hour home visit using multiple-choice, ranking, and open questions. During the interview, participants were informed about BCIs and the possible mental strategies.

RESULTS

Participants rated (in)direct forms of communication, computer use, and environmental control as the most desired cBCI applications. In addition, active cBCI control strategies were preferred over reactive strategies. Furthermore, individuals with progressive and SO disorders preferred to be informed about AT and cBCIs at the moment they would need it.

CONCLUSIONS

We show that individuals diagnosed with progressive and SO disorders preferred, in general, the same applications, mental strategies, and time of information. By collecting the opinion of a large sample of individuals with LIS, this study provides valuable information to stakeholders in cBCI and other AT development.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.

Locked-In Syndrome: Practical Rehabilitation Management

Locked-in syndrome is a rare and devastating condition that results in tetraplegia, lower cranial nerve paralysis, and anarthria with preserved cognition, vertical gaze, and upper eyelid movements. Although acute management is much like that of any severe stroke, rehabilitation and recovery of these patients have not been previously described. Challenges relevant to this population include blood pressure management and orthostasis, timing and appropriateness of reinstating oral feeding, ventilatory support, decannulation after tracheostomy, bowel and bladder management, vestibular dysfunction, and eye care. Targeted rehabilitation of head, neck, and trunk stability to improve function, and proper fit in an appropriate wheelchair are essential to assist with mobility. Rehabilitation interventions should include a focus on distal motor control and upright tolerance training followed by balance and mobility exercises. In addition, special considerations must be given to developing early methods of communication through use of augmentative systems to call for help and express needs. These systems along with additional technology provide the basis to promote connectivity to family and friends through the use of social media and the internet. Establishment of communication, mobility, and connectivity is essential in promoting independence, autonomy, and improving quality of life. Overall, with specialized rehabilitative care and access to the proper equipment, long-term outcomes and quality of life in these patients can be favorable.

EEG-based emotion recognition using 4D convolutional recurrent neural network

In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.