Brain-computer interface use is a skill that user and system acquire together

BCI move: exploring pediatric BCI-controlled power mobility

Introduction

Children and young people (CYP) with severe physical disabilities often experience barriers to independent mobility, placing them at risk for developmental impairments and restricting their independence and participation. Pilot work suggests that brain-computer interface (BCIs) could enable powered mobility control for children with motor disabilities. We explored how severely disabled CYP could use BCI to achieve individualized, functional power mobility goals and acquire power mobility skills. We also explored the practicality of pediatric BCI-enabled power mobility.

Methods

Nine CYP aged 7-17 years with severe physical disabilities and their caregivers participated in up to 12 BCI-enabled power mobility training sessions focused on a personalized power mobility goal. Goal achievement was assessed using the Canadian Occupational Performance Measure (COPM) and Goal Attainment Scaling (GAS). The Assessment for Learning Powered Mobility (ALP) was used to measure session-by-session power mobility skill acquisition. BCI set-up and calibration metrics, perceived workload, and participant engagement were also reported.

Results

Significant improvements in COPM performance (Z = -2.869, adjusted p = 0.012) and satisfaction scores (Z = -2.809, adjusted p = 0.015) and GAS T scores (Z = -2.805, p = 0.005) were observed following the intervention. ALP scores displayed a small but significant increase over time (R 2 = 0.07-0.19; adjusted p = <0.001-0.039), with 7/9 participants achieving increased overall ALP scores following the intervention. Setup and calibration times were practical although calibration consistency was highly variable. Participants reported moderate workload with no significant change over time (R 2 = 0.00-0.13; adjusted p = 0.006-1.000), although there was a trend towards increased frustration over time(R 2 = 0.13; adjusted p = 0.006).

Discussion

Participants were highly engaged throughout the intervention. BCI-enabled power mobility appears to help CYP with severe physical disabilities achieve personalized power mobility goals and acquire power mobility skills. BCI-enabled power mobility training also appears to be practical, but BCI performance optimization and skill acquisition may be needed to translate this technology into clinical use.

Making Video Games More Inclusive for People Living With Motor Neuron Disease: Scoping Review

BACKGROUND

Evidence suggests that individuals with motor neuron disease (MND), a terminal illness, find enjoyment and social connection through video games. However, MND-related barriers can make gaming challenging, exacerbating feelings of boredom, stress, isolation, and loss of control over daily life.

OBJECTIVE

We scoped the evidence to describe relevant research and practice regarding what may help reduce difficulties for people with MND when playing video games.

METHODS

A scoping review was conducted using the Arksey and O'Malley framework, recent scoping review guidance, and engaging with people with lived experience of MND. Peer-reviewed studies were sourced from PubMed and the Swinburne University of Technology Library. Gray literature was identified from government, not-for-profit, commercial, and community websites. Data were extracted and summarized from the collected literature.

RESULTS

The evidence base, consisting of quantitative and qualitative research, lived experience stories, information resources, reviews, and guidelines, included 85 documents. Only 8 (9%) directly addressed video games and people with MND; however, these studies were limited in depth and quality. The primary technologies examined included customized information and communications technology for communication and control of computing systems (including desktop, laptop, smartphone, tablet, and console systems) and video game software and hardware (including hand controllers and accessibility features, such as difficulty level, speed, and remappable buttons and controls). Common factors in the evidence base highlight barriers and enablers to enjoying video games for people with MND. These include technological, physical, social, and economic challenges. Addressing these requires greater involvement of people with MND in game and technology research and development. Changes to information and communications technology, game software and hardware, policies, and guidelines are needed to better meet their needs.

CONCLUSIONS

There is a significant gap in understanding the lived experience of people with MND with video games and what makes playing them easier, including appropriate customization of technology and the social experience of games. This gap perpetuates exclusion from gaming communities and recreation, potentially worsening social isolation. Existing evidence suggesting viable options for future research and practice. Video game and information and communications technology research and development must prioritize qualitative and quantitative research with people with MND at an appropriate scale, with a focus on lived experience, use of improved participant engagement techniques, and user-focused design for more inclusive games. Practical work needs to increase awareness of what can help make games more inclusive, including incorporation of accessibility early in the game production process, early incorporation of accessibility in game production, and affordable options for customized interfaces and other devices to play games.

Motor task-to-task transfer learning for motor imagery brain-computer interfaces

Motor imagery (MI) is one of the popular control paradigms in the non-invasive brain-computer interface (BCI) field. MI-BCI generally requires users to conduct the imagination of movement (e.g., left or right hand) to collect training data for generating a classification model during the calibration phase. However, this calibration phase is generally time-consuming and tedious, as users conduct the imagination of hand movement several times without being given feedback for an extended period. This obstacle makes MI-BCI non user-friendly and hinders its use. On the other hand, motor execution (ME) and motor observation (MO) are relatively easier tasks, yield lower fatigue than MI, and share similar neural mechanisms to MI. However, few studies have integrated these three tasks into BCIs. In this study, we propose a new task-to-task transfer learning approach of 3-motor tasks (ME, MO, and MI) for building a better user-friendly MI-BCI. For this study, 28 subjects participated in 3-motor tasks experiment, and electroencephalography (EEG) was acquired. User opinions regarding the 3-motor tasks were also collected through questionnaire survey. The 3-motor tasks showed a power decrease in the alpha rhythm, known as event-related desynchronization, but with slight differences in the temporal patterns. In the classification analysis, the cross-validated accuracy (within-task) was 67.05 % for ME, 65.93 % for MI, and 73.16 % for MO on average. Consistently with the results, the subjects scored MI (3.16) as the most difficult task compared with MO (1.42) and ME (1.41), with p < 0.05. In the analysis of task-to-task transfer learning, where training and testing are performed using different task datasets, the ME-trained model yielded an accuracy of 65.93 % (MI test), which is statistically similar to the within-task accuracy (p > 0.05). The MO-trained model achieved an accuracy of 60.82 % (MI test). On the other hand, combining two datasets yielded interesting results. ME and 50 % of the MI-trained model (50-shot) classified MI with a 69.21 % accuracy, which outperformed the within-task accuracy (p < 0.05), and MO and 50 % of the MI-trained model showed an accuracy of 66.75 %. Of the low performers with a within-task accuracy of 70 % or less, 90 % (n = 21) of the subjects improved in training with ME, and 76.2 % (n = 16) improved in training with MO on the MI test at 50-shot. These results demonstrate that task-to-task transfer learning is possible and could be a promising approach to building a user-friendly training protocol in MI-BCI.

Towards user-centric BCI design: Markov chain-based user assessment for mental imagery EEG-BCIs

Objective.While electroencephalography (EEG)-based brain-computer interfaces (BCIs) have many potential clinical applications, their use is impeded by poor performance for many users. To improve BCI performance, either via enhanced signal processing or user training, it is critical to understand and describe each user's ability to perform mental control tasks and produce discernible EEG patterns. While classification accuracy has predominantly been used to assess user performance, limitations and criticisms of this approach have emerged, thus prompting the need to develop novel user assessment approaches with greater descriptive capability. Here, we propose a combination of unsupervised clustering and Markov chain models to assess and describe user skill.Approach.Using unsupervisedK-means clustering, we segmented the EEG signal space into regions representing pattern states that users could produce. A user's movement through these pattern states while performing different tasks was modeled using Markov chains. Finally, using the steady-state distributions and entropy rates of the Markov chains, we proposed two metricstaskDistinctandrelativeTaskInconsistencyto assess, respectively, a user's ability to (i) produce distinct task-specific patterns for each mental task and (ii) maintain consistent patterns during individual tasks.Main results.Analysis of data from 14 adolescents using a three-class BCI revealed significant correlations between thetaskDistinctandrelativeTaskInconsistencymetrics and classification F1 score. Moreover, analysis of the pattern states and Markov chain models yielded descriptive information regarding user performance not immediately apparent from classification accuracy.Significance.Our proposed user assessment method can be used in concert with classifier-based analysis to further understand the extent to which users produce task-specific, time-evolving EEG patterns. In turn, this information could be used to enhance user training or classifier design.

A gap between reasons for skilled use of BCI speech devices and reasons for utterances, with implications for speech ownership

The skilled use of a speech BCI device will draw upon practical experience gained through the use of that very device. The reasons a user may have for using a device in a particular way, reflecting that skill gained via familiarity with the device, may differ significantly from the reasons that a speaker might have for their utterances. The potential divergence between reasons constituting skilled use and BCI-mediated speech output may serve to make clear an instrumental relationship between speaker and BCI speech device. This will affect the way in which the device and the speech it produces for the user can be thought of as being "reasons responsive", hence the way in which the user can be said to be in control of their device. Ultimately, this divergence will come down to how ownership of produced speech can be considered. The upshot will be that skillful use of a synthetic speech device might include practices that diverge from standard speech in significant ways. This might further indicate that synthetic speech devices ought to be considered as different from, not continuous with, standard speech.

A large EEG database with users' profile information for motor imagery brain-computer interface research

We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users' profiles and their BCI performances, (2) studying how EEG signals properties varies for different users' profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users' profile information into the design of EEG signal classification algorithms.

EEG-CDILNet: a lightweight and accurate CNN network using circular dilated convolution for motor imagery classification

Objective.The combination of the motor imagery (MI) electroencephalography (EEG) signals and deep learning-based methods is an effective way to improve MI classification accuracy. However, deep learning-based methods often need too many trainable parameters. As a result, the trade-off between the network decoding performance and computational cost has always been an important challenge in the MI classification research.Approach.In the present study, we proposed a new end-to-end convolutional neural network (CNN) model called the EEG-circular dilated convolution (CDIL) network, which takes into account both the lightweight model and the classification accuracy. Specifically, the depth-separable convolution was used to reduce the number of network parameters and extract the temporal and spatial features from the EEG signals. CDIL was used to extract the time-varying deep features that were generated in the previous stage. Finally, we combined the features extracted from the two stages and used the global average pooling to further reduce the number of parameters, in order to achieve an accurate MI classification. The performance of the proposed model was verified using three publicly available datasets.Main results.The proposed model achieved an average classification accuracy of 79.63% and 94.53% for the BCIIV2a and HGD four-classification task, respectively, and 87.82% for the BCIIV2b two-classification task. In particular, by comparing the number of parameters, computation and classification accuracy with other lightweight models, it was confirmed that the proposed model achieved a better balance between the decoding performance and computational cost. Furthermore, the structural feasibility of the proposed model was confirmed by ablation experiments and feature visualization.Significance.The results indicated that the proposed CNN model presented high classification accuracy with less computing resources, and can be applied in the MI classification research.

On the feasibility of simple brain-computer interface systems for enabling children with severe physical disabilities to explore independent movement

Introduction

Children with severe physical disabilities are denied their fundamental right to move, restricting their development, independence, and participation in life. Brain-computer interfaces (BCIs) could enable children with complex physical needs to access power mobility (PM) devices, which could help them move safely and independently. BCIs have been studied for PM control for adults but remain unexamined in children. In this study, we explored the feasibility of BCI-enabled PM control for children with severe physical disabilities, assessing BCI performance, standard PM skills and tolerability of BCI.

Materials and methods

Patient-oriented pilot trial. Eight children with quadriplegic cerebral palsy attended two sessions where they used a simple, commercial-grade BCI system to activate a PM trainer device. Performance was assessed through controlled activation trials (holding the PM device still or activating it upon verbal and visual cueing), and basic PM skills (driving time, number of activations, stopping) were assessed through distance trials. Setup and calibration times, headset tolerability, workload, and patient/caregiver experience were also evaluated.

Results

All participants completed the study with favorable tolerability and no serious adverse events or technological challenges. Average control accuracy was 78.3 ± 12.1%, participants were more reliably able to activate (95.7 ± 11.3%) the device than hold still (62.1 ± 23.7%). Positive trends were observed between performance and prior BCI experience and age. Participants were able to drive the PM device continuously an average of 1.5 meters for 3.0 s. They were able to stop at a target 53.1 ± 23.3% of the time, with significant variability. Participants tolerated the headset well, experienced mild-to-moderate workload and setup/calibration times were found to be practical. Participants were proud of their performance and both participants and families were eager to participate in future power mobility sessions.

Discussion

BCI-enabled PM access appears feasible in disabled children based on evaluations of performance, tolerability, workload, and setup/calibration. Performance was comparable to existing pediatric BCI literature and surpasses established cut-off thresholds (70%) of “effective” BCI use. Participants exhibited PM skills that would categorize them as “emerging operational learners.” Continued exploration of BCI-enabled PM for children with severe physical disabilities is justified.

Optimising the classification of feature-based attention in frequency-tagged electroencephalography data

Brain-computer interfaces (BCIs) are a rapidly expanding field of study and require accurate and reliable real-time decoding of patterns of neural activity. These protocols often exploit selective attention, a neural mechanism that prioritises the sensory processing of task-relevant stimulus features (feature-based attention) or task-relevant spatial locations (spatial attention). Within the visual modality, attentional modulation of neural responses to different inputs is well indexed by steady-state visual evoked potentials (SSVEPs). These signals are reliably present in single-trial electroencephalography (EEG) data, are largely resilient to common EEG artifacts, and allow separation of neural responses to numerous concurrently presented visual stimuli. To date, efforts to use single-trial SSVEPs to classify visual attention for BCI control have largely focused on spatial attention rather than feature-based attention. Here, we present a dataset that allows for the development and benchmarking of algorithms to classify feature-based attention using single-trial EEG data. The dataset includes EEG and behavioural responses from 30 healthy human participants who performed a feature-based motion discrimination task on frequency tagged visual stimuli.

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

Biosignal control is an interaction modality that allows users to interact with electronic devices by decoding the biological signals emanating from the movements or thoughts of the user. This manner of interaction with devices can enhance the sense of agency for users and enable persons suffering from a paralyzing condition to interact with everyday devices that would otherwise be challenging for them to use. It can also improve control of prosthetic devices and exoskeletons by making the interaction feel more natural and intuitive. However, with the current state of the art, several issues still need to be addressed to reliably decode user intent from biosignals and provide an improved user experience over other interaction modalities. One solution is to leverage advances in Deep Learning (DL) methods to provide more reliable decoding at the expense of added computational complexity. This scoping review introduces the basic concepts of DL and assists readers in deploying DL methods to a real-time control system that should operate under real-world conditions. The scope of this review covers any electronic device, but with an emphasis on robotic devices, as this is the most active area of research in biosignal control. We review the literature pertaining to the implementation and evaluation of control systems that incorporate DL to identify the main gaps and issues in the field, and formulate suggestions on how to mitigate them. Additionally, we formulate guidelines on the best approach to designing, implementing and evaluating research prototypes that use DL in their biosignal control systems.

A brief review of motor imagery and bimanual coordination

Motor imagery is increasingly being used in clinical settings, such as in neurorehabilitation and brain computer interface (BCI). In stroke, patients lose upper limb function and must re-learn bimanual coordination skills necessary for the activities of daily living. Physiotherapists integrate motor imagery with physical rehabilitation to accelerate recovery. In BCIs, users are often asked to imagine a movement, often with sparse instructions. The EEG pattern that coincides with this cognitive task is captured, then used to execute an external command, such as operating a neuroprosthetic device. As such, BCIs are dependent on the efficient and reliable interpretation of motor imagery. While motor imagery improves patient outcome and informs BCI research, the cognitive and neurophysiological mechanisms which underlie it are not clear. Certain types of motor imagery techniques are more effective than others. For instance, focusing on kinesthetic cues and adopting a first-person perspective are more effective than focusing on visual cues and adopting a third-person perspective. As motor imagery becomes more dominant in neurorehabilitation and BCIs, it is important to elucidate what makes these techniques effective. The purpose of this review is to examine the research to date that focuses on both motor imagery and bimanual coordination. An assessment of current research on these two themes may serve as a useful platform for scientists and clinicians seeking to use motor imagery to help improve bimanual coordination, either through augmenting physical therapy or developing more effective BCIs.

Frontolimbic alpha activity tracks intentional rest BCI control improvement through mindfulness meditation

Brain-computer interfaces (BCIs) are capable of translating human intentions into signals controlling an external device to assist patients with severe neuromuscular disorders. Prior work has demonstrated that participants with mindfulness meditation experience evince improved BCI performance, but the underlying neural mechanisms remain unclear. Here, we conducted a large-scale longitudinal intervention study by training participants in mindfulness-based stress reduction (MBSR; a standardized mind-body awareness training intervention), and investigated whether and how short-term MBSR affected sensorimotor rhythm (SMR)-based BCI performance. We hypothesize that MBSR training improves BCI performance by reducing mind wandering and enhancing self-awareness during the intentional rest BCI control, which would mainly be reflected by modulations of default-mode network and limbic network activity. We found that MBSR training significantly improved BCI performance compared to controls and these behavioral enhancements were accompanied by increased frontolimbic alpha activity (9-15 Hz) and decreased alpha connectivity among limbic network, frontoparietal network, and default-mode network. Furthermore, the modulations of frontolimbic alpha activity were positively correlated with the duration of meditation experience and the extent of BCI performance improvement. Overall, these data suggest that mindfulness allows participant to reach a state where they can modulate frontolimbic alpha power and improve BCI performance for SMR-based BCI control.

A review of user training methods in brain computer interfaces based on mental tasks

Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.

Brain-Machine Interfaces as Commodities: Exchanging Mind for Matter

Brain-machine interfaces (BMIs), which enable a two-way flow of signals, information, and directions between human neurons and computerized machines, offer spectacular opportunities for therapeutic and consumer applications, but they also present unique dangers to the safety, privacy, psychological health, and spiritual well-being of their users. The sale of these devices as commodities for profit exacerbates such issues and may subject the user to an unequal exchange with corporations. Catholic healthcare professionals and bioethicists should be especially concerned about the implications for the essential dignity of the persons using the new BMIs.

Summary

The commercial sale of brain-machine interfaces (BMIs) generates and exacerbates problems for end-users' safety, psychological health, and spiritual well-being.

Calibration time reduction through local activities estimation in motor imagery-based brain-computer interfaces

OBJECTIVE

One of the main limitations for the practical use of brain-computer interfaces (BCI) is the calibration phase. Several methods have been suggested for the truncating of this undesirable time, including various variants of the popular CSP method. In this study, we cope with the problem, using local activities estimation (LAE).

APPROACH

LAE is a spatial filtering technique that uses the EEG data of all electrodes along with their position information to emphasize the local activities. After spatial filtering by LAE, a few electrodes are selected based on physiological information. Then the features are extracted from the signal using FFT and classified by the support vector machine. In this study, the LAE is compared with CSP, RCSP, FBCSP and FBRCSP in two different electrode configurations of 118 and 64-channel.

MAIN RESULTS

The LAE outperforms CSP-based methods in all experiments using the different number of training samples. The LAE method also obtains an average classification accuracy of 84% even with a calibration time of fewer than 2 min Significance: Unlike CSP-based methods, the LAE does not use the covariance matrix, and also uses a priori physiological information. Therefore, LAE can significantly reduce the calibration time while maintaining proper accuracy. It works well even with a few training samples.

Functional disconnection of associative cortical areas predicts performance during BCI training

Brain-computer interfaces (BCIs) have been largely developed to allow communication, control, and neurofeedback in human beings. Despite their great potential, BCIs perform inconsistently across individuals and the neural processes that enable humans to achieve good control remain poorly understood. To address this question, we performed simultaneous high-density electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings in a motor imagery-based BCI training involving a group of healthy subjects. After reconstructing the signals at the cortical level, we showed that the reinforcement of motor-related activity during the BCI skill acquisition is paralleled by a progressive disconnection of associative areas which were not directly targeted during the experiments. Notably, these network connectivity changes reflected growing automaticity associated with BCI performance and predicted future learning rate. Altogether, our findings provide new insights into the large-scale cortical organizational mechanisms underlying BCI learning, which have implications for the improvement of this technology in a broad range of real-life applications.

Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain-computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user's environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.