Brain-machine interface (BMI) in paralysis

Easily Attach/Detach Reattachable EEG Headset with Candle-like Microneedle Electrodes

To expand the applications of the electroencephalogram (EEG), long-term measurement, a short installation time, and little stress on the participants are needed. In this study, we designed, fabricated, and evaluated an EEG headset with three candle-like microneedle electrodes (CMEs). The user is able to detach and reattach the electrodes, enabling long-term measurement with little stress. The design of the CMEs was experimentally determined by considering the skin-to-electrode impedance and user comfort. An EEG was successfully measured from areas with a high hair density without any preparation. The installation time was shorter than 60 s and the electrodes could be detached and reattached. The headset was designed such that the discomfort caused by its ear pads was higher than that caused by the electrodes. In 1 h experiments, the participants did not feel pain and the detachment of the CMEs was found to improve the comfort level of the participants in most cases. A successful demonstration of the long-term measurement of EEGs while watching a whole movie verified that the developed EEG headset with CMEs is applicable for EEG measurement in a variety of applications.

Multibranch convolutional neural network with contrastive representation learning for decoding same limb motor imagery tasks

Introduction

Emerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks.

Methods

The present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification.

Results

The experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively.

Discussion

Despite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

Spelling interface using intracortical signals in a completely locked-in patient enabled via auditory neurofeedback training

Patients with amyotrophic lateral sclerosis (ALS) can lose all muscle-based routes of communication as motor neuron degeneration progresses, and ultimately, they may be left without any means of communication. While others have evaluated communication in people with remaining muscle control, to the best of our knowledge, it is not known whether neural-based communication remains possible in a completely locked-in state. Here, we implanted two 64 microelectrode arrays in the supplementary and primary motor cortex of a patient in a completely locked-in state with ALS. The patient modulated neural firing rates based on auditory feedback and he used this strategy to select letters one at a time to form words and phrases to communicate his needs and experiences. This case study provides evidence that brain-based volitional communication is possible even in a completely locked-in state.

Multisite Simultaneous Neural Recording of Motor Pathway in Free-Moving Rats

Neural interfaces typically focus on one or two sites in the motoneuron system simultaneously due to the limitation of the recording technique, which restricts the scope of observation and discovery of this system. Herein, we built a system with various electrodes capable of recording a large spectrum of electrophysiological signals from the cortex, spinal cord, peripheral nerves, and muscles of freely moving animals. The system integrates adjustable microarrays, floating microarrays, and microwires to a commercial connector and cuff electrode on a wireless transmitter. To illustrate the versatility of the system, we investigated its performance for the behavior of rodents during tethered treadmill walking, untethered wheel running, and open field exploration. The results indicate that the system is stable and applicable for multiple behavior conditions and can provide data to support previously inaccessible research of neural injury, rehabilitation, brain-inspired computing, and fundamental neuroscience.

Neurodegenerative disorders management: state-of-art and prospects of nano-biotechnology

Neurodegenerative disorders (NDs) are highly prevalent among the aging population. It affects primarily the central nervous system (CNS) but the effects are also observed in the peripheral nervous system. Neural degeneration is a progressive loss of structure and function of neurons, which may ultimately involve cell death. Such patients suffer from debilitating memory loss and altered motor coordination which bring up non-affordable and unavoidable socio-economic burdens. Due to the unavailability of specific therapeutics and diagnostics, the necessity to control or manage NDs raised the demand to investigate and develop efficient alternative approaches. Keeping trends and advancements in view, this report describes both state-of-the-art and challenges in nano-biotechnology-based approaches to manage NDs, toward personalized healthcare management. Sincere efforts are being made to customize nano-theragnostics to control: therapeutic cargo packaging, delivery to the brain, nanomedicine of higher efficacy, deep brain stimulation, implanted stimulation, and managing brain cell functioning. These advancements are useful to design future therapy based on the severity of the patient's neurodegenerative disease. However, we observe a lack of knowledge shared among scientists of a variety of expertise to explore this multi-disciplinary research field for NDs management. Consequently, this review will provide a guideline platform that will be useful in developing novel smart nano-therapies by considering the aspects and advantages of nano-biotechnology to manage NDs in a personalized manner. Nano-biotechnology-based approaches have been proposed as effective and affordable alternatives at the clinical level due to recent advancements in nanotechnology-assisted theragnostics, targeted delivery, higher efficacy, and minimal side effects.

Four methods of brain pattern analyses of fMRI signals associated with wrist extension versus wrist flexion studied for potential use in future motor learning BCI

OBJECTIVE

In stroke survivors, a treatment-resistant problem is inability to volitionally differentiate upper limb wrist extension versus flexion. When one intends to extend the wrist, the opposite occurs, wrist flexion, rendering the limb non-functional. Conventional therapeutic approaches have had limited success in achieving functional recovery of patients with chronic and severe upper extremity impairments. Functional magnetic resonance imaging (fMRI) neurofeedback is an emerging strategy that has shown potential for stroke rehabilitation. There is a lack of information regarding unique blood-oxygenation-level dependent (BOLD) cortical activations uniquely controlling execution of wrist extension versus uniquely controlling wrist flexion. Therefore, a first step in providing accurate neural feedback and training to the stroke survivor is to determine the feasibility of classifying (or differentiating) brain activity uniquely associated with wrist extension from that of wrist flexion, first in healthy adults.

APPROACH

We studied brain signal of 10 healthy adults, who performed wrist extension and wrist flexion during fMRI data acquisition. We selected four types of analyses to study the feasibility of differentiating brain signal driving wrist extension versus wrist flexion, as follows: 1) general linear model (GLM) analysis; 2) support vector machine (SVM) classification; 3) 'Winner Take All'; and 4) Relative Dominance.

RESULTS

With these four methods and our data, we found that few voxels were uniquely active during either wrist extension or wrist flexion. SVM resulted in only minimal classification accuracies. There was no significant difference in activation magnitude between wrist extension versus flexion; however, clusters of voxels showed extension signal > flexion signal and other clusters vice versa. Spatial patterns of activation differed among subjects.

SIGNIFICANCE

We encountered a number of obstacles to obtaining clear group results in healthy adults. These obstacles included the following: high variability across healthy adults in all measures studied; close proximity of uniquely active voxels to voxels that were common to both the extension and flexion movements; in general, higher magnitude of signal for the voxels common to both movements versus the magnitude of any given uniquely active voxel for one type of movement. Our results indicate that greater precision in imaging will be required to develop a truly effective method for differentiating wrist extension versus wrist flexion from fMRI data.

Denoising Autoencoder-Based Feature Extraction to Robust SSVEP-Based BCIs

For subjects with amyotrophic lateral sclerosis (ALS), the verbal and nonverbal communication is greatly impaired. Steady state visually evoked potential (SSVEP)-based brain computer interfaces (BCIs) is one of successful alternative augmentative communications to help subjects with ALS communicate with others or devices. For practical applications, the performance of SSVEP-based BCIs is severely reduced by the effects of noises. Therefore, developing robust SSVEP-based BCIs is very important to help subjects communicate with others or devices. In this study, a noise suppression-based feature extraction and deep neural network are proposed to develop a robust SSVEP-based BCI. To suppress the effects of noises, a denoising autoencoder is proposed to extract the denoising features. To obtain an acceptable recognition result for practical applications, the deep neural network is used to find the decision results of SSVEP-based BCIs. The experimental results showed that the proposed approaches can effectively suppress the effects of noises and the performance of SSVEP-based BCIs can be greatly improved. Besides, the deep neural network outperforms other approaches. Therefore, the proposed robust SSVEP-based BCI is very useful for practical applications.

Decoding different working memory states during an operation span task from prefrontal fNIRS signals

We propose an effective and practical decoding method of different mental states for potential applications for the design of brain-computer interfaces, prediction of cognitive behaviour, and investigation of cognitive mechanism. Functional near infrared spectroscopy (fNIRS) signals that interrogated the prefrontal and parietal cortices and were evaluated by generalized linear model were recorded when nineteen healthy adults performed the operation span (OSPAN) task. The oxygenated hemoglobin changes during OSPAN, response, and rest periods were classified with a support vector machine (SVM). The relevance vector regression algorithm was utilized for prediction of cognitive performance based on multidomain features of fNIRS signals from the OSPAN task. We acquired decent classification accuracies for OSPAN vs. response (above 91.2%) and for OSPAN vs. rest (above 94.7%). Eight of the ten cognitive testing scores could be predicted from the combination of OSPAN and response features, which indicated the brain hemodynamic responses contain meaningful information suitable for predicting cognitive performance.

EEG power spectral density in locked-in and completely locked-in state patients: a longitudinal study

Persons with their eye closed and without any means of communication is said to be in a completely locked-in state (CLIS) while when they could still open their eyes actively or passively and have some means of communication are said to be in locked-in state (LIS). Two patients in CLIS without any means of communication, and one patient in the transition from LIS to CLIS with means of communication, who have Amyotrophic Lateral Sclerosis were followed at a regular interval for more than 1 year. During each visit, resting-state EEG was recorded before the brain-computer interface (BCI) based communication sessions. The resting-state EEG of the patients was analyzed to elucidate the evolution of their EEG spectrum over time with the disease's progression to provide future BCI-research with the relevant information to classify changes in EEG evolution. Comparison of power spectral density (PSD) of these patients revealed a significant difference in the PSD's of patients in CLIS without any means of communication and the patient in the transition from LIS to CLIS with means of communication. The EEG of patients without any means of communication is devoid of alpha, beta, and higher frequencies than the patient in transition who still had means of communication. The results show that the change in the EEG frequency spectrum may serve as an indicator of the communication ability of such patients.

Combined real-time fMRI and real time fNIRS brain computer interface (BCI): Training of volitional wrist extension after stroke, a case series pilot study

Objective

Pilot testing of real time functional magnetic resonance imaging (rt-fMRI) and real time functional near infrared spectroscopy (rt-fNIRS) as brain computer interface (BCI) neural feedback systems combined with motor learning for motor recovery in chronic severely impaired stroke survivors.

Approach

We enrolled a four-case series and administered three sequential rt-fMRI and ten rt-fNIRS neural feedback sessions interleaved with motor learning sessions. Measures were: Arm Motor Assessment Tool, functional domain (AMAT-F; 13 complex functional tasks), Fugl-Meyer arm coordination scale (FM); active wrist extension range of motion (ROM); volume of activation (fMRI); and fNIRS HbO concentration. Performance during neural feedback was assessed, in part, using percent successful brain modulations during rt-fNIRS.

Main results

Pre-/post-treatment mean clinically significant improvement in AMAT-F (.49 ± 0.22) and FM (10.0 ± 3.3); active wrist ROM improvement ranged from 20° to 50°. Baseline to follow-up change in brain signal was as follows: fMRI volume of activation was reduced in almost all ROIs for three subjects, and for one subject there was an increase or no change; fNIRS HbO was within normal range, except for one subject who increased beyond normal at post-treatment. During rt-fNIRS neural feedback training, there was successful brain signal modulation (42%–78%).

Significance

Severely impaired stroke survivors successfully engaged in spatially focused BCI systems, rt-fMRI and rt-fNIRS, to clinically significantly improve motor function. At the least, equivalency in motor recovery was demonstrated with prior long-duration motor learning studies (without neural feedback), indicating that no loss of motor improvement resulted from substituting neural feedback sessions for motor learning sessions. Given that the current neural feedback protocol did not prevent the motor improvements observed in other long duration studies, even in the presence of fewer sessions of motor learning in the current work, the results support further study of neural feedback and its potential for recovery of motor function in stroke survivors. In future work, expanding the sophistication of either or both rt-fMRI and rt-fNIRS could hold the potential for further reducing the number of hours of training needed and/or the degree of recovery.

ClinicalTrials.gov ID:NCT02856035.

Neurophysiological aspects of the completely locked-in syndrome in patients with advanced amyotrophic lateral sclerosis

OBJECTIVE

Amyotrophic lateral sclerosis (ALS) patients in completely locked-in syndrome (CLIS) are incapable of expressing themselves, and their state of consciousness and awareness is difficult to evaluate. Due to the complete paralysis included paralysis of eye muscles, any assessment of the perceptual and psychophysiological state can only be implemented in passive experimental paradigms with neurophysiological recordings.

METHODS

Four patients in CLIS were investigated in several experiments including resting state, visual stimulation (eyes open vs eyes closed), auditory stimulation (modified local-global paradigm), somatosensory stimulation (electrical stimulation of the median nerve), and during sleep.

RESULTS

All patients showed altered neurophysiological metrics, but a unique and common pattern could not be found between patients. However, slowing of the electroencephalography (EEG) and attenuation or absence of alpha wave activity was common in all patients. In two of the four patients, a slow dominant frequency emerged at 4 Hz with synchronized EEG at all channels. In the other two patients slowing of EEG appears less synchronized. EEGs between eyes open and eyes closed were significantly different in all patients. The dominant slow frequency during the day changes during slow-wave sleep (supposedly sleep stage 3) to even slower frequencies below 2 Hz. Somatosensory evoked potentials (SEPs) were absent or significantly altered in comparison to healthy subjects, similarly for auditory evoked potentials (AEPs).

CONCLUSIONS

The heterogeneity of the results underscores the fact that no single neurophysiological index is available to assess psychophysiological states in unresponsive ALS patients in CLIS. This caveat may also be valid for the assessment of cognitive processes; a functioning BCI can be the solution.

SIGNIFICANCE

Most of the studies of the neurophysiology of ALS patients focused on the early stage of the disease, and there are very few studies on the late stage when patients are completely paralyzed with no means of communication (i.e., CLIS). This study provides quantitative metrics of different neurophysiological aspects of these patients.

Force Decoding of Caudal Forelimb Area and Rostral Forelimb Area in Chronic Stroke Rats

Brain machine interfaces (BMIs) used for movement restoration primarily rely on studies of motor decoding. It has been proved that local field potentials (LFPs) from primary motor cortex and premotor cortex of normal rodents could be used for decoding motor signals. However, few studies have explored the decoding performance of these brain areas under motor cortex damage. In this work, we focus on force decoding performance of LFPs spectrum from both ipsilesional caudal forelimb area (CFA) and rostral forelimb area (RFA) of rodents with ischemia over CFA. After three months of ischemia induced by photothrombosis over CFA, the power of high-frequency bands (>120 Hz) from both CFA and RFA can decode force signals by Kalman filters. The fair performance of CFA indicates motor reorganization over penumbra. Further exploration of RFA decoding ability proves that at least four electrodes of RFA should be used on decoding and electrodes far from CFA of stroke rats could achieve almost as good results as those close to CFA of normal rats, which indicates the motor remapping. Experimental results show the long-term stability of PM LFPs decoding performance of stroke rats as the trained Kalman model could be used to accurately decode force some days later which provides a possibility for online decoding system. In conclusion, our work shows that even under CFA ischemia, high-frequency power of LFPs from RFA is still able to accurately decode force signals and has long stability, which provides the possibility of BMIs for motor function reconstruction of chronic stroke patients.

Visual-Electrotactile Stimulation Feedback to Improve Immersive Brain-Computer Interface Based on Hand Motor Imagery

In the aging society, the number of people suffering from vascular disorders is rapidly increasing and has become a social problem. The death rate due to stroke, which is the second leading cause of global mortality, has increased by 40% in the last two decades. Stroke can also cause paralysis. Of late, brain-computer interfaces (BCIs) have been garnering attention in the rehabilitation field as assistive technology. A BCI for the motor rehabilitation of patients with paralysis promotes neural plasticity, when subjects perform motor imagery (MI). Feedback, such as visual and proprioceptive, influences brain rhythm modulation to contribute to MI learning and motor function restoration. Also, virtual reality (VR) can provide powerful graphical options to enhance feedback visualization. This work aimed to improve immersive VR-BCI based on hand MI, using visual-electrotactile stimulation feedback instead of visual feedback. The MI tasks include grasping, flexion/extension, and their random combination. Moreover, the subjects answered a system perception questionnaire after the experiments. The proposed system was evaluated with twenty able-bodied subjects. Visual-electrotactile feedback improved the mean classification accuracy for the grasping (93.00%  $\pm$  3.50%) and flexion/extension (95.00%  $\pm$  5.27%) MI tasks. Additionally, the subjects achieved an acceptable mean classification accuracy (maximum of 86.5%  $\pm$  5.80%) for the random MI task, which required more concentration. The proprioceptive feedback maintained lower mean power spectral density in all channels and higher attention levels than those of visual feedback during the test trials for the grasping and flexion/extension MI tasks. Also, this feedback generated greater relative power in the $\mu$ -band for the premotor cortex, which indicated better MI preparation. Thus, electrotactile stimulation along with visual feedback enhanced the immersive VR-BCI classification accuracy by 5.5% and 4.5% for the grasping and flexion/extension MI tasks, respectively, retained the subject’s attention, and eased MI better than visual feedback alone.

Development of a brain-computer interface for patients in the critical care setting

Objective

Behaviorally unresponsive patients in intensive care units (ICU) are unable to consistently and effectively communicate their most fundamental physical needs. Brain-Computer Interface (BCI) technology has been established in the clinical context, but faces challenges in the critical care environment. Contrary to cue-based BCIs, which allow activation only during pre-determined periods of time, self-paced BCI systems empower patients to interact with others at any time. The study aims to develop a self-paced BCI for patients in the intensive care unit.

Methods

BCI experiments were conducted in 18 ICU patients and 5 healthy volunteers. The proposed self-paced BCI system analyzes EEG activity from patients while these are asked to control a beeping tone by performing a motor task (i.e., opening and closing a hand). Signal decoding is performed in real time and auditory feedback given via headphones. Performance of the BCI system was judged based on correlation between the optimal and the observed performance.

Results

All 5 healthy volunteers were able to successfully perform the BCI task, compared to chance alone (p<0.001). 5 of 14 (36%) conscious ICU patients were able to perform the BCI task. One of these 5 patients was quadriplegic and controlled the BCI system without any hand movements. None of the 4 unconscious patients were able to perform the BCI task.

Conclusions

More than one third of conscious ICU patients and all healthy volunteers were able to gain control over the self-paced BCI system. The initial 4 unconscious patients were not. Future studies will focus on studying the ability of behaviorally unresponsive patients with cognitive motor dissociation to control the self-paced BCI system.

A dataset of EEG and EOG from an auditory EOG-based communication system for patients in locked-in state

The dataset presented here contains recordings of electroencephalogram (EEG) and electrooculogram (EOG) from four advanced locked-in state (LIS) patients suffering from ALS (amyotrophic lateral sclerosis). These patients could no longer use commercial eye-trackers, but they could still move their eyes and used the remnant oculomotor activity to select letters to form words and sentences using a novel auditory communication system. Data were recorded from four patients during a variable range of visits (from 2 to 10), each visit comprised of 3.22 ± 1.21 days and consisted of 5.57 ± 2.61 sessions recorded per day. The patients performed a succession of different sessions, namely, Training, Feedback, Copy spelling, and Free spelling. The dataset provides an insight into the progression of ALS and presents a valuable opportunity to design and improve assistive and alternative communication technologies and brain-computer interfaces. It might also help redefine the course of progression in ALS, thereby improving clinical judgement and treatment.

Neural implant for the treatment of multiple sclerosis

The methods used to treat various neurological diseases are evolving. The facilities provided by the technology have led to creation of new treatment opportunities. Neuromodulation is one of these important methods. By definition, the neuromodulation is a change in neural activity which occurs by stimulating a specific area of nervous system. The mentioned stimulation can be electrical, magnetic, or chemical. This method is used in various diseases, such as stroke, Parkinson's, Alzheimer's, and amyotrophic lateral sclerosis (ALS). Multiple sclerosis (MS) is no exception in this regard and methods including the neurofeedback and transcranial magnetic stimulation (TMS) are used to treat various complications of the MS. One aspect of neuromodulation is the use of neural implant, which is applied nowadays, especially in the Parkinson's disease, and the use of microchips and prostheses to treat various symptoms in different neurological diseases has received significant attention. Although neural implant has been exploited to improve the symptoms of MS, they appear to have much greater potential to improve the condition of patients with MS. It seems that more attention to the symptoms of MS, on the one hand, and a new approach to the pathogenesis of this disease and considering it as a connectomopathy, on the other hand, can provide new opportunities for application of this method in the treatment of MS.

Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction

Brain-computer interfaces (BCIs) have long been seen as control interfaces that translate changes in brain activity, produced either by means of a volitional modulation or in response to an external stimulation. However, recent trends in the BCI and neurofeedback research highlight passive monitoring of a user's brain activity in order to estimate cognitive load, attention level, perceived errors and emotions. Extraction of such higher order information from brain signals is seen as a gateway for facilitation of interaction between humans and intelligent systems. Particularly in the field of robotics, passive BCIs provide a promising channel for prediction of user's cognitive and affective state for development of a user-adaptive interaction. In this paper, we first illustrate the state of the art in passive BCI technology and then provide examples of BCI employment in human-robot interaction (HRI). We finally discuss the prospects and challenges in integration of passive BCIs in socially demanding HRI settings. This work intends to inform HRI community of the opportunities offered by passive BCI systems for enhancement of human-robot interaction while recognizing potential pitfalls.

DIY hybrid SSVEP-P300 LED stimuli for BCI platform using EMOTIV EEG headset

A fully customisable chip-on board (COB) LED design to evoke two brain responses simultaneously (steady state visual evoked potential (SSVEP) and transient evoked potential, P300) is discussed in this paper. Considering different possible modalities in brain-computer interfacing (BCI), SSVEP is widely accepted as it requires a lesser number of electroencephalogram (EEG) electrodes and minimal training time. The aim of this work was to produce a hybrid BCI hardware platform to evoke SSVEP and P300 precisely with reduced fatigue and improved classification performance. The system comprises of four independent radial green visual stimuli controlled individually by a 32-bit microcontroller platform to evoke SSVEP and four red LEDs flashing at random intervals to generate P300 events. The system can also record the P300 event timestamps that can be used in classification, to improve the accuracy and reliability. The hybrid stimulus was tested for real-time classification accuracy by controlling a LEGO robot to move in four directions.

Identification of Motor and Mental Imagery EEG in Two and Multiclass Subject-Dependent Tasks Using Successive Decomposition Index

The development of fast and robust brain-computer interface (BCI) systems requires non-complex and efficient computational tools. The modern procedures adopted for this purpose are complex which limits their use in practical applications. In this study, for the first time, and to the best of our knowledge, a successive decomposition index (SDI)-based feature extraction approach is utilized for the classification of motor and mental imagery electroencephalography (EEG) tasks. First of all, the public datasets IVa, IVb, and V from BCI competition III were denoised using multiscale principal analysis (MSPCA), and then a SDI feature was calculated corresponding to each trial of the data. Finally, six benchmark machine learning and neural network classifiers were used to evaluate the performance of the proposed method. All the experiments were performed for motor and mental imagery datasets in binary and multiclass applications using a 10-fold cross-validation method. Furthermore, computerized automatic detection of motor and mental imagery using SDI (CADMMI-SDI) is developed to describe the proposed approach practically. The experimental results suggest that the highest classification accuracy of 97.46% (Dataset IVa), 99.52% (Dataset IVb), and 99.33% (Dataset V) was obtained using feedforward neural network classifier. Moreover, a series of experiments, namely, statistical analysis, channels variation, classifier parameters variation, processed and unprocessed data, and computational complexity, were performed and it was concluded that SDI is robust for noise, and a non-complex and efficient biomarker for the development of fast and accurate motor and mental imagery BCI systems.

Auditory Electrooculogram-based Communication System for ALS Patients in Transition from Locked-in to Complete Locked-in State

Patients in the transition from locked-in (i.e., a state of almost complete paralysis with voluntary eye movement control, eye blinks or twitches of face muscles, and preserved consciousness) to complete locked-in state (i.e., total paralysis including paralysis of eye-muscles and loss of gaze-fixation, combined with preserved consciousness) are left without any means of communication. An auditory communication system based on electrooculogram (EOG) was developed to enable such patients to communicate. Four amyotrophic lateral sclerosis patients in transition from locked-in state to completely locked-in state, with ALSFRS-R score of 0, unable to use eye trackers for communication, learned to use an auditory EOG-based communication system. The patients, with eye-movement amplitude between the range of ±200μV and ±40μV, were able to form complete sentences and communicate independently and freely, selecting letters from an auditory speller system. A follow-up of one year with one patient shows the feasibility of the proposed system in long-term use and the correlation between speller performance and eye-movement decay. The results of the auditory speller system have the potential to provide a means of communication to patient populations without gaze fixation ability and with low eye-movement amplitude range.

Open Software/Hardware Platform for Human-Computer Interface Based on Electrooculography (EOG) Signal Classification

Electrooculography (EOG) signals have been widely used in Human-Computer Interfaces (HCI). The HCI systems proposed in the literature make use of self-designed or closed environments, which restrict the number of potential users and applications. Here, we present a system for classifying four directions of eye movements employing EOG signals. The system is based on open source ecosystems, the Raspberry Pi single-board computer, the OpenBCI biosignal acquisition device, and an open-source python library. The designed system provides a cheap, compact, and easy to carry system that can be replicated or modified. We used Maximum, Minimum, and Median trial values as features to create a Support Vector Machine (SVM) classifier. A mean of 90% accuracy was obtained from 7 out of 10 subjects for online classification of Up, Down, Left, and Right movements. This classification system can be used as an input for an HCI, i.e., for assisted communication in paralyzed people.

Neurorestoration: Advances in human brain–computer interface using microelectrode arrays

Neural damage has been a great challenge to the medical field for a very long time. The emergence of brain–computer interfaces (BCIs) offered a new possibility to enhance the activity of daily living and provide a new formation of entertainment for those with disabilities. Intracortical BCIs, which require the implantation of microelectrodes, can receive neuronal signals with a high spatial and temporal resolution from the individual’s cortex. When BCI decoded cortical signals and mapped them to external devices, it displayed the ability not only to replace part of the human motor function but also to help individuals restore certain neurological functions. In this review, we focus on human intracortical BCI research using microelectrode arrays and summarize the main directions and the latest results in this field. In general, we found that intracortical BCI research based on motor neuroprosthetics and functional electrical stimulation have already achieved some simple functional replacement and treatment of motor function. Pioneering work in the posterior parietal cortex has given us a glimpse of the potential that intracortical BCIs have to control external devices and receive various sensory information.

Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction

Brain-computer interfaces (BCIs) have long been seen as control interfaces that translate changes in brain activity, produced either by means of a volitional modulation or in response to an external stimulation. However, recent trends in the BCI and neurofeedback research highlight passive monitoring of a user's brain activity in order to estimate cognitive load, attention level, perceived errors and emotions. Extraction of such higher order information from brain signals is seen as a gateway for facilitation of interaction between humans and intelligent systems. Particularly in the field of robotics, passive BCIs provide a promising channel for prediction of user's cognitive and affective state for development of a user-adaptive interaction. In this paper, we first illustrate the state of the art in passive BCI technology and then provide examples of BCI employment in human-robot interaction (HRI). We finally discuss the prospects and challenges in integration of passive BCIs in socially demanding HRI settings. This work intends to inform HRI community of the opportunities offered by passive BCI systems for enhancement of human-robot interaction while recognizing potential pitfalls.

Applications of brain-computer interfaces to the control of robotic and prosthetic arms

Brain-computer interfaces (BCIs) have the potential to improve the quality of life of individuals with severe motor disabilities. BCIs capture the user's brain activity and translate it into commands for the control of an effector, such as a computer cursor, robotic limb, or functional electrical stimulation device. Full dexterous manipulation of robotic and prosthetic arms via a BCI system has been a challenge because of the inherent need to decode high dimensional and preferably real-time control commands from the user's neural activity. Nevertheless, such functionality is fundamental if BCI-controlled robotic or prosthetic limbs are to be used for daily activities. In this chapter, we review how this challenge has been addressed by BCI researchers and how new solutions may improve the BCI user experience with robotic effectors.

A Newcomer's Guide to Functional Near Infrared Spectroscopy Experiments

This review presents a practical primer for functional near-infrared spectroscopy (fNIRS) with respect to technology, experimentation, and analysis software. Its purpose is to jump-start interested practitioners considering utilizing a non-invasive, versatile, nevertheless challenging window into the brain using optical methods. We briefly recapitulate relevant anatomical and optical foundations and give a short historical overview. We describe competing types of illumination (trans-illumination, reflectance, and differential reflectance) and data collection methods (continuous wave, time domain and frequency domain). Basic components (light sources, detection, and recording components) of fNIRS systems are presented. Advantages and limitations of fNIRS techniques are offered, followed by a list of very practical recommendations for its use. A variety of experimental and clinical studies with fNIRS are sampled, shedding light on many brain-related ailments. Finally, we describe and discuss a number of freely available analysis and presentation packages suited for data analysis. In conclusion, we recommend fNIRS due to its ever-growing body of clinical applications, state-of-the-art neuroimaging technique and manageable hardware requirements. It can be safely concluded that fNIRS adds a new arrow to the quiver of neuro-medical examinations due to both its great versatility and limited costs.

Semantic and BCI-performance in completely paralyzed patients: Possibility of language attrition in completely locked in syndrome

Patients with completely locked-in syndrome (CLIS) are incapable of any voluntary muscle movement and do not have any means of communication. Recently functional near infrared spectroscopy (fNIRS) based brain computer interface (BCI) has been successfully used to enable communication with these patients. The developed fNIRS-BCI system relies on the intactness of language comprehension in these patients in all dimensions of language. Interwoven language and motor cortex in brain, and lack of muscular activity in long run, can cause language attrition due to complete immobility in CLIS patients. In this study we have investigated effects of semantic content of sentences presented to a CLIS patient on the performance of the BCI system during a YES/NO paradigm. Comparison of communication success rate in BCI classification between different semantic categories indicate that semantic content of sentences presented to a CLIS patient can affect the BCI performance. Affected concepts are mostly associated with executive words. These findings can be beneficial towards development of more reliable communication device for patients in CLIS. In addition, these results may assist in elucidating the cognitive changes in completely paralyzed patients with the passage of time since the onset of total immovability.

Neurofeedback Training Enables Voluntary Alteration of β-Band Power in the Subthalamic Nucleus of Individuals with Parkinson's Disease

Highlighted Research Paper:Real-Time Neurofeedback to Modulate β-Band Power in the Subthalamic Nucleus in Parkinson’s Disease Patients, by Ryohei Fukuma, Takufumi Yanagisawa, Masataka Tanaka, Fumiaki Yoshida, Koichi Hosomi, Satoru Oshino, Naoki Tani and Haruhiko Kishima

Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain–computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user’s environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.

Uncovering Consciousness in Unresponsive ICU Patients: Technical, Medical and Ethical Considerations

This article is one of ten reviews selected from the Annual Update in Intensive Care and Emergency Medicine 2019. Other selected articles can be found online at https://www.biomedcentral.com/collections/annualupdate2019. Further information about the Annual Update in Intensive Care and Emergency Medicine is available from http://www.springer.com/series/8901.

Classification of Movement Intention Using Independent Components of Premovement EEG

Many previous studies on brain-machine interfaces (BMIs) have focused on electroencephalography (EEG) signals elicited during motor-command execution to generate device commands. However, exploiting pre-execution brain activity related to movement intention could improve the practical applicability of BMIs. Therefore, in this study we investigated whether EEG signals occurring before movement execution could be used to classify movement intention. Six subjects performed reaching tasks that required them to move a cursor to one of four targets distributed horizontally and vertically from the center. Using independent components of EEG acquired during a premovement phase, two-class classifications were performed for left vs. right trials and top vs. bottom trials using a support vector machine. Instructions were presented visually (test) and aurally (condition). In the test condition, accuracy for a single window was about 75%, and it increased to 85% in classification using two windows. In the control condition, accuracy for a single window was about 73%, and it increased to 80% in classification using two windows. Classification results showed that a combination of two windows from different time intervals during the premovement phase improved classification performance in the both conditions compared to a single window classification. By categorizing the independent components according to spatial pattern, we found that information depending on the modality can improve classification performance. We confirmed that EEG signals occurring during movement preparation can be used to control a BMI.

Classification of motor imagery and execution signals with population-level feature sets: implications for probe design in fNIRS based BCI

OBJECTIVE

The aim of this study was to introduce a novel methodology for classification of brain hemodynamic responses collected via functional near infrared spectroscopy (fNIRS) during rest, motor imagery (MI) and motor execution (ME) tasks which involves generating population-level training sets.

APPROACH

A 48-channel fNIRS system was utilized to obtain hemodynamic signals from the frontal (FC), primary motor (PMC) and somatosensory cortex (SMC) of ten subjects during an experimental paradigm consisting of MI and ME of various right hand movements. Classification accuracies of random forest (RF), support vector machines (SVM), and artificial neural networks (ANN) were computed at the single subject level by training each classifier with subject specific features, and at the group level by training with features from all subjects for ME versus Rest, MI versus Rest and MI versus ME conditions. The performances were also computed for channel data restricted to FC, PMC and SMC regions separately to determine optimal probe location.

MAIN RESULTS

RF, SVM and ANN had comparably high classification accuracies for ME versus Rest (%94, %96 and %98 respectively) and for MI versus Rest (%95, %95 and %98 respectively) when fed with group level feature sets. The accuracy performance of each algorithm in localized brain regions were comparable (>%93) to the accuracy performance obtained with whole brain channels (>%94) for both ME versus Rest and MI versus Rest conditions.

SIGNIFICANCE

By demonstrating the feasibility of generating a population level training set with a high classification performance for three different classification algorithms, the findings pave the path for removing the necessity to acquire subject specific training data and hold promise for a novel, real-time fNIRS based BCI system design which will be most effective for application to disease populations for whom obtaining data to train a classification algorithm is not possible.

Real-Time Neurofeedback to Modulate β-Band Power in the Subthalamic Nucleus in Parkinson's Disease Patients

The β-band oscillation in the subthalamic nucleus (STN) is a therapeutic target for Parkinson’s disease. Previous studies demonstrated that l-DOPA decreases the β-band (13–30 Hz) oscillations with improvement of motor symptoms. However, it has not been elucidated whether patients with Parkinson’s disease are able to control the β-band oscillation voluntarily. Here, we hypothesized that neurofeedback training to control the β-band power in the STN induces plastic changes in the STN of individuals with Parkinson’s disease. We recorded the signals from STN deep-brain stimulation electrodes during operations to replace implantable pulse generators in eight human patients (3 male) with bilateral electrodes. Four patients were induced to decrease the β-band power during the feedback training (down-training condition), whereas the other patients were induced to increase (up-training condition). All patients were blinded to their assigned condition. Adjacent contacts that showed the highest β-band power were selected for the feedback. During the 10 min training, patients were shown a circle whose diameter was controlled by the β-band power of the selected contacts. Powers in the β-band during 5 min resting sessions recorded before and after the feedback were compared. In the down-training condition, the β-band power of the selected contacts decreased significantly after feedback in all four patients (p < 0.05). In contrast, the β-band power significantly increased after feedback in two of four patients in the up-training condition. Overall, the patients could voluntarily control the β-band power in STN in the instructed direction (p < 0.05) through neurofeedback.

An adaptive CSP filter to investigate user independence in a 3-class MI-BCI paradigm

This paper describes the implementation of a Brain Computer Interface (BCI) scheme using a common spatial patterns (CSP) filter in combination with a Recursive Least Squares (RLS) approach to iteratively update the coefficients of the CSP filter. The proposed adaptive CSP (ACSP) algorithm is made more robust by introducing regularization using Diagonal Loading (DL), and thus will be able to significantly reduce the length of training sessions when introducing new patients to the BCI system. The system is tested on a 4-class multi-limb motor imagery (MI) data set from the BCI competition IV (2a), and a more complex single limb 3-class MI dataset recorded in-house. The latter dataset is produced to mimic an upper limb rehabilitation session, e.g., after stroke. The findings indicate that when extensive calibration data is available, the ACSP performs comparably to the CSP (kappa value of 0.523 and 0.502, respectively, for the 4-class problem); for reduced calibration sessions, the ACSP significantly improved the performance of the system (up to 4-fold). The proposed paradigm proved feasible and the ACSP algorithm seems to enable a user or semi user independent scenario, where the need for long system calibration sessions without feedback is eliminated.

HD-EEG Based Classification of Motor-Imagery Related Activity in Patients With Spinal Cord Injury

Brain computer interfaces (BCIs) are thought to revolutionize rehabilitation after SCI, e.g., by controlling neuroprostheses, exoskeletons, functional electrical stimulation, or a combination of these components. However, most BCI research was performed in healthy volunteers and it is unknown whether these results can be translated to patients with spinal cord injury because of neuroplasticity. We sought to examine whether high-density EEG (HD-EEG) could improve the performance of motor-imagery classification in patients with SCI. We recorded HD-EEG with 256 channels in 22 healthy controls and 7 patients with 14 recordings (4 patients had more than one recording) in an event related design. Participants were instructed acoustically to either imagine, execute, or observe foot and hand movements, or to rest. We calculated Fast Fourier Transform (FFT) and full frequency directed transfer function (ffDTF) for each condition and classified conditions pairwise with support vector machines when using only 2 channels over the sensorimotor area, full 10-20 montage, high-density montage of the sensorimotor cortex, and full HD-montage. Classification accuracies were comparable between patients and controls, with an advantage for controls for classifications that involved the foot movement condition. Full montages led to better results for both groups (p < 0.001), and classification accuracies were higher for FFT than for ffDTF (p < 0.001), for which the feature vector might be too long. However, full-montage 10–20 montage was comparable to high-density configurations. Motor-imagery driven control of neuroprostheses or BCI systems may perform as well in patients as in healthy volunteers with adequate technical configuration. We suggest the use of a whole-head montage and analysis of a broad frequency range.

A 20-Questions-Based Binary Spelling Interface for Communication Systems

Brain computer interfaces (BCIs) enables people with motor impairments to communicate using their brain signals by selecting letters and words from a screen. However, these spellers do not work for people in a complete locked-in state (CLIS). For these patients, a near infrared spectroscopy-based BCI has been developed, allowing them to reply to “yes”/”no” questions with a classification accuracy of 70%. Because of the non-optimal accuracy, a usual character-based speller for selecting letters or words cannot be used. In this paper, a novel spelling interface based on the popular 20-questions-game has been presented, which will allow patients to communicate using only “yes”/”no” answers, even in the presence of poor classification accuracy. The communication system is based on an artificial neural network (ANN) that estimates a statement thought by the patient asking less than 20 questions. The ANN has been tested in a web-based version with healthy participants and in offline simulations. Both results indicate that the proposed system can estimate a patient’s imagined sentence with an accuracy that varies from 40%, in the case of a “yes”/”no” classification accuracy of 70%, and up to 100% in the best case. These results show that the proposed spelling interface could allow patients in CLIS to express their own thoughts, instead of only answer to “yes”/”no” questions.

Progress in Neuroengineering for brain repair: New challenges and open issues

Background

In recent years, biomedical devices have proven to be able to target also different neurological disorders. Given the rapid ageing of the population and the increase of invalidating diseases affecting the central nervous system, there is a growing demand for biomedical devices of immediate clinical use. However, to reach useful therapeutic results, these tools need a multidisciplinary approach and a continuous dialogue between neuroscience and engineering, a field that is named neuroengineering. This is because it is fundamental to understand how to read and perturb the neural code in order to produce a significant clinical outcome.

Results

In this review, we first highlight the importance of developing novel neurotechnological devices for brain repair and the major challenges expected in the next years. We describe the different types of brain repair strategies being developed in basic and clinical research and provide a brief overview of recent advances in artificial intelligence that have the potential to improve the devices themselves. We conclude by providing our perspective on their implementation to humans and the ethical issues that can arise.

Conclusions

Neuroengineering approaches promise to be at the core of future developments for clinical applications in brain repair, where the boundary between biology and artificial intelligence will become increasingly less pronounced.

Toward an Attention-Based Diagnostic Tool for Patients With Locked-in Syndrome

Electroencephalography (EEG) has been proposed as a supplemental tool for reducing clinical misdiagnosis in severely brain-injured populations helping to distinguish conscious from unconscious patients. We studied the use of spectral entropy as a measure of focal attention in order to develop a motor-independent, portable, and objective diagnostic tool for patients with locked-in syndrome (LIS), answering the issues of accuracy and training requirement. Data from 20 healthy volunteers, 6 LIS patients, and 10 patients with a vegetative state/unresponsive wakefulness syndrome (VS/UWS) were included. Spectral entropy was computed during a gaze-independent 2-class (attention vs rest) paradigm, and compared with EEG rhythms (delta, theta, alpha, and beta) classification. Spectral entropy classification during the attention-rest paradigm showed 93% and 91% accuracy in healthy volunteers and LIS patients respectively. VS/UWS patients were at chance level. EEG rhythms classification reached a lower accuracy than spectral entropy. Resting-state EEG spectral entropy could not distinguish individual VS/UWS patients from LIS patients. The present study provides evidence that an EEG-based measure of attention could detect command-following in patients with severe motor disabilities. The entropy system could detect a response to command in all healthy subjects and LIS patients, while none of the VS/UWS patients showed a response to command using this system.

Portable Brain-Computer Interface for the Intensive Care Unit Patient Communication Using Subject-Dependent SSVEP Identification

A major predicament for Intensive Care Unit (ICU) patients is inconsistent and ineffective communication means. Patients rated most communication sessions as difficult and unsuccessful. This, in turn, can cause distress, unrecognized pain, anxiety, and fear. As such, we designed a portable BCI system for ICU communications (BCI4ICU) optimized to operate effectively in an ICU environment. The system utilizes a wearable EEG cap coupled with an Android app designed on a mobile device that serves as visual stimuli and data processing module. Furthermore, to overcome the challenges that BCI systems face today in real-world scenarios, we propose a novel subject-specific Gaussian Mixture Model- (GMM-) based training and adaptation algorithm. First, we incorporate subject-specific information in the training phase of the SSVEP identification model using GMM-based training and adaptation. We evaluate subject-specific models against other subjects. Subsequently, from the GMM discriminative scores, we generate the transformed vectors, which are passed to our predictive model. Finally, the adapted mixture mean scores of the subject-specific GMMs are utilized to generate the high-dimensional supervectors. Our experimental results demonstrate that the proposed system achieved 98.7% average identification accuracy, which is promising in order to provide effective and consistent communication for patients in the intensive care.

Convolutional neural network for high-accuracy functional near-infrared spectroscopy in a brain-computer interface: three-class classification of rest, right-, and left-hand motor execution

The aim of this work is to develop an effective brain-computer interface (BCI) method based on functional near-infrared spectroscopy (fNIRS). In order to improve the performance of the BCI system in terms of accuracy, the ability to discriminate features from input signals and proper classification are desired. Previous studies have mainly extracted features from the signal manually, but proper features need to be selected carefully. To avoid performance degradation caused by manual feature selection, we applied convolutional neural networks (CNNs) as the automatic feature extractor and classifier for fNIRS-based BCI. In this study, the hemodynamic responses evoked by performing rest, right-, and left-hand motor execution tasks were measured on eight healthy subjects to compare performances. Our CNN-based method provided improvements in classification accuracy over conventional methods employing the most commonly used features of mean, peak, slope, variance, kurtosis, and skewness, classified by support vector machine (SVM) and artificial neural network (ANN). Specifically, up to 6.49% and 3.33% improvement in classification accuracy was achieved by CNN compared with SVM and ANN, respectively.

Restoring Motor Functions After Stroke: Multiple Approaches and Opportunities

More than 1.5 million people suffer a stroke in Europe per year and more than 70% of stroke survivors experience limited functional recovery of their upper limb, resulting in diminished quality of life. Therefore, interventions to address upper-limb impairment are a priority for stroke survivors and clinicians. While a significant body of evidence supports the use of conventional treatments, such as intensive motor training or constraint-induced movement therapy, the limited and heterogeneous improvements they allow are, for most patients, usually not sufficient to return to full autonomy. Various innovative neurorehabilitation strategies are emerging in order to enhance beneficial plasticity and improve motor recovery. Among them, robotic technologies, brain-computer interfaces, or noninvasive brain stimulation (NIBS) are showing encouraging results. These innovative interventions, such as NIBS, will only provide maximized effects, if the field moves away from the “one-fits all” approach toward a “patient-tailored” approach. After summarizing the most commonly used rehabilitation approaches, we will focus on NIBS and highlight the factors that limit its widespread use in clinical settings. Subsequently, we will propose potential biomarkers that might help to stratify stroke patients in order to identify the individualized optimal therapy. We will discuss future methodological developments, which could open new avenues for poststroke rehabilitation, toward more patient-tailored precision medicine approaches and pathophysiologically motivated strategies.

Simultaneous functional near-infrared spectroscopy and electroencephalography for monitoring of human brain activity and oxygenation: a review

Multimodal monitoring has become particularly common in the study of human brain function. In this context, combined, synchronous measurements of functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) are getting increased interest. Because of the absence of electro-optical interference, it is quite simple to integrate these two noninvasive recording procedures of brain activity. fNIRS and EEG are both scalp-located procedures. fNIRS estimates brain hemodynamic fluctuations relying on spectroscopic measurements, whereas EEG captures the macroscopic temporal dynamics of brain electrical activity through passive voltages evaluations. The "orthogonal" neurophysiological information provided by the two technologies and the increasing interest in the neurovascular coupling phenomenon further encourage their integration. This review provides, together with an introduction regarding the principles and future directions of the two technologies, an evaluation of major clinical and nonclinical applications of this flexible, low-cost combination of neuroimaging modalities. fNIRS-EEG systems exploit the ability of the two technologies to be conducted in an environment or experimental setting and/or on subjects that are generally not suited for other neuroimaging modalities, such as functional magnetic resonance imaging, positron emission tomography, and magnetoencephalography. fNIRS-EEG brain monitoring settles itself as a useful multimodal tool for brain electrical and hemodynamic activity investigation.

Effects of Light and Sound on the Prefrontal Cortex Activation and Emotional Function: A Functional Near-Infrared Spectroscopy Study

We constructed a near infrared spectroscopy-based real-time feedback system to estimate the subjects' emotional states using the changes in oxygenated hemoglobin concentration [Δ(oxy-Hb)] in the prefrontal cortex (PFC). Using this system, we investigated the influences of continual mild and equivocal stimuli consisting of lights and a reconstructed waterfall sound on Δ[oxy-Hb] in the PFC. The visual (light) and auditory (sound) stimuli changed randomly and independently, depending on the emotional states of the individual subjects. The emotional states induced by the stimuli were examined via a questionnaire rated on an 11-point scale, from +5 (pleasant) to −5 (unpleasant), through 0 (neutral), after the 5-min experiments. Results from 757 subjects revealed that Δ[oxy-Hb] in the PFC exhibited a weak, but significant, correlation with emotional change, with the given continual and mild stimuli similar to that experienced in response to the intense pleasant/unpleasant stimuli. Based on the results we discuss the generation of pleasant/unpleasant weak emotional change induced by mild and weak stimuli such as light and sound.