Neural Internet: Web Surfing with Brain Potentials for the Completely Paralyzed

Subject Separation Network for Reducing Calibration Time of MI-Based BCI

Motor imagery brain-computer interface (MI-based BCIs) have demonstrated great potential in various applications. However, to well generalize classifiers to new subjects, a time-consuming calibration process is necessary due to high inter-subject variabilities of EEG signals. This process is costly and tedious, hindering the further expansion of MI-based BCIs outside of the laboratory. To reduce the calibration time of MI-based BCIs, we propose a novel domain adaptation framework that adapts multiple source subjects' labeled data to the unseen trials of target subjects. Firstly, we train one Subject Separation Network(SSN) for each of the source subjects in the dataset. Based on adversarial domain adaptation, a shared encoder is constructed to learn similar representations for both domains. Secondly, to model the factors that cause subject variabilities and eliminate the correlated noise existing in common feature space, private feature spaces orthogonal to the shared counterpart are learned for each subject. We use a shared decoder to validate that the model is actually learning from task-relevant neurophysiological information. At last, an ensemble classifier is built by the integration of the SSNs using the information extracted from each subject's task-relevant characteristics. To quantify the efficacy of the framework, we analyze the accuracy-calibration cost trade-off in MI-based BCIs, and theoretically guarantee a generalization bound on the target error. Visualizations of the transformed features illustrate the effectiveness of domain adaptation. The experimental results on the BCI Competition IV-IIa dataset prove the effectiveness of the proposed framework compared with multiple classification methods. We infer from our results that users could learn to control MI-based BCIs without a heavy calibration process. Our study further shows how to design and train Neural Networks to decode task-related information from different subjects and highlights the potential of deep learning methods for inter-subject EEG decoding.

How standardized are “standard protocols”? Variations in protocol and performance evaluation for slow cortical potential neurofeedback: A systematic review

Neurofeedback (NF) aims to alter neural activity by enhancing self-regulation skills. Over the past decade NF has received considerable attention as a potential intervention option for many somatic and mental conditions and ADHD in particular. However, placebo-controlled trials have demonstrated insufficient superiority of NF compared to treatment as usual and sham conditions. It has been argued that the reason for limited NF effects may be attributable to participants' challenges to self-regulate the targeted neural activity. Still, there is support of NF efficacy when only considering so-called “standard protocols,” such as Slow Cortical Potential NF training (SCP-NF). This PROSPERO registered systematic review following PRISMA criteria searched literature databases for studies applying SCP-NF protocols. Our review focus concerned the operationalization of self-regulatory success, and protocol-details that could influence the evaluation of self-regulation. Such details included; electrode placement, number of trials, length per trial, proportions of training modalities, handling of artifacts and skill-transfer into daily-life. We identified a total of 63 eligible reports published in the year 2000 or later. SCP-NF protocol-details varied considerably on most variables, except for electrode placement. However, due to the increased availability of commercial systems, there was a trend to more uniform protocol-details. Although, token-systems are popular in SCP-NF for ADHD, only half reported a performance-based component. Also, transfer exercises have become a staple part of SCP-NF. Furthermore, multiple operationalizations of regulatory success were identified, limiting comparability between studies, and perhaps usefulness of so-called transfer-exercises, which purpose is to facilitate the transfer of the self-regulatory skills into every-day life. While studies utilizing SCP as Brain-Computer-Interface mainly focused on the acquisition of successful self-regulation, clinically oriented studies often neglected this. Congruently, rates of successful regulators in clinical studies were mostly low (<50%). The relation between SCP self-regulation and behavior, and how symptoms in different disorders are affected, is complex and not fully understood. Future studies need to report self-regulation based on standardized measures, in order to facilitate both comparability and understanding of the effects on symptoms. When applied as treatment, future SCP-NF studies also need to put greater emphasis on the acquisition of self-regulation (before evaluating symptom outcomes).

Using Brain-Computer Interface to Control a Virtual Drone Using Non-Invasive Motor Imagery and Machine Learning

In recent years, the control of devices “by the power of the mind” has become a very controversial topic but has also been very well researched in the field of state-of-the-art gadgets, such as smartphones, laptops, tablets and even smart TVs, and also in medicine, to be used by people with disabilities for whom these technologies may be the only way to communicate with the outside world. It is well known that BCI control is a skill and can be improved through practice and training. This paper aims to improve and diversify signal processing methods for the implementation of a brain-computer interface (BCI) based on neurological phenomena recorded during motor tasks using motor imagery (MI). The aim of the research is to extract, select and classify the characteristics of electroencephalogram (EEG) signals, which are based on sensorimotor rhythms, for the implementation of BCI systems. This article investigates systems based on brain-computer interfaces, especially those that use the electroencephalogram as a method of acquisition of MI tasks. The purpose of this article is to allow users to manipulate quadcopter virtual structures (external, robotic objects) simply through brain activity, correlated with certain mental tasks using undecimal transformation (UWT) to reduce noise, Independent Component Analysis (ICA) together with determination coefficient (r2) and, for classification, a hybrid neural network consisting of Radial Basis Functions (RBF) and a multilayer perceptron–recurrent network (MLP–RNN), obtaining a classification accuracy of 95.5%. Following the tests performed, it can be stated that the use of biopotentials in human–computer interfaces is a viable method for applications in the field of BCI. The results presented show that BCI training can produce a rapid change in behavioral performance and cognitive properties. If more than one training session is used, the results may be beneficial for increasing poor cognitive performance. To achieve this goal, three steps were taken: understanding the functioning of BCI systems and the neurological phenomena involved; acquiring EEG signals based on sensorimotor rhythms recorded during MI tasks; applying and optimizing extraction methods, selecting and classifying characteristics using neuronal networks.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.

A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback

Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.

EEG- and EOG-Based Asynchronous Hybrid BCI: A System Integrating a Speller, a Web Browser, an E-Mail Client, and a File Explorer

This paper presents a new asynchronous hybrid brain-computer interface (BCI) system that integrates a speller, a web browser, an e-mail client, and a file explorer using electroencephalographic (EEG) and electrooculography (EOG) signals. More specifically, an EOG-based button selection method, which requires the user to blink his/her eyes synchronously with the target button's flashes during button selection, is first presented. Next, we propose a mouse control method by combining EEG and EOG signals, in which the left-/right-hand motor imagery (MI)-related EEG is used to control the horizontal movement of the mouse and the blink-related EOG is used to control the vertical movement of the mouse and to select/reject a target. These two methods are further combined to develop the integrated hybrid BCI system. With the hybrid BCI, users can input text, access the internet, communicate with others via e-mail, and manage files in their computer using only EEG and EOG without any body movements. Ten healthy subjects participated in a comprehensive online experiment, and superior performance was achieved compared with our previously developed P300- and MI-based BCI and some other asynchronous BCIs, therefore demonstrating the system's effectiveness.

Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain–computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user’s environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.

Cortical control of a tablet computer by people with paralysis

General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

EEG-Based Brain-Computer Interfaces for Communication and Rehabilitation of People with Motor Impairment: A Novel Approach of the 21 st Century

People with severe neurological impairments face many challenges in sensorimotor functions and communication with the environment; therefore they have increased demand for advanced, adaptive and personalized rehabilitation. During the last several decades, numerous studies have developed brain-computer interfaces (BCIs) with the goals ranging from providing means of communication to functional rehabilitation. Here we review the research on non-invasive, electroencephalography (EEG)-based BCI systems for communication and rehabilitation. We focus on the approaches intended to help severely paralyzed and locked-in patients regain communication using three different BCI modalities: slow cortical potentials, sensorimotor rhythms and P300 potentials, as operational mechanisms. We also review BCI systems for restoration of motor function in patients with spinal cord injury and chronic stroke. We discuss the advantages and limitations of these approaches and the challenges that need to be addressed in the future.

A hybrid BCI web browser based on EEG and EOG signals

In this study, we propose a new web browser based on a hybrid brain computer interface (BCI) combining electroencephalographic (EEG) and electrooculography (EOG) signals. Specifically, the user can control the horizontal movement of the mouse by imagining left/right hand motion, and control the vertical movement of the mouse, select/reject a target, or input text in an edit box by blinking eyes in synchrony with the flashes of the corresponding buttons on the GUI. Based on mouse control, target selection and text input, the user can open a web page of interest, select an intended target in the web and read the page content. An online experiment was conducted involving five healthy subjects. The experimental results demonstrated the effectiveness of the proposed method.

An Asynchronous P300-Based Brain-Computer Interface Web Browser for Severely Disabled People

This paper presents an electroencephalographic (EEG) P300-based brain-computer interface (BCI) Internet browser. The system uses the "odd-ball" row-col paradigm for generating the P300 evoked potentials on the scalp of the user, which are immediately processed and translated into web browser commands. There were previous approaches for controlling a BCI web browser. However, to the best of our knowledge, none of them was focused on an assistive context, failing to test their applications with a suitable number of end users. In addition, all of them were synchronous applications, where it was necessary to introduce a "read-mode" command in order to avoid a continuous command selection. Thus, the aim of this study is twofold: 1) to test our web browser with a population of multiple sclerosis (MS) patients in order to assess the usefulness of our proposal to meet their daily communication needs; and 2) to overcome the aforementioned limitation by adding a threshold that discerns between control and non-control states, allowing the user to calmly read the web page without undesirable selections. The browser was tested with sixteen MS patients and five healthy volunteers. Both quantitative and qualitative metrics were obtained. MS participants reached an average accuracy of 84.14%, whereas 95.75% was achieved by control subjects. Results show that MS patients can successfully control the BCI web browser, improving their personal autonomy.

Communications Technology and Motor Neuron Disease: An Australian Survey of People With Motor Neuron Disease

BACKGROUND

People with Motor Neuron Disease (MND), of which amyotrophic lateral sclerosis (ALS) is the most common form in adults, typically experience difficulties with communication and disabilities associated with movement. Assistive technology is essential to facilitate everyday activities, promote social support and enhance quality of life.

OBJECTIVE

This study aimed to explore the types of mainstream and commonly available communication technology used by people with MND including software and hardware, to identify the levels of confidence and skill that people with MND reported in using technology, to determine perceived barriers to the use of technology for communication, and to investigate the willingness of people with MND to adopt alternative modes of communication.

METHODS

An on-line survey was distributed to members of the New South Wales Motor Neuron Disease Association (MND NSW). Descriptive techniques were used to summarize frequencies of responses and cross tabulate data. Free-text responses to survey items and verbal comments from participants who chose to undertake the survey by telephone were analyzed using thematic analysis.

RESULTS

Responses from 79 MND NSW members indicated that 15-21% had difficulty with speaking, writing and/or using a keyboard. Commonly used devices were desktop computers, laptops, tablets and mobile phones. Most participants (84%) were connected to the Internet and used it for email (91%), to find out more about MND (59%), to follow the news (50%) or for on-line shopping (46%). A third of respondents used Skype or its equivalent, but few used this to interact with health professionals.

CONCLUSIONS

People with MND need greater awareness of technology options to access the most appropriate solutions. The timing for people with MND to make decisions about technology is critical. Health professionals need skills and knowledge about the application of technology to be able to work with people with MND to select the best communication technology options as early as possible after diagnosis. If people with MND are willing to trial telehealth technology, there is potential for tele-consultations via Skype or its equivalent, with health professionals. People with MND can benefit from health professional involvement to match technology to their functional limitations and personal preferences. However, health professionals need a comprehensive understanding of the application of available technology to achieve this.

Toward independent home use of brain-computer interfaces: a decision algorithm for selection of potential end-users

Noninvasive brain-computer interfaces (BCIs) use scalp-recorded electrical activity from the brain to control an application. Over the past 20 years, research demonstrating that BCIs can provide communication and control to individuals with severe motor impairment has increased almost exponentially. Although considerable effort has been dedicated to offline analysis for improving signal detection and translation, far less effort has been made to conduct online studies with target populations. Thus, there remains a great need for both long-term and translational BCI studies that include individuals with disabilities in their own homes. Completing these studies is the only sure means to answer questions about BCI utility and reliability. Here we suggest an algorithm for candidate selection for electroencephalographic (EEG)-based BCI home studies. This algorithm takes into account BCI end-users and their environment and should assist in study design and substantially improve subject retention rates, thereby improving the overall efficacy of BCI home studies. It is the result of a workshop at the Fifth International BCI Meeting that allowed us to leverage the expertise of multiple research laboratories and people from multiple backgrounds in BCI research.

Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state: a case study

OBJECTIVE

Despite intense brain-computer interface (BCI) research for >2 decades, BCIs have hardly been established at patients' homes. The current study aimed at demonstrating expert independent BCI home use by a patient in the locked-in state and the effect it has on quality of life.

DESIGN

In this case study, the P300 BCI-controlled application Brain Painting was facilitated and installed at the patient's home. Family and caregivers were trained in setting up the BCI system. After every BCI session, the end user indicated subjective level of control, loss of control, level of exhaustion, satisfaction, frustration, and enjoyment. To monitor BCI home use, evaluation data of every session were automatically sent and stored on a remote server. Satisfaction with the BCI as an assistive device and subjective workload was indicated by the patient. In accordance with the user-centered design, usability of the BCI was evaluated in terms of its effectiveness, efficiency, and satisfaction. The influence of the BCI on quality of life of the end user was assessed.

SETTING

At the patient's home.

PARTICIPANT

A 73-year-old patient with amyotrophic lateral sclerosis in the locked-in state.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

The BCI has been used by the patient independent of experts for >14 months. The patient painted in about 200 BCI sessions (1-3 times per week) with a mean painting duration of 81.86 minutes (SD=52.15, maximum: 230.41). BCI improved quality of life of the patient.

RESULTS

In most of the BCI sessions the end user's satisfaction was high (mean=7.4, SD=3.24; range, 0-10). Dissatisfaction occurred mostly because of technical problems at the beginning of the study or varying BCI control. The subjective workload was moderate (mean=40.61; range, 0-100). The end user was highy satisfied with all components of the BCI (mean 4.42-5.0; range, 1-5). A perfect match between the user and the BCI technology was achieved (mean: 4.8; range, 1-5). Brain Painting had a positive impact on the patient's life on all three dimensions: competence (1.5), adaptability (2.17) and self-esteem (1.5); (range: -3 = maximum negative impact; 3 maximum positive impact). The patient had her first public art exhibition in July 2013; future exhibitions are in preparation.

CONCLUSIONS

Independent BCI home use is possible with high satisfaction for the end user. The BCI indeed positively influenced quality of life of the patient and supports social inclusion. Results demonstrate that visual P300 BCIs can be valuable for patients in the locked-in state even if other means of communication are still available (eye tracker).

A review of EEG-based brain-computer interfaces as access pathways for individuals with severe disabilities

Electroencephalography (EEG) is a non-invasive method for measuring brain activity and is a strong candidate for brain-computer interface (BCI) development. While BCIs can be used as a means of communication for individuals with severe disabilities, the majority of existing studies have reported BCI evaluations by able-bodied individuals. Considering the many differences in body functions and usage scenarios between individuals with disabilities and able-bodied individuals, involvement of the target population in BCI evaluation is necessary. In this review, 39 studies reporting EEG-oriented BCI assessment by individuals with disabilities were identified in the past decade. With respect to participant populations, a need for assessing BCI performance for the pediatric population with severe disabilities was identified as an important future direction. Acquiring a reliable communication pathway during early stages of development is crucial in avoiding learned helplessness in pediatric-onset disabilities. With respect to evaluation, augmenting traditional measures of system performance with those relating to contextual factors was recommended for realizing user-centered designs appropriate for integration in real-life. Considering indicators of user state and developing more effective training paradigms are recommended for future studies of BCI involving individuals with disabilities.

Asynchronous BCI based on motor imagery with automated calibration and neurofeedback training

A new multiclass brain-computer interface (BCI) based on the modulation of sensorimotor oscillations by imagining movements is described. By the application of advanced signal processing tools, statistics and machine learning, this BCI system offers: 1) asynchronous mode of operation, 2) automatic selection of user-dependent parameters based on an initial calibration, 3) incremental update of the classifier parameters from feedback data. The signal classification uses spatially filtered signals and is based on spectral power estimation computed in individualized frequency bands, which are automatically identified by a specially tailored AR-based model. Relevant features are chosen by a criterion based on Mutual Information. Final recognition of motor imagery is effectuated by a multinomial logistic regression classifier. This BCI system was evaluated in two studies. In the first study, five participants trained the ability to imagine movements of the right hand, left hand and feet in response to visual cues. The accuracy of the classifier was evaluated across four training sessions with feedback. The second study assessed the information transfer rate (ITR) of the BCI in an asynchronous application. The subjects' task was to navigate a cursor along a computer rendered 2-D maze. A peak information transfer rate of 8.0 bit/min was achieved. Five subjects performed with a mean ITR of 4.5 bit/min and an accuracy of 74.84%. These results demonstrate that the use of automated interfaces to reduce complexity for the intended operator (outside the laboratory) is indeed possible. The signal processing and classifier source code embedded in BCI2000 is available from https://www.brain-project.org/downloads.html.

Brain computer interfaces, a review

A brain-computer interface (BCI) is a hardware and software communications system that permits cerebral activity alone to control computers or external devices. The immediate goal of BCI research is to provide communications capabilities to severely disabled people who are totally paralyzed or 'locked in' by neurological neuromuscular disorders, such as amyotrophic lateral sclerosis, brain stem stroke, or spinal cord injury. Here, we review the state-of-the-art of BCIs, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface. We discuss their advantages, drawbacks, and latest advances, and we survey the numerous technologies reported in the scientific literature to design each step of a BCI. First, the review examines the neuroimaging modalities used in the signal acquisition step, each of which monitors a different functional brain activity such as electrical, magnetic or metabolic activity. Second, the review discusses different electrophysiological control signals that determine user intentions, which can be detected in brain activity. Third, the review includes some techniques used in the signal enhancement step to deal with the artifacts in the control signals and improve the performance. Fourth, the review studies some mathematic algorithms used in the feature extraction and classification steps which translate the information in the control signals into commands that operate a computer or other device. Finally, the review provides an overview of various BCI applications that control a range of devices.

A telepresence mobile robot controlled with a noninvasive brain-computer interface

This paper reports an electroencephalogram-based brain-actuated telepresence system to provide a user with presence in remote environments through a mobile robot, with access to the Internet. This system relies on a P300-based brain-computer interface (BCI) and a mobile robot with autonomous navigation and camera orientation capabilities. The shared-control strategy is built by the BCI decoding of task-related orders (selection of visible target destinations or exploration areas), which can be autonomously executed by the robot. The system was evaluated using five healthy participants in two consecutive steps: 1) screening and training of participants and 2) preestablished navigation and visual exploration telepresence tasks. On the basis of the results, the following evaluation studies are reported: 1) technical evaluation of the device and its main functionalities and 2) the users' behavior study. The overall result was that all participants were able to complete the designed tasks, reporting no failures, which shows the robustness of the system and its feasibility to solve tasks in real settings where joint navigation and visual exploration were needed. Furthermore, the participants showed great adaptation to the telepresence system.

Goal selection versus process control while learning to use a brain-computer interface

A brain-computer interface (BCI) can be used to accomplish a task without requiring motor output. Two major control strategies used by BCIs during task completion are process control and goal selection. In process control, the user exerts continuous control and independently executes the given task. In goal selection, the user communicates their goal to the BCI and then receives assistance executing the task. A previous study has shown that goal selection is more accurate and faster in use. An unanswered question is, which control strategy is easier to learn? This study directly compares goal selection and process control while learning to use a sensorimotor rhythm-based BCI. Twenty young healthy human subjects were randomly assigned either to a goal selection or a process control-based paradigm for eight sessions. At the end of the study, the best user from each paradigm completed two additional sessions using all paradigms randomly mixed. The results of this study were that goal selection required a shorter training period for increased speed, accuracy, and information transfer over process control. These results held for the best subjects as well as in the general subject population. The demonstrated characteristics of goal selection make it a promising option to increase the utility of BCIs intended for both disabled and able-bodied users.

Clinical study of neurorehabilitation in stroke using EEG-based motor imagery brain-computer interface with robotic feedback

This clinical study investigates the ability of hemiparetic stroke patients in operating EEG-based motor imagery brain-computer interface (MI-BCI). It also assesses the efficacy in motor improvements on the stroke-affected upper limb using EEG-based MI-BCI with robotic feedback neurorehabilitation compared to robotic rehabilitation that delivers movement therapy. 54 hemiparetic stroke patients with mean age of 51.8 and baseline Fugl-Meyer Assessment (FMA) 14.9 (out of 66, higher = better) were recruited. Results showed that 48 subjects (89%) operated EEG-based MI-BCI better than at chance level, and their ability to operate EEG-based MI-BCI is not correlated to their baseline FMA (r=0.358). Those subjects who gave consent are randomly assigned to each group (N=11 and 14) for 12 1-hour rehabilitation sessions for 4 weeks. Significant gains in FMA scores were observed in both groups at post-rehabilitation (4.5, 6.2; p=0.032, 0.003) and 2-month post-rehabilitation (5.3, 7.3; p=0.020, 0.013), but no significant differences were observed between groups (p=0.512, 0.550). Hence, this study showed evidences that a majority of hemiparetic stroke patients can operate EEG-based MI-BCI, and that EEG-based MI-BCI with robotic feedback neurorehabilitation is effective in restoring upper extremities motor function in stroke.

EEG control of a virtual helicopter in 3-dimensional space using intelligent control strategies

Films like Firefox, Surrogates, and Avatar have explored the possibilities of using brain-computer interfaces (BCIs) to control machines and replacement bodies with only thought. Real world BCIs have made great progress toward that end. Invasive BCIs have enabled monkeys to fully explore 3-D space using neuroprosthetics. However, noninvasive BCIs have not been able to demonstrate such mastery of 3-D space. Here, we report our work, which demonstrates that human subjects can use a noninvasive BCI to fly a virtual helicopter to any point in a 3-D world. Through use of intelligent control strategies, we have facilitated the realization of controlled flight in 3-D space. We accomplished this through a reductionist approach that assigns subject-specific control signals to the crucial components of 3-D flight. Subject control of the helicopter was comparable when using either the BCI or a keyboard. By using intelligent control strategies, the strengths of both the user and the BCI system were leveraged and accentuated. Intelligent control strategies in BCI systems such as those presented here may prove to be the foundation for complex BCIs capable of doing more than we ever imagined.

Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges

In recent years, new research has brought the field of electroencephalogram (EEG)-based brain–computer interfacing (BCI) out of its infancy and into a phase of relative maturity through many demonstrated prototypes such as brain-controlled wheelchairs, keyboards, and computer games. With this proof-of-concept phase in the past, the time is now ripe to focus on the development of practical BCI technologies that can be brought out of the lab and into real-world applications. In particular, we focus on the prospect of improving the lives of countless disabled individuals through a combination of BCI technology with existing assistive technologies (AT). In pursuit of more practical BCIs for use outside of the lab, in this paper, we identify four application areas where disabled individuals could greatly benefit from advancements in BCI technology, namely, “Communication and Control”, “Motor Substitution”, “Entertainment”, and “Motor Recovery”. We review the current state of the art and possible future developments, while discussing the main research issues in these four areas. In particular, we expect the most progress in the development of technologies such as hybrid BCI architectures, user–machine adaptation algorithms, the exploitation of users’ mental states for BCI reliability and confidence measures, the incorporation of principles in human–computer interaction (HCI) to improve BCI usability, and the development of novel BCI technology including better EEG devices.

Nessi: an EEG-controlled web browser for severely paralyzed patients

We have previously demonstrated that an EEG-controlled web browser based on self-regulation of slow cortical potentials (SCPs) enables severely paralyzed patients to browse the internet independently of any voluntary muscle control. However, this system had several shortcomings, among them that patients could only browse within a limited number of web pages and had to select links from an alphabetical list, causing problems if the link names were identical or if they were unknown to the user (as in graphical links). Here we describe a new EEG-controlled web browser, called Nessi, which overcomes these shortcomings. In Nessi, the open source browser, Mozilla, was extended by graphical in-place markers, whereby different brain responses correspond to different frame colors placed around selectable items, enabling the user to select any link on a web page. Besides links, other interactive elements are accessible to the user, such as e-mail and virtual keyboards, opening up a wide range of hypertext-based applications.

Matching brain-machine interface performance to space applications

A brain-machine interface (BMI) is a particular class of human-machine interface (HMI). BMIs have so far been studied mostly as a communication means for people who have little or no voluntary control of muscle activity. For able-bodied users, such as astronauts, a BMI would only be practical if conceived as an augmenting interface. A method is presented for pointing out effective combinations of HMIs and applications of robotics and automation to space. Latency and throughput are selected as performance measures for a hybrid bionic system (HBS), that is, the combination of a user, a device, and a HMI. We classify and briefly describe HMIs and space applications and then compare the performance of classes of interfaces with the requirements of classes of applications, both in terms of latency and throughput. Regions of overlap correspond to effective combinations. Devices requiring simpler control, such as a rover, a robotic camera, or environmental controls are suitable to be driven by means of BMI technology. Free flyers and other devices with six degrees of freedom can be controlled, but only at low-interactivity levels. More demanding applications require conventional interfaces, although they could be controlled by BMIs once the same levels of performance as currently recorded in animal experiments are attained. Robotic arms and manipulators could be the next frontier for noninvasive BMIs. Integrating smart controllers in HBSs could improve interactivity and boost the use of BMI technology in space applications.

Principles of neural ensemble physiology underlying the operation of brain-machine interfaces

Research on brain-machine interfaces has been ongoing for at least a decade. During this period, simultaneous recordings of the extracellular electrical activity of hundreds of individual neurons have been used for direct, real-time control of various artificial devices. Brain-machine interfaces have also added greatly to our knowledge of the fundamental physiological principles governing the operation of large neural ensembles. Further understanding of these principles is likely to have a key role in the future development of neuroprosthetics for restoring mobility in severely paralysed patients.

Another kind of 'BOLD Response': answering multiple-choice questions via online decoded single-trial brain signals

The term 'locked-in'syndrome (LIS) describes a medical condition in which persons concerned are severely paralyzed and at the same time fully conscious and awake. The resulting anarthria makes it impossible for these patients to naturally communicate, which results in diagnostic as well as serious practical and ethical problems. Therefore, developing alternative, muscle-independent communication means is of prime importance. Such communication means can be realized via brain-computer interfaces (BCIs) circumventing the muscular system by using brain signals associated with preserved cognitive, sensory, and emotional brain functions. Primarily, BCIs based on electrophysiological measures have been developed and applied with remarkable success. Recently, also blood flow-based neuroimaging methods, such as functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS), have been explored in this context. After reviewing recent literature on the development of especially hemodynamically based BCIs, we introduce a highly reliable and easy-to-apply communication procedure that enables untrained participants to motor-independently and relatively effortlessly answer multiple-choice questions based on intentionally generated single-trial fMRI signals that can be decoded online. Our technique takes advantage of the participants' capability to voluntarily influence certain spatio-temporal aspects of the blood oxygenation level-dependent (BOLD) signal: source location (by using different mental tasks), signal onset and offset. We show that healthy participants are capable of hemodynamically encoding at least four distinct information units on a single-trial level without extensive pretraining and with little effort. Moreover, real-time data analysis based on simple multi-filter correlations allows for automated answer decoding with a high accuracy (94.9%) demonstrating the robustness of the presented method. Following our 'proof of concept', the next step will involve clinical trials with LIS patients, undertaken in close collaboration with their relatives and caretakers in order to elaborate individually tailored communication protocols. As our procedure can be easily transferred to MRI-equipped clinical sites, it may constitute a simple and effective possibility for online detection of residual consciousness and for LIS patients to communicate basic thoughts and needs in case no other alternative communication means are available (yet)--especially in the acute phase of the LIS. Future research may focus on further increasing the efficiency and accuracy of fMRI-based BCIs by implementing sophisticated data analysis methods (e.g., multivariate and independent component analysis) and neurofeedback training techniques. Finally, the presented BCI approach could be transferred to portable fNIRS systems as only this would enable hemodynamically based communication in daily life situations.

Clinical Applications of Brain-Computer Interfaces: Current State and Future Prospects

Brain-computer interfaces (BCIs) allow their users to communicate or control external devices using brain signals rather than the brain's normal output pathways of peripheral nerves and muscles. Motivated by the hope of restoring independence to severely disabled individuals and by interest in further extending human control of external systems, researchers from many fields are engaged in this challenging new work. BCI research and development have grown explosively over the past two decades. Efforts have recently begun to provide laboratory-validated BCI systems to severely disabled individuals for real-world applications. In this review, we discuss the current status and future prospects of BCI technology and its clinical applications. We will define BCI, review the BCI-relevant signals from the human brain, and describe the functional components of BCIs. We will also review current clinical applications of BCI technology, and identify potential users and potential applications. Finally, we will discuss current limitations of BCI technology, impediments to its widespread clinical use, and expectations for the future.

Brain-computer interface systems: progress and prospects

Brain-computer interface (BCI) systems support communication through direct measures of neural activity without muscle activity. BCIs may provide the best and sometimes the only communication option for users disabled by the most severe neuromuscular disorders and may eventually become useful to less severely disabled and/or healthy individuals across a wide range of applications. This review discusses the structure and functions of BCI systems, clarifies terminology and addresses practical applications. Progress and opportunities in the field are also identified and explicated.

Brain-computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation

Brain-computer interfaces (BCIs) are a rehabilitation tool for tetraplegic patients that aim to improve quality of life by augmenting communication, control of the environment, and self-care. The neurobiology of both rehabilitation and BCI control depends upon learning to modify the efficacy of spared neural ensembles that represent movement, sensation and cognition through progressive practice with feedback and reward. To serve patients, BCI systems must become safe, reliable, cosmetically acceptable, quickly mastered with minimal ongoing technical support, and highly accurate even in the face of mental distractions and the uncontrolled environment beyond a laboratory. BCI technologies may raise ethical concerns if their availability affects the decisions of patients who become locked-in with brain stem stroke or amyotrophic lateral sclerosis to be sustained with ventilator support. If BCI technology becomes flexible and affordable, volitional control of cortical signals could be employed for the rehabilitation of motor and cognitive impairments in hemiplegic or paraplegic patients by offering on-line feedback about cortical activity associated with mental practice, motor intention, and other neural recruitment strategies during progressive task-oriented practice. Clinical trials with measures of quality of life will be necessary to demonstrate the value of near-term and future BCI applications.