15 years of BCI research at Graz University of Technology: current projects

Authentication with a one-dimensional CNN model using EEG-based brain-computer interface

Brain-computer interface (BCI) technology uses electroencephalogram (EEG) signals to create a direct interaction between the human body and its surroundings. Motor imagery (MI) classification using EEG signals is an important application that can help a rehabilitated or motor-impaired stroke patient perform certain tasks. Robust classification of these signals is an important step toward making the use of EEG more practical in many applications and less dependent on trained professionals. Deep learning methods have produced impressive results in BCI in recent years, especially with the availability of large electroencephalography (EEG) data sets. Dealing with EEG-MI signals is difficult because noise and other signal sources can interfere with the electrical amplitude of the brain, and its generalization ability is limited, so it is difficult to improve EEG classifiers. To address these issues, this paper presents a methodology based on one-dimensional convolutional neural networks (1-D CNN) for motor imagery (MI) recognition for the right hand, left hand, feet, and sedentary task. The proposed model is a lightweight model with fewer parameters and has an accuracy of 91.75%. Then, in an innovative exploitation of the four output classes, there is an idea that allows people with disabilities who are deprived of security measures, such as entering a secret code, to use the output classification, such as password codes. It is also an idea for a unique authentication system that is more secure and less vulnerable to theft or the like for a healthy person at the same time.

Classification of EEG Signals Based on Sparrow Search Algorithm-Deep Belief Network for Brain-Computer Interface

In brain-computer interface (BCI) systems, challenges are presented by the recognition of motor imagery (MI) brain signals. Established recognition approaches have achieved favorable performance from patterns like SSVEP, AEP, and P300, whereas the classification methods for MI need to be improved. Hence, seeking a classification method that exhibits high accuracy and robustness for application in MI-BCI systems is essential. In this study, the Sparrow search algorithm (SSA)-optimized Deep Belief Network (DBN), called SSA-DBN, is designed to recognize the EEG features extracted by the Empirical Mode Decomposition (EMD). The performance of the DBN is enhanced by the optimized hyper-parameters obtained through the SSA. Our method's efficacy was tested on three datasets: two public and one private. Results indicate a relatively high accuracy rate, outperforming three baseline methods. Specifically, on the private dataset, our approach achieved an accuracy of 87.83%, marking a significant 10.38% improvement over the standard DBN algorithm. For the BCI IV 2a dataset, we recorded an accuracy of 86.14%, surpassing the DBN algorithm by 9.33%. In the SMR-BCI dataset, our method attained a classification accuracy of 87.21%, which is 5.57% higher than that of the conventional DBN algorithm. This study demonstrates enhanced classification capabilities in MI-BCI, potentially contributing to advancements in the field of BCI.

Prediction of Gait Kinematics and Kinetics: A Systematic Review of EMG and EEG Signal Use and Their Contribution to Prediction Accuracy

Human-machine interfaces hold promise in enhancing rehabilitation by predicting and responding to subjects' movement intent. In gait rehabilitation, neural network architectures utilize lower-limb muscle and brain activity to predict continuous kinematics and kinetics during stepping and walking. This systematic review, spanning five databases, assessed 16 papers meeting inclusion criteria. Studies predicted lower-limb kinematics and kinetics using electroencephalograms (EEGs), electromyograms (EMGs), or a combination with kinematic data and anthropological parameters. Long short-term memory (LSTM) and convolutional neural network (CNN) tools demonstrated highest accuracies. EEG focused on joint angles, while EMG predicted moments and torque joints. Useful EEG electrode locations included C3, C4, Cz, P3, F4, and F8. Vastus Lateralis, Rectus Femoris, and Gastrocnemius were the most commonly accessed muscles for kinematic and kinetic prediction using EMGs. No studies combining EEGs and EMGs to predict lower-limb kinematics and kinetics during stepping or walking were found, suggesting a potential avenue for future development in this technology.

BCI-Walls: A robust methodology to predict if conscious EEG changes can be detected in the presence of artefacts

Brain computer interfaces (BCI) depend on reliable realtime detection of conscious EEG changes for example to control a video game. However, scalp recordings are contaminated with non-stationary noise, such as facial muscle activity and eye movements. This interferes with the detection process making it potentially unreliable or even impossible. We have developed a new methodology which provides a hard and measurable criterion if conscious EEG changes can be detected in the presence of non-stationary noise by requiring the signal-to-noise ratio of a scalp recording to be greater than the SNR-wall which in turn is based on the highest and lowest noise variances of the recording. As an instructional example, we have recorded signals from the central electrode Cz during eight different activities causing non-stationary noise such as playing a video game or reading out loud. The results show that facial muscle activity and eye-movements have a strong impact on the detectability of EEG and that minimising both eye-movement artefacts and muscle noise is essential to be able to detect conscious EEG changes.

Motor Imagery Classification for Brain Computer Interface Using Deep Convolutional Neural Networks and Mixup Augmentation

Goal: Building a DL model that can be trained on small EEG training set of a single subject presents an interesting challenge that this work is trying to address. In particular, this study is trying to avoid the need for long EEG data collection sessions, and without combining multiple subjects training datasets, which has a detrimental effect on the classification performance due to the inter-individual variability among subjects. Methods: A customized Convolutional Neural Network with mixup augmentation was trained with [Formula: see text]120 EEG trials for only one subject per model. Results: Modified ResNet18 and DenseNet121 models with mixup augmentation achieved 0.920 (95% Confidence Interval: 0.908, 0.933) and 0.933 (95% Confidence Interval: 0.922, 0.945) classification accuracy, respectively. Conclusions: We show that the designed classifiers resulted in a higher classification performance in comparison to other DL classifiers of previous studies on the same dataset, despite the limited training dataset used in this work.

Electroencephalogram-Based Motor Imagery Classification Using Deep Residual Convolutional Networks

The classification of electroencephalogram (EEG) signals is of significant importance in brain-computer interface (BCI) systems. Aiming to achieve intelligent classification of motor imagery EEG types with high accuracy, a classification methodology using the wavelet packet decomposition (WPD) and the proposed deep residual convolutional networks (DRes-CNN) is proposed. Firstly, EEG waveforms are segmented into sub-signals. Then the EEG signal features are obtained through the WPD algorithm, and some selected wavelet coefficients are retained and reconstructed into EEG signals in their respective frequency bands. Subsequently, the reconstructed EEG signals were utilized as input of the proposed deep residual convolutional networks to classify EEG signals. Finally, EEG types of motor imagination are classified by the DRes-CNN classifier intelligently. The datasets from BCI Competition were used to test the performance of the proposed deep learning classifier. Classification experiments show that the average recognition accuracy of this method reaches 98.76%. The proposed method can be further applied to the BCI system of motor imagination control.

Beyond Technologies of Electroencephalography-Based Brain-Computer Interfaces: A Systematic Review From Commercial and Ethical Aspects

The deployment of electroencephalographic techniques for commercial applications has undergone a rapid growth in recent decades. As they continue to expand in the consumer markets as suitable techniques for monitoring the brain activity, their transformative potential necessitates equally significant ethical inquiries. One of the main questions, which arises then when evaluating these kinds of applications, is whether they should be aligned or not with the main ethical concerns reported by scholars and experts. Thus, the present work attempts to unify these disciplines of knowledge by performing a comprehensive scan of the major electroencephalographic market applications as well as their most relevant ethical concerns arising from the existing literature. In this literature review, different databases were consulted, which presented conceptual and empirical discussions and findings about commercial and ethical aspects of electroencephalography. Subsequently, the content was extracted from the articles and the main conclusions were presented. Finally, an external assessment of the outcomes was conducted in consultation with an expert panel in some of the topic areas such as biomedical engineering, biomechatronics, and neuroscience. The ultimate purpose of this review is to provide a genuine insight into the cutting-edge practical attempts at electroencephalography. By the same token, it seeks to highlight the overlap between the market needs and the ethical standards that should govern the deployment of electroencephalographic consumer-grade solutions, providing a practical approach that overcomes the engineering myopia of certain ethical discussions.

Modeling the Ongoing Dynamics of Short and Long-Range Temporal Correlations in Broadband EEG During Movement

Electroencephalogram (EEG) undergoes complex temporal and spectral changes during voluntary movement intention. Characterization of such changes has focused mostly on narrowband spectral processes such as Event-Related Desynchronization (ERD) in the sensorimotor rhythms because EEG is mostly considered as emerging from oscillations of the neuronal populations. However, the changes in the temporal dynamics, especially in the broadband arrhythmic EEG have not been investigated for movement intention detection. The Long-Range Temporal Correlations (LRTC) are ubiquitously present in several neuronal processes, typically requiring longer timescales to detect. In this paper, we study the ongoing changes in the dynamics of long- as well as short-range temporal dependencies in the single trial broadband EEG during movement intention. We obtained LRTC in 2 s windows of broadband EEG and modeled it using the Autoregressive Fractionally Integrated Moving Average (ARFIMA) model which allowed simultaneous modeling of short- and long-range temporal correlations. There were significant (p < 0.05) changes in both broadband long- and short-range temporal correlations during movement intention and execution. We discovered that the broadband LRTC and narrowband ERD are complementary processes providing distinct information about movement because eliminating LRTC from the signal did not affect the ERD and conversely, eliminating ERD from the signal did not affect LRTC. Exploring the possibility of applications in Brain Computer Interfaces (BCI), we used hybrid features with combinations of LRTC, ARFIMA, and ERD to detect movement intention. A significantly higher (p < 0.05) classification accuracy of 88.3 ± 4.2% was obtained using the combination of ARFIMA and ERD features together, which also predicted the earliest movement at 1 s before its onset. The ongoing changes in the long- and short-range temporal correlations in broadband EEG contribute to effectively capturing the motor command generation and can be used to detect movement successfully. These temporal dependencies provide different and additional information about the movement.

Feature extraction of four-class motor imagery EEG signals based on functional brain network

OBJECTIVE

A motor-imagery-based brain-computer interface (MI-BCI) provides an alternative way for people to interface with the outside world. However, the classification accuracy of MI signals remains challenging, especially with an increased number of classes and the presence of high variations with data from multiple individual people. This work investigates electroencephalogram (EEG) signal processing techniques, aiming to enhance the classification performance of multiple MI tasks in terms of tackling the challenges caused by the vast variety of subjects.

APPROACH

This work introduces a novel method to extract discriminative features by combining the features of functional brain networks with two other feature extraction algorithms: common spatial pattern (CSP) and local characteristic-scale decomposition (LCD). After functional brain networks are established from the MI EEG signals of the subjects, the measures of degree in the binary networks are extracted as additional features and fused with features in the frequency and spatial domains extracted by the CSP and LCD algorithms. A real-time BCI robot control system is designed and implemented with the proposed method. Subjects can control the movement of the robot through four classes of MI tasks. Both the BCI competition IV dataset 2a and real-time data acquired in our designed system are used to validate the performance of the proposed method.

MAIN RESULTS

As for the offline data experiment results, the average classification accuracy of the proposed method reaches 79.7%, outperforming the majority of popular algorithms. Experimental results with real-time data also prove the proposed method to be highly promising in its real-time performance.

SIGNIFICANCE

The experimental results show that our proposed method is robust in extracting discriminative brain activity features when performing different MI tasks, hence improving the classification accuracy in four-class MI tasks. The high classification accuracy and low computational demand show a considerable practicality for real-time rehabilitation systems.

EEG patterns of self-paced movement imaginations towards externally-cued and internally-selected targets

In this study, we investigate the neurophysiological signature of the interacting processes which lead to a single reach-and-grasp movement imagination (MI). While performing this task, the human healthy participants could either define their movement targets according to an external cue, or through an internal selection process. After defining their target, they could start the MI whenever they wanted. We recorded high density electroencephalographic (EEG) activity and investigated two neural correlates: the event-related potentials (ERPs) associated with the target selection, which reflect the perceptual and cognitive processes prior to the MI, and the movement-related cortical potentials (MRCPs), associated with the planning of the self-paced MI. We found differences in frontal and parietal areas between the late ERP components related to the internally-driven selection and the externally-cued process. Furthermore, we could reliably estimate the MI onset of the self-paced task. Next, we extracted MRCP features around the MI onset to train classifiers of movement vs. rest directly on self-paced MI data. We attained performance significantly higher than chance level for both time-locked and asynchronous classification. These findings contribute to the development of more intuitive brain-computer interfaces in which movement targets are defined internally and the movements are self-paced.

From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach

In this chapter, we give an overview of the Graz-BCI research, from the classic motor imagery detection to complex movement intentions decoding. We start by describing the classic motor imagery approach, its application in tetraplegic end users, and the significant improvements achieved using coadaptive brain-computer interfaces (BCIs). These strategies have the drawback of not mirroring the way one plans a movement. To achieve a more natural control-and to reduce the training time-the movements decoded by the BCI need to be closely related to the user's intention. Within this natural control, we focus on the kinematic level, where movement direction and hand position or velocity can be decoded from noninvasive recordings. First, we review movement execution decoding studies, where we describe the decoding algorithms, their performance, and associated features. Second, we describe the major findings in movement imagination decoding, where we emphasize the importance of estimating the sources of the discriminative features. Third, we introduce movement target decoding, which could allow the determination of the target without knowing the exact movement-by-movement details. Aside from the kinematic level, we also address the goal level, which contains relevant information on the upcoming action. Focusing on hand-object interaction and action context dependency, we discuss the possible impact of some recent neurophysiological findings in the future of BCI control. Ideally, the goal and the kinematic decoding would allow an appropriate matching of the BCI to the end users' needs, overcoming the limitations of the classic motor imagery approach.

Noninvasive Electroencephalogram Based Control of a Robotic Arm for Writing Task Using Hybrid BCI System

A novel hybrid brain-computer interface (BCI) based on the electroencephalogram (EEG) signal which consists of a motor imagery- (MI-) based online interactive brain-controlled switch, “teeth clenching” state detector, and a steady-state visual evoked potential- (SSVEP-) based BCI was proposed to provide multidimensional BCI control. MI-based BCI was used as single-pole double throw brain switch (SPDTBS). By combining the SPDTBS with 4-class SSEVP-based BCI, movement of robotic arm was controlled in three-dimensional (3D) space. In addition, muscle artifact (EMG) of “teeth clenching” condition recorded from EEG signal was detected and employed as interrupter, which can initialize the statement of SPDTBS. Real-time writing task was implemented to verify the reliability of the proposed noninvasive hybrid EEG-EMG-BCI. Eight subjects participated in this study and succeeded to manipulate a robotic arm in 3D space to write some English letters. The mean decoding accuracy of writing task was 0.93 ± 0.03. Four subjects achieved the optimal criteria of writing the word “HI” which is the minimum movement of robotic arm directions (15 steps). Other subjects had needed to take from 2 to 4 additional steps to finish the whole process. These results suggested that our proposed hybrid noninvasive EEG-EMG-BCI was robust and efficient for real-time multidimensional robotic arm control.

Pattern Recognition of Momentary Mental Workload Based on Multi-Channel Electrophysiological Data and Ensemble Convolutional Neural Networks

In this paper, we deal with the Mental Workload (MWL) classification problem based on the measured physiological data. First we discussed the optimal depth (i.e., the number of hidden layers) and parameter optimization algorithms for the Convolutional Neural Networks (CNN). The base CNNs designed were tested according to five classification performance indices, namely Accuracy, Precision, F-measure, G-mean, and required training time. Then we developed an Ensemble Convolutional Neural Network (ECNN) to enhance the accuracy and robustness of the individual CNN model. For the ECNN design, three model aggregation approaches (weighted averaging, majority voting and stacking) were examined and a resampling strategy was used to enhance the diversity of individual CNN models. The results of MWL classification performance comparison indicated that the proposed ECNN framework can effectively improve MWL classification performance and is featured by entirely automatic feature extraction and MWL classification, when compared with traditional machine learning methods.

An Efficient Framework for EEG Analysis with Application to Hybrid Brain Computer Interfaces Based on Motor Imagery and P300

The hybrid brain computer interface (BCI) based on motor imagery (MI) and P300 has been a preferred strategy aiming to improve the detection performance through combining the features of each. However, current methods used for combining these two modalities optimize them separately, which does not result in optimal performance. Here, we present an efficient framework to optimize them together by concatenating the features of MI and P300 in a block diagonal form. Then a linear classifier under a dual spectral norm regularizer is applied to the combined features. Under this framework, the hybrid features of MI and P300 can be learned, selected, and combined together directly. Experimental results on the data set of hybrid BCI based on MI and P300 are provided to illustrate competitive performance of the proposed method against other conventional methods. This provides an evidence that the method used here contributes to the discrimination performance of the brain state in hybrid BCI.

Motor Imagery Classification Using Mu and Beta Rhythms of EEG with Strong Uncorrelating Transform Based Complex Common Spatial Patterns

Recent studies have demonstrated the disassociation between the mu and beta rhythms of electroencephalogram (EEG) during motor imagery tasks. The proposed algorithm in this paper uses a fully data-driven multivariate empirical mode decomposition (MEMD) in order to obtain the mu and beta rhythms from the nonlinear EEG signals. Then, the strong uncorrelating transform complex common spatial patterns (SUTCCSP) algorithm is applied to the rhythms so that the complex data, constructed with the mu and beta rhythms, becomes uncorrelated and its pseudocovariance provides supplementary power difference information between the two rhythms. The extracted features using SUTCCSP that maximize the interclass variances are classified using various classification algorithms for the separation of the left- and right-hand motor imagery EEG acquired from the Physionet database. This paper shows that the supplementary information of the power difference between mu and beta rhythms obtained using SUTCCSP provides an important feature for the classification of the left- and right-hand motor imagery tasks. In addition, MEMD is proved to be a preferred preprocessing method for the nonlinear and nonstationary EEG signals compared to the conventional IIR filtering. Finally, the random forest classifier yielded a high performance for the classification of the motor imagery tasks.

Efficient resting-state EEG network facilitates motor imagery performance

OBJECTIVE

Motor imagery-based brain-computer interface (MI-BCI) systems hold promise in motor function rehabilitation and assistance for motor function impaired people. But the ability to operate an MI-BCI varies across subjects, which becomes a substantial problem for practical BCI applications beyond the laboratory.

APPROACH

Several previous studies have demonstrated that individual MI-BCI performance is related to the resting state of brain. In this study, we further investigate offline MI-BCI performance variations through the perspective of resting-state electroencephalography (EEG) network.

MAIN RESULTS

Spatial topologies and statistical measures of the network have close relationships with MI classification accuracy. Specifically, mean functional connectivity, node degrees, edge strengths, clustering coefficient, local efficiency and global efficiency are positively correlated with MI classification accuracy, whereas the characteristic path length is negatively correlated with MI classification accuracy. The above results indicate that an efficient background EEG network may facilitate MI-BCI performance. Finally, a multiple linear regression model was adopted to predict subjects' MI classification accuracy based on the efficiency measures of the resting-state EEG network, resulting in a reliable prediction.

SIGNIFICANCE

This study reveals the network mechanisms of the MI-BCI and may help to find new strategies for improving MI-BCI performance.

The feasibility of a brain-computer interface functional electrical stimulation system for the restoration of overground walking after paraplegia

BACKGROUND

Direct brain control of overground walking in those with paraplegia due to spinal cord injury (SCI) has not been achieved. Invasive brain-computer interfaces (BCIs) may provide a permanent solution to this problem by directly linking the brain to lower extremity prostheses. To justify the pursuit of such invasive systems, the feasibility of BCI controlled overground walking should first be established in a noninvasive manner. To accomplish this goal, we developed an electroencephalogram (EEG)-based BCI to control a functional electrical stimulation (FES) system for overground walking and assessed its performance in an individual with paraplegia due to SCI.

METHODS

An individual with SCI (T6 AIS B) was recruited for the study and was trained to operate an EEG-based BCI system using an attempted walking/idling control strategy. He also underwent muscle reconditioning to facilitate standing and overground walking with a commercial FES system. Subsequently, the BCI and FES systems were integrated and the participant engaged in several real-time walking tests using the BCI-FES system. This was done in both a suspended, off-the-ground condition, and an overground walking condition. BCI states, gyroscope, laser distance meter, and video recording data were used to assess the BCI performance.

RESULTS

During the course of 19 weeks, the participant performed 30 real-time, BCI-FES controlled overground walking tests, and demonstrated the ability to purposefully operate the BCI-FES system by following verbal cues. Based on the comparison between the ground truth and decoded BCI states, he achieved information transfer rates >3 bit/s and correlations >0.9. No adverse events directly related to the study were observed.

CONCLUSION

This proof-of-concept study demonstrates for the first time that restoring brain-controlled overground walking after paraplegia due to SCI is feasible. Further studies are warranted to establish the generalizability of these results in a population of individuals with paraplegia due to SCI. If this noninvasive system is successfully tested in population studies, the pursuit of permanent, invasive BCI walking prostheses may be justified. In addition, a simplified version of the current system may be explored as a noninvasive neurorehabilitative therapy in those with incomplete motor SCI.

A novel Morse code-inspired method for multiclass motor imagery brain-computer interface (BCI) design

Motor imagery (MI)-based brain-computer interfaces (BCIs) allow disabled individuals to control external devices voluntarily, helping us to restore lost motor functions. However, the number of control commands available in MI-based BCIs remains limited, limiting the usability of BCI systems in control applications involving multiple degrees of freedom (DOF), such as control of a robot arm. To address this problem, we developed a novel Morse code-inspired method for MI-based BCI design to increase the number of output commands. Using this method, brain activities are modulated by sequences of MI (sMI) tasks, which are constructed by alternately imagining movements of the left or right hand or no motion. The codes of the sMI task was detected from EEG signals and mapped to special commands. According to permutation theory, an sMI task with N-length allows 2 × (2(N)-1) possible commands with the left and right MI tasks under self-paced conditions. To verify its feasibility, the new method was used to construct a six-class BCI system to control the arm of a humanoid robot. Four subjects participated in our experiment and the averaged accuracy of the six-class sMI tasks was 89.4%. The Cohen's kappa coefficient and the throughput of our BCI paradigm are 0.88 ± 0.060 and 23.5bits per minute (bpm), respectively. Furthermore, all of the subjects could operate an actual three-joint robot arm to grasp an object in around 49.1s using our approach. These promising results suggest that the Morse code-inspired method could be used in the design of BCIs for multi-DOF control.

Individually adapted imagery improves brain-computer interface performance in end-users with disability

Brain-computer interfaces (BCIs) translate oscillatory electroencephalogram (EEG) patterns into action. Different mental activities modulate spontaneous EEG rhythms in various ways. Non-stationarity and inherent variability of EEG signals, however, make reliable recognition of modulated EEG patterns challenging. Able-bodied individuals who use a BCI for the first time achieve - on average - binary classification performance of about 75%. Performance in users with central nervous system (CNS) tissue damage is typically lower. User training generally enhances reliability of EEG pattern generation and thus also robustness of pattern recognition. In this study, we investigated the impact of mental tasks on binary classification performance in BCI users with central nervous system (CNS) tissue damage such as persons with stroke or spinal cord injury (SCI). Motor imagery (MI), that is the kinesthetic imagination of movement (e.g. squeezing a rubber ball with the right hand), is the "gold standard" and mainly used to modulate EEG patterns. Based on our recent results in able-bodied users, we hypothesized that pair-wise combination of "brain-teaser" (e.g. mental subtraction and mental word association) and "dynamic imagery" (e.g. hand and feet MI) tasks significantly increases classification performance of induced EEG patterns in the selected end-user group. Within-day (How stable is the classification within a day?) and between-day (How well does a model trained on day one perform on unseen data of day two?) analysis of variability of mental task pair classification in nine individuals confirmed the hypothesis. We found that the use of the classical MI task pair hand vs. feed leads to significantly lower classification accuracy - in average up to 15% less - in most users with stroke or SCI. User-specific selection of task pairs was again essential to enhance performance. We expect that the gained evidence will significantly contribute to make imagery-based BCI technology become accessible to a larger population of users including individuals with special needs due to CNS damage.

Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e., the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g., electroencephalography (EEG), and have demonstrated the capability of multidimensional prosthesis control. This paper reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications, are reviewed. Finally, limitations of SMR-BCIs and future outlooks are also discussed.

Classification of wheelchair commands using brain computer interface: comparison between able-bodied persons and patients with tetraplegia

This paper presents a three-class mental task classification for an electroencephalography based brain computer interface. Experiments were conducted with patients with tetraplegia and able bodied controls. In addition, comparisons with different time-windows of data were examined to find the time window with the highest classification accuracy. The three mental tasks used were letter composing, arithmetic and imagery of a Rubik's cube rolling forward; these tasks were associated with three wheelchair commands: left, right and forward, respectively. An eyes closed task was also recorded for the algorithms testing and used as an additional on/off command. The features extraction method was based on the spectrum from a Hilbert-Huang transform and the classification algorithm was based on an artificial neural network with a fuzzy particle swarm optimization with cross-mutated operation. The results show a strong eyes closed detection for both groups with average accuracy at above 90%. The overall result for the combined groups shows an improved average accuracy of 70.6% at 1s, 74.8% at 2s, 77.8% at 3s, 79.6% at 4s and 81.4% at 5s. The accuracy for individual groups were lower for patients with tetraplegia compared to the able-bodied group, however, does improve with increased duration of the time-window.

BCILAB: a platform for brain-computer interface development

OBJECTIVE

The past two decades have seen dramatic progress in our ability to model brain signals recorded by electroencephalography, functional near-infrared spectroscopy, etc., and to derive real-time estimates of user cognitive state, response, or intent for a variety of purposes: to restore communication by the severely disabled, to effect brain-actuated control and, more recently, to augment human-computer interaction. Continuing these advances, largely achieved through increases in computational power and methods, requires software tools to streamline the creation, testing, evaluation and deployment of new data analysis methods.

APPROACH

Here we present BCILAB, an open-source MATLAB-based toolbox built to address the need for the development and testing of brain-computer interface (BCI) methods by providing an organized collection of over 100 pre-implemented methods and method variants, an easily extensible framework for the rapid prototyping of new methods, and a highly automated framework for systematic testing and evaluation of new implementations.

MAIN RESULTS

To validate and illustrate the use of the framework, we present two sample analyses of publicly available data sets from recent BCI competitions and from a rapid serial visual presentation task. We demonstrate the straightforward use of BCILAB to obtain results compatible with the current BCI literature.

SIGNIFICANCE

The aim of the BCILAB toolbox is to provide the BCI community a powerful toolkit for methods research and evaluation, thereby helping to accelerate the pace of innovation in the field, while complementing the existing spectrum of tools for real-time BCI experimentation, deployment and use.

Automatic single-trial classification of prefrontal hemodynamic activity in an individual with Duchenne muscular dystrophy

Brain-computer interfaces (BCIs) allow users to control external devices via brain activity alone, circumventing the somatic nervous system and the need for overt movement. Essential to BCI development is the ability to accurately detect and classify patterns of activation associated with different mental tasks. Here, we investigate the ability to automatically distinguish a mental arithmetic (MA) task from a natural baseline state in an individual with Duchenne muscular dystrophy (DMD) using signals acquired via multichannel near-infrared spectroscopy (NIRS). Using dual-wavelength NIRS, we interrogated nine sites around the frontopolar locations while the individual performed MA to answer multiple-choice questions within a system-paced paradigm. An encouraging overall classification accuracy of 71.1% was obtained, which is comparable to the average accuracy we previously reported for healthy individuals performing the same task. This result demonstrates the potential of NIRS-BCI based on task-induced prefrontal activity for use by individuals with DMD.

Real-time two-dimensional asynchronous control of a computer cursor with a single subdural electrode

OBJECTIVE

To test the feasibility of controlling a computer cursor asynchronously in two dimensions using one subdural electrode.

DESIGN

Proof of concept study.

SETTING

Acute care hospital in Toronto, Canada.

PARTICIPANT

A 68-year-old woman with a subdural electrode implanted for the treatment of essential tremor (ET) using direct brain stimulation of the primary motor cortex (MI).

INTERVENTIONS

Power changes in the electrocorticography signals were used to implement a "brain switch". To activate the switch the subject had to decrease the power in the 7-13 Hz frequency range using motor imagery of the left hand. The brain switch was connected to a system for asynchronous control of movement in two dimensions. Each time the user reduced the amplitude in the 7-13 Hz frequency band below an experimentally defined threshold the direction of cursor changed randomly. The new direction was always different from those previously rejected ensuring the convergence of the system on the desired direction.

OUTCOME MEASURES

Training time, time and number of switch activations required to reach specific targets, information transfer rate.

RESULTS

The user was able to control the cursor to specific targets on the screen after only 15 minutes of training. Each target was reached in 51.7 ± 40.2 seconds (mean ± SD) and after 9.4 ± 6.8 switch activations. Information transfer rate of the system was estimated to be 0.11 bit/second.

CONCLUSION

A novel brain-machine interface for asynchronous two-dimensional control using one subdural electrode was developed.

Toward functioning and usable brain-computer interfaces (BCIs): a literature review

PURPOSE

The aim of this paper is to provide an exhaustive review of the literature about brain-computer interfaces (BCIs) that could be used with these paralysed patients. The electroencephalography (EEG) is the best candidate for the continuous use in the environment of patients' houses, due to its portability and ease of use. For this reason, the present paper will focus on this kind of BCI. Moreover, it is our aim to focus more on the patients, regarding their active role in the modulation of the brain activity. This leads to a differentiation between studies that use an active regulation and studies that use a non-active regulation.

METHOD

Relevant articles in the BCIs field were selected using MEDLINE and PsycINFO.

RESULTS

Research through data banks produced 980 results, which were reduced to 127 after exclusion criteria selection. These references were divided in four categories, based on the use of active or non-active regulation, and on the event related potential used.

CONCLUSIONS

In most of the examined works, the focus was on the development of systems and algorithms able to recognise and classify brain events. Although this kind of research is fundamental, a user-centred point of view was rarely adopted. [Box: see text].

Toward a model-based predictive controller design in brain-computer interfaces

A first step in designing a robust and optimal model-based predictive controller (MPC) for brain-computer interface (BCI) applications is presented in this article. An MPC has the potential to achieve improved BCI performance compared to the performance achieved by current ad hoc, nonmodel-based filter applications. The parameters in designing the controller were extracted as model-based features from motor imagery task-related human scalp electroencephalography. Although the parameters can be generated from any model-linear or non-linear, we here adopted a simple autoregressive model that has well-established applications in BCI task discriminations. It was shown that the parameters generated for the controller design can as well be used for motor imagery task discriminations with performance (with 8-23% task discrimination errors) comparable to the discrimination performance of the commonly used features such as frequency specific band powers and the AR model parameters directly used. An optimal MPC has significant implications for high performance BCI applications.

Optimization of electrode channels in Brain Computer Interfaces

What is the optimal number of electrodes one can use in discrimination of tasks for a Brain Computer Interface (BCI)? To address this question, the number and location of scalp electrodes in the acquisition of human electroencephalography (EEG) and discrimination of motor imagery tasks were optimized by using a systematic optimization approach. The systematic analysis results in the most reliable procedure in electrode optimization as well as a validating means for the other feature selection techniques. We acquired human scalp EEG in response to cue-based motor imagery tasks. We employed a systematic analysis by using all possible combinations of the channels and calculating task discrimination errors for each of these combinations by using linear discriminant analysis (LDA) for feature classification. Channel combination that resulted in the smallest discrimination error was selected as the optimum number of channels to be used in BCI applications. Results from the systematic analysis were compared with another feature selection algorithm: forward stepwise feature selection combined with LDA feature classification. Our results demonstrate the usefulness of the fully optimized technique for a reliable selection of scalp electrodes in BCI applications.

Toward a hybrid brain-computer interface based on imagined movement and visual attention

Brain-computer interface (BCI) systems do not work for all users. This article introduces a novel combination of tasks that could inspire BCI systems that are more accurate than conventional BCIs, especially for users who cannot attain accuracy adequate for effective communication. Subjects performed tasks typically used in two BCI approaches, namely event-related desynchronization (ERD) and steady state visual evoked potential (SSVEP), both individually and in a 'hybrid' condition that combines both tasks. Electroencephalographic (EEG) data were recorded across three conditions. Subjects imagined moving the left or right hand (ERD), focused on one of the two oscillating visual stimuli (SSVEP), and then simultaneously performed both tasks. Accuracy and subjective measures were assessed. Offline analyses suggested that half of the subjects did not produce brain patterns that could be accurately discriminated in response to at least one of the two tasks. If these subjects produced comparable EEG patterns when trying to use a BCI, these subjects would not be able to communicate effectively because the BCI would make too many errors. Results also showed that switching to a different task used in BCIs could improve accuracy in some of these users. Switching to a hybrid approach eliminated this problem completely, and subjects generally did not consider the hybrid condition more difficult. Results validate this hybrid approach and suggest that subjects who cannot use a BCI should consider switching to a different BCI approach, especially a hybrid BCI. Subjects proficient with both approaches might combine them to increase information throughput by improving accuracy, reducing selection time, and/or increasing the number of possible commands.

Decoding three-dimensional hand kinematics from electroencephalographic signals

The capacity to decode kinematics of intended movement from neural activity is necessary for the development of neuromotor prostheses such as smart artificial arms. Thus far, most of the progress in the development of neuromotor prostheses has been achieved by decoding kinematics of the hand from intracranial neural activity. The comparatively low signal-to-noise ratio and spatial resolution of neural data acquired non-invasively from the scalp via electroencephalography (EEG) have been presumed to prohibit the extraction of detailed information about hand kinematics. Here, we challenge this presumption by attempting to continuously decoding hand position, velocity, and acceleration from 55-channel EEG signals acquired during three-dimensional center-out reaching from five subjects. To preserve ecological validity, reaches were self-initiated, and targets were self-selected. After cross-validation, the overall mean correlation coefficients between measured and reconstructed position, velocity, and acceleration were 0.2, 0.3, and 0.3 respectively. These modest results support the continued development of non-invasive neuromotor prostheses for movement-impaired individuals.

Motor imagery task discrimination using wide-band frequency spectra with Slepian tapers

We here studied the efficacy of wide-band frequency spectra (WBFS) features using multi-taper (MT) spectral analysis in application to motor imagery based Brain Computer Interfaces. We acquired motor imagery task related human scalp electroencephalography (EEG) signals for left vs. right hand movements using 3 different pairs of visual arrow cues. Left vs. right movement imagery discrimination was conducted using a Naïve Bayesian classifier using WBFS features and commonly used Mu-Beta spectral features for EEG signals from central+parietal and central only electrode positions. Task discrimination accuracy results showed that WBFS features using MT spectral analysis provided significantly better performance (with a 95% confidence level) than that of using Mu-Beta spectral features commonly used. The use of central+parietal electrode signals improved discrimination accuracy significantly when compared to the accuracy using the central only signals, implying that sensory information enhanced task discrimination significantly.

Encoding of speed and direction of movement in the human supplementary motor area

OBJECT

The supplementary motor area (SMA) plays an important role in planning, initiation, and execution of motor acts. Patients with SMA lesions are impaired in various kinematic parameters, such as velocity and duration of movement. However, the relationships between neuronal activity and these parameters in the human brain have not been fully characterized. This is a study of single-neuron activity during a continuous volitional motor task, with the goal of clarifying these relationships for SMA neurons and other frontal lobe regions in humans.

METHODS

The participants were 7 patients undergoing evaluation for epilepsy surgery requiring implantation of intracranial depth electrodes. Single-unit recordings were conducted while the patients played a computer game involving movement of a cursor in a simple maze.

RESULTS

In the SMA proper, most of the recorded units exhibited a monotonic relationship between the unit firing rate and hand motion speed. The vast majority of SMA proper units with this property showed an inverse relation, that is, firing rate decrease with speed increase. In addition, most of the SMA proper units were selective to the direction of hand motion. These relationships were far less frequent in the pre-SMA, anterior cingulate gyrus, and orbitofrontal cortex.

CONCLUSIONS

The findings suggest that the SMA proper takes part in the control of kinematic parameters of endeffector motion, and thus lend support to the idea of connecting neuroprosthetic devices to the human SMA.

Dimensionality reduction and channel selection of motor imagery electroencephalographic data

The performance of spatial filters based on independent components analysis (ICA) was evaluated by employing principal component analysis (PCA) preprocessing for dimensional reduction. The PCA preprocessing was not found to be a suitable method that could retain motor imagery information in a smaller set of components. In contrast, 6 ICA components selected on the basis of visual inspection performed comparably (61.9%) to the full range of 22 components (63.9%). An automated selection of ICA components based on a variance criterion was also carried out. Only 8 components chosen this way performed better (63.1%) than visually selected components. A similar analysis on the reduced set of electrodes over mid-central and centro-parietal regions of the brain revealed that common spatial patterns (CSPs) and Infomax were able to detect motor imagery activity with a satisfactory accuracy.

Control of a neuroprosthesis for grasping using off-line classification of electrocorticographic signals: case study

STUDY DESIGN

Proof of concept study to control a neuroprosthesis for grasping using identification of arm movements from ECoG signals.

OBJECTIVE

To test the feasibility of using electrocorticographic (ECoG) signals as a control method for a neuroprosthesis for grasping.

SETTING

Acute care hospital, Toronto Western Hospital and spinal cord injury (SCI) rehabilitation centre, Toronto Rehabilitation Institute, Lyndhurst Centre. Both hospitals are located in Toronto, Canada.

METHODS

Two subjects participated in this study. The first subject had subdural electrodes implanted on the motor cortex for the treatment of essential tremor (ET). ECoG signals were recorded while the subject performed specific arm movements. The second subject had a complete SCI at C6 level (ASIA B score) and was fitted with a neuroprosthesis, capable of identifying arm movements from ECoG signals off-line, for grasping. To operate the neuroprosthesis, subject 2 issued a command that would trigger the release of a randomly selected ECoG signal recorded from subject 1, associated with a particular arm movement. The neuroprosthesis identified which arm movement was performed at the time of recording and used that information to trigger the stimulation sequence. A correct ECoG classification resulted in the neuroprosthesis producing the correct hand function (that is grasp and release).

RESULTS

The neuroprosthesis classified ECoG signals correctly delivering the correct stimulation strategy with 94.5% accuracy.

CONCLUSIONS

The feasibility of using ECoG signals as a control strategy for a neuroprosthesis for grasping was shown.

Model-based responses and features in Brain Computer Interfaces

Novel model based features are introduced in the discrimination of motor imagery tasks using human scalp electroencephalography (EEG) towards the development of Brain Computer Interfaces (BCI). We have acquired human scalp EEG under open-loop and feedback conditions in response to cue-based motor imagery tasks. EEG signals, transformed into frequency specific bands such as mu, beta and movement related potentials, were used for feature extraction with the aim to discriminate tasks. Data were classified using features such as power spectrum and model-based parameters. Two different feature selection methods: stepwise and principal component analysis (PCA), were combined with linear discriminant analysis (LDA). Different training/validation criteria were applied for classification of task related features. Results show that the scalp EEG correlate of the imagery tasks of hands/toes/tongue movements under open-loop conditions and left/right hand movements under feedback conditions, can be well discriminated with classification errors below 20%. Model based techniques, which resulted in classification errors in the range of 2%-30%, have the potential to use advanced control systems theory in the development of BCI to achieve improved performance compared to the performance achieved by currently applied proportional control or filter algorithms.

Clinical Applications of Brain-Computer Interfaces: Current State and Future Prospects

Brain-computer interfaces (BCIs) allow their users to communicate or control external devices using brain signals rather than the brain's normal output pathways of peripheral nerves and muscles. Motivated by the hope of restoring independence to severely disabled individuals and by interest in further extending human control of external systems, researchers from many fields are engaged in this challenging new work. BCI research and development have grown explosively over the past two decades. Efforts have recently begun to provide laboratory-validated BCI systems to severely disabled individuals for real-world applications. In this review, we discuss the current status and future prospects of BCI technology and its clinical applications. We will define BCI, review the BCI-relevant signals from the human brain, and describe the functional components of BCIs. We will also review current clinical applications of BCI technology, and identify potential users and potential applications. Finally, we will discuss current limitations of BCI technology, impediments to its widespread clinical use, and expectations for the future.

Flexibility and practicality graz brain-computer interface approach

"Graz brain-computer interface (BCI)" transforms changes in oscillatory electroencephalogram (EEG) activity into control signals for external devices and feedback. Steady-state evoked potentials (SSEPs) and event-related desynchronization (ERD) are employed to encode user messages. User-specific setup and training are important issues for robust and reliable classification. Furthermore, in order to implement small and thus affordable systems, focus is put on the minimization of the number of EEG sensors. The system also supports the self-paced operation mode, that is, users have on-demand access to the system at any time and can autonomously initiate communication. Flexibility, usability, and practicality are essential to increase user acceptance. Here, we illustrate the possibilities offered by now from EEG-based communication. Results of several studies with able-bodied and disabled individuals performed inside the laboratory and in real-world environments are presented; their characteristics are shown and open issues are mentioned. The applications include the control of neuroprostheses and spelling devices, the interaction with Virtual Reality, and the operation of off-the-shelf software such as Google Earth.