Motor imagery task classification for brain computer interface applications using spatiotemporal principle component analysis

Enhancing EEG-Based MI-BCIs with Class-Specific and Subject-Specific Features Detected by Neural Manifold Analysis

This paper presents an innovative approach leveraging Neuronal Manifold Analysis of EEG data to identify specific time intervals for feature extraction, effectively capturing both class-specific and subject-specific characteristics. Different pipelines were constructed and employed to extract distinctive features within these intervals, specifically for motor imagery (MI) tasks. The methodology was validated using the Graz Competition IV datasets 2A (four-class) and 2B (two-class) motor imagery classification, demonstrating an improvement in classification accuracy that surpasses state-of-the-art algorithms designed for MI tasks. A multi-dimensional feature space, constructed using NMA, was built to detect intervals that capture these critical characteristics, which led to significantly enhanced classification accuracy, especially for individuals with initially poor classification performance. These findings highlight the robustness of this method and its potential to improve classification performance in EEG-based MI-BCI systems.

Bilinear neural network with 3-D attention for brain decoding of motor imagery movements from the human EEG

Deep learning has achieved great success in areas such as computer vision and natural language processing. In the past, some work used convolutional networks to process EEG signals and reached or exceeded traditional machine learning methods. We propose a novel network structure and call it QNet. It contains a newly designed attention module: 3D-AM, which is used to learn the attention weights of EEG channels, time points, and feature maps. It provides a way to automatically learn the electrode and time selection. QNet uses a dual branch structure to fuse bilinear vectors for classification. It performs four, three, and two classes on the EEG Motor Movement/Imagery Dataset. The average cross-validation accuracy of 65.82%, 74.75%, and 82.88% was obtained, which are 7.24%, 4.93%, and 2.45% outperforms than the state-of-the-art, respectively. The article also visualizes the attention weights learned by QNet and shows its possible application for electrode channel selection.

Neuromagnetic Decoding of Simultaneous Bilateral Hand Movements for Multidimensional Brain-Machine Interfaces

To provide multidimensional control, we describe the first reported decoding of bilateral hand movements by using single-trial magnetoencephalography signals as a new approach to enhance a user's ability to interact with a complex environment through a multidimensional brain-machine interface. Ten healthy participants performed or imagined four types of bilateral hand movements during neuromagnetic measurements. By applying a support vector machine (SVM) method to classify the four movements regarding the sensor data obtained from the sensorimotor area, we found the mean accuracy of a two-class classification using the amplitudes of neuromagnetic fields to be particularly suitable for real-time applications, with accuracies comparable to those obtained in previous studies involving unilateral movement. The sensor data from over the sensorimotor cortex showed discriminative time-series waveforms and time-frequency maps in the bilateral hemispheres according to the four tasks. Furthermore, we used four-class classification algorithms based on the SVM method to decode all types of bilateral movements. Our results provided further proof that the slow components of neuromagnetic fields carry sufficient neural information to classify even bilateral hand movements and demonstrated the potential utility of decoding bilateral movements for engineering purposes such as multidimensional motor control.

Correlation Assisted Strong Uncorrelating Transform Complex Common Spatial Patterns for Spatially Distant Channel Data

The Strong Uncorrelating Transform Complex Common Spatial Patterns (SUTCCSP) algorithm, designed for multichannel data analysis, has a limitation on keeping the correlation information among channels during the simultaneous diagonalization process of the covariance and pseudocovariance matrices. This paper focuses on the importance of preserving the correlation information among multichannel data and proposes the correlation assisted SUTCCSP (CASUT) algorithm to address this issue. The performance of the proposed algorithm was demonstrated by classifying the motor imagery electroencephalogram (EEG) dataset. The features were first extracted using CSP algorithms including the proposed method, and then the random forest classifier was utilized for the classification. Experiments using CASUT yielded an average classification accuracy of 78.10 (%), which significantly outperformed those of original CSP, Complex Common Spatial Patterns (CCSP), and SUTCCSP with p-values less than 0.01, tested by the Wilcoxon signed rank test.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction: a review

Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported.

Motor Imagery Classification Using Mu and Beta Rhythms of EEG with Strong Uncorrelating Transform Based Complex Common Spatial Patterns

Recent studies have demonstrated the disassociation between the mu and beta rhythms of electroencephalogram (EEG) during motor imagery tasks. The proposed algorithm in this paper uses a fully data-driven multivariate empirical mode decomposition (MEMD) in order to obtain the mu and beta rhythms from the nonlinear EEG signals. Then, the strong uncorrelating transform complex common spatial patterns (SUTCCSP) algorithm is applied to the rhythms so that the complex data, constructed with the mu and beta rhythms, becomes uncorrelated and its pseudocovariance provides supplementary power difference information between the two rhythms. The extracted features using SUTCCSP that maximize the interclass variances are classified using various classification algorithms for the separation of the left- and right-hand motor imagery EEG acquired from the Physionet database. This paper shows that the supplementary information of the power difference between mu and beta rhythms obtained using SUTCCSP provides an important feature for the classification of the left- and right-hand motor imagery tasks. In addition, MEMD is proved to be a preferred preprocessing method for the nonlinear and nonstationary EEG signals compared to the conventional IIR filtering. Finally, the random forest classifier yielded a high performance for the classification of the motor imagery tasks.

EEG-based motor imagery classification accuracy improves with gradually increased channel number

The question of how many channels should be sed for classification remains a key issue in the study of Brain-Computer Interface. Several studies have shown that a reduced number of channels can achieve the optimal classification accuracy in the offline analysis of motor imagery paradigm, which does not have real-time feedback as in the online control. However, for the cursor movement control paradigm, it remains unclear as to how many channels should be selected in order to achieve the optimal classification. In the present study, we gradually increased the number of channels, and adopted the time-frequency-spatial synthesized method for left and right motor imagery classification. We compared the effect of increasing channel number in two datasets, an imagery-based cursor movement control dataset and a motor imagery tasks dataset. Our results indicated that for the former dataset, the more channels we used, the higher the accuracy rate was achieved, which is in contrast to the finding in the latter dataset that optimal performance was obtained at a subset number of channels. When gradually increasing the number of channels from 2 to all in the analysis of cursor movement control dataset, the average training and testing accuracies from three subjects improved from 68.7% to 90.4% and 63.7% to 87.7%, respectively.

Multidimensional analysis of the abnormal neural oscillations associated with lexical processing in schizophrenia

The neural mechanisms of language abnormalities, the core symptoms in schizophrenia, remain unclear. In this study, a new experimental paradigm, combining magnetoencephalography (MEG) techniques and machine intelligence methodologies, was designed to gain knowledge about the frequency, brain location, and time of occurrence of the neural oscillations that are associated with lexical processing in schizophrenia. The 248-channel MEG recordings were obtained from 12 patients with schizophrenia and 10 healthy controls, during a lexical processing task, where the patients discriminated correct from incorrect lexical stimuli that were visually presented. Event-related desynchronization/synchronization (ERD/ERS) was computed along the frequency, time, and space dimensions combined, that resulted in a large spectral-spatial-temporal ERD/ERS feature set. Machine intelligence techniques were then applied to select a small subset of oscillation patterns that are abnormal in patients with schizophrenia, according to their discriminating power in patient and control classification. Patients with schizophrenia showed abnormal ERD/ERS patterns during both lexical encoding and post-encoding periods. The top-ranked features were located at the occipital and left frontal-temporal areas, and covered a wide frequency range, including δ (1-4 Hz), α (8-12 Hz), β (12-32 Hz), and γ (32-48 Hz) bands. These top features could discriminate the patient group from the control group with 90.91% high accuracy, which demonstrates significant brain oscillation abnormalities in patients with schizophrenia at the specific frequency, time, and brain location indicated by these top features. As neural oscillation abnormality may be due to the mechanisms of the disease, the spectral, spatial, and temporal content of the discriminating features can offer useful information for helping understand the physiological basis of the language disorder in schizophrenia, as well as the pathology of the disease itself.

A frequency-temporal-spatial method for motor-related electroencephalography pattern recognition by comprehensive feature optimization

Either imagined or actual movements lead to a combination of electroencephalography signals with distinctive frequency, temporal and spatial characteristics, which correspond to various motor-related neural activities. This frequency-temporal-spatial pattern is the key of motor intention decoding which is the basis of brain-computer interfaces by motor imagery. We present a new method for motor-related electroencephalography recognition which comprehensively optimizes the frequency-time-space features in a user-specific way. The recognition work focuses on three points: proper time and frequency domain segmentation, spatial optimization based on common spatial pattern filters and feature importance evaluation. We show that by combining the advantages of these optimizational methods, the proposed algorithm effectively improves motor task classification, and the recognized signal chanracteristics can be used to visualize the motor related electroencephalography patterns under different conditions.

Single-trial analysis of cortical oscillatory activities during voluntary movements using empirical mode decomposition (EMD)-based spatiotemporal approach

This study presents a method based on empirical mode decomposition (EMD) and a spatial template-based matching approach to extract sensorimotor oscillatory activities from multi-channel magnetoencephalographic (MEG) measurements during right index finger lifting. The longitudinal gradiometer of the sensor unit which presents most prominent SEF was selected on which each single-trial recording was decomposed into a set of intrinsic mode functions (IMFs). The correlation between each IMF of the selected channel and raw data on other channels were created and represented as a spatial map. The sensorimotor-related IMFs with corresponding correlational spatial map exhibiting large values on primary sensorimotor area (SMI) were selected via spatial-template matching process. Trial-specific alpha and beta bands were determined in sensorimotor-related oscillatory activities using a two-spectrum comparison between the spectra obtained from baseline period (-4 to -3 s) and movement-onset period (-0.5 to 0.5 s). Sensorimotor-related oscillatory activities were filtered within the trial-specific frequency bands to resolve task-related oscillatory activities. Results demonstrated that the optimal phase and amplitude information were preserved not only for alpha suppression (event-related desynchronization) and beta rebound (event-related synchronization) but also for profound analysis of subtle dynamics across trials. The retention of high SNR in the extracted oscillatory activities allow various methods of source estimation that can be applied to study the intricate brain dynamics of motor control mechanisms. The present study enables the possibility of investigating cortical pathophysiology of movement disorder on a trial-by-trial basis which also permits an effective alternative for participants or patients who can not endure lengthy procedures or are incapable of sustaining long experiments.

Movement-related cortical evoked potentials using four-limb imagery

We compared the electroencephalographic changes during actual and imaginary movements with four limbs and classified optimally the responses during four-limb imagery. Evoked potentials in imagery exhibited lower and delayed peaks compared to actual-movement responses, but activations in the primary and the supplementary motor area were similar. Source-modeling analysis revealed that the motor and the parietal cortex were activated similarly, but several dipole sources were active in the frontal cortex for imagery. We compared thirteen classification methods and a combination of template matching and time-frequency methods showed the highest average of 70% classification rate for all limbs.

Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG

OBJECTIVE

To explore effective combinations of computational methods for the prediction of movement intention preceding the production of self-paced right and left hand movements from single trial scalp electroencephalogram (EEG).

METHODS

Twelve naïve subjects performed self-paced movements consisting of three key strokes with either hand. EEG was recorded from 128 channels. The exploration was performed offline on single trial EEG data. We proposed that a successful computational procedure for classification would consist of spatial filtering, temporal filtering, feature selection, and pattern classification. A systematic investigation was performed with combinations of spatial filtering using principal component analysis (PCA), independent component analysis (ICA), common spatial patterns analysis (CSP), and surface Laplacian derivation (SLD); temporal filtering using power spectral density estimation (PSD) and discrete wavelet transform (DWT); pattern classification using linear Mahalanobis distance classifier (LMD), quadratic Mahalanobis distance classifier (QMD), Bayesian classifier (BSC), multi-layer perceptron neural network (MLP), probabilistic neural network (PNN), and support vector machine (SVM). A robust multivariate feature selection strategy using a genetic algorithm was employed.

RESULTS

The combinations of spatial filtering using ICA and SLD, temporal filtering using PSD and DWT, and classification methods using LMD, QMD, BSC and SVM provided higher performance than those of other combinations. Utilizing one of the better combinations of ICA, PSD and SVM, the discrimination accuracy was as high as 75%. Further feature analysis showed that beta band EEG activity of the channels over right sensorimotor cortex was most appropriate for discrimination of right and left hand movement intention.

CONCLUSIONS

Effective combinations of computational methods provide possible classification of human movement intention from single trial EEG. Such a method could be the basis for a potential brain-computer interface based on human natural movement, which might reduce the requirement of long-term training.

SIGNIFICANCE

Effective combinations of computational methods can classify human movement intention from single trial EEG with reasonable accuracy.

Motor imagery classification by means of source analysis methods

We report our investigation of classification of imagined left and right hand movements by applying source analysis methods. Independent component analysis (ICA) is used as a spatio-temporal filter, then equivalent dipole analysis and cortical current density imaging methods are applied to reconstruct equivalent sources, to aid classification of motor imagery tasks in a human subject. The classification was considered correct if the equivalent source was found over the motor cortex in the corresponding hemisphere. A classification rate of about 80% was achieved in the human subject studied using both the equivalent dipole analysis and the cortical current density imaging analysis.

Classification of motor imagery EEG patterns and their topographic representation

We have developed a single trial motor imagery (MI) classification strategy for the brain computer interface (BCI) applications by using time-frequency synthesis approach to accommodate the individual difference, and using the spatial patterns derived from EEG rhythmic components as the feature description. The EEGs are decomposed into a series of frequency bands, and the instantaneous power is represented by the envelop of oscillatory activity, which formed the spatial patterns for a given electrode montage at a time-frequency grid. Time-frequency weights determined by training process were used to synthesize the contributions at the time-frequency domains. The overall classification accuracies for three selected human subjects performing left or right hand movement imagery tasks, were about 87 percent in the ten-fold cross validation without rejecting trials. The loci of motor imagery activity were shown in the spatial topography of differential mode patterns over the sensorimotor area. The present method promises to provide a useful alternative as a general purpose classification procedure for motor imagery classification.

Beyond mind-reading: multi-voxel pattern analysis of fMRI data

A key challenge for cognitive neuroscience is determining how mental representations map onto patterns of neural activity. Recently, researchers have started to address this question by applying sophisticated pattern-classification algorithms to distributed (multi-voxel) patterns of functional MRI data, with the goal of decoding the information that is represented in the subject's brain at a particular point in time. This multi-voxel pattern analysis (MVPA) approach has led to several impressive feats of mind reading. More importantly, MVPA methods constitute a useful new tool for advancing our understanding of neural information processing. We review how researchers are using MVPA methods to characterize neural coding and information processing in domains ranging from visual perception to memory search.

A wavelet-based time-frequency analysis approach for classification of motor imagery for brain-computer interface applications

Electroencephalogram (EEG) recordings during motor imagery tasks are often used as input signals for brain-computer interfaces (BCIs). The translation of these EEG signals to control signals of a device is based on a good classification of various kinds of imagination. We have developed a wavelet-based time-frequency analysis approach for classifying motor imagery tasks. Time-frequency distributions (TFDs) were constructed based on wavelet decomposition and event-related (de)synchronization patterns were extracted from symmetric electrode pairs. The weighted energy difference of the electrode pairs was then compared to classify the imaginary movement. The present method has been tested in nine human subjects and reached an averaged classification rate of 78%. The simplicity of the present technique suggests that it may provide an alternative method for EEG-based BCI applications.

Classification of motor imagery tasks for brain-computer interface applications by means of two equivalent dipoles analysis

We have developed a novel approach using source analysis for classifying motor imagery tasks. Two-equivalent-dipoles analysis was proposed to aid classification of motor imagery tasks for brain-computer interface (BCI) applications. By solving the electroencephalography (EEG) inverse problem of single trial data, it is found that the source analysis approach can aid classification of motor imagination of left- or right-hand movement without training. In four human subjects, an averaged accuracy of classification of 80% was achieved. The present study suggests the merits and feasibility of applying EEG inverse solutions to BCI applications from noninvasive EEG recordings.

Classifying EEG-based motor imagery tasks by means of time–frequency synthesized spatial patterns

OBJECTIVE

To develop a single trial motor imagery (MI) classification strategy for the brain-computer interface (BCI) applications by using time-frequency synthesis approach to accommodate the individual difference, and using the spatial patterns derived from electroencephalogram (EEG) rhythmic components as the feature description.

METHODS

The EEGs are decomposed into a series of frequency bands, and the instantaneous power is represented by the envelop of oscillatory activity, which forms the spatial patterns for a given electrode montage at a time-frequency grid. Time-frequency weights determined by training process are used to synthesize the contributions from the time-frequency domains.

RESULTS

The present method was tested in nine human subjects performing left or right hand movement imagery tasks. The overall classification accuracies for nine human subjects were about 80% in the 10-fold cross-validation, without rejecting any trials from the dataset. The loci of MI activity were shown in the spatial topography of differential-mode patterns over the sensorimotor area.

CONCLUSIONS

The present method does not contain a priori subject-dependent parameters, and is computationally efficient. The testing results are promising considering the fact that no trials are excluded due to noise or artifact.

SIGNIFICANCE

The present method promises to provide a useful alternative as a general purpose classification procedure for MI classification.

Motor imagery classification by means of source analysis for brain-computer interface applications

We report a pilot study of performing classification of motor imagery for brain-computer interface applications, by means of source analysis of scalp-recorded EEGs. Independent component analysis (ICA) was used as a spatio-temporal filter extracting signal components relevant to left or right motor imagery (MI) tasks. Source analysis methods including equivalent dipole analysis and cortical current density imaging were applied to reconstruct equivalent neural sources corresponding to MI, and classification was performed based on the inverse solutions. The classification was considered correct if the equivalent source was found over the motor cortex in the corresponding hemisphere. A classification rate of about 80% was achieved in the human subject studied using both the equivalent dipole analysis and the cortical current density imaging analysis. The present promising results suggest that the source analysis approach could manifest a clearer picture on the cortical activity, and thus facilitate the classification of MI tasks from scalp EEGs.