Multi-scale spatiotemporal attention network for neuron based motor imagery EEG classification

BACKGROUND

In recent times, the expeditious expansion of Brain-Computer Interface (BCI) technology in neuroscience, which relies on electroencephalogram (EEG) signals associated with motor imagery, has yielded outcomes that rival conventional approaches, notably due to the triumph of deep learning. Nevertheless, the task of developing and training a comprehensive network to extract the underlying characteristics of motor imagining EEG data continues to pose challenges.

NEW METHOD

This paper presents a multi-scale spatiotemporal self-attention (SA) network model that relies on an attention mechanism. This model aims to classify motor imagination EEG signals into four classes (left hand, right hand, foot, tongue/rest) by considering the temporal and spatial properties of EEG. It is employed to autonomously allocate greater weights to channels linked to motor activity and lesser weights to channels not related to movement, thus choosing the most suitable channels. Neuron utilises parallel multi-scale Temporal Convolutional Network (TCN) layers to extract feature information in the temporal domain at various scales, effectively eliminating temporal domain noise.

RESULTS

The suggested model achieves accuracies of 79.26%, 85.90%, and 96.96% on the BCI competition datasets IV-2a, IV-2b, and HGD, respectively.

COMPARISON WITH EXISTING METHODS

In terms of single-subject classification accuracy, this strategy demonstrates superior performance compared to existing methods.

CONCLUSION

The results indicate that the proposed strategy exhibits favourable performance, resilience, and transfer learning capabilities.

Deciphering Functional Connectivity Differences Between Motor Imagery and Execution of Target-Oriented Grasping

This study aimed to delineate overlapping and distinctive functional connectivity in visual motor imagery, kinesthetic motor imagery, and motor execution of target-oriented grasping action of the right hand. Functional magnetic resonance imaging data were obtained from 18 right-handed healthy individuals during each condition. Seed-based connectivity and multi-voxel pattern analyses were employed after selecting seed regions with the left primary motor cortex and supplementary motor area. There was equivalent seed-based connectivity during the three conditions in the bilateral frontoparietal and temporal areas. When the seed region was the left primary motor cortex, increased connectivity was observed in the left cuneus and superior frontal area during visual and kinesthetic motor imageries, respectively, compared with that during motor execution. Multi-voxel pattern analyses revealed that each condition was differentiated by spatially distributed connectivity patterns of the left primary motor cortex within the right cerebellum VI, cerebellum crus II, and left lingual area. When the seed region was the left supplementary motor area, the connectivity patterns within the right putamen, thalamus, cerebellar areas IV-V, and left superior parietal lobule were significantly classified above chance level across the three conditions. The present findings improve our understanding of the spatial representation of functional connectivity and its specific patterns among motor imagery and motor execution. The strength and fine-grained connectivity patterns of the brain areas can discriminate between motor imagery and motor execution.

Factors and strategies affecting motor imagery ability in people with multiple sclerosis: a systematic review

BACKGROUND

Although growing evidence has shown beneficial effects of motor imagery (MI) training in different populations including people with multiple sclerosis (pwMS), not all patients with neurological diseases may benefit from MI.

OBJECTIVES

To investigate factors and strategies affecting and enhancing MI ability in pwMS.

DATA SOURCES

MEDLINE/PubMed, PsycINFO, Cochrane Library, Scopus, EMBASE, EBSCOhost, Web of Science and REHABDATA databases, clinical trials registries, dissertation repositories, study bibliographies and internet search engines were searched through August 2021.

STUDY SELECTION

Any study type but single case studies investigating factors or strategies contributing to MI ability in pwMS.

STUDY APPRAISAL AND SYNTHESIS METHODS

Risk of bias (RoB) was assessed using the Joanna Briggs Institute Checklist for Case-Control and Analytical Cross-Sectional Studies and Cochrane RoB-2.0 tool for randomised trials. A qualitative synthesis was performed summarising main results.

RESULTS

Eight databases, 4 trial registries, 9 dissertation repositories, and 1 internet search engine were searched. Fourteen studies including 366 pwMS and 236 healthy controls were included. Most frequently, cognitive impairment was reported as a negative factor influencing MI ability in pwMS. Other negative factors were cognitive fatigue and disability. Inconsistent evidence was found on the contribution of MS phenotype, anxiety, and depression. Using a theory-based MI framework and familiarisation to MI and external cueing may enhance MI ability.

LIMITATIONS

Eligible studies were highly heterogeneous.

CONCLUSION AND IMPLICATIONS OF KEY FINDINGS

Cognitive impairment, cognitive fatigue and disability negatively influence MI ability in pwMS. Visual and/or auditory cueing of MI are strategies for facilitating MI ability.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO CRD42020173081 CONTRIBUTION OF THE PAPER.

Action Observation and Motor Imagery administered the day before surgery enhance functional recovery in patients after total hip arthroplasty: A randomized controlled trial

OBJECTIVE

To investigate the effects of Action Observation and Motor Imagery administered the day before surgery on functional recovery in patients after total hip arthroplasty.

DESIGN

Randomised controlled trial.

SETTING

Humanitas Clinical and Research Center, Milan, Italy.

PARTICIPANTS

Eighty inpatients with end-stage hip osteoarthritis undergoing total hip arthroplasty.

INTERVENTIONS

All patients followed a standardized postoperative rehabilitation program. Experimental group (AO + MI) performed two 12-minute Action Observation and Motor Imagery sessions on the preoperative day, whereas control group underwent usual care consisting of education without any additional preoperative activity.

OUTCOME MEASURES

A blinded physiotherapist assessed participants for functional mobility (Timed Up and Go - TUG) (primary outcome), maximum walking speed (10-Meter Walk Test - 10MWT), pain (Numeric Pain Rating Scale - NPRS) and fear of movement (Tampa Scale of Kinesiophobia - TSK) the day before and at four days after surgery.

RESULTS

No between-group differences were found at baseline. Although TUG and 10MWT worsened in both groups (p < 0.001), better TUG was found for AO + MI group at four days (mean difference -5.8 s, 95% confidence interval from -11.3 to -0.3 s, p = 0.039). NPRS (p < 0.001) and TSK (p = 0.036 for AO + MI group, p = 0.003 for control group) improved after surgery without between-group differences.

CONCLUSIONS

Patients undergoing Action Observation and Motor Imagery on the day before surgery showed less functional decline than control group in the first days after total hip arthroplasty. This intervention may contribute to a safer discharge with higher functional abilities in patients hospitalized for total hip arthroplasty.

Comparative Analysis of Spectral and Temporal Combinations in CSP-based Methods for Decoding Hand Motor Imagery Tasks

BACKGROUND

A widely used paradigm for brain-computer interfaces (BCI) is based on the detection of event-related (des)synchronization (ERD/S) in response to hand motor imagery (MI) tasks. The common spatial pattern (CSP) has been recognized as a powerful algorithm to design spatial filters for ERD/ERS detection. However, a limitation of CSP focus on identification only of discriminative spatial information but not the spectral one.

NEW METHOD

An open problem remains in literature related to extracting the most discriminative brain patterns in MI-based BCIs using an optimal time segment and spectral information that accounts for intersubject variability. In recent years, different variants of CSP-based methods have been proposed to address the problem of decoding motor imagery tasks under the intersubject variability of frequency bands related to ERD/ERS events, including Filter Bank Common Spatial Patterns (FBCSP) and Filter Bank Common Spatio-Spectral Patterns (FBCSSP).

COMPARISON WITH EXISTING METHODS

We performed a comparative study of different combinations of time segments and filter banks for three methods (CSP, FBCSP, and FBCSSP) to decode hand (right and left) motor imagery tasks using two different EEG datasets (Gigascience and BCI IVa competition).

RESULTS

The best configuration corresponds to a filter bank with 3 filters (8-15 Hz, 15-22 Hz and 22-29 Hz) using a time window of 1.5 seconds after the trigger, which provide accuracies of approximately 74% and an estimated ITRs of approximately 7 bits/min.

CONCLUSION

Discriminative information in time and spectral domains could be obtained using a convenient filter bank and a time segment configuration, to enhance the classification rate and ITR for detection of hand motor imagery tasks with CSP-related methods, to be used in the implementation of a real-time BCI system.

EEG characteristic investigation of the sixth-finger motor imagery and optimal channel selection for classification

OBJECTIVE

Supernumerary Robotic Limbs (SRL) are body augmentation robotic devices by adding extra limbs or fingers to the human body different from the traditional wearable robotic devices such as prosthesis and exoskeleton. We proposed a novel MI (Motor imagery)-based BCI paradigm based on the sixth-finger which imagines controlling the extra finger movements. The goal of this work is to investigate the EEG characteristics and the application potential of MI-based BCI systems based on the new imagination paradigm (the sixth finger MI).

APPROACH

14 subjects participated in the experiment involving the sixth finger MI tasks and rest state. Event-related spectral perturbation (ERSP) was adopted to analyse EEG spatial features and key-channel time-frequency features. Common spatial patterns (CSP) were used for feature extraction and classification was implemented by support vector machine (SVM). A genetic algorithm (GA) was used to select combinations of EEG channels that maximized classification accuracy and verified EEG patterns based on the sixth finger MI. And we conducted a longitudinal 4-week EEG control experiment based on the new paradigm.

MAIN RESULTS

ERD (event-related desynchronization) was found in the supplementary motor area (SMA) and primary motor area (M1) with a faint contralateral dominance. Unlike traditional MI based on the human hand, ERD was also found in frontal lobe. GA results showed that the distribution of the optimal 8-channel is similar to EEG topographical distributions, nearing parietal and frontal lobe. And the classification accuracy based on the optimal 8-channel (the highest accuracy of 80% and mean accuracy of 70%) was significantly better than that based on the random 8-channel (p<0.01).

SIGNIFICANCE

This work provided a new paradigm for MI-based MI system and verified its feasibility, widened the control bandwidth of the BCI system.

FB-CGANet: filter bank Channel Group Attention network for multi-class motor imagery classification

OBJECTIVE

Motor imagery-based brain computer interface (MI-BCI) is one of the most important BCI paradigms and can identify the target limb of subjects from the feature of MI-based Electroencephalography (EEG) signals. Deep learning methods, especially lightweight neural networks, provide an efficient technique for MI decoding, but the performance of lightweight neural networks is still limited and need further improving. This paper aimed to design a novel lightweight neural network for improving the performance of multi-class MI decoding.

APPROACH

A hybrid filter bank structure that can extract information in both time and frequency domain was proposed and combined with a novel channel attention method Channel Group Attention (CGA) to build a lightweight neural network Filter Bank Channel Group Attention Network (FB-CGANet). Accompanied with FB-CGANet, the Band Exchange data augmentation method was proposed to generate training data for networks with filter bank structure.

MAIN RESULTS

The proposed method can achieve higher 4-class average accuracy (79.4%) than compared methods on the BCI Competition IV IIa dataset in the experiment on the unseen evaluation data. Also, higher average accuracy (93.5%) than compared methods can be obtained in the cross-validation experiment.

SIGNIFICANCE

This work implies the effectiveness of channel attention and filter bank structure in lightweight neural networks and provides a novel option for multi-class motor imagery classification.

EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification

Classification of EEG-based motor imagery (MI) is a crucial non-invasive application in brain-computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 seconds to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. From the experiment results, it can be inferred that the EEG-Inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

OBJECTIVE

Motor imagery (MI) electroencephalography (EEG) classification is regarded as a promising technology for brain--computer interface (BCI) systems, which help people to communicate with the outside world using neural activities. However, decoding human intent accurately is a challenging task because of its small signal-to-noise ratio and non-stationary characteristics. Methods that directly extract features from raw EEG signals ignores key frequency domain information. One of the challenges in MI classification tasks is finding a way to supplement the frequency domain information ignored by the raw EEG signal.

APPROACH

In this study, we fuse different models using their complementary characteristics to develop a multiscale space-time-frequency feature-guided multitask learning convolutional neural network (CNN) architecture. The proposed method consists of four modules: the space-time feature-based representation module, time-frequency feature-based representation module, multimodal fused feature-guided generation module, and classification module. The proposed framework is based on multitask learning. The four modules are trained using three tasks simultaneously and jointly optimised.

RESULTS

The proposed method is evaluated using three public challenge datasets. Through quantitative analysis, we demonstrate that our proposed method outperforms most state-of-the-art machine learning and deep learning techniques for EEG classification, thereby demonstrating the robustness and effectiveness of our method. Moreover, the proposed method is employed to realize control of robot based on EEG signal, verifying its feasibility in real-time applications.

SIGNIFICANCE

To the best of our knowledge, a deep CNN architecture that fuses different input cases, which have complementary characteristics, has not been applied to BCI tasks. Because of the interaction of the three tasks in the multitask learning architecture, our method can improve the generalisation and accuracy of subject-dependent and subject-independent methods with limited annotated data.

Effects of Adding Motor Imagery to Early Physical Therapy in Patients with Knee Osteoarthritis who Had Received Total Knee Arthroplasty: A Randomized Clinical Trial

OBJECTIVE

To investigate the effects of the inclusion of motor imagery (MI) principles into early physical therapy on pain, disability, pressure pain thresholds (PPTs), and range of motion in the early postsurgical phase after total knee arthroplasty (TKA).

METHODS

A randomized clinical trial including patients with knee osteoarthritis who have received TKA was conducted. Participants were randomized to receive five treatment sessions of either physical therapy with or without MI principles in an early postsurgical phase after a TKA (five days after surgery). Pain intensity (visual analog scale [VAS], 0-100), pain-related disability (short-form Western Ontario McMaster Universities Osteoarthritis Index [WOMAC], 0-32), pressure pain thresholds (PPTs), and knee range of motion were assessed before and after five daily treatment sessions by an assessor blinded to the subject's condition.

RESULTS

Twenty-four participants completed data collection and treatment. The adjusted analysis revealed significant group*time interactions for WOMAC (F = 17.29, P = 0.001, η2 = 0.48) and VAS (F = 14.56, P < 0.001, η2 = 0.45); patients receiving physiotherapy and MI principles experienced greater improvements in pain (Δ -28.0, 95% confidence interval [CI] = -43.0 to -13.0) and pain-related disability (Δ -6.0, 95% CI = -8.3 to -3.7) than those receiving physiotherapy alone. No significant group*time interactions for knee range of motion and PPTs were observed (all, P  > 0.30).

CONCLUSIONS

The application of MI to early physiotherapy was effective for improving pain and disability, but not range of motion or pressure pain sensitivity, in the early postsurgical phase after TKA in people with knee osteoarthritis.

Hypoalgesic Effects of Aerobic and Isometric Motor Imagery and Action Observation Exercises on Asymptomatic Participants: A Randomized Controlled Pilot Trial

OBJECTIVES

The objective was to explore whether action observation (AO) and motor imagery (MI) of aerobic and isometric exercise could induce hypoalgesic responses in asymptomatic individuals compared with placebo observation (PO).

METHODS

A randomized controlled pilot trial was designed. Twenty-four healthy participants (mean age = 21.9 ± 2.1 years) were randomized into three groups: AO+MI (N = 8), AO, (N = 8), and PO (N = 8). All participants performed an actual aerobic running exercise (three series of 90 seconds at 85% of their VO2max and 30 seconds at 65% of their VO2max) and an isometric exercise protocol (isometric squats). A day later, they all performed the mental intervention, observing or imagining exercise execution performed the day before, according to their allocated group. Pressure pain thresholds (PPTs) of the quadriceps and epicondyle regions were assessed at baseline, postintervention, and 15 minutes postintervention.

RESULTS

Analysis of variance revealed statistically significant differences in the group*time interaction for PPT in the quadriceps. The AO group showed a statistically significant increase at postintervention and at 15 minutes postintervention. The AO+MI group obtained a statistically significant increase in the two PPT regions compared with the PO group at Δpre-post. The AO group obtained a greater increase in the PPT in the quadriceps femoris than the PO group at Δpre-post and Δpre-post 15 minutes.

CONCLUSIONS

AO and MI induce hypoalgesic responses compared with PO. AO isolated training showed pain modulation responses in the PPTs of the quadriceps region in young physically active adults. These findings highlight the potential role of brain training in pain management.

Directed EEG neural network analysis by LAPPS (p≤1) Penalized sparse Granger approach

The conventional multivariate Granger Analysis (GA) of directed interactions has been widely applied in brain network construction based on EEG recordings as well as fMRI. Nevertheless, EEG is usually inevitably contaminated by strong noise, which may cause network distortion due to the L2-norm used in GAs for directed network recovery. The Lp (p ≤1) norm has been shown to be more robust to outliers as compared to LASSO and L2-GAs. Motivated to construct the sparse brain networks under strong noise condition, we hereby introduce a new approach for GA analysis, termed LAPPS (Least Absolute LP (0<p<1) Penalized Solution). LAPPS utilizes the L1-loss function for the residual error to alleviate the effect of outliers, and another Lp-penalty term (p=0.5) to obtain the sparse connections while suppressing the spurious linkages in the networks. The simulation results reveal that LAPPS obtained the best performance under various noise conditions. In a real EEG data test when subjects performed the left and right hand Motor Imagery (MI) for brain network estimation, LAPPS also obtained a sparse network pattern with the hub at the contralateral brain primary motor areas consistent with the physiological basis of MI.

A Study on the Effect of Electrical Stimulation as a User Stimuli for Motor Imagery Classification in Brain-Machine Interface

Functional Electrical Stimulation (FES) provides a neuroprosthetic interface to non-recovered muscle groups by stimulating the affected region of the human body. FES in combination with Brain-machine interfacing (BMI) has a wide scope in rehabilitation because this system directly links the cerebral motor intention of the users with its corresponding peripheral muscle activations. In this paper, we examine the effect of FES on the electroencephalography (EEG) during motor imagery (left- and right-hand movement) training of the users. Results suggest a significant improvement in the classification accuracy when the subject was induced with FES stimuli as compared to the standard visual one.