1
|
Tang X, Zhang W, Wang H, Wang T, Tan C, Zou M, Xu Z. Dynamic pruning group equivariant network for motor imagery EEG recognition. Front Bioeng Biotechnol 2023; 11:917328. [PMID: 37324415 PMCID: PMC10267707 DOI: 10.3389/fbioe.2023.917328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 04/26/2023] [Indexed: 06/17/2023] Open
Abstract
Introduction: The decoding of the motor imaging electroencephalogram (MI-EEG) is the most critical part of the brain-computer interface (BCI) system. However, the inherent complexity of EEG signals makes it challenging to analyze and model them. Methods: In order to effectively extract and classify the features of EEG signals, a classification algorithm of motor imagery EEG signals based on dynamic pruning equal-variant group convolutional network is proposed. Group convolutional networks can learn powerful representations based on symmetric patterns, but they lack clear methods to learn meaningful relationships between them. The dynamic pruning equivariant group convolution proposed in this paper is used to enhance meaningful symmetric combinations and suppress unreasonable and misleading symmetric combinations. At the same time, a new dynamic pruning method is proposed to dynamically evaluate the importance of parameters, which can restore the pruned connections. Results and Discussion: The experimental results show that the pruning group equivariant convolution network is superior to the traditional benchmark method in the benchmark motor imagery EEG data set. This research can also be transferred to other research areas.
Collapse
Affiliation(s)
- Xianlun Tang
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Wei Zhang
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Huiming Wang
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Tianzhu Wang
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Cong Tan
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Mi Zou
- Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Zihui Xu
- Xinqiao Hospital, Army Medical University, Chongqing, China
| |
Collapse
|
2
|
Wang M, Zhou H, Li X, Chen S, Gao D, Zhang Y. Motor imagery classification method based on relative wavelet packet entropy brain network and improved lasso. Front Neurosci 2023; 17:1113593. [PMID: 36816135 PMCID: PMC9936148 DOI: 10.3389/fnins.2023.1113593] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 01/17/2023] [Indexed: 02/05/2023] Open
Abstract
Motor imagery (MI) electroencephalogram (EEG) signals have a low signal-to-noise ratio, which brings challenges in feature extraction and feature selection with high classification accuracy. In this study, we proposed an approach that combined an improved lasso with relief-f to extract the wavelet packet entropy features and the topological features of the brain function network. For signal denoising and channel filtering, raw MI EEG was filtered based on an R2 map, and then the wavelet soft threshold and one-to-one multi-class score common spatial pattern algorithms were used. Subsequently, the relative wavelet packet entropy and corresponding topological features of the brain network were extracted. After feature fusion, mutcorLasso and the relief-f method were applied for feature selection, followed by three classifiers and an ensemble classifier, respectively. The experiments were conducted on two public EEG datasets (BCI Competition III dataset IIIa and BCI Competition IV dataset IIa) to verify this proposed method. The results showed that the brain network topology features and feature selection methods can retain the information of EEG more effectively and reduce the computational complexity, and the average classification accuracy for both public datasets was above 90%; hence, this algorithms is suitable in MI-BCI and has potential applications in rehabilitation and other fields.
Collapse
Affiliation(s)
- Manqing Wang
- School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, China,School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Hui Zhou
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Xin Li
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Siyu Chen
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Dongrui Gao
- School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, China,School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Yongqing Zhang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China,*Correspondence: Yongqing Zhang ✉
| |
Collapse
|
3
|
Ji Y, Dong SY. Deep learning-based self-induced emotion recognition using EEG. Front Neurosci 2022; 16:985709. [PMID: 36188460 PMCID: PMC9523358 DOI: 10.3389/fnins.2022.985709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 08/29/2022] [Indexed: 11/13/2022] Open
Abstract
Emotion recognition from electroencephalogram (EEG) signals requires accurate and efficient signal processing and feature extraction. Deep learning technology has enabled the automatic extraction of raw EEG signal features that contribute to classifying emotions more accurately. Despite such advances, classification of emotions from EEG signals, especially recorded during recalling specific memories or imagining emotional situations has not yet been investigated. In addition, high-density EEG signal classification using deep neural networks faces challenges, such as high computational complexity, redundant channels, and low accuracy. To address these problems, we evaluate the effects of using a simple channel selection method for classifying self-induced emotions based on deep learning. The experiments demonstrate that selecting key channels based on signal statistics can reduce the computational complexity by 89% without decreasing the classification accuracy. The channel selection method with the highest accuracy was the kurtosis-based method, which achieved accuracies of 79.03% and 79.36% for the valence and arousal scales, respectively. The experimental results show that the proposed framework outperforms conventional methods, even though it uses fewer channels. Our proposed method can be beneficial for the effective use of EEG signals in practical applications.
Collapse
|
4
|
Korik A, McCreadie K, McShane N, Du Bois N, Khodadadzadeh M, Stow J, McElligott J, Carroll Á, Coyle D. Competing at the Cybathlon championship for people with disabilities: long-term motor imagery brain-computer interface training of a cybathlete who has tetraplegia. J Neuroeng Rehabil 2022; 19:95. [PMID: 36068570 PMCID: PMC9446658 DOI: 10.1186/s12984-022-01073-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 08/29/2022] [Indexed: 11/13/2022] Open
Abstract
Background The brain–computer interface (BCI) race at the Cybathlon championship, for people with disabilities, challenges teams (BCI researchers, developers and pilots with spinal cord injury) to control an avatar on a virtual racetrack without movement. Here we describe the training regime and results of the Ulster University BCI Team pilot who has tetraplegia and was trained to use an electroencephalography (EEG)-based BCI intermittently over 10 years, to compete in three Cybathlon events. Methods A multi-class, multiple binary classifier framework was used to decode three kinesthetically imagined movements (motor imagery of left arm, right arm, and feet), and relaxed state. Three game paradigms were used for training i.e., NeuroSensi, Triad, and Cybathlon Race: BrainDriver. An evaluation of the pilot’s performance is presented for two Cybathlon competition training periods—spanning 20 sessions over 5 weeks prior to the 2019 competition, and 25 sessions over 5 weeks in the run up to the 2020 competition. Results Having participated in BCI training in 2009 and competed in Cybathlon 2016, the experienced pilot achieved high two-class accuracy on all class pairs when training began in 2019 (decoding accuracy > 90%, resulting in efficient NeuroSensi and Triad game control). The BrainDriver performance (i.e., Cybathlon race completion time) improved significantly during the training period, leading up to the competition day, ranging from 274–156 s (255 ± 24 s to 191 ± 14 s mean ± std), over 17 days (10 sessions) in 2019, and from 230–168 s (214 ± 14 s to 181 ± 4 s), over 18 days (13 sessions) in 2020. However, on both competition occasions, towards the race date, the performance deteriorated significantly. Conclusions The training regime and framework applied were highly effective in achieving competitive race completion times. The BCI framework did not cope with significant deviation in electroencephalography (EEG) observed in the sessions occurring shortly before and during the race day. Changes in cognitive state as a result of stress, arousal level, and fatigue, associated with the competition challenge and performance pressure, were likely contributing factors to the non-stationary effects that resulted in the BCI and pilot achieving suboptimal performance on race day. Trial registration not registered Supplementary Information The online version contains supplementary material available at 10.1186/s12984-022-01073-9.
Collapse
Affiliation(s)
- Attila Korik
- Intelligent Systems Research Centre, Ulster University, Derry, UK.
| | - Karl McCreadie
- Intelligent Systems Research Centre, Ulster University, Derry, UK
| | - Niall McShane
- Intelligent Systems Research Centre, Ulster University, Derry, UK
| | - Naomi Du Bois
- Intelligent Systems Research Centre, Ulster University, Derry, UK
| | | | - Jacqui Stow
- National Rehabilitation Hospital of Ireland, Dun Laoghaire, Ireland
| | | | - Áine Carroll
- National Rehabilitation Hospital of Ireland, Dun Laoghaire, Ireland.,University College Dublin, Dublin, Ireland
| | - Damien Coyle
- Intelligent Systems Research Centre, Ulster University, Derry, UK
| |
Collapse
|
5
|
du Bois N, Bigirimana AD, Korik A, Kéthina LG, Rutembesa E, Mutabaruka J, Mutesa L, Prasad G, Jansen S, Coyle DH. Neurofeedback with low-cost, wearable electroencephalography (EEG) reduces symptoms in chronic Post-Traumatic Stress Disorder. J Affect Disord 2021; 295:1319-1334. [PMID: 34706446 DOI: 10.1016/j.jad.2021.08.071] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 07/19/2021] [Accepted: 08/25/2021] [Indexed: 11/30/2022]
Abstract
BACKGROUND The study examines the effectiveness of both neurofeedback and motor-imagery brain-computer interface (BCI) training, which promotes self-regulation of brain activity, using low-cost electroencephalography (EEG)-based wearable neurotechnology outside a clinical setting, as a potential treatment for post-traumatic stress disorder (PTSD) in Rwanda. METHODS Participants received training/treatment sessions along with a pre- and post- intervention clinical assessment, (N = 29; control n = 9, neurofeedback (NF, 7 sessions) n = 10, and motor-imagery (MI, 6 sessions) n = 10). Feedback was presented visually via a videogame. Participants were asked to regulate (NF) or intentionally modulate (MI) brain activity to affect/control the game. RESULTS The NF group demonstrated an increase in resting-state alpha 8-12 Hz bandpower following individual training sessions, termed alpha 'rebound' (Pz channel, p = 0.025, all channels, p = 0.024), consistent with previous research findings. This alpha 'rebound', unobserved in the MI group, produced a clinically relevant reduction in symptom severity in NF group, as revealed in three of seven clinical outcome measures: PCL-5 (p = 0.005), PTSD screen (p = 0.005), and HTQ (p = 0.005). LIMITATIONS Data collection took place in environments that posed difficulties in controlling environmental factors. Nevertheless, this limitation improves ecological validity, as neurotechnology treatments must be deployable outside controlled environments, to be a feasible technological treatment. CONCLUSIONS The study produced the first evidence to support a low-cost, neurotechnological solution for neurofeedback as an effective treatment of PTSD for victims of acute trauma in conflict zones in a developing country.
Collapse
Affiliation(s)
- N du Bois
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - A D Bigirimana
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - A Korik
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - L Gaju Kéthina
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - E Rutembesa
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - J Mutabaruka
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - L Mutesa
- Centre for Human Genetics, School of Medicine and Pharmacy, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - G Prasad
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom
| | - S Jansen
- Department of Clinical Psychology, College of Medicine and Health Sciences, University of Rwanda (UR), Huye, Rwanda
| | - D H Coyle
- Intelligent Systems Research Centre, Ulster University (UU), Magee Campus, NI, United Kingdom.
| |
Collapse
|
6
|
Tuncer T, Dogan S, Subasi A. LEDPatNet19: Automated Emotion Recognition Model based on Nonlinear LED Pattern Feature Extraction Function using EEG Signals. Cogn Neurodyn. [PMID: 35847545 PMCID: PMC9279545 DOI: 10.1007/s11571-021-09748-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Revised: 10/23/2021] [Accepted: 11/03/2021] [Indexed: 11/20/2022] Open
Abstract
Electroencephalography (EEG) signals collected from human brains have generally been used to diagnose diseases. Moreover, EEG signals can be used in several areas such as emotion recognition, driving fatigue detection. This work presents a new emotion recognition model by using EEG signals. The primary aim of this model is to present a highly accurate emotion recognition framework by using both a hand-crafted feature generation and a deep classifier. The presented framework uses a multilevel fused feature generation network. This network has three primary phases, which are tunable Q-factor wavelet transform (TQWT), statistical feature generation, and nonlinear textural feature generation phases. TQWT is applied to the EEG data for decomposing signals into different sub-bands and create a multilevel feature generation network. In the nonlinear feature generation, an S-box of the LED block cipher is utilized to create a pattern, which is named as Led-Pattern. Moreover, statistical feature extraction is processed using the widely used statistical moments. The proposed LED pattern and statistical feature extraction functions are applied to 18 TQWT sub-bands and an original EEG signal. Therefore, the proposed hand-crafted learning model is named LEDPatNet19. To select the most informative features, ReliefF and iterative Chi2 (RFIChi2) feature selector is deployed. The proposed model has been developed on the two EEG emotion datasets, which are GAMEEMO and DREAMER datasets. Our proposed hand-crafted learning network achieved 94.58%, 92.86%, and 94.44% classification accuracies for arousal, dominance, and valance cases of the DREAMER dataset. Furthermore, the best classification accuracy of the proposed model for the GAMEEMO dataset is equal to 99.29%. These results clearly illustrate the success of the proposed LEDPatNet19.
Collapse
|
7
|
Abstract
Hybrid EEG-fNIRS brain-computer interface (HBCI) is widely employed to enhance BCI performance. EEG and fNIRS signals are combined to increase the dimensionality of the information. Time windows are used to select EEG and fNIRS singles synchronously. However, it ignores that specific modal signals have their own characteristics, when the task is stimulated, the information between the modalities will mismatch at the moment, which has a significant impact on the classification performance. Here we propose a novel crossing time windows optimization for mental arithmetic (MA) based BCI. The EEG and fNIRS signals were segmented separately by sliding time windows. Then crossing time windows (CTW) were combined with each one segment from EEG and fNIRS selected independently. Furthermore, EEG and fNIRS features were extracted using Filter Bank Common Spatial Pattern (FBCSP) and statistical methods from each sample. Mutual information was calculated for FBCSP and statistical features to characterize the discrimination of crossing time windows, and the optimal window would be selected based on the largest mutual information. Finally, a sparse structured framework of Fisher Lasso feature selection (FLFS) was designed to select the joint features, and conventional Linear Discriminant Analysis (LDA) was employed to perform classification. We used proposed method for a MA dataset. The classification accuracy of the proposed method is 92.52 ± 5.38% and higher than other methods, which shows the rationality and superiority of the proposed method.
Collapse
Affiliation(s)
- Ming Meng
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Luyang Dai
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Qingshan She
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Yuliang Ma
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Wanzeng Kong
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| |
Collapse
|
8
|
Gonzalez H, George R, Muzaffar S, Acevedo J, Hoppner S, Mayr C, Yoo J, Fitzek F, Elfadel I. Hardware Acceleration of EEG-Based Emotion Classification Systems: A Comprehensive Survey. IEEE Trans Biomed Circuits Syst 2021; 15:412-442. [PMID: 34125683 DOI: 10.1109/tbcas.2021.3089132] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Recent years have witnessed a growing interest in EEG-based wearable classifiers of emotions, which could enable the real-time monitoring of patients suffering from neurological disorders such as Amyotrophic Lateral Sclerosis (ALS), Autism Spectrum Disorder (ASD), or Alzheimer's. The hope is that such wearable emotion classifiers would facilitate the patients' social integration and lead to improved healthcare outcomes for them and their loved ones. Yet in spite of their direct relevance to neuro-medicine, the hardware platforms for emotion classification have yet to fill up some important gaps in their various approaches to emotion classification in a healthcare context. In this paper, we present the first hardware-focused critical review of EEG-based wearable classifiers of emotions and survey their implementation perspectives, their algorithmic foundations, and their feature extraction methodologies. We further provide a neuroscience-based analysis of current hardware accelerators of emotion classifiers and use it to map out several research opportunities, including multi-modal hardware platforms, accelerators with tightly-coupled cores operating robustly in the near/supra-threshold region, and pre-processing libraries for universal EEG-based datasets.
Collapse
|
9
|
Roc A, Pillette L, Mladenovic J, Benaroch C, N'Kaoua B, Jeunet C, Lotte F. A review of user training methods in brain computer interfaces based on mental tasks. J Neural Eng 2020; 18. [PMID: 33181488 DOI: 10.1088/1741-2552/abca17] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 11/12/2020] [Indexed: 12/12/2022]
Abstract
Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.
Collapse
Affiliation(s)
| | | | | | - Camille Benaroch
- Inria Centre de recherche Bordeaux Sud-Ouest, Talence, 33405, FRANCE
| | - Bernard N'Kaoua
- Handicap, Activity, Cognition, Health, Inserm / University of Bordeaux, Talence, FRANCE
| | | | | |
Collapse
|
10
|
Aydin EA. Subject-Specific feature selection for near infrared spectroscopy based brain-computer interfaces. Comput Methods Programs Biomed 2020; 195:105535. [PMID: 32534382 DOI: 10.1016/j.cmpb.2020.105535] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Revised: 04/22/2020] [Accepted: 05/07/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Brain-computer interfaces (BCIs) enable people to control an external device by analyzing the brain's neural activity. Functional near-infrared spectroscopy (fNIRS), which is an emerging optical imaging technique, is frequently used in non-invasive BCIs. Determining the subject-specific features is an important concern in enhancing the classification accuracy as well as reducing the complexity of fNIRS based BCI systems. In this study, the effectiveness of subject-specific feature selection on classification accuracy of fNIRS signals is examined. METHODS In order to determine the subject-specific optimal feature subsets, stepwise regression analysis based on sequential feature selection (SWR-SFS) and ReliefF methods were employed. Feature selection is applied on time-domain features of fNIRS signals such as mean, slope, peak, skewness and kurtosis values of signals. Linear discriminant analysis, k nearest neighborhood and support vector machines are employed to evaluate the performance of the selected feature subsets. The proposed techniques are validated on benchmark motor imagery (MI) and mental arithmetic (MA) based fNIRS datasets collected from 29 healthy subjects. RESULTS Both SWR-SFS and reliefF feature selection methods have significantly improved the classification accuracy. However, the best results (88.67% (HbR) and 86.43% (HbO) for MA dataset and 77.01% (HbR) and 71.32% (HbO) for MI dataset) were achieved using SWR-SFS while feature selection provided extremely high feature reduction rates (89.50% (HbR) and 93.99% (HbO) for MA dataset and 94.04% (HbR) and 97.73% (HbO) for MI dataset). CONCLUSIONS The results of the study indicate that employing feature selection improves both MA and MI-based fNIRS signals classification performance significantly.
Collapse
Affiliation(s)
- Eda Akman Aydin
- Gazi University, Faculty of Technology, Department of Electrical and Electronics Engineering, 06500, Besevler, Ankara, Turkey.
| |
Collapse
|