1
|
Sun P, De Winne J, Zhang M, Devos P, Botteldooren D. Delayed knowledge transfer: Cross-modal knowledge transfer from delayed stimulus to EEG for continuous attention detection based on spike-represented EEG signals. Neural Netw 2025; 183:107003. [PMID: 39667216 DOI: 10.1016/j.neunet.2024.107003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 11/04/2024] [Accepted: 12/01/2024] [Indexed: 12/14/2024]
Abstract
Decoding visual and auditory stimuli from brain activities, such as electroencephalography (EEG), offers promising advancements for enhancing machine-to-human interaction. However, effectively representing EEG signals remains a significant challenge. In this paper, we introduce a novel Delayed Knowledge Transfer (DKT) framework that employs spiking neurons for attention detection, using our experimental EEG dataset. This framework extracts patterns from audiovisual stimuli to model brain responses in EEG signals, while accounting for inherent response delays. By aligning audiovisual features with EEG signals through a shared embedding space, our approach improves the performance of brain-computer interface (BCI) systems. We also present WithMeAttention, a multimodal dataset designed to facilitate research in continuously distinguishing between target and distractor responses. Our methodology demonstrates a 3% improvement in accuracy on the WithMeAttention dataset compared to a baseline model that decodes EEG signals from scratch. This significant performance increase highlights the effectiveness of our approach Comprehensive analysis across four distinct conditions shows that rhythmic enhancement of visual information can optimize multi-sensory information processing. Notably, the two conditions featuring rhythmic target presentation - with and without accompanying beeps - achieved significantly superior performance compared to other scenarios. Furthermore, the delay distribution observed under different conditions indicates that our delay layer effectively emulates the neural processing delays in response to stimuli.
Collapse
Affiliation(s)
- Pengfei Sun
- WAVES Research Group, Department of Information Technology, Ghent University, Gent, Belgium.
| | - Jorg De Winne
- WAVES Research Group, Department of Information Technology, Ghent University, Gent, Belgium.
| | - Malu Zhang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| | - Paul Devos
- WAVES Research Group, Department of Information Technology, Ghent University, Gent, Belgium.
| | - Dick Botteldooren
- WAVES Research Group, Department of Information Technology, Ghent University, Gent, Belgium.
| |
Collapse
|
2
|
Xu F, Shi W, Lv C, Sun Y, Guo S, Feng C, Zhang Y, Jung TP, Leng J. Enhancing Motor Imagery Classification with Residual Graph Convolutional Networks and Multi-Feature Fusion. Int J Neural Syst 2025; 35:2450069. [PMID: 39560446 DOI: 10.1142/s0129065724500692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2024]
Abstract
Stroke, an abrupt cerebrovascular ailment resulting in brain tissue damage, has prompted the adoption of motor imagery (MI)-based brain-computer interface (BCI) systems in stroke rehabilitation. However, analyzing electroencephalogram (EEG) signals from stroke patients poses challenges. To address the issues of low accuracy and efficiency in EEG classification, particularly involving MI, the study proposes a residual graph convolutional network (M-ResGCN) framework based on the modified S-transform (MST), and introduces the self-attention mechanism into residual graph convolutional network (ResGCN). This study uses MST to extract EEG time-frequency domain features, derives spatial EEG features by calculating the absolute Pearson correlation coefficient (aPcc) between channels, and devises a method to construct the adjacency matrix of the brain network using aPcc to measure the strength of the connection between channels. Experimental results involving 16 stroke patients and 16 healthy subjects demonstrate significant improvements in classification quality and robustness across tests and subjects. The highest classification accuracy reached 94.91% and a Kappa coefficient of 0.8918. The average accuracy and F1 scores from 10 times 10-fold cross-validation are 94.38% and 94.36%, respectively. By validating the feasibility and applicability of brain networks constructed using the aPcc in EEG signal analysis and feature encoding, it was established that the aPcc effectively reflects overall brain activity. The proposed method presents a novel approach to exploring channel relationships in MI-EEG and improving classification performance. It holds promise for real-time applications in MI-based BCI systems.
Collapse
Affiliation(s)
- Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| | - Weiyou Shi
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| | - Chengyan Lv
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| | - Yuan Sun
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| | - Shuai Guo
- Rehabilitation Center, Qilu Hospital of Shandong University, Jinan 250012, P. R. China
| | - Chao Feng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| | - Yang Zhang
- Rehabilitation and Physical Therapy Department, Shandong University of Traditional Chinese Medicine Affiliated Hospital, Jinan 250011, P. R. China
| | - Tzyy-Ping Jung
- Swartz Center for Computational Neuroscience, Institute of Neural Computation, University of California, San Diego, La Jolla, CA, USA
| | - Jiancai Leng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, P. R. China
| |
Collapse
|
3
|
Bao J, Wang G, Wang T, Wu N, Hu S, Lee WH, Lo SL, Yan X, Zheng Y, Wang G. A Feature Fusion Model Based on Temporal Convolutional Network for Automatic Sleep Staging Using Single-Channel EEG. IEEE J Biomed Health Inform 2024; 28:6641-6652. [PMID: 39504300 DOI: 10.1109/jbhi.2024.3457969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2024]
Abstract
Sleep staging is a crucial task in sleep monitoring and diagnosis, but clinical sleep staging is both time-consuming and subjective. In this study, we proposed a novel deep learning algorithm named feature fusion temporal convolutional network (FFTCN) for automatic sleep staging using single-channel EEG data. This algorithm employed a one-dimensional convolutional neural network (1D-CNN) to extract temporal features from raw EEG, and a two-dimensional CNN (2D-CNN) to extract time-frequency features from spectrograms generated through continuous wavelet transform (CWT) at the epoch level. These features were subsequently fused and further fed into a temporal convolutional network (TCN) to classify sleep stages at the sequence level. Moreover, a two-step training strategy was used to enhance the model's performance on an imbalanced dataset. Our proposed method exhibits superior performance in the 5-class classification task for healthy subjects, as evaluated on the SHHS-1, Sleep-EDF-153, and ISRUC-S1 datasets. This work provided a straightforward and promising method for improving the accuracy of automatic sleep staging using only single-channel EEG, and the proposed method exhibited great potential for future applications in professional sleep monitoring, which could effectively alleviate the workload of sleep technicians.
Collapse
|
4
|
Leng J, Yu X, Wang C, Zhao J, Zhu J, Chen X, Zhu Z, Jiang X, Zhao J, Feng C, Yang Q, Li J, Jiang L, Xu F, Zhang Y. Functional connectivity of EEG motor rhythms after spinal cord injury. Cogn Neurodyn 2024; 18:3015-3029. [PMID: 39555294 PMCID: PMC11564577 DOI: 10.1007/s11571-024-10136-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 02/22/2024] [Accepted: 03/22/2024] [Indexed: 11/19/2024] Open
Abstract
Spinal cord injury (SCI), which is the injury of the spinal cord site resulting in motor dysfunction, has prompted the use of motor imagery (MI)-based brain computer interface (BCI) systems for motor function reconstruction. However, analyzing electroencephalogram signals and brain function mechanisms for SCI patients is challenging. This is due to their low signal-to-noise ratio and high variability. We propose using the phase locking value (PLV) to construct the brain network in α and β rhythms for both SCI patients and healthy individuals. This approach aims to analyze the changes in brain network connectivity and brain function mechanisms following SCI. The results show that the connection strength of the α rhythm in the healthy control (HC) group is stronger than that in the SCI group, and the connection strength in the β rhythm of the SCI group is stronger than that in the HC group. Moreover, we extract the PLV with common spatial pattern (PLV-CSP) feature from the MI data of the SCI group. The experimental results for 12 SCI patients include that the peak classification accuracy is 100%, and the average accuracy of the ten-fold cross-verification is 95.6%. Our proposed approach can be used as a potential valuable method for SCI pathological studies and MI-based BCI rehabilitation systems.
Collapse
Affiliation(s)
- Jiancai Leng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Xin Yu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Chongfeng Wang
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Jinzhao Zhao
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Jianqun Zhu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Xinyi Chen
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Zhaoxin Zhu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Xiuquan Jiang
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Jiaqi Zhao
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Chao Feng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Qingbo Yang
- School of Mathematics and Statistics, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Jianfei Li
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Lin Jiang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, No.2006, Xiyuan Road West Hi-Tech District, Chengdu, 611731 Sichuan China
- School of Life Science and Technology, Center for Information in BioMedicine, University of Electronic Science and Technology of China, No.2006, Xiyuan Road West Hi-Tech District, Chengdu, 611731 Sichuan China
| | - Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), No.3501, Daxue Road Changqing District, Jinan, 250353 Shandong China
| | - Yang Zhang
- Rehabilitation and Physical Therapy Department, Shandong University of Traditional Chinese Medicine Affiliated Hospital, No.42, Wenhuaxi Road Lixia District, Jinan, 250012 Shandong China
| |
Collapse
|
5
|
Ieracitano C, Zhang X. Editorial Topical Collection: "Biomedical Imaging and Data Analytics for Disease Diagnosis and Treatment". Bioengineering (Basel) 2024; 11:726. [PMID: 39061808 PMCID: PMC11273676 DOI: 10.3390/bioengineering11070726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Accepted: 07/17/2024] [Indexed: 07/28/2024] Open
Abstract
The integration of biomedical imaging techniques with advanced data analytics is at the forefront of a transformative era in healthcare [...].
Collapse
Affiliation(s)
- Cosimo Ieracitano
- DICEAM Department, University Mediterranea of Reggio Calabria, via Zehender, Feo di Vito, 89122 Reggio Calabria, Italy
| | - Xuejun Zhang
- School of Computer, Electronics and Information, Guangxi University, Nanning 530004, China;
| |
Collapse
|
6
|
Ahmadian S, Rostami M, Farrahi V, Oussalah M. A novel physical activity recognition approach using deep ensemble optimized transformers and reinforcement learning. Neural Netw 2024; 173:106159. [PMID: 38342080 DOI: 10.1016/j.neunet.2024.106159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Revised: 12/02/2023] [Accepted: 02/01/2024] [Indexed: 02/13/2024]
Abstract
In recent years, human physical activity recognition has increasingly attracted attention from different research fields such as healthcare, computer-human interaction, lifestyle monitoring, and athletics. Deep learning models have been extensively employed in developing physical activity recognition systems. To improve these models, their hyperparameters need to be initialized with optimal values. However, tuning these hyperparameters manually is time-consuming and may lead to inaccurate results. Moreover, the application of these models to different data resources and the integration of their results into the overall data processing pipeline are challenging issues in physical activity recognition systems. In this paper, we propose a novel ensemble method for physical activity recognition based on a deep transformer-based time-series classification model that uses heart rate, speed, and distance time-series data to recognize physical activities. In particular, we develop a modified arithmetic optimization algorithm to automatically adjust the optimal values of the classification models' hyperparameters. Moreover, a reinforcement learning-based ensemble approach is proposed to optimally integrate the results of the classification models obtained using heart rate, speed, and distance time-series data and, subsequently, recognize the physical activities. Experiments performed on a real-world dataset demonstrated that the proposed method achieves promising efficiency in comparison to other state-of-the-art models. More specifically, the proposed method increases the performance compared to the second-best performer by around 3.44 %, 9.45 %, 5.43 %, 2.54 %, and 7.53 % based on accuracy, precision, recall, specificity, and F1-score evaluation metrics, respectively.
Collapse
Affiliation(s)
- Sajad Ahmadian
- Faculty of Information Technology, Kermanshah University of Technology, Kermanshah, Iran.
| | - Mehrdad Rostami
- Center for Machine Vision and Signal Analysis (CMVS), Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
| | - Vahid Farrahi
- Center for Machine Vision and Signal Analysis (CMVS), Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; Institute for Sport and Sport Science, TU Dortmund University, Dortmund, Germany; Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland
| | - Mourad Oussalah
- Center for Machine Vision and Signal Analysis (CMVS), Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland
| |
Collapse
|
7
|
Jiang M, Chaichanasittikarn O, Seet M, Ng D, Vyas R, Saini G, Dragomir A. Modulating Driver Alertness via Ambient Olfactory Stimulation: A Wearable Electroencephalography Study. SENSORS (BASEL, SWITZERLAND) 2024; 24:1203. [PMID: 38400361 PMCID: PMC10892239 DOI: 10.3390/s24041203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 01/31/2024] [Accepted: 02/07/2024] [Indexed: 02/25/2024]
Abstract
Poor alertness levels and related changes in cognitive efficiency are common when performing monotonous tasks such as extended driving. Recent studies have investigated driver alertness decrement and possible strategies for modulating alertness with the goal of improving reaction times to safety critical events. However, most studies rely on subjective measures in assessing alertness changes, while the use of olfactory stimuli, which are known to be strong modulators of cognitive states, has not been commensurately explored in driving alertness settings. To address this gap, in the present study we investigated the effectiveness of olfactory stimuli in modulating the alertness state of drivers and explored the utility of electroencephalography (EEG) in developing objective brain-based tools for assessing the resulting changes in cortical activity. Olfactory stimulation induced a significant differential effect on braking reaction time. The corresponding effect to the cortical activity was characterized using EEG-derived metrics and the devised machine learning framework yielded a high discriminating accuracy (92.1%). Furthermore, neural activity in the alpha frequency band was found to be significantly associated with the observed drivers' behavioral changes. Overall, our results demonstrate the potential of olfactory stimuli to modulate the alertness state and the efficiency of EEG in objectively assessing the resulting cognitive changes.
Collapse
Affiliation(s)
- Mengting Jiang
- N.1 Institute for Health, National University of Singapore, 28 Medical Drive, #05-COR, Singapore 117456, Singapore
- Laboratoire des Systèmes Perceptifs, Département d’Études Cognitives, École Normale Supérieure, PSL University, CNRS, 75005 Paris, France
| | - Oranatt Chaichanasittikarn
- N.1 Institute for Health, National University of Singapore, 28 Medical Drive, #05-COR, Singapore 117456, Singapore
| | - Manuel Seet
- N.1 Institute for Health, National University of Singapore, 28 Medical Drive, #05-COR, Singapore 117456, Singapore
| | - Desmond Ng
- International Operations, Procter & Gamble, 70 Biopolis Street, Singapore 138547, Singapore
| | - Rahul Vyas
- International Operations, Procter & Gamble, 70 Biopolis Street, Singapore 138547, Singapore
| | - Gaurav Saini
- International Operations, Procter & Gamble, 70 Biopolis Street, Singapore 138547, Singapore
| | - Andrei Dragomir
- N.1 Institute for Health, National University of Singapore, 28 Medical Drive, #05-COR, Singapore 117456, Singapore
| |
Collapse
|
8
|
Mammone N, Ieracitano C, Spataro R, Guger C, Cho W, Morabito FC. A Few-Shot Transfer Learning Approach for Motion Intention Decoding from Electroencephalographic Signals. Int J Neural Syst 2024; 34:2350068. [PMID: 38073546 DOI: 10.1142/s0129065723500685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2024]
Abstract
In this study, a few-shot transfer learning approach was introduced to decode movement intention from electroencephalographic (EEG) signals, allowing to recognize new tasks with minimal adaptation. To this end, a dataset of EEG signals recorded during the preparation of complex sub-movements was created from a publicly available data collection. The dataset was divided into two parts: the source domain dataset (including 5 classes) and the support (target domain) dataset, (including 2 classes) with no overlap between the two datasets in terms of classes. The proposed methodology consists in projecting EEG signals into the space-frequency-time domain, in processing such projections (rearranged in channels × frequency frames) by means of a custom EEG-based deep neural network (denoted as EEGframeNET5), and then adapting the system to recognize new tasks through a few-shot transfer learning approach. The proposed method achieved an average accuracy of 72.45 ± 4.19% in the 5-way classification of samples from the source domain dataset, outperforming comparable studies in the literature. In the second phase of the study, a few-shot transfer learning approach was proposed to adapt the neural system and make it able to recognize new tasks in the support dataset. The results demonstrated the system's ability to adapt and recognize new tasks with an average accuracy of 80 ± 0.12% in discriminating hand opening/closing preparation and outperforming reported results in the literature. This study suggests the effectiveness of EEG in capturing information related to the motor preparation of complex movements, potentially paving the way for BCI systems based on motion planning decoding. The proposed methodology could be straightforwardly extended to advanced EEG signal processing in other scenarios, such as motor imagery or neural disorder classification.
Collapse
Affiliation(s)
- Nadia Mammone
- DICEAM, University Mediterranea of Reggio Calabria Via Zehender, Loc. Feo di Vito, Reggio Calabria, 89122, Italy
| | - Cosimo Ieracitano
- DICEAM, University Mediterranea of Reggio Calabria Via Zehender, Loc. Feo di Vito, Reggio Calabria, 89122, Italy
| | - Rossella Spataro
- ALS Clinical Research Center, BiND, University of Palermo, Palermo, Italy
- Intensive Rehabilitation Unit, Villa delle Ginestre Hospital, Palermo, Italy
| | | | - Woosang Cho
- g.tec Medical Engineering GmbH, 4521, Schiedlberg, Austria
| | - Francesco Carlo Morabito
- DICEAM, University Mediterranea of Reggio Calabria Via Zehender, Loc. Feo di Vito, Reggio Calabria, 89122, Italy
| |
Collapse
|
9
|
Rafiei MH, Gauthier LV, Adeli H, Takabi D. Self-Supervised Learning for Electroencephalography. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:1457-1471. [PMID: 35867362 DOI: 10.1109/tnnls.2022.3190448] [Citation(s) in RCA: 65] [Impact Index Per Article: 65.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Decades of research have shown machine learning superiority in discovering highly nonlinear patterns embedded in electroencephalography (EEG) records compared with conventional statistical techniques. However, even the most advanced machine learning techniques require relatively large, labeled EEG repositories. EEG data collection and labeling are costly. Moreover, combining available datasets to achieve a large data volume is usually infeasible due to inconsistent experimental paradigms across trials. Self-supervised learning (SSL) solves these challenges because it enables learning from EEG records across trials with variable experimental paradigms, even when the trials explore different phenomena. It aggregates multiple EEG repositories to increase accuracy, reduce bias, and mitigate overfitting in machine learning training. In addition, SSL could be employed in situations where there is limited labeled training data, and manual labeling is costly. This article: 1) provides a brief introduction to SSL; 2) describes some SSL techniques employed in recent studies, including EEG; 3) proposes current and potential SSL techniques for future investigations in EEG studies; 4) discusses the cons and pros of different SSL techniques; and 5) proposes holistic implementation tips and potential future directions for EEG SSL practices.
Collapse
|
10
|
Nogay HS, Adeli H. Multiple Classification of Brain MRI Autism Spectrum Disorder by Age and Gender Using Deep Learning. J Med Syst 2024; 48:15. [PMID: 38252192 PMCID: PMC10803393 DOI: 10.1007/s10916-023-02032-0] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/31/2023] [Indexed: 01/23/2024]
Abstract
The fact that the rapid and definitive diagnosis of autism cannot be made today and that autism cannot be treated provides an impetus to look into novel technological solutions. To contribute to the resolution of this problem through multiple classifications by considering age and gender factors, in this study, two quadruple and one octal classifications were performed using a deep learning (DL) approach. Gender in one of the four classifications and age groups in the other were considered. In the octal classification, classes were created considering gender and age groups. In addition to the diagnosis of ASD (Autism Spectrum Disorders), another goal of this study is to find out the contribution of gender and age factors to the diagnosis of ASD by making multiple classifications based on age and gender for the first time. Brain structural MRI (sMRI) scans of participators with ASD and TD (Typical Development) were pre-processed in the system originally designed for this purpose. Using the Canny Edge Detection (CED) algorithm, the sMRI image data was cropped in the data pre-processing stage, and the data set was enlarged five times with the data augmentation (DA) techniques. The most optimal convolutional neural network (CNN) models were developed using the grid search optimization (GSO) algorism. The proposed DL prediction system was tested with the five-fold cross-validation technique. Three CNN models were designed to be used in the system. The first of these models is the quadruple classification model created by taking gender into account (model 1), the second is the quadruple classification model created by taking into account age (model 2), and the third is the eightfold classification model created by taking into account both gender and age (model 3). ). The accuracy rates obtained for all three designed models are 80.94, 85.42 and 67.94, respectively. These obtained accuracy rates were compared with pre-trained models by using the transfer learning approach. As a result, it was revealed that age and gender factors were effective in the diagnosis of ASD with the system developed for ASD multiple classifications, and higher accuracy rates were achieved compared to pre-trained models.
Collapse
Affiliation(s)
- Hidir Selcuk Nogay
- Electrical and Energy Department, Bursa Uludag University, Bursa, Turkey
| | - Hojjat Adeli
- Departments of Biomedical Informatics and Neuroscience, College of Medicine, The Ohio State University Neurology, 370 W. 9th Avenue, Columbus, OH, 43210, USA.
| |
Collapse
|
11
|
Ganjali M, Mehridehnavi A, Rakhshani S, Khorasani A. Unsupervised Neural Manifold Alignment for Stable Decoding of Movement from Cortical Signals. Int J Neural Syst 2024; 34:2450006. [PMID: 38063378 DOI: 10.1142/s0129065724500060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2023]
Abstract
The stable decoding of movement parameters using neural activity is crucial for the success of brain-machine interfaces (BMIs). However, neural activity can be unstable over time, leading to changes in the parameters used for decoding movement, which can hinder accurate movement decoding. To tackle this issue, one approach is to transfer neural activity to a stable, low-dimensional manifold using dimensionality reduction techniques and align manifolds across sessions by maximizing correlations of the manifolds. However, the practical use of manifold stabilization techniques requires knowledge of the true subject intentions such as target direction or behavioral state. To overcome this limitation, an automatic unsupervised algorithm is proposed that determines movement target intention before manifold alignment in the presence of manifold rotation and scaling across sessions. This unsupervised algorithm is combined with a dimensionality reduction and alignment method to overcome decoder instabilities. The effectiveness of the BMI stabilizer method is represented by decoding the two-dimensional (2D) hand velocity of two rhesus macaque monkeys during a center-out-reaching movement task. The performance of the proposed method is evaluated using correlation coefficient and R-squared measures, demonstrating higher decoding performance compared to a state-of-the-art unsupervised BMI stabilizer. The results offer benefits for the automatic determination of movement intents in long-term BMI decoding. Overall, the proposed method offers a promising automatic solution for achieving stable and accurate movement decoding in BMI applications.
Collapse
Affiliation(s)
- Mohammadali Ganjali
- Department of Biomedical Engineering, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Alireza Mehridehnavi
- Department of Biomedical Engineering, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Sajed Rakhshani
- Medical Image and Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Abed Khorasani
- Department of Neurology, Northwestern University, Chicago, IL, 60611, USA
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran
| |
Collapse
|
12
|
Hascher S, Shuster A, Mukamel R, Ossmy O. The power of multivariate approach in identifying EEG correlates of interlimb coupling. Front Hum Neurosci 2023; 17:1256497. [PMID: 37900731 PMCID: PMC10603300 DOI: 10.3389/fnhum.2023.1256497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 09/07/2023] [Indexed: 10/31/2023] Open
Abstract
Interlimb coupling refers to the interaction between movements of one limb and movements of other limbs. Understanding mechanisms underlying this effect is important to real life because it reflects the level of interdependence between the limbs that plays a role in daily activities including tool use, cooking, or playing musical instruments. Interlimb coupling involves multiple brain regions working together, including coordination of neural activity in sensory and motor regions across the two hemispheres. Traditional neuroscience research took a univariate approach to identify neural features that correspond to behavioural coupling measures. Yet, this approach reduces the complexity of the neural activity during interlimb tasks to one value. In this brief research report, we argue that identifying neural correlates of interlimb coupling would benefit from a multivariate approach in which full patterns from multiple sources are used to predict behavioural coupling. We demonstrate the feasibility of this approach in an exploratory EEG study where participants (n = 10) completed 240 trials of a well-established drawing paradigm that involves interlimb coupling. Using artificial neural network (ANN), we show that multivariate representation of the EEG signal significantly captures the interlimb coupling during bimanual drawing whereas univariate analyses failed to identify such correlates. Our findings demonstrate that analysing distributed patterns of multiple EEG channels is more sensitive than single-value techniques in uncovering subtle differences between multiple neural signals. Using such techniques can improve identification of neural correlates of complex motor behaviours.
Collapse
Affiliation(s)
- Sophie Hascher
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| | - Anastasia Shuster
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| | - Roy Mukamel
- Sagol School of Neuroscience and School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Ori Ossmy
- Centre for Brain and Cognitive Development, School of Psychological Sciences, Birkbeck, University of London, London, United Kingdom
| |
Collapse
|
13
|
Zhang R, Liu G, Wen Y, Zhou W. Self-attention-based convolutional neural network and time-frequency common spatial pattern for enhanced motor imagery classification. J Neurosci Methods 2023; 398:109953. [PMID: 37611877 DOI: 10.1016/j.jneumeth.2023.109953] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Revised: 07/20/2023] [Accepted: 08/19/2023] [Indexed: 08/25/2023]
Abstract
BACKGROUND Motor imagery (MI) based brain-computer interfaces (BCIs) have promising potentials in the field of neuro-rehabilitation. However, due to individual variations in active brain regions during MI tasks, the challenge of decoding MI EEG signals necessitates improved classification performance for practical application. NEW METHOD This study proposes a self-attention-based Convolutional Neural Network (CNN) in conjunction with a time-frequency common spatial pattern (TFCSP) for enhanced MI classification. Due to the limited availability of training data, a data augmentation strategy is employed to expand the scale of MI EEG datasets. The self-attention-based CNN is trained to automatically extract the temporal and spatial information from EEG signals, allowing the self-attention module to select active channels by calculating EEG channel weights. TFCSP is further implemented to extract multiscale time-frequency-space features from EEG data. Finally, the EEG features derived from TFCSP are concatenated with those from the self-attention-based CNN for MI classification. RESULTS The proposed method is evaluated on two publicly accessible datasets, BCI Competition IV IIa and BCI Competition III IIIa, yielding mean accuracies of 79.28 % and 86.39 %, respectively. CONCLUSIONS Compared with state-of-the-art methods, our approach achieves superior classification results in accuracy. Self-attention-based CNN combining with TFCSP can make full use of the time-frequency-space information of EEG, and enhance the classification performance.
Collapse
Affiliation(s)
- Rui Zhang
- School of Microelectronics, Shandong University, Jinan 250100, China
| | - Guoyang Liu
- School of Microelectronics, Shandong University, Jinan 250100, China
| | - Yiming Wen
- School of Microelectronics, Shandong University, Jinan 250100, China
| | - Weidong Zhou
- School of Microelectronics, Shandong University, Jinan 250100, China.
| |
Collapse
|
14
|
Kwak Y, Kong K, Song WJ, Kim SE. Subject-Invariant Deep Neural Networks Based on Baseline Correction for EEG Motor Imagery BCI. IEEE J Biomed Health Inform 2023; 27:1801-1812. [PMID: 37022076 DOI: 10.1109/jbhi.2023.3238421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Electroencephalography (EEG)-based brain-computer interface (BCI) systems have been extensively used in various applications, such as communication, control, and rehabilitation. However, individual anatomical and physiological differences cause subject-specific variability of EEG signals for the same task, and BCI systems thus require a calibration procedure that adjusts system parameters to each subject. To overcome this problem, we propose a subject-invariant deep neural network (DNN) using baseline-EEG signals that can be recorded from subjects resting in comfortable states. We first modeled the deep features of EEG signals as a decomposition of subject-invariant and subject-variant features corrupted by anatomical/physiological characteristics. Subject-variant features were then removed from the deep features by learning the network with a baseline correction module (BCM) using the underlying individual information in baseline-EEG signals. The subject-invariant loss forces the BCM to assemble subject-invariant features that have the same class, irrespective of the subject. Using 1-min baseline-EEG signals of the new subject, our algorithm can eliminate subject-variant components from test data without the calibration process. The experimental results show that our subject-invariant DNN framework significantly increases decoding accuracies of the conventional DNN methods for BCI systems. Furthermore, feature visualizations illustrate that the proposed BCM extracts subject-invariant features that are close to each other in the same class.
Collapse
|
15
|
Oikonomou VP, Georgiadis K, Kalaganis F, Nikolopoulos S, Kompatsiaris I. A Sparse Representation Classification Scheme for the Recognition of Affective and Cognitive Brain Processes in Neuromarketing. SENSORS (BASEL, SWITZERLAND) 2023; 23:2480. [PMID: 36904683 PMCID: PMC10007402 DOI: 10.3390/s23052480] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 02/09/2023] [Accepted: 02/20/2023] [Indexed: 06/18/2023]
Abstract
In this work, we propose a novel framework to recognize the cognitive and affective processes of the brain during neuromarketing-based stimuli using EEG signals. The most crucial component of our approach is the proposed classification algorithm that is based on a sparse representation classification scheme. The basic assumption of our approach is that EEG features from a cognitive or affective process lie on a linear subspace. Hence, a test brain signal can be represented as a linear (or weighted) combination of brain signals from all classes in the training set. The class membership of the brain signals is determined by adopting the Sparse Bayesian Framework with graph-based priors over the weights of linear combination. Furthermore, the classification rule is constructed by using the residuals of linear combination. The experiments on a publicly available neuromarketing EEG dataset demonstrate the usefulness of our approach. For the two classification tasks offered by the employed dataset, namely affective state recognition and cognitive state recognition, the proposed classification scheme manages to achieve a higher classification accuracy compared to the baseline and state-of-the art methods (more than 8% improvement in classification accuracy).
Collapse
Affiliation(s)
- Vangelis P. Oikonomou
- Information Technologies Institute, Centre for Research and Technology Hellas, CERTH-ITI, 6th km Charilaou-Thermi Road, 57001 Thessaloniki, Greece
| | | | | | | | | |
Collapse
|
16
|
Hossain KM, Islam MA, Hossain S, Nijholt A, Ahad MAR. Status of deep learning for EEG-based brain-computer interface applications. Front Comput Neurosci 2023; 16:1006763. [PMID: 36726556 PMCID: PMC9885375 DOI: 10.3389/fncom.2022.1006763] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 12/23/2022] [Indexed: 01/18/2023] Open
Abstract
In the previous decade, breakthroughs in the central nervous system bioinformatics and computational innovation have prompted significant developments in brain-computer interface (BCI), elevating it to the forefront of applied science and research. BCI revitalization enables neurorehabilitation strategies for physically disabled patients (e.g., disabled patients and hemiplegia) and patients with brain injury (e.g., patients with stroke). Different methods have been developed for electroencephalogram (EEG)-based BCI applications. Due to the lack of a large set of EEG data, methods using matrix factorization and machine learning were the most popular. However, things have changed recently because a number of large, high-quality EEG datasets are now being made public and used in deep learning-based BCI applications. On the other hand, deep learning is demonstrating great prospects for solving complex relevant tasks such as motor imagery classification, epileptic seizure detection, and driver attention recognition using EEG data. Researchers are doing a lot of work on deep learning-based approaches in the BCI field right now. Moreover, there is a great demand for a study that emphasizes only deep learning models for EEG-based BCI applications. Therefore, we introduce this study to the recent proposed deep learning-based approaches in BCI using EEG data (from 2017 to 2022). The main differences, such as merits, drawbacks, and applications are introduced. Furthermore, we point out current challenges and the directions for future studies. We argue that this review study will help the EEG research community in their future research.
Collapse
Affiliation(s)
- Khondoker Murad Hossain
- Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, Baltimore, MD, United States
| | - Md. Ariful Islam
- Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh
| | | | - Anton Nijholt
- Human Media Interaction, University of Twente, Enschede, Netherlands
| | - Md Atiqur Rahman Ahad
- Department of Computer Science and Digital Technology, University of East London, London, United Kingdom,*Correspondence: Md Atiqur Rahman Ahad ✉
| |
Collapse
|
17
|
Ieracitano C, Nicoletti F, Arcuri N, Ruggeri G, Versaci M, Morabito FC, Mammone N. A Deep Cognitive Venetian Blinds System for Automatic Estimation of Slat Orientation. Cognit Comput 2022. [DOI: 10.1007/s12559-022-10054-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
18
|
Xu F, Li J, Dong G, Li J, Chen X, Zhu J, Hu J, Zhang Y, Yue S, Wen D, Leng J. EEG decoding method based on multi-feature information fusion for spinal cord injury. Neural Netw 2022; 156:135-151. [DOI: 10.1016/j.neunet.2022.09.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Revised: 09/13/2022] [Accepted: 09/15/2022] [Indexed: 10/14/2022]
|
19
|
Yang B, Ma J, Qiu W, Zhang J, Wang X. The unilateral upper limb classification from fMRI-weighted EEG signals using convolutional neural network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
20
|
Ieracitano C, Morabito FC, Squartini S, Huang K, Li X, Mahmud M. Guest Editorial: Advances in Deep Learning for Clinical and Healthcare Applications. Cognit Comput 2022:1-3. [PMID: 35991007 PMCID: PMC9382598 DOI: 10.1007/s12559-022-10049-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/08/2022] [Indexed: 11/03/2022]
Affiliation(s)
- Cosimo Ieracitano
- DICEAM, University Mediterranea of Reggio Calabria, Reggio Calabria, Italy
| | | | | | | | - Xuelong Li
- Northwestern Polytechnical University, Xi’an, China
| | | |
Collapse
|
21
|
A novel multi-branch hybrid neural network for motor imagery EEG signal classification. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103718] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
22
|
Chen Y, Zhang D, Karimi HR, Deng C, Yin W. A new deep learning framework based on blood pressure range constraint for continuous cuffless BP estimation. Neural Netw 2022; 152:181-190. [DOI: 10.1016/j.neunet.2022.04.017] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 03/23/2022] [Accepted: 04/14/2022] [Indexed: 11/25/2022]
|
23
|
Sanchez-Reolid R, Martinez-Saez MC, Garcia-Martinez B, Fernandez-Aguilar L, Segura LR, Latorre JM, Fernandez-Caballero A. Emotion Classification from EEG with a Low-Cost BCI Versus a High-End Equipment. Int J Neural Syst 2022; 32:2250041. [DOI: 10.1142/s0129065722500411] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
24
|
Ieracitano C, Mammone N, Versaci M, Varone G, Ali AR, Armentano A, Calabrese G, Ferrarelli A, Turano L, Tebala C, Hussain Z, Sheikh Z, Sheikh A, Sceni G, Hussain A, Morabito FC. A fuzzy-enhanced deep learning approach for early detection of Covid-19 pneumonia from portable chest X-ray images. Neurocomputing 2022; 481:202-215. [PMID: 35079203 PMCID: PMC8776345 DOI: 10.1016/j.neucom.2022.01.055] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Revised: 12/17/2021] [Accepted: 01/14/2022] [Indexed: 12/20/2022]
Abstract
The Covid-19 pandemic is the defining global health crisis of our time. Chest X-Rays (CXR) have been an important imaging modality for assisting in the diagnosis and management of hospitalised Covid-19 patients. However, their interpretation is time intensive for radiologists. Accurate computer aided systems can facilitate early diagnosis of Covid-19 and effective triaging. In this paper, we propose a fuzzy logic based deep learning (DL) approach to differentiate between CXR images of patients with Covid-19 pneumonia and with interstitial pneumonias not related to Covid-19. The developed model here, referred to as CovNNet, is used to extract some relevant features from CXR images, combined with fuzzy images generated by a fuzzy edge detection algorithm. Experimental results show that using a combination of CXR and fuzzy features, within a deep learning approach by developing a deep network inputed to a Multilayer Perceptron (MLP), results in a higher classification performance (accuracy rate up to 81%), compared to benchmark deep learning approaches. The approach has been validated through additional datasets which are continously generated due to the spread of the virus and would help triage patients in acute settings. A permutation analysis is carried out, and a simple occlusion methodology for explaining decisions is also proposed. The proposed pipeline can be easily embedded into present clinical decision support systems.
Collapse
Affiliation(s)
- Cosimo Ieracitano
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella, Feo di Vito, 89124 Reggio Calabria, Italy
| | - Nadia Mammone
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella, Feo di Vito, 89124 Reggio Calabria, Italy
| | - Mario Versaci
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella, Feo di Vito, 89124 Reggio Calabria, Italy
| | - Giuseppe Varone
- Department of Neuroscience & Imaging, Universitá degli Studi G. d'Annunzio Chieti e Pescara, Pescara, Italy
| | - Abder-Rahman Ali
- Department of Computing Science and Mathematics, Faculty of Natural Sciences, University of Stirling, Scotland, UK
| | - Antonio Armentano
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Grazia Calabrese
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Anna Ferrarelli
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Lorena Turano
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Carmela Tebala
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Zain Hussain
- Edinburgh Medical School, College of Medicine and Veterinary Medicine, University of Edinburgh, Scotland, UK
| | - Zakariya Sheikh
- Edinburgh Medical School, College of Medicine and Veterinary Medicine, University of Edinburgh, Scotland, UK
| | - Aziz Sheikh
- Usher Institute, Edinburgh Medical School, University of Edinburgh, Scotland, UK
| | - Giuseppe Sceni
- Advanced Diagnostic and Therapeutic Technology Department, Grande Ospedale Metropolitano (GOM) Bianchi-Melacrino-Morelli of Reggio Calabria, Italy
| | - Amir Hussain
- School of Computing, Edinburgh Napier University, Scotland, UK
| | - Francesco Carlo Morabito
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella, Feo di Vito, 89124 Reggio Calabria, Italy
| |
Collapse
|
25
|
Hua Y, Shu X, Wang Z, Zhang L. Uncertainty-Guided Voxel-Level Supervised Contrastive Learning for Semi-Supervised Medical Image Segmentation. Int J Neural Syst 2022; 32:2250016. [DOI: 10.1142/s0129065722500162] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Semi-supervised learning reduces overfitting and facilitates medical image segmentation by regularizing the learning of limited well-annotated data with the knowledge provided by a large amount of unlabeled data. However, there are many misuses and underutilization of data in conventional semi-supervised methods. On the one hand, the model will deviate from the empirical distribution under the training of numerous unlabeled data. On the other hand, the model treats labeled and unlabeled data differently and does not consider inter-data information. In this paper, a semi-supervised method is proposed to exploit unlabeled data to further narrow the gap between the semi-supervised model and its fully-supervised counterpart. Specifically, the architecture of the proposed method is based on the mean-teacher framework, and the uncertainty estimation module is improved to impose constraints of consistency and guide the selection of feature representation vectors. Notably, a voxel-level supervised contrastive learning module is devised to establish a contrastive relationship between feature representation vectors, whether from labeled or unlabeled data. The supervised manner ensures that the network learns the correct knowledge, and the dense contrastive relationship further extracts information from unlabeled data. The above overcomes data misuse and underutilization in semi-supervised frameworks. Moreover, it favors the feature representation with intra-class compactness and inter-class separability and gains extra performance. Extensive experimental results on the left atrium dataset from Atrial Segmentation Challenge demonstrate that the proposed method has superior performance over the state-of-the-art methods.
Collapse
Affiliation(s)
- Yu Hua
- College of Computer Science, Sichuan University, Section 4, Southern 1st Ring Rd, Chengdu, Sichuan 610065, P. R. China
| | - Xin Shu
- College of Computer Science, Sichuan University, Section 4, Southern 1st Ring Rd, Chengdu, Sichuan 610065, P. R. China
| | - Zizhou Wang
- College of Computer Science, Sichuan University, Section 4, Southern 1st Ring Rd, Chengdu, Sichuan 610065, P. R. China
| | - Lei Zhang
- College of Computer Science, Sichuan University, Section 4, Southern 1st Ring Rd, Chengdu, Sichuan 610065, P. R. China
| |
Collapse
|
26
|
Xie P, Hao S, Zhao J, Liang Z, Li X. A Spatio-Temporal Method for Extracting Gamma-Band Features to Enhance Classification in a Rapid Serial Visual Presentation Task. Int J Neural Syst 2022; 32:2250010. [PMID: 35049411 DOI: 10.1142/s0129065722500101] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Rapid serial visual presentation (RSVP) is a type of electroencephalogram (EEG) pattern commonly used for target recognition. Besides delta- and theta-band responses already used for classification, RSVP task also evokes gamma-band responses having low amplitude and large individual difference. This paper proposes a filter bank spatio-temporal component analysis (FBSCA) method, extracting spatio-temporal features of the gamma-band responses for the first time, to enhance the RSVP classification performance. Considering the individual difference in time latency and responsive frequency, the proposed FBSCA method decomposes the gamma-band EEG data into sub-components in different time-frequency-space domains and seeks the weight coefficients to optimize the combinations of electrodes, common spatial pattern (CSP) components, time windows and frequency bands. Two state-of-the-art methods, i.e. hierarchical discriminant principal component analysis (HDPCA) and discriminative canonical pattern matching (DCPM), were used for comparison. The performance was evaluated in [Formula: see text] cross validations using a public dataset. Study results showed that the FBSCA method outperformed the other methods regardless of number of training trials. These results suggest that the proposed FBSCA method can enhance the RSVP classification.
Collapse
Affiliation(s)
- Ping Xie
- Key Laboratory of Intelligent Rehabilitation, and Neromodulation of Hebei Province, School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Shencai Hao
- Key Laboratory of Intelligent Rehabilitation, and Neromodulation of Hebei Province, School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Jing Zhao
- Key Laboratory of Intelligent Rehabilitation, and Neromodulation of Hebei Province, School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Zhenhu Liang
- Key Laboratory of Intelligent Rehabilitation, and Neromodulation of Hebei Province, School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Xiaoli Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, P. R. China
| |
Collapse
|
27
|
Cai Z, Wang L, Guo M, Xu G, Guo L, Li Y. From Intricacy to Conciseness: A Progressive Transfer Strategy for EEG-Based Cross-Subject Emotion Recognition. Int J Neural Syst 2022; 32:2250005. [PMID: 35023812 DOI: 10.1142/s0129065722500058] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Emotion plays a significant role in human daily activities, and it can be effectively recognized from EEG signals. However, individual variability limits the generalization of emotion classifiers across subjects. Domain adaptation (DA) is a reliable method to solve the issue. Due to the nonstationarity of EEG, the inferior-quality source domain data bring negative transfer in DA procedures. To solve this problem, an auto-augmentation joint distribution adaptation (AA-JDA) method and a burden-lightened and source-preferred JDA (BLSP-JDA) approach are proposed in this paper. The methods are based on a novel transfer idea, learning the specific knowledge of the target domain from the samples that are appropriate for transfer, which reduces the difficulty of transfer between two domains. On multiple emotion databases, our model shows state-of-the-art performance.
Collapse
Affiliation(s)
- Ziliang Cai
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| | - Lingyue Wang
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| | - Miaomiao Guo
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| | - Guizhi Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| | - Lei Guo
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| | - Ying Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300401, P. R. China
| |
Collapse
|
28
|
Ahmadi-Dastgerdi N, Hosseini-Nejad H, Amiri H, Shoeibi A, Gorriz JM. A Vector Quantization-Based Spike Compression Approach Dedicated to Multichannel Neural Recording Microsystems. Int J Neural Syst 2021; 32:2250001. [PMID: 34931938 DOI: 10.1142/s0129065722500010] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Implantable high-density multichannel neural recording microsystems provide simultaneous recording of brain activities. Wireless transmission of the entire recorded data causes high bandwidth usage, which is not tolerable for implantable applications. As a result, a hardware-friendly compression module is required to reduce the amount of data before it is transmitted. This paper presents a novel compression approach that utilizes a spike extractor and a vector quantization (VQ)-based spike compressor. In this approach, extracted spikes are vector quantized using an unsupervised learning process providing a high spike compression ratio (CR) of 10-80. A combination of extracting and compressing neural spikes results in a significant data reduction as well as preserving the spike waveshapes. The compression performance of the proposed approach was evaluated under variant conditions. We also developed new architectures such that the hardware blocks of our approach can be implemented more efficiently. The compression module was implemented in a 180-nm standard CMOS process achieving a SNDR of 14.49[Formula: see text]dB and a classification accuracy (CA) of 99.62% at a CR of 20, while consuming 4[Formula: see text][Formula: see text]W power and 0.16[Formula: see text]mm2 chip area per channel.
Collapse
Affiliation(s)
| | | | - Hadi Amiri
- School of Engineering Science, College of Engineering, University of Tehran, Tehran, Iran
| | - Afshin Shoeibi
- Faculty of Electrical Engineering, FPGA Research Lab K. N. Toosi, University of Technology, Tehran, Iran
| | - Juan Manuel Gorriz
- Department of Signal Processing Networking and Communications, University of Granada, Granada, Spain.,Department of Psychiatry, University of Cambridge, Cambridge, UK
| |
Collapse
|
29
|
Zhang X, Lu Z, Zhang T, Li H, Wang Y, Tao Q. Realizing the Application of EEG Modeling in BCI Classification: Based on a Conditional GAN Converter. Front Neurosci 2021; 15:727394. [PMID: 34867150 PMCID: PMC8636039 DOI: 10.3389/fnins.2021.727394] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 10/04/2021] [Indexed: 11/13/2022] Open
Abstract
Electroencephalogram (EEG) modeling in brain-computer interface (BCI) provides a theoretical foundation for its development. However, limited by the lack of guidelines in model parameter selection and the inability to obtain personal tissue information in practice, EEG modeling in BCI is mainly focused on the theoretical qualitative level which shows a gap between the theory and its application. Based on such problems, this work combined the surface EEG simulation with a converter based on the generative adversarial network (GAN), to establish the connection from simulated EEG to its application in BCI classification. For the scalp EEGs modeling, a mathematical model was built according to the physics of surface EEG, which consisted of the parallel 3-population neural mass model, the equivalent dipole, and the forward computation. For application, a converter based on the conditional GAN was designed, to transfer the simulated theoretical-only EEG to its practical version, in the lack of individual bio-information. To verify the feasibility, based on the latest microexpression-assisted BCI paradigm proposed by our group, the converted simulated EEGs were used in the training of BCI classifiers. The results indicated that, compared with training with insufficient real data, by adding the simulated EEGs, the overall performance showed a significant improvement (P = 0.04 < 0.05), and the test performance can be improved by 2.17% ± 4.23, in which the largest increase was up to 12.60% ± 1.81. Through this work, the link from theoretical EEG simulation to BCI classification has been initially established, providing an enhanced novel solution for the application of EEG modeling in BCI.
Collapse
Affiliation(s)
- Xiaodong Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China.,Shaanxi Key Laboratory of Intelligent Robot, Xi'an Jiaotong University, Xi'an, China
| | - Zhufeng Lu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China.,Shaanxi Key Laboratory of Intelligent Robot, Xi'an Jiaotong University, Xi'an, China
| | - Teng Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China.,Shaanxi Key Laboratory of Intelligent Robot, Xi'an Jiaotong University, Xi'an, China
| | - Hanzhe Li
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China.,Shaanxi Key Laboratory of Intelligent Robot, Xi'an Jiaotong University, Xi'an, China
| | - Yachun Wang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China.,Shaanxi Key Laboratory of Intelligent Robot, Xi'an Jiaotong University, Xi'an, China
| | - Qing Tao
- School of Mechanical Engineering, Xinjiang University, Wulumuqi, China
| |
Collapse
|
30
|
Karakullukcu N, Yilmaz B. Detection of Movement Intention in EEG-Based Brain-Computer Interfaces Using Fourier-Based Synchrosqueezing Transform. Int J Neural Syst 2021; 32:2150059. [PMID: 34806939 DOI: 10.1142/s0129065721500593] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Patients with motor impairments need caregivers' help to initiate the operation of brain-computer interfaces (BCI). This study aims to identify and characterize movement intention using multichannel electroencephalography (EEG) signals as a means to initiate BCI systems without extra accessories/methodologies. We propose to discriminate the resting and motor imagery (MI) states with high accuracy using Fourier-based synchrosqueezing transform (FSST) as a feature extractor. FSST has been investigated and compared with other popular approaches in 28 healthy subjects for a total of 6657 trials. The accuracy and f-measure values were obtained as 99.8% and 0.99, respectively, when FSST was used as the feature extractor and singular value decomposition (SVD) as the feature selection method and support vector machines as the classifier. Moreover, this study investigated the use of data that contain certain amount of noise without any preprocessing in addition to the clean counterparts. Furthermore, the statistical analysis of EEG channels with the best discrimination (of resting and MI states) characteristics demonstrated that F4-Fz-C3-Cz-C4-Pz channels and several statistical features had statistical significance levels, [Formula: see text], less than 0.05. This study showed that the preparation of the movement can be detected in real-time employing FSST-SVD combination and several channels with minimal pre-processing effort.
Collapse
Affiliation(s)
- Nedime Karakullukcu
- Electrical and Computer Engineering Department, Graduate School of Engineering and Sciences, Abdullah Gul University, 38080 Kayseri, Turkey.,Biomedical Instrumentation and Signal Analysis, Laboratory (BISA-Lab), School of Engineering, Abdullah Gul University, 38080 Kayseri, Turkey
| | - Bülent Yilmaz
- Electrical and Computer Engineering Department, Graduate School of Engineering and Sciences, Abdullah Gul University, 38080 Kayseri, Turkey.,Electrical-Electronics Engineering Department, School of Engineering, Abdullah Gul University, 38080 Kayseri, Turkey.,Biomedical Instrumentation and Signal Analysis Laboratory (BISA-Lab), School of Engineering, Abdullah Gul University, 38080 Kayseri, Turkey
| |
Collapse
|