1
|
Zou R, Zhao L, He S, Zhou X, Yin X. Effect of the period of EEG signals on the decoding of motor information. Phys Eng Sci Med 2024; 47:249-260. [PMID: 38150057 DOI: 10.1007/s13246-023-01361-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 11/22/2023] [Indexed: 12/28/2023]
Abstract
Decoding movement information from electroencephalogram to construct brain-computer interface has promising applications. The EEG data during the entire motor imagery (MI) period or movement execution (ME) period is generally decoded, and calculation of numerous information and massive dataset is time-consuming. In order to improve decoding efficiency, the joint topographic maps of the brain activation state of 15 subjects were studied during different periods. The results showed that the activation intensity of the preparation period in the motor imagery experiment was higher than during the exercise period, while during the exercise period, the activation intensity was higher than in the preparation period in the movement execution experiment. Hence, the wavelet neural network was used to decode the six-class movements including elbow flexion/extension, forearm pronation/supination and hand open/close in periods of MI/ME. The experimental results show that the accuracy obtained in the preparation period is the highest in the motor imagery experiment, which is 80.77%. On the other hand, the highest accuracy obtained in the exercise period of the movement execution experiment is 79.26%. It further proves that the optimized period is a key decoding factor to reduce the cost of calculation, and this new decoding method is effective to build a more intelligent brain-computer interface system.
Collapse
Affiliation(s)
- Renling Zou
- Department of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China.
| | - Liang Zhao
- Department of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Shuang He
- Department of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Xiaobo Zhou
- Department of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Xuezhi Yin
- Shanghai Berry Electronic Technology Co., Ltd, Shanghai, China
| |
Collapse
|
2
|
Jemal I, Abou-Abbas L, Henni K, Mitiche A, Mezghani N. Domain adaptation for EEG-based, cross-subject epileptic seizure prediction. Front Neuroinform 2024; 18:1303380. [PMID: 38371495 PMCID: PMC10869477 DOI: 10.3389/fninf.2024.1303380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 01/09/2024] [Indexed: 02/20/2024] Open
Abstract
The ability to predict the occurrence of an epileptic seizure is a safeguard against patient injury and health complications. However, a major challenge in seizure prediction arises from the significant variability observed in patient data. Common patient-specific approaches, which apply to each patient independently, often perform poorly for other patients due to the data variability. The aim of this study is to propose deep learning models which can handle this variability and generalize across various patients. This study addresses this challenge by introducing a novel cross-subject and multi-subject prediction models. Multiple-subject modeling broadens the scope of patient-specific modeling to account for the data from a dedicated ensemble of patients, thereby providing some useful, though relatively modest, level of generalization. The basic neural network architecture of this model is then adapted to cross-subject prediction, thereby providing a broader, more realistic, context of application. For accrued performance, and generalization ability, cross-subject modeling is enhanced by domain adaptation. Experimental evaluation using the publicly available CHB-MIT and SIENA data datasets shows that our multiple-subject model achieved better performance compared to existing works. However, the cross-subject faces challenges when applied to different patients. Finally, through investigating three domain adaptation methods, the model accuracy has been notably improved by 10.30% and 7.4% for the CHB-MIT and SIENA datasets, respectively.
Collapse
Affiliation(s)
- Imene Jemal
- Centre EMT, Institut National de la Recherche Scientifique, Montréal, QC, Canada
- Institute of Applied Artificial Intelligence (I2A), Université TÉLUQ, Montréal, QC, Canada
- Laboratoire de Recherche en Imagerie et Orthopédie, Centre de recherche du CHUM, Montréal, QC, Canada
| | - Lina Abou-Abbas
- Institute of Applied Artificial Intelligence (I2A), Université TÉLUQ, Montréal, QC, Canada
- Laboratoire de Recherche en Imagerie et Orthopédie, Centre de recherche du CHUM, Montréal, QC, Canada
| | - Khadidja Henni
- Institute of Applied Artificial Intelligence (I2A), Université TÉLUQ, Montréal, QC, Canada
- Laboratoire de Recherche en Imagerie et Orthopédie, Centre de recherche du CHUM, Montréal, QC, Canada
| | - Amar Mitiche
- Centre EMT, Institut National de la Recherche Scientifique, Montréal, QC, Canada
| | - Neila Mezghani
- Institute of Applied Artificial Intelligence (I2A), Université TÉLUQ, Montréal, QC, Canada
- Laboratoire de Recherche en Imagerie et Orthopédie, Centre de recherche du CHUM, Montréal, QC, Canada
| |
Collapse
|
3
|
Han J, Wei X, Faisal AA. EEG decoding for datasets with heterogenous electrode configurations using transfer learning graph neural networks. J Neural Eng 2023; 20:066027. [PMID: 37931308 DOI: 10.1088/1741-2552/ad09ff] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 11/06/2023] [Indexed: 11/08/2023]
Abstract
Objective. Brain-machine interfacing (BMI) has greatly benefited from adopting machine learning methods for feature learning that require extensive data for training, which are often unavailable from a single dataset. Yet, it is difficult to combine data across labs or even data within the same lab collected over the years due to the variation in recording equipment and electrode layouts resulting in shifts in data distribution, changes in data dimensionality, and altered identity of data dimensions. Our objective is to overcome this limitation and learn from many different and diverse datasets across labs with different experimental protocols.Approach. To tackle the domain adaptation problem, we developed a novel machine learning framework combining graph neural networks (GNNs) and transfer learning methodologies for non-invasive motor imagery (MI) EEG decoding, as an example of BMI. Empirically, we focus on the challenges of learning from EEG data with different electrode layouts and varying numbers of electrodes. We utilize three MI EEG databases collected using very different numbers of EEG sensors (from 22 channels to 64) and layouts (from custom layouts to 10-20).Main results. Our model achieved the highest accuracy with lower standard deviations on the testing datasets. This indicates that the GNN-based transfer learning framework can effectively aggregate knowledge from multiple datasets with different electrode layouts, leading to improved generalization in subject-independent MI EEG classification.Significance. The findings of this study have important implications for brain-computer-interface research, as they highlight a promising method for overcoming the limitations posed by non-unified experimental setups. By enabling the integration of diverse datasets with varying electrode layouts, our proposed approach can help advance the development and application of BMI technologies.
Collapse
Affiliation(s)
- Jinpei Han
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
| | - Xiaoxi Wei
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
| | - A Aldo Faisal
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
- Chair in Digital Health & Data Science, University of Bayreuth, 95447 Bayreuth, Germany
| |
Collapse
|
4
|
Wang H, Jiang J, Gan JQ, Wang H. Motor Imagery EEG Classification Based on a Weighted Multi-Branch Structure Suitable for Multisubject Data. IEEE Trans Biomed Eng 2023; 70:3040-3051. [PMID: 37186527 DOI: 10.1109/tbme.2023.3274231] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
OBJECTIVE Electroencephalogram (EEG) signal recognition based on deep learning technology requires the support of sufficient data. However, training data scarcity usually occurs in subject-specific motor imagery tasks unless multisubject data can be used to enlarge training data. Unfortunately, because of the large discrepancies between data distributions from different subjects, model performance could only be improved marginally or even worsened by simply training on multisubject data. METHOD This article proposes a novel weighted multi-branch (WMB) structure for handling multisubject data to solve the problem, in which each branch is responsible for fitting a pair of source-target subject data and adaptive weights are used to integrate all branches or select branches with the largest weights to make the final decision. The proposed WMB structure was applied to six well-known deep learning models (EEGNet, Shallow ConvNet, Deep ConvNet, ResNet, MSFBCNN, and EEG_TCNet) and comprehensive experiments were conducted on EEG datasets BCICIV-2a, BCICIV-2b, high gamma dataset (HGD) and two supplementary datasets. RESULT Superior results against the state-of-the-art models have demonstrated the efficacy of the proposed method in subject-specific motor imagery EEG classification. For example, the proposed WMB_EEGNet achieved classification accuracies of 84.14%, 90.23%, and 97.81% on BCICIV-2a, BCICIV-2b and HGD, respectively. CONCLUSION It is clear that the proposed WMB structure is capable to make good use of multisubject data with large distribution discrepancies for subject-specific EEG classification.
Collapse
|
5
|
Yang Y, Li F, Qin X, Wen H, Lin X, Huang D. Feature separation and adversarial training for the patient-independent detection of epileptic seizures. Front Comput Neurosci 2023; 17:1195334. [PMID: 37538929 PMCID: PMC10394297 DOI: 10.3389/fncom.2023.1195334] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 07/03/2023] [Indexed: 08/05/2023] Open
Abstract
An epileptic seizure is the external manifestation of abnormal neuronal discharges, which seriously affecting physical health. The pathogenesis of epilepsy is complex, and the types of epileptic seizures are diverse, resulting in significant variation in epileptic seizure data between subjects. If we feed epilepsy data from multiple patients directly into the model for training, it will lead to underfitting of the model. To overcome this problem, we propose a robust epileptic seizure detection model that effectively learns from multiple patients while eliminating the negative impact of the data distribution shift between patients. The model adopts a multi-level temporal-spectral feature extraction network to achieve feature extraction, a feature separation network to separate features into category-related and patient-related components, and an invariant feature extraction network to extract essential feature information related to categories. The proposed model is evaluated on the TUH dataset using leave-one-out cross-validation and achieves an average accuracy of 85.7%. The experimental results show that the proposed model is superior to the related literature and provides a valuable reference for the clinical application of epilepsy detection.
Collapse
Affiliation(s)
- Yong Yang
- Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu, Sichuan, China
- Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
- Chongqing School, University of Chinese Academy of Sciences, Chongqing, China
| | - Feng Li
- Department of Neurology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Xiaolin Qin
- Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu, Sichuan, China
| | - Han Wen
- Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu, Sichuan, China
| | - Xiaoguang Lin
- Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
- Chongqing School, University of Chinese Academy of Sciences, Chongqing, China
| | - Dong Huang
- Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
| |
Collapse
|
6
|
Cui J, Yuan L, Li R, Wang Z, Yang D, Jiang T. Benchmarking EEG-based Cross-dataset Driver Drowsiness Recognition with Deep Transfer Learning. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-6. [PMID: 38083525 DOI: 10.1109/embc40787.2023.10340982] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
It usually takes a long time to collect data for calibration when using electroencephalography (EEG) for driver drowsiness monitoring. Cross-dataset recognition is desirable since it can significantly save the calibration time when an existing dataset is used. However, the recognition accuracy is affected by the distribution drift problem caused by different experimental environments when building different datasets. In order to solve the problem, we propose a deep transfer learning model named Entropy-Driven Joint Adaptation Network (EDJAN), which can learn useful information from source and target domains simultaneously. An entropy-driven loss function is used to promote clustering of target-domain representations and an individual-level domain adaptation technique is proposed to alleviate the distribution discrepancy problem of test subjects. We use two public driving datasets SEEG-VIG and SADT to test the model on the cross-dataset setting. The proposed model achieved an accuracy of 83.3% when SADT is used as source domain and SEED-VIG is used as target domain and 76.7% accuracy on the reverse setting, which is higher than the other SOTA methods. The results are further analyzed with both global and local interpretation methods. Our work illuminates a promising direction of using EEG for calibration-free driver drowsiness recognition.
Collapse
|
7
|
Hu H, Yue K, Guo M, Lu K, Liu Y. Subject Separation Network for Reducing Calibration Time of MI-Based BCI. Brain Sci 2023; 13:brainsci13020221. [PMID: 36831764 PMCID: PMC9954620 DOI: 10.3390/brainsci13020221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/11/2023] [Accepted: 01/20/2023] [Indexed: 02/03/2023] Open
Abstract
Motor imagery brain-computer interface (MI-based BCIs) have demonstrated great potential in various applications. However, to well generalize classifiers to new subjects, a time-consuming calibration process is necessary due to high inter-subject variabilities of EEG signals. This process is costly and tedious, hindering the further expansion of MI-based BCIs outside of the laboratory. To reduce the calibration time of MI-based BCIs, we propose a novel domain adaptation framework that adapts multiple source subjects' labeled data to the unseen trials of target subjects. Firstly, we train one Subject Separation Network(SSN) for each of the source subjects in the dataset. Based on adversarial domain adaptation, a shared encoder is constructed to learn similar representations for both domains. Secondly, to model the factors that cause subject variabilities and eliminate the correlated noise existing in common feature space, private feature spaces orthogonal to the shared counterpart are learned for each subject. We use a shared decoder to validate that the model is actually learning from task-relevant neurophysiological information. At last, an ensemble classifier is built by the integration of the SSNs using the information extracted from each subject's task-relevant characteristics. To quantify the efficacy of the framework, we analyze the accuracy-calibration cost trade-off in MI-based BCIs, and theoretically guarantee a generalization bound on the target error. Visualizations of the transformed features illustrate the effectiveness of domain adaptation. The experimental results on the BCI Competition IV-IIa dataset prove the effectiveness of the proposed framework compared with multiple classification methods. We infer from our results that users could learn to control MI-based BCIs without a heavy calibration process. Our study further shows how to design and train Neural Networks to decode task-related information from different subjects and highlights the potential of deep learning methods for inter-subject EEG decoding.
Collapse
Affiliation(s)
- Haochen Hu
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Kang Yue
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Institute of Software, Chinese Academy of Sciences, Beijing 100045, China
| | - Mei Guo
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Kai Lu
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Yue Liu
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Correspondence:
| |
Collapse
|
8
|
Li H, Zhang D, Xie J. MI-DABAN: A dual-attention-based adversarial network for motor imagery classification. Comput Biol Med 2023; 152:106420. [PMID: 36529022 DOI: 10.1016/j.compbiomed.2022.106420] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 11/11/2022] [Accepted: 12/11/2022] [Indexed: 12/15/2022]
Abstract
The brain-computer interface (BCI) based on motor imagery electroencephalography (EEG) is widely used because of its convenience and safety. However, due to the distributional disparity between EEG signals, data from other subjects cannot be used directly to train a subject-specific classifier. For efficient use of the labeled data, domain transfer learning and adversarial learning are gradually applied to BCI classification tasks. While these methods improve classification performance, they only align globally and ignore task-specific class boundaries, which may lead to the blurring of features near the classification boundaries. Simultaneously, they employ fully shared generators to extract features, resulting in the loss of domain-specific information and the destruction of performance. To address these issues, we propose a novel dual-attention-based adversarial network for motor imagery classification (MI-DABAN). Our framework leverages multiple subjects' knowledge to improve a single subject's motor imagery classification performance by cleverly using a novel adversarial learning method and two unshared attention blocks. Specifically, without introducing additional domain discriminators, we iteratively maximize and minimize the output difference between the two classifiers to implement adversarial learning to ensure accurate domain alignment. Among them, maximization is used to identify easily confused samples near the decision boundary, and minimization is used to align the source and target domain distributions. Moreover, for the shallow features from source and target domains, we use two non-shared attention blocks to preserve domain-specific information, which can prevent the negative transfer of domain information and further improve the classification performance on test data. We conduct extensive experiments on two publicly available EEG datasets, namely BCI Competition IV Datasets 2a and 2b. The experiment results demonstrate our method's effectiveness and superiority.
Collapse
Affiliation(s)
- Huiying Li
- Jilin University, College of Computer Science and Technology, Changchun, Jilin Province, China; Key Laboratory of Symbol Computation and Knowledge Engineering, Jilin University, Changchun 130012, China.
| | - Dongxue Zhang
- Jilin University, College of Computer Science and Technology, Changchun, Jilin Province, China; Key Laboratory of Symbol Computation and Knowledge Engineering, Jilin University, Changchun 130012, China.
| | - Jingmeng Xie
- Xi'an Jiaotong University, College of Electronic information, Xi'an, Shanxi Province, China.
| |
Collapse
|
9
|
Zhang W, Wang Z, Wu D. Multi-Source Decentralized Transfer for Privacy-Preserving BCIs. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2710-2720. [PMID: 36112563 DOI: 10.1109/tnsre.2022.3207494] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Transfer learning, which utilizes labeled source domains to facilitate the learning in a target model, is effective in alleviating high intra- and inter-subject variations in electroencephalogram (EEG) based brain-computer interfaces (BCIs). Existing transfer learning approaches usually use the source subjects' EEG data directly, leading to privacy concerns. This paper considers a decentralized privacy-preserving transfer learning scenario: there are multiple source subjects, whose data and computations are kept local, and only the parameters or predictions of their pre-trained models can be accessed for privacy-protection; then, how to perform effective cross-subject transfer for a new subject with unlabeled EEG trials? We propose an offline unsupervised multi-source decentralized transfer (MSDT) approach, which first generates a pre-trained model from each source subject, and then performs decentralized transfer using the source model parameters (in gray-box settings) or predictions (in black-box settings). Experiments on two datasets from two BCI paradigms, motor imagery and affective BCI, demonstrated that MSDT outperformed several existing approaches, which do not consider privacy-protection at all. In other words, MSDT achieved both high privacy-protection and better classification performance.
Collapse
|
10
|
Wang H, Yua H, Wang H. EEG_GENet: A feature-level graph embedding method for motor imagery classification based on EEG signals. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
11
|
Domain adaptation for epileptic EEG classification using adversarial learning and Riemannian manifold. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103555] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
|
12
|
Wei W, Qiu S, Zhang Y, Mao J, He H. ERP prototypical matching net: a meta-learning method for zero-calibration RSVP-based image retrieval. J Neural Eng 2022; 19. [PMID: 35299166 DOI: 10.1088/1741-2552/ac5eb7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Accepted: 03/17/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Rapid serial visual presentation (RSVP)-based brain-computer interface (BCI) is an efficient information detection technology through detecting event-related potential (ERP) evoked by target visual stimuli. The BCI system requires a time-consuming calibration process to build a reliable decoding model for a new user. Therefore, zero-calibration has become an important topic in BCI research. APPROACH In this paper, we construct an RSVP dataset that includes 31 subjects, and propose a zero-calibration method based on a metric-based meta-learning: ERP Prototypical Matching Net (EPMN). EPMN learns a metric space where the distance between EEG features and ERP prototypes belonging to the same category is smaller than that of different categories. Here, we employ prototype learning to learn a common representation from ERP templates of different subjects as ERP prototypes. Also, a metric-learning loss function is proposed for maximizing the distance between different classes of EEG and ERP prototypes and minimize the distance between the same classes of EEG and ERP prototypes in the metric space. MAIN RESULTS The experimental results showed that EPMN achieved a balanced-accuracy of 86.34% and outperformed the comparable methods. Significance: Our EPMN can realize zero-calibration for an RSVP-based BCI system.
Collapse
Affiliation(s)
- Wei Wei
- Chinese Academy of Sciences Institute of Automation, 95 Zhongguancun East Road, Haidian District, Beijing, Beijing, Beijing, 100080, CHINA
| | - Shuang Qiu
- Chinese Academy of Sciences Institute of Automation, 95 Zhongguancun East Road, Haidian District, Beijing, Beijing, Beijing, 100190, CHINA
| | - Yukun Zhang
- Institute of Automation Chinese Academy of Sciences, 95 Zhongguancun East Road, Haidian District, Beijing, Beijing, 100190, CHINA
| | - Jiayu Mao
- Chinese Academy of Sciences Institute of Automation, 95 Zhongguancun East Road, Haidian District, Beijing, Beijing, Beijing, 100190, CHINA
| | - Huiguang He
- Chinese Academy of Sciences Institute of Automation, 95 Zhongguancun East Road, Haidian District, Beijing, Beijing, Beijing, 100190, CHINA
| |
Collapse
|
13
|
Song X, Zeng Y, Tong L, Shu J, Bao G, Yan B. P3-MSDA: Multi-Source Domain Adaptation Network for Dynamic Visual Target Detection. Front Hum Neurosci 2021; 15:685173. [PMID: 34434096 PMCID: PMC8381600 DOI: 10.3389/fnhum.2021.685173] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 07/07/2021] [Indexed: 11/13/2022] Open
Abstract
Single-trial electroencephalogram detection has been widely applied in brain-computer interface (BCI) systems. Moreover, an individual generalized model is significant for applying the dynamic visual target detection BCI system in real life because of the time jitter of the detection latency, the dynamics and complexity of visual background. Hence, we developed an unsupervised multi-source domain adaptation network (P3-MSDA) for dynamic visual target detection. In this network, a P3 map-clustering method was proposed for source domain selection. The adversarial domain adaptation was conducted for domain alignment to eliminate individual differences, and prediction probabilities were ranked and returned to guide the input of target samples for imbalanced data classification. The results showed that individuals with a strong P3 map selected by the proposed P3 map-clustering method perform best on the source domain. Compared with existing schemes, the proposed P3-MSDA network achieved the highest classification accuracy and F1 score using five labeled individuals with a strong P3 map as the source domain. These findings can have a significant meaning in building an individual generalized model for dynamic visual target detection.
Collapse
Affiliation(s)
- Xiyu Song
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Ying Zeng
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China.,Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Li Tong
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Jun Shu
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Guangcheng Bao
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China
| | - Bin Yan
- Henan Key Laboratory of Imaging and Intelligent Processing, Chinese People's Liberation Army (PLA) Strategic Support Force Information Engineering University, Zhengzhou, China
| |
Collapse
|
14
|
Ko W, Jeon E, Jeong S, Phyo J, Suk HI. A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces. Front Hum Neurosci 2021; 15:643386. [PMID: 34140883 PMCID: PMC8204721 DOI: 10.3389/fnhum.2021.643386] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2020] [Accepted: 04/27/2021] [Indexed: 11/28/2022] Open
Abstract
Brain-computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.
Collapse
Affiliation(s)
- Wonjun Ko
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Eunjin Jeon
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Seungwoo Jeong
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Jaeun Phyo
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Heung-Il Suk
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| |
Collapse
|
15
|
Shoeibi A, Khodatars M, Ghassemi N, Jafari M, Moridian P, Alizadehsani R, Panahiazar M, Khozeimeh F, Zare A, Hosseini-Nejad H, Khosravi A, Atiya AF, Aminshahidi D, Hussain S, Rouhani M, Nahavandi S, Acharya UR. Epileptic Seizures Detection Using Deep Learning Techniques: A Review. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:5780. [PMID: 34072232 PMCID: PMC8199071 DOI: 10.3390/ijerph18115780] [Citation(s) in RCA: 79] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Revised: 05/14/2021] [Accepted: 05/15/2021] [Indexed: 02/06/2023]
Abstract
A variety of screening approaches have been proposed to diagnose epileptic seizures, using electroencephalography (EEG) and magnetic resonance imaging (MRI) modalities. Artificial intelligence encompasses a variety of areas, and one of its branches is deep learning (DL). Before the rise of DL, conventional machine learning algorithms involving feature extraction were performed. This limited their performance to the ability of those handcrafting the features. However, in DL, the extraction of features and classification are entirely automated. The advent of these techniques in many areas of medicine, such as in the diagnosis of epileptic seizures, has made significant advances. In this study, a comprehensive overview of works focused on automated epileptic seizure detection using DL techniques and neuroimaging modalities is presented. Various methods proposed to diagnose epileptic seizures automatically using EEG and MRI modalities are described. In addition, rehabilitation systems developed for epileptic seizures using DL have been analyzed, and a summary is provided. The rehabilitation tools include cloud computing techniques and hardware required for implementation of DL algorithms. The important challenges in accurate detection of automated epileptic seizures using DL with EEG and MRI modalities are discussed. The advantages and limitations in employing DL-based techniques for epileptic seizures diagnosis are presented. Finally, the most promising DL models proposed and possible future works on automated epileptic seizure detection are delineated.
Collapse
Affiliation(s)
- Afshin Shoeibi
- Faculty of Electrical Engineering, Biomedical Data Acquisition Lab (BDAL), K. N. Toosi University of Technology, Tehran 1631714191, Iran;
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad 9177948974, Iran; (D.A.); (M.R.)
| | | | - Navid Ghassemi
- Faculty of Electrical Engineering, Biomedical Data Acquisition Lab (BDAL), K. N. Toosi University of Technology, Tehran 1631714191, Iran;
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad 9177948974, Iran; (D.A.); (M.R.)
| | - Mahboobeh Jafari
- Electrical and Computer Engineering Faculty, Semnan University, Semnan 3513119111, Iran;
| | - Parisa Moridian
- Faculty of Engineering, Science and Research Branch, Islamic Azad University, Tehran 1477893855, Iran;
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia; (R.A.); (F.K.); (A.K.); (S.N.)
| | - Maryam Panahiazar
- Institute for Computational Health Sciences, School of Medicine, University of California, San Francisco, CA 94143, USA;
| | - Fahime Khozeimeh
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia; (R.A.); (F.K.); (A.K.); (S.N.)
| | - Assef Zare
- Faculty of Electrical Engineering, Gonabad Branch, Islamic Azad University, Gonabad 6518115743, Iran;
| | - Hossein Hosseini-Nejad
- Faculty of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran 1631714191, Iran;
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia; (R.A.); (F.K.); (A.K.); (S.N.)
| | - Amir F. Atiya
- Department of Computer Engineering, Faculty of Engineering, Cairo University, Cairo 12613, Egypt;
| | - Diba Aminshahidi
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad 9177948974, Iran; (D.A.); (M.R.)
| | - Sadiq Hussain
- System Administrator at Dibrugarh University, Assam 786004, India;
| | - Modjtaba Rouhani
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad 9177948974, Iran; (D.A.); (M.R.)
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia; (R.A.); (F.K.); (A.K.); (S.N.)
| | - Udyavara Rajendra Acharya
- Department of Biomedical Engineering, School of Science and Technology, Singapore University of Social Sciences, Singapore 599494, Singapore;
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore 599489, Singapore
- Department of Bioinformatics and Medical Engineering, Taichung City 41354, Taiwan
| |
Collapse
|
16
|
Hong X, Zheng Q, Liu L, Chen P, Ma K, Gao Z, Zheng Y. Dynamic Joint Domain Adaptation Network for Motor Imagery Classification. IEEE Trans Neural Syst Rehabil Eng 2021; 29:556-565. [PMID: 33587702 DOI: 10.1109/tnsre.2021.3059166] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Electroencephalogram (EEG) has been widely used in brain computer interface (BCI) due to its convenience and reliability. The EEG-based BCI applications are majorly limited by the time-consuming calibration procedure for discriminative feature representation and classification. Existing EEG classification methods either heavily depend on the handcrafted features or require adequate annotated samples at each session for calibration. To address these issues, we propose a novel dynamic joint domain adaptation network based on adversarial learning strategy to learn domain-invariant feature representation, and thus improve EEG classification performance in the target domain by leveraging useful information from the source session. Specifically, we explore the global discriminator to align the marginal distribution across domains, and the local discriminator to reduce the conditional distribution discrepancy between sub-domains via conditioning on deep representation as well as the predicted labels from the classifier. In addition, we further investigate a dynamic adversarial factor to adaptively estimate the relative importance of alignment between the marginal and conditional distributions. To evaluate the efficacy of our method, extensive experiments are conducted on two public EEG datasets, namely, Datasets IIa and IIb of BCI Competition IV. The experimental results demonstrate that the proposed method achieves superior performance compared with the state-of-the-art methods.
Collapse
|
17
|
Lun X, Yu Z, Wang F, Chen T, Hou Y. A novel approach of CNN for human motor imagery recognition using the virtual electrode pairs. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2021. [DOI: 10.3233/jifs-202046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
In order to develop an efficient brain-computer interface system, the brain activity measured by electroencephalography needs to be accurately decoded. In this paper, a motor imagery classification approach is proposed, combining virtual electrodes on the cortex layer with a convolutional neural network; this can effectively improve the decoding performance of the brain-computer interface system. A three layer (cortex, skull, and scalp) head volume conduction model was established by using the symmetric boundary element method to map the scalp signal to the cortex area. Nine pairs of virtual electrodes were created on the cortex layer, and the features of the time and frequency sequence from the virtual electrodes were extracted by performing time-frequency analysis. Finally, the convolutional neural network was used to classify motor imagery tasks. The results show that the proposed approach is convergent in both the training model and the test model. Based on the Physionet motor imagery database, the averaged accuracy can reach 98.32% for a single subject, while the averaged values of accuracy, Kappa, precision, recall, and F1-score on the group-wise are 96.23%, 94.83%, 96.21%, 96.13%, and 96.14%, respectively. Based on the High Gamma database, the averaged accuracy has achieved 96.37% and 91.21% at the subject and group levels, respectively. Moreover, this approach is superior to those of other studies on the same database, which suggests robustness and adaptability to individual variability.
Collapse
Affiliation(s)
- Xiangmin Lun
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun, China
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Zhenglin Yu
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun, China
| | - Fang Wang
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Tao Chen
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| | - Yimin Hou
- School of Automation Engineering, Northeast Electric Power University, Jilin, China
| |
Collapse
|
18
|
Standardization-refinement domain adaptation method for cross-subject EEG-based classification in imagined speech recognition. Pattern Recognit Lett 2021. [DOI: 10.1016/j.patrec.2020.11.013] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
19
|
|
20
|
Wei W, Qiu S, Ma X, Li D, Wang B, He H. Reducing Calibration Efforts in RSVP Tasks With Multi-Source Adversarial Domain Adaptation. IEEE Trans Neural Syst Rehabil Eng 2020; 28:2344-2355. [DOI: 10.1109/tnsre.2020.3023761] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
21
|
Liang Y, Ma Y. Calibrating EEG features in motor imagery classification tasks with a small amount of current data using multisource fusion transfer learning. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.102101] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
|