1
|
Lee J, Kim M, Heo D, Kim J, Kim MK, Lee T, Park J, Kim H, Hwang M, Kim L, Kim SP. A comprehensive dataset for home appliance control using ERP-based BCIs with the application of inter-subject transfer learning. Front Hum Neurosci 2024; 18:1320457. [PMID: 38361913 PMCID: PMC10867822 DOI: 10.3389/fnhum.2024.1320457] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 01/08/2024] [Indexed: 02/17/2024] Open
Abstract
Brain-computer interfaces (BCIs) have a potential to revolutionize human-computer interaction by enabling direct links between the brain and computer systems. Recent studies are increasingly focusing on practical applications of BCIs-e.g., home appliance control just by thoughts. One of the non-invasive BCIs using electroencephalography (EEG) capitalizes on event-related potentials (ERPs) in response to target stimuli and have shown promise in controlling home appliance. In this paper, we present a comprehensive dataset of online ERP-based BCIs for controlling various home appliances in diverse stimulus presentation environments. We collected online BCI data from a total of 84 subjects among whom 60 subjects controlled three types of appliances (TV: 30, door lock: 15, and electric light: 15) with 4 functions per appliance, 14 subjects controlled a Bluetooth speaker with 6 functions via an LCD monitor, and 10 subjects controlled air conditioner with 4 functions via augmented reality (AR). Using the dataset, we aimed to address the issue of inter-subject variability in ERPs by employing the transfer learning in two different approaches. The first approach, "within-paradigm transfer learning," aimed to generalize the model within the same paradigm of stimulus presentation. The second approach, "cross-paradigm transfer learning," involved extending the model from a 4-class LCD environment to different paradigms. The results demonstrated that transfer learning can effectively enhance the generalizability of BCIs based on ERP across different subjects and environments.
Collapse
Affiliation(s)
- Jongmin Lee
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Minju Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Dojin Heo
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Jongsu Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Min-Ki Kim
- The Institute of Healthcare Convergence, College of Medicine, Catholic Kwandong University, Gangneung-si, Republic of Korea
| | - Taejun Lee
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Jongwoo Park
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - HyunYoung Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Minho Hwang
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Laehyun Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Sung-Phil Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| |
Collapse
|
2
|
Gao DR, Li J, Wang MQ, Wang LT, Zhang YQ. Automatic sleep staging of single-channel EEG based on domain adversarial neural networks and domain self-attention. Front Neurosci 2023; 17:1143495. [PMID: 37090812 PMCID: PMC10117677 DOI: 10.3389/fnins.2023.1143495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 03/13/2023] [Indexed: 04/09/2023] Open
Abstract
The diagnosis and management of sleep problems depend heavily on sleep staging. For autonomous sleep staging, many data-driven deep learning models have been presented by trying to construct a large-labeled auxiliary sleep dataset and test it by electroencephalograms on different subjects. These approaches suffer a significant setback cause it assumes the training and test data come from the same or similar distribution. However, this is almost impossible in scenario cross-dataset due to inherent domain shift between domains. Unsupervised domain adaption was recently created to address the domain shift issue. However, only a few customized UDA solutions for sleep staging due to two limitations in previous UDA methods. First, the domain classifier does not consider boundaries between classes. Second, they depend on a shared model to align the domain that could miss the information of domains when extracting features. Given those restrictions, we present a novel UDA approach that combines category decision boundaries and domain discriminator to align the distributions of source and target domains. Also, to keep the domain-specific features, we create an unshared attention method. In addition, we investigated effective data augmentation in cross-dataset sleep scenarios. The experimental results on three datasets validate the efficacy of our approach and show that the proposed method is superior to state-of-the-art UDA methods on accuracy and MF1-Score.
Collapse
Affiliation(s)
- Dong-Rui Gao
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
- School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Jing Li
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Man-Qing Wang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
- School of Life Sciences and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Lu-Tao Wang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
| | - Yong-Qing Zhang
- School of Computer Science, Chengdu University of Information Technology, Chengdu, China
- *Correspondence: Yong-Qing Zhang
| |
Collapse
|
3
|
Miao M, Zheng L, Xu B, Yang Z, Hu W. A multiple frequency bands parallel spatial–temporal 3D deep residual learning framework for EEG-based emotion recognition. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104141] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
4
|
Albahri AS, Al-qaysi ZT, Alzubaidi L, Alnoor A, Albahri OS, Alamoodi AH, Bakar AA. A Systematic Review of Using Deep Learning Technology in the Steady-State Visually Evoked Potential-Based Brain-Computer Interface Applications: Current Trends and Future Trust Methodology. Int J Telemed Appl 2023; 2023:7741735. [PMID: 37168809 PMCID: PMC10164869 DOI: 10.1155/2023/7741735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 02/01/2023] [Accepted: 03/16/2023] [Indexed: 05/13/2023] Open
Abstract
The significance of deep learning techniques in relation to steady-state visually evoked potential- (SSVEP-) based brain-computer interface (BCI) applications is assessed through a systematic review. Three reliable databases, PubMed, ScienceDirect, and IEEE, were considered to gather relevant scientific and theoretical articles. Initially, 125 papers were found between 2010 and 2021 related to this integrated research field. After the filtering process, only 30 articles were identified and classified into five categories based on their type of deep learning methods. The first category, convolutional neural network (CNN), accounts for 70% (n = 21/30). The second category, recurrent neural network (RNN), accounts for 10% (n = 3/30). The third and fourth categories, deep neural network (DNN) and long short-term memory (LSTM), account for 6% (n = 30). The fifth category, restricted Boltzmann machine (RBM), accounts for 3% (n = 1/30). The literature's findings in terms of the main aspects identified in existing applications of deep learning pattern recognition techniques in SSVEP-based BCI, such as feature extraction, classification, activation functions, validation methods, and achieved classification accuracies, are examined. A comprehensive mapping analysis was also conducted, which identified six categories. Current challenges of ensuring trustworthy deep learning in SSVEP-based BCI applications were discussed, and recommendations were provided to researchers and developers. The study critically reviews the current unsolved issues of SSVEP-based BCI applications in terms of development challenges based on deep learning techniques and selection challenges based on multicriteria decision-making (MCDM). A trust proposal solution is presented with three methodology phases for evaluating and benchmarking SSVEP-based BCI applications using fuzzy decision-making techniques. Valuable insights and recommendations for researchers and developers in the SSVEP-based BCI and deep learning are provided.
Collapse
Affiliation(s)
- A. S. Albahri
- Iraqi Commission for Computers and Informatics (ICCI), Baghdad, Iraq
| | - Z. T. Al-qaysi
- Department of Computer Science, Computer Science and Mathematics College, Tikrit University, Tikrit, Iraq
| | - Laith Alzubaidi
- School of Mechanical, Medical, and Process Engineering, Queensland University of Technology, Brisbane, QLD 4000, Australia
- ARC Industrial Transformation Training Centre—Joint Biomechanics, Queensland University of Technology, Brisbane, QLD 4000, Australia
| | | | - O. S. Albahri
- Computer Techniques Engineering Department, Mazaya University College, Nasiriyah, Iraq
- Department of Computer Science and Information Technology, La Trobe University, Melbourne, VIC, Australia
| | - A. H. Alamoodi
- Faculty of Computing and Meta-Technology (FKMT), Universiti Pendidikan Sultan Idris (UPSI), Perak, Malaysia
| | | |
Collapse
|
5
|
Xu DQ, Li MA. A dual alignment-based multi-source domain adaptation framework for motor imagery EEG classification. APPL INTELL 2022; 53:10766-10788. [PMID: 36039116 PMCID: PMC9402410 DOI: 10.1007/s10489-022-04077-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/08/2022] [Indexed: 11/25/2022]
Affiliation(s)
- Dong-qin Xu
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 China
| | - Ming-ai Li
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 China
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing, 100124 China
- Engineering Research Center of Digital Community, Ministry of Education, Beijing, 100124 China
| |
Collapse
|
6
|
Jeong JH, Cho JH, Lee YE, Lee SH, Shin GH, Kweon YS, Millán JDR, Müller KR, Lee SW. 2020 International brain-computer interface competition: A review. Front Hum Neurosci 2022; 16:898300. [PMID: 35937679 PMCID: PMC9354666 DOI: 10.3389/fnhum.2022.898300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 07/01/2022] [Indexed: 11/16/2022] Open
Abstract
The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.
Collapse
Affiliation(s)
- Ji-Hoon Jeong
- School of Computer Science, Chungbuk National University, Cheongju, South Korea
| | - Jeong-Hyun Cho
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Young-Eun Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Seo-Hyun Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Gi-Hwan Shin
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Young-Seok Kweon
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - José del R. Millán
- Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, United States
| | - Klaus-Robert Müller
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Machine Learning Group, Department of Computer Science, Berlin Institute of Technology, Berlin, Germany
- Max Planck Institute for Informatics, Saarbrucken, Germany
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Seong-Whan Lee
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| |
Collapse
|
7
|
Ko W, Jeon E, Yoon JS, Suk HI. Semi-supervised generative and discriminative adversarial learning for motor imagery-based brain-computer interface. Sci Rep 2022; 12:4587. [PMID: 35301366 PMCID: PMC8931045 DOI: 10.1038/s41598-022-08490-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 02/28/2022] [Indexed: 11/22/2022] Open
Abstract
Convolutional neural networks (CNNs), which can recognize structural/configuration patterns in data with different architectures, have been studied for feature extraction. However, challenges remain regarding leveraging advanced deep learning methods in BCIs. We focus on problems of small-sized training samples and interpretability of the learned parameters and leverages a semi-supervised generative and discriminative learning framework that effectively utilizes synthesized samples with real samples to discover class-discriminative features. Our framework learns the distributional characteristics of EEG signals in an embedding space using a generative model. By using artificially generated and real EEG signals, our framework finds class-discriminative spatio-temporal feature representations that help to correctly discriminate input EEG signals. It is noteworthy that the framework facilitates the exploitation of real, unlabeled samples to better uncover the underlying patterns inherent in a user’s EEG signals. To validate our framework, we conducted experiments comparing our method with conventional linear models by utilizing variants of three existing CNN architectures as generator networks and measuring the performance on three public datasets. Our framework exhibited statistically significant improvements over the competing methods. We investigated the learned network via activation pattern maps and visualized generated artificial samples to empirically justify the stability and neurophysiological plausibility of our model.
Collapse
Affiliation(s)
- Wonjun Ko
- Department of Brain and Cognitive Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Eunjin Jeon
- Department of Brain and Cognitive Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Jee Seok Yoon
- Department of Brain and Cognitive Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Heung-Il Suk
- Department of Brain and Cognitive Engineering, Korea University, Seoul, 02841, Republic of Korea. .,Department of Artificial Intelligence, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|
8
|
Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12031695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.
Collapse
|
9
|
Bhosale S, Chakraborty R, Kopparapu SK. Calibration free meta learning based approach for subject independent EEG emotion recognition. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103289] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
10
|
Pilyugina N, Tsukahara A, Tanaka K. Comparing Methods of Feature Extraction of Brain Activities for Octave Illusion Classification Using Machine Learning. SENSORS 2021; 21:s21196407. [PMID: 34640727 PMCID: PMC8512176 DOI: 10.3390/s21196407] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 09/21/2021] [Accepted: 09/22/2021] [Indexed: 11/16/2022]
Abstract
The aim of this study was to find an efficient method to determine features that characterize octave illusion data. Specifically, this study compared the efficiency of several automatic feature selection methods for automatic feature extraction of the auditory steady-state responses (ASSR) data in brain activities to distinguish auditory octave illusion and nonillusion groups by the difference in ASSR amplitudes using machine learning. We compared univariate selection, recursive feature elimination, principal component analysis, and feature importance by testifying the results of feature selection methods by using several machine learning algorithms: linear regression, random forest, and support vector machine. The univariate selection with the SVM as the classification method showed the highest accuracy result, 75%, compared to 66.6% without using feature selection. The received results will be used for future work on the explanation of the mechanism behind the octave illusion phenomenon and creating an algorithm for automatic octave illusion classification.
Collapse
Affiliation(s)
- Nina Pilyugina
- Graduate School of Advanced Science and Technology, Tokyo Denki University, Hiki-gun, Saitama 350-0394, Japan
- Correspondence:
| | - Akihiko Tsukahara
- Graduate School of Science and Engineering, Tokyo Denki University, Hiki-gun, Saitama 350-0394, Japan; (A.T.); (K.T.)
| | - Keita Tanaka
- Graduate School of Science and Engineering, Tokyo Denki University, Hiki-gun, Saitama 350-0394, Japan; (A.T.); (K.T.)
| |
Collapse
|