1
|
Hamzah HA, Abdalla KK. EEG-based emotion recognition systems; comprehensive study. Heliyon 2024; 10:e31485. [PMID: 38818173 PMCID: PMC11137547 DOI: 10.1016/j.heliyon.2024.e31485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Accepted: 05/16/2024] [Indexed: 06/01/2024] Open
Abstract
Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.
Collapse
Affiliation(s)
- Hussein Ali Hamzah
- Electrical Engineering Department, College of Engineering, University of Babylon, Iraq
| | - Kasim K. Abdalla
- Electrical Engineering Department, College of Engineering, University of Babylon, Iraq
| |
Collapse
|
2
|
Khabti J, AlAhmadi S, Soudani A. Optimal Channel Selection of Multiclass Motor Imagery Classification Based on Fusion Convolutional Neural Network with Attention Blocks. SENSORS (BASEL, SWITZERLAND) 2024; 24:3168. [PMID: 38794022 PMCID: PMC11125262 DOI: 10.3390/s24103168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Revised: 05/08/2024] [Accepted: 05/14/2024] [Indexed: 05/26/2024]
Abstract
The widely adopted paradigm in brain-computer interfaces (BCIs) involves motor imagery (MI), enabling improved communication between humans and machines. EEG signals derived from MI present several challenges due to their inherent characteristics, which lead to a complex process of classifying and finding the potential tasks of a specific participant. Another issue is that BCI systems can result in noisy data and redundant channels, which in turn can lead to increased equipment and computational costs. To address these problems, the optimal channel selection of a multiclass MI classification based on a Fusion convolutional neural network with Attention blocks (FCNNA) is proposed. In this study, we developed a CNN model consisting of layers of convolutional blocks with multiple spatial and temporal filters. These filters are designed specifically to capture the distribution and relationships of signal features across different electrode locations, as well as to analyze the evolution of these features over time. Following these layers, a Convolutional Block Attention Module (CBAM) is used to, further, enhance EEG signal feature extraction. In the process of channel selection, the genetic algorithm is used to select the optimal set of channels using a new technique to deliver fixed as well as variable channels for all participants. The proposed methodology is validated showing 6.41% improvement in multiclass classification compared to most baseline models. Notably, we achieved the highest results of 93.09% for binary classes involving left-hand and right-hand movements. In addition, the cross-subject strategy for multiclass classification yielded an impressive accuracy of 68.87%. Following channel selection, multiclass classification accuracy was enhanced, reaching 84.53%. Overall, our experiments illustrated the efficiency of the proposed EEG MI model in both channel selection and classification, showing superior results with either a full channel set or a reduced number of channels.
Collapse
Affiliation(s)
- Joharah Khabti
- Department of Computer Science, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia; (S.A.); (A.S.)
| | | | | |
Collapse
|
3
|
Du X, Ding X, Xi M, Lv Y, Qiu S, Liu Q. A Data Augmentation Method for Motor Imagery EEG Signals Based on DCGAN-GP Network. Brain Sci 2024; 14:375. [PMID: 38672024 PMCID: PMC11048538 DOI: 10.3390/brainsci14040375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Revised: 04/09/2024] [Accepted: 04/11/2024] [Indexed: 04/28/2024] Open
Abstract
Motor imagery electroencephalography (EEG) signals have garnered attention in brain-computer interface (BCI) research due to their potential in promoting motor rehabilitation and control. However, the limited availability of labeled data poses challenges for training robust classifiers. In this study, we propose a novel data augmentation method utilizing an improved Deep Convolutional Generative Adversarial Network with Gradient Penalty (DCGAN-GP) to address this issue. We transformed raw EEG signals into two-dimensional time-frequency maps and employed a DCGAN-GP network to generate synthetic time-frequency representations resembling real data. Validation experiments were conducted on the BCI IV 2b dataset, comparing the performance of classifiers trained with augmented and unaugmented data. Results demonstrated that classifiers trained with synthetic data exhibit enhanced robustness across multiple subjects and achieve higher classification accuracy. Our findings highlight the effectiveness of utilizing a DCGAN-GP-generated synthetic EEG data to improve classifier performance in distinguishing different motor imagery tasks. Thus, the proposed data augmentation method based on a DCGAN-GP offers a promising avenue for enhancing BCI system performance, overcoming data scarcity challenges, and bolstering classifier robustness, thereby providing substantial support for the broader adoption of BCI technology in real-world applications.
Collapse
Affiliation(s)
| | - Xiaohui Ding
- Communication and Network Laboratory, Dalian University, Dalian 116622, China; (X.D.); (M.X.); (Y.L.); (S.Q.); (Q.L.)
| | | | | | | | | |
Collapse
|
4
|
Liang G, Cao D, Wang J, Zhang Z, Wu Y. EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding. IEEE Trans Neural Syst Rehabil Eng 2024; 32:1535-1545. [PMID: 38536681 DOI: 10.1109/tnsre.2024.3382226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/13/2024]
Abstract
The motor imagery brain-computer interface (MI-BCI) based on electroencephalography (EEG) is a widely used human-machine interface paradigm. However, due to the non-stationarity and individual differences among subjects in EEG signals, the decoding accuracy is limited, affecting the application of the MI-BCI. In this paper, we propose the EISATC-Fusion model for MI EEG decoding, consisting of inception block, multi-head self-attention (MSA), temporal convolutional network (TCN), and layer fusion. Specifically, we design a DS Inception block to extract multi-scale frequency band information. And design a new cnnCosMSA module based on CNN and cos attention to solve the attention collapse and improve the interpretability of the model. The TCN module is improved by the depthwise separable convolution to reduces the parameters of the model. The layer fusion consists of feature fusion and decision fusion, fully utilizing the features output by the model and enhances the robustness of the model. We improve the two-stage training strategy for model training. Early stopping is used to prevent model overfitting, and the accuracy and loss of the validation set are used as indicators for early stopping. The proposed model achieves within-subject classification accuracies of 84.57% and 87.58% on BCI Competition IV Datasets 2a and 2b, respectively. And the model achieves cross-subject classification accuracies of 67.42% and 71.23% (by transfer learning) when training the model with two sessions and one session of Dataset 2a, respectively. The interpretability of the model is demonstrated through weight visualization method.
Collapse
|
5
|
Martini DN, Mancini M, Antonellis P, McDonnell P, Vitorio R, Stuart S, King LA. Prefrontal Cortex Activity During Gait in People With Persistent Symptoms After Concussion. Neurorehabil Neural Repair 2024:15459683241240423. [PMID: 38506532 DOI: 10.1177/15459683241240423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/21/2024]
Abstract
BACKGROUND Concussions result in transient symptoms stemming from a cortical metabolic energy crisis. Though this metabolic energy crisis typically resolves in a month, symptoms can persist for years. The symptomatic period is associated with gait dysfunction, the cortical underpinnings of which are poorly understood. Quantifying prefrontal cortex (PFC) activity during gait may provide insight into post-concussion gait dysfunction. The purpose of this study was to explore the effects of persisting concussion symptoms on PFC activity during gait. We hypothesized that adults with persisting concussion symptoms would have greater PFC activity during gait than controls. Within the concussed group, we hypothesized that worse symptoms would relate to increased PFC activity during gait, and that increased PFC activity would relate to worse gait characteristics. METHODS The Neurobehavior Symptom Inventory (NSI) characterized concussion symptoms. Functional near-infrared spectroscopy quantified PFC activity (relative concentration changes of oxygenated hemoglobin [HbO2]) in 14 people with a concussion and 25 controls. Gait was assessed using six inertial sensors in the concussion group. RESULTS Average NSI total score was 26.4 (13.2). HbO2 was significantly higher (P = .007) for the concussed group (0.058 [0.108]) compared to the control group (-0.016 [0.057]). Within the concussion group, HbO2 correlated with NSI total symptom score (ρ = .62; P = .02), sagittal range of motion (r = .79; P = .001), and stride time variability (r = -.54; P = .046). CONCLUSION These data suggest PFC activity relates to symptom severity and some gait characteristics in people with persistent concussion symptoms. Identifying the neurophysiological underpinnings to gait deficits post-concussion expands our knowledge of motor behavior deficits in people with persistent concussion symptoms.
Collapse
Affiliation(s)
- Douglas N Martini
- Department of Kinesiology, University of Massachusetts Amherst, Amherst, MA, USA
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
| | - Martina Mancini
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
| | | | - Paul McDonnell
- Department of Kinesiology, University of Massachusetts Amherst, Amherst, MA, USA
| | - Rodrigo Vitorio
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, UK
| | - Samuel Stuart
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, UK
| | - Laurie A King
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
| |
Collapse
|
6
|
Yu S, Wang Z, Wang F, Chen K, Yao D, Xu P, Zhang Y, Wang H, Zhang T. Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model. Cereb Cortex 2024; 34:bhad511. [PMID: 38183186 DOI: 10.1093/cercor/bhad511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 12/06/2023] [Accepted: 12/08/2023] [Indexed: 01/07/2024] Open
Abstract
Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.
Collapse
Affiliation(s)
- Shiqi Yu
- Microecology Research Center, Baiyun Branch, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
- Mental Health Education Center and School of Science, Xihua University, Chengdu 610039, China
| | - Zedong Wang
- Microecology Research Center, Baiyun Branch, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
| | - Fei Wang
- School of Computer and Software, Chengdu Jincheng College, Chengdu 610097, China
| | - Kai Chen
- Mental Health Education Center and School of Science, Xihua University, Chengdu 610039, China
| | - Dezhong Yao
- Key Laboratory for Neuroinformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Peng Xu
- Key Laboratory for Neuroinformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Yong Zhang
- Microecology Research Center, Baiyun Branch, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
| | - Hesong Wang
- Microecology Research Center, Baiyun Branch, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
| | - Tao Zhang
- Microecology Research Center, Baiyun Branch, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
- Mental Health Education Center and School of Science, Xihua University, Chengdu 610039, China
- Key Laboratory for Neuroinformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu 611731, China
| |
Collapse
|
7
|
Xie X, Chen L, Qin S, Zha F, Fan X. Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification. Front Neurorobot 2024; 18:1343249. [PMID: 38352723 PMCID: PMC10861766 DOI: 10.3389/fnbot.2024.1343249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 01/15/2024] [Indexed: 02/16/2024] Open
Abstract
Introduction As an interactive method gaining popularity, brain-computer interfaces (BCIs) aim to facilitate communication between the brain and external devices. Among the various research topics in BCIs, the classification of motor imagery using electroencephalography (EEG) signals has the potential to greatly improve the quality of life for people with disabilities. Methods This technology assists them in controlling computers or other devices like prosthetic limbs, wheelchairs, and drones. However, the current performance of EEG signal decoding is not sufficient for real-world applications based on Motor Imagery EEG (MI-EEG). To address this issue, this study proposes an attention-based bidirectional feature pyramid temporal convolutional network model for the classification task of MI-EEG. The model incorporates a multi-head self-attention mechanism to weigh significant features in the MI-EEG signals. It also utilizes a temporal convolution network (TCN) to separate high-level temporal features. The signals are enhanced using the sliding-window technique, and channel and time-domain information of the MI-EEG signals is extracted through convolution. Results Additionally, a bidirectional feature pyramid structure is employed to implement attention mechanisms across different scales and multiple frequency bands of the MI-EEG signals. The performance of our model is evaluated on the BCI Competition IV-2a dataset and the BCI Competition IV-2b dataset, and the results showed that our model outperformed the state-of-the-art baseline model, with an accuracy of 87.5 and 86.3% for the subject-dependent, respectively. Discussion In conclusion, the BFATCNet model offers a novel approach for EEG-based motor imagery classification in BCIs, effectively capturing relevant features through attention mechanisms and temporal convolutional networks. Its superior performance on the BCI Competition IV-2a and IV-2b datasets highlights its potential for real-world applications. However, its performance on other datasets may vary, necessitating further research on data augmentation techniques and integration with multiple modalities to enhance interpretability and generalization. Additionally, reducing computational complexity for real-time applications is an important area for future work.
Collapse
Affiliation(s)
- Xinghe Xie
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
- Faculty of Applied Science, Macao Polytechnic University, Macau, Macao SAR, China
| | - Liyan Chen
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
| | - Shujia Qin
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
| | - Fusheng Zha
- Harbin Institute of Technology, Harbin, Heilongjiang Province, China
| | - Xinggang Fan
- Information Engineering College, Zhijiang College of Zhejiang University of Technology, Shaoxing, China
| |
Collapse
|
8
|
Zhu L, Xu M, Zhu J, Huang A, Zhang J. A time segment adaptive optimization method based on separability criterion and correlation analysis for motor imagery BCIs. Comput Methods Biomech Biomed Engin 2024:1-14. [PMID: 38193151 DOI: 10.1080/10255842.2023.2301421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 12/24/2023] [Indexed: 01/10/2024]
Abstract
Motor imagery (MI) plays a crucial role in brain-computer interface (BCI), and the classification of MI tasks using electroencephalogram (EEG) is currently under extensive investigation. During MI classification, individual differences among subjects in terms of response and time latency need to be considered. Optimizing the time segment for different subjects can enhance subsequent classification performance. In view of the individual differences of subjects in motor imagery tasks, this article proposes a Time Segment Adaptive Optimization method based on Separability criterion and Correlation analysis (TSAOSC). The fundamental principle of this method involves applying the separability criterion to various sizes of time windows within the training data, identifying the optimal raw reference signal, and adaptively adjusting the time segment position for each trial's data by analyzing its relationship with the optimal reference signal. We evaluated our method on three BCI competition datasets, respectively. The utilization of the TSAOSC method in the experiments resulted in an enhancement of 4.90% in average classification accuracy compared to its absence. Additionally, building upon the TSAOSC approach, this study proposes a Nonlinear-TSAOSC method (N-TSAOSC) for analyzing EEG signals with nonlinearity, which shows improvements in the classification accuracy of certain subjects. The results of the experiments demonstrate that the proposed method is an effective time segment optimization method, and it can be integrated into other algorithms to further improve their accuracy.
Collapse
Affiliation(s)
- Lei Zhu
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Mengxuan Xu
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Jieping Zhu
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Aiai Huang
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Jianhai Zhang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou, China
| |
Collapse
|
9
|
Rahman A, Debnath T, Kundu D, Khan MSI, Aishi AA, Sazzad S, Sayduzzaman M, Band SS. Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunities. AIMS Public Health 2024; 11:58-109. [PMID: 38617415 PMCID: PMC11007421 DOI: 10.3934/publichealth.2024004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 12/18/2023] [Indexed: 04/16/2024] Open
Abstract
In recent years, machine learning (ML) and deep learning (DL) have been the leading approaches to solving various challenges, such as disease predictions, drug discovery, medical image analysis, etc., in intelligent healthcare applications. Further, given the current progress in the fields of ML and DL, there exists the promising potential for both to provide support in the realm of healthcare. This study offered an exhaustive survey on ML and DL for the healthcare system, concentrating on vital state of the art features, integration benefits, applications, prospects and future guidelines. To conduct the research, we found the most prominent journal and conference databases using distinct keywords to discover scholarly consequences. First, we furnished the most current along with cutting-edge progress in ML-DL-based analysis in smart healthcare in a compendious manner. Next, we integrated the advancement of various services for ML and DL, including ML-healthcare, DL-healthcare, and ML-DL-healthcare. We then offered ML and DL-based applications in the healthcare industry. Eventually, we emphasized the research disputes and recommendations for further studies based on our observations.
Collapse
Affiliation(s)
- Anichur Rahman
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
| | - Tanoy Debnath
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
- Department of CSE, Green University of Bangladesh, 220/D, Begum Rokeya Sarani, Dhaka -1207, Bangladesh
| | - Dipanjali Kundu
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Md. Saikat Islam Khan
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
| | - Airin Afroj Aishi
- Department of Computing and Information System, Daffodil International University, Savar, Dhaka, Bangladesh
| | - Sadia Sazzad
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Mohammad Sayduzzaman
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Shahab S. Band
- Department of Information Management, International Graduate School of Artificial Intelligence, National Yunlin University of Science and Technology, Taiwan
| |
Collapse
|
10
|
Wang W, Li B, Wang H, Wang X, Qin Y, Shi X, Liu S. EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification. Med Biol Eng Comput 2024; 62:107-120. [PMID: 37728715 DOI: 10.1007/s11517-023-02931-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 09/07/2023] [Indexed: 09/21/2023]
Abstract
Motor imagery (MI) electroencephalogram (EEG) signal is recognized as a promising paradigm for brain-computer interface (BCI) systems and has been extensively employed in various BCI applications, including assisting disabled individuals, controlling devices and environments, and enhancing human capabilities. The high-performance decoding capability of MI-EEG signals is a key issue that impacts the development of the industry. However, decoding MI-EEG signals is challenging due to the low signal-to-noise ratio and inter-subject variability. In response to the aforementioned core problems, this paper proposes a novel end-to-end network, a fusion multi-branch 1D convolutional neural network (EEG-FMCNN), to decode MI-EEG signals without pre-processing. The utilization of multi-branch 1D convolution not only exhibits a certain level of noise tolerance but also addresses the issue of inter-subject variability to some extent. This is attributed to the ability of multi-branch architectures to capture information from different frequency bands, enabling the establishment of optimal convolutional scales and depths. Furthermore, we incorporate 1D squeeze-and-excitation (SE) blocks and shortcut connections at appropriate locations to further enhance the generalization and robustness of the network. In the BCI Competition IV-2a dataset, our proposed model has obtained good experimental results, achieving accuracies of 78.82% and 68.41% for subject-dependent and subject-independent modes, respectively. In addition, extensive ablative experiments and fine-tuning experiments were conducted, resulting in a notable 7% improvement in the average performance of the network, which holds significant implications for the generalization and application of the network.
Collapse
Affiliation(s)
- Wenlong Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China
| | - Baojiang Li
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China.
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China.
| | - Haiyan Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China
| | - Xichao Wang
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China
| | - Yuxin Qin
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China
| | - Xingbin Shi
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- Intelligent Decision and Control Technology Institute, Shanghai Dianji University, Shanghai, 201306, China
| | - Shuxin Liu
- The School of Electrical Engineering, Shanghai Dianji University, Shanghai, 201306, China
- The Key Laboratory of Cognitive Computing and Intelligent Information Processing of Fujian Education Institutions (Wuyi University), Fujian, 354300, China
| |
Collapse
|
11
|
Rodríguez-Azar PI, Mejía-Muñoz JM, Cruz-Mejía O, Torres-Escobar R, López LVR. Fog Computing for Control of Cyber-Physical Systems in Industry Using BCI. SENSORS (BASEL, SWITZERLAND) 2023; 24:149. [PMID: 38203012 PMCID: PMC10781321 DOI: 10.3390/s24010149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 12/23/2023] [Accepted: 12/25/2023] [Indexed: 01/12/2024]
Abstract
Brain-computer interfaces use signals from the brain, such as EEG, to determine brain states, which in turn can be used to issue commands, for example, to control industrial machinery. While Cloud computing can aid in the creation and operation of industrial multi-user BCI systems, the vast amount of data generated from EEG signals can lead to slow response time and bandwidth problems. Fog computing reduces latency in high-demand computation networks. Hence, this paper introduces a fog computing solution for BCI processing. The solution consists in using fog nodes that incorporate machine learning algorithms to convert EEG signals into commands to control a cyber-physical system. The machine learning module uses a deep learning encoder to generate feature images from EEG signals that are subsequently classified into commands by a random forest. The classification scheme is compared using various classifiers, being the random forest the one that obtained the best performance. Additionally, a comparison was made between the fog computing approach and using only cloud computing through the use of a fog computing simulator. The results indicate that the fog computing method resulted in less latency compared to the solely cloud computing approach.
Collapse
Affiliation(s)
- Paula Ivone Rodríguez-Azar
- Departamento de Ingeniería Industrial y Manufactura, Instituto de Ingeniería y Tecnología, Universidad Autónoma de Ciudad Juárez, Ciudad Juárez 32310, Mexico
| | - Jose Manuel Mejía-Muñoz
- Departamento de Ingeniería Eléctrica, Instituto de Ingenieria y Tecnologia, Universidad Autónoma de Ciudad Juárez, Ciudad Juárez 32310, Mexico;
| | - Oliverio Cruz-Mejía
- Departamento de Ingeniería Industrial, FES Aragón, Universidad Nacional Autónoma de México, Mexico 57171, Mexico;
| | | | - Lucero Verónica Ruelas López
- Departamento de Ingeniería Eléctrica, Instituto de Ingenieria y Tecnologia, Universidad Autónoma de Ciudad Juárez, Ciudad Juárez 32310, Mexico;
| |
Collapse
|
12
|
Liu M, Li T, Zhang X, Yang Y, Zhou Z, Fu T. IMH-Net: a convolutional neural network for end-to-end EEG motor imagery classification. Comput Methods Biomech Biomed Engin 2023:1-14. [PMID: 37936533 DOI: 10.1080/10255842.2023.2275244] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 10/17/2023] [Indexed: 11/09/2023]
Abstract
As the main component of Brain-computer interface (BCI) technology, the classification algorithm based on EEG has developed rapidly. The previous algorithms were often based on subject-dependent settings, resulting in BCI needing to be calibrated for new users. In this work, we propose IMH-Net, an end-to-end subject-independent model. The model first uses Inception blocks extracts the frequency domain features of the data, then further compresses the feature vectors to extract the spatial domain features, and finally learns the global information and classification through Multi-Head Attention mechanism. On the OpenBMI dataset, IMH-Net obtained 73.90 ± 13.10% accuracy and 73.09 ± 14.99% F1-score in subject-independent manner, which improved the accuracy by 1.96% compared with the comparison model. On the BCI competition IV dataset 2a, this model also achieved the highest accuracy and F1-score in subject-dependent manner. The IMH-Net model we proposed can improve the accuracy of subject-independent Motor Imagery (MI), and the robustness of the algorithm is high, which has strong practical value in the field of BCI.
Collapse
Affiliation(s)
- Menghao Liu
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Tingting Li
- Department of Anesthesiology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Xu Zhang
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Yang Yang
- Shanghai Lanhui Medical Technology Co., Ltd, Shanghai, China
| | - Zhiyong Zhou
- Mechanical College, Shanghai Dianji University, Shanghai, China
| | - Tianhao Fu
- Mechanical College, Shanghai Dianji University, Shanghai, China
| |
Collapse
|
13
|
Lun X, Zhang Y, Zhu M, Lian Y, Hou Y. A Combined Virtual Electrode-Based ESA and CNN Method for MI-EEG Signal Feature Extraction and Classification. SENSORS (BASEL, SWITZERLAND) 2023; 23:8893. [PMID: 37960592 PMCID: PMC10649179 DOI: 10.3390/s23218893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 10/27/2023] [Accepted: 10/30/2023] [Indexed: 11/15/2023]
Abstract
A Brain-Computer Interface (BCI) is a medium for communication between the human brain and computers, which does not rely on other human neural tissues, but only decodes Electroencephalography (EEG) signals and converts them into commands to control external devices. Motor Imagery (MI) is an important BCI paradigm that generates a spontaneous EEG signal without external stimulation by imagining limb movements to strengthen the brain's compensatory function, and it has a promising future in the field of computer-aided diagnosis and rehabilitation technology for brain diseases. However, there are a series of technical difficulties in the research of motor imagery-based brain-computer interface (MI-BCI) systems, such as: large individual differences in subjects and poor performance of the cross-subject classification model; a low signal-to-noise ratio of EEG signals and poor classification accuracy; and the poor online performance of the MI-BCI system. To address the above problems, this paper proposed a combined virtual electrode-based EEG Source Analysis (ESA) and Convolutional Neural Network (CNN) method for MI-EEG signal feature extraction and classification. The outcomes reveal that the online MI-BCI system developed based on this method can improve the decoding ability of multi-task MI-EEG after training, it can learn generalized features from multiple subjects in cross-subject experiments and has some adaptability to the individual differences of new subjects, and it can decode the EEG intent online and realize the brain control function of the intelligent cart, which provides a new idea for the research of an online MI-BCI system.
Collapse
Affiliation(s)
| | | | | | | | - Yimin Hou
- School of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (X.L.); (Y.Z.); (M.Z.); (Y.L.)
| |
Collapse
|
14
|
Zhi H, Yu Z, Yu T, Gu Z, Yang J. A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding. IEEE Trans Neural Syst Rehabil Eng 2023; 31:3988-3998. [PMID: 37815970 DOI: 10.1109/tnsre.2023.3323325] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/12/2023]
Abstract
Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs.
Collapse
|
15
|
Liu K, Yang M, Xing X, Yu Z, Wu W. SincMSNet: a Sinc filter convolutional neural network for EEG motor imagery classification. J Neural Eng 2023; 20:056024. [PMID: 37683664 DOI: 10.1088/1741-2552/acf7f4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 09/08/2023] [Indexed: 09/10/2023]
Abstract
Objective.Motor imagery (MI) is widely used in brain-computer interfaces (BCIs). However, the decode of MI-EEG using convolutional neural networks (CNNs) remains a challenge due to individual variability.Approach.We propose a fully end-to-end CNN called SincMSNet to address this issue. SincMSNet employs the Sinc filter to extract subject-specific frequency band information and utilizes mixed-depth convolution to extract multi-scale temporal information for each band. It then applies a spatial convolutional block to extract spatial features and uses a temporal log-variance block to obtain classification features. The model of SincMSNet is trained under the joint supervision of cross-entropy and center loss to achieve inter-class separable and intra-class compact representations of EEG signals.Main results.We evaluated the performance of SincMSNet on the BCIC-IV-2a (four-class) and OpenBMI (two-class) datasets. SincMSNet achieves impressive results, surpassing benchmark methods. In four-class and two-class inter-session analysis, it achieves average accuracies of 80.70% and 71.50% respectively. In four-class and two-class single-session analysis, it achieves average accuracies of 84.69% and 76.99% respectively. Additionally, visualizations of the learned band-pass filter bands by Sinc filters demonstrate the network's ability to extract subject-specific frequency band information from EEG.Significance.This study highlights the potential of SincMSNet in improving the performance of MI-EEG decoding and designing more robust MI-BCIs. The source code for SincMSNet can be found at:https://github.com/Want2Vanish/SincMSNet.
Collapse
Affiliation(s)
- Ke Liu
- Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
- Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Mingzhao Yang
- Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Xin Xing
- Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, People's Republic of China
| | - Zhuliang Yu
- College of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, People's Republic of China
| | - Wei Wu
- Alto Neuroscience, Inc., Los Altos, CA 94022, United States of America
| |
Collapse
|
16
|
Chowdhury RR, Muhammad Y, Adeel U. Enhancing Cross-Subject Motor Imagery Classification in EEG-Based Brain-Computer Interfaces by Using Multi-Branch CNN. SENSORS (BASEL, SWITZERLAND) 2023; 23:7908. [PMID: 37765965 PMCID: PMC10536894 DOI: 10.3390/s23187908] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 08/23/2023] [Accepted: 09/11/2023] [Indexed: 09/29/2023]
Abstract
A brain-computer interface (BCI) is a computer-based system that allows for communication between the brain and the outer world, enabling users to interact with computers using neural activity. This brain signal is obtained from electroencephalogram (EEG) signals. A significant obstacle to the development of BCIs based on EEG is the classification of subject-independent motor imagery data since EEG data are very individualized. Deep learning techniques such as the convolutional neural network (CNN) have illustrated their influence on feature extraction to increase classification accuracy. In this paper, we present a multi-branch (five branches) 2D convolutional neural network that employs several hyperparameters for every branch. The proposed model achieved promising results for cross-subject classification and outperformed EEGNet, ShallowConvNet, DeepConvNet, MMCNN, and EEGNet_Fusion on three public datasets. Our proposed model, EEGNet Fusion V2, achieves 89.6% and 87.8% accuracy for the actual and imagined motor activity of the eegmmidb dataset and scores of 74.3% and 84.1% for the BCI IV-2a and IV-2b datasets, respectively. However, the proposed model has a bit higher computational cost, i.e., it takes around 3.5 times more computational time per sample than EEGNet_Fusion.
Collapse
Affiliation(s)
- Radia Rayan Chowdhury
- Department of Computing & Games, School of Computing, Engineering & Digital Technologies, Teesside University, Middlesbrough TS1 3BX, UK
| | - Yar Muhammad
- Department of Computing & Games, School of Computing, Engineering & Digital Technologies, Teesside University, Middlesbrough TS1 3BX, UK
- Department of Computer Science, School of Physics, Engineering & Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK
| | - Usman Adeel
- Department of Computing & Games, School of Computing, Engineering & Digital Technologies, Teesside University, Middlesbrough TS1 3BX, UK
| |
Collapse
|
17
|
Huang Y, Zheng J, Xu B, Li X, Liu Y, Wang Z, Feng H, Cao S. An improved model using convolutional sliding window-attention network for motor imagery EEG classification. Front Neurosci 2023; 17:1204385. [PMID: 37662108 PMCID: PMC10469504 DOI: 10.3389/fnins.2023.1204385] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 07/26/2023] [Indexed: 09/05/2023] Open
Abstract
Introduction The classification model of motor imagery-based electroencephalogram (MI-EEG) is a new human-computer interface pattern and a new neural rehabilitation assessment method for diseases such as Parkinson's and stroke. However, existing MI-EEG models often suffer from insufficient richness of spatiotemporal feature extraction, learning ability, and dynamic selection ability. Methods To solve these problems, this work proposed a convolutional sliding window-attention network (CSANet) model composed of novel spatiotemporal convolution, sliding window, and two-stage attention blocks. Results The model outperformed existing state-of-the-art (SOTA) models in within- and between-individual classification tasks on commonly used MI-EEG datasets BCI-2a and Physionet MI-EEG, with classification accuracies improved by 4.22 and 2.02%, respectively. Discussion The experimental results also demonstrated that the proposed type token, sliding window, and local and global multi-head self-attention mechanisms can significantly improve the model's ability to construct, learn, and adaptively select multi-scale spatiotemporal features in MI-EEG signals, and accurately identify electroencephalogram signals in the unilateral motor area. This work provided a novel and accurate classification model for MI-EEG brain-computer interface tasks and proposed a feasible neural rehabilitation assessment scheme based on the model, which could promote the further development and application of MI-EEG methods in neural rehabilitation.
Collapse
Affiliation(s)
- Yuxuan Huang
- School of Computer Science and Technology, Donghua University, Shanghai, China
| | - Jianxu Zheng
- Department of Neurosurgery and State Key Laboratory of Trauma, Burn and Combined Injury, Southwest Hospital, Third Military Medical University (Army Medical University), Chongqing, China
| | - Binxing Xu
- School of Computer Science and Technology, Donghua University, Shanghai, China
| | - Xuhang Li
- School of Computer Science and Technology, Donghua University, Shanghai, China
| | - Yu Liu
- School of Computer Science and Technology, Donghua University, Shanghai, China
| | - Zijian Wang
- School of Computer Science and Technology, Donghua University, Shanghai, China
| | - Hua Feng
- Department of Neurosurgery and State Key Laboratory of Trauma, Burn and Combined Injury, Southwest Hospital, Third Military Medical University (Army Medical University), Chongqing, China
| | - Shiqi Cao
- Department of Orthopaedics of TCM Clinical Unit, The Sixth Medical Center, Chinese PLA General Hospital, Beijing, China
| |
Collapse
|
18
|
Park HJ, Lee B. Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network. Front Hum Neurosci 2023; 17:1186594. [PMID: 37645689 PMCID: PMC10461632 DOI: 10.3389/fnhum.2023.1186594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 07/21/2023] [Indexed: 08/31/2023] Open
Abstract
Introduction In this study, we classified electroencephalography (EEG) data of imagined speech using signal decomposition and multireceptive convolutional neural network. The imagined speech EEG with five vowels /a/, /e/, /i/, /o/, and /u/, and mute (rest) sounds were obtained from ten study participants. Materials and methods First, two different signal decomposition methods were applied for comparison: noise-assisted multivariate empirical mode decomposition and wavelet packet decomposition. Six statistical features were calculated from the decomposed eight sub-frequency bands EEG. Next, all features obtained from each channel of the trial were vectorized and used as the input vector of classifiers. Lastly, EEG was classified using multireceptive field convolutional neural network and several other classifiers for comparison. Results We achieved an average classification rate of 73.09 and up to 80.41% in a multiclass (six classes) setup (Chance: 16.67%). In comparison with various other classifiers, significant improvements for other classifiers were achieved (p-value < 0.05). From the frequency sub-band analysis, high-frequency band regions and the lowest-frequency band region contain more information about imagined vowel EEG data. The misclassification and classification rate of each vowel imaginary EEG was analyzed through a confusion matrix. Discussion Imagined speech EEG can be classified successfully using the proposed signal decomposition method and a convolutional neural network. The proposed classification method for imagined speech EEG can contribute to developing a practical imagined speech-based brain-computer interfaces system.
Collapse
Affiliation(s)
- Hyeong-jun Park
- Department of Biomedical Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Boreom Lee
- Department of Biomedical Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
- AI Graduate School, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| |
Collapse
|
19
|
Li M, Qi Y, Pan G. Encrypt with Your Mind: Reliable and Revocable Brain Biometrics via Multidimensional Gaussian Fitted Bit Allocation. Bioengineering (Basel) 2023; 10:912. [PMID: 37627797 PMCID: PMC10451328 DOI: 10.3390/bioengineering10080912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 07/19/2023] [Accepted: 07/30/2023] [Indexed: 08/27/2023] Open
Abstract
Biometric features, e.g., fingerprints, the iris, and the face, have been widely used to authenticate individuals. However, most biometrics are not cancellable, i.e., once these biometric features are cloned or stolen, they cannot be replaced easily. Unlike traditional biometrics, brain biometrics are extremely difficult to clone or forge due to the natural randomness across different individuals, which makes them an ideal option for identity authentication. Most existing brain biometrics are based on electroencephalogram (EEG), which is usually demonstrated unstable performance due to the low signal-to-noise ratio (SNR). For the first time, we propose the use of intracortical brain signals, which have higher resolution and SNR, to realize the construction of the high-performance brain biometrics. Specifically, we put forward a novel brain-based key generation approach called multidimensional Gaussian fitted bit allocation (MGFBA). The proposed MGFBA method extracts keys from the local field potential of ten rats with high reliability and high entropy. We found that with the proposed MGFBA, the average effective key length of the brain biometrics was 938 bits, while achieving high authentication accuracy of 88.1% at a false acceptance rate of 1.9%, which is significantly improved compared to conventional EEG-based approaches. In addition, the proposed MGFBA-based keys can be conveniently revoked using different motor behaviors with high entropy. Experimental results demonstrate the potential of using intracortical brain signals for reliable authentication and other security applications.
Collapse
Affiliation(s)
- Ming Li
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
| | - Yu Qi
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- Affiliated Mental Health Center & Hangzhou Seventh Peoples Hospital, MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University School of Medicine, Hangzhou 310030, China
| | - Gang Pan
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
| |
Collapse
|
20
|
Degirmenci M, Yuce YK, Perc M, Isler Y. Statistically significant features improve binary and multiple Motor Imagery task predictions from EEGs. Front Hum Neurosci 2023; 17:1223307. [PMID: 37497042 PMCID: PMC10366537 DOI: 10.3389/fnhum.2023.1223307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 06/23/2023] [Indexed: 07/28/2023] Open
Abstract
In recent studies, in the field of Brain-Computer Interface (BCI), researchers have focused on Motor Imagery tasks. Motor Imagery-based electroencephalogram (EEG) signals provide the interaction and communication between the paralyzed patients and the outside world for moving and controlling external devices such as wheelchair and moving cursors. However, current approaches in the Motor Imagery-BCI system design require effective feature extraction methods and classification algorithms to acquire discriminative features from EEG signals due to the non-linear and non-stationary structure of EEG signals. This study investigates the effect of statistical significance-based feature selection on binary and multi-class Motor Imagery EEG signal classifications. In the feature extraction process performed 24 different time-domain features, 15 different frequency-domain features which are energy, variance, and entropy of Fourier transform within five EEG frequency subbands, 15 different time-frequency domain features which are energy, variance, and entropy of Wavelet transform based on five EEG frequency subbands, and 4 different Poincare plot-based non-linear parameters are extracted from each EEG channel. A total of 1,364 Motor Imagery EEG features are supplied from 22 channel EEG signals for each input EEG data. In the statistical significance-based feature selection process, the best one among all possible combinations of these features is tried to be determined using the independent t-test and one-way analysis of variance (ANOVA) test on binary and multi-class Motor Imagery EEG signal classifications, respectively. The whole extracted feature set and the feature set that contain statistically significant features only are classified in this study. We implemented 6 and 7 different classifiers in multi-class and binary (two-class) classification tasks, respectively. The classification process is evaluated using the five-fold cross-validation method, and each classification algorithm is tested 10 times. These repeated tests provide to check the repeatability of the results. The maximum of 61.86 and 47.36% for the two-class and four-class scenarios, respectively, are obtained with Ensemble Subspace Discriminant among all these classifiers using selected features including only statistically significant features. The results reveal that the introduced statistical significance-based feature selection approach improves the classifier performances by achieving higher classifier performances with fewer relevant components in Motor Imagery task classification. In conclusion, the main contribution of the presented study is two-fold evaluation of non-linear parameters as an alternative to the commonly used features and the prediction of multiple Motor Imagery tasks using statistically significant features.
Collapse
Affiliation(s)
- Murside Degirmenci
- Department of Biomedical Technologies, Izmir Katip Celebi University, İzmir, Türkiye
| | - Yilmaz Kemal Yuce
- Department of Computer Engineering, Alanya Alaaddin Keykubat University, Antalya, Türkiye
| | - Matjaž Perc
- Faculty of Natural Sciences and Mathematics, University of Maribor, Maribor, Slovenia
- Department of Medical Research, China Medical University Hospital, China Medical University, Taichung, Taiwan
- Alma Mater Europaea, Maribor, Slovenia
- Complexity Science Hub Vienna, Vienna, Austria
- Department of Physics, Kyung Hee University, Seoul, Republic of Korea
| | - Yalcin Isler
- Department of Biomedical Engineering, Izmir Katip Celebi University, İzmir, Türkiye
| |
Collapse
|
21
|
Li M, Qi Y, Pan G. Optimal Feature Analysis for Identification Based on Intracranial Brain Signals with Machine Learning Algorithms. Bioengineering (Basel) 2023; 10:801. [PMID: 37508828 PMCID: PMC10376518 DOI: 10.3390/bioengineering10070801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 06/05/2023] [Accepted: 06/29/2023] [Indexed: 07/30/2023] Open
Abstract
Biometrics, e.g., fingerprints, the iris, and the face, have been widely used to authenticate individuals. However, most biometrics are not cancellable, i.e., once these traditional biometrics are cloned or stolen, they cannot be replaced easily. Unlike traditional biometrics, brain biometrics are extremely difficult to clone or forge due to the natural randomness across different individuals, which makes them an ideal option for identity authentication. Most existing brain biometrics are based on an electroencephalogram (EEG), which typically demonstrates unstable performance due to the low signal-to-noise ratio (SNR). Thus, in this paper, we propose the use of intracortical brain signals, which have higher resolution and SNR, to realize the construction of a high-performance brain biometric. Significantly, this is the first study to investigate the features of intracortical brain signals for identification. Specifically, several features based on local field potential are computed for identification, and their performance is compared with different machine learning algorithms. The results show that frequency domain features and time-frequency domain features are excellent for intra-day and inter-day identification. Furthermore, the energy features perform best among all features with 98% intra-day and 93% inter-day identification accuracy, which demonstrates the great potential of intracraial brain signals to be biometrics. This paper may serve as a guidance for future intracranial brain researches and the development of more reliable and high-performance brain biometrics.
Collapse
Affiliation(s)
- Ming Li
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
| | - Yu Qi
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- Affiliated Mental Health Center & Hangzhou Seventh Peoples Hospital, MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University School of Medicine, Hangzhou 310030, China
| | - Gang Pan
- State Key Lab of Brain-Machine Intelligence, Hangzhou 310018, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
| |
Collapse
|
22
|
Tan X, Wang D, Chen J, Xu M. Transformer-Based Network with Optimization for Cross-Subject Motor Imagery Identification. Bioengineering (Basel) 2023; 10:bioengineering10050609. [PMID: 37237679 DOI: 10.3390/bioengineering10050609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 05/14/2023] [Accepted: 05/16/2023] [Indexed: 05/28/2023] Open
Abstract
Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.
Collapse
Affiliation(s)
- Xiyue Tan
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Dan Wang
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Jiaming Chen
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Meng Xu
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
23
|
Gwon D, Won K, Song M, Nam CS, Jun SC, Ahn M. Review of public motor imagery and execution datasets in brain-computer interfaces. Front Hum Neurosci 2023; 17:1134869. [PMID: 37063105 PMCID: PMC10101208 DOI: 10.3389/fnhum.2023.1134869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 03/10/2023] [Indexed: 04/18/2023] Open
Abstract
The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.
Collapse
Affiliation(s)
- Daeun Gwon
- Department of Computer Science and Electrical Engineering, Handong Global University, Pohang, Republic of Korea
| | - Kyungho Won
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Minseok Song
- Department of Computer Science and Electrical Engineering, Handong Global University, Pohang, Republic of Korea
| | - Chang S. Nam
- Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC, United States
- Department of Industrial and Management Systems Engineering, Kyung Hee University, Yongin-si, Republic of Korea
| | - Sung Chan Jun
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
- AI Graudate School, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Minkyu Ahn
- Department of Computer Science and Electrical Engineering, Handong Global University, Pohang, Republic of Korea
- School of Computer Science and Electrical Engineering, Handong Global University, Pohang, Republic of Korea
| |
Collapse
|
24
|
Robust Classification and Detection of Big Medical Data Using Advanced Parallel K-Means Clustering, YOLOv4, and Logistic Regression. Life (Basel) 2023; 13:life13030691. [PMID: 36983845 PMCID: PMC10056696 DOI: 10.3390/life13030691] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 02/24/2023] [Accepted: 02/28/2023] [Indexed: 03/08/2023] Open
Abstract
Big-medical-data classification and image detection are crucial tasks in the field of healthcare, as they can assist with diagnosis, treatment planning, and disease monitoring. Logistic regression and YOLOv4 are popular algorithms that can be used for these tasks. However, these techniques have limitations and performance issue with big medical data. In this study, we presented a robust approach for big-medical-data classification and image detection using logistic regression and YOLOv4, respectively. To improve the performance of these algorithms, we proposed the use of advanced parallel k-means pre-processing, a clustering technique that identified patterns and structures in the data. Additionally, we leveraged the acceleration capabilities of a neural engine processor to further enhance the speed and efficiency of our approach. We evaluated our approach on several large medical datasets and showed that it could accurately classify large amounts of medical data and detect medical images. Our results demonstrated that the combination of advanced parallel k-means pre-processing, and the neural engine processor resulted in a significant improvement in the performance of logistic regression and YOLOv4, making them more reliable for use in medical applications. This new approach offers a promising solution for medical data classification and image detection and may have significant implications for the field of healthcare.
Collapse
|
25
|
Convolutional Neural Network with a Topographic Representation Module for EEG-Based Brain-Computer Interfaces. Brain Sci 2023; 13:brainsci13020268. [PMID: 36831811 PMCID: PMC9954538 DOI: 10.3390/brainsci13020268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 02/02/2023] [Accepted: 02/03/2023] [Indexed: 02/08/2023] Open
Abstract
Convolutional neural networks (CNNs) have shown great potential in the field of brain-computer interfaces (BCIs) due to their ability to directly process raw electroencephalogram (EEG) signals without artificial feature extraction. Some CNNs have achieved better classification accuracy than that of traditional methods. Raw EEG signals are usually represented as a two-dimensional (2-D) matrix composed of channels and time points, ignoring the spatial topological information of electrodes. Our goal is to make a CNN that takes raw EEG signals as inputs have the ability to learn spatial topological features and improve its classification performance while basically maintaining its original structure. We propose an EEG topographic representation module (TRM). This module consists of (1) a mapping block from raw EEG signals to a 3-D topographic map and (2) a convolution block from the topographic map to an output with the same size as the input. According to the size of the convolutional kernel used in the convolution block, we design two types of TRMs, namely TRM-(5,5) and TRM-(3,3). We embed the two TRM types into three widely used CNNs (ShallowConvNet, DeepConvNet and EEGNet) and test them on two publicly available datasets (the Emergency Braking During Simulated Driving Dataset (EBDSDD) and the High Gamma Dataset (HGD)). Results show that the classification accuracies of all three CNNs are improved on both datasets after using the TRMs. With TRM-(5,5), the average classification accuracies of DeepConvNet, EEGNet and ShallowConvNet are improved by 6.54%, 1.72% and 2.07% on the EBDSDD and by 6.05%, 3.02% and 5.14% on the HGD, respectively; with TRM-(3,3), they are improved by 7.76%, 1.71% and 2.17% on the EBDSDD and by 7.61%, 5.06% and 6.28% on the HGD, respectively. We improve the classification performance of three CNNs on both datasets through the use of TRMs, indicating that they have the capability to mine spatial topological EEG information. More importantly, since the output of a TRM has the same size as the input, CNNs with raw EEG signals as inputs can use this module without changing their original structures.
Collapse
|
26
|
Rajesh Kumar DT, Mahalaxmi U, MM R, Bhatt DD. Optimization enabled deep residual neural network for motor imagery EEG signal classification. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
27
|
A Review of Online Classification Performance in Motor Imagery-Based Brain–Computer Interfaces for Stroke Neurorehabilitation. SIGNALS 2023. [DOI: 10.3390/signals4010004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
Motor imagery (MI)-based brain–computer interfaces (BCI) have shown increased potential for the rehabilitation of stroke patients; nonetheless, their implementation in clinical practice has been restricted due to their low accuracy performance. To date, although a lot of research has been carried out in benchmarking and highlighting the most valuable classification algorithms in BCI configurations, most of them use offline data and are not from real BCI performance during the closed-loop (or online) sessions. Since rehabilitation training relies on the availability of an accurate feedback system, we surveyed articles of current and past EEG-based BCI frameworks who report the online classification of the movement of two upper limbs in both healthy volunteers and stroke patients. We found that the recently developed deep-learning methods do not outperform the traditional machine-learning algorithms. In addition, patients and healthy subjects exhibit similar classification accuracy in current BCI configurations. Lastly, in terms of neurofeedback modality, functional electrical stimulation (FES) yielded the best performance compared to non-FES systems.
Collapse
|
28
|
Yacine F, Salah H, Amar K, Ahmad K. A novel ANN adaptive Riemannian-based kernel classification for motor imagery. Biomed Phys Eng Express 2022; 9. [PMID: 36535004 DOI: 10.1088/2057-1976/acaca2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 12/19/2022] [Indexed: 12/23/2022]
Abstract
More recently, a number of studies show the interest of the use of the Riemannian geometry in EEG classification. The idea is to exploit the EEG covariance matrices, instead of the raw EEG data, and use the Riemannian geometry to directly classify these matrices. This paper presents a novel Artificial Neural Network approach based on an Adaptive Riemannian Kernel, named ARK-ANN, to classify Electroencephalographic (EEG) motor imaging signals in the context of Brain Computer Interface (BCI). A multilayer perceptron is used to classify the covariance matrices of Motor Imagery (MI) signals employing an adaptive optimization of the testing set. The contribution of a geodesic filter is also assessed for the ANN and the original method which uses an SVM classifier. The results demonstrate that the ARK-ANN performs better than the other methods and the geodesic filter gives slightly better results in the ARK-SVM, considered here as the reference method, in the case of inter-subject classification (accuracy of 87.4% and 86% for ARK-ANN and ARK-SVM, respectively). Regarding the cross-subject classification, the proposed method gives an accuracy of 77.3% and increases the precision by 8.2% in comparison to the SVM based method.
Collapse
Affiliation(s)
- Fodil Yacine
- Laboratoire d'Analyse et Modélisation des Phénomènes Aléatoires (LAMPA), University Mouloud Mammeri of Tizi-Ouzou (UMMTO), Algeria
| | - Haddab Salah
- Laboratoire d'Analyse et Modélisation des Phénomènes Aléatoires (LAMPA), University Mouloud Mammeri of Tizi-Ouzou (UMMTO), Algeria
| | - Kachenoura Amar
- Laboratoire Traitement du Signal et de l'Image (LTSI), University Rennes, Inserm, LTSI-UMR 1099, Rennes, France
| | - Karfoul Ahmad
- Laboratoire Traitement du Signal et de l'Image (LTSI), University Rennes, Inserm, LTSI-UMR 1099, Rennes, France
| |
Collapse
|
29
|
Mattiev J, Sajovic J, Drevenšek G, Rogelj P. Assessment of Model Accuracy in Eyes Open and Closed EEG Data: Effect of Data Pre-Processing and Validation Methods. BIOENGINEERING (BASEL, SWITZERLAND) 2022; 10:bioengineering10010042. [PMID: 36671614 PMCID: PMC9854523 DOI: 10.3390/bioengineering10010042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 12/21/2022] [Accepted: 12/25/2022] [Indexed: 12/31/2022]
Abstract
Eyes open and eyes closed data is often used to validate novel human brain activity classification methods. The cross-validation of models trained on minimally preprocessed data is frequently utilized, regardless of electroencephalography data comprised of data resulting from muscle activity and environmental noise, affecting classification accuracy. Moreover, electroencephalography data of a single subject is often divided into smaller parts, due to limited availability of large datasets. The most frequently used method for model validation is cross-validation, even though the results may be affected by overfitting to the specifics of brain activity of limited subjects. To test the effects of preprocessing and classifier validation on classification accuracy, we tested fourteen classification algorithms implemented in WEKA and MATLAB, tested on comprehensively and simply preprocessed electroencephalography data. Hold-out and cross-validation were used to compare the classification accuracy of eyes open and closed data. The data of 50 subjects, with four minutes of data with eyes closed and open each was used. The algorithms trained on simply preprocessed data were superior to the ones trained on comprehensively preprocessed data in cross-validation testing. The reverse was true when hold-out accuracy was examined. Significant increases in hold-out accuracy were observed if the data of different subjects was not strictly separated between the test and training datasets, showing the presence of overfitting. The results show that comprehensive data preprocessing can be advantageous for subject invariant classification, while higher subject-specific accuracy can be attained with simple preprocessing. Researchers should thus state the final intended use of their classifier.
Collapse
Affiliation(s)
- Jamolbek Mattiev
- Department of Information Technologies, Urgench State University, Khamid Alimdjan 14, Urgench 220100, Uzbekistan
- Correspondence:
| | - Jakob Sajovic
- Department of Orthodontics, University Medical Centre Ljubljana, Hrvatski trg 6, 1000 Ljubljana, Slovenia
- Faculty of Medicine, University of Ljubljana, Korytkova 2, 1000 Ljubljana, Slovenia
| | - Gorazd Drevenšek
- Faculty of Medicine, University of Ljubljana, Korytkova 2, 1000 Ljubljana, Slovenia
- Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska, Glagoljaška 8, 6000 Koper, Slovenia
| | - Peter Rogelj
- Faculty of Mathematics, Natural Sciences and Information Technologies, University of Primorska, Glagoljaška 8, 6000 Koper, Slovenia
| |
Collapse
|
30
|
Ramasubramanian B, Reddy VS, Chellappan V, Ramakrishna S. Emerging Materials, Wearables, and Diagnostic Advancements in Therapeutic Treatment of Brain Diseases. BIOSENSORS 2022; 12:1176. [PMID: 36551143 PMCID: PMC9775999 DOI: 10.3390/bios12121176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 12/07/2022] [Accepted: 12/07/2022] [Indexed: 06/17/2023]
Abstract
Among the most critical health issues, brain illnesses, such as neurodegenerative conditions and tumors, lower quality of life and have a significant economic impact. Implantable technology and nano-drug carriers have enormous promise for cerebral brain activity sensing and regulated therapeutic application in the treatment and detection of brain illnesses. Flexible materials are chosen for implantable devices because they help reduce biomechanical mismatch between the implanted device and brain tissue. Additionally, implanted biodegradable devices might lessen any autoimmune negative effects. The onerous subsequent operation for removing the implanted device is further lessened with biodegradability. This review expands on current developments in diagnostic technologies such as magnetic resonance imaging, computed tomography, mass spectroscopy, infrared spectroscopy, angiography, and electroencephalogram while providing an overview of prevalent brain diseases. As far as we are aware, there hasn't been a single review article that addresses all the prevalent brain illnesses. The reviewer also looks into the prospects for the future and offers suggestions for the direction of future developments in the treatment of brain diseases.
Collapse
Affiliation(s)
- Brindha Ramasubramanian
- Department of Mechanical Engineering, Center for Nanofibers & Nanotechnology, National University of Singapore, Singapore 117574, Singapore
- Institute of Materials Research and Engineering (IMRE), Agency for Science, Technology and Research (A*STAR), #08-03, 2 Fusionopolis Way, Innovis, Singapore 138634, Singapore
| | - Vundrala Sumedha Reddy
- Department of Mechanical Engineering, Center for Nanofibers & Nanotechnology, National University of Singapore, Singapore 117574, Singapore
| | - Vijila Chellappan
- Institute of Materials Research and Engineering (IMRE), Agency for Science, Technology and Research (A*STAR), #08-03, 2 Fusionopolis Way, Innovis, Singapore 138634, Singapore
| | - Seeram Ramakrishna
- Department of Mechanical Engineering, Center for Nanofibers & Nanotechnology, National University of Singapore, Singapore 117574, Singapore
| |
Collapse
|
31
|
Upadhyay HK, Juneja S, Muhammad G, Nauman A, Awad NA. Analysis of IoT-Related Ergonomics-Based Healthcare Issues Using Analytic Hierarchy Process Methodology. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22218232. [PMID: 36365939 PMCID: PMC9655769 DOI: 10.3390/s22218232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 10/18/2022] [Accepted: 10/21/2022] [Indexed: 05/12/2023]
Abstract
The objective of the present work is for assessing ergonomics-based IoT (Internet of Things) related healthcare issues with the use of a popular multi-criteria decision-making technique named the analytic hierarchy process (AHP). Multiple criteria decision making (MCDM) is a technique that combines alternative performance across numerous contradicting, qualitative, and/or quantitative criteria, resulting in a solution requiring a consensus. The AHP is a flexible strategy for organizing and simplifying complex MCDM concerns by disassembling a compound decision problem into an ordered array of relational decision components (evaluation criteria, sub-criteria, and substitutions). A total of twelve IoT-related ergonomics-based healthcare issues have been recognized as Lumbago (lower backache), Cervicalgia (neck ache), shoulder pain; digital eye strain, hearing impairment, carpal tunnel syndrome; distress, exhaustion, depression; obesity, high blood pressure, hyperglycemia. "Distress" has proven itself the most critical IoT-related ergonomics-based healthcare issue, followed by obesity, depression, and exhaustion. These IoT-related ergonomics-based healthcare issues in four categories (excruciating issues, eye-ear-nerve issues, psychosocial issues, and persistent issues) have been compared and ranked. Based on calculated mathematical values, "psychosocial issues" have been ranked in the first position followed by "persistent issues" and "eye-ear-nerve issues". In several industrial systems, the results may be of vital importance for increasing the efficiency of human force, particularly a human-computer interface for prolonged hours.
Collapse
Affiliation(s)
| | - Sapna Juneja
- KIET Group of Institutions, Delhi NCR, Ghaziabad 201206, India
- Correspondence: (S.J.); (G.M.)
| | - Ghulam Muhammad
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (S.J.); (G.M.)
| | - Ali Nauman
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Korea
| | - Nancy Awadallah Awad
- Department of Computer and Information Systems, Sadat Academy for Management Sciences, Cairo 11742, Egypt
| |
Collapse
|
32
|
Li X, Chen Y, Tu X, Huang H. Development of a Three-Dimensional Nerve Stretch Growth Device towards an Implantable Neural Interface. MICROMACHINES 2022; 13:1558. [PMID: 36295911 PMCID: PMC9610007 DOI: 10.3390/mi13101558] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 09/09/2022] [Accepted: 09/19/2022] [Indexed: 06/16/2023]
Abstract
Because of rising traumatic accidents and diseases, the number of patients suffering from nerve injury is increasing. Without effective rehabilitation therapy, the patients will get motor or sensory function losses or even a lifelong disability. As for amputees, neural interface technology can be used to splice nerves and electrical wires together in a way that allows them to control an artificial limb as if it was a natural extension of the body. However, the means the need for an autologous nerve to stimulate axonal regeneration and extension into target tissues, which are limited by the supply of donor nerves. Based on the principle of mechanical force regulating axon growth, in this paper, we developed a three-dimensional nerve stretch growth device for an implantable neural interface. The device consists of three motors controlled by single chip microcomputer and some mechanical parts. The stability and reliability of the device were tested. Then, we used neurons derived from human pluripotent stem cells by small chemical molecules to explore the optimal three-dimensional stretch culture parameters. Furthermore, we found that the axons were intact through 10 rotations per day and 1 mm of horizontal pulling per day. The results of this research will provide convenience for patients treated through an implantable neural interface.
Collapse
Affiliation(s)
- Xiao Li
- School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China
| | - Yongguang Chen
- School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China
| | - Xikai Tu
- School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China
| | - Hailong Huang
- Department of Rehabilitation Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430077, China
| |
Collapse
|
33
|
Gao S, Yang J, Shen T, Jiang W. A Parallel Feature Fusion Network Combining GRU and CNN for Motor Imagery EEG Decoding. Brain Sci 2022; 12:brainsci12091233. [PMID: 36138969 PMCID: PMC9496764 DOI: 10.3390/brainsci12091233] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 08/13/2022] [Accepted: 09/09/2022] [Indexed: 11/29/2022] Open
Abstract
In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.
Collapse
|
34
|
Singamaneni KK, Dhiman G, Juneja S, Muhammad G, AlQahtani SA, Zaki J. A Novel QKD Approach to Enhance IIOT Privacy and Computational Knacks. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22186741. [PMID: 36146089 PMCID: PMC9504852 DOI: 10.3390/s22186741] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Revised: 08/26/2022] [Accepted: 08/29/2022] [Indexed: 05/17/2023]
Abstract
The industry-based internet of things (IIoT) describes how IIoT devices enhance and extend their capabilities for production amenities, security, and efficacy. IIoT establishes an enterprise-to-enterprise setup that means industries have several factories and manufacturing units that are dependent on other sectors for their services and products. In this context, individual industries need to share their information with other external sectors in a shared environment which may not be secure. The capability to examine and inspect such large-scale information and perform analytical protection over the large volumes of personal and organizational information demands authentication and confidentiality so that the total data are not endangered after illegal access by hackers and other unauthorized persons. In parallel, these large volumes of confidential industrial data need to be processed within reasonable time for effective deliverables. Currently, there are many mathematical-based symmetric and asymmetric key cryptographic approaches and identity- and attribute-based public key cryptographic approaches that exist to address the abovementioned concerns and limitations such as computational overheads and taking more time for crucial generation as part of the encipherment and decipherment process for large-scale data privacy and security. In addition, the required key for the encipherment and decipherment process may be generated by a third party which may be compromised and lead to man-in-the-middle attacks, brute force attacks, etc. In parallel, there are some other quantum key distribution approaches available to produce keys for the encipherment and decipherment process without the need for a third party. However, there are still some attacks such as photon number splitting attacks and faked state attacks that may be possible with these existing QKD approaches. The primary motivation of our work is to address and avoid such abovementioned existing problems with better and optimal computational overhead for key generation, encipherment, and the decipherment process compared to the existing conventional models. To overcome the existing problems, we proposed a novel dynamic quantum key distribution (QKD) algorithm for critical public infrastructure, which will secure all cyber-physical systems as part of IIoT. In this paper, we used novel multi-state qubit representation to support enhanced dynamic, chaotic quantum key generation with high efficiency and low computational overhead. Our proposed QKD algorithm can create a chaotic set of qubits that act as a part of session-wise dynamic keys used to encipher the IIoT-based large scales of information for secure communication and distribution of sensitive information.
Collapse
Affiliation(s)
- Kranthi Kumar Singamaneni
- Department of Computer Science and Engineering, School of Technology, GITAM Deemed to be University, Visakhapatnam 530045, Andhra Pradesh, India
| | - Gaurav Dhiman
- Department of Electrical and Computer Engineering, Lebanese American University, Beirut 1102 2801, Lebanon
- University Centre for Research and Development, Department of Computer Science and Engineering, Chandigarh University, Gharuan, Mohali 140413, Punjab, India
- Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun 248002, Uttarakhand, India
| | - Sapna Juneja
- KIET Group of Institutions, Delhi NCR, Ghaziabad 201206, Uttar Pradesh, India
| | - Ghulam Muhammad
- Research Chair of New Emerging Technologies and 5G Networks and Beyond, Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence:
| | - Salman A. AlQahtani
- Research Chair of New Emerging Technologies and 5G Networks and Beyond, Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - John Zaki
- Department of Computer and Systems, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt
| |
Collapse
|
35
|
He Y, Lu Z, Wang J, Ying S, Shi J. A Self-Supervised Learning Based Channel Attention MLP-Mixer Network for Motor Imagery Decoding. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2406-2417. [PMID: 35976835 DOI: 10.1109/tnsre.2022.3199363] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Convolutional Neural Network (CNN) is commonly used for the Electroencephalogram (EEG) based motor-imagery (MI) decoding. However, its performance is generally limited due to the small size sample problem. An alternative way to address such issue is to segment EEG trials into small slices for data augmentation, but this approach usually inevitably loses the valuable long-range dependencies of temporal information in EEG signals. To this end, we propose a novel self-supervised learning (SSL) based channel attention MLP-Mixer network (S-CAMLP-Net) for MI decoding with EEG. Specifically, a new EEG slice prediction task is designed as the pretext task to capture the long-range information of EEG trials in the time domain. In the downstream task, a newly proposed MLP-Mixer is applied to the classification task for signals rather than for images. Moreover, in order to effectively learn the discriminative spatial representations in EEG slices, an attention mechanism is integrated into MLP-Mixer to adaptively estimate the importance of each EEG channel without any prior information. Thus, the proposed S-CAMLP-Net can effectively learn more long-range temporal information and global spatial features of EEG signals. Extensive experiments are conducted on the public MI-2 dataset and the BCI Competition IV Dataset 2A. The experimental results indicate that our proposed S-CAMLP-Net achieves superior classification performance over all the compared algorithms.
Collapse
|
36
|
Na Y, Joo H, Trang LT, Quan LDA, Woo J. Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses. Front Neurosci 2022; 16:906616. [PMID: 36061597 PMCID: PMC9433707 DOI: 10.3389/fnins.2022.906616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 07/25/2022] [Indexed: 11/29/2022] Open
Abstract
Auditory prostheses provide an opportunity for rehabilitation of hearing-impaired patients. Speech intelligibility can be used to estimate the extent to which the auditory prosthesis improves the user’s speech comprehension. Although behavior-based speech intelligibility is the gold standard, precise evaluation is limited due to its subjectiveness. Here, we used a convolutional neural network to predict speech intelligibility from electroencephalography (EEG). Sixty-four–channel EEGs were recorded from 87 adult participants with normal hearing. Sentences spectrally degraded by a 2-, 3-, 4-, 5-, and 8-channel vocoder were used to set relatively low speech intelligibility conditions. A Korean sentence recognition test was used. The speech intelligibility scores were divided into 41 discrete levels ranging from 0 to 100%, with a step of 2.5%. Three scores, namely 30.0, 37.5, and 40.0%, were not collected. The speech features, i.e., the speech temporal envelope (ENV) and phoneme (PH) onset, were used to extract continuous-speech EEGs for speech intelligibility prediction. The deep learning model was trained by a dataset of event-related potentials (ERP), correlation coefficients between the ERPs and ENVs, between the ERPs and PH onset, or between ERPs and the product of the multiplication of PH and ENV (PHENV). The speech intelligibility prediction accuracies were 97.33% (ERP), 99.42% (ENV), 99.55% (PH), and 99.91% (PHENV). The models were interpreted using the occlusion sensitivity approach. While the ENV models’ informative electrodes were located in the occipital area, the informative electrodes of the phoneme models, i.e., PH and PHENV, were based on the occlusion sensitivity map located in the language processing area. Of the models tested, the PHENV model obtained the best speech intelligibility prediction accuracy. This model may promote clinical prediction of speech intelligibility with a comfort speech intelligibility test.
Collapse
Affiliation(s)
- Youngmin Na
- Department of Biomedical Engineering, University of Ulsan, Ulsan, South Korea
| | - Hyosung Joo
- Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan, South Korea
| | - Le Thi Trang
- Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan, South Korea
| | - Luong Do Anh Quan
- Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan, South Korea
| | - Jihwan Woo
- Department of Biomedical Engineering, University of Ulsan, Ulsan, South Korea
- Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan, South Korea
- *Correspondence: Jihwan Woo,
| |
Collapse
|
37
|
Lomelin-Ibarra VA, Gutierrez-Rodriguez AE, Cantoral-Ceballos JA. Motor Imagery Analysis from Extensive EEG Data Representations Using Convolutional Neural Networks. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22166093. [PMID: 36015854 PMCID: PMC9414220 DOI: 10.3390/s22166093] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 08/06/2022] [Accepted: 08/12/2022] [Indexed: 05/28/2023]
Abstract
Motor imagery is a complex mental task that represents muscular movement without the execution of muscular action, involving cognitive processes of motor planning and sensorimotor proprioception of the body. Since the mental task has similar behavior to that of the motor execution process, it can be used to create rehabilitation routines for patients with some motor skill impairment. However, due to the nature of this mental task, its execution is complicated. Hence, the classification of these signals in scenarios such as brain-computer interface systems tends to have a poor performance. In this work, we study in depth different forms of data representation of motor imagery EEG signals for distinct CNN-based models as well as novel EEG data representations including spectrograms and multidimensional raw data. With the aid of transfer learning, we achieve results up to 93% accuracy, exceeding the current state of the art. However, although these results are strong, they entail the use of high computational resources to generate the samples, since they are based on spectrograms. Thus, we searched further for alternative forms of EEG representations, based on 1D, 2D, and 3D variations of the raw data, leading to promising results for motor imagery classification that still exceed the state of the art. Hence, in this work, we focus on exploring alternative methods to process and improve the classification of motor imagery features with few preprocessing techniques.
Collapse
|
38
|
Wei Y, Li J, Ji H, Jin L, Liu L, Bai Z, Ye C. A Semi-Supervised Progressive Learning Algorithm for Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2067-2076. [PMID: 35853068 DOI: 10.1109/tnsre.2022.3192448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Brain-computer interface (BCI) usually suffers from the problem of low recognition accuracy and large calibration time, especially when identifying motor imagery tasks for subjects with indistinct features and classifying fine grained motion control tasks by electroencephalogram (EEG)-electromyogram (EMG) fusion analysis. To fill the research gap, this paper presents an end-to-end semi-supervised learning framework for EEG classification and EEG-EMG fusion analysis. Benefiting from the proposed metric learning based label estimation strategy, sampling criterion and progressive learning scheme, the proposed framework efficiently extracts distinctive feature embedding from the unlabeled EEG samples and achieves a 5.40% improvement on BCI Competition IV Dataset IIa with 80% unlabeled samples and an average 3.35% improvement on two public BCI datasets. By employing synchronous EMG features as pseudo labels for the unlabeled EEG samples, the proposed framework further extracts deep level features of the synergistic complementarity between the EEG signals and EMG features based on the deep encoders, which improves the performance of hybrid BCI (with a 5.53% improvement for the Upper Limb Motion Dataset and an average 4.34% improvement on two hybrid datasets). Moreover, the ablation experiments show that the proposed framework can substantially improve the performance of the deep encoders (with an average 5.53% improvement). The proposed framework not only largely improves the performance of deep networks in the BCI system, but also significantly reduces the calibration time for EEG-EMG fusion analysis, which shows great potential for building an efficient and high-performance hybrid BCI for the motor rehabilitation process.
Collapse
|
39
|
Altuwaijri GA, Muhammad G. Electroencephalogram-Based Motor Imagery Signals Classification Using a Multi-Branch Convolutional Neural Network Model with Attention Blocks. Bioengineering (Basel) 2022; 9:bioengineering9070323. [PMID: 35877374 PMCID: PMC9311604 DOI: 10.3390/bioengineering9070323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 06/29/2022] [Accepted: 07/12/2022] [Indexed: 11/24/2022] Open
Abstract
Brain signals can be captured via electroencephalogram (EEG) and be used in various brain–computer interface (BCI) applications. Classifying motor imagery (MI) using EEG signals is one of the important applications that can help a stroke patient to rehabilitate or perform certain tasks. Dealing with EEG-MI signals is challenging because the signals are weak, may contain artefacts, are dependent on the patient’s mood and posture, and have low signal-to-noise ratio. This paper proposes a multi-branch convolutional neural network model called the Multi-Branch EEGNet with Convolutional Block Attention Module (MBEEGCBAM) using attention mechanism and fusion techniques to classify EEG-MI signals. The attention mechanism is applied both channel-wise and spatial-wise. The proposed model is a lightweight model that has fewer parameters and higher accuracy compared to other state-of-the-art models. The accuracy of the proposed model is 82.85% and 95.45% using the BCI-IV2a motor imagery dataset and the high gamma dataset, respectively. Additionally, when using the fusion approach (FMBEEGCBAM), it achieves 83.68% and 95.74% accuracy, respectively.
Collapse
|
40
|
Ahmad F, Ahmad A, Hussain I, Muhammad G, Uddin Z, AlQahtani SA. Proactive Caching in D2D Assisted Multitier Cellular Network. SENSORS 2022; 22:s22145078. [PMID: 35890758 PMCID: PMC9322377 DOI: 10.3390/s22145078] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 06/29/2022] [Accepted: 07/01/2022] [Indexed: 02/04/2023]
Abstract
Cache-enabled networks suffer hugely from the challenge of content caching and content delivery. In this regard, cache-enabled device-to-device (D2D) assisted multitier cellular networks are expected to relieve the network data pressure and effectively solve the problem of content placement and content delivery. Consequently, the user can have a better opportunity to get their favored contents from nearby cache-enabled transmitters (CETs) through reliable and good-quality links; however, as expected, designing an effective caching policy is a challenging task due to the limited cache memory of CETs and uncertainty in user preferences. In this article, we introduce a joint content placement and content delivery technique for D2D assisted multitier cellular networks (D2DMCN). A support vector machine (SVM) is employed to predict the content popularity to determine which content is to be cached and where it is to be cached, thereby increasing the overall cache hit ratio (CHR). The content request is satisfied either by the neighboring node through the D2D link or by the cache-enabled base stations (BSs) of the multitier cellular networks (MCNs). Similarly, to solve the problem of optimal content delivery, the Hungarian algorithm is employed aiming to improve the quality of satisfaction. The simulation results indicate that the proposed content placement strategy effectively optimizes the overall cache hit ratio of the system. Similarly, an effective content delivery approach reduces the request content delivery delay and power consumption.
Collapse
Affiliation(s)
- Fawad Ahmad
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah 47040, Pakistan; (F.A.); (A.A.); (Z.U.)
| | - Ayaz Ahmad
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah 47040, Pakistan; (F.A.); (A.A.); (Z.U.)
| | - Irshad Hussain
- Faculty of Electrical and Computer Engineering, University of Engineering and Technology Peshawar, Peshawar 25000, Pakistan
- Correspondence: (I.H.); (G.M.)
| | - Ghulam Muhammad
- Research Chair of New Emerging Technologies and 5G Networks and Beyond, Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia;
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (I.H.); (G.M.)
| | - Zahoor Uddin
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah 47040, Pakistan; (F.A.); (A.A.); (Z.U.)
| | - Salman A. AlQahtani
- Research Chair of New Emerging Technologies and 5G Networks and Beyond, Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia;
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| |
Collapse
|
41
|
Al-Hammadi M, Bencherif MA, Alsulaiman M, Muhammad G, Mekhtiche MA, Abdul W, Alohali YA, Alrayes TS, Mathkour H, Faisal M, Algabri M, Altaheri H, Alfakih T, Ghaleb H. Spatial Attention-Based 3D Graph Convolutional Neural Network for Sign Language Recognition. SENSORS 2022; 22:s22124558. [PMID: 35746341 PMCID: PMC9227856 DOI: 10.3390/s22124558] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 06/10/2022] [Accepted: 06/13/2022] [Indexed: 02/04/2023]
Abstract
Sign language is the main channel for hearing-impaired people to communicate with others. It is a visual language that conveys highly structured components of manual and non-manual parameters such that it needs a lot of effort to master by hearing people. Sign language recognition aims to facilitate this mastering difficulty and bridge the communication gap between hearing-impaired people and others. This study presents an efficient architecture for sign language recognition based on a convolutional graph neural network (GCN). The presented architecture consists of a few separable 3DGCN layers, which are enhanced by a spatial attention mechanism. The limited number of layers in the proposed architecture enables it to avoid the common over-smoothing problem in deep graph neural networks. Furthermore, the attention mechanism enhances the spatial context representation of the gestures. The proposed architecture is evaluated on different datasets and shows outstanding results.
Collapse
Affiliation(s)
- Muneer Al-Hammadi
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7034 Trondheim, Norway
| | - Mohamed A. Bencherif
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (M.A.B.); (G.M.)
| | - Mansour Alsulaiman
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Ghulam Muhammad
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence: (M.A.B.); (G.M.)
| | - Mohamed Amine Mekhtiche
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Wadood Abdul
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Yousef A. Alohali
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Tareq S. Alrayes
- Department of Special Education, College of Education, King Saud University, Riyadh 11543, Saudi Arabia;
| | - Hassan Mathkour
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Mohammed Faisal
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Center of AI & Robotics, Kuwait College of Science and Technology (KCST), Kuwait City 35004, Kuwait
| | - Mohammed Algabri
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Hamdi Altaheri
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Taha Alfakih
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
| | - Hamid Ghaleb
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia; (M.A.-H.); (M.A.); (M.A.M.); (W.A.); (Y.A.A.); (H.M.); (M.F.); (M.A.); (H.A.); (T.A.); (H.G.)
- Software Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| |
Collapse
|
42
|
Du Y, Liu J. IENet: a robust convolutional neural network for EEG based brain-computer interfaces. J Neural Eng 2022; 19. [PMID: 35605585 DOI: 10.1088/1741-2552/ac7257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 05/22/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) based on electroencephalogram (EEG) develop into novel application areas with more complex scenarios, which put forward higher requirements for the robustness of EEG signal processing algorithms. Deep learning can automatically extract discriminative features and potential dependencies via deep structures, demonstrating strong analytical capabilities in numerous domains such as computer vision (CV) and natural language processing (NLP). Making full use of deep learning technology to design a robust algorithm that is capable of analyzing EEG across BCI paradigms is our main work in this paper. APPROACH Inspired by InceptionV4 and InceptionTime architecture, we introduce a neural network ensemble named InceptionEEG-Net (IENet), where multi-scale convolutional layer and convolution of length 1 enable model to extract rich high-dimensional features with limited parameters. In addition, we propose the average receptive field gain for convolutional neural networks (CNNs), which optimizes IENet to detect long patterns at a smaller cost. We compare with the current state-of-the-art method across five EEG-BCI paradigms: steady-state visual evoked potentials, epilepsy EEG, overt attention P300 visual-evoked potentials, covert attention P300 visual-evoked potentials and movement-related cortical potentials. MAIN RESULTS The classification results show that the generalizability of IENet is on par with the state-of-the-art paradigm-agnostic models on test datasets. Furthermore, the feature explainability analysis of IENet illustrates its capability to extract neurophysiologically interpretable features for different BCI paradigms, ensuring the reliability of algorithm. Significance. It can be seen from our results that IENet can generalize to different BCI paradigms. And it is essential for deep CNNs to increase the receptive field size using average receptive field gain.
Collapse
Affiliation(s)
- Yipeng Du
- SCCE, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing 100083 P. R.China, Beijing, Beijing, 100083, CHINA
| | - Jian Liu
- SCCE, University of Science and Technology Beijing, 30 Xueyuan Road, Haidian District, Beijing 100083 P. R.China, Beijing, 100083, CHINA
| |
Collapse
|
43
|
Nafisah SI, Muhammad G. Tuberculosis detection in chest radiograph using convolutional neural network architecture and explainable artificial intelligence. Neural Comput Appl 2022; 36:1-21. [PMID: 35462630 PMCID: PMC9016694 DOI: 10.1007/s00521-022-07258-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2021] [Accepted: 03/29/2022] [Indexed: 12/18/2022]
Abstract
In most regions of the world, tuberculosis (TB) is classified as a malignant infectious disease that can be fatal. Using advanced tools and technology, automatic analysis and classification of chest X-rays (CXRs) into TB and non-TB can be a reliable alternative to the subjective assessment performed by healthcare professionals. Thus, in the study, we propose an automatic TB detection system using advanced deep learning (DL) models. A significant portion of a CXR image is dark, providing no information for diagnosis and potentially confusing DL models. Therefore, in the proposed system, we use sophisticated segmentation networks to extract the region of interest from multimedia CXRs. Then, segmented images are fed into the DL models. For the subjective assessment, we use explainable artificial intelligence to visualize TB-infected parts of the lung. We use different convolutional neural network (CNN) models in our experiments and compare their classification performance using three publicly available CXR datasets. EfficientNetB3, one of the CNN models, achieves the highest accuracy of 99.1%, with a receiver operating characteristic of 99.9%, and an average accuracy of 98.7%. Experiment results confirm that using segmented lung CXR images produces better performance than does using raw lung CXR images.
Collapse
Affiliation(s)
- Saad I. Nafisah
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, 11543 Saudi Arabia
| | - Ghulam Muhammad
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, 11543 Saudi Arabia
| |
Collapse
|
44
|
Altuwaijri GA, Muhammad G, Altaheri H, Alsulaiman M. A Multi-Branch Convolutional Neural Network with Squeeze-and-Excitation Attention Blocks for EEG-Based Motor Imagery Signals Classification. Diagnostics (Basel) 2022; 12:diagnostics12040995. [PMID: 35454043 PMCID: PMC9032940 DOI: 10.3390/diagnostics12040995] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/10/2022] [Accepted: 04/12/2022] [Indexed: 02/04/2023] Open
Abstract
Electroencephalography-based motor imagery (EEG-MI) classification is a critical component of the brain-computer interface (BCI), which enables people with physical limitations to communicate with the outside world via assistive technology. Regrettably, EEG decoding is challenging because of the complexity, dynamic nature, and low signal-to-noise ratio of the EEG signal. Developing an end-to-end architecture capable of correctly extracting EEG data’s high-level features remains a difficulty. This study introduces a new model for decoding MI known as a Multi-Branch EEGNet with squeeze-and-excitation blocks (MBEEGSE). By clearly specifying channel interdependencies, a multi-branch CNN model with attention blocks is employed to adaptively change channel-wise feature responses. When compared to existing state-of-the-art EEG motor imagery classification models, the suggested model achieves good accuracy (82.87%) with reduced parameters in the BCI-IV2a motor imagery dataset and (96.15%) in the high gamma dataset.
Collapse
Affiliation(s)
- Ghadir Ali Altuwaijri
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia; (G.A.A.); (H.A.); (M.A.)
| | - Ghulam Muhammad
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia; (G.A.A.); (H.A.); (M.A.)
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia
- Correspondence:
| | - Hamdi Altaheri
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia; (G.A.A.); (H.A.); (M.A.)
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia
| | - Mansour Alsulaiman
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia; (G.A.A.); (H.A.); (M.A.)
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia
| |
Collapse
|
45
|
Deep EEG feature learning via stacking common spatial pattern and support matrix machine. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103531] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
|
46
|
|
47
|
Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12031695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.
Collapse
|
48
|
Lu W, Jiang W, Zhang N, Xue F. Design of Intelligent Nursing Decision Support System Based on Multiattribute Decision Model. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:1088393. [PMID: 35126894 PMCID: PMC8816552 DOI: 10.1155/2022/1088393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/09/2021] [Revised: 10/29/2021] [Accepted: 11/09/2021] [Indexed: 11/17/2022]
Abstract
Nursing medical histories are copied randomly, leading to the occurrence of incorrectly altered histories, and medical errors are on the rise year by year. Moreover, erroneous falsification of nursing medical history is a challenging issue that needs to be urgently addressed in domestic nursing medical history. In this manuscript, a caregiver association method, which is based on a multiattribute decision model, is proposed which is specifically designed for the caregiver association selection problem in intelligent caregiving decision making. In this mode, through the selection and modeling of key performance indicators and the design of combined weights, the purpose of regulating the priority of attributes is achieved. At the same time, a comprehensive evaluation index is designed to realize the joint optimization of multiple attributes. Simulation results show that the proposed nursing workstation can not only tradeoff the quality of nursing staff experience and system performance but also balance the distribution of nursing work and effectively reduce the association delay and system energy consumption..
Collapse
Affiliation(s)
- Wenjing Lu
- Nursing Department, The Second Affiliated Hospital of Air Force Military Medical University, Xi'an 710038, China
| | - Wei Jiang
- Nursing Department, The Second Affiliated Hospital of Air Force Military Medical University, Xi'an 710038, China
| | - Na Zhang
- Nursing Department, The Second Affiliated Hospital of Air Force Military Medical University, Xi'an 710038, China
| | - Feng Xue
- Nursing Department, The Second Affiliated Hospital of Air Force Military Medical University, Xi'an 710038, China
| |
Collapse
|
49
|
Altuwaijri GA, Muhammad G. A Multibranch of Convolutional Neural Network Models for Electroencephalogram-Based Motor Imagery Classification. BIOSENSORS 2022; 12:22. [PMID: 35049650 PMCID: PMC8773854 DOI: 10.3390/bios12010022] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Revised: 12/19/2021] [Accepted: 12/30/2021] [Indexed: 06/14/2023]
Abstract
Automatic high-level feature extraction has become a possibility with the advancement of deep learning, and it has been used to optimize efficiency. Recently, classification methods for Convolutional Neural Network (CNN)-based electroencephalography (EEG) motor imagery have been proposed, and have achieved reasonably high classification accuracy. These approaches, however, use the CNN single convolution scale, whereas the best convolution scale varies from subject to subject. This limits the precision of classification. This paper proposes multibranch CNN models to address this issue by effectively extracting the spatial and temporal features from raw EEG data, where the branches correspond to different filter kernel sizes. The proposed method's promising performance is demonstrated by experimental results on two public datasets, the BCI Competition IV 2a dataset and the High Gamma Dataset (HGD). The results of the technique show a 9.61% improvement in the classification accuracy of multibranch EEGNet (MBEEGNet) from the fixed one-branch EEGNet model, and 2.95% from the variable EEGNet model. In addition, the multibranch ShallowConvNet (MBShallowConvNet) improved the accuracy of a single-scale network by 6.84%. The proposed models outperformed other state-of-the-art EEG motor imagery classification methods.
Collapse
Affiliation(s)
- Ghadir Ali Altuwaijri
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia;
- Computer Sciences and Information Technology College, Majmaah University, Al Majma’ah 11952, Saudi Arabia
| | - Ghulam Muhammad
- Department of Computer Engineering, College of Computer and Information Sciences (CCIS), King Saud University, Riyadh 11543, Saudi Arabia;
- Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia
| |
Collapse
|
50
|
Zhu L, Xu Z, Fang T. Analysis of Cardiac Ultrasound Images of Critically Ill Patients Using Deep Learning. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:6050433. [PMID: 34745504 PMCID: PMC8566033 DOI: 10.1155/2021/6050433] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 10/06/2021] [Indexed: 11/17/2022]
Abstract
Cardiovascular disease remains a substantial cause of morbidity and mortality in the developed world and is becoming an increasingly important cause of death in developing countries too. While current cardiovascular treatments can assist to reduce the risk of this disease, a large number of patients still retain a high risk of experiencing a life-threatening cardiovascular event. Thus, the advent of new treatments methods capable of reducing this residual risk remains an important healthcare objective. This paper proposes a deep learning-based method for section recognition of cardiac ultrasound images of critically ill cardiac patients. A convolution neural network (CNN) is used to classify the standard ultrasound video data. The ultrasound video data is parsed into a static image, and InceptionV3 and ResNet50 networks are used to classify eight ultrasound static sections, and the ResNet50 with better classification accuracy is selected as the standard network for classification. The correlation between the ultrasound video data frames is used to construct the ResNet50 + LSTM model. Next, the time-series features of the two-dimensional image sequence are extracted and the classification of the ultrasound section video data is realized. Experimental results show that the proposed cardiac ultrasound image recognition model has good performance and can meet the requirements of clinical section classification accuracy.
Collapse
Affiliation(s)
- Lingxia Zhu
- Department of Emergency and Critical Care Medicine, The Second Affiliated Hospital of Soochow University, Suzhou 215004, China
| | - Zhiping Xu
- Department of Emergency and Critical Care Medicine, The Second Affiliated Hospital of Soochow University, Suzhou 215004, China
| | - Ting Fang
- Department of Pediatric Emergency and Critical Care Medicine, Children's Hospital of Soochow University, Suzhou 215003, China
| |
Collapse
|