1
|
Sireesha V, Tallapragada VVS, Naresh M, Pradeep Kumar GV. EEG-BCI-based motor imagery classification using double attention convolutional network. Comput Methods Biomech Biomed Engin 2025; 28:581-600. [PMID: 38164118 DOI: 10.1080/10255842.2023.2298369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 11/07/2023] [Accepted: 12/13/2023] [Indexed: 01/03/2024]
Abstract
This article aims to improve and diversify signal processing techniques to execute a brain-computer interface (BCI) based on neurological phenomena observed when performing motor tasks using motor imagery (MI). The noise present in the original data, such as intermodulation noise, crosstalk, and other unwanted noise, is removed by Modify Least Mean Square (M-LMS) in the pre-processing stage. Traditional LMSs were unable to extract all the noise from the images. After pre-processing, the required features, such as statistical features, entropy features, etc., were extracted using Common Spatial Pattern (CSP) and Pearson's Correlation Coefficient (PCC) instead of the traditional single feature extraction model. The arithmetic optimization algorithm cannot select the features accurately and fails to reduce the feature dimensionality of the data. Thus, an Extended Arithmetic operation optimization (ExAo) algorithm is used to select the most significant attributes from the extracted features. The proposed model uses Double Attention Convolutional Neural Networks (DAttnConvNet) to classify the types of EEG signals based on optimal feature selection. Here, the attention mechanism is used to select and optimize the features to improve the classification accuracy and efficiency of the model. In EEG motor imagery datasets, the proposed model has been analyzed under class, which obtained an accuracy of 99.98% in class Baseline (B), 99.82% in class Imagined movement of a right fist (R) and 99.61% in class Imagined movement of both fists (RL). In the EEG dataset, the proposed model can obtain a high accuracy of 97.94% compared to EEG datasets of other models.
Collapse
Affiliation(s)
- V Sireesha
- Department of Computer Science and Engineering, School of Technology, GITAM University, Hyderabad, India
| | | | - M Naresh
- Department of ECE, Matrusri Engineering College, Saidabad, Hyderabad, India
| | - G V Pradeep Kumar
- Department of ECE, Chaitanya Bharathi Institute of Technology, Hyderabad, India
| |
Collapse
|
2
|
Xu G, Wang Z, Hu H, Zhao X, Li R, Zhou T, Xu T. Riemannian Locality Preserving Method for Transfer Learning With Applications on Brain-Computer Interface. IEEE J Biomed Health Inform 2024; 28:4565-4576. [PMID: 38758616 DOI: 10.1109/jbhi.2024.3402324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/19/2024]
Abstract
Brain-computer interfaces (BCIs) have been widely focused and extensively studied in recent years for their huge prospect of medical rehabilitation and commercial applications. Transfer learning exploits the information in the source domain and applies in another different but related domain (target domain), and is therefore introduced into the BCIs to figure out the inter-subject variances of electroencephalography (EEG) signals. In this article, a novel transfer learning method is proposed to preserve the Riemannian locality of data structure in both the source and target domains and simultaneously realize the joint distribution adaptation of both domains to enhance the effectiveness of transfer learning. Specifically, a Riemannian graph is first defined and constructed based on the Riemannian distance to represent the Riemannian geometry information. To simultaneously align the marginal and conditional distribution of source and target domains and preserve the Riemannian locality of data structure in both domains, the Riemannian graph is embedded in the joint distribution adaptation (JDA) framework and forms the proposed Riemannian locality preserving-based transfer learning (RLPTL). To validate the effect of the proposed method, it is compared with several existing methods on two open motor imagery datasets, and both multi-source domains (MSD) and single-source domains (SSD) experiments are considered. Experimental results show that the proposed method achieves the highest accuracies in MSD and SSD experiments on three datasets and outperforms eight baseline methods, which demonstrates that the proposed method creates a feasible and efficient way to realize transfer learning.
Collapse
|
3
|
Martinez-Peon D, Garcia-Hernandez NV, Benavides-Bravo FG, Parra-Vega V. Characterization and classification of kinesthetic motor imagery levels. J Neural Eng 2024; 21:046024. [PMID: 38963179 DOI: 10.1088/1741-2552/ad5f27] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 06/27/2024] [Indexed: 07/05/2024]
Abstract
Objective.Kinesthetic Motor Imagery (KMI) represents a robust brain paradigm intended for electroencephalography (EEG)-based commands in brain-computer interfaces (BCIs). However, ensuring high accuracy in multi-command execution remains challenging, with data from C3 and C4 electrodes reaching up to 92% accuracy. This paper aims to characterize and classify EEG-based KMI of multilevel muscle contraction without relying on primary motor cortex signals.Approach.A new method based on Hurst exponents is introduced to characterize EEG signals of multilevel KMI of muscle contraction from electrodes placed on the premotor, dorsolateral prefrontal, and inferior parietal cortices. EEG signals were recorded during a hand-grip task at four levels of muscle contraction (0%, 10%, 40%, and 70% of the maximal isometric voluntary contraction). The task was executed under two conditions: first, physically, to train subjects in achieving muscle contraction at each level, followed by mental imagery under the KMI paradigm for each contraction level. EMG signals were recorded in both conditions to correlate muscle contraction execution, whether correct or null accurately. Independent component analysis (ICA) maps EEG signals from the sensor to the source space for preprocessing. For characterization, three algorithms based on Hurst exponents were used: the original (HO), using partitions (HRS), and applying semivariogram (HV). Finally, seven classifiers were used: Bayes network (BN), naive Bayes (NB), support vector machine (SVM), random forest (RF), random tree (RT), multilayer perceptron (MP), and k-nearest neighbors (kNN).Main results.A combination of the three Hurst characterization algorithms produced the highest average accuracy of 96.42% from kNN, followed by MP (92.85%), SVM (92.85%), NB (91.07%), RF (91.07%), BN (91.07%), and RT (80.35%). of 96.42% for kNN.Significance.Results show the feasibility of KMI multilevel muscle contraction detection and, thus, the viability of non-binary EEG-based BCI applications without using signals from the motor cortex.
Collapse
Affiliation(s)
- D Martinez-Peon
- Department of Electrical and Electronic Engineering, National Technological Institute of Mexico (TecNM)- IT Nuevo Leon, Guadalupe, Mexico
| | - N V Garcia-Hernandez
- National Council on Science and Technology, Saltillo, Mexico
- Robotics and Advanced Manufacturing, Research Center for Advanced Studies (Cinvestav), Saltillo, Mexico
| | - F G Benavides-Bravo
- Department of Basic Sciences, National Technological Institute of Mexico (TecNM)- IT Nuevo Leon, Guadalupe, Mexico
| | - V Parra-Vega
- Robotics and Advanced Manufacturing, Research Center for Advanced Studies (Cinvestav), Saltillo, Mexico
| |
Collapse
|
4
|
Barnova K, Mikolasova M, Kahankova RV, Jaros R, Kawala-Sterniuk A, Snasel V, Mirjalili S, Pelc M, Martinek R. Implementation of artificial intelligence and machine learning-based methods in brain-computer interaction. Comput Biol Med 2023; 163:107135. [PMID: 37329623 DOI: 10.1016/j.compbiomed.2023.107135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 05/13/2023] [Accepted: 06/04/2023] [Indexed: 06/19/2023]
Abstract
Brain-computer interfaces are used for direct two-way communication between the human brain and the computer. Brain signals contain valuable information about the mental state and brain activity of the examined subject. However, due to their non-stationarity and susceptibility to various types of interference, their processing, analysis and interpretation are challenging. For these reasons, the research in the field of brain-computer interfaces is focused on the implementation of artificial intelligence, especially in five main areas: calibration, noise suppression, communication, mental condition estimation, and motor imagery. The use of algorithms based on artificial intelligence and machine learning has proven to be very promising in these application domains, especially due to their ability to predict and learn from previous experience. Therefore, their implementation within medical technologies can contribute to more accurate information about the mental state of subjects, alleviate the consequences of serious diseases or improve the quality of life of disabled patients.
Collapse
Affiliation(s)
- Katerina Barnova
- Department of Cybernetics and Biomedical Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia.
| | - Martina Mikolasova
- Department of Cybernetics and Biomedical Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia.
| | - Radana Vilimkova Kahankova
- Department of Cybernetics and Biomedical Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia
| | - Rene Jaros
- Department of Cybernetics and Biomedical Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia.
| | - Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Poland.
| | - Vaclav Snasel
- Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia.
| | - Seyedali Mirjalili
- Centre for Artificial Intelligence Research and Optimisation, Torrens University Australia, Australia.
| | - Mariusz Pelc
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Poland; School of Computing and Mathematical Sciences, University of Greenwich, London, UK.
| | - Radek Martinek
- Department of Cybernetics and Biomedical Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Czechia; Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, Poland.
| |
Collapse
|
5
|
Hu Z, Niu Q, Hsiao BS, Yao X, Zhang Y. Bioactive polymer-enabled conformal neural interface and its application strategies. MATERIALS HORIZONS 2023; 10:808-828. [PMID: 36597872 DOI: 10.1039/d2mh01125e] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Neural interface is a powerful tool to control the varying neuron activities in the brain, where the performance can directly affect the quality of recording neural signals and the reliability of in vivo connection between the brain and external equipment. Recent advances in bioelectronic innovation have provided promising pathways to fabricate flexible electrodes by integrating electrodes on bioactive polymer substrates. These bioactive polymer-based electrodes can enable the conformal contact with irregular tissue and result in low inflammation when compared to conventional rigid inorganic electrodes. In this review, we focus on the use of silk fibroin and cellulose biopolymers as well as certain synthetic polymers to offer the desired flexibility for constructing electrode substrates for a conformal neural interface. First, the development of a neural interface is reviewed, and the signal recording methods and tissue response features of the implanted electrodes are discussed in terms of biocompatibility and flexibility of corresponding neural interfaces. Following this, the material selection, structure design and integration of conformal neural interfaces accompanied by their effective applications are described. Finally, we offer our perspectives on the evolution of desired bioactive polymer-enabled neural interfaces, regarding the biocompatibility, electrical properties and mechanical softness.
Collapse
Affiliation(s)
- Zhanao Hu
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, Shanghai Engineering Research Center of Nano-Biomaterials and Regenerative Medicine, College of Materials Science and Engineering, Donghua University, Shanghai, 201620, People's Republic of China.
| | - Qianqian Niu
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, Shanghai Engineering Research Center of Nano-Biomaterials and Regenerative Medicine, College of Materials Science and Engineering, Donghua University, Shanghai, 201620, People's Republic of China.
| | - Benjamin S Hsiao
- Department of Chemistry, Stony Brook University, Stony Brook, New York, 11794-3400, USA
| | - Xiang Yao
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, Shanghai Engineering Research Center of Nano-Biomaterials and Regenerative Medicine, College of Materials Science and Engineering, Donghua University, Shanghai, 201620, People's Republic of China.
| | - Yaopeng Zhang
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, Shanghai Engineering Research Center of Nano-Biomaterials and Regenerative Medicine, College of Materials Science and Engineering, Donghua University, Shanghai, 201620, People's Republic of China.
| |
Collapse
|
6
|
Fumanal-Idocin J, Rodriguez-Martinez I, Indurain A, Minárová M, Bustince H. Almost aggregations in the gravitational clustering to perform anomaly detection. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.07.167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
7
|
Gao S, Yang J, Shen T, Jiang W. A Parallel Feature Fusion Network Combining GRU and CNN for Motor Imagery EEG Decoding. Brain Sci 2022; 12:brainsci12091233. [PMID: 36138969 PMCID: PMC9496764 DOI: 10.3390/brainsci12091233] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 08/13/2022] [Accepted: 09/09/2022] [Indexed: 11/29/2022] Open
Abstract
In recent years, deep-learning-based motor imagery (MI) electroencephalography (EEG) decoding methods have shown great potential in the field of the brain–computer interface (BCI). The existing literature is relatively mature in decoding methods for two classes of MI tasks. However, with the increase in MI task classes, decoding studies for four classes of MI tasks need to be further explored. In addition, it is difficult to obtain large-scale EEG datasets. When the training data are limited, deep-learning-based decoding models are prone to problems such as overfitting and poor robustness. In this study, we design a data augmentation method for MI-EEG. The original EEG is slid along the time axis and reconstructed to expand the size of the dataset. Second, we combine the gated recurrent unit (GRU) and convolutional neural network (CNN) to construct a parallel-structured feature fusion network to decode four classes of MI tasks. The parallel structure can avoid temporal, frequency and spatial features interfering with each other. Experimenting on the well-known four-class MI dataset BCI Competition IV 2a shows a global average classification accuracy of 80.7% and a kappa value of 0.74. The proposed method improves the robustness of deep learning to decode small-scale EEG datasets and alleviates the overfitting phenomenon caused by insufficient data. The method can be applied to BCI systems with a small amount of daily recorded data.
Collapse
|
8
|
Fumanal-Idocin J, Wang YK, Lin CT, Fernandez J, Sanz JA, Bustince H. Motor-Imagery-Based Brain-Computer Interface Using Signal Derivation and Aggregation Functions. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:7944-7955. [PMID: 34033571 DOI: 10.1109/tcyb.2021.3073210] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Brain-computer interface (BCI) technologies are popular methods of communication between the human brain and external devices. One of the most popular approaches to BCI is motor imagery (MI). In BCI applications, the electroencephalography (EEG) is a very popular measurement for brain dynamics because of its noninvasive nature. Although there is a high interest in the BCI topic, the performance of existing systems is still far from ideal, due to the difficulty of performing pattern recognition tasks in EEG signals. This difficulty lies in the selection of the correct EEG channels, the signal-to-noise ratio of these signals, and how to discern the redundant information among them. BCI systems are composed of a wide range of components that perform signal preprocessing, feature extraction, and decision making. In this article, we define a new BCI framework, called enhanced fusion framework, where we propose three different ideas to improve the existing MI-based BCI frameworks. First, we include an additional preprocessing step of the signal: a differentiation of the EEG signal that makes it time invariant. Second, we add an additional frequency band as a feature for the system: the sensorimotor rhythm band, and we show its effect on the performance of the system. Finally, we make a profound study of how to make the final decision in the system. We propose the usage of both up to six types of different classifiers and a wide range of aggregation functions (including classical aggregations, Choquet and Sugeno integrals, and their extensions and overlap functions) to fuse the information given by the considered classifiers. We have tested this new system on a dataset of 20 volunteers performing MI-based brain-computer interface experiments. On this dataset, the new system achieved 88.80% accuracy. We also propose an optimized version of our system that is able to obtain up to 90.76%. Furthermore, we find that the pair Choquet/Sugeno integrals and overlap functions are the ones providing the best results.
Collapse
|
9
|
Kwak Y, Song WJ, Kim SE. FGANet: fNIRS-guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng 2022; 30:329-339. [PMID: 35130163 DOI: 10.1109/tnsre.2022.3149899] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.
Collapse
|
10
|
Marco-Detchart C, Lucca G, Lopez-Molina C, De Miguel L, Pereira Dimuro G, Bustince H. Neuro-inspired edge feature fusion using Choquet integrals. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.10.016] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
11
|
Padfield N, Ren J, Qing C, Murray P, Zhao H, Zheng J. Multi-segment Majority Voting Decision Fusion for MI EEG Brain-Computer Interfacing. Cognit Comput 2021. [DOI: 10.1007/s12559-021-09953-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
12
|
Feng Z, Sun Y, Qian L, Qi Y, Wang Y, Guan C, Sun Y. Design a novel BCI for neurorehabilitation using concurrent LFP and EEG features: a case study. IEEE Trans Biomed Eng 2021; 69:1554-1563. [PMID: 34582344 DOI: 10.1109/tbme.2021.3115799] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Brain-computer interfaces (BCI) that enables people with severe motor disabilities to use their brain signals for direct control of objects have attracted increased interest in rehabilitation. To date, no study has investigated feasibility of the BCI framework incorporating both intracortical and scalp signals. Methods: Concurrent local field potential (LFP) from the hand-knob area and scalp EEG were recorded in a paraplegic patient undergoing a spike-based close-loop neurorehabilitation training. Based upon multimodal spatio-spectral feature extraction and Naive Bayes classification, we developed, for the first time, a novel LFP-EEG-BCI for motor intention decoding. A transfer learning (TL) approach was employed to further improve the feasibility. The performance of the proposed LFP-EEG-BCI for four-class upper-limb motor intention decoding was assessed. Results: Using a decision fusion strategy, we showed that the LFP-EEG-BCI significantly (p <0.05) outperformed single modal BCI (LFP-BCI and EEG-BCI) in terms of decoding accuracy with the best performance achieved using regularized common spatial pattern features. Interrogation of feature characteristics revealed discriminative spatial and spectral patterns, which may lead to new insights for better understanding of brain dynamics during different motor imagery tasks and promote development of efficient decoding algorithms. Moreover, we showed that similar classification performance could be obtained with few training trials, therefore highlighting the efficacy of TL. Conclusion: The present findings demonstrated the superiority of the novel LFP-EEG-BCI in motor intention decoding. Significance: This work introduced a novel LFP-EEG-BCI that may lead to new directions for developing practical neurorehabilitation systems with high detection accuracy and multi-paradigm feasibility in clinical applications.
Collapse
|
13
|
Lv Z, Qiao L, Wang Q, Piccialli F. Advanced Machine-Learning Methods for Brain-Computer Interfacing. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2021; 18:1688-1698. [PMID: 32750892 DOI: 10.1109/tcbb.2020.3010014] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The brain-computer interface (BCI) connects the brain and the external world through an information transmission channel by interpreting the physiological information of the brain during thinking activities. The effective classification of electroencephalogram (EEG) signals is the key to improving the performance of the system. To improve the classification accuracy of EEG signals in the BCI system, the transfer learning algorithm and the improved Common Spatial Pattern (CSP) algorithm are combined to construct a data classification model. Finally, the effectiveness of the proposed algorithm is verified. The results show that in actual and imagined movements, the accuracy of the left- and right-hand movements at different speeds is higher than when the speeds are the same. The proposed Adaptive Composite Common Spatial Pattern (ACCSP) and Self Adaptive Common Spatial Pattern (SACSP) algorithms have good classification effects on 5 subjects, with an average classification accuracy rate of 83.58 percent, which is an increase of 6.96 percent compared with traditional algorithms. When the training sample size is 10, the classification accuracy of the ACCSP algorithm is higher than that of the traditional CSP algorithm. The improved CSP algorithm combined with transfer learning embodies a good classification effect in both ACCSP and SACSP. Especially, the performance of SACSP mode is better. Combining the improved CSP algorithm proposed with the CSP-based transfer learning algorithm can improve the classification accuracy of the BCI classifier.
Collapse
|
14
|
Gu X, Cao Z, Jolfaei A, Xu P, Wu D, Jung TP, Lin CT. EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2021; 18:1645-1666. [PMID: 33465029 DOI: 10.1109/tcbb.2021.3052811] [Citation(s) in RCA: 93] [Impact Index Per Article: 23.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.
Collapse
|
15
|
Alhade BA, Ahmed IS, Kamil BZ. Brain Computer Interface using EEG Based Sequential Minimal Optimization algorithms. JOURNAL OF PHYSICS: CONFERENCE SERIES 2021; 1879:022092. [DOI: 10.1088/1742-6596/1879/2/022092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
Abstract
The concept of interfacing brains with robots/machines has been capturing human interests for a long time. The technology of the Brain-computer interface (BCI) has been aimed at building an interface between the brain and any electronic/electrical device (such as, smart home appliance, a wheelchair, and robotics devices) with the use of the electroencephalogram (EEG) that can be defined as a non-invasive approach for the measurement of the electrical potentials from the electrodes that have been placed on the scalp, produced by the activity of the brain. Over the past years, pattern classification was a highly challenging research field. Presently, the tasks of the pattern classification. In this paper, we chose motor imagery with the use of the single trial EEG signal, the SOM has been utilized to classify the signal processing algorithm ( FICA). In comparison to other algorithms of the EEG signal analyses. It has achieved a classification accuracy of up to 88.% in comparison with the other method where the reported accuracy has been 65%. The SOM classification algorithm has been fast, simple, efficient, and easy to use. It achieved satisfactory results at the BCI.
Collapse
|
16
|
Singanamalla SKR, Lin CT. Spiking Neural Network for Augmenting Electroencephalographic Data for Brain Computer Interfaces. Front Neurosci 2021; 15:651762. [PMID: 33867928 PMCID: PMC8047134 DOI: 10.3389/fnins.2021.651762] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2021] [Accepted: 02/22/2021] [Indexed: 11/28/2022] Open
Abstract
With the advent of advanced machine learning methods, the performance of brain–computer interfaces (BCIs) has improved unprecedentedly. However, electroencephalography (EEG), a commonly used brain imaging method for BCI, is characterized by a tedious experimental setup, frequent data loss due to artifacts, and is time consuming for bulk trial recordings to take advantage of the capabilities of deep learning classifiers. Some studies have tried to address this issue by generating artificial EEG signals. However, a few of these methods are limited in retaining the prominent features or biomarker of the signal. And, other deep learning-based generative methods require a huge number of samples for training, and a majority of these models can handle data augmentation of one category or class of data at any training session. Therefore, there exists a necessity for a generative model that can generate synthetic EEG samples with as few available trials as possible and generate multi-class while retaining the biomarker of the signal. Since EEG signal represents an accumulation of action potentials from neuronal populations beneath the scalp surface and as spiking neural network (SNN), a biologically closer artificial neural network, communicates via spiking behavior, we propose an SNN-based approach using surrogate-gradient descent learning to reconstruct and generate multi-class artificial EEG signals from just a few original samples. The network was employed for augmenting motor imagery (MI) and steady-state visually evoked potential (SSVEP) data. These artificial data are further validated through classification and correlation metrics to assess its resemblance with original data and in-turn enhanced the MI classification performance.
Collapse
Affiliation(s)
- Sai Kalyan Ranga Singanamalla
- Computational Intelligence and Brain Computer Interface Lab, School of Computer Science, University of Technology Sydney, Sydney, NSW, Australia
| | - Chin-Teng Lin
- Computational Intelligence and Brain Computer Interface Lab, School of Computer Science, University of Technology Sydney, Sydney, NSW, Australia.,Centre for Artificial Intelligence, University of Technology Sydney, Sydney, NSW, Australia
| |
Collapse
|
17
|
Deligani RJ, Borgheai SB, McLinden J, Shahriari Y. Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework. BIOMEDICAL OPTICS EXPRESS 2021; 12:1635-1650. [PMID: 33796378 PMCID: PMC7984774 DOI: 10.1364/boe.413666] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Revised: 02/15/2021] [Accepted: 02/16/2021] [Indexed: 05/26/2023]
Abstract
Multimodal data fusion is one of the current primary neuroimaging research directions to overcome the fundamental limitations of individual modalities by exploiting complementary information from different modalities. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are especially compelling modalities due to their potentially complementary features reflecting the electro-hemodynamic characteristics of neural responses. However, the current multimodal studies lack a comprehensive systematic approach to properly merge the complementary features from their multimodal data. Identifying a systematic approach to properly fuse EEG-fNIRS data and exploit their complementary potential is crucial in improving performance. This paper proposes a framework for classifying fused EEG-fNIRS data at the feature level, relying on a mutual information-based feature selection approach with respect to the complementarity between features. The goal is to optimize the complementarity, redundancy and relevance between multimodal features with respect to the class labels as belonging to a pathological condition or healthy control. Nine amyotrophic lateral sclerosis (ALS) patients and nine controls underwent multimodal data recording during a visuo-mental task. Multiple spectral and temporal features were extracted and fed to a feature selection algorithm followed by a classifier, which selected the optimized subset of features through a cross-validation process. The results demonstrated considerably improved hybrid classification performance compared to the individual modalities and compared to conventional classification without feature selection, suggesting a potential efficacy of our proposed framework for wider neuro-clinical applications.
Collapse
Affiliation(s)
- Roohollah Jafari Deligani
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - Seyyed Bahram Borgheai
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - John McLinden
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - Yalda Shahriari
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
- Interdisciplinary Neuroscience Program;
University of Rhode Island, Kingston, RI
02881, USA
| |
Collapse
|
18
|
Cao L, Chen S, Jia J, Fan C, Wang H, Xu Z. An Inter- and Intra-Subject Transfer Calibration Scheme for Improving Feedback Performance of Sensorimotor Rhythm-Based BCI Rehabilitation. Front Neurosci 2021; 14:629572. [PMID: 33584182 PMCID: PMC7876404 DOI: 10.3389/fnins.2020.629572] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2020] [Accepted: 12/21/2020] [Indexed: 01/11/2023] Open
Abstract
The Brain Computer Interface (BCI) system is a typical neurophysiological application which helps paralyzed patients with human-machine communication. Stroke patients with motor disabilities are able to perform BCI tasks for clinical rehabilitation. This paper proposes an effective scheme of transfer calibration for BCI rehabilitation. The inter- and intra-subject transfer learning approaches can improve the low-precision classification performance for experimental feedback. The results imply that the systematical scheme is positive in increasing the confidence of voluntary training for stroke patients. In addition, it also reduces the time consumption of classifier calibration.
Collapse
Affiliation(s)
- Lei Cao
- Department of Artificial Intelligence, Shanghai Maritime University, Shanghai, China
| | - Shugeng Chen
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, Shanghai, China
| | - Jie Jia
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, Shanghai, China
| | | | - Haoran Wang
- Department of Computer Science and Technology, Tongji University, Shanghai, China
| | - Zhixiong Xu
- Department of Artificial Intelligence, Shanghai Maritime University, Shanghai, China
| |
Collapse
|
19
|
Pan H, Mi W, Wen F, Zhong W. An adaptive decoder design based on the receding horizon optimization in BMI system. Cogn Neurodyn 2020; 14:281-290. [PMID: 32399071 DOI: 10.1007/s11571-019-09567-4] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2019] [Revised: 12/10/2019] [Accepted: 12/13/2019] [Indexed: 10/25/2022] Open
Abstract
In a motor brain-machine interface system, since the electroencephalogram signal is changing through out the process of the arm movement, the offline trained decoder with fixed weights is often unable to convert the electroencephalogram signal accurately, resulting in poor recovery of joint motor function. In this paper, a receding horizon optimization strategy is chosen to online update the decoder weights and design an adaptive Wiener-filter-based decoder. Firstly, a classical Wiener-filter-based decoder with fixed weights is brief reviewed. Secondly, the weights in Wiener-filter-based decoder are updated by minimizing the cost function, which is composed by the sum of squared position errors in the given horizon at each sampling time. The simulation shows that the recovery effect of joint motor function and neuron activity in the BMI system with the adaptive decoder are both better than that in the BMI system with the fixed decoder.
Collapse
Affiliation(s)
- Hongguang Pan
- 1College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an, 710054 China
| | - Wenyu Mi
- 1College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an, 710054 China
| | - Fan Wen
- 1College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an, 710054 China
| | - Weimin Zhong
- 2Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, 200237 China
| |
Collapse
|
20
|
A closed-loop brain–machine interface framework design for motor rehabilitation. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.101877] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
21
|
Chikara RK, Lo WC, Ko LW. Exploration of Brain Connectivity during Human Inhibitory Control Using Inter-Trial Coherence. SENSORS 2020; 20:s20061722. [PMID: 32204504 PMCID: PMC7147711 DOI: 10.3390/s20061722] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/11/2020] [Revised: 03/11/2020] [Accepted: 03/16/2020] [Indexed: 11/16/2022]
Abstract
Inhibitory control is a cognitive process that inhibits a response. It is used in everyday activities, such as driving a motorcycle, driving a car and playing a game. The effect of this process can be compared to the red traffic light in the real world. In this study, we investigated brain connectivity under human inhibitory control using the phase lag index and inter-trial coherence (ITC). The human brain connectivity gives a more accurate representation of the functional neural network. Results of electroencephalography (EEG), the data sets were generated from twelve healthy subjects during left and right hand inhibitions using the auditory stop-signal task, showed that the inter-trial coherence in delta (1-4 Hz) and theta (4-7 Hz) band powers increased over the frontal and temporal lobe of the brain. These EEG delta and theta band activities neural markers have been related to human inhibition in the frontal lobe. In addition, inter-trial coherence in the delta-theta and alpha (8-12 Hz) band powers increased at the occipital lobe through visual stimulation. Moreover, the highest brain connectivity was observed under inhibitory control in the frontal lobe between F3-F4 channels compared to temporal and occipital lobes. The greater EEG coherence and phase lag index in the frontal lobe is associated with the human response inhibition. These findings revealed new insights to understand the neural network of brain connectivity and underlying mechanisms during human response inhibition.
Collapse
Affiliation(s)
- Rupesh Kumar Chikara
- Department of Biological Science and Technology, College of Biological Science and Technology, National Chiao Tung University, Hsinchu 300, Taiwan;
- Center For Intelligent Drug Systems and Smart Bio-devices (IDS2B), National Chiao Tung University, Hsinchu 300, Taiwan
| | - Wei-Cheng Lo
- Department of Biological Science and Technology, College of Biological Science and Technology, National Chiao Tung University, Hsinchu 300, Taiwan;
- Institute of Bioinformatics and Systems Biology, National Chiao Tung University, Hsinchu 300, Taiwan
- Correspondence: (W.-C.L.); (L.-W.K.)
| | - Li-Wei Ko
- Department of Biological Science and Technology, College of Biological Science and Technology, National Chiao Tung University, Hsinchu 300, Taiwan;
- Center For Intelligent Drug Systems and Smart Bio-devices (IDS2B), National Chiao Tung University, Hsinchu 300, Taiwan
- Institute of Bioinformatics and Systems Biology, National Chiao Tung University, Hsinchu 300, Taiwan
- The Drug Development and Value Creation Research Center, Kaohsiung Medical University, Kaohsiung 80708, Taiwan
- Correspondence: (W.-C.L.); (L.-W.K.)
| |
Collapse
|
22
|
Brain Computer Interface-Based Action Observation Game Enhances Mu Suppression in Patients with Stroke. ELECTRONICS 2019. [DOI: 10.3390/electronics8121466] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Action observation (AO), based on the mirror neuron theory, is a promising strategy to promote motor cortical activation in neurorehabilitation. Brain computer interface (BCI) can detect a user’s intention and provide them with brain state-dependent feedback to assist with patient rehabilitation. We investigated the effects of a combined BCI-AO game on power of mu band attenuation in stroke patients. Nineteen patients with subacute stroke were recruited. A BCI-AO game provided real-time feedback to participants regarding their attention to a flickering action video using steady-state visual-evoked potentials. All participants watched a video of repetitive grasping actions under two conditions: (1) BCI-AO game and (2) conventional AO, in random order. In the BCI-AO game, feedback on participants’ observation scores and observation time was provided. In conventional AO, a non-flickering video and no feedback were provided. The magnitude of mu suppression in the central motor, temporal, parietal, and occipital areas was significantly higher in the BCI-AO game than in the conventional AO. The magnitude of mu suppression was significantly higher in the BCI-AO game than in the conventional AO both in the affected and unaffected hemispheres. These results support the facilitatory effects of the BCI-AO game on mu suppression over conventional AO.
Collapse
|
23
|
Zhao X, Zhang H, Zhu G, You F, Kuang S, Sun L. A Multi-Branch 3D Convolutional Neural Network for EEG-Based Motor Imagery Classification. IEEE Trans Neural Syst Rehabil Eng 2019; 27:2164-2177. [DOI: 10.1109/tnsre.2019.2938295] [Citation(s) in RCA: 93] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
24
|
Neural Activities Classification of Human Inhibitory Control Using Hierarchical Model. SENSORS 2019; 19:s19173791. [PMID: 31480570 PMCID: PMC6749522 DOI: 10.3390/s19173791] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/14/2019] [Revised: 08/18/2019] [Accepted: 08/29/2019] [Indexed: 11/30/2022]
Abstract
Human inhibitory control refers to the suppression of behavioral response in real environments, such as when driving a car or riding a motorcycle, playing a game and operating a machine. The P300 wave is a neural marker of human inhibitory control, and it can be used to recognize the symptoms of attention deficit hyperactivity disorder (ADHD) in human. In addition, the P300 neural marker can be considered as a stop command in the brain-computer interface (BCI) technologies. Therefore, the present study of electroencephalography (EEG) recognizes the mindset of human inhibition by observing the brain dynamics, like P300 wave in the frontal lobe, supplementary motor area, and in the right temporoparietal junction of the brain, all of them have been associated with response inhibition. Our work developed a hierarchical classification model to identify the neural activities of human inhibition. To accomplish this goal phase-locking value (PLV) method was used to select coupled brain regions related to inhibition because this method has demonstrated the best performance of the classification system. The PLVs were used with pattern recognition algorithms to classify a successful-stop versus a failed-stop in left-and right-hand inhibitions. The results demonstrate that quadratic discriminant analysis (QDA) yielded an average classification accuracy of 94.44%. These findings implicate the neural activities of human inhibition can be utilized as a stop command in BCI technologies, as well as to identify the symptoms of ADHD patients in clinical research.
Collapse
|
25
|
Ko LW, Komarov O, Lin SC. Enhancing the Hybrid BCI Performance With the Common Frequency Pattern in Dual-Channel EEG. IEEE Trans Neural Syst Rehabil Eng 2019; 27:1360-1369. [PMID: 31180893 DOI: 10.1109/tnsre.2019.2920748] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
The brain-computer interface establishes a direct communication pathway between the human brain and an external device by recognizing specific patterns in cortical activities. The principle of hybridization stands for combining at least two different BCI modalities into a single interface with the aim of improving the information transfer rate by increasing the recognition accuracy and number of choices available for the user. This study proposes a simultaneous hybrid BCI system that recognizes the motor imagery (MI) and the steady-state visually evoked potentials (SSVEP) using the EEG signals from a dual-channel EEG setting with sensors placed over the central area (C3 and C4 channels). The data processing implements a supervised optimization algorithm for the feature extraction, named the common frequency pattern, which finds the optimal spectral filter that maximizes the separability of the data by classes. The experiment compares the classification accuracy in a two-class task using the MI, SSVEP and hybrid approaches on seventeen healthy 18-29 years old subjects with various dual-channel setups and complete set of thirty EEG electrodes. The designed system reaches a high accuracy of 97.4 ± 1.1% in the hybrid task using the C3-C4 channel configuration, which is marginally lower than the 98.8 ± 0.5% accuracy achieved with the complete set of channels while applying the support vector classifier; in the plain SSVEP task the accuracy drops from 91.3 ± 3.9% to 86.0 ± 2.5% while moving from the occipital to central area under the dual-channel condition. The results demonstrate that by combining the principles of hybridization and data-driven spectral filtering for the feature selection it is feasible to compensate a lack of spatial information and implement the proposed BCI using a portable few channel EEG device even under sub-optimal conditions for the sensors placement.
Collapse
|