A novel classification framework using multiple bandwidth method with optimized CNN for brain–computer interfaces with EEG-fNIRS signals

Temporal attention fusion network with custom loss function for EEG-fNIRS classification

Objective.Methods that can detect brain activities accurately are crucial owing to the increasing prevalence of neurological disorders. In this context, a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offers a powerful approach to understanding normal and pathological brain functions, thereby overcoming the limitations of each modality, such as susceptibility to artifacts of EEG and limited temporal resolution of fNIRS. However, challenges such as class imbalance and inter-class variability within multisubject data hinder their full potential.Approach.To address this issue, we propose a novel temporal attention fusion network (TAFN) with a custom loss function. The TAFN model incorporates attention mechanisms to its long short-term memory and temporal convolutional layers to accurately capture spatial and temporal dependencies in the EEG-fNIRS data. The custom loss function combines class weights and asymmetric loss terms to ensure the precise classification of cognitive and motor intentions, along with addressing class imbalance issues.Main results.Rigorous testing demonstrated the exceptional cross-subject accuracy of the TAFN, exceeding 99% for cognitive tasks and 97% for motor imagery (MI) tasks. Additionally, the ability of the model to detect subtle differences in epilepsy was analyzed using scalp topography in MI tasks.Significance.This study presents a technique that outperforms traditional methods for detecting high-precision brain activity with subtle differences in the associated patterns. This makes it a promising tool for applications such as epilepsy and seizure detection, in which discerning subtle pattern differences is of paramount importance.

A hybrid capsule attention-based convolutional bi-GRU method for multi-class mental task classification based brain-computer Interface

Electroencephalography analysis is critical for brain computer interface research. The primary goal of brain-computer interface is to establish communication between impaired people and others via brain signals. The classification of multi-level mental activities using the brain-computer interface has recently become more difficult, which affects the accuracy of the classification. However, several deep learning-based techniques have attempted to identify mental tasks using multidimensional data. The hybrid capsule attention-based convolutional bidirectional gated recurrent unit model was introduced in this study as a hybrid deep learning technique for multi-class mental task categorization. Initially, the obtained electroencephalography data is pre-processed with a digital low-pass Butterworth filter and a discrete wavelet transform to remove disturbances. The spectrally adaptive common spatial pattern is used to extract characteristics from pre-processed electroencephalography data. The retrieved features were then loaded into the suggested classification model, which was used to extract the features deeply and classify the mental tasks. To improve classification results, the model's parameters are fine-tuned using a dung beetle optimization approach. Finally, the proposed classifier is assessed for several types of mental task classification using the provided dataset. The simulation results are compared with the existing state-of-the-art techniques in terms of accuracy, precision, recall, etc. The accuracy obtained using the proposed approach is 97.87%, which is higher than that of the other existing methods.

Correlation-Filter-Based Channel and Feature Selection Framework for Hybrid EEG-fNIRS BCI Applications

The proposed study is based on a feature and channel selection strategy that uses correlation filters for brain-computer interface (BCI) applications using electroencephalography (EEG)-functional near-infrared spectroscopy (fNIRS) brain imaging modalities. The proposed approach fuses the complementary information of the two modalities to train the classifier. The channels most closely correlated with brain activity are extracted using a correlation-based connectivity matrix for fNIRS and EEG separately. Furthermore, the training vector is formed through the identification and fusion of the statistical features of both modalities (i.e., slope, skewness, maximum, skewness, mean, and kurtosis). The constructed fused feature vector is passed through various filters (including ReliefF, minimum redundancy maximum relevance, chi-square test, analysis of variance, and Kruskal-Wallis filters) to remove redundant information before training. Traditional classifiers such as neural networks, support-vector machines, linear discriminant analysis, and ensembles were used for the purpose of training and testing. A publicly available dataset with motor imagery information was used for validation of the proposed approach. Our findings indicate that the proposed correlation-filter-based channel and feature selection framework significantly enhances the classification accuracy of hybrid EEG-fNIRS. The ReliefF-based filter outperformed other filters with the ensemble classifier with a high accuracy of 94.77 ± 4.26%. The statistical analysis also validated the significance (p < 0.01) of the results. A comparison of the proposed framework with the prior findings was also presented. Our results show that the proposed approach can be used in future EEG-fNIRS-based hybrid BCI applications.

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

Motor imagery (MI) electroencephalogram (EEG) signal is recognized as a promising paradigm for brain-computer interface (BCI) systems and has been extensively employed in various BCI applications, including assisting disabled individuals, controlling devices and environments, and enhancing human capabilities. The high-performance decoding capability of MI-EEG signals is a key issue that impacts the development of the industry. However, decoding MI-EEG signals is challenging due to the low signal-to-noise ratio and inter-subject variability. In response to the aforementioned core problems, this paper proposes a novel end-to-end network, a fusion multi-branch 1D convolutional neural network (EEG-FMCNN), to decode MI-EEG signals without pre-processing. The utilization of multi-branch 1D convolution not only exhibits a certain level of noise tolerance but also addresses the issue of inter-subject variability to some extent. This is attributed to the ability of multi-branch architectures to capture information from different frequency bands, enabling the establishment of optimal convolutional scales and depths. Furthermore, we incorporate 1D squeeze-and-excitation (SE) blocks and shortcut connections at appropriate locations to further enhance the generalization and robustness of the network. In the BCI Competition IV-2a dataset, our proposed model has obtained good experimental results, achieving accuracies of 78.82% and 68.41% for subject-dependent and subject-independent modes, respectively. In addition, extensive ablative experiments and fine-tuning experiments were conducted, resulting in a notable 7% improvement in the average performance of the network, which holds significant implications for the generalization and application of the network.

A hybrid deep leaning model for prediction and parametric sensitivity analysis of noise annoyance

Noise annoyance is recognized as an expression of physiological and psychological strain in acoustical environment. The studies on prediction of noise annoyance and parametric sensitivity analysis of factors affecting it have been rarely reported in India. A hybrid ConvLSTM technique was developed in the study to predict traffic-induced noise annoyance in 484 people based on ambient noise levels, as well as survey information. Ambient noise levels were obtained at different locations of Dhanbad city using sound level meter at varying intervals, viz. 09AM-12PM, 03PM-06PM, and 08PM-11PM. The proposed method was compared with some well-known neural network techniques such as K-nearest neighbors (KNN), artificial neural network (ANN), recurrent neural network (RNN), and long-short-term memory (LSTM). The experimental results indicate that the proposed method outperforms other techniques and can be a reliable approach for prediction of noise annoyance with an accuracy of 93.8%. It can be concluded from noise maps that the noise levels in all locations of the Dhanbad city were higher than 70 dB(A) and noise sensitivity is the most important input variable of traffic-induced noise annoyance, followed by honking noise, education, exposure hours, L_Aeq, sleeping disorder, and chronic disease. The study shall facilitate in developing a decision support tool for prediction of noise annoyance and promoting implementation of suitable public policy in urban cities.

Deep neural network to differentiate brain activity between patients with euthymic bipolar disorders and healthy controls during verbal fluency performance: A multichannel near-infrared spectroscopy study

In this study, we aimed to differentiate between euthymic bipolar disorder (BD) patients and healthy controls (HC) based on frontal activity measured by fNIRS that were converted to spectrograms with Convolutional Neural Networks (CNN). And also, we investigated brain regions that cause this distinction. In total, 29 BD patients and 28 HCs were recruited. Their brain cortical activities were measured using fNIRS while performing letter versions of VFT. Each one of the 24 fNIRS channels was converted to a 2D spectrogram on which a CNN architecture was designed and utilized for classification. We found that our CNN algorithm using fNIRS activity during a VFT is able to differentiate subjects with BD from healthy controls with 90% accuracy, 80% sensitivity, and 100% specificity. Moreover, validation performance reached an AUC of 94%. From our individual channel analyses, we observed channels corresponding to the left inferior frontal gyrus (left-IFC), medial frontal cortex (MFC), right dorsolateral prefrontal cortex (DLPFC), Broca area, and right premotor have considerable activity variation to distinguish patients from HC. fNIRS activity during VFT can be used as a potential marker to classify euthymic BD patients from HCs. Activity particularly in the MFC, left-IFC, Broca's area, and DLPFC have a considerable variation to distinguish patients from healthy controls.

Improved classification performance of EEG-fNIRS multimodal brain-computer interface based on multi-domain features and multi-level progressive learning

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have potentially complementary characteristics that reflect the electrical and hemodynamic characteristics of neural responses, so EEG-fNIRS-based hybrid brain-computer interface (BCI) is the research hotspots in recent years. However, current studies lack a comprehensive systematic approach to properly fuse EEG and fNIRS data and exploit their complementary potential, which is critical for improving BCI performance. To address this issue, this study proposes a novel multimodal fusion framework based on multi-level progressive learning with multi-domain features. The framework consists of a multi-domain feature extraction process for EEG and fNIRS, a feature selection process based on atomic search optimization, and a multi-domain feature fusion process based on multi-level progressive machine learning. The proposed method was validated on EEG-fNIRS-based motor imagery (MI) and mental arithmetic (MA) tasks involving 29 subjects, and the experimental results show that multi-domain features provide better classification performance than single-domain features, and multi-modality provides better classification performance than single-modality. Furthermore, the experimental results and comparison with other methods demonstrated the effectiveness and superiority of the proposed method in EEG and fNIRS information fusion, it can achieve an average classification accuracy of 96.74% in the MI task and 98.42% in the MA task. Our proposed method may provide a general framework for future fusion processing of multimodal brain signals based on EEG-fNIRS.

A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity

The third type of neural network called spiking is developed due to a more accurate representation of neuronal activity in living organisms. Spiking neural networks have many different parameters that can be difficult to adjust manually to the current classification problem. The analysis and selection of coefficients’ values in the network can be analyzed as an optimization problem. A practical method for automatic selection of them can decrease the time needed to develop such a model. In this paper, we propose the use of a heuristic approach to analyze and select coefficients with the idea of collaborative working. The proposed idea is based on parallel analyzing of different coefficients and choosing the best of them or average ones. This type of optimization problem allows the selection of all variables, which can significantly affect the convergence of the accuracy. Our proposal was tested using network simulators and popular databases to indicate the possibilities of the described approach. Five different heuristic algorithms were tested and the best results were reached by Cuckoo Search Algorithm, Grasshopper Optimization Algorithm, and Polar Bears Algorithm.