Cross-Subject Emotion Recognition Brain-Computer Interface Based on fNIRS and DBJNet

EEG-fNIRS-Based Emotion Recognition Using Graph Convolution and Capsule Attention Network

Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) can objectively reflect a person's emotional state and have been widely studied in emotion recognition. However, the effective feature fusion and discriminative feature learning from EEG-fNIRS data is challenging. In order to improve the accuracy of emotion recognition, a graph convolution and capsule attention network model (GCN-CA-CapsNet) is proposed. Firstly, EEG-fNIRS signals are collected from 50 subjects induced by emotional video clips. And then, the features of the EEG and fNIRS are extracted; the EEG-fNIRS features are fused to generate higher-quality primary capsules by graph convolution with the Pearson correlation adjacency matrix. Finally, the capsule attention module is introduced to assign different weights to the primary capsules, and higher-quality primary capsules are selected to generate better classification capsules in the dynamic routing mechanism. We validate the efficacy of the proposed method on our emotional EEG-fNIRS dataset with an ablation study. Extensive experiments demonstrate that the proposed GCN-CA-CapsNet method achieves a more satisfactory performance against the state-of-the-art methods, and the average accuracy can increase by 3-11%.

Evolving brain network dynamics in early childhood: Insights from modular graph metrics

Modular dynamic graph theory metrics effectively capture the patterns of dynamic information interaction during human brain development. While existing research has employed modular algorithms to examine the overall impact of dynamic changes in community structure throughout development, there is a notable gap in understanding the cross-community dynamic changes within different functional networks during early childhood and their potential contributions to the efficiency of brain information transmission. This study seeks to address this gap by tracing the trajectories of cross-community structural changes within early childhood functional networks and modeling their contributions to information transmission efficiency. We analyzed 194 functional imaging scans from 83 children aged 2 to 8 years, who participated in passive viewing functional magnetic resonance imaging sessions. Utilizing sliding windows and modular algorithms, we evaluated three spatiotemporal metrics-temporal flexibility, spatiotemporal diversity, and within-community spatiotemporal diversity-and four centrality metrics: within-community degree centrality, eigenvector centrality, between-community degree centrality, and between-community eigenvector centrality. Mixed-effects linear models revealed significant age-related increases in the temporal flexibility of the default mode network (DMN), executive control network (ECN), and salience network (SN), indicating frequent adjustments in community structure within these networks during early childhood. Additionally, the spatiotemporal diversity of the SN also displayed significant age-related increases, highlighting its broad pattern of cross-community dynamic interactions. Conversely, within-community spatiotemporal diversity in the language network exhibited significant age-related decreases, reflecting the network's gradual functional specialization. Furthermore, our findings indicated significant age-related increases in between-community degree centrality across the DMN, ECN, SN, language network, and dorsal attention network, while between-community eigenvector centrality also increased significantly for the DMN, ECN, and SN. However, within-community eigenvector centrality remained stable across all functional networks during early childhood. These results suggest that while centrality of cross-community interactions in early childhood functional networks increases, centrality within communities remains stable. Finally, mediation analysis was conducted to explore the relationships between age, brain dynamic graph metrics, and both global and local efficiency based on community structure. The results indicated that the dynamic graph metrics of the SN primarily mediated the relationship between age and the decrease in global efficiency, while those of the DMN, language network, ECN, dorsal attention network, and SN primarily mediated the relationship between age and the increase in local efficiency. This pattern suggests a developmental trajectory in early childhood from global information integration to local information segregation, with the SN playing a pivotal role in this transformation. This study provides novel insights into the mechanisms by which early childhood brain functional development impacts information transmission efficiency through cross-community adjustments in functional networks.

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism

Brain Computer Interface (BCI) is a highly promising human-computer interaction method that can utilize brain signals to control external devices. BCI based on functional near-infrared spectroscopy (fNIRS) is considered a relatively new and promising paradigm. fNIRS is a technique of measuring functional changes in cerebral hemodynamics. It detects changes in the hemodynamic activity of the cerebral cortex by measuring oxyhemoglobin and deoxyhemoglobin (HbR) concentrations and inversely predicts the neural activity of the brain. At the present time, Deep learning (DL) methods have not been widely used in fNIRS decoding, and there are fewer studies considering both spatial and temporal dimensions for fNIRS classification. To solve these problems, we proposed an end-to-end hybrid neural network for feature extraction of fNIRS. The method utilizes a spatial-temporal convolutional layer for automatic extraction of temporally valid information and uses a spatial attention mechanism to extract spatially localized information. A temporal convolutional network (TCN) is used to further utilize the temporal information of fNIRS before the fully connected layer. We validated our approach on a publicly available dataset including 29 subjects, including left-hand and right-hand motor imagery (MI), mental arithmetic (MA), and a baseline task. The results show that the method has few training parameters and high accuracy, providing a meaningful reference for BCI development.

Cross-Subject Motor Imagery Decoding by Transfer Learning of Tactile ERD

For Brain-Computer Interface (BCI) based on motor imagery (MI), the MI task is abstract and spontaneous, presenting challenges in measurement and control and resulting in a lower signal-to-noise ratio. The quality of the collected MI data significantly impacts the cross-subject calibration results. To address this challenge, we introduce a novel cross-subject calibration method based on passive tactile afferent stimulation, in which data induced by tactile stimulation is utilized to calibrate transfer learning models for cross-subject decoding. During the experiments, tactile stimulation was applied to either the left or right hand, with subjects only required to sense tactile stimulation. Data from these tactile tasks were used to train or fine-tune models and subsequently applied to decode pure MI data. We evaluated BCI performance using both the classical Common Spatial Pattern (CSP) combined with the Linear Discriminant Analysis (LDA) algorithm and a state-of-the-art deep transfer learning model. The results demonstrate that the proposed calibration method achieved decoding performance at an equivalent level to traditional MI calibration, with the added benefit of outperforming traditional MI calibration with fewer trials. The simplicity and effectiveness of the proposed cross-subject tactile calibration method make it valuable for practical applications of BCI, especially in clinical settings.

A 4-Channel Neural Stimulation IC Design With Charge Balancing and Multiple Current Output Modes

This article proposes a neural stimulation integrated circuit design with multiple current output modes. In the cathodic stimulation phase and anodic stimulation phase, each output current waveform can be independently selected to either exponential waveform or square wave, so the stimulator holds four stimulation modes. To minimize the headroom voltage of the output stage and enhance the power efficiency of the proposed stimulator, we introduce the exponentially decaying current which is realized by the exponential current generation circuit in this work. It can enhance the longer duration of the stimulation pulse as well. In case the residual charge may cause harm to patients, a charge balancing technique is implemented in this work for all operation modes. The four-channel stimulator IC is implemented in a 180-nm CMOS process, occupying a core area of 1.93 mm2. The measurement results show that the proposed stimulator realized a maximum power efficiency of 91.3% and the maximum stimulation duration is 3 times larger than previous works. Moreover, even in exponential output waveform mode, the maximum residual charge in a single cycle is only 255 pC due to the proposed charge balancing technique. The experiment results based on the PBS solution also show that the stimulator IC can remove residual charges within 60 μs, and the electrode voltage remains stable within a safe range under multicycle stimulation.