Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI

Integrating fMRI spatial network dynamics and EEG spectral power: insights into resting state connectivity

Introduction

The Integration of functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) has allowed for a novel exploration of the brain's spatial-temporal resolution. While functional brain networks show variations in both spatial and temporal dimensions, most studies focus on fixed spatial networks that change together over time.

Methods

In this study, for the first time, we link spatially dynamic brain networks with EEG spectral properties recorded simultaneously, which allows us to concurrently capture high spatial and temporal resolutions offered by these complementary imaging modalities. We estimated time-resolved brain networks using sliding window-based spatially constrained independent component analysis (scICA), producing resting brain networks that evolved over time at the voxel level. Next, we assessed their coupling with four time-varying EEG spectral power (delta, theta, alpha, and beta).

Results

Our analysis demonstrated how the networks' volumes and their voxel-level activities vary over time and revealed significant correlations with time-varying EEG spectral power. For instance, we found a strong association between increasing volume of the primary visual network and alpha band power, consistent with our hypothesis for eyes open resting state scan. Similarly, the alpha, theta, and delta power of the Pz electrode were localized to voxel-level activities of primary visual, cerebellum, and temporal networks, respectively. We also identified a strong correlation between the primary motor network and alpha (mu rhythm) and beta activity. This is consistent with motor tasks during rest, though this remains to be tested directly.

Discussion

These association between space and frequency observed during rest offer insights into the brain's spatial-temporal characteristics and enhance our understanding of both spatially varying fMRI networks and EEG band power.

Application status and prospects of multimodal EEG-fMRI in HIV-associated neurocognitive disorders

HIV-associated neurocognitive disorders (HAND) are one of the common complications in people living with HIV (PLWH), which can affect their attention, working memory, and other related cognitive functions. With the widespread use of combination antiretroviral therapy (cART), the incidence of HAND has declined. However, HAND is still an important complication of HIV, which not only affects the quality of life of patients but also affects their adherence to HIV treatment. Its diagnosis mainly relies on neurocognitive tests, which have a certain degree of subjectivity, making it difficult to diagnose and classify HAND accurately, and there is an urgent need to explore more sensitive biomarkers. Multimodal brain imaging has seen a surge in recent years with simultaneous EEG-fMRI being at the forefront of cognitive multimodal neuroimaging. It is a complementary fusion technique that effectively combines the high spatial resolution of fMRI with the high temporal resolution of EEG, compensating for the shortcomings of a single technique and providing a new method for studying cognitive function. It is expected to reveal the underlying mechanisms of HAND and provide high spatiotemporal warning biomarkers of HAND, which will provide a new perspective for the early diagnosis and treatment of HAND and contribute to the improvement of patient prognosis.

Distinct basal ganglia contributions to learning from implicit and explicit value signals in perceptual decision-making

Metacognitive evaluations of confidence provide an estimate of decision accuracy that could guide learning in the absence of explicit feedback. We examine how humans might learn from this implicit feedback in direct comparison with that of explicit feedback, using simultaneous EEG-fMRI. Participants performed a motion direction discrimination task where stimulus difficulty was increased to maintain performance, with intermixed explicit- and no-feedback trials. We isolate single-trial estimates of post-decision confidence using EEG decoding, and find these neural signatures re-emerge at the time of feedback together with separable signatures of explicit feedback. We identified these signatures of implicit versus explicit feedback along a dorsal-ventral gradient in the striatum, a finding uniquely enabled by an EEG-fMRI fusion. These two signals appear to integrate into an aggregate representation in the external globus pallidus, which could broadcast updates to improve cortical decision processing via the thalamus and insular cortex, irrespective of the source of feedback.

Mapping the Neural Basis of Neuroeconomics with Functional Magnetic Resonance Imaging: A Narrative Literature Review

Neuroeconomics merges neuroscience, economics, and psychology to investigate the neural basis of decision making. Decision making involves assessing outcomes with subjective value, shaped by emotions and experiences, which are crucial in economic decisions. Functional MRI (fMRI) reveals key areas of the brain, including the ventro-medial prefrontal cortex, that are involved in subjective value representation. Collaborative interdisciplinary efforts are essential for advancing the field of neuroeconomics, with implications for clinical interventions and policy design. This review explores subjective value in neuroeconomics, highlighting brain regions identified through fMRI studies.

Subject-Specific Modeling of EEG-fNIRS Neurovascular Coupling by Task-Related Tensor Decomposition

Neurovascular coupling (NVC) connects neural activity with hemodynamics and plays a vital role in sustaining brain function. Combining electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) is a promising way to explore the NVC. However, the high-order property of EEG data and variability of hemodynamic response function (HRF) across subjects have not been well considered in existing NVC studies. In this study, we proposed a novel framework to enhance the subject-specific parametric modeling of NVC from simultaneous EEG-fNIRS measurement. Specifically, task-related tensor decomposition of high-order EEG data was performed to extract the underlying connections in the temporal-spectral-spatial structures of EEG activities and identify the most relevant temporal signature within multiple trials. Subject-specific HRFs were estimated by parameters optimization of a double gamma function model. A canonical motor task experiment was designed to induce neural activity and validate the effectiveness of the proposed framework. The results indicated that the proposed framework significantly improves the reproducibility of EEG components and the correlation between the predicted hemodynamic activities and the real fNIRS signal. Moreover, the estimated parameters characterized the NVC differences in the task with two speeds. Therefore, the proposed framework provides a feasible solution for the quantitative assessment of the NVC function.

Exploring changes in brain function in IBD patients using SPCCA: a study of simultaneous EEG-fMRI

Research on functional changes in the brain of inflammatory bowel disease (IBD) patients is emerging around the world, which brings new perspectives to medical research. In this paper, the methods of canonical correlation analysis (CCA), kernel canonical correlation analysis (KCCA), and sparsity preserving canonical correlation analysis (SPCCA) were applied to the fusion of simultaneous EEG-fMRI data from 25 IBD patients and 15 healthy individuals. The CCA, KCCA and SPCCA fusion methods were used for data processing to compare the results obtained by the three methods. The results clearly show that there is a significant difference in the activation intensity between IBD and healthy control (HC), not only in the frontal lobe (p < 0.01) and temporal lobe (p < 0.01) regions, but also in the posterior cingulate gyrus (p < 0.01), gyrus rectus (p < 0.01), and amygdala (p < 0.01) regions, which are usually neglected. The mean difference in the SPCCA activation intensity was 60.1. However, the mean difference in activation intensity was only 36.9 and 49.8 by using CCA and KCCA. In addition, the correlation of the relevant components selected during the SPCCA calculation was high, with correlation components of up to 0.955; alternatively, the correlations obtained from CCA and KCCA calculations were only 0.917 and 0.926, respectively. It can be seen that SPCCA is indeed superior to CCA and KCCA in processing high-dimensional multimodal data. This work reveals the process of analyzing the brain activation state in IBD disease, provides a further perspective for the study of brain function, and opens up a new avenue for studying the SPCCA method and the change in the intensity of brain activation in IBD disease.

Distinct spatiotemporal brainstem pathways of outcome valence during reward- and punishment-based learning

Learning to seek rewards and avoid punishments, based on positive and negative choice outcomes, is essential for human survival. Yet, the neural underpinnings of outcome valence in the human brainstem and the extent to which they differ in reward and punishment learning contexts remain largely elusive. Here, using simultaneously acquired electroencephalography and functional magnetic resonance imaging data, we show that during reward learning the substantia nigra (SN)/ventral tegmental area (VTA) and locus coeruleus are initially activated following negative outcomes, while the VTA subsequently re-engages exhibiting greater responses for positive than negative outcomes, consistent with an early arousal/avoidance response and a later value-updating process, respectively. During punishment learning, we show that distinct raphe nucleus and SN subregions are activated only by negative outcomes with a sustained post-outcome activity across time, supporting the involvement of these brainstem subregions in avoidance behavior. Finally, we demonstrate that the coupling of these brainstem structures with other subcortical and cortical areas helps to shape participants' serial choice behavior in each context.

Transcriptomic and Macroscopic Architectures of Multimodal Covariance Network Reveal Molecular-Structural-Functional Co-alterations

Human cognition is usually underpinned by intrinsic structure and functional neural co-activation in spatially distributed brain regions. Owing to lacking an effective approach to quantifying the covarying of structure and functional responses, how the structural-functional circuits interact and how genes encode the relationships, to deepen our knowledge of human cognition and disease, are still unclear. Here, we propose a multimodal covariance network (MCN) construction approach to capture interregional covarying of the structural skeleton and transient functional activities for a single individual. We further explored the potential association between brain-wide gene expression patterns and structural-functional covarying in individuals involved in a gambling task and individuals with major depression disorder (MDD), adopting multimodal data from a publicly available human brain transcriptomic atlas and 2 independent cohorts. MCN analysis showed a replicable cortical structural-functional fine map in healthy individuals, and the expression of cognition- and disease phenotype-related genes was found to be spatially correlated with the corresponding MCN differences. Further analysis of cell type-specific signature genes suggests that the excitatory and inhibitory neuron transcriptomic changes could account for most of the observed correlation with task-evoked MCN differences. In contrast, changes in MCN of MDD patients were enriched for biological processes related to synapse function and neuroinflammation in astrocytes, microglia, and neurons, suggesting its promising application in developing targeted therapies for MDD patients. Collectively, these findings confirmed the correlations of MCN-related differences with brain-wide gene expression patterns, which captured genetically validated structural-functional differences at the cellular level in specific cognitive processes and psychiatric patients.

Pupillary response is associated with the reset and switching of functional brain networks during salience processing

The interface between processing internal goals and salient events in the environment involves various top-down processes. Previous studies have identified multiple brain areas for salience processing, including the salience network (SN), dorsal attention network, and the locus coeruleus-norepinephrine (LC-NE) system. However, interactions among these systems in salience processing remain unclear. Here, we simultaneously recorded pupillometry, EEG, and fMRI during an auditory oddball paradigm. The analyses of EEG and fMRI data uncovered spatiotemporally organized target-associated neural correlates. By modeling the target-modulated effective connectivity, we found that the target-evoked pupillary response is associated with the network directional couplings from late to early subsystems in the trial, as well as the network switching initiated by the SN. These findings indicate that the SN might cooperate with the pupil-indexed LC-NE system in the reset and switching of cortical networks, and shed light on their implications in various cognitive processes and neurological diseases.

An EEG-fNIRS neurovascular coupling analysis method to investigate cognitive-motor interference

BACKGROUND AND OBJECTIVE

The simultaneous execution of a motor and cognitive dual task may lead to the deterioration of task performance in one or both tasks due to cognitive-motor interference (CMI). Neuroimaging techniques are promising ways to reveal the underlying neural mechanism of CMI. However, existing studies have only explored CMI from a single neuroimaging modality, which lack built-in validation and comparison of analysis results. This work is aimed to establish an effective analysis framework to comprehensively investigate the CMI by exploring the electrophysiological and hemodynamic activities as well as their neurovascular coupling.

METHODS

Experiments including an upper limb single motor task, single cognitive task, and cognitive-motor dual task were designed and performed with 16 healthy young participants. Bimodal signals of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were recorded simultaneously during the experiments. A novel bimodal signal analysis framework was proposed to extract the task-related components for EEG and fNIRS signals respectively and analyze their correlation. Indicators including within-class similarity and between-class distance were utilized to validate the effectiveness of the proposed analysis framework compared to the canonical channel-averaged method. Statistical analysis was performed to investigate the difference in the behavior and neural correlates between the single and dual tasks.

RESULTS

Our results revealed that the extra cognitive interference caused divided attention in the dual task, which led to the decreased neurovascular coupling between fNIRS and EEG in all theta, alpha, and beta rhythms. The proposed framework was demonstrated to have a better ability in characterizing the neural patterns than the canonical channel-averaged method with significantly higher within-class similarity and between-class distance indicators.

CONCLUSIONS

This study proposed a method to investigate CMI by exploring the task-related electrophysiological and hemodynamic activities as well as their neurovascular coupling. Our concurrent EEG-fNIRS study provides new insight into the EEG-fNIRS correlation analysis and novel evidence for the mechanism of neurovascular coupling in the CMI.

On the Role of Neural Oscillations Across Timescales in Speech and Music Processing

This mini review is aimed at a clinician-scientist seeking to understand the role of oscillations in neural processing and their functional relevance in speech and music perception. We present an overview of neural oscillations, methods used to study them, and their functional relevance with respect to music processing, aging, hearing loss, and disorders affecting speech and language. We first review the oscillatory frequency bands and their associations with speech and music processing. Next we describe commonly used metrics for quantifying neural oscillations, briefly touching upon the still-debated mechanisms underpinning oscillatory alignment. Following this, we highlight key findings from research on neural oscillations in speech and music perception, as well as contributions of this work to our understanding of disordered perception in clinical populations. Finally, we conclude with a look toward the future of oscillatory research in speech and music perception, including promising methods and potential avenues for future work. We note that the intention of this mini review is not to systematically review all literature on cortical tracking of speech and music. Rather, we seek to provide the clinician-scientist with foundational information that can be used to evaluate and design research studies targeting the functional role of oscillations in speech and music processing in typical and clinical populations.

Reconstructing Rapid Natural Vision with fMRI-Conditional Video Generative Adversarial Network

Recent functional magnetic resonance imaging (fMRI) studies have made significant progress in reconstructing perceived visual content, which advanced our understanding of the visual mechanism. However, reconstructing dynamic natural vision remains a challenge because of the limitation of the temporal resolution of fMRI. Here, we developed a novel fMRI-conditional video generative adversarial network (f-CVGAN) to reconstruct rapid video stimuli from evoked fMRI responses. In this model, we employed a generator to produce spatiotemporal reconstructions and employed two separate discriminators (spatial and temporal discriminators) for the assessment. We trained and tested the f-CVGAN on two publicly available video-fMRI datasets, and the model produced pixel-level reconstructions of 8 perceived video frames from each fMRI volume. Experimental results showed that the reconstructed videos were fMRI-related and captured important spatial and temporal information of the original stimuli. Moreover, we visualized the cortical importance map and found that the visual cortex is extensively involved in the reconstruction, whereas the low-level visual areas (V1/V2/V3/V4) showed the largest contribution. Our work suggests that slow blood oxygen level-dependent signals describe neural representations of the fast perceptual process that can be decoded in practice.

Neurophysiological dynamics for psychological resilience: A view from the temporal axis

When an individual is faced with adversity, the brain and body work cooperatively to adapt to it. This adaptive process is termed psychological resilience, and recent studies have identified several neurophysiological factors ("neurophysiological resilience"), such as monoamines, oscillatory brain activity, hemodynamics, autonomic activity, stress hormones, and immune systems. Each factor is activated in an interactive manner during specific time windows after exposure to stress. Thus, the differences in psychological resilience levels among individuals can be characterized by differences in the temporal dynamics of neurophysiological resilience. In this review, after briefly introducing the frequently used approaches in this research field and the well-known factors of neurophysiological resilience, we summarize the temporal dynamics of neurophysiological resilience. This viewpoint clarifies an important time window, the more-than-one-hour scale, but the neurophysiological dynamics during this window remain elusive. To address this issue, we propose exploring brain-wide oscillatory activities using concurrent functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) techniques.

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision.