Building an EEG-fMRI Multi-Modal Brain Graph: A Concurrent EEG-fMRI Study

Multi-modal cross-domain self-supervised pre-training for fMRI and EEG fusion

Neuroimaging techniques including functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) have shown promise in detecting functional abnormalities in various brain disorders. However, existing studies often focus on a single domain or modality, neglecting the valuable complementary information offered by multiple domains from both fMRI and EEG, which is crucial for a comprehensive representation of disorder pathology. This limitation poses a challenge in effectively leveraging the synergistic information derived from these modalities. To address this, we propose a Multi-modal Cross-domain Self-supervised Pre-training Model (MCSP), a novel approach that leverages self-supervised learning to synergize multi-modal information across spatial, temporal, and spectral domains. Our model employs cross-domain self-supervised loss that bridges domain differences by implementing domain-specific data augmentation and contrastive loss, enhancing feature discrimination. Furthermore, MCSP introduces cross-modal self-supervised loss to capitalize on the complementary information of fMRI and EEG, facilitating knowledge distillation within domains and maximizing cross-modal feature convergence. We constructed a large-scale pre-training dataset and pretrained MCSP model by leveraging proposed self-supervised paradigms to fully harness multimodal neuroimaging data. Through comprehensive experiments, we have demonstrated the superior performance and generalizability of our model on multiple classification tasks. Our study contributes a significant advancement in the fusion of fMRI and EEG, marking a novel integration of cross-domain features, which enriches the existing landscape of neuroimaging research, particularly within the context of mental disorder studies.

Integrating fMRI spatial network dynamics and EEG spectral power: insights into resting state connectivity

Introduction

The Integration of functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) has allowed for a novel exploration of the brain's spatial-temporal resolution. While functional brain networks show variations in both spatial and temporal dimensions, most studies focus on fixed spatial networks that change together over time.

Methods

In this study, for the first time, we link spatially dynamic brain networks with EEG spectral properties recorded simultaneously, which allows us to concurrently capture high spatial and temporal resolutions offered by these complementary imaging modalities. We estimated time-resolved brain networks using sliding window-based spatially constrained independent component analysis (scICA), producing resting brain networks that evolved over time at the voxel level. Next, we assessed their coupling with four time-varying EEG spectral power (delta, theta, alpha, and beta).

Results

Our analysis demonstrated how the networks' volumes and their voxel-level activities vary over time and revealed significant correlations with time-varying EEG spectral power. For instance, we found a strong association between increasing volume of the primary visual network and alpha band power, consistent with our hypothesis for eyes open resting state scan. Similarly, the alpha, theta, and delta power of the Pz electrode were localized to voxel-level activities of primary visual, cerebellum, and temporal networks, respectively. We also identified a strong correlation between the primary motor network and alpha (mu rhythm) and beta activity. This is consistent with motor tasks during rest, though this remains to be tested directly.

Discussion

These association between space and frequency observed during rest offer insights into the brain's spatial-temporal characteristics and enhance our understanding of both spatially varying fMRI networks and EEG band power.

Recent advancements in multimodal human-robot interaction

Robotics have advanced significantly over the years, and human-robot interaction (HRI) is now playing an important role in delivering the best user experience, cutting down on laborious tasks, and raising public acceptance of robots. New HRI approaches are necessary to promote the evolution of robots, with a more natural and flexible interaction manner clearly the most crucial. As a newly emerging approach to HRI, multimodal HRI is a method for individuals to communicate with a robot using various modalities, including voice, image, text, eye movement, and touch, as well as bio-signals like EEG and ECG. It is a broad field closely related to cognitive science, ergonomics, multimedia technology, and virtual reality, with numerous applications springing up each year. However, little research has been done to summarize the current development and future trend of HRI. To this end, this paper systematically reviews the state of the art of multimodal HRI on its applications by summing up the latest research articles relevant to this field. Moreover, the research development in terms of the input signal and the output signal is also covered in this manuscript.

Simultaneous electroencephalography-functional magnetic resonance imaging for assessment of human brain function

Electroencephalography (EEG) and functional Magnetic Resonance Imaging (MRI) have long been used as tools to examine brain activity. Since both methods are very sensitive to changes of synaptic activity, simultaneous recording of EEG and fMRI can provide both high temporal and spatial resolution. Therefore, the two modalities are now integrated into a hybrid tool, EEG-fMRI, which encapsulates the useful properties of the two. Among other benefits, EEG-fMRI can contribute to a better understanding of brain connectivity and networks. This review lays its focus on the methodologies applied in performing EEG-fMRI studies, namely techniques used for the recording of EEG inside the scanner, artifact removal, and statistical analysis of the fMRI signal. We will investigate simultaneous resting-state and task-based EEG-fMRI studies and discuss their clinical and technological perspectives. Moreover, it is established that the brain regions affected by a task-based neural activity might not be limited to the regions in which they have been initiated. Advanced methods can help reveal the regions responsible for or affected by a developed neural network. Therefore, we have also looked into studies related to characterization of structure and dynamics of brain networks. The reviewed literature suggests that EEG-fMRI can provide valuable complementary information about brain neural networks and functions.

Mapping relationships among schizophrenia, bipolar and schizoaffective disorders: A deep classification and clustering framework using fMRI time series

BACKGROUND

Psychiatric disorders are categorized using self-report and observational information rather than biological data. There is also considerable symptomatic overlap between different types of psychiatric disorders, which makes diagnostic categorization and multi-class classification challenging.

METHODS

In this work, we propose a unified framework for supervised classification and unsupervised clustering of psychotic disorders using brain imaging data. A new multi-scale recurrent neural network (MsRNN) model was developed and applied to fMRI time courses (TCs) for multi-class classification. The high-level representations of the original TCs were then submitted to a tSNE clustering model for visualizing the group differences between disorders. A leave-one-feature-out approach was used for disorder-related biomarker identification.

RESULTS

When studying fMRI from schizophrenia, psychotic bipolar disorder, schizoaffective disorder, and healthy individuals, the accuracy of a 4-class classification reached 46%, significantly above chance. The hippocampus, supplementary motor area and paracentral lobule were discovered as the most contributing regional TCs in the multi-class classification. Beyond this, visualization of the tSNE clustering suggested that the disease severity can be captured and schizoaffective disorder (SAD) may be separated into two subtypes. SAD cluster1 has significantly higher Positive And Negative Syndrome Scale (PANSS) scores than SAD cluster2 in PANSS negative2 (emotional withdrawal), general2 (anxiety), general3 (guilt feelings), general4 (tension).

CONCLUSIONS

The proposed deep classification and clustering framework is not only able to identify psychiatric disorders with high accuracy, but also interpret the correlation between brain networks and specific psychiatric disorders, and reveal the relationship between them. This work provides a promising way to investigate a spectrum of similar disorders using neuroimaging-based measures.

Graph theory approach for the structural-functional brain connectome of depression

To decipher the organizational styles of neural underpinning in major depressive disorder (MDD), the current article reviewed recent neuroimaging studies (published during 2015-2020) that applied graph theory approach to the diffusion tensor imaging data or functional brain activation data acquired during task-free resting state. The global network organization of resting-state functional connectivity network in MDD were diverse according to the onset age and medication status. Intra-modular functional connections were weaker in MDD compared to healthy controls (HC) for default mode and limbic networks. Weaker local graph metrics of default mode, frontoparietal, and salience network components in MDD compared to HC were also found. On the contrary, brain regions comprising the limbic, sensorimotor, and subcortical networks showed higher local graph metrics in MDD compared to HC. For the brain white matter-based structural connectivity network, the global network organization was comparable to HC in adult MDD but was attenuated in late-life depression. Local graph metrics of limbic, salience, default-mode, subcortical, insular, and frontoparietal network components in structural connectome were affected from the severity of depressive symptoms, burden of perceived stress, and treatment effects. Collectively, the current review illustrated changed global network organization of structural and functional brain connectomes in MDD compared to HC and were varied according to the onset age and medication status. Intra-modular functional connectivity within the default mode and limbic networks were weaker in MDD compared to HC. Local graph metrics of structural connectome for MDD reflected severity of depressive symptom and perceived stress, and were also changed after treatments. Further studies that explore the graph metrics-based neural correlates of clinical features, cognitive styles, treatment response and prognosis in MDD are required.

Biomarkers for autism spectrum disorder: opportunities for magnetoencephalography (MEG)

This paper reviews a candidate biomarker for ASD, the M50 auditory evoked response component, detected by magnetoencephalography (MEG) and presents a position on the roles and opportunities for such a biomarker, as well as converging evidence from allied imaging techniques (magnetic resonance imaging, MRI and spectroscopy, MRS). Data is presented on prolonged M50 latencies in ASD as well as extension to include children with ASD with significant language and cognitive impairments in whom M50 latency delays are exacerbated. Modeling of the M50 latency by consideration of the properties of auditory pathway white matter is shown to be successful in typical development but challenged by heterogeneity in ASD; this, however, is capitalized upon to identify a distinct subpopulation of children with ASD whose M50 latencies lie well outside the range of values predictable from the typically developing model. Interestingly, this subpopulation is characterized by low levels of the inhibitory neurotransmitter GABA. Following from this, we discuss a potential use of the M50 latency in indicating “target engagement” acutely with administration of a GABA-B agonist, potentially distinguishing “responders” from “non-responders” with the implication of optimizing inclusion for clinical trials of such agents. Implications for future application, including potential evaluation of infants with genetic risk factors, are discussed. As such, the broad scope of potential of a representative candidate biological marker, the M50 latency, is introduced along with potential future applications.

This paper outlines a strategy for understanding brain dysfunction in individuals with intellectual and developmental disabilities (IDD). It is proposed that a multimodal approach (collection of brain structure, chemistry, and neuronal functional data) will identify IDD subpopulations who share a common disease pathway, and thus identify individuals with IDD who might ultimately benefit from specific treatments. After briefly demonstrating the need and potential for scope, examples from studies examining brain function and structure in children with autism spectrum disorder (ASD) illustrate how measures of brain neuronal function (from magnetoencephalography, MEG), brain structure (from magnetic resonance imaging, MRI, especially diffusion MRI), and brain chemistry (MR spectroscopy) can help us better understand the heterogeneity in ASD and form the basis of multivariate biological markers (biomarkers) useable to define clinical subpopulations. Similar approaches can be applied to understand brain dysfunction in neurodevelopmental disorders (NDD) in general. In large part, this paper represents our endeavors as part of the CHOP/Penn NICHD-funded intellectual and developmental disabilities research center (IDDRC) over the past decade.

Dynamic Functional Connectivity of EEG: From Identifying Fingerprints to Gender Differences to a General Blueprint for the Brain's Functional Organization

An individual's brain functional organization is unique and can reliably be observed using modalities such as functional magnetic resonance imaging (fMRI). Here we demonstrate that a quantification of the dynamics of functional connectivity (FC) as measured using electroencephalography (EEG) offers an alternative means of observing an individual's brain functional organization. Using data from both healthy individuals as well as from patients with Parkinson's disease (PD) (n = 103 healthy individuals, n = 57 PD patients), we show that “dynamic FC” (DFC) profiles can be used to identify individuals in a large group. Furthermore, we show that DFC profiles predict gender and exhibit characteristics shared both among individuals as well as between both hemispheres. Furthermore, DFC profile characteristics are frequency band specific, indicating that they reflect distinct processes in the brain. Our empirically derived method of DFC demonstrates the potential of studying the dynamics of the functional organization of the brain using EEG.

Tracking spatial dynamics of functional connectivity during a task

Functional connectivity (FC) measured from functional magnetic resonance imaging (fMRI) provides a powerful tool to explore brain organization. Studies of the temporal dynamics of brain organization have shown a large temporal variability of the functional connectome, which may be associated with mental status transitions and/or adaptive process. Most dynamic studies, e.g. functional connectome and functional network connectivity (FNC), have focused on the macroscopic FC changes, i.e. the changes of temporal coherence across various brain network sources, nodes and/or regions of interest, where it is assumed within the network or node that the FC is static. In this paper, we develop a novel method to examine the spatial dynamics of FC, without the assumption of its intra-network stationarity. We applied our approach to fMRI data during an auditory oddball task (AOD) from twenty-two subjects, in an attempt to capture/validate the approach by evaluating whether spatial connectivity varies with task condition. The results showed that connectivity networks exhibit spatial variability over time, in addition to participating in conventional temporal dynamics, i.e. cross-network variability or dynamic functional network connectivity (dFNC). Furthermore, we studied the relationship of spatial dynamic in FC to cognitive processes, by performing a cluster analysis to evaluate an individual's functional correspondence towards the 'target' (oddball) detection from AOD task, and extracting cognitive task correspondence states as well as their dynamic FC spatial maps segregated by such states. We found a clear trend in different task-guided states, particularly, a prominent reduction of task stimulus synchrony state along with strong anticorrelation between default mode network (DMN) and cognitive attentional networks. We also observed an increasing occurrence of the task desynchrony state which showed an absence of DMN anticorrelation. The results highlight the impact of a well-studied cognitive task on the observed spatial dynamic structure. We also showed that the FC spatial dynamic pattern from our method largely corresponds to macroscopic dFNC patterns, but with more details and specifications over space, meanwhile the connectivity within the source itself provides novel information and varies over time. Overall, we demonstrate clear evidence of the presence of the (usually ignored) spatial dynamics of connectivity, its links to the task and implications of cognition/mental status.

Blind Visualization of Task-Related Networks From Visual Oddball Simultaneous EEG-fMRI Data: Spectral or Spatiospectral Model?

Various disease conditions can alter EEG event-related responses and fMRI-BOLD signals. We hypothesized that event-related responses and their clinical alterations are imprinted in the EEG spectral domain as event-related (spatio)spectral patterns (ERSPat). We tested four EEG-fMRI fusion models utilizing EEG power spectra fluctuations (i.e., absolute spectral model - ASM; relative spectral model - RSM; absolute spatiospectral model - ASSM; and relative spatiospectral model - RSSM) for fully automated and blind visualization of task-related neural networks. Two (spatio)spectral patterns (high δ 4 band and low β 1 band) demonstrated significant negative linear relationship (p FWE < 0.05) to the frequent stimulus and three patterns (two low δ 2 and δ 3 bands, and narrow θ 1 band) demonstrated significant positive relationship (p < 0.05) to the target stimulus. These patterns were identified as ERSPats. EEG-fMRI F-map of each δ 4 model showed strong engagement of insula, cuneus, precuneus, basal ganglia, sensory-motor, motor and dorsal part of fronto-parietal control (FPCN) networks with fast HRF peak and noticeable trough. ASM and RSSM emphasized spatial statistics, and the relative power amplified the relationship to the frequent stimulus. For the δ 4 model, we detected a reduced HRF peak amplitude and a magnified HRF trough amplitude in the frontal part of the FPCN, default mode network (DMN) and in the frontal white matter. The frequent-related β 1 patterns visualized less significant and distinct suprathreshold spatial associations. Each θ 1 model showed strong involvement of lateralized left-sided sensory-motor and motor networks with simultaneous basal ganglia co-activations and reduced HRF peak and amplified HRF trough in the frontal part of the FPCN and DMN. The ASM θ 1 model preserved target-related EEG-fMRI associations in the dorsal part of the FPCN. For δ 4, β 1, and θ 1 bands, all models provided high local F-statistics in expected regions. The most robust EEG-fMRI associations were observed for ASM and RSSM.

Age-related differences in network structure and dynamic synchrony of cognitive control

Cognitive trajectories vary greatly across older individuals, and the neural mechanisms underlying these differences remain poorly understood. Here, we investigate the cognitive variability in older adults by linking the influence of white matter microstructure on the task-related organization of fast and effective communications between brain regions. Using diffusion tensor imaging and electroencephalography, we show that individual differences in white matter network organization are associated with network clustering and efficiency in the alpha and high-gamma bands, and that functional network dynamics partly explain individual differences in cognitive control performance in older adults. We show that older individuals with high versus low structural network clustering differ in task-related network dynamics and cognitive performance. These findings were corroborated by investigating magnetoencephalography networks in an independent dataset. This multimodal (fMRI and biological markers) brain connectivity framework of individual differences provides a holistic account of how differences in white matter microstructure underlie age-related variability in dynamic network organization and cognitive performance.

Brain Network Constancy and Participant Recognition: an Integrated Approach to Big Data and Complex Network Analysis

With the development of big data sharing and data standardization, electroencephalogram (EEG) data are increasingly used in the exploration of human cognitive behavior. Most of the existing studies focus on the changes of human brain network topology (the number of connections, degree distribution, clustering coefficient phantom) in various cognitive behaviors. However, there has been little exploration into the steady state of multi-cognitive behaviors and the recognition of multi-participant brain networks. To solve these two problems, we used EEG data of 99 healthy participants from the PhysioBank to study multi-cognitive behaviors. Specifically, we calculated the symbolic transfer entropy (STE) between 64 electrode sequences of EEG data and constructed the brain networks of various cognitive behaviors of each participant using the directed minimum spanning tree (DMST) algorithm. We then investigated the eigenvalue spectrum of the STE matrix of each individual's cognitive behavior. The results also showed that the spectrum distributions of different cognitive states of the same participant remained relatively stable, but those of the same cognitive state of different participants varied considerably, verifying the relative stability and uniqueness of the human brain network similar to a human's fingerprint. Based on these features, we used the spectral distribution set of 99 participants of various cognitive states as the original data set and developed a spectral distribution set scoring (SDSS) method to identify the brain network participants. It was found that most labels (69.35%) of the test participant with the highest score were identical to the labeled participant. This study provided further evidence for the existence of human brain fingerprints, and furnished a new approach for dynamic identification of brain fingerprints.

BOLD and EEG signal variability at rest differently relate to aging in the human brain

Variability of neural activity is regarded as a crucial feature of healthy brain function, and several neuroimaging approaches have been employed to assess it noninvasively. Studies on the variability of both evoked brain response and spontaneous brain signals have shown remarkable changes with aging but it is unclear if the different measures of brain signal variability - identified with either hemodynamic or electrophysiological methods - reflect the same underlying physiology. In this study, we aimed to explore age differences of spontaneous brain signal variability with two different imaging modalities (EEG, fMRI) in healthy younger (25 ​± ​3 years, N ​= ​135) and older (67 ​± ​4 years, N ​= ​54) adults. Consistent with the previous studies, we found lower blood oxygenation level dependent (BOLD) variability in the older subjects as well as less signal variability in the amplitude of low-frequency oscillations (1-12 ​Hz), measured in source space. These age-related reductions were mostly observed in the areas that overlap with the default mode network. Moreover, age-related increases of variability in the amplitude of beta-band frequency EEG oscillations (15-25 ​Hz) were seen predominantly in temporal brain regions. There were significant sex differences in EEG signal variability in various brain regions while no significant sex differences were observed in BOLD signal variability. Bivariate and multivariate correlation analyses revealed no significant associations between EEG- and fMRI-based variability measures. In summary, we show that both BOLD and EEG signal variability reflect aging-related processes but are likely to be dominated by different physiological origins, which relate differentially to age and sex.

Towards understanding neural network signatures of motor skill learning in Parkinson's disease and healthy aging

In the past decade, neurorehabilitation has been shown to be an effective therapeutic supplement for patients with Parkinson's disease (PD). However, patients still experience severe problems with the consolidation of learned motor skills. Knowledge on the neural correlates underlying this process is thus essential to optimize rehabilitation for PD. This review investigates the existing studies on neural network connectivity changes in relation to motor learning in healthy aging and PD and critically evaluates the imaging methods used from a methodological point of view. The results indicate that despite neurodegeneration there is still potential to modify connectivity within and between motor and cognitive networks in response to motor training, although these alterations largely bypass the most affected regions in PD. However, so far training-related changes are inferred and possible relationships are not substantiated by brain-behavior correlations. Furthermore, the studies included suffer from many methodological drawbacks. This review also highlights the potential for using neural network measures as predictors for the response to rehabilitation, mainly based on work in young healthy adults. We speculate that future approaches, including graph theory and multimodal neuroimaging, may be more sensitive than brain activation patterns and model-based connectivity maps to capture the effects of motor learning. Overall, this review suggests that methodological developments in neuroimaging will eventually provide more detailed knowledge on how neural networks are modified by training, thereby paving the way for optimized neurorehabilitation for patients.

An approach to directly link ICA and seed-based functional connectivity: Application to schizophrenia

Independent component analysis (ICA) and seed-based analyses are widely used techniques for studying intrinsic neuronal activity in task-based or resting scans. In this work, we show there is a direct link between the two, and show that there are some important differences between the two approaches in terms of what information they capture. We developed an enhanced connectivity-matrix independent component analysis (cmICA) for calculating whole brain voxel maps of functional connectivity, which reduces the computational complexity of voxel-based connectivity analysis on performing many temporal correlations. We also show there is a mathematical equivalency between parcellations on voxel-to-voxel functional connectivity and simplified cmICA. Next, we used this cost-efficient data-driven method to examine the resting state fMRI connectivity in schizophrenia patients (SZ) and healthy controls (HC) on a whole brain scale and further quantified the relationship between brain functional connectivity and cognitive performances measured by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery. Current results suggest that SZ exhibit a wide-range abnormality, primarily a decrease, in functional connectivity both between networks and within different network hubs. Specific functional connectivity decreases were associated with MATRICS performance deficits. In addition, we found that resting state functional connectivity decreases was extensively associated with aging regardless of groups. In contrast, there was no relationship between positive and negative symptoms in the patients and functional connectivity. In sum, we have developed a novel mathematical relationship between ICA and seed-based connectivity that reduces computational complexity, which has broad applicability, and showed a specific application of this approach to characterize connectivity changes associated with cognitive scores in SZ.

Metastable States of Multiscale Brain Networks Are Keys to Crack the Timing Problem

The dynamics of the environment where we live in and the interaction with it, predicting events, provided strong evolutionary pressures for the brain functioning to process temporal information and generate timed responses. As a result, the human brain is able to process temporal information and generate temporal patterns. Despite the clear importance of temporal processing to cognition, learning, communication and sensory, motor and emotional processing, the basal mechanisms of how animals differentiate simple intervals or provide timed responses are still under debate. The lesson we learned from the last decade of research in neuroscience is that functional and structural brain connectivity matter. Specifically, it has been accepted that the organization of the brain in interacting segregated networks enables its function. In this paper we delineate the route to a promising approach for investigating timing mechanisms. We illustrate how novel insight into timing mechanisms can come by investigating brain functioning as a multi-layer dynamical network whose clustered dynamics is bound to report the presence of metastable states. We anticipate that metastable dynamics underlie the real-time coordination necessary for the brain's dynamic functioning associated to time perception. This new point of view will help further clarifying mechanisms of neuropsychiatric disorders.

From eyes-closed to eyes-open: Role of cholinergic projections in EC-to-EO alpha reactivity revealed by combining EEG and MRI

Alpha rhythm (8 to 12 Hz) observed in EEG over human posterior cortex is prominent during eyes-closed (EC) resting and attenuates during eyes-open (EO) resting. Research shows that the degree of EC-to-EO alpha blocking or alpha desynchronization, termed alpha reactivity here, is a neural marker of cognitive health. We tested the role of acetylcholine in EC-to-EO alpha reactivity by applying a multimodal neuroimaging approach to a cohort of young adults and a cohort of older adults. In the young cohort, simultaneous EEG-fMRI was recorded from twenty-one young adults during both EO and EC resting. In the older cohort, functional MRI was recorded from forty older adults during EO and EC resting, along with FLAIR and diffusion MRI. For a subset of twenty older adults, EEG was recorded during EO and EC resting in a separate session. In both young and older adults, functional connectivity between the basal nucleus of Meynert (BNM), the major source of cortical acetylcholine, and the visual cortex increased from EC to EO, and this connectivity increase was positively associated with alpha reactivity; namely, the stronger the BNM-visual cortex functional connectivity increase from EC to EO, the larger the EC-to-EO alpha desynchronization. In older adults, lesions of the fiber tracts linking BNM and visual cortex quantified by leukoaraiosis volume, associated with reduced alpha reactivity. These findings support a role of acetylcholine and particularly cholinergic pathways in mediating EC-to-EO alpha reactivity and suggest that impaired alpha reactivity could serve as a marker of the integrity of the cholinergic system.

On the nature and use of models in network neuroscience

Network theory provides an intuitively appealing framework for studying relationships among interconnected brain mechanisms and their relevance to behaviour. As the space of its applications grows, so does the diversity of meanings of the term network model. This diversity can cause confusion, complicate efforts to assess model validity and efficacy, and hamper interdisciplinary collaboration. In this Review, we examine the field of network neuroscience, focusing on organizing principles that can help overcome these challenges. First, we describe the fundamental goals in constructing network models. Second, we review the most common forms of network models, which can be described parsimoniously along the following three primary dimensions: from data representations to first-principles theory; from biophysical realism to functional phenomenology; and from elementary descriptions to coarse-grained approximations. Third, we draw on biology, philosophy and other disciplines to establish validation principles for these models. We close with a discussion of opportunities to bridge model types and point to exciting frontiers for future pursuits.

Working memory capacity and the functional connectome - insights from resting-state fMRI and voxelwise centrality mapping

The functional connectome represents a comprehensive network map of functional connectivity throughout the human brain. To date, the relationship between the organization of functional connectivity and cognitive performance measures is still poorly understood. In the present study we use resting-state functional magnetic resonance imaging (fMRI) data to explore the link between the functional connectome and working memory capacity in an individual differences design. Working memory capacity, which refers to the maximum amount of context information that an individual can retain in the absence of external stimulation, was assessed outside the MRI scanner and estimated based on behavioral data from a change detection task. Resting-state time series were analyzed by means of voxelwise degree and eigenvector centrality mapping, which are data-driven network analytic approaches for the characterization of functional connectivity. We found working memory capacity to be inversely correlated with both centrality in the right intraparietal sulcus. Exploratory analyses revealed that this relationship was putatively driven by an increase in negative connectivity strength of the structure. This resting-state connectivity finding fits previous task based activation studies that have shown that this area responds to manipulations of working memory load.

Exploration of Neural Activity under Cognitive Reappraisal Using Simultaneous EEG-fMRI Data and Kernel Canonical Correlation Analysis

BACKGROUND

Neural activity under cognitive reappraisal can be more accurately investigated using simultaneous EEG- (electroencephalography) fMRI (functional magnetic resonance imaging) than using EEG or fMRI only. Complementary spatiotemporal information can be found from simultaneous EEG-fMRI data to study brain function.

METHOD

An effective EEG-fMRI fusion framework is proposed in this work. EEG-fMRI data is simultaneously sampled on fifteen visually stimulated healthy adult participants. Net-station toolbox and empirical mode decomposition are employed for EEG denoising. Sparse spectral clustering is used to construct fMRI masks that are used to constrain fMRI activated regions. A kernel-based canonical correlation analysis is utilized to fuse nonlinear EEG-fMRI data.

RESULTS

The experimental results show a distinct late positive potential (LPP, latency 200-700ms) from the correlated EEG components that are reconstructed from nonlinear EEG-fMRI data. Peak value of LPP under reappraisal state is smaller than that under negative state, however, larger than that under neutral state. For correlated fMRI components, obvious activation can be observed in cerebral regions, e.g., the amygdala, temporal lobe, cingulate gyrus, hippocampus, and frontal lobe. Meanwhile, in these regions, activated intensity under reappraisal state is obviously smaller than that under negative state and larger than that under neutral state.

CONCLUSIONS

The proposed EEG-fMRI fusion approach provides an effective way to study the neural activities of cognitive reappraisal with high spatiotemporal resolution. It is also suitable for other neuroimaging technologies using simultaneous EEG-fMRI data.

Application of Graph Theory to Assess Static and Dynamic Brain Connectivity: Approaches for Building Brain Graphs

Human brain connectivity is complex. Graph theory based analysis has become a powerful and popular approach for analyzing brain imaging data, largely because of its potential to quantitatively illuminate the networks, the static architecture in structure and function, the organization of dynamic behavior over time, and disease related brain changes. The first step in creating brain graphs is to define the nodes and edges connecting them. We review a number of approaches for defining brain nodes including fixed versus data-driven nodes. Expanding the narrow view of most studies which focus on static and/or single modality brain connectivity, we also survey advanced approaches and their performances in building dynamic and multi-modal brain graphs. We show results from both simulated and real data from healthy controls and patients with mental illnesse. We outline the advantages and challenges of these various techniques. By summarizing and inspecting recent studies which analyzed brain imaging data based on graph theory, this article provides a guide for developing new powerful tools to explore complex brain networks.

Multimodal, noninvasive seizure network mapping software: A novel tool for preoperative epilepsy evaluation

BACKGROUND

Despite rigorous preoperative evaluation, epilepsy surgery achieves seizure freedom in only two-thirds of cases. Current preoperative evaluation does not include a detailed network analysis despite the association of network-level changes with epilepsy.

OBJECTIVE

We sought to create a software algorithm to map individualized epilepsy networks by combining noninvasive electroencephalography (EEG) source localization and nonconcurrent resting state functional magnetic resonance imaging (rsfMRI).

METHODS

Scalp EEG and rsfMRI data were acquired for three sample cases: one healthy control case, one case of right temporal lobe epilepsy, and one case of bitemporal seizure onset. Data from rsfMRI were preprocessed, and a time-series function was extracted. Connection coefficients were used to threshold out spurious connections and model global functional networks in a 3D map. Epileptic discharges were localized using a forward model of cortical mesh dipoles followed by an empirical Bayesian approach of inverse source reconstruction and co-registered with rsfMRI. Co-activating brain regions were mapped.

RESULTS

Three illustrative sample cases are presented. In the healthy control case, the software showed symmetrical global connectivity. In the right temporal lobe epilepsy case, asymmetry was found in the global connectivity metrics with a paucity of connectivity ipsilateral to the epileptogenic cortex. The superior longitudinal fasciculus, uncinate fasciculus, and commissural fibers connecting disparate and discontinuous cortical regions involved in the epilepsy network were visualized. In the case with bitemporal lobe epilepsy, global connectivity was symmetric. It showed a network of correlating cortical activity local to epileptogenic tissue in both temporal lobes. The network involved white matter tracks in a similar pattern to those seen in the right temporal case.

CONCLUSIONS

This modeling algorithm allows better definition of the global brain network and potentially demonstrates differences in connectivity between an epileptic and a non-epileptic brain. This finding may be useful for mapping cortico-cortical connections representing the putative epilepsy networks. With this methodology, we localized the epileptogenic brain and showed network asymmetry and long-distance cortical co-activation. This software tool is the first to use a multimodal, nonconcurrent, and noninvasive approach to model and visualize the epilepsy network.

Maturation trajectories of cortical resting-state networks depend on the mediating frequency band

The functional significance of resting state networks and their abnormal manifestations in psychiatric disorders are firmly established, as is the importance of the cortical rhythms in mediating these networks. Resting state networks are known to undergo substantial reorganization from childhood to adulthood, but whether distinct cortical rhythms, which are generated by separable neural mechanisms and are often manifested abnormally in psychiatric conditions, mediate maturation differentially, remains unknown. Using magnetoencephalography (MEG) to map frequency band specific maturation of resting state networks from age 7 to 29 in 162 participants (31 independent), we found significant changes with age in networks mediated by the beta (13-30 Hz) and gamma (31-80 Hz) bands. More specifically, gamma band mediated networks followed an expected asymptotic trajectory, but beta band mediated networks followed a linear trajectory. Network integration increased with age in gamma band mediated networks, while local segregation increased with age in beta band mediated networks. Spatially, the hubs that changed in importance with age in the beta band mediated networks had relatively little overlap with those that showed the greatest changes in the gamma band mediated networks. These findings are relevant for our understanding of the neural mechanisms of cortical maturation, in both typical and atypical development.

A Single Session of rTMS Enhances Small-Worldness in Writer's Cramp: Evidence from Simultaneous EEG-fMRI Multi-Modal Brain Graph

Background and Purpose: Repetitive transcranial magnetic stimulation (rTMS) induces widespread changes in brain connectivity. As the network topology differences induced by a single session of rTMS are less known we undertook this study to ascertain whether the network alterations had a small-world morphology using multi-modal graph theory analysis of simultaneous EEG-fMRI.

Method: Simultaneous EEG-fMRI was acquired in duplicate before (R1) and after (R2) a single session of rTMS in 14 patients with Writer’s Cramp (WC). Whole brain neuronal and hemodynamic network connectivity were explored using the graph theory measures and clustering coefficient, path length and small-world index were calculated for EEG and resting state fMRI (rsfMRI). Multi-modal graph theory analysis was used to evaluate the correlation of EEG and fMRI clustering coefficients.

Result: A single session of rTMS was found to increase the clustering coefficient and small-worldness significantly in both EEG and fMRI (p < 0.05). Multi-modal graph theory analysis revealed significant modulations in the fronto-parietal regions immediately after rTMS. The rsfMRI revealed additional modulations in several deep brain regions including cerebellum, insula and medial frontal lobe.

Conclusion: Multi-modal graph theory analysis of simultaneous EEG-fMRI can supplement motor physiology methods in understanding the neurobiology of rTMS in vivo. Coinciding evidence from EEG and rsfMRI reports small-world morphology for the acute phase network hyper-connectivity indicating changes ensuing low-frequency rTMS is probably not “noise”.

Stable Scalp EEG Spatiospectral Patterns Across Paradigms Estimated by Group ICA

Electroencephalography (EEG) oscillations reflect the superposition of different cortical sources with potentially different frequencies. Various blind source separation (BSS) approaches have been developed and implemented in order to decompose these oscillations, and a subset of approaches have been developed for decomposition of multi-subject data. Group independent component analysis (Group ICA) is one such approach, revealing spatiospectral maps at the group level with distinct frequency and spatial characteristics. The reproducibility of these distinct maps across subjects and paradigms is relatively unexplored domain, and the topic of the present study. To address this, we conducted separate group ICA decompositions of EEG spatiospectral patterns on data collected during three different paradigms or tasks (resting-state, semantic decision task and visual oddball task). K-means clustering analysis of back-reconstructed individual subject maps demonstrates that fourteen different independent spatiospectral maps are present across the different paradigms/tasks, i.e. they are generally stable.

Comparing brain graphs in which nodes are regions of interest or independent components: A simulation study

BACKGROUND

A key challenge in building a brain graph using fMRI data is how to define the nodes. Spatial brain components estimated by independent components analysis (ICA) and regions of interest (ROIs) determined by brain atlas are two popular methods to define nodes in brain graphs. It is difficult to evaluate which method is better in real fMRI data.

NEW METHOD

Here we perform a simulation study and evaluate the accuracies of a few graph metrics in graphs with nodes of ICA components, ROIs, or modified ROIs in four simulation scenarios.

RESULTS

Graph measures with ICA nodes are more accurate than graphs with ROI nodes in all cases. Graph measures with modified ROI nodes are modulated by artifacts. The correlations of graph metrics across subjects between graphs with ICA nodes and ground truth are higher than the correlations between graphs with ROI nodes and ground truth in scenarios with large overlapped spatial sources. Moreover, moving the location of ROIs would largely decrease the correlations in all scenarios.

COMPARISON WITH EXISTING METHOD (S)

Evaluating graphs with different nodes is promising in simulated data rather than real data because different scenarios can be simulated and measures of different graphs can be compared with a known ground truth.

CONCLUSION

Since ROIs defined using brain atlas may not correspond well to real functional boundaries, overall findings of this work suggest that it is more appropriate to define nodes using data-driven ICA than ROI approaches in real fMRI data.

Dynamic spatiotemporal variability of alpha-BOLD relationships during the resting-state and task-evoked responses

Accurate characterization of the spatiotemporal relationship between two of the most prominent neuroimaging measures of neuronal activity, the 8-13Hz, occipito-parietal EEG alpha oscillation and the BOLD fMRI signal, must encompass the intrinsically dynamic nature of both alpha power and brain function. Here, during the eyes-open resting state, we use a 16s sliding-window analysis and demonstrate that the mean spatial network of dynamic alpha-BOLD correlations is highly comparable to the static network calculated over six minutes. However, alpha-BOLD correlations showed substantial spatiotemporal variability within-subjects and passed through many different configurations such that the static network was fully represented in only ~10% of 16s epochs, with visual and parietal regions (coherent on average) often opposingly correlated with each other or with alpha. We find that the common assumption of static-alpha BOLD correlations greatly oversimplifies temporal variation in brain network dynamics. Fluctuations in alpha-BOLD coupling significantly depended upon the instantaneous amplitude of alpha power, and primary and lateral visual areas were most strongly negatively correlated with alpha during different alpha power states, possibly suggesting the action of multiple alpha mechanisms. Dynamic alpha-BOLD correlations could not be explained by eye-blinks/movements, head motion or non-neuronal physiological variability. Individual's mean alpha power and frequency were found to contribute to between-subject variability in alpha-BOLD correlations. Additionally, application to a visual stimulation dataset showed that dynamic alpha-BOLD correlations provided functional information pertaining to the brain's response to stimulation by exhibiting spatiotemporal fluctuations related to variability in the trial-by-trial BOLD response magnitude. Significantly weaker visual alpha-BOLD correlations were found both preceding and following small amplitude BOLD response trials compared to large response trials.