Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis

BrainDAS: Structure-aware domain adaptation network for multi-site brain network analysis

In the medical field, datasets are mostly integrated across sites due to difficult data acquisition and insufficient data at a single site. The domain shift problem caused by the heterogeneous distribution among multi-site data makes autism spectrum disorder (ASD) hard to identify. Recently, domain adaptation has received considerable attention as a promising solution. However, domain adaptation on graph data like brain networks has not been fully studied. It faces two major challenges: (1) complex graph structure; and (2) multiple source domains. To overcome the issues, we propose an end-to-end structure-aware domain adaptation framework for brain network analysis (BrainDAS) using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed approach contains two stages: supervision-guided multi-site graph domain adaptation with dynamic kernel generation and graph classification with attention-based graph pooling. We evaluate our BrainDAS on a public dataset provided by Autism Brain Imaging Data Exchange (ABIDE) which includes 871 subjects from 17 different sites, surpassing state-of-the-art algorithms in several different evaluation settings. Furthermore, our promising results demonstrate the interpretability and generalization of the proposed method. Our code is available at https://github.com/songruoxian/BrainDAS.

Anatomy-Guided Spatio-Temporal Graph Convolutional Networks (AG-STGCNs) for Modeling Functional Connectivity Between Gyri and Sulci Across Multiple Task Domains

The cerebral cortex is folded as gyri and sulci, which provide the foundation to unveil anatomo-functional relationship of brain. Previous studies have extensively demonstrated that gyri and sulci exhibit intrinsic functional difference, which is further supported by morphological, genetic, and structural evidences. Therefore, systematically investigating the gyro-sulcal (G-S) functional difference can help deeply understand the functional mechanism of brain. By integrating functional magnetic resonance imaging (fMRI) with advanced deep learning models, recent studies have unveiled the temporal difference in functional activity between gyri and sulci. However, the potential difference of functional connectivity, which represents functional dependency between gyri and sulci, is much unknown. Moreover, the regularity and variability of the G-S functional connectivity difference across multiple task domains remains to be explored. To address the two concerns, this study developed new anatomy-guided spatio-temporal graph convolutional networks (AG-STGCNs) to investigate the regularity and variability of functional connectivity differences between gyri and sulci across multiple task domains. Based on 830 subjects with seven different task-based and one resting state fMRI (rs-fMRI) datasets from the public Human Connectome Project (HCP), we consistently found that there are significant differences of functional connectivity between gyral and sulcal regions within task domains compared with resting state (RS). Furthermore, there is considerable variability of such functional connectivity and information flow between gyri and sulci across different task domains, which are correlated with individual cognitive behaviors. Our study helps better understand the functional segregation of gyri and sulci within task domains as well as the anatomo-functional-behavioral relationship of the human brain.

Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network Classification

Dynamic brain network has the advantage over static brain network in characterizing the variation pattern of functional brain connectivity, and it has attracted increasing attention in brain disease diagnosis. However, most of the existing dynamic brain networks analysis methods rely on extracting features from independent brain networks divided by sliding windows, making them hard to reveal the high-order dynamic evolution laws of functional brain networks. Additionally, they cannot effectively extract the spatio-temporal topology features in dynamic brain networks. In this paper, we propose to use optimal transport (OT) theory to capture the topology evolution of the dynamic brain networks, and develop a multi-channel spatio-temporal graph convolutional network that collaboratively extracts the temporal and spatial features from the evolution networks. Specifically, we first adaptively evaluate the graph hubness of brain regions in the brain network of each time window, which comprehensively models information transmission among multiple brain regions. Second, the hubness propagation information across adjacent time windows is captured by optimal transport, describing high-order topology evolution of dynamic brain networks. Moreover, we develop a spatio-temporal graph convolutional network with attention mechanism to collaboratively extract the intrinsic temporal and spatial topology information from the above networks. Finally, the multi-layer perceptron is adopted for classifying the dynamic brain network. The extensive experiment on the collected epilepsy dataset and the public ADNI dataset show that our proposed method not only outperforms several state-of-the-art methods in brain disease diagnosis, but also reveals the key dynamic alterations of brain connectivities between patients and healthy controls.

Constructing hierarchical attentive functional brain networks for early AD diagnosis

Analyzing functional brain networks (FBN) with deep learning has demonstrated great potential for brain disorder diagnosis. The conventional construction of FBN is typically conducted at a single scale with a predefined brain region atlas. However, numerous studies have identified that the structure and function of the brain are hierarchically organized in nature. This urges the need of representing FBN in a hierarchical manner for more effective analysis of the complementary diagnostic insights at different scales. To this end, this paper proposes to build hierarchical FBNs adaptively within the Transformer framework. Specifically, a sparse attention-based node-merging module is designed to work alongside the conventional network feature extraction modules in each layer. The proposed module generates coarser nodes for further FBN construction and analysis by combining fine-grained nodes. By stacking multiple such layers, a hierarchical representation of FBN can be adaptively learned in an end-to-end manner. The hierarchical structure can not only integrate the complementary information from multiscale FBN for joint analysis, but also reduce the model complexity due to decreasing node sizes. Moreover, this paper argues that the nodes defined by the existing atlases are not necessarily the optimal starting level to build FBN hierarchy and exploring finer nodes may further enrich the FBN representation. In this regard, each predefined node in an atlas is split into multiple sub-nodes, overcoming the scale limitation of the existing atlases. Extensive experiments conducted on various data sets consistently demonstrate the superior performance of the proposed method over the competing methods.

A novel spatiotemporal graph convolutional network framework for functional connectivity biomarkers identification of Alzheimer's disease

BACKGROUND

Functional connectivity (FC) biomarkers play a crucial role in the early diagnosis and mechanistic study of Alzheimer's disease (AD). However, the identification of effective FC biomarkers remains challenging. In this study, we introduce a novel approach, the spatiotemporal graph convolutional network (ST-GCN) combined with the gradient-based class activation mapping (Grad-CAM) model (STGC-GCAM), to effectively identify FC biomarkers for AD.

METHODS

This multi-center cross-racial retrospective study involved 2,272 participants, including 1,105 cognitively normal (CN) subjects, 790 mild cognitive impairment (MCI) individuals, and 377 AD patients. All participants underwent functional magnetic resonance imaging (fMRI) and T1-weighted MRI scans. In this study, firstly, we optimized the STGC-GCAM model to enhance classification accuracy. Secondly, we identified novel AD-associated biomarkers using the optimized model. Thirdly, we validated the imaging biomarkers using Kaplan-Meier analysis. Lastly, we performed correlation analysis and causal mediation analysis to confirm the physiological significance of the identified biomarkers.

RESULTS

The STGC-GCAM model demonstrated great classification performance (The average area under the curve (AUC) values for different categories were: CN vs MCI = 0.98, CN vs AD = 0.95, MCI vs AD = 0.96, stable MCI vs progressive MCI = 0.79). Notably, the model identified specific brain regions, including the sensorimotor network (SMN), visual network (VN), and default mode network (DMN), as key differentiators between patients and CN individuals. These brain regions exhibited significant associations with the severity of cognitive impairment (p < 0.05). Moreover, the topological features of important brain regions demonstrated excellent predictive capability for the conversion from MCI to AD (Hazard ratio = 3.885, p < 0.001). Additionally, our findings revealed that the topological features of these brain regions mediated the impact of amyloid beta (Aβ) deposition (bootstrapped average causal mediation effect: β = -0.01 [-0.025, 0.00], p < 0.001) and brain glucose metabolism (bootstrapped average causal mediation effect: β = -0.02 [-0.04, -0.001], p < 0.001) on cognitive status.

CONCLUSIONS

This study presents the STGC-GCAM framework, which identifies FC biomarkers using a large multi-site fMRI dataset.

Cross-Domain Identification of Multisite Major Depressive Disorder Using End-to-End Brain Dynamic Attention Network

Establishing objective and quantitative imaging markers at individual level can assist in accurate diagnosis of Major Depressive Disorder (MDD). However, the clinical heterogeneity of MDD and the shift to multisite data decreased identification accuracy. To address these issues, the Brain Dynamic Attention Network (BDANet) is innovatively proposed, and analyzed bimodal scans from 2055 participants of the Rest-meta-MDD consortium. The end-to-end BDANet contains two crucial components. The Dynamic BrainGraph Generator dynamically focuses and represents topological relationships between Regions of Interest, overcoming limitations of static methods. The Ensemble Classifier is constructed to obfuscate domain sources to achieve inter-domain alignment. Finally, BDANet dynamically generates sample-specific brain graphs by downstream recognition tasks. The proposed BDANet achieved an accuracy of 81.6%. The regions with high attribution for classification were mainly located in the insula, cingulate cortex and auditory cortex. The level of brain connectivity in p24 region was negatively correlated ( [Formula: see text]) with the severity of MDD. Additionally, sex differences in connectivity strength were observed in specific brain regions and functional subnetworks ( [Formula: see text] or [Formula: see text]). These findings based on a large multisite dataset support the conclusion that BDANet can better solve the problem of the clinical heterogeneity of MDD and the shift of multisite data. It also illustrates the potential utility of BDANet for personalized accurate identification, treatment and intervention of MDD.

Meta Transfer of Self-Supervised Knowledge: Foundation Model in Action for Post-Traumatic Epilepsy Prediction

Despite the impressive advancements achieved using deep-learning for functional brain activity analysis, the heterogeneity of functional patterns and scarcity of imaging data still pose challenges in tasks such as prediction of future onset of Post-Traumatic Epilepsy (PTE) from data acquired shortly after traumatic brain injury (TBI). Foundation models pre-trained on separate large-scale datasets can improve the performance from scarce and heterogeneous datasets. For functional Magnetic Resonance Imaging (fMRI), while data may be abundantly available from healthy controls, clinical data is often scarce, limiting the ability of foundation models to identify clinically-relevant features. We overcome this limitation by introducing a novel training strategy for our foundation model by integrating meta-learning with self-supervised learning to improve the generalization from normal to clinical features. In this way we enable generalization to other downstream clinical tasks, in our case prediction of PTE. To achieve this, we perform self-supervised training on the control dataset to focus on inherent features that are not limited to a particular supervised task while applying meta-learning, which strongly improves the model's generalizability using bi-level optimization. Through experiments on neurological disorder classification tasks, we demonstrate that the proposed strategy significantly improves task performance on small-scale clinical datasets. To explore the generalizability of the foundation model in downstream applications, we then apply the model to an unseen TBI dataset for prediction of PTE using zero-shot learning. Results further demonstrated the enhanced generalizability of our foundation model.

A Siamese Network With Node Convolution for Individualized Predictions Based on Connectivity Maps Extracted From Resting-State fMRI Data

Deep learning has demonstrated great potential for objective diagnosis of neuropsychiatric disorders based on neuroimaging data, which includes the promising resting-state functional magnetic resonance imaging (RS-fMRI). However, the insufficient sample size has long been a bottleneck for deep model training for the purpose. In this study, we proposed a Siamese network with node convolution (SNNC) for individualized predictions based on RS-fMRI data. With the involvement of Siamese network, which uses sample pair (rather than a single sample) as input, the problem of insufficient sample size can largely be alleviated. To adapt to connectivity maps extracted from RS-fMRI data, we applied node convolution to each of the two branches of the Siamese network. For regression purposes, we replaced the contrastive loss in classic Siamese network with the mean square error loss and thus enabled Siamese network to quantitatively predict label differences. The label of a test sample can be predicted based on any of the training samples, by adding the label of the training sample to the predicted label difference between them. The final prediction for a test sample in this study was made by averaging the predictions based on each of the training samples. The performance of the proposed SNNC was evaluated with age and IQ predictions based on a public dataset (Cam-CAN). The results indicated that SNNC can make effective predictions even with a sample size of as small as 40, and SNNC achieved state-of-the-art accuracy among a variety of deep models and standard machine learning approaches.

Modularity-Constrained Dynamic Representation Learning for Interpretable Brain Disorder Analysis with Functional MRI

Resting-state functional MRI (rs-fMRI) is increasingly used to detect altered functional connectivity patterns caused by brain disorders, thereby facilitating objective quantification of brain pathology. Existing studies typically extract fMRI features using various machine/deep learning methods, but the generated imaging biomarkers are often challenging to interpret. Besides, the brain operates as a modular system with many cognitive/topological modules, where each module contains subsets of densely inter-connected regions-of-interest (ROIs) that are sparsely connected to ROIs in other modules. However, current methods cannot effectively characterize brain modularity. This paper proposes a modularity-constrained dynamic representation learning (MDRL) framework for interpretable brain disorder analysis with rs-fMRI. The MDRL consists of 3 parts: (1) dynamic graph construction, (2) modularity-constrained spatiotemporal graph neural network (MSGNN) for dynamic feature learning, and (3) prediction and biomarker detection. In particular, the MSGNN is designed to learn spatiotemporal dynamic representations of fMRI, constrained by 3 functional modules (i.e., central executive network, salience network, and default mode network). To enhance discriminative ability of learned features, we encourage the MSGNN to reconstruct network topology of input graphs. Experimental results on two public and one private datasets with a total of 1,155 subjects validate that our MDRL outperforms several state-of-the-art methods in fMRI-based brain disorder analysis. The detected fMRI biomarkers have good explainability and can be potentially used to improve clinical diagnosis.

Multi-Scale Dynamic Graph Learning for Brain Disorder Detection With Functional MRI

Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in the detection of brain disorders such as autism spectrum disorder based on various machine/deep learning techniques. Learning-based methods typically rely on functional connectivity networks (FCNs) derived from blood-oxygen-level-dependent time series of rs-fMRI data to capture interactions between brain regions-of-interest (ROIs). Graph neural networks have been recently used to extract fMRI features from graph-structured FCNs, but cannot effectively characterize spatiotemporal dynamics of FCNs, e.g., the functional connectivity of brain ROIs is dynamically changing in a short period of time. Also, many studies usually focus on single-scale topology of FCN, thereby ignoring the potential complementary topological information of FCN at different spatial resolutions. To this end, in this paper, we propose a multi-scale dynamic graph learning (MDGL) framework to capture multi-scale spatiotemporal dynamic representations of rs-fMRI data for automated brain disorder diagnosis. The MDGL framework consists of three major components: 1) multi-scale dynamic FCN construction using multiple brain atlases to model multi-scale topological information, 2) multi-scale dynamic graph representation learning to capture spatiotemporal information conveyed in fMRI data, and 3) multi-scale feature fusion and classification. Experimental results on two datasets show that MDGL outperforms several state-of-the-art methods.

Graph Self-Supervised Learning With Application to Brain Networks Analysis

The less training data and insufficient supervision limit the performance of the deep supervised models for brain disease diagnosis. It is significant to construct a learning framework that can capture more information in limited data and insufficient supervision. To address these issues, we focus on self-supervised learning and aim to generalize the self-supervised learning to the brain networks, which are non-Euclidean graph data. More specifically, we propose an ensemble masked graph self-supervised framework named BrainGSLs, which incorporates 1) a local topological-aware encoder that takes the partially visible nodes as input and learns these latent representations, 2) a node-edge bi-decoder that reconstructs the masked edges by the representations of both the masked and visible nodes, 3) a signal representation learning module for capturing temporal representations from BOLD signals and 4) a classifier used for the classification. We evaluate our model on three real medical clinical applications: diagnosis of Autism Spectrum Disorder (ASD), diagnosis of Bipolar Disorder (BD) and diagnosis of Major Depressive Disorder (MDD). The results suggest that the proposed self-supervised training has led to remarkable improvement and outperforms state-of-the-art methods. Moreover, our method is able to identify the biomarkers associated with the diseases, which is consistent with the previous studies. We also explore the correlation of these three diseases and find the strong association between ASD and BD. To the best of our knowledge, our work is the first attempt of applying the idea of self-supervised learning with masked autoencoder on the brain network analysis.

Early Diagnosis and Biomarkers of Alzheimer's Disease Based on Spatio-temporal Graph Convolution Network

Functional magnetic resonance imaging (fMRI) could detect the dynamic activity of brain function and communication. Previous studies have found reduced brain functional connectivity in Alzheimer's disease (AD) patients. In this study, we proposed to process fMRI data by spatio-temporal graph convolution network (ST-GCN) to achieve an early differential diagnosis of AD and to extract image markers using gradient-weighted class activation mapping (Grad-CAM). The data used in this study were from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, Xuanwu Hospital, and Tongji Hospital. The study included 1105 normal controls and 790 patients with mild cognitive impairment (MCI). The grid search method of K-fold cross-validation was used to train the model. In addition, we used Grad-CAM to extract image markers and carried out visualization analysis. This model obtains better AD diagnosis power: accuracy = 0.92, sensitivity = 0.97, specificity = 0.89, and area under the curve=0.96. Salient brain regions extracted by Grad-CAM include the paracentral lobule, inferior occipital gyrus, middle frontal gyrus, superior temporal gyrus, cuneus, posterior cingulate gyrus, and superior parietal gyrus. Our proposed ST-GAN model will help to explore objective markers that can be used for the early diagnosis of AD.Clinical relevance- Our proposed model shows great potential for enhancing the understanding of the pathology of AD by detecting functional connectivity interruptions.

A transformer model for learning spatiotemporal contextual representation in fMRI data

Representation learning is a core component in data-driven modeling of various complex phenomena. Learning a contextually informative representation can especially benefit the analysis of fMRI data because of the complexities and dynamic dependencies present in such datasets. In this work, we propose a framework based on transformer models to learn an embedding of the fMRI data by taking the spatiotemporal contextual information in the data into account. This approach takes the multivariate BOLD time series of the regions of the brain as well as their functional connectivity network simultaneously as the input to create a set of meaningful features that can in turn be used in various downstream tasks such as classification, feature extraction, and statistical analysis. The proposed spatiotemporal framework uses the attention mechanism as well as the graph convolution neural network to jointly inject the contextual information regarding the dynamics in time series data and their connectivity into the representation. We demonstrate the benefits of this framework by applying it to two resting-state fMRI datasets, and provide further discussion on various aspects and advantages of it over a number of other commonly adopted architectures.

Through the looking glass: Deep interpretable dynamic directed connectivity in resting fMRI

Brain network interactions are commonly assessed via functional (network) connectivity, captured as an undirected matrix of Pearson correlation coefficients. Functional connectivity can represent static and dynamic relations, but often these are modeled using a fixed choice for the data window Alternatively, deep learning models may flexibly learn various representations from the same data based on the model architecture and the training task. However, the representations produced by deep learning models are often difficult to interpret and require additional posthoc methods, e.g., saliency maps. In this work, we integrate the strengths of deep learning and functional connectivity methods while also mitigating their weaknesses. With interpretability in mind, we present a deep learning architecture that exposes a directed graph layer that represents what the model has learned about relevant brain connectivity. A surprising benefit of this architectural interpretability is significantly improved accuracy in discriminating controls and patients with schizophrenia, autism, and dementia, as well as age and gender prediction from functional MRI data. We also resolve the window size selection problem for dynamic directed connectivity estimation as we estimate windowing functions from the data, capturing what is needed to estimate the graph at each time-point. We demonstrate efficacy of our method in comparison with multiple existing models that focus on classification accuracy, unlike our interpretability-focused architecture. Using the same data but training different models on their own discriminative tasks we are able to estimate task-specific directed connectivity matrices for each subject. Results show that the proposed approach is also more robust to confounding factors compared to standard dynamic functional connectivity models. The dynamic patterns captured by our model are naturally interpretable since they highlight the intervals in the signal that are most important for the prediction. The proposed approach reveals that differences in connectivity among sensorimotor networks relative to default-mode networks are an important indicator of dementia and gender. Dysconnectivity between networks, specially sensorimotor and visual, is linked with schizophrenic patients, however schizophrenic patients show increased intra-network default-mode connectivity compared to healthy controls. Sensorimotor connectivity was important for both dementia and schizophrenia prediction, but schizophrenia is more related to dysconnectivity between networks whereas, dementia bio-markers were mostly intra-network connectivity.

Diffusion Kernel Attention Network for Brain Disorder Classification

Constructing and analyzing functional brain networks (FBN) has become a promising approach to brain disorder classification. However, the conventional successive construct-and-analyze process would limit the performance due to the lack of interactions and adaptivity among the subtasks in the process. Recently, Transformer has demonstrated remarkable performance in various tasks, attributing to its effective attention mechanism in modeling complex feature relationships. In this paper, for the first time, we develop Transformer for integrated FBN modeling, analysis and brain disorder classification with rs-fMRI data by proposing a Diffusion Kernel Attention Network to address the specific challenges. Specifically, directly applying Transformer does not necessarily admit optimal performance in this task due to its extensive parameters in the attention module against the limited training samples usually available. Looking into this issue, we propose to use kernel attention to replace the original dot-product attention module in Transformer. This significantly reduces the number of parameters to train and thus alleviates the issue of small sample while introducing a non-linear attention mechanism to model complex functional connections. Another limit of Transformer for FBN applications is that it only considers pair-wise interactions between directly connected brain regions but ignores the important indirect connections. Therefore, we further explore diffusion process over the kernel attention to incorporate wider interactions among indirectly connected brain regions. Extensive experimental study is conducted on ADHD-200 data set for ADHD classification and on ADNI data set for Alzheimer's disease classification, and the results demonstrate the superior performance of the proposed method over the competing methods.

Joint Graph Convolution for Analyzing Brain Structural and Functional Connectome

The white-matter (micro-)structural architecture of the brain promotes synchrony among neuronal populations, giving rise to richly patterned functional connections. A fundamental problem for systems neuroscience is determining the best way to relate structural and functional networks quantified by diffusion tensor imaging and resting-state functional MRI. As one of the state-of-the-art approaches for network analysis, graph convolutional networks (GCN) have been separately used to analyze functional and structural networks, but have not been applied to explore inter-network relationships. In this work, we propose to couple the two networks of an individual by adding inter-network edges between corresponding brain regions, so that the joint structure-function graph can be directly analyzed by a single GCN. The weights of inter-network edges are learnable, reflecting non-uniform structure-function coupling strength across the brain. We apply our Joint-GCN to predict age and sex of 662 participants from the public dataset of the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) based on their functional and micro-structural white-matter networks. Our results support that the proposed Joint-GCN outperforms existing multi-modal graph learning approaches for analyzing structural and functional networks.

Collaborative learning of graph generation, clustering and classification for brain networks diagnosis

PURPOSE

Accurate diagnosis of autism spectrum disorder (ASD) plays a key role in improving the condition and quality of life for patients. In this study, we mainly focus on ASD diagnosis with functional brain networks (FBNs). The major challenge for brain networks modeling is the high dimensional connectivity in brain networks and limited number of subjects, which hinders the classification capability of graph convolutional networks (GCNs).

METHOD

To alleviate the influence of the limited data and high dimensional connectivity, we introduce a unified three-stage graph learning framework for brain network classification, involving multi-graph clustering, graph generation and graph classification. The framework combining Graph Generation, Clustering and Classification Networks (GraphCGC-Net) enhances the critical connections by multi-graph clustering (MGC) with a supervision scheme, and generates realistic brain networks by simultaneously preserving the global consistent distribution and local topology properties.

RESULTS

To demonstrate the effectiveness of our approach, we evaluate the performance of the proposed method on the Autism Brain Imaging Data Exchange (ABIDE) dataset and conduct extensive experiments on the ASD classification problem. Our proposed method achieves an average accuracy of 70.45% and an AUC of 72.76% on ABIDE. Compared with the traditional GCN model, the proposed GraphCGC-Net obtains 9.3%, and 10.64% improvement in terms of accuracy and AUC metrics, respectively.

CONCLUSION

The comprehensive experiments demonstrate that our GraphCGC-Net is effective for graph classification in brain disorders diagnosis. Moreover, we find that MGC can generate biologically meaningful subnetworks, which is highly consistent with the previous neuroimaging-derived biomarker evidence of ASD. More importantly, the promising results suggest that applying generative adversarial networks (GANs) in brain networks to improve the classification performance is worth further investigation.

Multigraph Fusion for Dynamic Graph Convolutional Network

Graph convolutional network (GCN) outputs powerful representation by considering the structure information of the data to conduct representation learning, but its robustness is sensitive to the quality of both the feature matrix and the initial graph. In this article, we propose a novel multigraph fusion method to produce a high-quality graph and a low-dimensional space of original high-dimensional data for the GCN model. Specifically, the proposed method first extracts the common information and the complementary information among multiple local graphs to obtain a unified local graph, which is then fused with the global graph of the data to obtain the initial graph for the GCN model. As a result, the proposed method conducts the graph fusion process twice to simultaneously learn the low-dimensional space and the intrinsic graph structure of the data in a unified framework. Experimental results on real datasets demonstrated that our method outperformed the comparison methods in terms of classification tasks.

A novel 5D brain parcellation approach based on spatio-temporal encoding of resting fMRI data from deep residual learning

OBJECTIVE

Brain parcellation is an essential aspect of computational neuroimaging research and deals with segmenting the brain into (possibly overlapping) sub-regions employed to study brain anatomy or function. In the context of functional parcellation, brain organization which is often measured via temporal metrics such as coherence, is highly dynamic. This dynamic aspect is ignored in most research, which typically applies anatomically based, fixed regions for each individual, and can produce misleading results.

METHODS

In this work, we propose a novel spatio-temporal-network (5D) brain parcellation scheme utilizing a deep residual network to predict the probability of each voxel belonging to a brain network at each point in time.

RESULTS

We trained 53 4D brain networks and evaluate the ability of these networks to capture spatial and temporal dynamics as well as to show sensitivity to individual or group-level variation (in our case with age).

CONCLUSION

The proposed system generates informative spatio-temporal networks that vary not only across individuals but also over time and space.

SIGNIFICANCE

The dynamic 5D nature of the developed approach provides a powerful framework that expands on existing work and has potential to identify novel and typically ignored findings when studying the healthy and disordered brain.

Representation learning of resting state fMRI with variational autoencoder

Resting state functional magnetic resonance imaging (rsfMRI) data exhibits complex but structured patterns. However, the underlying origins are unclear and entangled in rsfMRI data. Here we establish a variational auto-encoder, as a generative model trainable with unsupervised learning, to disentangle the unknown sources of rsfMRI activity. After being trained with large data from the Human Connectome Project, the model has learned to represent and generate patterns of cortical activity and connectivity using latent variables. The latent representation and its trajectory represent the spatiotemporal characteristics of rsfMRI activity. The latent variables reflect the principal gradients of the latent trajectory and drive activity changes in cortical networks. Representational geometry captured as covariance or correlation between latent variables, rather than cortical connectivity, can be used as a more reliable feature to accurately identify subjects from a large group, even if only a short period of data is available in each subject. Our results demonstrate that VAE is a valuable addition to existing tools, particularly suited for unsupervised representation learning of resting state fMRI activity.

A Deep Learning Model for Data-Driven Discovery of Functional Connectivity

Functional connectivity (FC) studies have demonstrated the overarching value of studying the brain and its disorders through the undirected weighted graph of functional magnetic resonance imaging (fMRI) correlation matrix. However, most of the work with the FC depends on the way the connectivity is computed, and it further depends on the manual post-hoc analysis of the FC matrices. In this work, we propose a deep learning architecture BrainGNN that learns the connectivity structure as part of learning to classify subjects. It simultaneously applies a graphical neural network to this learned graph and learns to select a sparse subset of brain regions important to the prediction task. We demonstrate that the model’s state-of-the-art classification performance on a schizophrenia fMRI dataset and demonstrate how introspection leads to disorder relevant findings. The graphs that are learned by the model exhibit strong class discrimination and the sparse subset of relevant regions are consistent with the schizophrenia literature.

Deep Generative Analysis for Task-Based Functional MRI Experiments

While functional magnetic resonance imaging (fMRI) remains one of the most widespread and important methods in basic and clinical neuroscience, the data it produces-time series of brain volumes-continue to pose daunting analysis challenges. The current standard ("mass univariate") approach involves constructing a matrix of task regressors, fitting a separate general linear model at each volume pixel ("voxel"), computing test statistics for each model, and correcting for false positives post hoc using bootstrap or other resampling methods. Despite its simplicity, this approach has enjoyed great success over the last two decades due to: 1) its ability to produce effect maps highlighting brain regions whose activity significantly correlates with a given variable of interest; and 2) its modeling of experimental effects as separable and thus easily interpretable. However, this approach suffers from several well-known drawbacks, namely: inaccurate assumptions of linearity and noise Gaussianity; a limited ability to capture individual effects and variability; and difficulties in performing proper statistical testing secondary to independently fitting voxels. In this work, we adopt a different approach, modeling entire volumes directly in a manner that increases model flexibility while preserving interpretability. Specifically, we use a generalized additive model (GAM) in which the effects of each regressor remain separable, the product of a spatial map produced by a variational autoencoder and a (potentially nonlinear) gain modeled by a covariate-specific Gaussian Process. The result is a model that yields group-level effect maps comparable or superior to the ones obtained with standard fMRI analysis software while also producing single-subject effect maps capturing individual differences. This suggests that generative models with a decomposable structure might offer a more flexible alternative for the analysis of task-based fMRI data.