Attention-Diffusion-Bilinear Neural Network for Brain Network Analysis

Riemannian manifold-based disentangled representation learning for multi-site functional connectivity analysis

Functional connectivity (FC), derived from resting-state functional magnetic resonance imaging (rs-fMRI), has been widely used to characterize brain abnormalities in disorders. FC is usually defined as a correlation matrix that is a symmetric positive definite (SPD) matrix lying on the Riemannian manifold. Recently, a number of learning-based methods have been proposed for FC analysis, while the geometric properties of Riemannian manifold have not yet been fully explored in previous studies. Also, most existing methods are designed to target one imaging site of fMRI data, which may result in limited training data for learning reliable and robust models. In this paper, we propose a novel Riemannian Manifold-based Disentangled Representation Learning (RM-DRL) framework which is capable of learning invariant representations from fMRI data across multiple sites for brain disorder diagnosis. In RM-DRL, we first employ an SPD-based encoder module to learn a latent unified representation of FC from different sites, which can preserve the Riemannian geometry of the SPD matrices. In latent space, a disentangled representation module is then designed to split the learned features into domain-specific and domain-invariant parts, respectively. Finally, a decoder module is introduced to ensure that sufficient information can be preserved during disentanglement learning. These designs allow us to introduce four types of training objectives to improve the disentanglement learning. Our RM-DRL method is evaluated on the public multi-site ABIDE dataset, showing superior performance compared with several state-of-the-art methods.

The impact of indirect structure on functional connectivity in schizophrenia using a multiplex brain network

It is known that abnormal functional connectivity (FC) in schizophrenia (SZ) is closely related to structural connectivity (SC). We speculate that indirect SC also have an impact on FC in SZ patients. Conventional single-layer network has limitations for studying the relationship between indirect SC and FC. Thus, this study constructed a multiplex network based on structural connectivity and functional connectivity (SC-FC). The SC-FC bandwidth and SC-FC cost are used to analyze the impact of indirect SC on FC. Moreover, this paper proposed mediation ability, mediation cost, mediated strength and mediated cost to quantify the effects of mediator nodes and mediated nodes on indirect SC. The results show that SZ patients exhibit lower SC-FC bandwidth and SC-FC cost compared to healthy controls (HC), which could be caused by the limbic and subcortical network (LSN), default mode network (DMN) and visual network (VN). The mediator and mediated nodes in indirect SC of SZ patients also showed diminished effects. These findings suggest that functional communication ability and cost in SZ patients are influenced by indirect SC. This study provides new perspectives for understanding the relationship between indirect SC and FC, and provides strong evidence for interpreting the physiological mechanisms of SZ patients.

Classifying disorders of consciousness using a novel dual-level and dual-modal graph learning model

BACKGROUND

Disorders of consciousness (DoC) are a group of conditions that affect the level of awareness and communication in patients. While neuroimaging techniques can provide useful information about the brain structure and function in these patients, most existing methods rely on a single modality for analysis and rarely account for brain injury. To address these limitations, we propose a novel method that integrates two neuroimaging modalities, functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), to enhance the classification of subjects into different states of consciousness.

METHOD AND RESULTS

The main contributions of our work are threefold: first, after constructing a dual-model individual graph using functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), we introduce a brain injury mask mechanism that consolidates damaged brain regions into a single graph node, enhancing the modeling of brain injuries and reducing deformation effects. Second, to address over-smoothing, we construct a dual-level graph that dynamically construct a population-level graph with node features from individual graphs, to promote the clustering of similar subjects while distinguishing dissimilar ones. Finally, we employ a subgraph exploration model with task-fMRI data to validate the interpretability of our model, confirming that the selected brain regions are task-relevant in cognition. Our experimental results on data from 89 healthy participants and 204 patients with DoC from Huashan Hospital, Fudan University, demonstrate that our method achieves high accuracy in classifying patients into unresponsive wakefulness syndrome (UWS), minimally conscious state (MCS), or normal conscious state, outperforming current state-of-the-art methods. The explainability results of our method identified a subset of brain regions that are important for consciousness, such as the default mode network, the salience network, the dorsal attention network, and the visual network. Our method also revealed the relationship between brain networks and language processing in consciousness, and showed that language-related subgraphs can distinguish MCS from UWS patients.

CONCLUSION

We proposed a novel graph learning method for classifying DoC based on fMRI and DTI data, introducing a brain injury mask mechanism to effectively handle damaged brains. The classification results demonstrate the effectiveness of our method in distinguishing subjects across different states of consciousness, while the explainability results identify key brain regions relevant to this classification. Our study provides new evidence for the role of brain networks and language processing in consciousness, with potential implications for improving the diagnosis and prognosis of patients with DoC.

Siamese Graph Convolutional Network quantifies increasing structure-function discrepancy over the cognitive decline continuum

BACKGROUND AND OBJECTIVE

Alzheimer's disease dementia (ADD) is well known to induce alterations in both structural and functional brain connectivity. However, reported changes in connectivity are mostly limited to global/local network features, which have poor specificity for diagnostic purposes. Following recent advances in machine learning, deep neural networks, particularly Graph Neural Network (GNN) based approaches, have found applications in brain research as well. The majority of existing applications of GNNs employ a single network (uni-modal or structure/function unified), despite the widely accepted view that there is a nontrivial interdependence between the brain's structural connectivity and the neural activity patterns, which is hypothesized to be disrupted in ADD. This disruption is quantified as a discrepancy score by the proposed "structure-function discrepancy learning network" (sfDLN) and its distribution is studied over the spectrum of clinical cognitive decline. The measured discrepancy score is utilized as a diagnostic biomarker and is compared with state-of-the-art diagnostic classifiers.

METHODS

sfDLN is a GNN with a siamese architecture built on the hypothesis that the mismatch between structural and functional connectivity patterns increases over the cognitive decline spectrum, starting from subjective cognitive impairment (SCI), passing through a mid-stage mild cognitive impairment (MCI), and ending up with ADD. The structural brain connectome (sNET) built using diffusion MRI-based tractography and the novel, sparse (lean) functional brain connectome (ℓNET) built using fMRI are input to sfDLN. The siamese sfDLN is trained to extract connectome representations and a discrepancy (dissimilarity) score that complies with the proposed hypothesis and is blindly tested on an MCI group.

RESULTS

The sfDLN generated structure-function discrepancy scores show high disparity between ADD and SCI subjects. Leave-one-out experiments of SCI-ADD classification over a cohort of 42 subjects reach 88% accuracy, surpassing state-of-the-art GNN-based classifiers in the literature. Furthermore, a blind assessment over a cohort of 46 MCI subjects confirmed that it captures the intermediary character of the MCI group. GNNExplainer module employed to investigate the anatomical determinants of the observed discrepancy confirms that sfDLN attends to cortical regions neurologically relevant to ADD.

CONCLUSION

In support of our hypothesis, the harmony between the structural and functional organization of the brain degrades with increasing cognitive decline. This discrepancy, shown to be rooted in brain regions neurologically relevant to ADD, can be quantified by sfDLN and outperforms state-of-the-art GNN-based ADD classification methods when used as a biomarker.

Continuous Dictionary of Nodes Model and Bilinear-Diffusion Representation Learning for Brain Disease Analysis

Brain networks based on functional magnetic resonance imaging (fMRI) provide a crucial perspective for diagnosing brain diseases. Representation learning has recently attracted tremendous attention due to its strong representation capability, which can be naturally applied to brain disease analysis. However, traditional representation learning only considers direct and local node interactions in original brain networks, posing challenges in constructing higher-order brain networks to represent indirect and extensive node interactions. To address this problem, we propose the Continuous Dictionary of Nodes model and Bilinear-Diffusion (CDON-BD) network for brain disease analysis. The CDON model is innovatively used to learn the original brain network, with its encoder weights directly regarded as latent features. To fully integrate latent features, we further utilize Bilinear Pooling to construct higher-order brain networks. The Diffusion Module is designed to capture extensive node interactions in higher-order brain networks. Compared to state-of-the-art methods, CDON-BD demonstrates competitive classification performance on two real datasets. Moreover, the higher-order representations learned by our method reveal brain regions relevant to the diseases, contributing to a better understanding of the pathology of brain diseases.

Identification of abnormal closed-loop pathways in patients with MRI-negative pharmacoresistant epilepsy

Epilepsy is a disorder of brain networks, that is usually combined with cognitive and emotional impairment. However, most of the current research on closed-loop pathways in epilepsy is limited to the neuronal level or has focused only on known closed-loop pathways, and studies on abnormalities in closed-loop pathways in epilepsy at the whole-brain network level are lacking. A total of 26 patients with magnetic resonance imaging-negative pharmacoresistant epilepsy (MRIneg-PRE) and 26 healthy controls (HCs) were included in this study. Causal brain networks and temporal-lag brain networks were constructed from resting-state functional MRI data, and the Johnson algorithm was used to identify stable closed-loop pathways. Abnormal closed-loop pathways in the MRIneg-PRE cohort compared with the HC group were identified, and the associations of these pathways with indicators of cognitive and emotional impairments were examined via Pearson correlation analysis. The results revealed that the abnormal stable closed-loop pathways were distributed across the frontal, parietal, and occipital lobes and included altered functional connectivity values both within and between cerebral hemispheres. Four abnormal closed-loop pathways in the occipital lobe were associated with emotional and cognitive impairments. These abnormal pathways may serve as biomarkers for the diagnosis and guidance of individualized treatments for MRIneg-PRE patients.

A Multiview Brain Network Transformer Fusing Individualized Information for Autism Spectrum Disorder Diagnosis

Functional connectivity (FC) networks, built from analyses of resting-state magnetic resonance imaging (rs-fMRI), serve as efficacious biomarkers for identifying Autism Spectrum Disorders (ASD) patients. Given the neurobiological heterogeneity across individuals and the unique presentation of ASD symptoms, the fusion of individualized information into diagnosis becomes essential. However, this aspect is overlooked in most methods. Furthermore, the existing methods typically focus on studying direct pairwise connections between brain ROIs, while disregarding interactions between indirectly connected neighbors. To overcome above challenges, we build common FC and individualized FC by tangent pearson embedding (TP) and common orthogonal basis extraction (COBE) respectively, and present a novel multiview brain transformer (MBT) aimed at effectively fusing common and indivinformation of subjects. MBT is mainly constructed by transformer layers with diffusion kernel (DK), fusion quality-inspired weighting module (FQW), similarity loss and orthonormal clustering fusion readout module (OCFRead). DK transformer can incorporate higher-order random walk methods to capture wider interactions among indirectly connected brain regions. FQW promotes adaptive fusion of features between views, and similarity loss and OCFRead are placed on the last layer to accomplish the ultimate integration of information. In our method, TP, DK and FQW modules all help to model wider connectivity in the brain that make up for the shortcomings of traditional methods. We conducted experiments on the public ABIDE dataset based on AAL and CC200 respectively. Our framework has shown promising results, outperforming state-of-the-art methods on both templates. This suggests its potential as a valuable approach for clinical ASD diagnosis.

Randomizing Human Brain Function Representation for Brain Disease Diagnosis

Resting-state fMRI (rs-fMRI) is an effective tool for quantifying functional connectivity (FC), which plays a crucial role in exploring various brain diseases. Due to the high dimensionality of fMRI data, FC is typically computed based on the region of interest (ROI), whose parcellation relies on a pre-defined atlas. However, utilizing the brain atlas poses several challenges including 1) subjective selection bias in choosing from various brain atlases, 2) parcellation of each subject's brain with the same atlas yet disregarding individual specificity; 3) lack of interaction between brain region parcellation and downstream ROI-based FC analysis. To address these limitations, we propose a novel randomizing strategy for generating brain function representation to facilitate neural disease diagnosis. Specifically, we randomly sample brain patches, thus avoiding ROI parcellations of the brain atlas. Then, we introduce a new brain function representation framework for the sampled patches. Each patch has its function description by referring to anchor patches, as well as the position description. Furthermore, we design an adaptive-selection-assisted Transformer network to optimize and integrate the function representations of all sampled patches within each brain for neural disease diagnosis. To validate our framework, we conduct extensive evaluations on three datasets, and the experimental results establish the effectiveness and generality of our proposed method, offering a promising avenue for advancing neural disease diagnosis beyond the confines of traditional atlas-based methods. Our code is available at https://github.com/mjliu2020/RandomFR.

End-stage renal disease accompanied by mild cognitive impairment: A study and analysis of trimodal brain network fusion

The function and structure of brain networks (BN) may undergo changes in patients with end-stage renal disease (ESRD), particularly in those accompanied by mild cognitive impairment (ESRDaMCI). Many existing methods for fusing BN focus on extracting interaction features between pairs of network nodes from each mode and combining them. This approach overlooks the correlation between different modal features during feature extraction and the potentially valuable information that may exist between more than two brain regions. To address this issue, we propose a model using a multi-head self-attention mechanism to fuse brain functional networks, white matter structural networks, and gray matter structural networks, which results in the construction of brain fusion networks (FBN). Initially, three networks are constructed: the brain function network, the white matter structure network, and the individual-based gray matter structure network. The multi-head self-attention mechanism is then applied to fuse the three types of networks, generating attention weights that are transformed into an optimized model. The optimized model introduces hypergraph popular regular term and L1 norm regular term, leading to the formation of FBN. Finally, FBN is employed in the diagnosis and prediction of ESRDaMCI to evaluate its classification performance and investigate the correlation between discriminative brain regions and cognitive dysfunction. Experimental results demonstrate that the optimal classification accuracy achieved is 92.80%, which is at least 3.63% higher than the accuracy attained using other methods. This outcome confirms the effectiveness of our proposed method. Additionally, the identification of brain regions significantly associated with scores on the Montreal cognitive assessment scale may shed light on the underlying pathogenesis of ESRDaMCI.

Constructing hierarchical attentive functional brain networks for early AD diagnosis

Analyzing functional brain networks (FBN) with deep learning has demonstrated great potential for brain disorder diagnosis. The conventional construction of FBN is typically conducted at a single scale with a predefined brain region atlas. However, numerous studies have identified that the structure and function of the brain are hierarchically organized in nature. This urges the need of representing FBN in a hierarchical manner for more effective analysis of the complementary diagnostic insights at different scales. To this end, this paper proposes to build hierarchical FBNs adaptively within the Transformer framework. Specifically, a sparse attention-based node-merging module is designed to work alongside the conventional network feature extraction modules in each layer. The proposed module generates coarser nodes for further FBN construction and analysis by combining fine-grained nodes. By stacking multiple such layers, a hierarchical representation of FBN can be adaptively learned in an end-to-end manner. The hierarchical structure can not only integrate the complementary information from multiscale FBN for joint analysis, but also reduce the model complexity due to decreasing node sizes. Moreover, this paper argues that the nodes defined by the existing atlases are not necessarily the optimal starting level to build FBN hierarchy and exploring finer nodes may further enrich the FBN representation. In this regard, each predefined node in an atlas is split into multiple sub-nodes, overcoming the scale limitation of the existing atlases. Extensive experiments conducted on various data sets consistently demonstrate the superior performance of the proposed method over the competing methods.

Beta-informativeness-diffusion multilayer graph embedding for brain network analysis

Brain network analysis provides essential insights into the diagnosis of brain disease. Integrating multiple neuroimaging modalities has been demonstrated to be more effective than using a single modality for brain network analysis. However, a majority of existing brain network analysis methods based on multiple modalities often overlook both complementary information and unique characteristics from various modalities. To tackle this issue, we propose the Beta-Informativeness-Diffusion Multilayer Graph Embedding (BID-MGE) method. The proposed method seamlessly integrates structural connectivity (SC) and functional connectivity (FC) to learn more comprehensive information for diagnosing neuropsychiatric disorders. Specifically, a novel beta distribution mapping function (beta mapping) is utilized to increase vital information and weaken insignificant connections. The refined information helps the diffusion process concentrate on crucial brain regions to capture more discriminative features. To maximize the preservation of the unique characteristics of each modality, we design an optimal scale multilayer brain network, the inter-layer connections of which depend on node informativeness. Then, a multilayer informativeness diffusion is proposed to capture complementary information and unique characteristics from various modalities and generate node representations by incorporating the features of each node with those of their connected nodes. Finally, the node representations are reconfigured using principal component analysis (PCA), and cosine distances are calculated with reference to multiple templates for statistical analysis and classification. We implement the proposed method for brain network analysis of neuropsychiatric disorders. The results indicate that our method effectively identifies crucial brain regions associated with diseases, providing valuable insights into the pathology of the disease, and surpasses other advanced methods in classification performance.

A Multi-Graph Cross-Attention-Based Region-Aware Feature Fusion Network Using Multi-Template for Brain Disorder Diagnosis

Functional connectivity (FC) networks based on resting-state functional magnetic imaging (rs-fMRI) are reliable and sensitive for brain disorder diagnosis. However, most existing methods are limited by using a single template, which may be insufficient to reveal complex brain connectivities. Furthermore, these methods usually neglect the complementary information between static and dynamic brain networks, and the functional divergence among different brain regions, leading to suboptimal diagnosis performance. To address these limitations, we propose a novel multi-graph cross-attention based region-aware feature fusion network (MGCA-RAFFNet) by using multi-template for brain disorder diagnosis. Specifically, we first employ multi-template to parcellate the brain space into different regions of interest (ROIs). Then, a multi-graph cross-attention network (MGCAN), including static and dynamic graph convolutions, is developed to explore the deep features contained in multi-template data, which can effectively analyze complex interaction patterns of brain networks for each template, and further adopt a dual-view cross-attention (DVCA) to acquire complementary information. Finally, to efficiently fuse multiple static-dynamic features, we design a region-aware feature fusion network (RAFFNet), which is beneficial to improve the feature discrimination by considering the underlying relations among static-dynamic features in different brain regions. Our proposed method is evaluated on both public ADNI-2 and ABIDE-I datasets for diagnosing mild cognitive impairment (MCI) and autism spectrum disorder (ASD). Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/mylbuaa/MGCA-RAFFNet.

A Causality-Driven Graph Convolutional Network for Postural Abnormality Diagnosis in Parkinsonians

Abnormal posture is a common movement disorder in the progress of Parkinson's disease (PD), and this abnormality can increase the risk of falls or even disabilities. The conventional assessment approach depends on the judgment of well-trained experts via canonical scales. However, this approach requires extensive clinical expertise and is highly subjective. Considering the potential of quantitative susceptibility mapping (QSM) in PD diagnosis, this study explored the QSM-based method for the automated classification between PD patients with and without postural abnormalities. Nevertheless, a major challenge is that unstable non-causal features typically lead to less reliable performance. Therefore, we propose a causality-driven graph-convolutional-network framework based on multi-instance learning, where performance stability is enhanced through the invariant prediction principle and causal interventions. Specifically, we adopt an intervention strategy that combines a non-causal intervenor with causal prediction. A stability constraint is proposed to ensure robust integrated prediction under different interventions. Moreover, an intra-class homogeneity constraint is enforced for each individually-learned causality scoring module to promote the extraction of group-level general features, and hence achieve a balance between subject-specific and group-level features. The proposed method demonstrated promising performance through extensive experiments on a real clinical dataset. Also, the features extracted by our method coincide with those reported in previous medical studies on PD posture abnormalities. In general, our work provides a clinically-valuable approach for automated, objective, and reliable diagnosis of postural abnormalities in Parkinsonians. Our source code is publicly available at https://github.com/SJTUBME-QianLab/CausalGCN-PDPA.

A comprehensive survey of complex brain network representation

Recent years have shown great merits in utilizing neuroimaging data to understand brain structural and functional changes, as well as its relationship to different neurodegenerative diseases and other clinical phenotypes. Brain networks, derived from different neuroimaging modalities, have attracted increasing attention due to their potential to gain system-level insights to characterize brain dynamics and abnormalities in neurological conditions. Traditional methods aim to pre-define multiple topological features of brain networks and relate these features to different clinical measures or demographical variables. With the enormous successes in deep learning techniques, graph learning methods have played significant roles in brain network analysis. In this survey, we first provide a brief overview of neuroimaging-derived brain networks. Then, we focus on presenting a comprehensive overview of both traditional methods and state-of-the-art deep-learning methods for brain network mining. Major models, and objectives of these methods are reviewed within this paper. Finally, we discuss several promising research directions in this field.

Multimodal Classification Framework Based on Hypergraph Latent Relation for End-Stage Renal Disease Associated with Mild Cognitive Impairment

Combined arterial spin labeling (ASL) and functional magnetic resonance imaging (fMRI) can reveal more comprehensive properties of the spatiotemporal and quantitative properties of brain networks. Imaging markers of end-stage renal disease associated with mild cognitive impairment (ESRDaMCI) will be sought from these properties. The current multimodal classification methods often neglect to collect high-order relationships of brain regions and remove noise from the feature matrix. A multimodal classification framework is proposed to address this issue using hypergraph latent relation (HLR). A brain functional network with hypergraph structural information is constructed by fMRI data. The feature matrix is obtained through graph theory (GT). The cerebral blood flow (CBF) from ASL is selected as the second modal feature matrix. Then, the adaptive similarity matrix is constructed by learning the latent relation between feature matrices. Latent relation adaptive similarity learning (LRAS) is introduced to multi-task feature learning to construct a multimodal feature selection method based on latent relation (LRMFS). The experimental results show that the best classification accuracy (ACC) reaches 88.67%, at least 2.84% better than the state-of-the-art methods. The proposed framework preserves more valuable information between brain regions and reduces noise among feature matrixes. It provides an essential reference value for ESRDaMCI recognition.

Graph Neural Networks in Network Neuroscience

Noninvasive medical neuroimaging has yielded many discoveries about the brain connectivity. Several substantial techniques mapping morphological, structural and functional brain connectivities were developed to create a comprehensive road map of neuronal activities in the human brain -namely brain graph. Relying on its non-euclidean data type, graph neural network (GNN) provides a clever way of learning the deep graph structure and it is rapidly becoming the state-of-the-art leading to enhanced performance in various network neuroscience tasks. Here we review current GNN-based methods, highlighting the ways that they have been used in several applications related to brain graphs such as missing brain graph synthesis and disease classification. We conclude by charting a path toward a better application of GNN models in network neuroscience field for neurological disorder diagnosis and population graph integration. The list of papers cited in our work is available at https://github.com/basiralab/GNNs-in-Network-Neuroscience.

Tree-structured neural networks: Spatiotemporal dynamics and optimal control

How the network topology drives the response dynamic is a basic question that has not yet been fully answered in neural networks. Elucidating the internal relation between topological structures and dynamics is instrumental in our understanding of brain function. Recent studies have revealed that the ring structure and star structure have a great influence on the dynamical behavior of neural networks. In order to further explore the role of topological structures in the response dynamic, we construct a new tree structure that differs from the ring structure and star structure of traditional neural networks. Considering the diffusion effect, we propose a diffusion neural network model with binary tree structure and multiple delays. How to design control strategies to optimize brain function has also been an open question. Thus, we put forward a novel full-dimensional nonlinear state feedback control strategy to optimize relevant neurodynamics. Some conditions about the local stability and Hopf bifurcation are obtained, and it is proved that the Turing instability does not occur. Moreover, for the formation of the spatially homogeneous periodic solution, some diffusion conditions are also fused together. Finally, several numerical examples are carried out to illustrate the results' correctness. Meanwhile, some comparative experiments are rendered to reveal the effectiveness of the proposed control strategy.

Class activation attention transfer neural networks for MCI conversion prediction

Accurate prediction of the trajectory of Alzheimer's disease (AD) from an early stage is of substantial value for treatment and planning to delay the onset of AD. We propose a novel attention transfer method to train a 3D convolutional neural network to predict which patients with mild cognitive impairment (MCI) will progress to AD within 3 years. A model is first trained on a separate but related source task (task we are transferring information from) to automatically learn regions of interest (ROI) from a given image. Next we train a model to simultaneously classify progressive MCI (pMCI) and stable MCI (sMCI) (the target task we want to solve) and the ROIs learned from the source task. The predicted ROIs are then used to focus the model's attention on certain areas of the brain when classifying pMCI versus sMCI. Thus, in contrast to traditional transfer learning, we transfer attention maps instead of transferring model weights from a source task to the target classification task. Our Method outperformed all methods tested including traditional transfer learning and methods that used expert knowledge to define ROI. Furthermore, the attention map transferred from the source task highlights known Alzheimer's pathology.

Multi-Graph Attention Networks With Bilinear Convolution for Diagnosis of Schizophrenia

The explorations of brain functional connectivity (FC) network using resting-state functional magnetic resonance imaging (rs-fMRI) can provide crucial insights into discriminative analysis of neuropsychiatric disorders such as schizophrenia (SZ). Graph attention network (GAT), which could capture the local stationary on the network topology and aggregate the features of neighboring nodes, has advantages in learning the feature representation of brain regions. However, GAT only can obtain the node-level features that reflect local information, ignoring the spatial information within the connectivity-based features that proved to be important for SZ diagnosis. In addition, existing graph learning techniques usually rely on a single graph topology to represent neighborhood information, and only consider a single correlation measure for connectivity features. Comprehensive analysis of multiple graph topologies and multiple measures of FC can leverage their complementary information that may contribute to identifying patients. In this paper, we propose a multi-graph attention network (MGAT) with bilinear convolution (BC) neural network framework for SZ diagnosis and functional connectivity analysis. Besides multiple correlation measures to construct connectivity networks from different perspectives, we further propose two different graph construction methods to capture both the low- and high-level graph topologies, respectively. Especially, the MGAT module is developed to learn multiple node interaction features on each graph topology, and the BC module is utilized to learn the spatial connectivity features of the brain network for disease prediction. Importantly, the rationality and advantages of our proposed method can be validated by the experiments on SZ identification. Therefore, we speculate that this framework may also be potentially used as a diagnostic tool for other neuropsychiatric disorders.

Towards better interpretable and generalizable AD detection using collective artificial intelligence

Alzheimer's Disease is the most common cause of dementia. Accurate diagnosis and prognosis of this disease are essential to design an appropriate treatment plan, increasing the life expectancy of the patient. Intense research has been conducted on the use of machine learning to identify Alzheimer's Disease from neuroimaging data, such as structural magnetic resonance imaging. In recent years, advances of deep learning in computer vision suggest a new research direction for this problem. Current deep learning-based approaches in this field, however, have a number of drawbacks, including the interpretability of model decisions, a lack of generalizability information and a lower performance compared to traditional machine learning techniques. In this paper, we design a two-stage framework to overcome these limitations. In the first stage, an ensemble of 125 U-Nets is used to grade the input image, producing a 3D map that reflects the disease severity at voxel-level. This map can help to localize abnormal brain areas caused by the disease. In the second stage, we model a graph per individual using the generated grading map and other information about the subject. We propose to use a graph convolutional neural network classifier for the final classification. As a result, our framework demonstrates comparative performance to the state-of-the-art methods in different datasets for both diagnosis and prognosis. We also demonstrate that the use of a large ensemble of U-Nets offers a better generalization capacity for our framework.

An explainable autoencoder with multi-paradigm fMRI fusion for identifying differences in dynamic functional connectivity during brain development

Multi-paradigm deep learning models show great potential for dynamic functional connectivity (dFC) analysis by integrating complementary information. However, many of them cannot use information from different paradigms effectively and have poor explainability, that is, the ability to identify significant features that contribute to decision making. In this paper, we propose a multi-paradigm fusion-based explainable deep sparse autoencoder (MF-EDSAE) to address these issues. Considering explainability, the MF-EDSAE is constructed based on a deep sparse autoencoder (DSAE). For integrating information effectively, the MF-EDASE contains the nonlinear fusion layer and multi-paradigm hypergraph regularization. We apply the model to the Philadelphia Neurodevelopmental Cohort and demonstrate it achieves better performance in detecting dynamic FC (dFC) that differ significantly during brain development than the single-paradigm DSAE. The experimental results show that children have more dispersive dFC patterns than adults. The function of the brain transits from undifferentiated systems to specialized networks during brain development. Meanwhile, adults have stronger connectivities between task-related functional networks for a given task than children. As the brain develops, the patterns of the global dFC change more quickly when stimulated by a task.

GATE: Graph CCA for Temporal Self-Supervised Learning for Label-Efficient fMRI Analysis

In this work, we focus on the challenging task, neuro-disease classification, using functional magnetic resonance imaging (fMRI). In population graph-based disease analysis, graph convolutional neural networks (GCNs) have achieved remarkable success. However, these achievements are inseparable from abundant labeled data and sensitive to spurious signals. To improve fMRI representation learning and classification under a label-efficient setting, we propose a novel and theory-driven self-supervised learning (SSL) framework on GCNs, namely Graph CCA for Temporal sElf-supervised learning on fMRI analysis (GATE). Concretely, it is demanding to design a suitable and effective SSL strategy to extract formation and robust features for fMRI. To this end, we investigate several new graph augmentation strategies from fMRI dynamic functional connectives (FC) for SSL training. Further, we leverage canonical-correlation analysis (CCA) on different temporal embeddings and present the theoretical implications. Consequently, this yields a novel two-step GCN learning procedure comprised of (i) SSL on an unlabeled fMRI population graph and (ii) fine-tuning on a small labeled fMRI dataset for a classification task. Our method is tested on two independent fMRI datasets, demonstrating superior performance on autism and dementia diagnosis. Our code is available at https://github.com/LarryUESTC/GATE.

Hypergraph representation of multimodal brain networks for patients with end-stage renal disease associated with mild cognitive impairment

The structure and function of brain networks (BN) may be altered in patients with end-stage renal disease (ESRD). However, there are relatively few attentions on ESRD associated with mild cognitive impairment (ESRDaMCI). Most studies focus on the pairwise relationships between brain regions, without taking into account the complementary information of functional connectivity (FC) and structural connectivity (SC). To address the problem, a hypergraph representation method is proposed to construct a multimodal BN for ESRDaMCI. First, the activity of nodes is determined by connection features extracted from functional magnetic resonance imaging (fMRI) (i.e., FC), and the presence of edges is determined by physical connections of nerve fibers extracted from diffusion kurtosis imaging (DKI) (i.e., SC). Then, the connection features are generated through bilinear pooling and transformed into an optimization model. Next, a hypergraph is constructed according to the generated node representation and connection features, and the node degree and edge degree of the hypergraph are calculated to obtain the hypergraph manifold regularization (HMR) term. The HMR and L₁ norm regularization terms are introduced into the optimization model to achieve the final hypergraph representation of multimodal BN (HRMBN). Experimental results show that the classification performance of HRMBN is significantly better than that of several state-of-the-art multimodal BN construction methods. Its best classification accuracy is 91.0891%, at least 4.3452% higher than that of other methods, verifying the effectiveness of our method. The HRMBN not only achieves better results in ESRDaMCI classification, but also identifies the discriminative brain regions of ESRDaMCI, which provides a reference for the auxiliary diagnosis of ESRD.

Brain Age Prediction in Developing Childhood with Multimodal Magnetic Resonance Images

It is well known that brain development is very fast and complex in the early childhood with age-based neurological and physiological changes of brain structure and function. The brain maturity is an important indicator for evaluating the normal development of children. In this paper, we propose a multimodal regression framework to combine the features from structural magnetic resonance imaging (sMRI) and diffusion tensor imaging (DTI) data for age prediction of children. First, three types of features are extracted from sMRI and DTI data. Second, we propose to combine the sparse coding and Q-Learning for feature selection from each modality. Finally, the ensemble regression is performed by random forest based on proximity measures to fuse multimodal features for age prediction. The proposed method is evaluated on 212 participants, including 76 young children less than 2 years old and 136 children aged from 2-15 years old recruited from Shanghai Children's Hospital. The results show that integrating multimodal features has achieved the highest accuracies with the root mean squared error (RMSE) of 0.208 years and mean absolute error (MAE) of 0.150 years for age prediction of young children (0-2), and RMSE of 1.666 years and MAE of 1.087 years for older children (2-15). We have shown that the selected features by Q-Learning can consistently improve the prediction accuracy. The comparison of prediction results demonstrates that the proposed method performs better than other competing methods.

Altered higher-order coupling between brain structure and function with embedded vector representations of connectomes in schizophrenia

It has been shown that the functional dependency of the brain exists in both direct and indirect regional relationships. Therefore, it is necessary to map higher-order coupling in brain structure and function to understand brain dynamic. However, how to quantify connections between not directly regions remains unknown to schizophrenia. The word2vec is a common algorithm through create embeddings of words to solve these problems. We apply the node2vec embedding representation to characterize features on each node, their pairwise relationship can give rise to correspondence relationships between brain regions. Then we adopt pearson correlation to quantify the higher-order coupling between structure and function in normal controls and schizophrenia. In addition, we construct direct and indirect connections to quantify the coupling between their respective functional connections. The results showed that higher-order coupling is significantly higher in schizophrenia. Importantly, the anomalous cause of coupling mainly focus on indirect structural connections. The indirect structural connections play an essential role in functional connectivity–structural connectivity (SC–FC) coupling. The similarity between embedded representations capture more subtle network underlying information, our research provides new perspectives for understanding SC–FC coupling. A strong indication that the structural backbone of the brain has an intimate influence on the resting-state functional.

Diffusion Kernel Attention Network for Brain Disorder Classification

Constructing and analyzing functional brain networks (FBN) has become a promising approach to brain disorder classification. However, the conventional successive construct-and-analyze process would limit the performance due to the lack of interactions and adaptivity among the subtasks in the process. Recently, Transformer has demonstrated remarkable performance in various tasks, attributing to its effective attention mechanism in modeling complex feature relationships. In this paper, for the first time, we develop Transformer for integrated FBN modeling, analysis and brain disorder classification with rs-fMRI data by proposing a Diffusion Kernel Attention Network to address the specific challenges. Specifically, directly applying Transformer does not necessarily admit optimal performance in this task due to its extensive parameters in the attention module against the limited training samples usually available. Looking into this issue, we propose to use kernel attention to replace the original dot-product attention module in Transformer. This significantly reduces the number of parameters to train and thus alleviates the issue of small sample while introducing a non-linear attention mechanism to model complex functional connections. Another limit of Transformer for FBN applications is that it only considers pair-wise interactions between directly connected brain regions but ignores the important indirect connections. Therefore, we further explore diffusion process over the kernel attention to incorporate wider interactions among indirectly connected brain regions. Extensive experimental study is conducted on ADHD-200 data set for ADHD classification and on ADNI data set for Alzheimer's disease classification, and the results demonstrate the superior performance of the proposed method over the competing methods.

Predicting brain structural network using functional connectivity

Uncovering the non-trivial brain structure-function relationship is fundamentally important for revealing organizational principles of human brain. However, it is challenging to infer a reliable relationship between individual brain structure and function, e.g., the relations between individual brain structural connectivity (SC) and functional connectivity (FC). Brain structure-function displays a distributed and heterogeneous pattern, that is, many functional relationships arise from non-overlapping sets of anatomical connections. This complex relation can be interwoven with widely existed individual structural and functional variations. Motivated by the advances of generative adversarial network (GAN) and graph convolutional network (GCN) in the deep learning field, in this work, we proposed a multi-GCN based GAN (MGCN-GAN) to infer individual SC based on corresponding FC by automatically learning the complex associations between individual brain structural and functional networks. The generator of MGCN-GAN is composed of multiple multi-layer GCNs which are designed to model complex indirect connections in brain network. The discriminator of MGCN-GAN is a single multi-layer GCN which aims to distinguish the predicted SC from real SC. To overcome the inherent unstable behavior of GAN, we designed a new structure-preserving (SP) loss function to guide the generator to learn the intrinsic SC patterns more effectively. Using Human Connectome Project (HCP) dataset and Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset as test beds, our MGCN-GAN model can generate reliable individual SC from FC. This result implies that there may exist a common regulation between specific brain structural and functional architectures across different individuals.

Classification of EEG Signal-Based Encephalon Magnetic Signs for Identification of Epilepsy-Based Neurological Disorder

Magnetoencephalography (MEG) is now widely used in clinical examinations and medical research in many fields. Resting-state magnetoencephalography-based brain network analysis can be used to study the physiological or pathological mechanisms of the brain. Furthermore, magnetoencephalography analysis has a significant reference value for the diagnosis of epilepsy. The scope of the proposed research is that this research demonstrates how to locate spikes in the phase locking functional brain connectivity network of the Desikan-Killiany brain region division using a neural network approach. It also improves detection accuracy and reduces missed and false detection rates. The automatic classification of epilepsy encephalomagnetic signals can make timely judgments on the patient's condition, which is of tremendous clinical significance. The existing literature's research on the automatic type of epilepsy EEG signals is relatively sufficient, but the research on epilepsy EEG signals is relatively weak. A full-band machine learning automatic discrimination method of epilepsy brain magnetic spikes based on the brain functional connection network is proposed. The four classifiers are comprehensively compared. The classifier with the best effect is selected, and the discrimination accuracy can reach 93.8%. Therefore, this method has a good application prospect in automatically identifying and labeling epileptic spikes in magnetoencephalography.

BMNet: A New Region-Based Metric Learning Method for Early Alzheimer's Disease Identification With FDG-PET Images

18F-fluorodeoxyglucose (FDG)-positron emission tomography (PET) reveals altered brain metabolism in individuals with mild cognitive impairment (MCI) and Alzheimer's disease (AD). Some biomarkers derived from FDG-PET by computer-aided-diagnosis (CAD) technologies have been proved that they can accurately diagnosis normal control (NC), MCI, and AD. However, existing FDG-PET-based researches are still insufficient for the identification of early MCI (EMCI) and late MCI (LMCI). Compared with methods based other modalities, current methods with FDG-PET are also inadequate in using the inter-region-based features for the diagnosis of early AD. Moreover, considering the variability in different individuals, some hard samples which are very similar with both two classes limit the classification performance. To tackle these problems, in this paper, we propose a novel bilinear pooling and metric learning network (BMNet), which can extract the inter-region representation features and distinguish hard samples by constructing the embedding space. To validate the proposed method, we collect 898 FDG-PET images from Alzheimer's disease neuroimaging initiative (ADNI) including 263 normal control (NC) patients, 290 EMCI patients, 147 LMCI patients, and 198 AD patients. Following the common preprocessing steps, 90 features are extracted from each FDG-PET image according to the automatic anatomical landmark (AAL) template and then sent into the proposed network. Extensive fivefold cross-validation experiments are performed for multiple two-class classifications. Experiments show that most metrics are improved after adding the bilinear pooling module and metric losses to the Baseline model respectively. Specifically, in the classification task between EMCI and LMCI, the specificity improves 6.38% after adding the triple metric loss, and the negative predictive value (NPV) improves 3.45% after using the bilinear pooling module. In addition, the accuracy of classification between EMCI and LMCI achieves 79.64% using imbalanced FDG-PET images, which illustrates that the proposed method yields a state-of-the-art result of the classification accuracy between EMCI and LMCI based on PET images.

Multimodal Brain Network Jointly Construction and Fusion for Diagnosis of Epilepsy

Brain network analysis has been proved to be one of the most effective methods in brain disease diagnosis. In order to construct discriminative brain networks and improve the performance of disease diagnosis, many machine learning–based methods have been proposed. Recent studies show that combining functional and structural brain networks is more effective than using only single modality data. However, in the most of existing multi-modal brain network analysis methods, it is a common strategy that constructs functional and structural network separately, which is difficult to embed complementary information of different modalities of brain network. To address this issue, we propose a unified brain network construction algorithm, which jointly learns both functional and structural data and effectively face the connectivity and node features for improving classification. First, we conduct space alignment and brain network construction under a unified framework, and then build the correlation model among all brain regions with functional data by low-rank representation so that the global brain region correlation can be captured. Simultaneously, the local manifold with structural data is embedded into this model to preserve the local structural information. Second, the PageRank algorithm is adaptively used to evaluate the significance of different brain regions, in which the interaction of multiple brain regions is considered. Finally, a multi-kernel strategy is utilized to solve the data heterogeneity problem and merge the connectivity as well as node information for classification. We apply the proposed method to the diagnosis of epilepsy, and the experimental results show that our method can achieve a promising performance.

A Survey on Deep Learning for Neuroimaging-Based Brain Disorder Analysis

Deep learning has recently been used for the analysis of neuroimages, such as structural magnetic resonance imaging (MRI), functional MRI, and positron emission tomography (PET), and it has achieved significant performance improvements over traditional machine learning in computer-aided diagnosis of brain disorders. This paper reviews the applications of deep learning methods for neuroimaging-based brain disorder analysis. We first provide a comprehensive overview of deep learning techniques and popular network architectures by introducing various types of deep neural networks and recent developments. We then review deep learning methods for computer-aided analysis of four typical brain disorders, including Alzheimer's disease, Parkinson's disease, Autism spectrum disorder, and Schizophrenia, where the first two diseases are neurodegenerative disorders and the last two are neurodevelopmental and psychiatric disorders, respectively. More importantly, we discuss the limitations of existing studies and present possible future directions.

A novel node-level structure embedding and alignment representation of structural networks for brain disease analysis

Brain networks based on various neuroimaging technologies, such as diffusion tensor image (DTI) and functional magnetic resonance imaging (fMRI), have been widely applied to brain disease analysis. Currently, there are several node-level structural measures (e.g., local clustering coefficients and node degrees) for representing and analyzing brain networks since they usually can reflect the topological structure of brain regions. However, these measures typically describe specific types of structural information, ignoring important network properties (i.e., small structural changes) that could further improve the performance of brain network analysis. To overcome this problem, in this paper, we first define a novel node-level structure embedding and alignment (nSEA) representation to accurately characterize the node-level structural information of the brain network. Different from existing measures that characterize a specific type of structural properties with a single value, our proposed nSEA method can learn a vector representation for each node, thus contain richer structure information to capture small structural changes. Furthermore, we develop an nSEA representation based learning (nSEAL) framework for brain disease analysis. Specifically, we first perform structural embedding to calculate node vector representations for each brain network and then align vector representations of all brain networks into the common space for two group-level network analyses, including a statistical analysis and brain disease classifications. Experiment results on a real schizophrenia dataset demonstrate that our proposed method not only discover disease-related brain regions that could help to better understand the pathology of brain diseases, but also improve the classification performance of brain diseases, compared with state-of-the-art methods.