Adversarial Learning Based Node-Edge Graph Attention Networks for Autism Spectrum Disorder Identification

Toward Integrating Federated Learning With Split Learning via Spatio-Temporal Graph Framework for Brain Disease Prediction

Functional Magnetic Resonance Imaging (fMRI) is used for extracting blood oxygen signals from brain regions to map brain functional connectivity for brain disease prediction. Despite its effectiveness, fMRI has not been widely used: on the one hand, collecting and labeling the data is time-consuming and costly, which limits the amount of valid data collected at a single healthcare site; on the other hand, integrating data from multiple sites is challenging due to data privacy restrictions. To address these issues, we propose a novel, integrated Federated learning and Split learning Spatio-temporal Graph framework (F G). Specifically, we introduce federated learning and split learning techniques to split a spatio-temporal model into a client temporal model and a server spatial model. In the client temporal model, we propose a time-aware mechanism to focus on changes in brain functional states and use an InceptionTime model to extract information about changes in the brain states of each subject. In the server spatial model, we propose a united graph convolutional network to integrate multiple graph convolutional networks. Integrating federated learning and split learning, F G can utilize multi-site fMRI data without violating data privacy protection and reduce the risk of overfitting as it is capable of learning from limited training data sets. Moreover, it boosts the extraction of spatio-temporal features of fMRI using spatio-temporal graph networks. Experiments on ABIDE and ADHD200 datasets demonstrate that our proposed method outperforms state-of-the-art methods. In addition, we explore biomarkers associated with brain disease prediction using community discovery algorithms using intermediate results of F G. The source code is available at https://github.com/yutian0315/FS2G.

A short investigation of the effect of the selection of human brain atlases on the performance of ASD's classification models

This study investigated the impact of brain atlas selection on the classification accuracy of Autism Spectrum Disorder (ASD) models using functional Magnetic Resonance Imaging (fMRI) data. Brain atlases, such as AAL, CC200, Harvard-Oxford, and Yeo 7/17, are used to define regions of interest (ROIs) for fMRI analysis and play a crucial role in enabling researchers to study connectivity patterns and neural dynamics in ASD patients. Through a systematic review, we examined the performance of different atlases in various machine-learning and deep-learning frameworks for ASD classification. The results reveal that atlas selection significantly affects classification accuracy, with denser atlases, such as CC400, providing higher granularity, whereas coarser atlases such as AAL, offer computational efficiency. Furthermore, we discuss the dynamics of combining multiple atlases to enhance feature extraction and explore the implications of atlas selection across diverse datasets. Our findings emphasize the need for standardized approaches to atlas selection and highlight future research directions, including the integration of novel atlases, advanced data augmentation techniques, and end-to-end deep-learning models. This study provides valuable insights into optimizing fMRI-based ASD diagnosis and underscores the importance of interpreting atlas-specific features for an improved understanding of brain connectivity in ASD.

MHNet: Multi-view High-Order Network for Diagnosing Neurodevelopmental Disorders Using Resting-State fMRI

Deep learning models have shown promise in diagnosing neurodevelopmental disorders (NDD) like ASD and ADHD. However, many models either use graph neural networks (GNN) to construct single-level brain functional networks (BFNs) or employ spatial convolution filtering for local information extraction from rs-fMRI data, often neglecting high-order features crucial for NDD classification. We introduce a Multi-view High-order Network (MHNet) to capture hierarchical and high-order features from multi-view BFNs derived from rs-fMRI data for NDD prediction. MHNet has two branches: the Euclidean Space Features Extraction (ESFE) module and the Non-Euclidean Space Features Extraction (Non-ESFE) module, followed by a Feature Fusion-based Classification (FFC) module for NDD identification. ESFE includes a Functional Connectivity Generation (FCG) module and a High-order Convolutional Neural Network (HCNN) module to extract local and high-order features from BFNs in Euclidean space. Non-ESFE comprises a Generic Internet-like Brain Hierarchical Network Generation (G-IBHN-G) module and a High-order Graph Neural Network (HGNN) module to capture topological and high-order features in non-Euclidean space. Experiments on three public datasets show that MHNet outperforms state-of-the-art methods using both AAL1 and Brainnetome Atlas templates. Extensive ablation studies confirm the superiority of MHNet and the effectiveness of using multi-view fMRI information and high-order features. Our study also offers atlas options for constructing more sophisticated hierarchical networks and explains the association between key brain regions and NDD. MHNet leverages multi-view feature learning from both Euclidean and non-Euclidean spaces, incorporating high-order information from BFNs to enhance NDD classification performance.

mGNN-bw: Multi-Scale Graph Neural Network Based on Biased Random Walk Path Aggregation for ASD Diagnosis

In recent years, computationally assisted diagnosis for classifying autism spectrum disorder (ASD) and typically developing (TD) individuals based on neuroimaging data, such as functional magnetic resonance imaging (fMRI), has garnered significant attention. Studies have shown that long-range functional connectivity patterns in ASD patients exhibit significant abnormalities, and individual brain networks display considerable heterogeneity. However, current graph neural networks (GNNs) used in ASD research have failed to adequately capture long-range connectivity and have overlooked individual differences. To address these limitations, this study proposes a novel multi-scale graph neural network based on biased random walks (mGNN-bw). The model introduces a co-optimization strategy between sub-models and the main model, leveraging node pooling scores from sub-models to guide biased random walks, effectively capturing long-range connectivity. By constructing high-order brain networks through path encoding and aggregation, and integrating them with low-order brain networks based on Pearson correlation, the model achieves a robust multi-scale feature representation. Experimental results on the publicly available ABIDE I dataset demonstrate the superior performance of our approach, achieving accuracy rates of 74.8% and 73.2% using CC200 and AAL atlases, respectively, outperforming existing methods. Additionally, the model identifies key ASD-associated brain regions, including the frontal lobe, insula, cingulate, and calcarine, supported by existing research. The proposed method significantly contributes to the clinical diagnosis of ASD.

An objective quantitative diagnosis of depression using a local-to-global multimodal fusion graph neural network

This study developed an artificial intelligence (AI) system using a local-global multimodal fusion graph neural network (LGMF-GNN) to address the challenge of diagnosing major depressive disorder (MDD), a complex disease influenced by social, psychological, and biological factors. Utilizing functional MRI, structural MRI, and electronic health records, the system offers an objective diagnostic method by integrating individual brain regions and population data. Tested across cohorts from China, Japan, and Russia with 1,182 healthy controls and 1,260 MDD patients from 24 institutions, it achieved a classification accuracy of 78.75%, an area under the receiver operating characteristic curve (AUROC) of 80.64%, and correctly identified MDD subtypes. The system further discovered distinct brain connectivity patterns in MDD, including reduced functional connectivity between the left gyrus rectus and right cerebellar lobule VIIB, and increased connectivity between the left Rolandic operculum and right hippocampus. Anatomically, MDD is associated with thickness changes of the gray and white matter interface, indicating potential neuropathological conditions or brain injuries.

Site-Invariant Meta-Modulation Learning for Multisite Autism Spectrum Disorders Diagnosis

Large amounts of fMRI data are essential to building generalized predictive models for brain disease diagnosis. In order to conduct extensive data analysis, it is often necessary to gather data from multiple organizations. However, the site variation inherent in multisite resting-state functional magnetic resonance imaging (rs-fMRI) leads to unfavorable heterogeneity in data distribution, negatively impacting the identification of biomarkers and the diagnostic decision. Several existing methods have alleviated this shift of domain distribution (i.e., multisite problem). Statistical tuning schemes directly regress out site disparity factors from the data prior to model training. Such methods have a limitation in processing data each time through variance estimation according to the added site. In the model adjustment approaches, domain adaptation (DA) methods adjust the features or models of the source domain according to the target domain during model training. Thus, it is inevitable that it needs updating model parameters according to the samples of a target site, causing great limitations in practical applicability. Meanwhile, the approach of domain generalization (DG) aims to create a universal model that can be quickly adapted to multiple domains. In this study, we propose a novel framework for disease diagnosis that alleviates the multisite problem by adaptively calibrating site-specific features into site-invariant features. Specifically, it applies directly to samples from unseen sites without the need for fine-tuning. With a learning-to-learn strategy that learns how to calibrate the features under the various domain shift environments, our novel modulation mechanism extracts site-invariant features. In our experiments over the Autism Brain Imaging Data Exchange (ABIDE I and II) dataset, we validated the generalization ability of the proposed network by improving diagnostic accuracy in both seen and unseen multisite samples.

Bilinear Perceptual Fusion Algorithm Based on Brain Functional and Structural Data for ASD Diagnosis and Regions of Interest Identification

Autism spectrum disorder (ASD) is a serious mental disorder with a complex pathogenesis mechanism and variable presentation among individuals. Although many deep learning algorithms have been used to diagnose ASD, most of them focus on a single modality of data, resulting in limited information extraction and poor stability. In this paper, we propose a bilinear perceptual fusion (BPF) algorithm that leverages data from multiple modalities. In our algorithm, different schemes are used to extract features according to the characteristics of functional and structural data. Through bilinear operations, the associations between the functional and structural features of each region of interest (ROI) are captured. Then the associations are used to integrate the feature representation. Graph convolutional neural networks (GCNs) can effectively utilize topology and node features in brain network analysis. Therefore, we design a deep learning framework called BPF-GCN and conduct experiments on publicly available ASD dataset. The results show that the classification accuracy of BPF-GCN reached 82.35%, surpassing existing methods. This demonstrates the superiority of its classification performance, and the framework can extract ROIs related to ASD. Our work provides a valuable reference for the timely diagnosis and treatment of ASD.

Spatial-Temporal Co-Attention Learning for Diagnosis of Mental Disorders From Resting-State fMRI Data

Neuroimaging techniques have been widely adopted to detect the neurological brain structures and functions of the nervous system. As an effective noninvasive neuroimaging technique, functional magnetic resonance imaging (fMRI) has been extensively used in computer-aided diagnosis (CAD) of mental disorders, e.g., autism spectrum disorder (ASD) and attention deficit/hyperactivity disorder (ADHD). In this study, we propose a spatial-temporal co-attention learning (STCAL) model for diagnosing ASD and ADHD from fMRI data. In particular, a guided co-attention (GCA) module is developed to model the intermodal interactions of spatial and temporal signal patterns. A novel sliding cluster attention module is designed to address global feature dependency of self-attention mechanism in fMRI time series. Comprehensive experimental results demonstrate that our STCAL model can achieve competitive accuracies of 73.0 ± 4.5%, 72.0 ± 3.8%, and 72.5 ± 4.2% on the ABIDE I, ABIDE II, and ADHD-200 datasets, respectively. Moreover, the potential for feature pruning based on the co-attention scores is validated by the simulation experiment. The clinical interpretation analysis of STCAL can allow medical professionals to concentrate on the discriminative regions of interest and key time frames from fMRI data.

ASD-SWNet: a novel shared-weight feature extraction and classification network for autism spectrum disorder diagnosis

The traditional diagnostic process for autism spectrum disorder (ASD) is subjective, where early and accurate diagnosis significantly affects treatment outcomes and life quality. Thus, improving ASD diagnostic methods is critical. This paper proposes ASD-SWNet, a new shared-weight feature extraction and classification network. It resolves the issue found in previous studies of inefficiently integrating unsupervised and supervised learning, thereby enhancing diagnostic precision. The approach utilizes functional magnetic resonance imaging to improve diagnostic accuracy, featuring an autoencoder (AE) with Gaussian noise for robust feature extraction and a tailored convolutional neural network (CNN) for classification. The shared-weight mechanism utilizes features learned by the AE to initialize the convolutional layer weights of the CNN, thereby integrating AE and CNN for joint training. A novel data augmentation strategy for time-series medical data is also introduced, tackling the problem of small sample sizes. Tested on the ABIDE-I dataset through nested ten-fold cross-validation, the method achieved an accuracy of 76.52% and an AUC of 0.81. This approach surpasses existing methods, showing significant enhancements in diagnostic accuracy and robustness. The contribution of this paper lies not only in proposing new methods for ASD diagnosis but also in offering new approaches for other neurological brain diseases.

Anatomy-Guided Spatio-Temporal Graph Convolutional Networks (AG-STGCNs) for Modeling Functional Connectivity Between Gyri and Sulci Across Multiple Task Domains

The cerebral cortex is folded as gyri and sulci, which provide the foundation to unveil anatomo-functional relationship of brain. Previous studies have extensively demonstrated that gyri and sulci exhibit intrinsic functional difference, which is further supported by morphological, genetic, and structural evidences. Therefore, systematically investigating the gyro-sulcal (G-S) functional difference can help deeply understand the functional mechanism of brain. By integrating functional magnetic resonance imaging (fMRI) with advanced deep learning models, recent studies have unveiled the temporal difference in functional activity between gyri and sulci. However, the potential difference of functional connectivity, which represents functional dependency between gyri and sulci, is much unknown. Moreover, the regularity and variability of the G-S functional connectivity difference across multiple task domains remains to be explored. To address the two concerns, this study developed new anatomy-guided spatio-temporal graph convolutional networks (AG-STGCNs) to investigate the regularity and variability of functional connectivity differences between gyri and sulci across multiple task domains. Based on 830 subjects with seven different task-based and one resting state fMRI (rs-fMRI) datasets from the public Human Connectome Project (HCP), we consistently found that there are significant differences of functional connectivity between gyral and sulcal regions within task domains compared with resting state (RS). Furthermore, there is considerable variability of such functional connectivity and information flow between gyri and sulci across different task domains, which are correlated with individual cognitive behaviors. Our study helps better understand the functional segregation of gyri and sulci within task domains as well as the anatomo-functional-behavioral relationship of the human brain.

Gradient Matching Federated Domain Adaptation for Brain Image Classification

Federated learning has shown its unique advantages in many different tasks, including brain image analysis. It provides a new way to train deep learning models while protecting the privacy of medical image data from multiple sites. However, previous studies suggest that domain shift across different sites may influence the performance of federated models. As a solution, we propose a gradient matching federated domain adaptation (GM-FedDA) method for brain image classification, aiming to reduce domain discrepancy with the assistance of a public image dataset and train robust local federated models for target sites. It mainly includes two stages: 1) pretraining stage; we propose a one-common-source adversarial domain adaptation (OCS-ADA) strategy, i.e., adopting ADA with gradient matching loss to pretrain encoders for reducing domain shift at each target site (private data) with the assistance of a common source domain (public data) and 2) fine-tuning stage; we develop a gradient matching federated (GM-Fed) fine-tuning method for updating local federated models pretrained with the OCS-ADA strategy, i.e., pushing the optimization direction of a local federated model toward its specific local minimum by minimizing gradient matching loss between sites. Using fully connected networks as local models, we validate our method with the diagnostic classification tasks of schizophrenia and major depressive disorder based on multisite resting-state functional MRI (fMRI), respectively. Results show that the proposed GM-FedDA method outperforms other commonly used methods, suggesting the potential of our method in brain imaging analysis and other fields, which need to utilize multisite data while preserving data privacy.

Brain Structural Connectivity Guided Vision Transformers for Identification of Functional Connectivity Characteristics in Preterm Neonates

Preterm birth is the leading cause of death in children under five years old, and is associated with a wide sequence of complications in both short and long term. In view of rapid neurodevelopment during the neonatal period, preterm neonates may exhibit considerable functional alterations compared to term ones. However, the identified functional alterations in previous studies merely achieve moderate classification performance, while more accurate functional characteristics with satisfying discrimination ability for better diagnosis and therapeutic treatment is underexplored. To address this problem, we propose a novel brain structural connectivity (SC) guided Vision Transformer (SCG-ViT) to identify functional connectivity (FC) differences among three neonatal groups: preterm, preterm with early postnatal experience, and term. Particularly, inspired by the neuroscience-derived information, a novel patch token of SC/FC matrix is defined, and the SC matrix is then adopted as an effective mask into the ViT model to screen out input FC patch embeddings with weaker SC, and to focus on stronger ones for better classification and identification of FC differences among the three groups. The experimental results on multi-modal MRI data of 437 neonatal brains from publicly released Developing Human Connectome Project (dHCP) demonstrate that SCG-ViT achieves superior classification ability compared to baseline models, and successfully identifies holistically different FC patterns among the three groups. Moreover, these different FCs are significantly correlated with the differential gene expressions of the three groups. In summary, SCG-ViT provides a powerfully brain-guided pipeline of adopting large-scale and data-intensive deep learning models for medical imaging-based diagnosis.

B2-ViT Net: Broad Vision Transformer Network With Broad Attention for Seizure Prediction

Seizure prediction are necessary for epileptic patients. The global spatial interactions among channels, and long-range temporal dependencies play a crucial role in seizure onset prediction. In addition, it is necessary to search for seizure prediction features in a vast space to learn new generalized feature representations. Many previous deep learning algorithms have achieved some results in automatic seizure prediction. However, most of them do not consider global spatial interactions among channels and long-range temporal dependencies together, and only learn the feature representation in the deep space. To tackle these issues, in this study, an novel bi-level programming seizure prediction model, B2-ViT Net, is proposed for learning the new generalized spatio-temporal long-range correlation features, which can characterize the global interactions among channels in spatial, and long-range dependencies in temporal required for seizure prediction. In addition, the proposed model can comprehensively learn generalized seizure prediction features in a vast space due to its strong deep and broad feature search capabilities. Sufficient experiments are conducted on two public datasets, CHB-MIT and Kaggle datasets. Compared with other existing methods, our proposed model has shown promising results in automatic seizure prediction tasks, and provides a certain degree of interpretability.

A Causality-Aware Graph Convolutional Network Framework for Rigidity Assessment in Parkinsonians

Rigidity is one of the common motor disorders in Parkinson's disease (PD), which lead to life quality deterioration. The widely-used rating-scale-based approach for rigidity assessment still depends on the availability of experienced neurologists and is limited by rating subjectivity. Given the recent successful applications of quantitative susceptibility mapping (QSM) in auxiliary PD diagnosis, automated assessment of PD rigidity can be essentially achieved through QSM analysis. However, a major challenge is the performance instability due to the confounding factors (e.g., noise and distribution shift) which conceal the truly-causal features. Therefore, we propose a causality-aware graph convolutional network (GCN) framework, where causal feature selection is combined with causal invariance to ensure that causality-informed model decisions are reached. Firstly, a GCN model that integrates causal feature selection is systematically constructed at three graph levels: node, structure, and representation. In this model, a causal diagram is learned to extract a subgraph with truly-causal information. Secondly, a non-causal perturbation strategy is developed along with an invariance constraint to ensure the stability of the assessment results under different distributions, and thus avoid spurious correlations caused by distribution shifts. The superiority of the proposed method is shown by extensive experiments and the clinical value is revealed by the direct relevance of selected brain regions to rigidity in PD. Besides, its extensibility is verified on other two tasks: PD bradykinesia and mental state for Alzheimer's disease. Overall, we provide a clinically-potential tool for automated and stable assessment of PD rigidity. Our source code will be available at https://github.com/SJTUBME-QianLab/Causality-Aware-Rigidity.

Delineating a Pathway for the Discovery of Functional Connectome Biomarkers of Autism

The promise of individually tailored care for autism has driven efforts to establish biomarkers. This chapter appraises the state of precision-medicine research focused on biomarkers based on the functional brain connectome. This work is grounded on abundant evidence supporting the brain dysconnection model of autism and the advantages of resting-state functional MRI (R-fMRI) for studying the brain in vivo. After considering biomarker requirements of consistency and clinical relevance, we provide a scoping review of R-fMRI studies of individual prediction in autism. In the past 10 years, responding to the availability of open data through the Autism Brain Imaging Data Exchange, machine learning studies have surged. Nearly all have focused on diagnostic label classification. These efforts have shown that autism prediction is feasible using functional connectome markers, with accuracy reported well above chance. In parallel, emerging approaches more directly addressing autism heterogeneity are paving the way for much-needed biomarkers of longitudinal outcome and treatment response. We conclude with key challenges to be addressed by the next generation of studies.

MS²-GNN: Exploring GNN-Based Multimodal Fusion Network for Depression Detection

Major depressive disorder (MDD) is one of the most common and severe mental illnesses, posing a huge burden on society and families. Recently, some multimodal methods have been proposed to learn a multimodal embedding for MDD detection and achieved promising performance. However, these methods ignore the heterogeneity/homogeneity among various modalities. Besides, earlier attempts ignore interclass separability and intraclass compactness. Inspired by the above observations, we propose a graph neural network (GNN)-based multimodal fusion strategy named modal-shared modal-specific GNN, which investigates the heterogeneity/homogeneity among various psychophysiological modalities as well as explores the potential relationship between subjects. Specifically, we develop a modal-shared and modal-specific GNN architecture to extract the inter/intramodal characteristics. Furthermore, a reconstruction network is employed to ensure fidelity within the individual modality. Moreover, we impose an attention mechanism on various embeddings to obtain a multimodal compact representation for the subsequent MDD detection task. We conduct extensive experiments on two public depression datasets and the favorable results demonstrate the effectiveness of the proposed algorithm.

The Combination of a Graph Neural Network Technique and Brain Imaging to Diagnose Neurological Disorders: A Review and Outlook

Neurological disorders (NDs), such as Alzheimer's disease, have been a threat to human health all over the world. It is of great importance to diagnose ND through combining artificial intelligence technology and brain imaging. A graph neural network (GNN) can model and analyze the brain, imaging from morphology, anatomical structure, function features, and other aspects, thus becoming one of the best deep learning models in the diagnosis of ND. Some researchers have investigated the application of GNN in the medical field, but the scope is broad, and its application to NDs is less frequent and not detailed enough. This review focuses on the research progress of GNNs in the diagnosis of ND. Firstly, we systematically investigated the GNN framework of ND, including graph construction, graph convolution, graph pooling, and graph prediction. Secondly, we investigated common NDs using the GNN diagnostic model in terms of data modality, number of subjects, and diagnostic accuracy. Thirdly, we discussed some research challenges and future research directions. The results of this review may be a valuable contribution to the ongoing intersection of artificial intelligence technology and brain imaging.

Multi-Scale Dynamic Graph Learning for Brain Disorder Detection With Functional MRI

Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in the detection of brain disorders such as autism spectrum disorder based on various machine/deep learning techniques. Learning-based methods typically rely on functional connectivity networks (FCNs) derived from blood-oxygen-level-dependent time series of rs-fMRI data to capture interactions between brain regions-of-interest (ROIs). Graph neural networks have been recently used to extract fMRI features from graph-structured FCNs, but cannot effectively characterize spatiotemporal dynamics of FCNs, e.g., the functional connectivity of brain ROIs is dynamically changing in a short period of time. Also, many studies usually focus on single-scale topology of FCN, thereby ignoring the potential complementary topological information of FCN at different spatial resolutions. To this end, in this paper, we propose a multi-scale dynamic graph learning (MDGL) framework to capture multi-scale spatiotemporal dynamic representations of rs-fMRI data for automated brain disorder diagnosis. The MDGL framework consists of three major components: 1) multi-scale dynamic FCN construction using multiple brain atlases to model multi-scale topological information, 2) multi-scale dynamic graph representation learning to capture spatiotemporal information conveyed in fMRI data, and 3) multi-scale feature fusion and classification. Experimental results on two datasets show that MDGL outperforms several state-of-the-art methods.

Detection of autism spectrum disorder using graph representation learning algorithms and deep neural network, based on fMRI signals

Introduction

Can we apply graph representation learning algorithms to identify autism spectrum disorder (ASD) patients within a large brain imaging dataset? ASD is mainly identified by brain functional connectivity patterns. Attempts to unveil the common neural patterns emerged in ASD are the essence of ASD classification. We claim that graph representation learning methods can appropriately extract the connectivity patterns of the brain, in such a way that the method can be generalized to every recording condition, and phenotypical information of subjects. These methods can capture the whole structure of the brain, both local and global properties.

Methods

The investigation is done for the worldwide brain imaging multi-site database known as ABIDE I and II (Autism Brain Imaging Data Exchange). Among different graph representation techniques, we used AWE, Node2vec, Struct2vec, multi node2vec, and Graph2Img. The best approach was Graph2Img, in which after extracting the feature vectors representative of the brain nodes, the PCA algorithm is applied to the matrix of feature vectors. The classifier adapted to the features embedded in graphs is an LeNet deep neural network.

Results and discussion

Although we could not outperform the previous accuracy of 10-fold cross-validation in the identification of ASD versus control patients in this dataset, for leave-one-site-out cross-validation, we could obtain better results (our accuracy: 80%). The result is that graph embedding methods can prepare the connectivity matrix more suitable for applying to a deep network.

Differentiate preterm and term infant brains and characterize the corresponding biomarkers via DICCCOL-based multi-modality graph neural networks

Preterm birth is a worldwide problem that affects infants throughout their lives significantly. Therefore, differentiating brain disorders, and further identifying and characterizing the corresponding biomarkers are key issues to investigate the effects of preterm birth, which facilitates the interventions for neuroprotection and improves outcomes of prematurity. Until now, many efforts have been made to study the effects of preterm birth; however, most of the studies merely focus on either functional or structural perspective. In addition, an effective framework not only jointly studies the brain function and structure at a group-level, but also retains the individual differences among the subjects. In this study, a novel dense individualized and common connectivity-based cortical landmarks (DICCCOL)-based multi-modality graph neural networks (DM-GNN) framework is proposed to differentiate preterm and term infant brains and characterize the corresponding biomarkers. This framework adopts the DICCCOL system as the initialized graph node of GNN for each subject, utilizing both functional and structural profiles and effectively retaining the individual differences. To be specific, functional magnetic resonance imaging (fMRI) of the brain provides the features for the graph nodes, and brain fiber connectivity is utilized as the structural representation of the graph edges. Self-attention graph pooling (SAGPOOL)-based GNN is then applied to jointly study the function and structure of the brain and identify the biomarkers. Our results successfully demonstrate that the proposed framework can effectively differentiate the preterm and term infant brains. Furthermore, the self-attention-based mechanism can accurately calculate the attention score and recognize the most significant biomarkers. In this study, not only 87.6% classification accuracy is observed for the developing Human Connectome Project (dHCP) dataset, but also distinguishing features are explored and extracted. Our study provides a novel and uniform framework to differentiate brain disorders and characterize the corresponding biomarkers.

FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net

An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.

Predicting escitalopram treatment response from pre-treatment and early response resting state fMRI in a multi-site sample: A CAN-BIND-1 report

•

Baseline measures alone not able to predict escitalopram response above default.

•

This poor baseline performance contradicts results from smaller studies.

•

Accuracy improved using change in functional connectivity from baseline to week 2.

•

Measures of early change following treatment may be crucial for accurate prediction.

Many previous intervention studies have used functional magnetic resonance imaging (fMRI) data to predict the antidepressant response of patients with major depressive disorder (MDD); however, practical constraints have limited many of those attempts to small, single centre studies which may not adequately reflect how these models will generalize when used in clinical practice. Not only does the act of collecting data at multiple sites generally increase sample sizes (a critical point in machine learning development) it also generates a more heterogeneous dataset due to systematic differences in scanners at different sites, and geographical differences in patient populations. As part of the Canadian Biomarker Integration Network in Depression (CAN-BIND-1) study, 144 MDD patients from six sites underwent resting state fMRI prior to starting escitalopram treatment, and again two weeks after the start. Here, we consider ways to use machine learning techniques to produce models that can predict response (measured at eight weeks after initiation), based on various parcellations, functional connectivity (FC) metrics, dimensionality reduction algorithms, and base learners, and also whether to use scans from one or both time points. Models that use only baseline (pre-treatment) or only week 2 (early-response) whole-brain FC features consistently failed to perform significantly better than default models. Utilizing the change in FC between these two time points, however, yielded significant results, with the best performing analytical pipeline achieving 69.6% (SD 10.8) accuracy. These results appear contrary to findings from many smaller single-site studies, which report substantially higher predictive accuracies from models trained on only baseline resting state FC features, suggesting these models may not generalize well beyond data used for development. Further, these results indicate the potential value of collecting data both before and shortly after treatment initiation.