Virtual Adversarial Training-Based Deep Feature Aggregation Network From Dynamic Effective Connectivity for MCI Identification

Dysregulated brain dynamics in the visualmotor network in type 2 diabetes patients and their relationship with cognitive impairment

OBJECTIVE

Type 2 diabetes mellitus (T2DM) is a significant risk factor for mild cognitive impairment (MCI). Here, we identified a T2DM-specific effective connectivity (EC) network, the dynamic features of which could be used to distinguish T2DM patients with MCI from healthy controls (HC) and correlation with cognitive performance.

METHODS

Local and multicentered T2DM patients and matched HC who underwent functional magnetic resonance imaging were recruited. Their static and dynamic effective connectivity were compared. The relationships between connectome characteristics and cognitive performance were also evaluated.

RESULTS

The nodes of the T2DM-related static causality network included the anterior central gyrus, tail of the parahippocampal gyrus, posterior superior temporal sulcus, posterior central parietal lobe, posterior central gyrus and V5 region of the occipital lobe. The V5 region of the visual cortex was the core node. In the multicentered dataset, compared with the HC group, the T2DM with MCI group had significantly greater fractional window and mean dwell time. Fractional windows of the state, which was dominated by the interaction of the nodes from SomMot_Network, Limbic_Network, Default_Network, in the T2DM-specific network increased with poorer cognitive performance in T2DM with MCI patients.

CONCLUSION

Our findings provide insights into the neurobiological mechanisms of the cognitive impairment of T2DM patients from a dynamic network perspective, which may ultimately inform more targeted and effective strategies to prevent MCI.

DEAF-Net: Detail-Enhanced Attention Feature Fusion Network for Retinal Vessel Segmentation

Retinal vessel segmentation is crucial for the diagnosis of ophthalmic and cardiovascular diseases. However, retinal vessels are densely and irregularly distributed, with many capillaries blending into the background, and exhibit low contrast. Moreover, the encoder-decoder-based network for retinal vessel segmentation suffers from irreversible loss of detailed features due to multiple encoding and decoding, leading to incorrect segmentation of the vessels. Meanwhile, the single-dimensional attention mechanisms possess limitations, neglecting the importance of multidimensional features. To solve these issues, in this paper, we propose a detail-enhanced attention feature fusion network (DEAF-Net) for retinal vessel segmentation. First, the detail-enhanced residual block (DERB) module is proposed to strengthen the capacity for detailed representation, ensuring that intricate features are efficiently maintained during the segmentation of delicate vessels. Second, the multidimensional collaborative attention encoder (MCAE) module is proposed to optimize the extraction of multidimensional information. Then, the dynamic decoder (DYD) module is introduced to preserve spatial information during the decoding process and reduce the information loss caused by upsampling operations. Finally, the proposed detail-enhanced feature fusion (DEFF) module composed of DERB, MCAE and DYD modules fuses feature maps from both encoding and decoding and achieves effective aggregation of multi-scale contextual information. The experiments conducted on the datasets of DRIVE, CHASEDB1, and STARE, achieving Sen of 0.8305, 0.8784, and 0.8654, and AUC of 0.9886, 0.9913, and 0.9911 on DRIVE, CHASEDB1, and STARE, respectively, demonstrate the performance of our proposed network, particularly in the segmentation of fine retinal vessels.

Discovery of novel VEGFR2 inhibitors against non-small cell lung cancer based on fingerprint-enhanced graph attention convolutional network

Despite the proven inhibitory effects of drugs targeting vascular endothelial growth factor receptor 2 (VEGFR2) on solid tumors, including non-small cell lung cancer (NSCLC), the development of anti-NSCLC drugs solely targeting VEGFR2 still faces risks such as off-target effects and limited efficacy. This study aims to develop a novel fingerprint-enhanced graph attention convolutional network (FnGATGCN) model for predicting the activity of anti-NSCLC drugs. Employing a multimodal fusion strategy, the model integrates a feature extraction layer that comprises molecular graph feature extraction and molecular fingerprint feature extraction. The performance evaluation results indicate that the model exhibits high accuracy and stability in predicting activity. Moreover, we explored the relationship between molecular features and biological activity through visualization analysis, thus improving the interpretability of the approach. Utilizing this model, we screened the ZINC database and conducted high-precision molecular docking, leading to the identification of 11 potential active molecules. Subsequently, molecular dynamics simulations and free energy calculations were performed. The results demonstrate that all 11 aforementioned molecules can stably bind to VEGFR2 under dynamic conditions. Among the short-listed compounds, the top six exhibited satisfactory inhibitory activity against VEGFR2 and A549 cells. Especially, compound Z-3 displayed VEGFR2 inhibitory with IC50 values of 0.88 μM, and anti-proliferative activity against A549 cells with IC50 values of 4.23 ± 0.45 μM. This approach combines the advantages of target-based and phenotype-based screening, facilitating the rapid and efficient identification of candidate compounds with dual activity against VEGFR2 and A549 cell lines. It provides new insights and methods for the development of anti-NSCLC drugs. Furthermore, further biological activity tests revealed that Z1-Z3 and Z6 manifested relatively strong antiproliferative activities against NCI-H23 and NCI-H460, and relatively low toxicity towards GES-1. The hit compounds were promising candidates for the further development of novel VEGFR2 inhibitors against NSCLC.

Constructing High-Order Functional Connectivity Networks With Temporal Information From fMRI Data

Conducting functional connectivity analysis on functional magnetic resonance imaging (fMRI) data presents a significant and intricate challenge. Contemporary studies typically analyze fMRI data by constructing high-order functional connectivity networks (FCNs) due to their strong interpretability. However, these approaches often overlook temporal information, resulting in suboptimal accuracy. Temporal information plays a vital role in reflecting changes in blood oxygenation level-dependent signals. To address this shortcoming, we have devised a framework for extracting temporal dependencies from fMRI data and inferring high-order functional connectivity among regions of interest (ROIs). Our approach postulates that the current state can be determined by the FCN and the state at the previous time, effectively capturing temporal dependencies. Furthermore, we enhance FCN by incorporating high-order features through hypergraph-based manifold regularization. Our algorithm involves causal modeling of the dynamic brain system, and the obtained directed FC reveals differences in the flow of information under different patterns. We have validated the significance of integrating temporal information into FCN using four real-world fMRI datasets. On average, our framework achieves 12% higher accuracy than non-temporal hypergraph-based and low-order FCNs, all while maintaining a short processing time. Notably, our framework successfully identifies the most discriminative ROIs, aligning with previous research, and thereby facilitating cognitive and behavioral studies.

Temporal Dynamic Synchronous Functional Brain Network for Schizophrenia Classification and Lateralization Analysis

Available evidence suggests that dynamic functional connectivity can capture time-varying abnormalities in brain activity in resting-state cerebral functional magnetic resonance imaging (rs-fMRI) data and has a natural advantage in uncovering mechanisms of abnormal brain activity in schizophrenia (SZ) patients. Hence, an advanced dynamic brain network analysis model called the temporal brain category graph convolutional network (Temporal-BCGCN) was employed. Firstly, a unique dynamic brain network analysis module, DSF-BrainNet, was designed to construct dynamic synchronization features. Subsequently, a revolutionary graph convolution method, TemporalConv, was proposed based on the synchronous temporal properties of features. Finally, the first modular test tool for abnormal hemispherical lateralization in deep learning based on rs-fMRI data, named CategoryPool, was proposed. This study was validated on COBRE and UCLA datasets and achieved 83.62% and 89.71% average accuracies, respectively, outperforming the baseline model and other state-of-the-art methods. The ablation results also demonstrate the advantages of TemporalConv over the traditional edge feature graph convolution approach and the improvement of CategoryPool over the classical graph pooling approach. Interestingly, this study showed that the lower-order perceptual system and higher-order network regions in the left hemisphere are more severely dysfunctional than in the right hemisphere in SZ, reaffirmings the importance of the left medial superior frontal gyrus in SZ. Our code was available at: https://github.com/swfen/Temporal-BCGCN.

Enhancing Autism Spectrum Disorder identification in multi-site MRI imaging: A multi-head cross-attention and multi-context approach for addressing variability in un-harmonized data

Multi-site MRI imaging poses a significant challenge due to the potential variations in images across different scanners at different sites. This variability can introduce ambiguity in further image analysis. Consequently, the image analysis techniques become site-dependent and scanner-dependent, implying that adjustments in the analysis methodologies are necessary for each scanner configuration. Further, implementing real-time modifications becomes intricate, particularly when incorporating a new type of scanner, as it requires adapting the analysis methods accordingly. Taking into account the aforementioned challenge, we have considered its implications for an Autism spectrum disorder (ASD) application. Our objective is to minimize the impact of site and scanner variability in the analysis, aiming to develop a model that remains effective across different scanners and sites. This entails devising a methodology that allows the same model to function seamlessly across multiple scanner configurations and sites. ASD, a behavioral disorder affecting child development, requires early detection. Clinical observation is time-consuming, prompting the use of fMRI with machine/deep learning for expedited diagnosis. Previous methods leverage fMRI's functional connectivity but often rely on less generalized feature extractors and classifiers. Hence, there is significant room for improvement in the generalizability of detection methods across multi-site data, which is acquired from multiple scanners with different settings. In this study, we propose a Cross-Combination Multi-Scale Multi-Context Framework (CCMSMCF) capable of performing neuroimaging-based diagnostic classification of mental disorders for a multi-site dataset. Thus, this framework attains a degree of internal data harmonization, rendering it to some extent site and scanner-agnostic. Our proposed network, CCMSMCF, is constructed by integrating two sub-modules: the Multi-Head Attention Cross-Scale Module (MHACSM) and the Residual Multi-Context Module (RMCN). We also employ multiple loss functions in a novel manner for training the model, which includes Binary Cross Entropy, Dice loss, and Embedding Coupling loss. The model is validated on the Autism Brain Imaging Data Exchange I (ABIDE-I) dataset, which includes data from multiple scanners across different sites, and achieves promising results.

Dynamic functional connections analysis with spectral learning for brain disorder detection

Dynamic functional connections (dFCs), can reveal neural activities, which provides an insightful way of mining the temporal patterns within the human brain and further detecting brain disorders. However, most existing studies focus on the dFCs estimation to identify brain disorders by shallow temporal features and methods, which cannot capture the inherent temporal patterns of dFCs effectively. To address this problem, this study proposes a novel method, named dynamic functional connections analysis with spectral learning (dCSL), to explore inherently temporal patterns of dFCs and further detect the brain disorders. Concretely, dCSL includes two components, dFCs estimation module and dFCs analysis module. In the former, dFCs are estimated via the sliding window technique. In the latter, the spectral kernel mapping is first constructed by combining the Fourier transform with the non-stationary kernel. Subsequently, the spectral kernel mapping is stacked into a deep kernel network to explore higher-order temporal patterns of dFCs through spectral learning. The proposed dCSL, sharing the benefits of deep architecture and non-stationary kernel, can not only calculate the long-range relationship but also explore the higher-order temporal patterns of dFCs. To evaluate the proposed method, a set of brain disorder classification tasks are conducted on several public datasets. As a result, the proposed dCSL achieves 5% accuracy improvement compared with the widely used approaches for analyzing sequence data, 1.3% accuracy improvement compared with the state-of-the-art methods for dFCs. In addition, the discriminative brain regions are explored in the ASD detection task. The findings in this study are consistent with the clinical performance in ASD.

Early diagnosis of Alzheimer's disease using a group self-calibrated coordinate attention network based on multimodal MRI

Convolutional neural networks (CNNs) for extracting structural information from structural magnetic resonance imaging (sMRI), combined with functional magnetic resonance imaging (fMRI) and neuropsychological features, has emerged as a pivotal tool for early diagnosis of Alzheimer's disease (AD). However, the fixed-size convolutional kernels in CNNs have limitations in capturing global features, reducing the effectiveness of AD diagnosis. We introduced a group self-calibrated coordinate attention network (GSCANet) designed for the precise diagnosis of AD using multimodal data, including encompassing Haralick texture features, functional connectivity, and neuropsychological scores. GSCANet utilizes a parallel group self-calibrated module to enhance original spatial features, expanding the field of view and embedding spatial data into channel information through a coordinate attention module, which ensures long-term contextual interaction. In a four-classification comparison (AD vs. early MCI (EMCI) vs. late MCI (LMCI) vs. normal control (NC)), GSCANet demonstrated an accuracy of 78.70%. For the three-classification comparison (AD vs. MCI vs. NC), it achieved an accuracy of 83.33%. Moreover, our method exhibited impressive accuracies in the AD vs. NC (92.81%) and EMCI vs. LMCI (84.67%) classifications. GSCANet improves classification performance at different stages of AD by employing group self-calibrated to expand features receptive field and integrating coordinated attention to facilitate significant interactions among channels and spaces. Providing insights into AD mechanisms and showcasing scalability for various disease predictions.

Sparse Graph Representation Learning Based on Reinforcement Learning for Personalized Mild Cognitive Impairment (MCI) Diagnosis

Resting-state functional magnetic resonance imaging (rs-fMRI) has gained attention as a reliable technique for investigating the intrinsic function patterns of the brain. It facilitates the extraction of functional connectivity networks (FCNs) that capture synchronized activity patterns among regions of interest (ROIs). Analyzing FCNs enables the identification of distinctive connectivity patterns associated with mild cognitive impairment (MCI). For MCI diagnosis, various sparse representation techniques have been introduced, including statistical- and deep learning-based methods. However, these methods face limitations due to their reliance on supervised learning schemes, which restrict the exploration necessary for probing novel solutions. To overcome such limitation, prior work has incorporated reinforcement learning (RL) to dynamically select ROIs, but effective exploration remains challenging due to the vast search space during training. To tackle this issue, in this study, we propose an advanced RL-based framework that utilizes a divide-and-conquer approach to decompose the FCN construction task into smaller sub-problems in a subject-specific manner, enabling efficient exploration under each sub-problem condition. Additionally, we leverage the learned value function to determine the sparsity level of FCNs, considering individual characteristics of FCNs. We validate the effectiveness of our proposed framework by demonstrating its superior performance in MCI diagnosis on publicly available cohort datasets.

A Multiview Brain Network Transformer Fusing Individualized Information for Autism Spectrum Disorder Diagnosis

Functional connectivity (FC) networks, built from analyses of resting-state magnetic resonance imaging (rs-fMRI), serve as efficacious biomarkers for identifying Autism Spectrum Disorders (ASD) patients. Given the neurobiological heterogeneity across individuals and the unique presentation of ASD symptoms, the fusion of individualized information into diagnosis becomes essential. However, this aspect is overlooked in most methods. Furthermore, the existing methods typically focus on studying direct pairwise connections between brain ROIs, while disregarding interactions between indirectly connected neighbors. To overcome above challenges, we build common FC and individualized FC by tangent pearson embedding (TP) and common orthogonal basis extraction (COBE) respectively, and present a novel multiview brain transformer (MBT) aimed at effectively fusing common and indivinformation of subjects. MBT is mainly constructed by transformer layers with diffusion kernel (DK), fusion quality-inspired weighting module (FQW), similarity loss and orthonormal clustering fusion readout module (OCFRead). DK transformer can incorporate higher-order random walk methods to capture wider interactions among indirectly connected brain regions. FQW promotes adaptive fusion of features between views, and similarity loss and OCFRead are placed on the last layer to accomplish the ultimate integration of information. In our method, TP, DK and FQW modules all help to model wider connectivity in the brain that make up for the shortcomings of traditional methods. We conducted experiments on the public ABIDE dataset based on AAL and CC200 respectively. Our framework has shown promising results, outperforming state-of-the-art methods on both templates. This suggests its potential as a valuable approach for clinical ASD diagnosis.

MCAN: Multimodal Causal Adversarial Networks for Dynamic Effective Connectivity Learning From fMRI and EEG Data

Dynamic effective connectivity (DEC) is the accumulation of effective connectivity in the time dimension, which can describe the continuous neural activities in the brain. Recently, learning DEC from functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) data has attracted the attention of neuroinformatics researchers. However, the current methods fail to consider the gap between the fMRI and EEG modality, which can not precisely learn the DEC network from multimodal data. In this paper, we propose a multimodal causal adversarial network for DEC learning, named MCAN. The MCAN contains two modules: multimodal causal generator and multimodal causal discriminator. First, MCAN employs a multimodal causal generator with an attention-guided layer to produce a posterior signal and output a set of DEC networks. Then, the proposed method uses a multimodal causal discriminator to unsupervised calculate the joint gradient, which directs the update of the whole network. The experimental results on simulated data sets show that MCAN is superior to other state-of-the-art methods in learning the network structure of DEC and can effectively estimate the brain states. The experimental results on real data sets show that MCAN can better reveal abnormal patterns of brain activity and has good application potential in brain network analysis.

STDCformer: Spatial-temporal dual-path cross-attention model for fMRI-based autism spectrum disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) is a non-invasive neuroimaging technique widely utilized in the research of Autism Spectrum Disorder (ASD), providing preliminary insights into the potential biological mechanisms underlying ASD. Deep learning techniques have demonstrated significant potential in the analysis of rs-fMRI. However, accurately distinguishing between healthy control group and ASD has been a longstanding challenge. In this regard, this work proposes a model featuring a dual-path cross-attention framework for spatial and temporal patterns, named STDCformer, aiming to enhance the accuracy of ASD identification. STDCformer can preserve both temporal-specific patterns and spatial-specific patterns while explicitly interacting spatiotemporal information in depth. The embedding layer of the STDCformer embeds temporal and spatial patterns in dual paths. For the temporal path, we introduce a perturbation positional encoding to improve the issue of signal misalignment caused by individual differences. For the spatial path, we propose a correlation metric based on Gramian angular field similarity to establish a more specific whole-brain functional network. Subsequently, we interleave the query and key vectors of dual paths to interact spatial and temporal information. We further propose integrating the dual-path attention into a tensor that retains spatiotemporal dimensions and utilizing 2D convolution for feed-forward processing. Our attention layer allows the model to represent spatiotemporal correlations of signals at multiple scales to alleviate issues of information distortion and loss. Our STDCformer demonstrates competitive results compared to state-of-the-art methods on the ABIDE dataset. Additionally, we conducted interpretative analyses of the model to preliminarily discuss the potential physiological mechanisms of ASD. This work once again demonstrates the potential of deep learning technology in identifying ASD and developing neuroimaging biomarkers for ASD.

Anatomy-Guided Spatio-Temporal Graph Convolutional Networks (AG-STGCNs) for Modeling Functional Connectivity Between Gyri and Sulci Across Multiple Task Domains

The cerebral cortex is folded as gyri and sulci, which provide the foundation to unveil anatomo-functional relationship of brain. Previous studies have extensively demonstrated that gyri and sulci exhibit intrinsic functional difference, which is further supported by morphological, genetic, and structural evidences. Therefore, systematically investigating the gyro-sulcal (G-S) functional difference can help deeply understand the functional mechanism of brain. By integrating functional magnetic resonance imaging (fMRI) with advanced deep learning models, recent studies have unveiled the temporal difference in functional activity between gyri and sulci. However, the potential difference of functional connectivity, which represents functional dependency between gyri and sulci, is much unknown. Moreover, the regularity and variability of the G-S functional connectivity difference across multiple task domains remains to be explored. To address the two concerns, this study developed new anatomy-guided spatio-temporal graph convolutional networks (AG-STGCNs) to investigate the regularity and variability of functional connectivity differences between gyri and sulci across multiple task domains. Based on 830 subjects with seven different task-based and one resting state fMRI (rs-fMRI) datasets from the public Human Connectome Project (HCP), we consistently found that there are significant differences of functional connectivity between gyral and sulcal regions within task domains compared with resting state (RS). Furthermore, there is considerable variability of such functional connectivity and information flow between gyri and sulci across different task domains, which are correlated with individual cognitive behaviors. Our study helps better understand the functional segregation of gyri and sulci within task domains as well as the anatomo-functional-behavioral relationship of the human brain.

Verbal memory network mapping in individual patients predicts postoperative functional impairments

Verbal memory decline is a significant concern following temporal lobe surgeries in patients with epilepsy, emphasizing the need for precision presurgical verbal memory mapping to optimize functional outcomes. However, the inter-individual variability in functional networks and brain function-structural dissociations pose challenges when relying solely on group-level atlases or anatomical landmarks for surgical guidance. Here, we aimed to develop and validate a personalized functional mapping technique for verbal memory using precision resting-state functional MRI (rs-fMRI) and neurosurgery. A total of 38 patients with refractory epilepsy scheduled for surgical interventions were enrolled and 28 patients were analyzed in the study. Baseline 30-min rs-fMRI scanning, verbal memory and language assessments were collected for each patient before surgery. Personalized verbal memory networks (PVMN) were delineated based on preoperative rs-fMRI data for each patient. The accuracy of PVMN was assessed by comparing post-operative functional impairments and the overlapping extent between PVMN and surgical lesions. A total of 14 out of 28 patients experienced clinically meaningful declines in verbal memory after surgery. The personalized network and the group-level atlas exhibited 100% and 75.0% accuracy in predicting postoperative verbal memory declines, respectively. Moreover, six patients with extra-temporal lesions that overlapped with PVMN showed selective impairments in verbal memory. Furthermore, the lesioned ratio of the personalized network rather than the group-level atlas was significantly correlated with postoperative declines in verbal memory (personalized networks: r = -0.39, p = .038; group-level atlas: r = -0.19, p = .332). In conclusion, our personalized functional mapping technique, using precision rs-fMRI, offers valuable insights into individual variability in the verbal memory network and holds promise in precision verbal memory network mapping in individuals.

A Learnable Counter-Condition Analysis Framework for Functional Connectivity-Based Neurological Disorder Diagnosis

To understand the biological characteristics of neurological disorders with functional connectivity (FC), recent studies have widely utilized deep learning-based models to identify the disease and conducted post-hoc analyses via explainable models to discover disease-related biomarkers. Most existing frameworks consist of three stages, namely, feature selection, feature extraction for classification, and analysis, where each stage is implemented separately. However, if the results at each stage lack reliability, it can cause misdiagnosis and incorrect analysis in afterward stages. In this study, we propose a novel unified framework that systemically integrates diagnoses (i.e., feature selection and feature extraction) and explanations. Notably, we devised an adaptive attention network as a feature selection approach to identify individual-specific disease-related connections. We also propose a functional network relational encoder that summarizes the global topological properties of FC by learning the inter-network relations without pre-defined edges between functional networks. Last but not least, our framework provides a novel explanatory power for neuroscientific interpretation, also termed counter-condition analysis. We simulated the FC that reverses the diagnostic information (i.e., counter-condition FC): converting a normal brain to be abnormal and vice versa. We validated the effectiveness of our framework by using two large resting-state functional magnetic resonance imaging (fMRI) datasets, Autism Brain Imaging Data Exchange (ABIDE) and REST-meta-MDD, and demonstrated that our framework outperforms other competing methods for disease identification. Furthermore, we analyzed the disease-related neurological patterns based on counter-condition analysis.

A Multi-Graph Cross-Attention-Based Region-Aware Feature Fusion Network Using Multi-Template for Brain Disorder Diagnosis

Functional connectivity (FC) networks based on resting-state functional magnetic imaging (rs-fMRI) are reliable and sensitive for brain disorder diagnosis. However, most existing methods are limited by using a single template, which may be insufficient to reveal complex brain connectivities. Furthermore, these methods usually neglect the complementary information between static and dynamic brain networks, and the functional divergence among different brain regions, leading to suboptimal diagnosis performance. To address these limitations, we propose a novel multi-graph cross-attention based region-aware feature fusion network (MGCA-RAFFNet) by using multi-template for brain disorder diagnosis. Specifically, we first employ multi-template to parcellate the brain space into different regions of interest (ROIs). Then, a multi-graph cross-attention network (MGCAN), including static and dynamic graph convolutions, is developed to explore the deep features contained in multi-template data, which can effectively analyze complex interaction patterns of brain networks for each template, and further adopt a dual-view cross-attention (DVCA) to acquire complementary information. Finally, to efficiently fuse multiple static-dynamic features, we design a region-aware feature fusion network (RAFFNet), which is beneficial to improve the feature discrimination by considering the underlying relations among static-dynamic features in different brain regions. Our proposed method is evaluated on both public ADNI-2 and ABIDE-I datasets for diagnosing mild cognitive impairment (MCI) and autism spectrum disorder (ASD). Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/mylbuaa/MGCA-RAFFNet.

Effective hyper-connectivity network construction and learning: Application to major depressive disorder identification

Functional connectivity (FC) derived from resting-state fMRI (rs-fMRI) is a primary approach for identifying brain diseases, but it is limited to capturing the pairwise correlation between regions-of-interest (ROIs) in the brain. Thus, hyper-connectivity which describes the higher-order relationship among multiple ROIs is receiving increasing attention. However, most hyper-connectivity methods overlook the directionality of connections. The direction of information flow constitutes a pivotal factor in shaping brain activity and cognitive processes. Neglecting this directional aspect can lead to an incomplete understanding of high-order interactions within the brain. To this end, we propose a novel effective hyper-connectivity (EHC) network that integrates direction detection and hyper-connectivity modeling. It characterizes the high-order directional information flow among multiple ROIs, providing a more comprehensive understanding of brain activity. Then, we develop a directed hypergraph convolutional network (DHGCN) to acquire deep representations from EHC network and functional indicators of ROIs. In contrast to conventional hypergraph convolutional networks designed for undirected hypergraphs, DHGCN is specifically tailored to handle directed hypergraph data structures. Moreover, unlike existing methods that primarily focus on fMRI time series, our proposed DHGCN model also incorporates multiple functional indicators, providing a robust framework for feature learning. Finally, deep representations generated via DHGCN, combined with demographic factors, are used for major depressive disorder (MDD) identification. Experimental results demonstrate that the proposed framework outperforms both FC and undirected hyper-connectivity models, as well as surpassing other state-of-the-art methods. The identification of EHC abnormalities through our framework can enhance the analysis of brain function in individuals with MDD.

Deep Fusion of Multi-Template Using Spatio-Temporal Weighted Multi-Hypergraph Convolutional Networks for Brain Disease Analysis

Conventional functional connectivity network (FCN) based on resting-state fMRI (rs-fMRI) can only reflect the relationship between pairwise brain regions. Thus, the hyper-connectivity network (HCN) has been widely used to reveal high-order interactions among multiple brain regions. However, existing HCN models are essentially spatial HCN, which reflect the spatial relevance of multiple brain regions, but ignore the temporal correlation among multiple time points. Furthermore, the majority of HCN construction and learning frameworks are limited to using a single template, while the multi-template carries richer information. To address these issues, we first employ multiple templates to parcellate the rs-fMRI into different brain regions. Then, based on the multi-template data, we propose a spatio-temporal weighted HCN (STW-HCN) to capture more comprehensive high-order temporal and spatial properties of brain activity. Next, a novel deep fusion model of multi-template called spatio-temporal weighted multi-hypergraph convolutional network (STW-MHGCN) is proposed to fuse the STW-HCN of multiple templates, which extracts the deep interrelation information between different templates. Finally, we evaluate our method on the ADNI-2 and ABIDE-I datasets for mild cognitive impairment (MCI) and autism spectrum disorder (ASD) analysis. Experimental results demonstrate that the proposed method is superior to the state-of-the-art approaches in MCI and ASD classification, and the abnormal spatio-temporal hyper-edges discovered by our method have significant significance for the brain abnormalities analysis of MCI and ASD.

Adversarially Trained Persistent Homology Based Graph Convolutional Network for Disease Identification Using Brain Connectivity

Brain disease propagation is associated with characteristic alterations in the structural and functional connectivity networks of the brain. To identify disease-specific network representations, graph convolutional networks (GCNs) have been used because of their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks. However, existing GCNs generally focus on learning the discriminative region of interest (ROI) features, often ignoring important topological information that enables the integration of connectome patterns of brain activity. In addition, most methods fail to consider the vulnerability of GCNs to perturbations in network properties of the brain, which considerably degrades the reliability of diagnosis results. In this study, we propose an adversarially trained persistent homology-based graph convolutional network (ATPGCN) to capture disease-specific brain connectome patterns and classify brain diseases. First, the brain functional/structural connectivity is constructed using different neuroimaging modalities. Then, we develop a novel strategy that concatenates the persistent homology features from a brain algebraic topology analysis with readout features of the global pooling layer of a GCN model to collaboratively learn the individual-level representation. Finally, we simulate the adversarial perturbations by targeting the risk ROIs from clinical prior, and incorporate them into a training loop to evaluate the robustness of the model. The experimental results on three independent datasets demonstrate that ATPGCN outperforms existing classification methods in disease identification and is robust to minor perturbations in network architecture. Our code is available at https://github.com/CYB08/ATPGCN.

A comprehensive survey of complex brain network representation

Recent years have shown great merits in utilizing neuroimaging data to understand brain structural and functional changes, as well as its relationship to different neurodegenerative diseases and other clinical phenotypes. Brain networks, derived from different neuroimaging modalities, have attracted increasing attention due to their potential to gain system-level insights to characterize brain dynamics and abnormalities in neurological conditions. Traditional methods aim to pre-define multiple topological features of brain networks and relate these features to different clinical measures or demographical variables. With the enormous successes in deep learning techniques, graph learning methods have played significant roles in brain network analysis. In this survey, we first provide a brief overview of neuroimaging-derived brain networks. Then, we focus on presenting a comprehensive overview of both traditional methods and state-of-the-art deep-learning methods for brain network mining. Major models, and objectives of these methods are reviewed within this paper. Finally, we discuss several promising research directions in this field.

Autism spectrum disorder diagnosis based on deep unrolling-based spatial constraint representation

Accurate diagnosis of autism spectrum disorder (ASD) is crucial for effective treatment and prognosis. Functional brain networks (FBNs) constructed from functional magnetic resonance imaging (fMRI) have become a popular tool for ASD diagnosis. However, existing model-driven approaches used to construct FBNs lack the ability to capture potential non-linear relationships between data and labels. Moreover, most existing studies treat the FBNs construction and disease classification as separate steps, leading to large inter-subject variability in the estimated FBNs and reducing the statistical power of subsequent group comparison. To address these limitations, we propose a new approach to FBNs construction called the deep unrolling-based spatial constraint representation (DUSCR) model and integrate it with a convolutional classifier to create an end-to-end framework for ASD recognition. Specifically, the model spatial constraint representation (SCR) is solved using a proximal gradient descent algorithm, and we unroll it into deep networks using the deep unrolling algorithm. Classification is then performed using a convolutional prototype learning model. We evaluated the effectiveness of the proposed method on the ABIDE I dataset and observed a significant improvement in model performance and classification accuracy. The resting state fMRI images are preprocessed into time series data and 3D coordinates of each region of interest. The data are fed into the DUSCR model, a model for building functional brain networks using deep learning instead of traditional models, that we propose, and then the outputs are fed into the convolutional classifier with prototype learning to determine whether the patient has ASD disease.

DAUF: A disease-related attentional UNet framework for progressive and stable mild cognitive impairment identification

Identifying progressive mild cognitive impairment (pMCI) and stable mild cognitive impairment (sMCI) plays a significant role in early Alzheimer's disease (AD) diagnosis, which can effectively boost the life quality of patients. Recently, convolutional neural network (CNN)- based methods using structural magnetic resonance imaging (sMRI) images have shown effective for AD identification. However, these CNN-based methods fail to effectively explore the feature extraction of disease-related multi-scale tissues, such as ventricles, hippocampi and cerebral cortex. To address this issue, we propose an end-to-end disease-related attentional UNet framework (DAUF) for identifying pMCI and sMCI, by embedding a devised dual disease-related attention module (D2AM) and a novel tree-structured feature fusion classifier (TFFC). Specifically, D2AM leverages the complementarity between feature maps and attention maps and the complementary features from the encoder and decoder, so as to highlight discriminative semantic and detailed features. Additionally, TFFC is a powerfully joint multi-scale feature fusion and classification head, by employing the homogeneity among multi-scale features, so that the discriminative features of the multi-scale tissues are adequately fused for enhancing classification performance. Finally, extensive experiments demonstrate the superior performance of DAUF, with the effectiveness of D2AM and TFFC on identifying pMCI and sMCI subjects.

A new weakly supervised deep neural network for recognizing Alzheimer's disease

Alzheimer's disease (AD) is a chronic neurodegenerative disease that mainly affects older adults, causing memory loss and decline in thinking skills. In recent years, many traditional machine learning and deep learning methods have been used to assist in the diagnosis of AD, and most existing methods focus on early prediction of disease on a supervised basis. In reality, there is a massive amount of medical data available. However, some of those data have problems with the low-quality or lack of labels, and the cost of labeling them will be too high. To solve above problem, a new Weakly Supervised Deep Learning model (WSDL) is proposed, which adds attention mechanisms and consistency regularization to the EfficientNet framework and uses data augmentation techniques on the original data that can take full advantage of this unlabeled data. Validation of the proposed WSDL method on the brain MRI datasets of the Alzheimer's Disease Neuroimaging Program by setting five different unlabeled ratios to complete weakly supervised training showed better performance according to the compared experimental results with others baselines.

Global Transformer and Dual Local Attention Network via Deep-Shallow Hierarchical Feature Fusion for Retinal Vessel Segmentation

Clinically, retinal vessel segmentation is a significant step in the diagnosis of fundus diseases. However, recent methods generally neglect the difference of semantic information between deep and shallow features, which fail to capture the global and local characterizations in fundus images simultaneously, resulting in the limited segmentation performance for fine vessels. In this article, a global transformer (GT) and dual local attention (DLA) network via deep-shallow hierarchical feature fusion (GT-DLA-dsHFF) are investigated to solve the above limitations. First, the GT is developed to integrate the global information in the retinal image, which effectively captures the long-distance dependence between pixels, alleviating the discontinuity of blood vessels in the segmentation results. Second, DLA, which is constructed using dilated convolutions with varied dilation rates, unsupervised edge detection, and squeeze-excitation block, is proposed to extract local vessel information, consolidating the edge details in the segmentation result. Finally, a novel deep-shallow hierarchical feature fusion (dsHFF) algorithm is studied to fuse the features in different scales in the deep learning framework, respectively, which can mitigate the attenuation of valid information in the process of feature fusion. We verified the GT-DLA-dsHFF on four typical fundus image datasets. The experimental results demonstrate our GT-DLA-dsHFF achieves superior performance against the current methods and detailed discussions verify the efficacy of the proposed three modules. Segmentation results of diseased images show the robustness of our proposed GT-DLA-dsHFF. Implementation codes will be available on https://github.com/YangLibuaa/GT-DLA-dsHFF.

Personalized Functional Connectivity Based Spatio-Temporal Aggregated Attention Network for MCI Identification

Functional connectivity (FC) networks deri- ved from resting-state magnetic resonance image (rs-fMRI) are effective biomarkers for identifying mild cognitive impairment (MCI) patients. However, most FC identification methods simply extract features from group-averaged brain templates, and neglect inter-subject functional variations. Furthermore, the existing methods generally concentrate on spatial correlation among brain regions, resulting in the inefficient capture of the fMRI temporal features. To address these limitations, we propose a novel personalized functional connectivity based dual-branch graph neural network with spatio-temporal aggregated attention (PFC-DBGNN-STAA) for MCI identification. Specifically, a personalized functional connectivity (PFC) template is firstly constructed to align 213 functional regions across samples and generate discriminative individualized FC features. Secondly, a dual-branch graph neural network (DBGNN) is conducted by aggregating features from the individual- and group-level templates with the cross-template FC, which is beneficial to improve the feature discrimination by considering dependency between templates. Finally, a spatio-temporal aggregated attention (STAA) module is investigated to capture the spatial and dynamic relationships between functional regions, which solves the limitation of insufficient temporal information utilization. We evaluate our proposed method on 442 samples from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, and achieve the accuracies of 90.1%, 90.3%, 83.3% for normal control (NC) vs. early MCI (EMCI), EMCI vs. late MCI (LMCI), and NC vs. EMCI vs. LMCI classification tasks, respectively, indicating that our method boosts MCI identification performance and outperforms state-of-the-art methods.

Dynamic multi-site graph convolutional network for autism spectrum disorder identification

Multi-site learning has attracted increasing interests in autism spectrum disorder (ASD) identification tasks by its efficacy on capturing data heterogeneity of neuroimaging taken from different medical sites. However, existing multi-site graph convolutional network (MSGCN) often ignores the correlations between different sites, and may obtain suboptimal identification results. Moreover, current feature extraction methods characterizing temporal variations of functional magnetic resonance imaging (fMRI) signals require the time series to be of the same length and cannot be directly applied to multi-site fMRI datasets. To address these problems, we propose a dual graph based dynamic multi-site graph convolutional network (DG-DMSGCN) for multi-site ASD identification. First, a sliding-window dual-graph convolutional network (SW-DGCN) is introduced for feature extraction, simultaneously capturing temporal and spatial features of fMRI data with different series lengths. Then we aggregate the features extracted from multiple medical sites through a novel dynamic multi-site graph convolutional network (DMSGCN), which effectively considers the correlations between different sites and is beneficial to improve identification performance. We evaluate the proposed DG-DMSGCN on public ABIDE I dataset containing data from 17 medical sites. The promising results obtained by our framework outperforms the state-of-the-art methods with increase in identification accuracy, indicating that it has a potential clinical prospect for practical ASD diagnosis. Our codes are available on https://github.com/Junling-Du/DG-DMSGCN.

Multicenter and Multichannel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network

For significant memory concern (SMC) and mild cognitive impairment (MCI), their classification performance is limited by confounding features, diverse imaging protocols, and limited sample size. To address the above limitations, we introduce a dual-modality fused brain connectivity network combining resting-state functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), and propose three mechanisms in the current graph convolutional network (GCN) to improve classifier performance. First, we introduce a DTI-strength penalty term for constructing functional connectivity networks. Stronger structural connectivity and bigger structural strength diversity between groups provide a higher opportunity for retaining connectivity information. Second, a multi-center attention graph with each node representing a subject is proposed to consider the influence of data source, gender, acquisition equipment, and disease status of those training samples in GCN. The attention mechanism captures their different impacts on edge weights. Third, we propose a multi-channel mechanism to improve filter performance, assigning different filters to features based on feature statistics. Applying those nodes with low-quality features to perform convolution would also deteriorate filter performance. Therefore, we further propose a pooling mechanism, which introduces the disease status information of those training samples to evaluate the quality of nodes. Finally, we obtain the final classification results by inputting the multi-center attention graph into the multi-channel pooling GCN. The proposed method is tested on three datasets (i.e., an ADNI 2 dataset, an ADNI 3 dataset, and an in-house dataset). Experimental results indicate that the proposed method is effective and superior to other related algorithms, with a mean classification accuracy of 93.05% in our binary classification tasks. Our code is available at: https://github.com/Xuegang-S.

Exploring high-order correlations with deep-broad learning for autism spectrum disorder diagnosis

Recently, a lot of research has been conducted on diagnosing neurological disorders, such as autism spectrum disorder (ASD). Functional magnetic resonance imaging (fMRI) is the commonly used technique to assist in the diagnosis of ASD. In the past years, some conventional methods have been proposed to extract the low-order functional connectivity network features for ASD diagnosis, which ignore the complexity and global features of the brain network. Most deep learning-based methods generally have a large number of parameters that need to be adjusted during the learning process. To overcome the limitations mentioned above, we propose a novel deep-broad learning method for learning the higher-order brain functional connectivity network features to assist in ASD diagnosis. Specifically, we first construct the high-order functional connectivity network that describes global correlations of the brain regions based on hypergraph, and then we use the deep-broad learning method to extract the high-dimensional feature representations for brain networks sequentially. The evaluation of the proposed method is conducted on Autism Brain Imaging Data Exchange (ABIDE) dataset. The results show that our proposed method can achieve 71.8% accuracy on the multi-center dataset and 70.6% average accuracy on 17 single-center datasets, which are the best results compared with the state-of-the-art methods. Experimental results demonstrate that our method can describe the global features of the brain regions and get rich discriminative information for the classification task.

Deep Spatio-Temporal Attention-based Recurrent Network from Dynamic Adaptive Functional Connectivity for MCI Identification

Most existing methods of constructing dynamic functional connectivity (dFC) network obtain the connectivity strength via the sliding window correlation (SWC) method, which estimates the connectivity strength at each time segment, rather than at each time point, and thus is difficult to produce accurate dFC network due to the influence of the window type and window width. Furthermore, the deep learning methods may not capture the discriminative spatio-temporal information that is closely related to disease, thus impacting the performance of (mild cognitive impairment) MCI identification. In this paper, a novel spatio-temporal attention-based bidirectional gated recurrent unit (STA-BiGRU) network is proposed to extract inherent spatio-temporal information from a dynamic adaptive functional connectivity (dAFC) network for MCI diagnosis. Specifically, we adopt a group lasso-based Kalman filter algorithm to obtain the dAFC network with more accurate connectivity strength at each time step. Then a spatial attention module with self-attention and a temporal attention module with multiple temporal attention vectors are incorporated into the BiGRU network to extract more discriminative disease-related spatio-temporal information. Finally, the spatio-temporal regularizations are employed to better guide the attention learning of STA-BiGRU network to enhance the robustness of the deep network. Experimental results show that the proposed framework achieves mean accuracies of 90.2%, 90.0%, and 81.5%, respectively, for three MCI classification tasks. This study provides a more effective deep spatio-temporal attention-based recurrent network and obtains good performance and interpretability of deep learning for psychiatry diagnosis research.

Dual Encoder-Based Dynamic-Channel Graph Convolutional Network With Edge Enhancement for Retinal Vessel Segmentation

Retinal vessel segmentation with deep learning technology is a crucial auxiliary method for clinicians to diagnose fundus diseases. However, the deep learning approaches inevitably lose the edge information, which contains spatial features of vessels while performing down-sampling, leading to the limited segmentation performance of fine blood vessels. Furthermore, the existing methods ignore the dynamic topological correlations among feature maps in the deep learning framework, resulting in the inefficient capture of the channel characterization. To address these limitations, we propose a novel dual encoder-based dynamic-channel graph convolutional network with edge enhancement (DE-DCGCN-EE) for retinal vessel segmentation. Specifically, we first design an edge detection-based dual encoder to preserve the edge of vessels in down-sampling. Secondly, we investigate a dynamic-channel graph convolutional network to map the image channels to the topological space and synthesize the features of each channel on the topological map, which solves the limitation of insufficient channel information utilization. Finally, we study an edge enhancement block, aiming to fuse the edge and spatial features in the dual encoder, which is beneficial to improve the accuracy of fine blood vessel segmentation. Competitive experimental results on five retinal image datasets validate the efficacy of the proposed DE-DCGCN-EE, which achieves more remarkable segmentation results against the other state-of-the-art methods, indicating its potential clinical application.