1
|
Qu G, Zhou Z, Calhoun VD, Zhang A, Wang YP. Integrated brain connectivity analysis with fMRI, DTI, and sMRI powered by interpretable graph neural networks. Med Image Anal 2025; 103:103570. [PMID: 40250104 DOI: 10.1016/j.media.2025.103570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2024] [Revised: 03/19/2025] [Accepted: 03/25/2025] [Indexed: 04/20/2025]
Abstract
Multimodal neuroimaging data modeling has become a widely used approach but confronts considerable challenges due to their heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret diverse datasets within a cohesive analytical framework. In our research, we combine functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) for joint analysis. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from multiple modalities - functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI - within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating an amalgamation of multimodal imaging data. This technique enhances interpretability at the connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved prediction accuracy and uncovers crucial anatomical features and neural connections, deepening our understanding of brain structure and function. This study not only advances multimodal neuroimaging analytics by offering a novel method for integrative analysis of diverse imaging modalities but also improves the understanding of intricate relationships between brain's structural and functional networks and cognitive development.
Collapse
Affiliation(s)
- Gang Qu
- Biomedical Engineering Department, Tulane University, New Orleans, LA 70118, USA.
| | - Ziyu Zhou
- Computer Science Department, Tulane University, New Orleans, LA 70118, USA
| | - Vince D Calhoun
- Tri-Institutional Center for Translational Research in Neuro Imaging and Data Science (TreNDS) - Georgia State, Georgia Tech and Emory, Atlanta, GA 30303, USA
| | - Aiying Zhang
- School of Data Science, University of Virginia, Charlottesville, VA 22903, USA.
| | - Yu-Ping Wang
- Biomedical Engineering Department, Tulane University, New Orleans, LA 70118, USA.
| |
Collapse
|
2
|
Gao Y, Li Z, Zhai X, Zhang G, Zhang L, Huang T, Han L, Wang J, Yan R, Li Y, Zhao H, Zhao Q, Wei Z, Xie B, Sun Y, Zhao J, Cui H. MRI-based habitat radiomics combined with vision transformer for identifying vulnerable intracranial atherosclerotic plaques and predicting stroke events: a multicenter, retrospective study. EClinicalMedicine 2025; 82:103186. [PMID: 40235946 PMCID: PMC11999680 DOI: 10.1016/j.eclinm.2025.103186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/02/2024] [Revised: 03/16/2025] [Accepted: 03/18/2025] [Indexed: 04/17/2025] Open
Abstract
Background Accurate identification of high-risk vulnerable plaques and assessment of stroke risk are crucial for clinical decision-making, yet reliable non-invasive predictive tools are currently lacking. This study aimed to develop an artificial intelligence model based on high-resolution vessel wall imaging (HR-VWI) to assist in the identification of vulnerable plaques and prediction of stroke recurrence risk in patients with symptomatic intracranial atherosclerotic stenosis (sICAS). Methods Between June 2018 and June 2024, a retrospective collection of HR-VWI images from 1806 plaques in 726 sICAS patients across four medical institutions was conducted. K-means clustering was applied to the T1-weighted imaging (T1WI) and T1-weighted imaging with contrast enhancement (T1CE) sequences. Following feature extraction and selection, radiomic models and habitat models were constructed. Additionally, the Vision Transformer (ViT) architecture was utilized for HR-VWI image analysis to build a deep learning model. A stacking fusion strategy was employed to integrate the habitat model and ViT model, enabling effective identification of high-risk vulnerable plaques in the intracranial region and prediction of stroke recurrence risk. Model performance was evaluated using receiver operating characteristic (ROC) curves, and model comparisons were conducted using the DeLong test. Furthermore, decision curve analysis and calibration curves were utilized to assess the practicality and clinical value of the model. Findings The fused Habitat + ViT model exhibited excellent performance in both the validation and test sets. In the validation set, the model achieved an area under the curve (AUC) of 0.949 (95% CI: 0.927-0.969), with a sensitivity of 0.879 (95% CI: 0.840-0.945), a specificity of 0.905 (95% CI: 0.842-0.949), and an accuracy of 0.897 (95% CI: 0.870-0.926). In the test set, the AUC increased to 0.960 (95% CI: 0.941-0.973), with specificity rising to 0.963 and an accuracy of 0.885 (95% CI: 0.857-0.913). The DeLong test revealed statistically significant differences in AUC between the fused model and the single-modal models (test set, vs. ViT p = 0.000; vs. Habitat p = 0.000) Cox regression analysis showed that the Habitat + ViT index, based on the prediction probability of the Habitat + ViT model, was an independent predictor of stroke recurrence (HR: 2.07; 95% CI: 1.12-3.81), with significant predictive power for stroke events at multiple time points. Specifically, measured by AUC values, the model's predictive performance at 1, 2, 3, and 4 years was 0.751 (95% CI: 0.679-0.823), 0.820 (95% CI: 0.760-0.876), 0.815 (95% CI: 0.753-0.877), and 0.780 (95% CI: 0.680-0.873), respectively. Interpretation The integrated Habitat + ViT model based on HR-VWI demonstrated superior performance in identifying high-risk vulnerable plaques in sICAS patients and predicting stroke recurrence risk, providing valuable support for clinical decision-making. Funding This study was supported by the National Natural Science Foundation of China (grant 82204933). Henan Key Laboratory of Neurorestoratology (HNSJXF-2021-004), 2019 Joint Construction Project of Henan Provincial Health Committee and Ministry of Health (SB201901061), and the Xin Xiang City Acute Ischemic Stroke Precision Prevention and Treatment Key Laboratory.
Collapse
Affiliation(s)
- Yu Gao
- Department of MRI Center, The First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou, China
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Ziang Li
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Xiaoyang Zhai
- Department of MRI Center, The First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou, China
| | - Gang Zhang
- Department of MRI Center, The First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou, China
| | - Lan Zhang
- Department of MRI Center, The First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou, China
| | - Tingting Huang
- Department of MRI Center, The First Affiliated Hospital of Henan University of Chinese Medicine, Zhengzhou, China
| | - Lin Han
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Jie Wang
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Ruifang Yan
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Yongdong Li
- Institute of Diagnostic and Interventional Radiology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - HongLing Zhao
- Department of Neurology Center, Xinxiang Central Hospital, Xin Xiang, China
| | - Qiuyi Zhao
- The Second School of Clinical Medicine, ZhengZhou University, Zhengzhou, China
| | - Zhengqi Wei
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Beichen Xie
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Yancong Sun
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Jianhua Zhao
- Department of Neurology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| | - Hongkai Cui
- Department of Radiology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
- Department of Neurology Center, The First Affiliated Hospital of Xinxiang Medical University, Xin Xiang, China
| |
Collapse
|
3
|
Mao J, Liu J, Tian X, Pan Y, Trucco E, Lin H. Toward Integrating Federated Learning With Split Learning via Spatio-Temporal Graph Framework for Brain Disease Prediction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2025; 44:1334-1346. [PMID: 39509311 DOI: 10.1109/tmi.2024.3493195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2024]
Abstract
Functional Magnetic Resonance Imaging (fMRI) is used for extracting blood oxygen signals from brain regions to map brain functional connectivity for brain disease prediction. Despite its effectiveness, fMRI has not been widely used: on the one hand, collecting and labeling the data is time-consuming and costly, which limits the amount of valid data collected at a single healthcare site; on the other hand, integrating data from multiple sites is challenging due to data privacy restrictions. To address these issues, we propose a novel, integrated Federated learning and Split learning Spatio-temporal Graph framework (F G). Specifically, we introduce federated learning and split learning techniques to split a spatio-temporal model into a client temporal model and a server spatial model. In the client temporal model, we propose a time-aware mechanism to focus on changes in brain functional states and use an InceptionTime model to extract information about changes in the brain states of each subject. In the server spatial model, we propose a united graph convolutional network to integrate multiple graph convolutional networks. Integrating federated learning and split learning, F G can utilize multi-site fMRI data without violating data privacy protection and reduce the risk of overfitting as it is capable of learning from limited training data sets. Moreover, it boosts the extraction of spatio-temporal features of fMRI using spatio-temporal graph networks. Experiments on ABIDE and ADHD200 datasets demonstrate that our proposed method outperforms state-of-the-art methods. In addition, we explore biomarkers associated with brain disease prediction using community discovery algorithms using intermediate results of F G. The source code is available at https://github.com/yutian0315/FS2G.
Collapse
|
4
|
Zhu X, Sun S, Lin L, Wu Y, Ma X. Transformer-based approaches for neuroimaging: an in-depth review of their role in classification and regression tasks. Rev Neurosci 2025; 36:209-228. [PMID: 39333087 DOI: 10.1515/revneuro-2024-0088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2024] [Accepted: 09/13/2024] [Indexed: 09/29/2024]
Abstract
In the ever-evolving landscape of deep learning (DL), the transformer model emerges as a formidable neural network architecture, gaining significant traction in neuroimaging-based classification and regression tasks. This paper presents an extensive examination of transformer's application in neuroimaging, surveying recent literature to elucidate its current status and research advancement. Commencing with an exposition on the fundamental principles and structures of the transformer model and its variants, this review navigates through the methodologies and experimental findings pertaining to their utilization in neuroimage classification and regression tasks. We highlight the transformer model's prowess in neuroimaging, showcasing its exceptional performance in classification endeavors while also showcasing its burgeoning potential in regression tasks. Concluding with an assessment of prevailing challenges and future trajectories, this paper proffers insights into prospective research directions. By elucidating the current landscape and envisaging future trends, this review enhances comprehension of transformer's role in neuroimaging tasks, furnishing valuable guidance for further inquiry.
Collapse
Affiliation(s)
- Xinyu Zhu
- Department of Biomedical Engineering, 12496 College of Chemistry and Life Sciences, Beijing University of Technology , Beijing, 100124, China
| | - Shen Sun
- Department of Biomedical Engineering, 12496 College of Chemistry and Life Sciences, Beijing University of Technology , Beijing, 100124, China
| | - Lan Lin
- Department of Biomedical Engineering, 12496 College of Chemistry and Life Sciences, Beijing University of Technology , Beijing, 100124, China
| | - Yutong Wu
- Department of Biomedical Engineering, 12496 College of Chemistry and Life Sciences, Beijing University of Technology , Beijing, 100124, China
| | - Xiangge Ma
- Department of Biomedical Engineering, 12496 College of Chemistry and Life Sciences, Beijing University of Technology , Beijing, 100124, China
| |
Collapse
|
5
|
Wu Y, Zhang C, Ma X, Zhu X, Lin L, Tian M. ds-FCRN: three-dimensional dual-stream fully convolutional residual networks and transformer-based global-local feature learning for brain age prediction. Brain Struct Funct 2025; 230:32. [PMID: 39826018 DOI: 10.1007/s00429-024-02889-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2024] [Accepted: 12/23/2024] [Indexed: 01/20/2025]
Abstract
The brain undergoes atrophy and cognitive decline with advancing age. The utilization of brain age prediction represents a pioneering methodology in the examination of brain aging. This study aims to develop a deep learning model with high predictive accuracy and interpretability for brain age prediction tasks. The gray matter (GM) density maps obtained from T1 MRI data of 16,377 healthy participants aged 45 to 82 years from the UKB database were included in this study (mean age, 64.27 ± 7.52 , 7811 men). We propose an innovative deep learning architecture for predicting brain age based on GM density maps. The architecture combines a 3D dual-stream fully convolutional residual network (ds-FCRN) with a Transformer-based global-local feature learning paradigm to enhance prediction accuracy. Moreover, we employed Shapley values to elucidate the influence of various brain regions on prediction precision. On a test set of 3,276 healthy subjects (mean age, 64.15 ± 7.45 , 1561 men), our 3D ds-FCRN model achieved a mean absolute error of 2.2 years in brain age prediction, outperforming existing models on the same dataset. The posterior interpretation revealed that the temporal lobe plays the most significant role in the brain age prediction process, while frontal lobe aging is associated with the greatest number of lifestyle factors. Our designed 3D ds-FCRN model achieved high predictive accuracy and high decision transparency. The brain age vectors constructed using Shapley values provided brain region-level insights into life factors associated with abnormal brain aging.
Collapse
Affiliation(s)
- Yutong Wu
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China
| | - Chen Zhang
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China
| | - Xiangge Ma
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China
| | - Xinyu Zhu
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China
| | - Lan Lin
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China.
| | - Miao Tian
- Department of Biomedical Engineering, College of Chemistry and Life Sciences, Beijing University of Technology, Beijing, 100124, China.
| |
Collapse
|
6
|
Liu X, Zheng G, Beheshti I, Ji S, Gou Z, Cui W. Low-Rank Tensor Fusion for Enhanced Deep Learning-Based Multimodal Brain Age Estimation. Brain Sci 2024; 14:1252. [PMID: 39766451 PMCID: PMC11674316 DOI: 10.3390/brainsci14121252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2024] [Revised: 11/26/2024] [Accepted: 12/10/2024] [Indexed: 01/11/2025] Open
Abstract
Background/Objectives: A multimodal brain age estimation model could provide enhanced insights into brain aging. However, effectively integrating multimodal neuroimaging data to enhance the accuracy of brain age estimation remains a challenging task. Methods: In this study, we developed an innovative data fusion technique employing a low-rank tensor fusion algorithm, tailored specifically for deep learning-based frameworks aimed at brain age estimation. Specifically, we utilized structural magnetic resonance imaging (sMRI), diffusion tensor imaging (DTI), and magnetoencephalography (MEG) to extract spatial-temporal brain features with different properties. These features were fused using the low-rank tensor algorithm and employed as predictors for estimating brain age. Results: Our prediction model achieved a desirable prediction accuracy on the independent test samples, demonstrating its robust performance. Conclusions: The results of our study suggest that the low-rank tensor fusion algorithm has the potential to effectively integrate multimodal data into deep learning frameworks for estimating brain age.
Collapse
Affiliation(s)
- Xia Liu
- School of Management Science and Information Engineering, Hebei University of Economics and Businesses, Shijiazhuang 050061, China; (X.L.); (Z.G.); (W.C.)
| | - Guowei Zheng
- School of Computer Science and Technology, Harbin Institute of Technology, Weihai 264209, China;
| | - Iman Beheshti
- Department of Human Anatomy and Cell Science, University of Manitoba, Winnipeg, MB R3T 2N2, Canada
| | - Shanling Ji
- Institute of Mental Health, Jining Medical University, Jining 272111, China;
| | - Zhinan Gou
- School of Management Science and Information Engineering, Hebei University of Economics and Businesses, Shijiazhuang 050061, China; (X.L.); (Z.G.); (W.C.)
| | - Wenkuo Cui
- School of Management Science and Information Engineering, Hebei University of Economics and Businesses, Shijiazhuang 050061, China; (X.L.); (Z.G.); (W.C.)
| |
Collapse
|
7
|
Sun J, Han JDJ, Chen W. Exploring the relationship among Alzheimer's disease, aging and cognitive scores through neuroimaging-based approach. Sci Rep 2024; 14:27472. [PMID: 39523370 PMCID: PMC11551169 DOI: 10.1038/s41598-024-78712-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2024] [Accepted: 11/04/2024] [Indexed: 11/16/2024] Open
Abstract
Alzheimer's disease (AD) is a fatal neurodegenerative disorder, with the Mini-Mental State Examination (MMSE) and Clinical Dementia Rating (CDR) serving significant roles in monitoring its progression. We hypothesize that while cognitive assessment scores can detect AD-related brain changes, the targeted brain regions may differ. Additionally, given AD's strong association with aging, we propose that specific brain regions are influenced by both AD pathology and aging, exhibiting strong correlations with both. To test these hypotheses, we developed a 3D convolutional network with a mixed-attention mechanism to recognize AD subjects from structural magnetic resonance imaging (sMRI) data and utilize 3D convolutional methods to pinpoint brain regions significantly correlated with the AD, MMSE, CDR and age. All models were trained and internally validated on 417 samples from the Alzheimer's Disease Neuroimaging Initiative (ADNI), and the classification model was externally validated on 382 samples from the Australian Imaging and Lifestyle flagship (AIBL). This approach provided robust support for using MMSE and CDR in assessing AD progression and visually illustrated the relationship between aging and AD. The analysis revealed correlations among the four identification tasks (AD, MMSE, CDR and age) and highlighted asymmetric brain lesions in both AD and aging. Notably, we found that AD can accelerate aging to some extent, and a significant correlation exists between the rate of aging and cognitive assessment scores. This offers new insights into the relationship between AD and aging.
Collapse
Affiliation(s)
- Jinhui Sun
- School of Cyber Science and Engineering, Qufu Normal University, Qufu, 273165, People's Republic of China
| | - Jing-Dong J Han
- Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, Center for Quantitative Biology (CQB), Peking University, Beijing, 100871, People's Republic of China.
| | - Weiyang Chen
- School of Cyber Science and Engineering, Qufu Normal University, Qufu, 273165, People's Republic of China.
| |
Collapse
|
8
|
Luo J, Zhang Z, Ma X, Yan C, Luo H. GTasm: a genome assembly method using graph transformers and HiFi reads. Front Genet 2024; 15:1495657. [PMID: 39525812 PMCID: PMC11543488 DOI: 10.3389/fgene.2024.1495657] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2024] [Accepted: 10/14/2024] [Indexed: 11/16/2024] Open
Abstract
Motivation Genome assembly aims to reconstruct the whole chromosome-scale genome sequence. Obtaining accurate and complete chromosome-scale genome sequence serve as an indispensable foundation for downstream genomics analyses. Due to the complex repeat regions contained in genome sequence, the assembly results commonly are fragmented. Long reads with high accuracy rate can greatly enhance the integrity of genome assembly results. Results Here we introduce GTasm, an assembly method that uses graph transformer network to find optimal assembly results based on assembly graphs. Based on assembly graph, GTasm first extracts features about vertices and edges. Then, GTasm scores the edges by graph transformer model, and adopt a heuristic algorithm to find optimal paths in the assembly graph, each path corresponding to a contig. The graph transformer model is trained using simulated HiFi reads from CHM13, and GTasm is compared with other assembly methods using real HIFI read set. Through experimental result, GTasm can produce well assembly results, and achieve good performance on NA50 and NGA50 evaluation indicators. Applying deep learning models to genome assembly can improve the continuity and accuracy of assembly results. The code is available from https://github.com/chu-xuezhe/GTasm.
Collapse
Affiliation(s)
- Junwei Luo
- School of Software, Henan Polytechnic University, Jiaozuo, China
| | - Ziheng Zhang
- School of Software, Henan Polytechnic University, Jiaozuo, China
| | - Xinliang Ma
- School of Software, Henan Polytechnic University, Jiaozuo, China
| | - Chaokun Yan
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| | - Huimin Luo
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| |
Collapse
|
9
|
Gaser C, Kalc P, Cole JH. A perspective on brain-age estimation and its clinical promise. NATURE COMPUTATIONAL SCIENCE 2024; 4:744-751. [PMID: 39048692 DOI: 10.1038/s43588-024-00659-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 06/12/2024] [Indexed: 07/27/2024]
Abstract
Brain-age estimation has gained increased attention in the neuroscientific community owing to its potential use as a biomarker of brain health. The difference between estimated and chronological age based on neuroimaging data enables a unique perspective on brain development and aging, with multiple open questions still remaining in the brain-age research field. This Perspective presents an overview of current advancements in the field and envisions the future evolution of the brain-age framework before its potential deployment in hospital settings.
Collapse
Affiliation(s)
- Christian Gaser
- Structural Brain Mapping Group, Department of Neurology, Jena University Hospital, Jena, Germany.
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany.
- German Centre for Mental Health (DZPG), Jena-Halle-Magdeburg, Jena, Germany.
| | - Polona Kalc
- Structural Brain Mapping Group, Department of Neurology, Jena University Hospital, Jena, Germany
| | - James H Cole
- Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK
- Dementia Research Centre, Queen Square Institute of Neurology, University College London, London, UK
| |
Collapse
|
10
|
Peng L, Cai S, Wu Z, Shang H, Zhu X, Li X. MMGPL: Multimodal Medical Data Analysis with Graph Prompt Learning. Med Image Anal 2024; 97:103225. [PMID: 38908306 DOI: 10.1016/j.media.2024.103225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Revised: 04/12/2024] [Accepted: 05/27/2024] [Indexed: 06/24/2024]
Abstract
Prompt learning has demonstrated impressive efficacy in the fine-tuning of multimodal large models to a wide range of downstream tasks. Nonetheless, applying existing prompt learning methods for the diagnosis of neurological disorder still suffers from two issues: (i) existing methods typically treat all patches equally, despite the fact that only a small number of patches in neuroimaging are relevant to the disease, and (ii) they ignore the structural information inherent in the brain connection network which is crucial for understanding and diagnosing neurological disorders. To tackle these issues, we introduce a novel prompt learning model by learning graph prompts during the fine-tuning process of multimodal models for diagnosing neurological disorders. Specifically, we first leverage GPT-4 to obtain relevant disease concepts and compute semantic similarity between these concepts and all patches. Secondly, we reduce the weight of irrelevant patches according to the semantic similarity between each patch and disease-related concepts. Moreover, we construct a graph among tokens based on these concepts and employ a graph convolutional network layer to extract the structural information of the graph, which is used to prompt the pre-trained multimodal models for diagnosing neurological disorders. Extensive experiments demonstrate that our method achieves superior performance for neurological disorder diagnosis compared with state-of-the-art methods and validated by clinicians.
Collapse
Affiliation(s)
- Liang Peng
- Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518000, China
| | - Songyue Cai
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Zongqian Wu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Huifang Shang
- Department of Neurology, Laboratory of Neurodegenerative Disorders, National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu 610041, China
| | - Xiaofeng Zhu
- Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518000, China.
| | - Xiaoxiao Li
- School of Electrical and Computer Engineering, University of British Columbia, Vancouver BC V6T 1Z4, Canada
| |
Collapse
|
11
|
Jiang N, Wang G, Ye C, Liu T, Yan T. Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning. IEEE J Biomed Health Inform 2024; 28:5528-5539. [PMID: 38889024 DOI: 10.1109/jbhi.2024.3416038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/20/2024]
Abstract
Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.
Collapse
|
12
|
Johnson R, Li MM, Noori A, Queen O, Zitnik M. Graph Artificial Intelligence in Medicine. Annu Rev Biomed Data Sci 2024; 7:345-368. [PMID: 38749465 PMCID: PMC11344018 DOI: 10.1146/annurev-biodatasci-110723-024625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/23/2024]
Abstract
In clinical artificial intelligence (AI), graph representation learning, mainly through graph neural networks and graph transformer architectures, stands out for its capability to capture intricate relationships and structures within clinical datasets. With diverse data-from patient records to imaging-graph AI models process data holistically by viewing modalities and entities within them as nodes interconnected by their relationships. Graph AI facilitates model transfer across clinical tasks, enabling models to generalize across patient populations without additional parameters and with minimal to no retraining. However, the importance of human-centered design and model interpretability in clinical decision-making cannot be overstated. Since graph AI models capture information through localized neural transformations defined on relational datasets, they offer both an opportunity and a challenge in elucidating model rationale. Knowledge graphs can enhance interpretability by aligning model-driven insights with medical knowledge. Emerging graph AI models integrate diverse data modalities through pretraining, facilitate interactive feedback loops, and foster human-AI collaboration, paving the way toward clinically meaningful predictions.
Collapse
Affiliation(s)
- Ruth Johnson
- Berkowitz Family Living Laboratory, Harvard Medical School, Boston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Michelle M Li
- Bioinformatics and Integrative Genomics Program, Harvard Medical School, Boston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Ayush Noori
- Department of Computer Science, Harvard John A. Paulson School of Engineering and Applied Sciences, Allston, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Owen Queen
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| | - Marinka Zitnik
- Harvard Data Science Initiative, Cambridge, Massachusetts, USA
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Allston, Massachusetts, USA
- Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA;
| |
Collapse
|
13
|
Aghaei A, Ebrahimi Moghaddam M. Brain age gap estimation using attention-based ResNet method for Alzheimer's disease detection. Brain Inform 2024; 11:16. [PMID: 38833039 DOI: 10.1186/s40708-024-00230-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 05/27/2024] [Indexed: 06/06/2024] Open
Abstract
This study investigates the correlation between brain age and chronological age in healthy individuals using brain MRI images, aiming to identify potential biomarkers for neurodegenerative diseases like Alzheimer's. To achieve this, a novel attention-based ResNet method, 3D-Attention-Resent-SVR, is proposed to accurately estimate brain age and distinguish between Cognitively Normal (CN) and Alzheimer's disease (AD) individuals by computing the brain age gap (BAG). Unlike conventional methods, which often rely on single datasets, our approach addresses potential biases by employing four datasets for training and testing. The results, based on a combined dataset from four public sources comprising 3844 data points, demonstrate the model's efficacy with a mean absolute error (MAE) of 2.05 for brain age gap estimation. Moreover, the model's generalizability is showcased by training on three datasets and testing on a separate one, yielding a remarkable MAE of 2.4. Furthermore, leveraging BAG as the sole biomarker, our method achieves an accuracy of 92% and an AUC of 0.87 in Alzheimer's disease detection on the ADNI dataset. These findings underscore the potential of our approach in assisting with early detection and disease monitoring, emphasizing the strong correlation between BAG and AD.
Collapse
Affiliation(s)
- Atefe Aghaei
- Faculty of Computer Science and Engineering, Shahid Beheshti University, Tehran, Iran
| | | |
Collapse
|
14
|
Zhang X, Duan SY, Wang SQ, Chen YW, Lai SX, Zou JS, Cheng Y, Guan JT, Wu RH, Zhang XL. A ResNet mini architecture for brain age prediction. Sci Rep 2024; 14:11185. [PMID: 38755275 PMCID: PMC11098808 DOI: 10.1038/s41598-024-61915-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 05/10/2024] [Indexed: 05/18/2024] Open
Abstract
The brain presents age-related structural and functional changes in the human life, with different extends between subjects and groups. Brain age prediction can be used to evaluate the development and aging of human brain, as well as providing valuable information for neurodevelopment and disease diagnosis. Many contributions have been made for this purpose, resorting to different machine learning methods. To solve this task and reduce memory resource consumption, we develop a mini architecture of only 10 layers by modifying the deep residual neural network (ResNet), named ResNet mini architecture. To support the ResNet mini architecture in brain age prediction, the brain age dataset (OpenNeuro #ds000228) that consists of 155 study participants (three classes) and the Alzheimer MRI preprocessed dataset that consists of 6400 images (four classes) are employed. We compared the performance of the ResNet mini architecture with other popular networks using the two considered datasets. Experimental results show that the proposed architecture exhibits generality and robustness with high accuracy and less parameter number.
Collapse
Affiliation(s)
- Xuan Zhang
- College of Engineering, Shantou University, Shantou, 515063, China
| | - Si-Yuan Duan
- College of Computer Science, Sichuan University, Chengdu, 610065, China
| | - Si-Qi Wang
- College of Engineering, Shantou University, Shantou, 515063, China
| | - Yao-Wen Chen
- College of Engineering, Shantou University, Shantou, 515063, China
| | - Shi-Xin Lai
- College of Engineering, Shantou University, Shantou, 515063, China
| | - Ji-Sheng Zou
- College of Engineering, Shantou University, Shantou, 515063, China
| | - Yan Cheng
- Department of Radiology, Second Hospital of Shandong University, Jinan, 250033, China
| | - Ji-Tian Guan
- Department of Radiology, Second Affiliated Hospital of Shantou University Medical College, Shantou, 515041, China
| | - Ren-Hua Wu
- Department of Radiology, Second Affiliated Hospital of Shantou University Medical College, Shantou, 515041, China.
| | - Xiao-Lei Zhang
- Department of Radiology, Second Affiliated Hospital of Shantou University Medical College, Shantou, 515041, China.
| |
Collapse
|
15
|
Lv H, Zeng N, Li M, Sun J, Wu N, Xu M, Chen Q, Zhao X, Chen S, Liu W, Li X, Zhao P, Wintermark M, Hui Y, Li J, Wu S, Wang Z. Association between Body Mass Index and Brain Health in Adults: A 16-Year Population-Based Cohort and Mendelian Randomization Study. HEALTH DATA SCIENCE 2024; 4:0087. [PMID: 38500551 PMCID: PMC10944701 DOI: 10.34133/hds.0087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 02/26/2024] [Indexed: 03/20/2024]
Abstract
Background: The cumulative effect of body mass index (BMI) on brain health remains ill-defined. The effects of overweight on brain health across different age groups need clarification. We analyzed the effect of cumulative BMI on neuroimaging features of brain health in adults of different ages. Methods: This study was based on a multicenter, community-based cohort study. We modeled the trajectories of BMI over 16 years to evaluate cumulative exposure. Multimodality neuroimaging data were collected once for volumetric measurements of the brain macrostructure, white matter hyperintensity (WMH), and brain microstructure. We used a generalized linear model to evaluate the association between cumulative BMI and neuroimaging features. Two-sample Mendelian randomization analysis was performed using summary level of BMI genetic data from 681,275 individuals and neuroimaging genetic data from 33,224 individuals to analyze the causal relationships. Results: Clinical and neuroimaging data were obtained from 1,074 adults (25 to 83 years). For adults aged under 45 years, brain volume differences in participants with a cumulative BMI of >26.2 kg/m2 corresponded to 12.0 years [95% confidence interval (CI), 3.0 to 20.0] of brain aging. Differences in WMH were statistically substantial for participants aged over 60 years, with a 6.0-ml (95% CI, 1.5 to 10.5) larger volume. Genetic analysis indicated causal relationships between high BMI and smaller gray matter and higher fractional anisotropy in projection fibers. Conclusion: High cumulative BMI is associated with smaller brain volume, larger volume of white matter lesions, and abnormal microstructural integrity. Adults younger than 45 years are suggested to maintain their BMI below 26.2 kg/m2 for better brain health. Trial Registration: This study was registered on clinicaltrials.gov (Clinical Indicators and Brain Image Data: A Cohort Study Based on Kailuan Cohort; No. NCT05453877; https://clinicaltrials.gov/ct2/show/NCT05453877).
Collapse
Affiliation(s)
- Han Lv
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Na Zeng
- Peking University School of Public Health, Beijing 100191, China
| | - Mengyi Li
- Department of General Surgery, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Jing Sun
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Ning Wu
- Department of Medical Imaging Technology,
Capital Medical University Yanjing College, Beijing 101300, China
| | - Mingze Xu
- Center for MRI Research,
Peking University Academy for Advanced Interdisciplinary Studies, Beijing 100871, China
| | - Qian Chen
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Xinyu Zhao
- Clinical Epidemiology and Evidence-based Medicine Unit, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Shuohua Chen
- Department of Cardiology, Kailuan General Hospital, Hebei, Tangshan 063000, China
| | - Wenjuan Liu
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Xiaoshuai Li
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Pengfei Zhao
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| | - Max Wintermark
- Department of Neuroradiology,
The University of Texas MD Anderson Cancer Center, Houston, TX 78701, USA
| | - Ying Hui
- Department of Radiology, Kailuan General Hospital, Hebei, Tangshan 063000, China
| | - Jing Li
- Department of Radiology, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine,
Tsinghua University, Beijing, China
| | - Shouling Wu
- Department of Cardiology, Kailuan General Hospital, Hebei, Tangshan 063000, China
| | - Zhenchang Wang
- Department of Radiology, Beijing Friendship Hospital,
Capital Medical University, Beijing 100050, China
| |
Collapse
|
16
|
Lim H, Joo Y, Ha E, Song Y, Yoon S, Shin T. Brain Age Prediction Using Multi-Hop Graph Attention Combined with Convolutional Neural Network. Bioengineering (Basel) 2024; 11:265. [PMID: 38534539 DOI: 10.3390/bioengineering11030265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 02/28/2024] [Accepted: 03/01/2024] [Indexed: 03/28/2024] Open
Abstract
Convolutional neural networks (CNNs) have been used widely to predict biological brain age based on brain magnetic resonance (MR) images. However, CNNs focus mainly on spatially local features and their aggregates and barely on the connective information between distant regions. To overcome this issue, we propose a novel multi-hop graph attention (MGA) module that exploits both the local and global connections of image features when combined with CNNs. After insertion between convolutional layers, MGA first converts the convolution-derived feature map into graph-structured data by using patch embedding and embedding-distance-based scoring. Multi-hop connections between the graph nodes are modeled by using the Markov chain process. After performing multi-hop graph attention, MGA re-converts the graph into an updated feature map and transfers it to the next convolutional layer. We combined the MGA module with sSE (spatial squeeze and excitation)-ResNet18 for our final prediction model (MGA-sSE-ResNet18) and performed various hyperparameter evaluations to identify the optimal parameter combinations. With 2788 three-dimensional T1-weighted MR images of healthy subjects, we verified the effectiveness of MGA-sSE-ResNet18 with comparisons to four established, general-purpose CNNs and two representative brain age prediction models. The proposed model yielded an optimal performance with a mean absolute error of 2.822 years and Pearson's correlation coefficient (PCC) of 0.968, demonstrating the potential of the MGA module to improve the accuracy of brain age prediction.
Collapse
Affiliation(s)
- Heejoo Lim
- Division of Mechanical and Biomedical Engineering, Ewha W. University, Seoul 03760, Republic of Korea
- Graduate Program in Smart Factory, Ewha W. University, Seoul 03760, Republic of Korea
| | - Yoonji Joo
- Ewha Brain Institute, Ewha W. University, Seoul 03760, Republic of Korea
| | - Eunji Ha
- Ewha Brain Institute, Ewha W. University, Seoul 03760, Republic of Korea
| | - Yumi Song
- Ewha Brain Institute, Ewha W. University, Seoul 03760, Republic of Korea
- Department of Brain and Cognitive Sciences, Ewha W. University, Seoul 03760, Republic of Korea
| | - Sujung Yoon
- Ewha Brain Institute, Ewha W. University, Seoul 03760, Republic of Korea
- Department of Brain and Cognitive Sciences, Ewha W. University, Seoul 03760, Republic of Korea
| | - Taehoon Shin
- Division of Mechanical and Biomedical Engineering, Ewha W. University, Seoul 03760, Republic of Korea
- Graduate Program in Smart Factory, Ewha W. University, Seoul 03760, Republic of Korea
| |
Collapse
|
17
|
Wu Y, Chen Y, Yang Y, Lin C, Su S, Zhao J, Wu S, Wu G, Liu H, Liu X, Yang Z, Zhang J, Huang B. Predicting brain age using partition modeling strategy and atlas-based attentional enhancement in the Chinese population. Cereb Cortex 2024; 34:bhae030. [PMID: 38342684 DOI: 10.1093/cercor/bhae030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 01/13/2024] [Accepted: 01/15/2024] [Indexed: 02/13/2024] Open
Abstract
As a biomarker of human brain health during development, brain age is estimated based on subtle differences in brain structure from those under typical developmental. Magnetic resonance imaging (MRI) is a routine diagnostic method in neuroimaging. Brain age prediction based on MRI has been widely studied. However, few studies based on Chinese population have been reported. This study aimed to construct a brain age predictive model for the Chinese population across its lifespan. We developed a partition prediction method based on transfer learning and atlas attention enhancement. The participants were separated into four age groups, and a deep learning model was trained for each group to identify the brain regions most critical for brain age prediction. The Atlas attention-enhancement method was also used to help the models focus only on critical brain regions. The proposed method was validated using 354 participants from domestic datasets. For prediction performance in the testing sets, the mean absolute error was 2.218 ± 1.801 years, and the Pearson correlation coefficient (r) was 0.969, exceeding previous results for wide-range brain age prediction. In conclusion, the proposed method could provide brain age estimation to assist in assessing the status of brain health.
Collapse
Affiliation(s)
- Yingtong Wu
- Medical AI Lab, School of Biomedical Engineering, Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
- Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Key Laboratory for MRI, Institute of Biomedical and Health Engineering, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen 518055, Guangdong Province, China
| | - Yingqian Chen
- Department of Radiology, the First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou 510080, Guangdong Province, China
| | - Yang Yang
- Department of Radiology, Suining Central Hospital, 127 Desheng West Road, Suining 629099, Sichuan Province, China
- Medical Imaging Center of Guizhou Province, Department of Radiology, The Affiliated Hospital of Zunyi Medical University, 149 Dalian Road, Zunyi 563000, Guizhou Province, China
| | - Chuxuan Lin
- Medical AI Lab, School of Biomedical Engineering, Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
- Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
| | - Shu Su
- Department of Radiology, the First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou 510080, Guangdong Province, China
| | - Jing Zhao
- Department of Radiology, the First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou 510080, Guangdong Province, China
| | - Songxiong Wu
- Radiology Department, Shenzhen University General Hospital and Shenzhen University Clinical Medical Academy, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
| | - Guangyao Wu
- Radiology Department, Shenzhen University General Hospital and Shenzhen University Clinical Medical Academy, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
| | - Heng Liu
- Medical Imaging Center of Guizhou Province, Department of Radiology, The Affiliated Hospital of Zunyi Medical University, 149 Dalian Road, Zunyi 563000, Guizhou Province, China
| | - Xia Liu
- Department of Radiology, Shenzhen Mental Health Center, Shenzhen Kangning Hospital, 1080 Cuizhu Road, Shenzhen 518118, Guangdong Province, China
| | - Zhiyun Yang
- Department of Radiology, the First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou 510080, Guangdong Province, China
| | - Jian Zhang
- Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, 1068 Xueyuan Avenue, Shenzhen 518055, Guangdong Province, China
- School of Pharmaceutical Sciences, Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
| | - Bingsheng Huang
- Medical AI Lab, School of Biomedical Engineering, Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
- Guangdong Key Laboratory of Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, 1066 Xueyuan Avenue, Shenzhen 518060, Guangdong Province, China
| |
Collapse
|
18
|
Dular L, Špiclin Ž. BASE: Brain Age Standardized Evaluation. Neuroimage 2024; 285:120469. [PMID: 38065279 DOI: 10.1016/j.neuroimage.2023.120469] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 10/31/2023] [Accepted: 11/20/2023] [Indexed: 01/13/2024] Open
Abstract
Brain age, most commonly inferred from T1-weighted magnetic resonance images (T1w MRI), is a robust biomarker of brain health and related diseases. Superior accuracy in brain age prediction, often falling within a 2-3 year range, is achieved predominantly through deep neural networks. However, comparing study results is difficult due to differences in datasets, evaluation methodologies and metrics. Addressing this, we introduce Brain Age Standardized Evaluation (BASE), which includes (i) a standardized T1w MRI dataset including multi-site, new unseen site, test-retest and longitudinal data, and an associated (ii) evaluation protocol, including repeated model training and upon based comprehensive set of performance metrics measuring accuracy, robustness, reproducibility and consistency aspects of brain age predictions, and (iii) statistical evaluation framework based on linear mixed-effects models for rigorous performance assessment and cross-comparison. To showcase BASE, we comprehensively evaluate four deep learning based brain age models, appraising their performance in scenarios that utilize multi-site, test-retest, unseen site, and longitudinal T1w brain MRI datasets. Ensuring full reproducibility and application in future studies, we have made all associated data information and code publicly accessible at https://github.com/AralRalud/BASE.git.
Collapse
Affiliation(s)
- Lara Dular
- University of Ljubljana, Faculty of Electrical Engineering, Tržaška cesta 25, Ljubljana, 1000, Slovenia
| | - Žiga Špiclin
- University of Ljubljana, Faculty of Electrical Engineering, Tržaška cesta 25, Ljubljana, 1000, Slovenia.
| |
Collapse
|
19
|
Moon HS, Mahzarnia A, Stout J, Anderson RJ, Badea CT, Badea A. Feature attention graph neural network for estimating brain age and identifying important neural connections in mouse models of genetic risk for Alzheimer's disease. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.13.571574. [PMID: 38168445 PMCID: PMC10760088 DOI: 10.1101/2023.12.13.571574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
Alzheimer's disease (AD) remains one of the most extensively researched neurodegenerative disorders due to its widespread prevalence and complex risk factors. Age is a crucial risk factor for AD, which can be estimated by the disparity between physiological age and estimated brain age. To model AD risk more effectively, integrating biological, genetic, and cognitive markers is essential. Here, we utilized mouse models expressing the major APOE human alleles and human nitric oxide synthase 2 to replicate genetic risk for AD and a humanized innate immune response. We estimated brain age employing a multivariate dataset that includes brain connectomes, APOE genotype, subject traits such as age and sex, and behavioral data. Our methodology used Feature Attention Graph Neural Networks (FAGNN) for integrating different data types. Behavioral data were processed with a 2D Convolutional Neural Network (CNN), subject traits with a 1D CNN, brain connectomes through a Graph Neural Network using quadrant attention module. The model yielded a mean absolute error for age prediction of 31.85 days, with a root mean squared error of 41.84 days, outperforming other, reduced models. In addition, FAGNN identified key brain connections involved in the aging process. The highest weights were assigned to the connections between cingulum and corpus callosum, striatum, hippocampus, thalamus, hypothalamus, cerebellum, and piriform cortex. Our study demonstrates the feasibility of predicting brain age in models of aging and genetic risk for AD. To verify the validity of our findings, we compared Fractional Anisotropy (FA) along the tracts of regions with the highest connectivity, the Return-to-Origin Probability (RTOP), Return-to-Plane Probability (RTPP), and Return-to-Axis Probability (RTAP), which showed significant differences between young, middle-aged, and old age groups. Younger mice exhibited higher FA, RTOP, RTAP, and RTPP compared to older groups in the selected connections, suggesting that degradation of white matter tracts plays a critical role in aging and for FAGNN's selections. Our analysis suggests a potential neuroprotective role of APOE2, relative to APOE3 and APOE4, where APOE2 appears to mitigate age-related changes. Our findings highlighted a complex interplay of genetics and brain aging in the context of AD risk modeling.
Collapse
Affiliation(s)
- Hae Sol Moon
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| | - Ali Mahzarnia
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| | - Jacques Stout
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| | - Robert J Anderson
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| | - Cristian T. Badea
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
| | - Alexandra Badea
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
- Quantitative Imaging and Analysis Laboratory, Department of Radiology, Duke University School of Medicine, Durham, NC, USA
- Brain Imaging and Analysis Center, Duke University School of Medicine, Durham, NC, USA
- Department of Neurology, Duke University School of Medicine, Durham, NC, USA
| |
Collapse
|
20
|
Wong SB, Tsao Y, Tsai WH, Wang TS, Wu HC, Wang SS. Application of bidirectional long short-term memory network for prediction of cognitive age. Sci Rep 2023; 13:20197. [PMID: 37980387 PMCID: PMC10657465 DOI: 10.1038/s41598-023-47606-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Accepted: 11/16/2023] [Indexed: 11/20/2023] Open
Abstract
Electroencephalography (EEG) measures changes in neuronal activity and can reveal significant changes from infancy to adulthood concomitant with brain maturation, making it a potential physiological marker of brain maturation and cognition. To investigate a promising deep learning tool for EEG classification, we applied the bidirectional long short-term memory (BLSTM) algorithm to analyze EEG data from the pediatric EEG laboratory of Taipei Tzu Chi Hospital. The trained BLSTM model was 86% accurate when identifying EEGs from young children (8 months-6 years) and adolescents (12-20 years). However, there was only a modest classification accuracy (69.3%) when categorizing EEG samples into three age groups (8 months-6 years, 6-12 years, and 12-20 years). For EEG samples from patients with intellectual disability, the prediction accuracy of the trained BLSTM model was 46.4%, which was significantly lower than its accuracy for EEGs from neurotypical patients, indicating that the individual's intelligence plays a major role in the age prediction. This study confirmed that scalp EEG can reflect brain maturation and the BLSTM algorithm is a feasible deep learning tool for the identification of cognitive age. The trained model can potentially be applied to clinical services as a supportive measurement of neurodevelopmental status.
Collapse
Affiliation(s)
- Shi-Bing Wong
- Department of Pediatrics, Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, New Taipei City, Taiwan.
- School of Medicine, Tzu Chi University, Hualien, Taiwan.
| | - Yu Tsao
- Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan
| | - Wen-Hsin Tsai
- Department of Pediatrics, Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, New Taipei City, Taiwan
- School of Medicine, Tzu Chi University, Hualien, Taiwan
| | - Tzong-Shi Wang
- School of Medicine, Tzu Chi University, Hualien, Taiwan
- Department of Psychiatry, Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, New Taipei City, Taiwan
| | - Hsin-Chi Wu
- School of Medicine, Tzu Chi University, Hualien, Taiwan
- Department of Physical Medicine and Rehabilitation, Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, New Taipei City, Taiwan
| | - Syu-Siang Wang
- Department of Electrical Engineering, Yuan Ze University, Taoyuan, Taiwan.
| |
Collapse
|
21
|
Tang H, Ma G, Zhang Y, Ye K, Guo L, Liu G, Huang Q, Wang Y, Ajilore O, Leow AD, Thompson PM, Huang H, Zhan L. A comprehensive survey of complex brain network representation. META-RADIOLOGY 2023; 1:100046. [PMID: 39830588 PMCID: PMC11741665 DOI: 10.1016/j.metrad.2023.100046] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/22/2025]
Abstract
Recent years have shown great merits in utilizing neuroimaging data to understand brain structural and functional changes, as well as its relationship to different neurodegenerative diseases and other clinical phenotypes. Brain networks, derived from different neuroimaging modalities, have attracted increasing attention due to their potential to gain system-level insights to characterize brain dynamics and abnormalities in neurological conditions. Traditional methods aim to pre-define multiple topological features of brain networks and relate these features to different clinical measures or demographical variables. With the enormous successes in deep learning techniques, graph learning methods have played significant roles in brain network analysis. In this survey, we first provide a brief overview of neuroimaging-derived brain networks. Then, we focus on presenting a comprehensive overview of both traditional methods and state-of-the-art deep-learning methods for brain network mining. Major models, and objectives of these methods are reviewed within this paper. Finally, we discuss several promising research directions in this field.
Collapse
Affiliation(s)
- Haoteng Tang
- Department of Computer Science, College of Engineering and Computer Science, University of Texas Rio Grande Valley, 1201 W University Dr, Edinburg, 78539, TX, USA
| | - Guixiang Ma
- Intel Labs, 2111 NE 25th Ave, Hillsboro, 97124, OR, USA
| | - Yanfu Zhang
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, 3700 O’Hara St., Pittsburgh, 15261, PA, USA
| | - Kai Ye
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, 3700 O’Hara St., Pittsburgh, 15261, PA, USA
| | - Lei Guo
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, 3700 O’Hara St., Pittsburgh, 15261, PA, USA
| | - Guodong Liu
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, 3700 O’Hara St., Pittsburgh, 15261, PA, USA
| | - Qi Huang
- Department of Radiology, Utah Center of Advanced Imaging, University of Utah, 729 Arapeen Drive, Salt Lake City, 84108, UT, USA
| | - Yalin Wang
- School of Computing and Augmented Intelligence, Arizona State University, 699 S Mill Ave., Tempe, 85281, AZ, USA
| | - Olusola Ajilore
- Department of Psychiatry, University of Illinois Chicago, 1601 W. Taylor St., Chicago, 60612, IL, USA
| | - Alex D. Leow
- Department of Psychiatry, University of Illinois Chicago, 1601 W. Taylor St., Chicago, 60612, IL, USA
| | - Paul M. Thompson
- Department of Neurology, University of Southern California, 2001 N. Soto St., Los Angeles, 90032, CA, USA
| | - Heng Huang
- Department of Computer Science, University of Maryland, 8125 Paint Branch Dr, College Park, 20742, MD, USA
| | - Liang Zhan
- Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh, 3700 O’Hara St., Pittsburgh, 15261, PA, USA
| |
Collapse
|