Towards the Interpretability of Deep Learning Models for Multi-modal Neuroimaging: Finding Structural Changes of the Ageing Brain

Integrated brain connectivity analysis with fMRI, DTI, and sMRI powered by interpretable graph neural networks

Multimodal neuroimaging data modeling has become a widely used approach but confronts considerable challenges due to their heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret diverse datasets within a cohesive analytical framework. In our research, we combine functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), and structural MRI (sMRI) for joint analysis. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging-derived features from multiple modalities - functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI - within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating an amalgamation of multimodal imaging data. This technique enhances interpretability at the connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved prediction accuracy and uncovers crucial anatomical features and neural connections, deepening our understanding of brain structure and function. This study not only advances multimodal neuroimaging analytics by offering a novel method for integrative analysis of diverse imaging modalities but also improves the understanding of intricate relationships between brain's structural and functional networks and cognitive development.

Estimating Chronological Age From the Electrical Activity of the Brain: How EEG-Age Can Be Used as a Marker of General Brain Functioning

With an aging global population, the number of older adults with age-related changes in the brain, including dementia, will continue to increase unless we can make progress in the early detection and treatment of such conditions. There is extensive literature on the effects of aging on the EEG, particularly a decline in the Peak Alpha Frequency (PAF), but here, in a reversal of convention, we used the EEG power-frequency spectrum to estimate chronological age. The motivation for this approach was that an individual's brain age might act as a proxy for their general brain functioning, whereby a discrepancy between chronological age and EEG age could prove clinically informative by implicating deleterious conditions. With a sample of sixty healthy adults, whose ages ranged from 20 to 78 years, and using multivariate methods to analyze the broad EEG spectrum (0.1-45 Hz), strong positive correlations between chronological age and EEG age emerged. Furthermore, EEG age was a more accurate estimate and accounted for more variance in chronological age than well-established PAF-based estimates of age, indicating that EEG age could be a more comprehensive measure of general brain functioning. We conclude that EEG age could become a biomarker for neural and cognitive integrity.

Illuminating Salient Contributions in Neuron Activation With Attribution Equilibrium

With the remarkable success of deep neural networks, there is a growing interest in research aimed at providing clear interpretations of their decision-making processes. In this paper, we introduce Attribution Equilibrium, a novel method to decompose output predictions into fine-grained attributions, balancing positive and negative relevance for clearer visualization of the evidence behind a network decision. We carefully analyze conventional approaches to decision explanation and present a different perspective on the conservation of evidence. We define the evidence as a gap between positive and negative influences among gradient-derived initial contribution maps. Then, we incorporate antagonistic elements and a user-defined criterion for the degree of positive attribution during propagation. Additionally, we consider the role of inactivated neurons in the propagation rule, thereby enhancing the discernment of less relevant elements such as the background. We conduct various assessments in a verified experimental environment with PASCAL VOC 2007, MS COCO 2014, and ImageNet datasets. The results demonstrate that our method outperforms existing attribution methods both qualitatively and quantitatively in identifying the key input features that influence model decisions.

ds-FCRN: three-dimensional dual-stream fully convolutional residual networks and transformer-based global-local feature learning for brain age prediction

The brain undergoes atrophy and cognitive decline with advancing age. The utilization of brain age prediction represents a pioneering methodology in the examination of brain aging. This study aims to develop a deep learning model with high predictive accuracy and interpretability for brain age prediction tasks. The gray matter (GM) density maps obtained from T1 MRI data of 16,377 healthy participants aged 45 to 82 years from the UKB database were included in this study (mean age, 64.27 ± 7.52 , 7811 men). We propose an innovative deep learning architecture for predicting brain age based on GM density maps. The architecture combines a 3D dual-stream fully convolutional residual network (ds-FCRN) with a Transformer-based global-local feature learning paradigm to enhance prediction accuracy. Moreover, we employed Shapley values to elucidate the influence of various brain regions on prediction precision. On a test set of 3,276 healthy subjects (mean age, 64.15 ± 7.45 , 1561 men), our 3D ds-FCRN model achieved a mean absolute error of 2.2 years in brain age prediction, outperforming existing models on the same dataset. The posterior interpretation revealed that the temporal lobe plays the most significant role in the brain age prediction process, while frontal lobe aging is associated with the greatest number of lifestyle factors. Our designed 3D ds-FCRN model achieved high predictive accuracy and high decision transparency. The brain age vectors constructed using Shapley values provided brain region-level insights into life factors associated with abnormal brain aging.

Attention over vulnerable brain regions associating cerebral palsy disorder and biological markers

INTRODUCTION

Cerebral palsy (CP) is a neurological disorder caused by cerebral ischemia and hypoxia during fetal brain development.Early intervention in CP favors medications and therapies; however, monitoring early brain development in children with CP is critical. It is essential to thoroughly examine brain-vulnerable regions associated with biological traits (BTs).Variations in BTs were evident in children with CP; however, it is critical to explore the BTs' impact on the brains of healthy controls (HC) and CP-disordered children.

OBJECTIVE

This study associates BTs with HC and CP children.This study investigates the neurodevelopment of HC and CP underlying BTs. This study establishes a benchmark for the association of BT with HC and CP children.

METHOD

The proposed AWG-Net is composed of customized spatial-channel (CSC) and multi-head self (MHA) attentions, where CSC blocks are incorporated at the first few stages, MHA at later stages, and cumulative-dense structures to propagate susceptible regions to deeper layers. The training samples include T1-w, T2-w, Flair, and Sag, annotated with age, gender, and weight.

RESULTS

The significant results for HC and CP are age (HC: MAE = 1.05, MCS10=85.63, R2=0.844; CP: MAE = 1.16, MCS10=84.79, R2=0.717), gender (HC: Acc = 82.98%, CP: Acc = 82.00%), and weight (HC: MAE = 4.65, MCS10=56.30, R2=0.78; CP: MAE = 2.85, MCS10=70.24, R2=0.82). Vulnerable regions for age are the cerebellar hemisphere, frontal, occipital, and parietal bones in HC and inconsistent in CP. HC and CP commonalities are in the frontal bone and cerebellar hemisphere with HC and discrepant in the occipital and temporal bones for CP. Similarly, gender differences are found for HC and CP.

CONCLUSION

Age and gender are marginally less affected by the brain regions vulnerable to CP than weight estimation. T1-w is appropriate for age, weight, and gender. The learned trends are different for HC and CP in brain development and gender while slightly different in the case of weight.

Glycemic control contributes to the neuroprotective effects of Mediterranean and green-Mediterranean diets on brain age: the DIRECT PLUS brain-magnetic resonance imaging randomized controlled trial

BACKGROUND

We recently reported that Mediterranean (MED) and green-MED diets significantly attenuated age-related brain atrophy by ∼50% within 18 mo.

OBJECTIVE

The objective of this study was to explore the contribution of specific diet-induced parameters to brain-volume deviation from chronologic age.

METHODS

A post hoc analysis of the 18-mo DIRECT PLUS trial, where participants were randomly assigned to the following groups: 1) healthy dietary guidelines, 2) MED diet, or 3) green-MED diet, high in polyphenols, and low in red meat. Both MED groups consumed 28 g walnuts/d (+440 mg/d polyphenols). The green-MED group further consumed green tea (3-4 cups/d) and Mankai green shake (Wolffia globosa aquatic plant) (+800 mg/d polyphenols). We collected blood samples through the intervention and followed brain structure volumes by magnetic resonance imaging (MRI). We used hippocampal occupancy (HOC) score (hippocampal and inferior lateral-ventricle volumes ratio) as a neurodegeneration marker and brain-age proxy. We applied multivariate linear regression models.

RESULTS

Of 284 participants [88% male; age = 51.1 y; body mass index = 31.2 kg/m2; hemoglobin A1c (HbA1c) = 5.48%; APOE-ε4 genotype = 15.7%], 224 completed the trial with eligible whole-brain MRIs. Individuals with higher HOC deviations (i.e., younger brain age) presented lower body weight [r = -0.204; 95% confidence interval (CI): -0.298, -0.101], waist circumference (r = -0.207; 95% CI: -0.310, -0.103), diastolic (r = -0.186; 95% CI: -0.304, -0.072), systolic blood pressure (r = -0.189; 95% CI: -0.308, -0.061), insulin (r = -0.099; 95% CI: -0.194, -0.004), and HbA1c (r = -0.164; 95% CI: -0.337, -0.006) levels. After 18 mo, greater changes in HOC deviations (i.e., brain-age decline attenuation) were independently associated with improved HbA1c (β = -0.254; 95% CI: -0.392, -0.117), HOMA-IR (β = -0.200; 95% CI: -0.346, -0.055), fasting glucose (β = -0.155; 95% CI: -0.293, -0.016), and c-reactive protein (β = -0.153; 95% CI: -0.296, -0.010). Improvement in diabetes status was associated with greater HOC deviation changes than either no change in diabetes status (0.010; 95% CI: 0.002, 0.019) or with an unfavorable change (0.012; 95% CI: 0.002, 0.023). A decline in HbA1c was further associated with greater deviation changes in the thalamus, caudate nucleus, and cerebellum (P < 0.05). Greater consumption of Mankai and green tea (green-MED diet components) were associated with greater HOC deviation changes beyond weight loss.

CONCLUSIONS

Glycemic control contributes to the neuroprotective effects of the MED and green-MED diets on brain age. Polyphenols-rich diet components as Mankai and green tea may contribute to a more youthful brain age. This trial was registered at clinicaltrials.gov at clinicaltrials.gov as NCT03020186.

Brain age prediction via cross-stratified ensemble learning

As an important biomarker of neural aging, the brain age reflects the integrity and health of the human brain. Accurate prediction of brain age could help to understand the underlying mechanism of neural aging. In this study, a cross-stratified ensemble learning algorithm with staking strategy was proposed to obtain brain age and the derived predicted age difference (PAD) using T1-weighted magnetic resonance imaging (MRI) data. The approach was characterized as by implementing two modules: one was three base learners of 3D-DenseNet, 3D-ResNeXt, 3D-Inception-v4; another was 14 secondary learners of liner regressions. To evaluate performance, our method was compared with single base learners, regular ensemble learning algorithms, and state-of-the-art (SOTA) methods. The results demonstrated that our proposed model outperformed others models, with three metrics of mean absolute error (MAE), root mean-squared error (RMSE), and coefficient of determination (R2) of 2.9405 years, 3.9458 years, and 0.9597, respectively. Furthermore, there existed significant differences in PAD among the three groups of normal control (NC), mild cognitive impairment (MCI) and Alzheimer's disease (AD), with an increased trend across NC, MCI, and AD. It was concluded that the proposed algorithm could be effectively used in computing brain aging and PAD, and offering potential for early diagnosis and assessment of normal brain aging and AD.

Machine Learning and Deep Learning Approaches in Lifespan Brain Age Prediction: A Comprehensive Review

The concept of 'brain age', derived from neuroimaging data, serves as a crucial biomarker reflecting cognitive vitality and neurodegenerative trajectories. In the past decade, machine learning (ML) and deep learning (DL) integration has transformed the field, providing advanced models for brain age estimation. However, achieving precise brain age prediction across all ages remains a significant analytical challenge. This comprehensive review scrutinizes advancements in ML- and DL-based brain age prediction, analyzing 52 peer-reviewed studies from 2020 to 2024. It assesses various model architectures, highlighting their effectiveness and nuances in lifespan brain age studies. By comparing ML and DL, strengths in forecasting and methodological limitations are revealed. Finally, key findings from the reviewed articles are summarized and a number of major issues related to ML/DL-based lifespan brain age prediction are discussed. Through this study, we aim at the synthesis of the current state of brain age prediction, emphasizing both advancements and persistent challenges, guiding future research, technological advancements, and improving early intervention strategies for neurodegenerative diseases.

Markers of Biological Brain Aging Mediate Effects of Vascular Risk Factors on Cognitive and Motor Functions: A Multivariate Imaging Analysis of 40,579 Individuals

The increasing global life expectancy brings forth challenges associated with age-related cognitive and motor declines. To better understand underlying mechanisms, we investigated the connection between markers of biological brain aging based on magnetic resonance imaging (MRI), cognitive and motor performance, as well as modifiable vascular risk factors, using a large-scale neuroimaging analysis in 40,579 individuals of the population-based UK Biobank and Hamburg City Health Study. Employing partial least squares correlation analysis (PLS), we investigated multivariate associative effects between three imaging markers of biological brain aging - relative brain age, white matter hyperintensities of presumed vascular origin, and peak-width of skeletonized mean diffusivity - and multi-domain cognitive test performances and motor test results. The PLS identified a latent dimension linking higher markers of biological brain aging to poorer cognitive and motor performances, accounting for 94.7% of shared variance. Furthermore, a mediation analysis revealed that biological brain aging mediated the relationship of vascular risk factors - including hypertension, glucose, obesity, and smoking - to cognitive and motor function. These results were replicable in both cohorts. By integrating multi-domain data with a comprehensive methodological approach, our study contributes evidence of a direct association between vascular health, biological brain aging, and functional cognitive as well as motor performance, emphasizing the need for early and targeted preventive strategies to maintain cognitive and motor independence in aging populations.

Structural indices of brain aging in methamphetamine use disorder

BACKGROUND

Methamphetamine use is surging globally. It has been linked to premature stroke, Parkinsonism, and dementia, suggesting that it may accelerate brain aging.

METHODS

We performed a retrospective study to determine if structural indices of brain aging were more prevalent prior to old age (26 - 54 years) in individuals with Methamphetamine Use Disorder (MUD), who were in early abstinence (M ± SD = 22.1 ± 25.6 days) than in healthy control (HC) participants. We compared T1-weighted MRI brain scans in age- and sex-matched groups (n = 89/group) on three structural features of brain aging: the brain volume/cerebrospinal fluid (BV/CSF) index, volume of white matter hypointensities/lesions, and choroid plexus volume.

RESULTS

The MUD group had a lower mean BV/CSF index and larger volumes of white matter hypointensities and choroid plexus (p-values < 0.01). Regression analyses showed significant age-by-group effects, indicating different age trajectories of the BV/CSF index and choroid plexus volume, consistent with abnormal global brain atrophy and choroid plexus pathology in the MUD group. Significant age and group main effects reflected a larger volume of white matter hypointensities for older participants across groups and for the MUD group irrespective of age. None of the three measures of brain aging correlated significantly with recent use or duration of recent abstinence from methamphetamine.

CONCLUSIONS

Premature brain pathology, which may reflect cerebrovascular damage and dysfunction of the choroid plexus, occurs in people with MUD. Such pathology may affect cognition and thereby efficacy of behavioral treatments for MUD.

Benchmarking the influence of pre-training on explanation performance in MR image classification

Convolutional Neural Networks (CNNs) are frequently and successfully used in medical prediction tasks. They are often used in combination with transfer learning, leading to improved performance when training data for the task are scarce. The resulting models are highly complex and typically do not provide any insight into their predictive mechanisms, motivating the field of "explainable" artificial intelligence (XAI). However, previous studies have rarely quantitatively evaluated the "explanation performance" of XAI methods against ground-truth data, and transfer learning and its influence on objective measures of explanation performance has not been investigated. Here, we propose a benchmark dataset that allows for quantifying explanation performance in a realistic magnetic resonance imaging (MRI) classification task. We employ this benchmark to understand the influence of transfer learning on the quality of explanations. Experimental results show that popular XAI methods applied to the same underlying model differ vastly in performance, even when considering only correctly classified examples. We further observe that explanation performance strongly depends on the task used for pre-training and the number of CNN layers pre-trained. These results hold after correcting for a substantial correlation between explanation and classification performance.

Assessing Machine Learning Models for Predicting Age with Intracranial Vessel Tortuosity and Thickness Information

This study aimed to develop and validate machine learning (ML) models that predict age using intracranial vessels' tortuosity and diameter features derived from magnetic resonance angiography (MRA) data. A total of 171 subjects' three-dimensional (3D) time-of-flight MRA image data were considered for analysis. After annotations of two endpoints in each arterial segment, tortuosity features such as the sum of the angle metrics, triangular index, relative length, and product of the angle distance, as well as the vessels' diameter features, were extracted and used to train and validate the ML models for age prediction. Features extracted from the right and left internal carotid arteries (ICA) and basilar arteries were considered as the inputs to train and validate six ML regression models with a four-fold cross validation. The random forest regression model resulted in the lowest root mean square error of 14.9 years and the highest average coefficient of determination of 0.186. The linear regression model showed the lowest average mean absolute percentage error (MAPE) and the highest average Pearson correlation coefficient (0.532). The mean diameter of the right ICA vessel segment was the most important feature contributing to prediction of age in two out of the four regression models considered. An ML of tortuosity descriptors and diameter features extracted from MRA data showed a modest correlation between real age and ML-predicted age. Further studies are warranted for the assessment of the model's age predictions in patients with intracranial vessel diseases.

Genetic heart-brain connections

Multiorgan imaging unveils the intertwined nature of the human heart and brain.

A systematic review of multimodal brain age studies: Uncovering a divergence between model accuracy and utility

Brain aging is a complex, multifaceted process that can be challenging to model in ways that are accurate and clinically useful. One of the most common approaches has been to apply machine learning to neuroimaging data with the goal of predicting age in a data-driven manner. Building on initial brain age studies that were derived solely from T1-weighted scans (i.e., unimodal), recent studies have incorporated features across multiple imaging modalities (i.e., "multimodal"). In this systematic review, we show that unimodal and multimodal models have distinct advantages. Multimodal models are the most accurate and sensitive to differences in chronic brain disorders. In contrast, unimodal models from functional magnetic resonance imaging were most sensitive to differences across a broad array of phenotypes. Altogether, multimodal imaging has provided us valuable insight for improving the accuracy of brain age models, but there is still much untapped potential with regard to achieving widespread clinical utility.

Impact of obesity and diet on brain structure and function: a gut-brain-body crosstalk

Most societies witness an ever increasing prevalence of both obesity and dementia, a scenario related to often underestimated individual and public health burden. Overnutrition and weight gain have been linked with abnormal functionality of homoeostasis brain networks and changes in higher cognitive functions such as reward evaluation, executive functions and learning and memory. In parallel, evidence has accumulated that modifiable factors such as obesity and diet impact the gut-brain axis and modulate brain health and cognition through various pathways. Using neuroimaging data from epidemiological studies and randomised clinical trials, we aim to shed light on the underlying mechanisms and to determine both determinants and consequences of obesity and diet at the level of human brain structure and function. We analysed multimodal 3T MRI of about 2600 randomly selected adults (47 % female, 18-80 years of age, BMI 18-47 kg/m²) of the LIFE-Adult study, a deeply phenotyped population-based cohort. In addition, brain MRI data of controlled intervention studies on weight loss and healthy diets acquired in lean, overweight and obese participants may help to understand the role of the gut-brain axis in food craving and cognitive ageing. We find that higher BMI and visceral fat accumulation correlate with accelerated brain age, microstructure of the hypothalamus, lower thickness and connectivity in default mode- and reward-related areas, as well as with subtle grey matter atrophy and white matter lesion load in non-demented individuals. Mediation analyses indicated that higher visceral fat affects brain tissue through systemic low-grade inflammation, and that obesity-related regional changes translate into cognitive disadvantages. Considering longitudinal studies, some, but not all data indicate beneficial effects of weight loss and healthy diets such as plant-based nutrients and dietary patterns on brain ageing and cognition. Confounding effects of concurrent changes in other lifestyle factors or false positives might help to explain these findings. Therefore a more holistic intervention approach, along with open science tools such as data and code sharing, in-depth pre-registration and pooling of data could help to overcome these limitations. In addition, as higher BMI relates to increased head micro-movements during MRI, and as head motion in turn systematically induces image artefacts, future studies need to rigorously control for head motion during MRI to enable valid neuroimaging results. In sum, our results support the view that overweight and obesity are intertwined with markers of brain health in the general population, and that weight loss and plant-based diets may help to promote brain plasticity. Meta-analyses and longitudinal cohort studies are underway to further differentiate causation from correlation in obesity- and nutrition-brain research.