Global-Local Transformer for Brain Age Estimation

Calcium deblooming in coronary computed tomography angiography via semantic-oriented generative adversarial network

Calcium blooming artifact produced by calcified plaque in coronary computed tomography angiography (CCTA) is a significant contributor to false-positive results for radiologists. Most previous research focused on general noise reduction of CT images, while performance was limited when facing the blooming artifact. To address this problem, we designed an automated and robust semantics-oriented adversarial network that fully exploits the calcified plaques as semantic regions in the CCTA. The semantic features were extracted using a feature extraction module and implemented through a global-local fusion module, a generator with a semantic similarity module, and a matrix discriminator. The effectiveness of our network was validated both on a virtual and a clinical dataset. The clinical dataset consists of 372 CCTA and corresponding coronary angiogram (CAG) results, with the assistance of two cardiac radiologists (with 10 and 21 years of experience) for clinical evaluation. The proposed method effectively reduces artifacts for three major coronary arteries and significantly improves the specificity and positive predictive value for the diagnosis of coronary stenosis.

Enhancing brain age estimation under uncertainty: A spectral-normalized neural gaussian process approach utilizing 2.5D slicing

Brain age gap, the difference between estimated brain age and chronological age via magnetic resonance imaging, has emerged as a pivotal biomarker in the detection of brain abnormalities. While deep learning is accurate in estimating brain age, the absence of uncertainty estimation may pose risks in clinical use. Moreover, current 3D brain age models are intricate, and using 2D slices hinders comprehensive dimensional data integration. Here, we introduced Spectral-normalized Neural Gaussian Process (SNGP) accompanied by 2.5D slice approach for seamless uncertainty integration in a single network with low computational expenses, and extra dimensional data integration without added model complexity. Subsequently, we compared different deep learning methods for estimating brain age uncertainty via the Pearson correlation coefficient, a metric that helps circumvent systematic underestimation of uncertainty during training. SNGP shows excellent uncertainty estimation and generalization on a dataset of 11 public datasets (N = 6327), with competitive predictive performance (MAE=2.95). Besides, SNGP demonstrates superior generalization performance (MAE=3.47) on an independent validation set (N = 301). Additionally, we conducted five controlled experiments to validate our method. Firstly, uncertainty adjustment in brain age estimation improved the detection of accelerated brain aging in adolescents with ADHD, with a 38% increase in effect size after adjustment. Secondly, the SNGP model exhibited OOD detection capabilities, showing significant differences in uncertainty across Asian and non-Asian datasets. Thirdly, the performance of DenseNet as a backbone for SNGP was slightly better than ResNeXt, attributed to DenseNet's feature reuse capability, with robust generalization on an independent validation set. Fourthly, site effect harmonization led to a decline in model performance, consistent with previous studies. Finally, the 2.5D slice approach significantly outperformed 2D methods, improving model performance without increasing network complexity. In conclusion, we present a cost-effective method for estimating brain age with uncertainty, utilizing 2.5D slicing for enhanced performance, showcasing promise for clinical applications.

Vision transformer and deep learning based weighted ensemble model for automated spine fracture type identification with GAN generated CT images

The most common causes of spine fractures, or vertebral column fractures (VCF), are traumas like falls, injuries from sports, or accidents. CT scans are affordable and effective at detecting VCF types in an accurate manner. VCF type identification in cervical, thoracic, and lumbar (C3-L5) regions is limited and sensitive to inter-observer variability. To solve this problem, this work introduces an autonomous approach for identifying VCF type by developing a novel ensemble model of Vision Transformers (ViT) and best-performing deep learning (DL) models. It assists orthopaedicians in easy and early identification of VCF types. The performance of numerous fine-tuned DL architectures, including VGG16, ResNet50, and DenseNet121, was investigated, and an ensemble classification model was developed to identify the best-performing combination of DL models. A ViT model is also trained to identify VCF. Later, the best-performing DL models and ViT were fused by weighted average technique for type identification. To overcome data limitations, an extended Deep Convolutional Generative Adversarial Network (DCGAN) and Progressive Growing Generative Adversarial Network (PGGAN) were developed. The VGG16-ResNet50-ViT ensemble model outperformed all ensemble models and got an accuracy of 89.98%. Extended DCGAN and PGGAN augmentation increased the accuracy of type identification to 90.28% and 93.68%, respectively. This demonstrates efficacy of PGGANs in augmenting VCF images. The study emphasizes the distinctive contributions of the ResNet50, VGG16, and ViT models in feature extraction, generalization, and global shape-based pattern capturing in VCF type identification. CT scans collected from a tertiary care hospital are used to validate these models.

MRI-based habitat radiomics combined with vision transformer for identifying vulnerable intracranial atherosclerotic plaques and predicting stroke events: a multicenter, retrospective study

Background

Accurate identification of high-risk vulnerable plaques and assessment of stroke risk are crucial for clinical decision-making, yet reliable non-invasive predictive tools are currently lacking. This study aimed to develop an artificial intelligence model based on high-resolution vessel wall imaging (HR-VWI) to assist in the identification of vulnerable plaques and prediction of stroke recurrence risk in patients with symptomatic intracranial atherosclerotic stenosis (sICAS).

Methods

Between June 2018 and June 2024, a retrospective collection of HR-VWI images from 1806 plaques in 726 sICAS patients across four medical institutions was conducted. K-means clustering was applied to the T1-weighted imaging (T1WI) and T1-weighted imaging with contrast enhancement (T1CE) sequences. Following feature extraction and selection, radiomic models and habitat models were constructed. Additionally, the Vision Transformer (ViT) architecture was utilized for HR-VWI image analysis to build a deep learning model. A stacking fusion strategy was employed to integrate the habitat model and ViT model, enabling effective identification of high-risk vulnerable plaques in the intracranial region and prediction of stroke recurrence risk. Model performance was evaluated using receiver operating characteristic (ROC) curves, and model comparisons were conducted using the DeLong test. Furthermore, decision curve analysis and calibration curves were utilized to assess the practicality and clinical value of the model.

Findings

The fused Habitat + ViT model exhibited excellent performance in both the validation and test sets. In the validation set, the model achieved an area under the curve (AUC) of 0.949 (95% CI: 0.927-0.969), with a sensitivity of 0.879 (95% CI: 0.840-0.945), a specificity of 0.905 (95% CI: 0.842-0.949), and an accuracy of 0.897 (95% CI: 0.870-0.926). In the test set, the AUC increased to 0.960 (95% CI: 0.941-0.973), with specificity rising to 0.963 and an accuracy of 0.885 (95% CI: 0.857-0.913). The DeLong test revealed statistically significant differences in AUC between the fused model and the single-modal models (test set, vs. ViT p = 0.000; vs. Habitat p = 0.000) Cox regression analysis showed that the Habitat + ViT index, based on the prediction probability of the Habitat + ViT model, was an independent predictor of stroke recurrence (HR: 2.07; 95% CI: 1.12-3.81), with significant predictive power for stroke events at multiple time points. Specifically, measured by AUC values, the model's predictive performance at 1, 2, 3, and 4 years was 0.751 (95% CI: 0.679-0.823), 0.820 (95% CI: 0.760-0.876), 0.815 (95% CI: 0.753-0.877), and 0.780 (95% CI: 0.680-0.873), respectively.

Interpretation

The integrated Habitat + ViT model based on HR-VWI demonstrated superior performance in identifying high-risk vulnerable plaques in sICAS patients and predicting stroke recurrence risk, providing valuable support for clinical decision-making.

Funding

This study was supported by the National Natural Science Foundation of China (grant 82204933). Henan Key Laboratory of Neurorestoratology (HNSJXF-2021-004), 2019 Joint Construction Project of Henan Provincial Health Committee and Ministry of Health (SB201901061), and the Xin Xiang City Acute Ischemic Stroke Precision Prevention and Treatment Key Laboratory.

Genetically supported targets and drug repurposing for brain aging: A systematic study in the UK Biobank

Brain age gap (BAG), the deviation between estimated brain age and chronological age, is a promising marker of brain health. However, the genetic architecture and reliable targets for brain aging remains poorly understood. In this study, we estimate magnetic resonance imaging (MRI)-based brain age using deep learning models trained on the UK Biobank and validated with three external datasets. A genome-wide association study for BAG identified two unreported loci and seven previously reported loci. By integrating Mendelian Randomization (MR) and colocalization analysis on eQTL and pQTL data, we prioritized seven genetically supported druggable genes, including MAPT, TNFSF12, GZMB, SIRPB1, GNLY, NMB, and C1RL, as promising targets for brain aging. We rediscovered 13 potential drugs with evidence from clinical trials of aging and prioritized several drugs with strong genetic support. Our study provides insights into the genetic basis of brain aging, potentially facilitating drug development for brain aging to extend the health span.

Heterogeneous Graph Representation Learning Framework for Resting-State Functional Connectivity Analysis

Brain functional connectivity analysis is important for understanding brain development and brain disorders. Recent studies have suggested that the variations of functional connectivity among multiple subnetworks are closely related to the development of diseases. However, the existing works failed to sufficiently capture the complex correlation patterns among the subnetworks and ignored the learning of heterogeneous structural information across the subnetworks. To address these issues, we formulate a new paradigm for constructing and analyzing high-order heterogeneous functional brain networks via meta-paths and propose a Heterogeneous Graph representation Learning framework (BrainHGL). Our framework consists of three key aspects: 1) Meta-path encoding for capturing rich heterogeneous topological information, 2) Meta-path interaction for exploiting complex association patterns among subnetworks and 3) Meta-path aggregation for better meta-path fusion. To the best of our knowledge, we are the first to formulate the heterogeneous brain networks for better exploiting the relationship between the subnetwork interactions and the mental disease We evaluate BrainHGL on the private center Nanjing Medical University dataset (center NMU) and the public Autism Brain Imaging Data Exchange (ABIDE) dataset. We demonstrate the effectiveness of the proposed model across various disease classification tasks, including major depression disorder (MDD), bipolar disorder (BD) and autism spectrum disorder (ASD) diagnoses. In addition, our model provides deeper insights into disease interpretability, including the critical brain subnetwork connectivities, brain regions and functional pathways. We also identified disease subtypes consistent with previous neuroscientific studies by our model, which benefits the disease identification performance. The code is available at https://github.com/IntelliDAL/Graph/BrainHGL.

Principles of artificial intelligence in radiooncology

PURPOSE

In the rapidly expanding field of artificial intelligence (AI) there is a wealth of literature detailing the myriad applications of AI, particularly in the realm of deep learning. However, a review that elucidates the technical principles of deep learning as relevant to radiation oncology in an easily understandable manner is still notably lacking. This paper aims to fill this gap by providing a comprehensive guide to the principles of deep learning that is specifically tailored toward radiation oncology.

METHODS

In light of the extensive variety of AI methodologies, this review selectively concentrates on the specific domain of deep learning. It emphasizes the principal categories of deep learning models and delineates the methodologies for training these models effectively.

RESULTS

This review initially delineates the distinctions between AI and deep learning as well as between supervised and unsupervised learning. Subsequently, it elucidates the fundamental principles of major deep learning models, encompassing multilayer perceptrons (MLPs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, generative adversarial networks (GANs), diffusion-based generative models, and reinforcement learning. For each category, it presents representative networks alongside their specific applications in radiation oncology. Moreover, the review outlines critical factors essential for training deep learning models, such as data preprocessing, loss functions, optimizers, and other pivotal training parameters including learning rate and batch size.

CONCLUSION

This review provides a comprehensive overview of deep learning principles tailored toward radiation oncology. It aims to enhance the understanding of AI-based research and software applications, thereby bridging the gap between complex technological concepts and clinical practice in radiation oncology.

Transformer-based approaches for neuroimaging: an in-depth review of their role in classification and regression tasks

In the ever-evolving landscape of deep learning (DL), the transformer model emerges as a formidable neural network architecture, gaining significant traction in neuroimaging-based classification and regression tasks. This paper presents an extensive examination of transformer's application in neuroimaging, surveying recent literature to elucidate its current status and research advancement. Commencing with an exposition on the fundamental principles and structures of the transformer model and its variants, this review navigates through the methodologies and experimental findings pertaining to their utilization in neuroimage classification and regression tasks. We highlight the transformer model's prowess in neuroimaging, showcasing its exceptional performance in classification endeavors while also showcasing its burgeoning potential in regression tasks. Concluding with an assessment of prevailing challenges and future trajectories, this paper proffers insights into prospective research directions. By elucidating the current landscape and envisaging future trends, this review enhances comprehension of transformer's role in neuroimaging tasks, furnishing valuable guidance for further inquiry.

ds-FCRN: three-dimensional dual-stream fully convolutional residual networks and transformer-based global-local feature learning for brain age prediction

The brain undergoes atrophy and cognitive decline with advancing age. The utilization of brain age prediction represents a pioneering methodology in the examination of brain aging. This study aims to develop a deep learning model with high predictive accuracy and interpretability for brain age prediction tasks. The gray matter (GM) density maps obtained from T1 MRI data of 16,377 healthy participants aged 45 to 82 years from the UKB database were included in this study (mean age, 64.27 ± 7.52 , 7811 men). We propose an innovative deep learning architecture for predicting brain age based on GM density maps. The architecture combines a 3D dual-stream fully convolutional residual network (ds-FCRN) with a Transformer-based global-local feature learning paradigm to enhance prediction accuracy. Moreover, we employed Shapley values to elucidate the influence of various brain regions on prediction precision. On a test set of 3,276 healthy subjects (mean age, 64.15 ± 7.45 , 1561 men), our 3D ds-FCRN model achieved a mean absolute error of 2.2 years in brain age prediction, outperforming existing models on the same dataset. The posterior interpretation revealed that the temporal lobe plays the most significant role in the brain age prediction process, while frontal lobe aging is associated with the greatest number of lifestyle factors. Our designed 3D ds-FCRN model achieved high predictive accuracy and high decision transparency. The brain age vectors constructed using Shapley values provided brain region-level insights into life factors associated with abnormal brain aging.

A review of artificial intelligence-based brain age estimation and its applications for related diseases

The study of brain age has emerged over the past decade, aiming to estimate a person's age based on brain imaging scans. Ideally, predicted brain age should match chronological age in healthy individuals. However, brain structure and function change in the presence of brain-related diseases. Consequently, brain age also changes in affected individuals, making the brain age gap (BAG)-the difference between brain age and chronological age-a potential biomarker for brain health, early screening, and identifying age-related cognitive decline and disorders. With the recent successes of artificial intelligence in healthcare, it is essential to track the latest advancements and highlight promising directions. This review paper presents recent machine learning techniques used in brain age estimation (BAE) studies. Typically, BAE models involve developing a machine learning regression model to capture age-related variations in brain structure from imaging scans of healthy individuals and automatically predict brain age for new subjects. The process also involves estimating BAG as a measure of brain health. While we discuss recent clinical applications of BAE methods, we also review studies of biological age that can be integrated into BAE research. Finally, we point out the current limitations of BAE's studies.

BOston Neonatal Brain Injury Data for Hypoxic Ischemic Encephalopathy (BONBID-HIE): I. MRI and Lesion Labeling

Hypoxic ischemic encephalopathy (HIE) is a brain injury that occurs in 1 ~ 5/1000 term neonates. Accurate identification and segmentation of HIE-related lesions in neonatal brain magnetic resonance images (MRIs) is the first step toward identifying high-risk patients, understanding neurological symptoms, evaluating treatment effects, and predicting outcomes. We release the first public dataset containing neonatal brain diffusion MRI and expert annotation of lesions from 133 patients diagnosed with HIE. HIE-related lesions in brain MRI are often diffuse (i.e., multi-focal), and small (over half the patients in our data having lesions occupying <1% of the brain volume (including ventricles)). Segmentation for HIE MRI data is remarkably different from, and arguably more challenging than, other segmentation tasks such as brain tumors with focal and relatively large lesions. We hope that this dataset can help fuel the development of MRI lesion segmentation methods for HIE and small diffuse lesions in general.

Predicting brain age with global-local attention network from multimodal neuroimaging data: Accuracy, generalizability, and behavioral associations

Brain age, an emerging biomarker for brain diseases and aging, is typically predicted using single-modality T1-weighted structural MRI data. This study investigates the benefits of integrating structural MRI with diffusion MRI to enhance brain age prediction. We propose an attention-based deep learning model that fuses global-context information from structural MRI with local details from diffusion metrics. The model was evaluated using two large datasets: the Human Connectome Project (HCP, n = 1064, age 22-37) and the Cambridge Center for Aging and Neuroscience (Cam-CAN, n = 639, age 18-88). It was tested for generalizability and robustness on three independent datasets (n = 546, age 20-86), reproducibility on a test-retest dataset (n = 44, age 22-35), and longitudinal consistency (n = 129, age 46-92). We also examined the relationship between predicted brain age and behavioral measures. Results showed that the multimodal model improved prediction accuracy, achieving mean absolute errors (MAEs) of 2.44 years in the HCP dataset (sagittal plane) and 4.36 years in the Cam-CAN dataset (axial plane). The corresponding R2 values were 0.258 and 0.914, respectively, reflecting the model's ability to explain variance in the predictions across both datasets. Compared to single-modality models, the multimodal approach showed better generalization, reducing MAEs by 10-76 % and enhancing robustness by 22-82 %. While the multimodal model exhibited superior reproducibility, the sMRI model showed slightly better longitudinal consistency. Importantly, the multimodal model revealed unique associations between predicted brain age and behavioral measures, such as walking endurance and loneliness in the HCP dataset, which were not detected with chronological age alone. In the Cam-CAN dataset, brain age and chronological age exhibited similar correlations with behavioral measures. By integrating sMRI and dMRI through an attention-based model, our proposed approach enhances predictive accuracy and provides deeper insights into the relationship between brain aging and behavior.

Signed Curvature Graph Representation Learning of Brain Networks for Brain Age Estimation

Graph Neural Networks (GNNs) play a pivotal role in learning representations of brain networks for estimating brain age. However, the over-squashing impedes interactions between long-range nodes, hindering the ability of message-passing mechanism-based GNNs to learn the topological structure of brain networks. Graph rewiring methods and curvature GNNs have been proposed to alleviate over-squashing. However, most graph rewiring methods overlook node features and curvature GNNs neglect the geometric properties of signed curvature. In this study, a Signed Curvature GNN (SCGNN) was proposed to rewire the graph based on node features and curvature, and learn the representation of signed curvature. First, a Mutual Information Ollivier-Ricci Flow (MORF) was proposed to add connections in the neighborhood of edge with the minimal negative curvature based on the maximum mutual information between node features, improving the efficiency of information interaction between nodes. Then, a Signed Curvature Convolution (SCC) was proposed to aggregate node features based on positive and negative curvature, facilitating the model's ability to capture the complex topological structures of brain networks. Additionally, an Ollivier-Ricci Gradient Pooling (ORG-Pooling) was proposed to select the key nodes and topology structures by curvature gradient and attention mechanism, accurately obtaining the global representation for brain age estimation. Experiments conducted on six public datasets with structural magnetic resonance imaging (sMRI), spanning ages from 18 to 91 years, validate that our method achieves promising performance compared with existing methods. Furthermore, we employed the gaps between brain age and chronological age for identifying Alzheimer's Disease (AD), yielding the best classification performance.

Multi-Label Chest X-Ray Image Classification With Single Positive Labels

Deep learning approaches for multi-label Chest X-ray (CXR) images classification usually require large-scale datasets. However, acquiring such datasets with full annotations is costly, time-consuming, and prone to noisy labels. Therefore, we introduce a weakly supervised learning problem called Single Positive Multi-label Learning (SPML) into CXR images classification (abbreviated as SPML-CXR), in which only one positive label is annotated per image. A simple solution to SPML-CXR problem is to assume that all the unannotated pathological labels are negative, however, it might introduce false negative labels and decrease the model performance. To this end, we present a Multi-level Pseudo-label Consistency (MPC) framework for SPML-CXR. First, inspired by the pseudo-labeling and consistency regularization in semi-supervised learning, we construct a weak-to-strong consistency framework, where the model prediction on weakly-augmented image is treated as the pseudo label for supervising the model prediction on a strongly-augmented version of the same image, and define an Image-level Perturbation-based Consistency (IPC) regularization to recover the potential mislabeled positive labels. Besides, we incorporate Random Elastic Deformation (RED) as an additional strong augmentation to enhance the perturbation. Second, aiming to expand the perturbation space, we design a perturbation stream to the consistency framework at the feature-level and introduce a Feature-level Perturbation-based Consistency (FPC) regularization as a supplement. Third, we design a Transformer-based encoder module to explore the sample relationship within each mini-batch by a Batch-level Transformer-based Correlation (BTC) regularization. Extensive experiments on the CheXpert and MIMIC-CXR datasets have shown the effectiveness of our MPC framework for solving the SPML-CXR problem.

Inferring neurocognition using artificial intelligence on brain MRIs

Brain magnetic resonance imaging (MRI) offers a unique lens to study neuroanatomic support of human neurocognition. A core mystery is the MRI explanation of individual differences in neurocognition and its manifestation in intelligence. The past four decades have seen great advancement in studying this century-long mystery, but the sample size and population-level studies limit the explanation at the individual level. The recent rise of big data and artificial intelligence offers novel opportunities. Yet, data sources, harmonization, study design, and interpretation must be carefully considered. This review aims to summarize past work, discuss rising opportunities and challenges, and facilitate further investigations on artificial intelligence inferring human neurocognition.

Deep learning of structural MRI predicts fluid, crystallized, and general intelligence

Can brain structure predict human intelligence? T1-weighted structural brain magnetic resonance images (sMRI) have been correlated with intelligence. However, the population-level association does not fully account for individual variability in intelligence. To address this, studies have emerged recently to predict individual subject's intelligence or neurocognitive scores. However, they are mostly on predicting fluid intelligence (the ability to solve new problems). Studies are lacking to predict crystallized intelligence (the ability to accumulate knowledge) or general intelligence (fluid and crystallized intelligence combined). This study tests whether deep learning of sMRI can predict an individual subject's verbal, comprehensive, and full-scale intelligence quotients (VIQ, PIQ, and FSIQ), which reflect fluid and crystallized intelligence. We performed a comprehensive set of 432 experiments, using different input image channels, six deep learning models, and two outcome settings, in 850 healthy and autistic subjects 6-64 years of age. Our findings indicate a statistically significant potential of T1-weighted sMRI in predicting intelligence, with a Pearson correlation exceeding 0.21 (p < 0.001). Interestingly, we observed that an increase in the complexity of deep learning models does not necessarily translate to higher accuracy in intelligence prediction. The interpretations of our 2D and 3D CNNs, based on GradCAM, align well with the Parieto-Frontal Integration Theory (P-FIT), reinforcing the theory's suggestion that human intelligence is a result of interactions among various brain regions, including the occipital, temporal, parietal, and frontal lobes. These promising results invite further studies and open new questions in the field.

A perspective on brain-age estimation and its clinical promise

Brain-age estimation has gained increased attention in the neuroscientific community owing to its potential use as a biomarker of brain health. The difference between estimated and chronological age based on neuroimaging data enables a unique perspective on brain development and aging, with multiple open questions still remaining in the brain-age research field. This Perspective presents an overview of current advancements in the field and envisions the future evolution of the brain-age framework before its potential deployment in hospital settings.

Accelerating denoising diffusion probabilistic model via truncated inverse processes for medical image segmentation

Medical image segmentation demands precise accuracy and the capability to assess segmentation uncertainty for informed clinical decision-making. Denoising Diffusion Probability Models (DDPMs), with their advancements in image generation, can treat segmentation as a conditional generation task, providing accurate segmentation and uncertainty estimation. However, current DDPMs used in medical image segmentation suffer from low inference efficiency and prediction errors caused by excessive noise at the end of the forward process. To address this issue, we propose an accelerated denoising diffusion probabilistic model via truncated inverse processes (ADDPM) that is specifically designed for medical image segmentation. The inverse process of ADDPM starts from a non-Gaussian distribution and terminates early once a prediction with relatively low noise is obtained after multiple iterations of denoising. We employ a separate powerful segmentation network to obtain pre-segmentation and construct the non-Gaussian distribution of the segmentation based on the forward diffusion rule. By further adopting a separate denoising network, the final segmentation can be obtained with just one denoising step from the predictions with low noise. ADDPM greatly reduces the number of denoising steps to approximately one-tenth of that in vanilla DDPMs. Our experiments on four segmentation tasks demonstrate that ADDPM outperforms both vanilla DDPMs and existing representative accelerating DDPMs methods. Moreover, ADDPM can be easily integrated with existing advanced segmentation models to improve segmentation performance and provide uncertainty estimation. Implementation code: https://github.com/Guoxt/ADDPM.

Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning

Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

Ensemble Vision Transformer for Dementia Diagnosis

In recent years, deep learning has gained momentum in computer-aided Alzheimer's Disease (AD) diagnosis. This study introduces a novel approach, Monte Carlo Ensemble Vision Transformer (MC-ViT), which develops an ensemble approach with Vision transformer (ViT). Instead of using traditional ensemble methods that deploy multiple learners, our approach employs a single vision transformer learner. By harnessing Monte Carlo sampling, this method produces a broad spectrum of classification decisions, enhancing the MC-ViT performance. This novel technique adeptly overcomes the limitation of 3D patch convolutional neural networks that only characterize partial of the whole brain anatomy, paving the way for a neural network adept at discerning 3D inter-feature correlations. Evaluations using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset with 7199 scans and Open Access Series of Imaging Studies-3 (OASIS-3) with 1992 scans showcased its performance. With minimal preprocessing, our approach achieved an impressive 90% accuracy in AD classification, surpassing both 2D-slice CNNs and 3D CNNs.

Brain age prediction using interpretable multi-feature-based convolutional neural network in mild traumatic brain injury

BACKGROUND

Convolutional neural network (CNN) can capture the structural features changes of brain aging based on MRI, thus predict brain age in healthy individuals accurately. However, most studies use single feature to predict brain age in healthy individuals, ignoring adding information from multiple sources and the changes in brain aging patterns after mild traumatic brain injury (mTBI) were still unclear.

METHODS

Here, we leveraged the structural data from a large, heterogeneous dataset (N = 1464) to implement an interpretable 3D combined CNN model for brain-age prediction. In addition, we also built an atlas-based occlusion analysis scheme with a fine-grained human Brainnetome Atlas to reveal the age-sstratified contributed brain regions for brain-age prediction in healthy controls (HCs) and mTBI patients. The correlations between brain predicted age gaps (brain-PAG) following mTBI and individual's cognitive impairment, as well as the level of plasma neurofilament light were also examined.

RESULTS

Our model utilized multiple 3D features derived from T1w data as inputs, and reduced the mean absolute error (MAE) of age prediction to 3.08 years and improved Pearson's r to 0.97 on 154 HCs. The strong generalizability of our model was also validated across different centers. Regions contributing the most significantly to brain age prediction were the caudate and thalamus for HCs and patients with mTBI, and the contributive regions were mostly located in the subcortical areas throughout the adult lifespan. The left hemisphere was confirmed to contribute more in brain age prediction throughout the adult lifespan. Our research showed that brain-PAG in mTBI patients was significantly higher than that in HCs in both acute and chronic phases. The increased brain-PAG in mTBI patients was also highly correlated with cognitive impairment and a higher level of plasma neurofilament light, a marker of neurodegeneration. The higher brain-PAG and its correlation with severe cognitive impairment showed a longitudinal and persistent nature in patients with follow-up examinations.

CONCLUSION

We proposed an interpretable deep learning framework on a relatively large dataset to accurately predict brain age in both healthy individuals and mTBI patients. The interpretable analysis revealed that the caudate and thalamus became the most contributive role across the adult lifespan in both HCs and patients with mTBI. The left hemisphere contributed significantly to brain age prediction may enlighten us to be concerned about the lateralization of brain abnormality in neurological diseases in the future. The proposed interpretable deep learning framework might also provide hope for testing the performance of related drugs and treatments in the future.

Machine Learning and Deep Learning Approaches in Lifespan Brain Age Prediction: A Comprehensive Review

The concept of 'brain age', derived from neuroimaging data, serves as a crucial biomarker reflecting cognitive vitality and neurodegenerative trajectories. In the past decade, machine learning (ML) and deep learning (DL) integration has transformed the field, providing advanced models for brain age estimation. However, achieving precise brain age prediction across all ages remains a significant analytical challenge. This comprehensive review scrutinizes advancements in ML- and DL-based brain age prediction, analyzing 52 peer-reviewed studies from 2020 to 2024. It assesses various model architectures, highlighting their effectiveness and nuances in lifespan brain age studies. By comparing ML and DL, strengths in forecasting and methodological limitations are revealed. Finally, key findings from the reviewed articles are summarized and a number of major issues related to ML/DL-based lifespan brain age prediction are discussed. Through this study, we aim at the synthesis of the current state of brain age prediction, emphasizing both advancements and persistent challenges, guiding future research, technological advancements, and improving early intervention strategies for neurodegenerative diseases.

Exploring the potential of representation and transfer learning for anatomical neuroimaging: Application to psychiatry

The perspective of personalized medicine for brain disorders requires efficient learning models for anatomical neuroimaging-based prediction of clinical conditions. There is now a consensus on the benefit of deep learning (DL) in addressing many medical imaging tasks, such as image segmentation. However, for single-subject prediction problems, recent studies yielded contradictory results when comparing DL with Standard Machine Learning (SML) on top of classical feature extraction. Most existing comparative studies were limited in predicting phenotypes of little clinical interest, such as sex and age, and using a single dataset. Moreover, they conducted a limited analysis of the employed image pre-processing and feature selection strategies. This paper extensively compares DL and SML prediction capacity on five multi-site problems, including three increasingly complex clinical applications in psychiatry namely schizophrenia, bipolar disorder, and Autism Spectrum Disorder (ASD) diagnosis. To compensate for the relative scarcity of neuroimaging data on these clinical datasets, we also evaluate three pre-training strategies for transfer learning from brain imaging of the general healthy population: self-supervised learning, generative modeling and supervised learning with age. Overall, we find similar performance between randomly initialized DL and SML for the three clinical tasks and a similar scaling trend for sex prediction. This was replicated on an external dataset. We also show highly correlated discriminative brain regions between DL and linear ML models in all problems. Nonetheless, we demonstrate that self-supervised pre-training on large-scale healthy population imaging datasets (N≈10k), along with Deep Ensemble, allows DL to learn robust and transferable representations to smaller-scale clinical datasets (N≤1k). It largely outperforms SML on 2 out of 3 clinical tasks both in internal and external test sets. These findings suggest that the improvement of DL over SML in anatomical neuroimaging mainly comes from its capacity to learn meaningful and useful abstract representations of the brain anatomy, and it sheds light on the potential of transfer learning for personalized medicine in psychiatry.

Brain age gap estimation using attention-based ResNet method for Alzheimer's disease detection

This study investigates the correlation between brain age and chronological age in healthy individuals using brain MRI images, aiming to identify potential biomarkers for neurodegenerative diseases like Alzheimer's. To achieve this, a novel attention-based ResNet method, 3D-Attention-Resent-SVR, is proposed to accurately estimate brain age and distinguish between Cognitively Normal (CN) and Alzheimer's disease (AD) individuals by computing the brain age gap (BAG). Unlike conventional methods, which often rely on single datasets, our approach addresses potential biases by employing four datasets for training and testing. The results, based on a combined dataset from four public sources comprising 3844 data points, demonstrate the model's efficacy with a mean absolute error (MAE) of 2.05 for brain age gap estimation. Moreover, the model's generalizability is showcased by training on three datasets and testing on a separate one, yielding a remarkable MAE of 2.4. Furthermore, leveraging BAG as the sole biomarker, our method achieves an accuracy of 92% and an AUC of 0.87 in Alzheimer's disease detection on the ADNI dataset. These findings underscore the potential of our approach in assisting with early detection and disease monitoring, emphasizing the strong correlation between BAG and AD.

The role of cortical structural variance in deep learning-based prediction of fetal brain age

Background

Deep-learning-based brain age estimation using magnetic resonance imaging data has been proposed to identify abnormalities in brain development and the risk of adverse developmental outcomes in the fetal brain. Although saliency and attention activation maps have been used to understand the contribution of different brain regions in determining brain age, there has been no attempt to explain the influence of shape-related cortical structural features on the variance of predicted fetal brain age.

Methods

We examined the association between the predicted brain age difference (PAD: predicted brain age-chronological age) from our convolution neural networks-based model and global and regional cortical structural measures, such as cortical volume, surface area, curvature, gyrification index, and folding depth, using regression analysis.

Results

Our results showed that global brain volume and surface area were positively correlated with PAD. Additionally, higher cortical surface curvature and folding depth led to a significant increase in PAD in specific regions, including the perisylvian areas, where dramatic agerelated changes in folding structures were observed in the late second trimester. Furthermore, PAD decreased with disorganized sulcal area patterns, suggesting that the interrelated arrangement and areal patterning of the sulcal folds also significantly affected the prediction of fetal brain age.

Conclusion

These results allow us to better understand the variance in deep learning-based fetal brain age and provide insight into the mechanism of the fetal brain age prediction model.

A ResNet mini architecture for brain age prediction

The brain presents age-related structural and functional changes in the human life, with different extends between subjects and groups. Brain age prediction can be used to evaluate the development and aging of human brain, as well as providing valuable information for neurodevelopment and disease diagnosis. Many contributions have been made for this purpose, resorting to different machine learning methods. To solve this task and reduce memory resource consumption, we develop a mini architecture of only 10 layers by modifying the deep residual neural network (ResNet), named ResNet mini architecture. To support the ResNet mini architecture in brain age prediction, the brain age dataset (OpenNeuro #ds000228) that consists of 155 study participants (three classes) and the Alzheimer MRI preprocessed dataset that consists of 6400 images (four classes) are employed. We compared the performance of the ResNet mini architecture with other popular networks using the two considered datasets. Experimental results show that the proposed architecture exhibits generality and robustness with high accuracy and less parameter number.

Transformer based multi-modal MRI fusion for prediction of post-menstrual age and neonatal brain development analysis

The brain development during the perinatal period is characterized by rapid changes in both structure and function, which have significant impact on the cognitive and behavioral abilities later in life. Accurate assessment of brain age is a crucial indicator for brain development maturity and can help predict the risk of neonatal pathology. However, evaluating neonatal brains using magnetic resonance imaging (MRI) is challenging due to its complexity, multi-dimension, and noise with subtle alterations. In this paper, we propose a multi-modal deep learning framework based on transformers for precise post-menstrual age (PMA) estimation and brain development analysis using T2-weighted structural MRI (T2-sMRI) and diffusion MRI (dMRI) data. First, we build a two-stream dense network to learn modality-specific features from T2-sMRI and dMRI of brain individually. Then, a transformer module based on self-attention mechanism integrates these features for PMA prediction and preterm/term classification. Finally, saliency maps on brain templates are used to enhance the interpretability of results. Our method is evaluated on the multi-modal MRI dataset of the developing Human Connectome Project (dHCP), which contains 592 neonates, including 478 term-born and 114 preterm-born subjects. The results demonstrate that our method achieves a 0.5-week mean absolute error (MAE) in PMA estimation for term-born subjects. Notably, preterm-born subjects exhibit delayed brain development, worsening with increasing prematurity. Our method also achieves 95% accuracy in classification of term-born and preterm-born subjects, revealing significant group differences.

Multi-Template Meta-Information Regularized Network for Alzheimer's Disease Diagnosis Using Structural MRI

Structural magnetic resonance imaging (sMRI) has been widely applied in computer-aided Alzheimer's disease (AD) diagnosis, owing to its capabilities in providing detailed brain morphometric patterns and anatomical features in vivo. Although previous works have validated the effectiveness of incorporating metadata (e.g., age, gender, and educational years) for sMRI-based AD diagnosis, existing methods solely paid attention to metadata-associated correlation to AD (e.g., gender bias in AD prevalence) or confounding effects (e.g., the issue of normal aging and metadata-related heterogeneity). Hence, it is difficult to fully excavate the influence of metadata on AD diagnosis. To address these issues, we constructed a novel Multi-template Meta-information Regularized Network (MMRN) for AD diagnosis. Specifically, considering diagnostic variation resulting from different spatial transformations onto different brain templates, we first regarded different transformations as data augmentation for self-supervised learning after template selection. Since the confounding effects may arise from excessive attention to meta-information owing to its correlation with AD, we then designed the modules of weakly supervised meta-information learning and mutual information minimization to learn and disentangle meta-information from learned class-related representations, which accounts for meta-information regularization for disease diagnosis. We have evaluated our proposed MMRN on two public multi-center cohorts, including the Alzheimer's Disease Neuroimaging Initiative (ADNI) with 1,950 subjects and the National Alzheimer's Coordinating Center (NACC) with 1,163 subjects. The experimental results have shown that our proposed method outperformed the state-of-the-art approaches in both tasks of AD diagnosis, mild cognitive impairment (MCI) conversion prediction, and normal control (NC) vs. MCI vs. AD classification.

A joint brain extraction and image quality assessment framework for fetal brain MRI slices

Brain extraction and image quality assessment are two fundamental steps in fetal brain magnetic resonance imaging (MRI) 3D reconstruction and quantification. However, the randomness of fetal position and orientation, the variability of fetal brain morphology, maternal organs around the fetus, and the scarcity of data samples, all add excessive noise and impose a great challenge to automated brain extraction and quality assessment of fetal MRI slices. Conventionally, brain extraction and quality assessment are typically performed independently. However, both of them focus on the brain image representation, so they can be jointly optimized to ensure the network learns more effective features and avoid overfitting. To this end, we propose a novel two-stage dual-task deep learning framework with a brain localization stage and a dual-task stage for joint brain extraction and quality assessment of fetal MRI slices. Specifically, the dual-task module compactly contains a feature extraction module, a quality assessment head and a segmentation head with feature fusion for simultaneous brain extraction and quality assessment. Besides, a transformer architecture is introduced into the feature extraction module and the segmentation head. We utilize a multi-step training strategy to guarantee a stable and successful training of all modules. Finally, we validate our method by a 5-fold cross-validation and ablation study on a dataset with fetal brain MRI slices in different qualities, and perform a cross-dataset validation in addition. Experiments show that the proposed framework achieves very promising performance.

Efficient Supervised Pretraining of Swin-Transformer for Virtual Staining of Microscopy Images

Fluorescence staining is an important technique in life science for labeling cellular constituents. However, it also suffers from being time-consuming, having difficulty in simultaneous labeling, etc. Thus, virtual staining, which does not rely on chemical labeling, has been introduced. Recently, deep learning models such as transformers have been applied to virtual staining tasks. However, their performance relies on large-scale pretraining, hindering their development in the field. To reduce the reliance on large amounts of computation and data, we construct a Swin-transformer model and propose an efficient supervised pretraining method based on the masked autoencoder (MAE). Specifically, we adopt downsampling and grid sampling to mask 75% of pixels and reduce the number of tokens. The pretraining time of our method is only 1/16 compared with the original MAE. We also design a supervised proxy task to predict stained images with multiple styles instead of masked pixels. Additionally, most virtual staining approaches are based on private datasets and evaluated by different metrics, making a fair comparison difficult. Therefore, we develop a standard benchmark based on three public datasets and build a baseline for the convenience of future researchers. We conduct extensive experiments on three benchmark datasets, and the experimental results show the proposed method achieves the best performance both quantitatively and qualitatively. In addition, ablation studies are conducted, and experimental results illustrate the effectiveness of the proposed pretraining method. The benchmark and code are available at https://github.com/birkhoffkiki/CAS-Transformer.

Brain Age Prediction Using Multi-Hop Graph Attention Combined with Convolutional Neural Network

Convolutional neural networks (CNNs) have been used widely to predict biological brain age based on brain magnetic resonance (MR) images. However, CNNs focus mainly on spatially local features and their aggregates and barely on the connective information between distant regions. To overcome this issue, we propose a novel multi-hop graph attention (MGA) module that exploits both the local and global connections of image features when combined with CNNs. After insertion between convolutional layers, MGA first converts the convolution-derived feature map into graph-structured data by using patch embedding and embedding-distance-based scoring. Multi-hop connections between the graph nodes are modeled by using the Markov chain process. After performing multi-hop graph attention, MGA re-converts the graph into an updated feature map and transfers it to the next convolutional layer. We combined the MGA module with sSE (spatial squeeze and excitation)-ResNet18 for our final prediction model (MGA-sSE-ResNet18) and performed various hyperparameter evaluations to identify the optimal parameter combinations. With 2788 three-dimensional T1-weighted MR images of healthy subjects, we verified the effectiveness of MGA-sSE-ResNet18 with comparisons to four established, general-purpose CNNs and two representative brain age prediction models. The proposed model yielded an optimal performance with a mean absolute error of 2.822 years and Pearson's correlation coefficient (PCC) of 0.968, demonstrating the potential of the MGA module to improve the accuracy of brain age prediction.

Is Attention all You Need in Medical Image Analysis? A Review

Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. Despite their important advances, typical CNN have relatively limited capabilities in modelling "global" pixel interactions, which restricts their generalisation ability to understand out-of-distribution data with different "global" information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments ("Transf/Attention") which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced an analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated.

Brain Age Prediction Based on Quantitative Susceptibility Mapping Using the Segmentation Transformer

The process of brain aging is intricate, encompassing significant structural and functional changes, including myelination and iron deposition in the brain. Brain age could act as a quantitative marker to evaluate the degree of the individual's brain evolution. Quantitative susceptibility mapping (QSM) is sensitive to variations in magnetically responsive substances such as iron and myelin, making it a favorable tool for estimating brain age. In this study, we introduce an innovative 3D convolutional network named Segmentation-Transformer-Age-Network (STAN) to predict brain age based on QSM data. STAN employs a two-stage network architecture. The first-stage network learns to extract informative features from the QSM data through segmentation training, while the second-stage network predicts brain age by integrating the global and local features. We collected QSM images from 712 healthy participants, with 548 for training and 164 for testing. The results demonstrate that the proposed method achieved a high accuracy brain age prediction with a mean absolute error (MAE) of 4.124 years and a coefficient of determination (R2) of 0.933. Furthermore, the gaps between the predicted brain age and the chronological age of Parkinson's disease patients were significantly higher than those of healthy subjects (P<0.01). We thus believe that using QSM-based predicted brain age offers a more reliable and accurate phenotype, with the potentiality to serve as a biomarker to explore the process of advanced brain aging.

Brain age prediction across the human lifespan using multimodal MRI data

Measuring differences between an individual's age and biological age with biological information from the brain have the potential to provide biomarkers of clinically relevant neurological syndromes that arise later in human life. To explore the effect of multimodal brain magnetic resonance imaging (MRI) features on the prediction of brain age, we investigated how multimodal brain imaging data improved age prediction from more imaging features of structural or functional MRI data by using partial least squares regression (PLSR) and longevity data sets (age 6-85 years). First, we found that the age-predicted values for each of these ten features ranged from high to low: cortical thickness (R = 0.866, MAE = 7.904), all seven MRI features (R = 0.8594, MAE = 8.24), four features in structural MRI (R = 0.8591, MAE = 8.24), fALFF (R = 0.853, MAE = 8.1918), gray matter volume (R = 0.8324, MAE = 8.931), three rs-fMRI feature (R = 0.7959, MAE = 9.744), mean curvature (R = 0.7784, MAE = 10.232), ReHo (R = 0.7833, MAE = 10.122), ALFF (R = 0.7517, MAE = 10.844), and surface area (R = 0.719, MAE = 11.33). In addition, the significance of the volume and size of brain MRI data in predicting age was also studied. Second, our results suggest that all multimodal imaging features, except cortical thickness, improve brain-based age prediction. Third, we found that the left hemisphere contributed more to the age prediction, that is, the left hemisphere showed a greater weight in the age prediction than the right hemisphere. Finally, we found a nonlinear relationship between the predicted age and the amount of MRI data. Combined with multimodal and lifespan brain data, our approach provides a new perspective for chronological age prediction and contributes to a better understanding of the relationship between brain disorders and aging.

MultiIB-TransUNet: Transformer with multiple information bottleneck blocks for CT and ultrasound image segmentation

BACKGROUND

Accurate medical image segmentation is crucial for disease diagnosis and surgical planning. Transformer networks offer a promising alternative for medical image segmentation as they can learn global features through self-attention mechanisms. To further enhance performance, many researchers have incorporated more Transformer layers into their models. However, this approach often results in the model parameters increasing significantly, causing a potential rise in complexity. Moreover, the datasets of medical image segmentation usually have fewer samples, which leads to the risk of overfitting of the model.

PURPOSE

This paper aims to design a medical image segmentation model that has fewer parameters and can effectively alleviate overfitting.

METHODS

We design a MultiIB-Transformer structure consisting of a single Transformer layer and multiple information bottleneck (IB) blocks. The Transformer layer is used to capture long-distance spatial relationships to extract global feature information. The IB block is used to compress noise and improve model robustness. The advantage of this structure is that it only needs one Transformer layer to achieve the state-of-the-art (SOTA) performance, significantly reducing the number of model parameters. In addition, we designed a new skip connection structure. It only needs two 1× 1 convolutions, the high-resolution feature map can effectively have both semantic and spatial information, thereby alleviating the semantic gap.

RESULTS

The proposed model is on the Breast UltraSound Images (BUSI) dataset, and the IoU and F1 evaluation indicators are 67.75 and 87.78. On the Synapse multi-organ segmentation dataset, the Param, Hausdorff Distance (HD) and Dice Similarity Cofficient (DSC) evaluation indicators are 22.30, 20.04 and 81.83.

CONCLUSIONS

Our proposed model (MultiIB-TransUNet) achieved superior results with fewer parameters compared to other models.

A deep learning model for brain age prediction using minimally preprocessed T1w images as input

Introduction

In the last few years, several models trying to calculate the biological brain age have been proposed based on structural magnetic resonance imaging scans (T1-weighted MRIs, T1w) using multivariate methods and machine learning. We developed and validated a convolutional neural network (CNN)-based biological brain age prediction model that uses one T1w MRI preprocessing step when applying the model to external datasets to simplify implementation and increase accessibility in research settings. Our model only requires rigid image registration to the MNI space, which is an advantage compared to previous methods that require more preprocessing steps, such as feature extraction.

Methods

We used a multicohort dataset of cognitively healthy individuals (age range = 32.0-95.7 years) comprising 17,296 MRIs for training and evaluation. We compared our model using hold-out (CNN1) and cross-validation (CNN2-4) approaches. To verify generalisability, we used two external datasets with different populations and MRI scan characteristics to evaluate the model. To demonstrate its usability, we included the external dataset's images in the cross-validation training (CNN3). To ensure that our model used only the brain signal on the image, we also predicted brain age using skull-stripped images (CNN4).

Results

The trained models achieved a mean absolute error of 2.99, 2.67, 2.67, and 3.08 years for CNN1-4, respectively. The model's performance in the external dataset was in the typical range of mean absolute error (MAE) found in the literature for testing sets. Adding the external dataset to the training set (CNN3), overall, MAE is unaffected, but individual cohort MAE improves (5.63-2.25 years). Salience maps of predictions reveal that periventricular, temporal, and insular regions are the most important for age prediction.

Discussion

We provide indicators for using biological (predicted) brain age as a metric for age correction in neuroimaging studies as an alternative to the traditional chronological age. In conclusion, using different approaches, our CNN-based model showed good performance using one T1w brain MRI preprocessing step. The proposed CNN model is made publicly available for the research community to be easily implemented and used to study ageing and age-related disorders.

Classification of MDD using a Transformer classifier with large-scale multisite resting-state fMRI data

Major depressive disorder (MDD) is one of the most common psychiatric disorders worldwide with high recurrence rate. Identifying MDD patients, particularly those with recurrent episodes with resting-state fMRI, may reveal the relationship between MDD and brain function. We proposed a Transformer-Encoder model, which utilized functional connectivity extracted from large-scale multisite rs-fMRI datasets to classify MDD and HC. The model discarded the Transformer's Decoder part, reducing the model's complexity and decreasing the number of parameters to adapt to the limited sample size and it does not require a complex feature selection process and achieves end-to-end classification. Additionally, our model is suitable for classifying data combined from multiple brain atlases and has an optional unsupervised pre-training module to acquire optimal initial parameters and speed up the training process. The model's performance was tested on a large-scale multisite dataset and identified brain regions affected by MDD using the Grad-CAM method. After conducting five-fold cross-validation, our model achieved an average classification accuracy of 68.61% on a dataset consisting of 1611 samples. For the selected recurrent MDD dataset, the model reached an average classification accuracy of 78.11%. Abnormalities were detected in the frontal gyri and cerebral cortex of MDD patients in both datasets. Furthermore, the identified brain regions in the recurrent MDD dataset generally exhibited a higher contribution to the model's performance.

Deep Learning: A Primer for Neurosurgeons

This chapter explores the transformative impact of deep learning (DL) on neurosurgery, elucidating its pivotal role in enhancing diagnostic performance, surgical planning, execution, and postoperative assessment. It delves into various deep learning architectures, including convolutional and recurrent neural networks, and their applications in analyzing neuroimaging data for brain tumors, spinal cord injuries, and other neurological conditions. The integration of DL in neurosurgical robotics and the potential for fully autonomous surgical procedures are discussed, highlighting advancements in surgical precision and patient outcomes. The chapter also examines the challenges of data privacy, quality, and interpretability that accompany the implementation of DL in neurosurgery. The potential for DL to revolutionize neurosurgical practices through improved diagnostics, patient-specific surgical planning, and the advent of intelligent surgical robots is underscored, promising a future where technology and healthcare converge to offer unprecedented solutions in neurosurgery.

BASE: Brain Age Standardized Evaluation

Brain age, most commonly inferred from T1-weighted magnetic resonance images (T1w MRI), is a robust biomarker of brain health and related diseases. Superior accuracy in brain age prediction, often falling within a 2-3 year range, is achieved predominantly through deep neural networks. However, comparing study results is difficult due to differences in datasets, evaluation methodologies and metrics. Addressing this, we introduce Brain Age Standardized Evaluation (BASE), which includes (i) a standardized T1w MRI dataset including multi-site, new unseen site, test-retest and longitudinal data, and an associated (ii) evaluation protocol, including repeated model training and upon based comprehensive set of performance metrics measuring accuracy, robustness, reproducibility and consistency aspects of brain age predictions, and (iii) statistical evaluation framework based on linear mixed-effects models for rigorous performance assessment and cross-comparison. To showcase BASE, we comprehensively evaluate four deep learning based brain age models, appraising their performance in scenarios that utilize multi-site, test-retest, unseen site, and longitudinal T1w brain MRI datasets. Ensuring full reproducibility and application in future studies, we have made all associated data information and code publicly accessible at https://github.com/AralRalud/BASE.git.

Feature attention graph neural network for estimating brain age and identifying important neural connections in mouse models of genetic risk for Alzheimer's disease

Alzheimer's disease (AD) remains one of the most extensively researched neurodegenerative disorders due to its widespread prevalence and complex risk factors. Age is a crucial risk factor for AD, which can be estimated by the disparity between physiological age and estimated brain age. To model AD risk more effectively, integrating biological, genetic, and cognitive markers is essential. Here, we utilized mouse models expressing the major APOE human alleles and human nitric oxide synthase 2 to replicate genetic risk for AD and a humanized innate immune response. We estimated brain age employing a multivariate dataset that includes brain connectomes, APOE genotype, subject traits such as age and sex, and behavioral data. Our methodology used Feature Attention Graph Neural Networks (FAGNN) for integrating different data types. Behavioral data were processed with a 2D Convolutional Neural Network (CNN), subject traits with a 1D CNN, brain connectomes through a Graph Neural Network using quadrant attention module. The model yielded a mean absolute error for age prediction of 31.85 days, with a root mean squared error of 41.84 days, outperforming other, reduced models. In addition, FAGNN identified key brain connections involved in the aging process. The highest weights were assigned to the connections between cingulum and corpus callosum, striatum, hippocampus, thalamus, hypothalamus, cerebellum, and piriform cortex. Our study demonstrates the feasibility of predicting brain age in models of aging and genetic risk for AD. To verify the validity of our findings, we compared Fractional Anisotropy (FA) along the tracts of regions with the highest connectivity, the Return-to-Origin Probability (RTOP), Return-to-Plane Probability (RTPP), and Return-to-Axis Probability (RTAP), which showed significant differences between young, middle-aged, and old age groups. Younger mice exhibited higher FA, RTOP, RTAP, and RTPP compared to older groups in the selected connections, suggesting that degradation of white matter tracts plays a critical role in aging and for FAGNN's selections. Our analysis suggests a potential neuroprotective role of APOE2, relative to APOE3 and APOE4, where APOE2 appears to mitigate age-related changes. Our findings highlighted a complex interplay of genetics and brain aging in the context of AD risk modeling.

Deep learning-based multi-stage postoperative type-b aortic dissection segmentation using global-local fusion learning

Objective.Type-b aortic dissection (AD) is a life-threatening cardiovascular disease and the primary treatment is thoracic endovascular aortic repair (TEVAR). Due to the lack of a rapid and accurate segmentation technique, the patient-specific postoperative AD model is unavailable in clinical practice, resulting in impracticable 3D morphological and hemodynamic analyses during TEVAR assessment. This work aims to construct a deep learning-based segmentation framework for postoperative type-b AD.Approach.The segmentation is performed in a two-stage manner. A multi-class segmentation of the contrast-enhanced aorta, thrombus (TH), and branch vessels (BV) is achieved in the first stage based on the cropped image patches. True lumen (TL) and false lumen (FL) are extracted from a straightened image containing the entire aorta in the second stage. A global-local fusion learning mechanism is designed to improve the segmentation of TH and BR by compensating for the missing contextual features of the cropped images in the first stage.Results.The experiments are conducted on a multi-center dataset comprising 133 patients with 306 follow-up images. Our framework achieves the state-of-the-art dice similarity coefficient (DSC) of 0.962, 0.921, 0.811, and 0.884 for TL, FL, TH, and BV, respectively. The global-local fusion learning mechanism increases the DSC of TH and BV by 2.3% (p< 0.05) and 1.4% (p< 0.05), respectively, based on the baseline. Segmenting TH in stage 1 can achieve significantly better DSC for FL (0.921 ± 0.055 versus 0.857 ± 0.220,p< 0.01) and TH (0.811 ± 0.137 versus 0.797 ± 0.146,p< 0.05) than in stage 2. Our framework supports more accurate vascular volume quantifications compared with previous segmentation model, especially for the patients with enlarged TH+FL after TEVAR, and shows good generalizability to different hospital settings.Significance.Our framework can quickly provide accurate patient-specific AD models, supporting the clinical practice of 3D morphological and hemodynamic analyses for quantitative and more comprehensive patient-specific TEVAR assessments.

Application of bidirectional long short-term memory network for prediction of cognitive age

Electroencephalography (EEG) measures changes in neuronal activity and can reveal significant changes from infancy to adulthood concomitant with brain maturation, making it a potential physiological marker of brain maturation and cognition. To investigate a promising deep learning tool for EEG classification, we applied the bidirectional long short-term memory (BLSTM) algorithm to analyze EEG data from the pediatric EEG laboratory of Taipei Tzu Chi Hospital. The trained BLSTM model was 86% accurate when identifying EEGs from young children (8 months-6 years) and adolescents (12-20 years). However, there was only a modest classification accuracy (69.3%) when categorizing EEG samples into three age groups (8 months-6 years, 6-12 years, and 12-20 years). For EEG samples from patients with intellectual disability, the prediction accuracy of the trained BLSTM model was 46.4%, which was significantly lower than its accuracy for EEGs from neurotypical patients, indicating that the individual's intelligence plays a major role in the age prediction. This study confirmed that scalp EEG can reflect brain maturation and the BLSTM algorithm is a feasible deep learning tool for the identification of cognitive age. The trained model can potentially be applied to clinical services as a supportive measurement of neurodevelopmental status.

Artificial Intelligence-Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

BACKGROUND

Transformer-based models are gaining popularity in medical imaging and cancer imaging applications. Many recent studies have demonstrated the use of transformer-based models for brain cancer imaging applications such as diagnosis and tumor segmentation.

OBJECTIVE

This study aims to review how different vision transformers (ViTs) contributed to advancing brain cancer diagnosis and tumor segmentation using brain image data. This study examines the different architectures developed for enhancing the task of brain tumor segmentation. Furthermore, it explores how the ViT-based models augmented the performance of convolutional neural networks for brain cancer imaging.

METHODS

This review performed the study search and study selection following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The search comprised 4 popular scientific databases: PubMed, Scopus, IEEE Xplore, and Google Scholar. The search terms were formulated to cover the interventions (ie, ViTs) and the target application (ie, brain cancer imaging). The title and abstract for study selection were performed by 2 reviewers independently and validated by a third reviewer. Data extraction was performed by 2 reviewers and validated by a third reviewer. Finally, the data were synthesized using a narrative approach.

RESULTS

Of the 736 retrieved studies, 22 (3%) were included in this review. These studies were published in 2021 and 2022. The most commonly addressed task in these studies was tumor segmentation using ViTs. No study reported early detection of brain cancer. Among the different ViT architectures, Shifted Window transformer-based architectures have recently become the most popular choice of the research community. Among the included architectures, UNet transformer and TransUNet had the highest number of parameters and thus needed a cluster of as many as 8 graphics processing units for model training. The brain tumor segmentation challenge data set was the most popular data set used in the included studies. ViT was used in different combinations with convolutional neural networks to capture both the global and local context of the input brain imaging data.

CONCLUSIONS

It can be argued that the computational complexity of transformer architectures is a bottleneck in advancing the field and enabling clinical transformations. This review provides the current state of knowledge on the topic, and the findings of this review will be helpful for researchers in the field of medical artificial intelligence and its applications in brain cancer.

Multi-view fusion-based local-global dynamic pyramid convolutional cross-tansformer network for density classification in mammography

Object.Breast density is an important indicator of breast cancer risk. However, existing methods for breast density classification do not fully utilise the multi-view information produced by mammography and thus have limited classification accuracy.Method.In this paper, we propose a multi-view fusion network, denoted local-global dynamic pyramidal-convolution transformer network (LG-DPTNet), for breast density classification in mammography. First, for single-view feature extraction, we develop a dynamic pyramid convolutional network to enable the network to adaptively learn global and local features. Second, we address the problem exhibited by traditional multi-view fusion methods, this is based on a cross-transformer that integrates fine-grained information and global contextual information from different views and thereby provides accurate predictions for the network. Finally, we use an asymmetric focal loss function instead of traditional cross-entropy loss during network training to solve the problem of class imbalance in public datasets, thereby further improving the performance of the model.Results.We evaluated the effectiveness of our method on two publicly available mammography datasets, CBIS-DDSM and INbreast, and achieved areas under the curve (AUC) of 96.73% and 91.12%, respectively.Conclusion.Our experiments demonstrated that the devised fusion model can more effectively utilise the information contained in multiple views than existing models and exhibits classification performance that is superior to that of baseline and state-of-the-art methods.

Human-to-monkey transfer learning identifies the frontal white matter as a key determinant for predicting monkey brain age

The application of artificial intelligence (AI) to summarize a whole-brain magnetic resonance image (MRI) into an effective "brain age" metric can provide a holistic, individualized, and objective view of how the brain interacts with various factors (e.g., genetics and lifestyle) during aging. Brain age predictions using deep learning (DL) have been widely used to quantify the developmental status of human brains, but their wider application to serve biomedical purposes is under criticism for requiring large samples and complicated interpretability. Animal models, i.e., rhesus monkeys, have offered a unique lens to understand the human brain - being a species in which aging patterns are similar, for which environmental and lifestyle factors are more readily controlled. However, applying DL methods in animal models suffers from data insufficiency as the availability of animal brain MRIs is limited compared to many thousands of human MRIs. We showed that transfer learning can mitigate the sample size problem, where transferring the pre-trained AI models from 8,859 human brain MRIs improved monkey brain age estimation accuracy and stability. The highest accuracy and stability occurred when transferring the 3D ResNet [mean absolute error (MAE) = 1.83 years] and the 2D global-local transformer (MAE = 1.92 years) models. Our models identified the frontal white matter as the most important feature for monkey brain age predictions, which is consistent with previous histological findings. This first DL-based, anatomically interpretable, and adaptive brain age estimator could broaden the application of AI techniques to various animal or disease samples and widen opportunities for research in non-human primate brains across the lifespan.

Clinical applications of radiomics in non-small cell lung cancer patients with immune checkpoint inhibitor-related pneumonitis

Immune checkpoint inhibitors (ICIs) modulate the body's immune function to treat tumors but may also induce pneumonitis. Immune checkpoint inhibitor-related pneumonitis (ICIP) is a serious immune-related adverse event (irAE). Immunotherapy is currently approved as a first-line treatment for non-small cell lung cancer (NSCLC), and the incidence of ICIP in NSCLC patients can be as high as 5%-19% in clinical practice. ICIP can be severe enough to lead to the death of NSCLC patients, but there is a lack of a gold standard for the diagnosis of ICIP. Radiomics is a method that uses computational techniques to analyze medical images (e.g., CT, MRI, PET) and extract important features from them, which can be used to solve classification and regression problems in the clinic. Radiomics has been applied to predict and identify ICIP in NSCLC patients in the hope of transforming clinical qualitative problems into quantitative ones, thus improving the diagnosis and treatment of ICIP. In this review, we summarize the pathogenesis of ICIP and the process of radiomics feature extraction, review the clinical application of radiomics in ICIP of NSCLC patients, and discuss its future application prospects.

Determining regional brain growth in premature and mature infants in relation to age at MRI using deep neural networks

Neonatal MRIs are used increasingly in preterm infants. However, it is not always feasible to analyze this data. Having a tool that assesses brain maturation during this period of extraordinary changes would be immensely helpful. Approaches based on deep learning approaches could solve this task since, once properly trained and validated, they can be used in practically any system and provide holistic quantitative information in a matter of minutes. However, one major deterrent for radiologists is that these tools are not easily interpretable. Indeed, it is important that structures driving the results be detailed and survive comparison to the available literature. To solve these challenges, we propose an interpretable pipeline based on deep learning to predict postmenstrual age at scan, a key measure for assessing neonatal brain development. For this purpose, we train a state-of-the-art deep neural network to segment the brain into 87 different regions using normal preterm and term infants from the dHCP study. We then extract informative features for brain age estimation using the segmented MRIs and predict the brain age at scan with a regression model. The proposed framework achieves a mean absolute error of 0.46 weeks to predict postmenstrual age at scan. While our model is based solely on structural T2-weighted images, the results are superior to recent, arguably more complex approaches. Furthermore, based on the extracted knowledge from the trained models, we found that frontal and parietal lobes are among the most important structures for neonatal brain age estimation.

BOston Neonatal Brain Injury Dataset for Hypoxic Ischemic Encephalopathy (BONBID-HIE): Part I. MRI and Manual Lesion Annotation

Hypoxic ischemic encephalopathy (HIE) is a brain injury that occurs in 1 ~ 5/1000 term neonates. Accurate identification and segmentation of HIE-related lesions in neonatal brain magnetic resonance images (MRIs) is the first step toward predicting prognosis, identifying high-risk patients, and evaluating treatment effects. It will lead to a more accurate estimation of prognosis, a better understanding of neurological symptoms, and a timely prediction of response to therapy. We release the first public dataset containing neonatal brain diffusion MRI and expert annotation of lesions from 133 patients diagnosed with HIE. HIE-related lesions in brain MRI are often diffuse (i.e., multi-focal), and small (over half the patients in our data having lesions occupying <1% of brain volume). Segmentation for HIE MRI data is remarkably different from, and arguably more challenging than, other segmentation tasks such as brain tumors with focal and relatively large lesions. We hope that this dataset can help fuel the development of MRI lesion segmentation methods for HIE and small diffuse lesions in general.