Multi-Task Weakly-Supervised Attention Network for Dementia Status Estimation With Structural MRI

Hybrid multi-modality multi-task learning for forecasting progression trajectories in subjective cognitive decline

While numerous studies strive to exploit the complementary potential of MRI and PET using learning-based methods, the effective fusion of the two modalities remains a tricky problem due to their inherently distinctive properties. In addition, current studies often face the problem of small sample sizes and missing PET data due to factors such as patient withdrawal or low image quality. To this end, we propose a hybrid multi-modality multi-task learning (HM2L) framework with cross-domain knowledge transfer for forecasting trajectories of SCD progression. Our HM2L comprises (1) missing PET imputation, (2) multi-modality feature extraction for MRI and PET feature learning with a novel softmax-triplet constraint, (3) attention-based multi-modality fusion of MRI and PET features, and (4) multi-task prediction of category labels and clinical scores such as Mini-Mental State Examination (MMSE) and Geriatric Depression Scale (GDS). To handle problems with small sample sizes, a transfer learning strategy is developed to enable knowledge transfer from a relatively large scale dataset with MRI and PET from 795 subjects to two small-scale SCD cohorts with a total of 136 subjects. Experimental results indicate HM2L surpasses several state-of-the-art methods in jointly predicting category labels and clinical scores of subjective cognitive decline. Results show that the MMSE scores of SCD subjects who develop mild cognitive impairment during the 2-year/7-year follow-up are significantly lower than those of subjects who remain stable, while there exists a complex relationship between SCD progression with GDS.

Broad Multitask Learning System With Group Sparse Regularization

The broad learning system (BLS) featuring lightweight, incremental extension, and strong generalization capabilities has been successful in its applications. Despite these advantages, BLS struggles in multitask learning (MTL) scenarios with its limited ability to simultaneously unravel multiple complex tasks where existing BLS models cannot adequately capture and leverage essential information across tasks, decreasing their effectiveness and efficacy in MTL scenarios. To address these limitations, we proposed an innovative MTL framework explicitly designed for BLS, named group sparse regularization for broad multitask learning system using related task-wise (BMtLS-RG). This framework combines a task-related BLS learning mechanism with a group sparse optimization strategy, significantly boosting BLS's ability to generalize in MTL environments. The task-related learning component harnesses task correlations to enable shared learning and optimize parameters efficiently. Meanwhile, the group sparse optimization approach helps minimize the effects of irrelevant or noisy data, thus enhancing the robustness and stability of BLS in navigating complex learning scenarios. To address the varied requirements of MTL challenges, we presented two additional variants of BMtLS-RG: BMtLS-RG with sharing parameters of feature mapped nodes (BMtLS-RGf), which integrates a shared feature mapping layer, and BMtLS-RGf and enhanced nodes (BMtLS-RGfe), which further includes an enhanced node layer atop the shared feature mapping structure. These adaptations provide customized solutions tailored to the diverse landscape of MTL problems. We compared BMtLS-RG with state-of-the-art (SOTA) MTL and BLS algorithms through comprehensive experimental evaluation across multiple practical MTL and UCI datasets. BMtLS-RG outperformed SOTA methods in 97.81% of classification tasks and achieved optimal performance in 96.00% of regression tasks, demonstrating its superior accuracy and robustness. Furthermore, BMtLS-RG exhibited satisfactory training efficiency, outperforming existing MTL algorithms by 8.04-42.85 times.

Modality-Aware Discriminative Fusion Network for Integrated Analysis of Brain Imaging Genomics

Mild cognitive impairment (MCI) represents an early stage of Alzheimer's disease (AD), characterized by subtle clinical symptoms that pose challenges for accurate diagnosis. The quest for the identification of MCI individuals has highlighted the importance of comprehending the underlying mechanisms of disease causation. Integrated analysis of brain imaging and genomics offers a promising avenue for predicting MCI risk before clinical symptom onset. However, most existing methods face challenges in: 1) mining the brain network-specific topological structure and addressing the single nucleotide polymorphisms (SNPs)-related noise contamination and 2) extracting the discriminative properties of brain imaging genomics, resulting in limited accuracy for MCI diagnosis. To this end, a modality-aware discriminative fusion network (MA-DFN) is proposed to integrate the complementary information from brain imaging genomics to diagnose MCI. Specifically, we first design two modality-specific feature extraction modules: the graph convolutional network with edge-augmented self-attention module (GCN-EASA) and the deep adversarial denoising autoencoder module (DAD-AE), to capture the topological structure of brain networks and the intrinsic distribution of SNPs. Subsequently, a discriminative-enhanced fusion network with correlation regularization module (DFN-CorrReg) is employed to enhance inter-modal consistency and between-class discrimination in brain imaging and genomics. Compared to other state-of-the-art approaches, MA-DFN not only exhibits superior performance in stratifying cognitive normal (CN) and MCI individuals but also identifies disease-related brain regions and risk SNPs locus, which hold potential as putative biomarkers for MCI diagnosis.

Development and validation of a deep learning model based on cascade mask regional convolutional neural network to noninvasively and accurately identify human round spermatids

INTRODUCTION

The difficulty of identifying human round spermatids (hRSs) has impeded applications of the human round spermatid injection (ROSI) technique. RSs can be accurately screened through flow cytometric analysis utilizing the Hoechst fluorescence profile reflecting DNA, but this method is not suitable for isolating hRSs due to the toxicity associated with Hoechst staining.

OBJECTIVE

To evaluate the capacity of a deep learning model grounded in a cascade mask region-based convolutional neural network (R-CNN) for the noninvasive and accurate identification of hRSs.

METHODS

In this study, we presented the development and validation of a deep learning model for identifying hRSs through the analysis of 3457 optical light microscope images of sorted hRSs obtained via flow cytometric analysis. The model's accuracy and specificity were evaluated by calculating the mean average precision (mAP). Furthermore, a double-blind experiment was conducted to access the reliability of the proposed model in accurately identifying hRSs. It detected the expression of protamine (PRM1) and/or peanut lectin (PNA), which are established markers for RSs.

RESULTS

Our deep learning-based model demonstrated a high precision, achieving a mAP of over 0.80 for isolating hRSs in test datasets. The expression of PRM1 and/or PNA was observed in all cells noninvasively selected by our AI model during an independent double-blind test. This phenomenon confirmed the accuracy and effectiveness of the proposed model. The model's capability for noninvasive and accurate isolation of hRSs among spermatogenic cells highlighted its robustness and generalizability for clinical applications.

CONCLUSION

The deep learning AI model based on a cascade R-CNN has the ability to accurately identify hRSs among spermatogenic cells. The application of this noninvasive method, which requires no additional procedures in clinical practice, is able to facilitate the widespread implementation of ROSI technique. Therefore, it can provide patients with spermatogenic arrest the opportunity to become biological fathers.

VSNet: Vessel Structure-aware Network for hepatic and portal vein segmentation

Identifying and segmenting hepatic and portal veins (two predominant vascular systems in the liver, from CT scans) play a crucial role for clinicians in preoperative planning for treatment strategies. However, existing segmentation models often struggle to capture fine details of minor veins. In this article, we introduce Vessel Structure-aware Network (VSNet), a multi-task learning model with vessel-growing decoder, to address the challenge. VSNet excels at accurate segmentation by capturing the topological features of both minor veins while preserving correct connectivity from minor vessels to trucks. We also build and publish the largest dataset (303 cases) for hepatic and portal vessel segmentation. Through comprehensive experiments, we demonstrate that VSNet achieves the best Dice for hepatic vein of 0.824 and portal vein of 0.807 on our proposed dataset, significantly outperforming other popular segmentation models. The source code and dataset are publicly available at https://github.com/XXYZB/VSNet.

Pyramid Pixel Context Adaption Network for Medical Image Classification With Supervised Contrastive Learning

Spatial attention (SA) mechanism has been widely incorporated into deep neural networks (DNNs), significantly lifting the performance in computer vision tasks via long-range dependency modeling. However, it may perform poorly in medical image analysis. Unfortunately, the existing efforts are often unaware that long-range dependency modeling has limitations in highlighting subtle lesion regions. To overcome this limitation, we propose a practical yet lightweight architectural unit, pyramid pixel context adaption (PPCA) module, which exploits multiscale pixel context information to recalibrate pixel position in a pixel-independent manner dynamically. PPCA first applies a well-designed cross-channel pyramid pooling (CCPP) to aggregate multiscale pixel context information, then eliminates the inconsistency among them by the well-designed pixel normalization (PN), and finally estimates per pixel attention weight via a pixel context integration. By embedding PPCA into a DNN with negligible overhead, the PPCA network (PPCANet) is developed for medical image classification. In addition, we introduce supervised contrastive learning to enhance feature representation by exploiting the potential of label information via supervised contrastive loss (CL). The extensive experiments on six medical image datasets show that the PPCANet outperforms state-of-the-art (SOTA) attention-based networks and recent DNNs. We also provide visual analysis and ablation study to explain the behavior of PPCANet in the decision-making process.

Community Graph Convolution Neural Network for Alzheimer's Disease Classification and Pathogenetic Factors Identification

As a complex neural network system, the brain regions and genes collaborate to effectively store and transmit information. We abstract the collaboration correlations as the brain region gene community network (BG-CN) and present a new deep learning approach, such as the community graph convolutional neural network (Com-GCN), for investigating the transmission of information within and between communities. The results can be used for diagnosing and extracting causal factors for Alzheimer's disease (AD). First, an affinity aggregation model for BG-CN is developed to describe intercommunity and intracommunity information transmission. Second, we design the Com-GCN architecture with intercommunity convolution and intracommunity convolution operations based on the affinity aggregation model. Through sufficient experimental validation on the AD neuroimaging initiative (ADNI) dataset, the design of Com-GCN matches the physiological mechanism better and improves the interpretability and classification performance. Furthermore, Com-GCN can identify lesioned brain regions and disease-causing genes, which may assist precision medicine and drug design in AD and serve as a valuable reference for other neurological disorders.

Modeling High-Order Relationships: Brain-Inspired Hypergraph-Induced Multimodal-Multitask Framework for Semantic Comprehension

Semantic comprehension aims to reasonably reproduce people's real intentions or thoughts, e.g., sentiment, humor, sarcasm, motivation, and offensiveness, from multiple modalities. It can be instantiated as a multimodal-oriented multitask classification issue and applied to scenarios, such as online public opinion supervision and political stance analysis. Previous methods generally employ multimodal learning alone to deal with varied modalities or solely exploit multitask learning to solve various tasks, a few to unify both into an integrated framework. Moreover, multimodal-multitask cooperative learning could inevitably encounter the challenges of modeling high-order relationships, i.e., intramodal, intermodal, and intertask relationships. Related research of brain sciences proves that the human brain possesses multimodal perception and multitask cognition for semantic comprehension via decomposing, associating, and synthesizing processes. Thus, establishing a brain-inspired semantic comprehension framework to bridge the gap between multimodal and multitask learning becomes the primary motivation of this work. Motivated by the superiority of the hypergraph in modeling high-order relations, in this article, we propose a hypergraph-induced multimodal-multitask (HIMM) network for semantic comprehension. HIMM incorporates monomodal, multimodal, and multitask hypergraph networks to, respectively, mimic the decomposing, associating, and synthesizing processes to tackle the intramodal, intermodal, and intertask relationships accordingly. Furthermore, temporal and spatial hypergraph constructions are designed to model the relationships in the modality with sequential and spatial structures, respectively. Also, we elaborate a hypergraph alternative updating algorithm to ensure that vertices aggregate to update hyperedges and hyperedges converge to update their connected vertices. Experiments on the dataset with two modalities and five tasks verify the effectiveness of HIMM on semantic comprehension.

Spatial-Temporal Co-Attention Learning for Diagnosis of Mental Disorders From Resting-State fMRI Data

Neuroimaging techniques have been widely adopted to detect the neurological brain structures and functions of the nervous system. As an effective noninvasive neuroimaging technique, functional magnetic resonance imaging (fMRI) has been extensively used in computer-aided diagnosis (CAD) of mental disorders, e.g., autism spectrum disorder (ASD) and attention deficit/hyperactivity disorder (ADHD). In this study, we propose a spatial-temporal co-attention learning (STCAL) model for diagnosing ASD and ADHD from fMRI data. In particular, a guided co-attention (GCA) module is developed to model the intermodal interactions of spatial and temporal signal patterns. A novel sliding cluster attention module is designed to address global feature dependency of self-attention mechanism in fMRI time series. Comprehensive experimental results demonstrate that our STCAL model can achieve competitive accuracies of 73.0 ± 4.5%, 72.0 ± 3.8%, and 72.5 ± 4.2% on the ABIDE I, ABIDE II, and ADHD-200 datasets, respectively. Moreover, the potential for feature pruning based on the co-attention scores is validated by the simulation experiment. The clinical interpretation analysis of STCAL can allow medical professionals to concentrate on the discriminative regions of interest and key time frames from fMRI data.

Understanding machine learning applications in dementia research and clinical practice: a review for biomedical scientists and clinicians

Several (inter)national longitudinal dementia observational datasets encompassing demographic information, neuroimaging, biomarkers, neuropsychological evaluations, and muti-omics data, have ushered in a new era of potential for integrating machine learning (ML) into dementia research and clinical practice. ML, with its proficiency in handling multi-modal and high-dimensional data, has emerged as an innovative technique to facilitate early diagnosis, differential diagnosis, and to predict onset and progression of mild cognitive impairment and dementia. In this review, we evaluate current and potential applications of ML, including its history in dementia research, how it compares to traditional statistics, the types of datasets it uses and the general workflow. Moreover, we identify the technical barriers and challenges of ML implementations in clinical practice. Overall, this review provides a comprehensive understanding of ML with non-technical explanations for broader accessibility to biomedical scientists and clinicians.

PatchNet: Maximize the Exploration of Congeneric Semantics for Weakly Supervised Semantic Segmentation

With the increase in the number of image data and the lack of corresponding labels, weakly supervised learning has drawn a lot of attention recently in computer vision tasks, especially in the fine-grained semantic segmentation problem. To alleviate human efforts from expensive pixel-by-pixel annotations, our method focuses on weakly supervised semantic segmentation (WSSS) with image-level labels, which are much easier to obtain. As a considerable gap exists between pixel-level segmentation and image-level labels, how to reflect the image-level semantic information on each pixel is an important question. To explore the congeneric semantic regions from the same class to the maximum, we construct the patch-level semantic augmentation network (PatchNet) based on the self-detected patches from different images that contain the same class labels. Patches can frame the objects as much as possible and include as little background as possible. The patch-level semantic augmentation network that is established with patches as the nodes can maximize the mutual learning of similar objects. We regard the embedding vectors of patches as nodes and use a transformer-based complementary learning module to construct weighted edges according to the embedding similarity between different nodes. Moreover, to better supplement semantic information, we propose softcomplementary loss functions matched with the whole network structure. We conduct experiments on the popular PASCAL VOC 2012 and MS COCO 2014 benchmarks, and our model yields the state-of-the-art performance.

Multitask Learning for Joint Diagnosis of Multiple Mental Disorders in Resting-State fMRI

Facing the increasing worldwide prevalence of mental disorders, the symptom-based diagnostic criteria struggle to address the urgent public health concern due to the global shortfall in well-qualified professionals. Thanks to the recent advances in neuroimaging techniques, functional magnetic resonance imaging (fMRI) has surfaced as a new solution to characterize neuropathological biomarkers for detecting functional connectivity (FC) anomalies in mental disorders. However, the existing computer-aided diagnosis models for fMRI analysis suffer from unstable performance on large datasets. To address this issue, we propose an efficient multitask learning (MTL) framework for joint diagnosis of multiple mental disorders using resting-state fMRI data. A novel multiobjective evolutionary clustering algorithm is presented to group regions of interests (ROIs) into different clusters for FC pattern analysis. On the optimal clustering solution, the multicluster multigate mixture-of-expert model is used for the final classification by capturing the highly consistent feature patterns among related diagnostic tasks. Extensive simulation experiments demonstrate that the performance of the proposed framework is superior to that of the other state-of-the-art methods. Moreover, the potential for practical application of the framework is also validated in terms of limited computational resources, real-time analysis, and insufficient training data. The proposed model can identify the remarkable interpretative biomarkers associated with specific mental disorders for clinical interpretation analysis.

Attention-Like Multimodality Fusion With Data Augmentation for Diagnosis of Mental Disorders Using MRI

The globally rising prevalence of mental disorders leads to shortfalls in timely diagnosis and therapy to reduce patients' suffering. Facing such an urgent public health problem, professional efforts based on symptom criteria are seriously overstretched. Recently, the successful applications of computer-aided diagnosis approaches have provided timely opportunities to relieve the tension in healthcare services. Particularly, multimodal representation learning gains increasing attention thanks to the high temporal and spatial resolution information extracted from neuroimaging fusion. In this work, we propose an efficient multimodality fusion framework to identify multiple mental disorders based on the combination of functional and structural magnetic resonance imaging. A multioutput conditional generative adversarial network (GAN) is developed to address the scarcity of multimodal data for augmentation. Based on the augmented training data, the multiheaded gating fusion model is proposed for classification by extracting the complementary features across different modalities. The experiments demonstrate that the proposed model can achieve robust accuracies of 75.1 ± 1.5 %, 72.9 ± 1.1 %, and 87.2 ± 1.5 % for autism spectrum disorder (ASD), attention deficit/hyperactivity disorder, and schizophrenia, respectively. In addition, the interpretability of our model is expected to enable the identification of remarkable neuropathology diagnostic biomarkers, leading to well-informed therapeutic decisions.

A Causality-Aware Graph Convolutional Network Framework for Rigidity Assessment in Parkinsonians

Rigidity is one of the common motor disorders in Parkinson's disease (PD), which lead to life quality deterioration. The widely-used rating-scale-based approach for rigidity assessment still depends on the availability of experienced neurologists and is limited by rating subjectivity. Given the recent successful applications of quantitative susceptibility mapping (QSM) in auxiliary PD diagnosis, automated assessment of PD rigidity can be essentially achieved through QSM analysis. However, a major challenge is the performance instability due to the confounding factors (e.g., noise and distribution shift) which conceal the truly-causal features. Therefore, we propose a causality-aware graph convolutional network (GCN) framework, where causal feature selection is combined with causal invariance to ensure that causality-informed model decisions are reached. Firstly, a GCN model that integrates causal feature selection is systematically constructed at three graph levels: node, structure, and representation. In this model, a causal diagram is learned to extract a subgraph with truly-causal information. Secondly, a non-causal perturbation strategy is developed along with an invariance constraint to ensure the stability of the assessment results under different distributions, and thus avoid spurious correlations caused by distribution shifts. The superiority of the proposed method is shown by extensive experiments and the clinical value is revealed by the direct relevance of selected brain regions to rigidity in PD. Besides, its extensibility is verified on other two tasks: PD bradykinesia and mental state for Alzheimer's disease. Overall, we provide a clinically-potential tool for automated and stable assessment of PD rigidity. Our source code will be available at https://github.com/SJTUBME-QianLab/Causality-Aware-Rigidity.

Updated Models of Alzheimer's Disease with Deep Neural Networks

Background

In recent years, researchers have focused on developing precise models for the progression of Alzheimer's disease (AD) using deep neural networks. Forecasting the progression of AD through the analysis of time series data represents a promising approach.

Objective

The primary objective of this research is to formulate an effective methodology for forecasting the progression of AD through the integration of multi-task learning techniques and the analysis of pertinent medical data.

Methods

This study primarily utilized volumetric measurements obtained through magnetic resonance imaging (MRI), trajectories of cognitive assessments, and clinical status indicators. The research encompassed 150 patients diagnosed with AD who underwent examination between 2020 and 2022 in Beijing, China. A multi-task learning approach was employed to train forecasting models using MRI data, trajectories of cognitive assessments, and clinical status. Correlation analysis was conducted at various time points.

Results

At the baseline, a robust correlation was observed among the forecasting tasks: 0.75 for volumetric MRI measurements, 0.62 for trajectories of cognitive assessment, and 0.48 for clinical status. The implementation of a multi-task learning framework enhanced performance by 12.7% for imputing missing values and 14.8% for prediction accuracy.

Conclusions

The findings of our study, indicate that multi-task learning can effectively predict the progression of AD. However, it is important to note that the study's generalizability may be limited due to the restricted dataset and the specific population under examination. These conclusions represent a significant stride toward more precise diagnosis and treatment of this neurological disorder.

A neuroimaging biomarker for Individual Brain-Related Abnormalities In Neurodegeneration (IBRAIN): a cross-sectional study

Background

Alzheimer's disease (AD) is a prevalent neurodegenerative disorder that poses a worldwide public health challenge. A neuroimaging biomarker would significantly improve early diagnosis and intervention, ultimately enhancing the quality of life for affected individuals and reducing the burden on healthcare systems.

Methods

Cross-sectional and longitudinal data (10,099 participants with 13,380 scans) from 12 independent datasets were used in the present study (this study was performed between September 1, 2021 and February 15, 2023). The Individual Brain-Related Abnormalities In Neurodegeneration (IBRAIN) score was developed via integrated regional- and network-based measures under an ensemble machine learning model based on structural MRI data. We systematically assessed whether IBRAIN could be a neuroimaging biomarker for AD.

Findings

IBRAIN accurately differentiated individuals with AD from NCs (AUC = 0.92) and other neurodegenerative diseases, including Frontotemporal dementia (FTD), Parkinson's disease (PD), Vascular dementia (VaD) and Amyotrophic Lateral Sclerosis (ALS) (AUC = 0.92). IBRAIN was significantly correlated to clinical measures and gene expression, enriched in immune process and protein metabolism. The IBRAIN score exhibited a significant ability to reveal the distinct progression of prodromal AD (i.e., Mild cognitive impairment, MCI) (Hazard Ratio (HR) = 6.52 [95% CI: 4.42∼9.62], p < 1 × 10-16), which offers similar powerful performance with Cerebrospinal Fluid (CSF) Aβ (HR = 3.78 [95% CI: 2.63∼5.43], p = 2.13 × 10-14) and CSF Tau (HR = 3.77 [95% CI: 2.64∼5.39], p = 9.53 × 10-15) based on the COX and Log-rank test. Notably, the IBRAIN shows comparable sensitivity (beta = -0.70, p < 1 × 10-16) in capturing longitudinal changes in individuals with conversion to AD than CSF Aβ (beta = -0.26, p = 4.40 × 10-9) and CSF Tau (beta = 0.12, p = 1.02 × 10-5).

Interpretation

Our findings suggested that IBRAIN is a biologically relevant, specific, and sensitive neuroimaging biomarker that can serve as a clinical measure to uncover prodromal AD progression. It has strong potential for application in future clinical practice and treatment trials.

Funding

Science and Technology Innovation 2030 Major Projects, the National Natural Science Foundation of China, Beijing Natural Science Funds, the Fundamental Research Funds for the CentralUniversity, and the Startup Funds for Talents at Beijing Normal University.

Neuroimaging mechanisms of acupuncture on functional reorganization for post-stroke motor improvement: a machine learning-based functional magnetic resonance imaging study

Objective

Motor recovery is crucial in stroke rehabilitation, and acupuncture can influence recovery. Neuroimaging and machine learning approaches provide new research directions to explore the brain functional reorganization and acupuncture mechanisms after stroke. We applied machine learning to predict the classification of the minimal clinically important differences (MCID) for motor improvement and identify the neuroimaging features, in order to explore brain functional reorganization and acupuncture mechanisms for motor recovery after stroke.

Methods

In this study, 49 patients with unilateral motor pathway injury (basal ganglia and/or corona radiata) after ischemic stroke were included and evaluated the motor function by Fugl-Meyer Assessment scores (FMA) at baseline and at 2-week follow-up sessions. Patients were divided by the difference between the twice FMA scores into one group showing minimal clinically important difference (MCID group, n = 28) and the other group with no minimal clinically important difference (N-MCID, n = 21). Machine learning was performed by PRoNTo software to predict the classification of the patients and identify the feature brain regions of interest (ROIs). In addition, a matched group of healthy controls (HC, n = 26) was enrolled. Patients and HC underwent magnetic resonance imaging examination in the resting state and in the acupuncture state (acupuncture at the Yanglingquan point on one side) to compare the differences in brain functional connectivity (FC) and acupuncture effects.

Results

Through machine learning, we obtained a balance accuracy rate of 75.51% and eight feature ROIs. Compared to HC, we found that the stroke patients with lower FC between these feature ROIs with other brain regions, while patients in the MCID group exhibited a wider range of lower FC. When acupuncture was applied to Yanglingquan (GB 34), the abnormal FC of patients was decreased, with different targets of effects in different groups.

Conclusion

Feature ROIs identified by machine learning can predict the classification of stroke patients with different motor improvements, and the FC between these ROIs with other brain regions is decreased. Acupuncture can modulate the bilateral cerebral hemispheres to restore abnormal FC via different targets, thereby promoting motor recovery after stroke.

Clinical trial registration

https://www.chictr.org.cn/showproj.html?proj=37359, ChiCTR1900022220.

MPS-FFA: A multiplane and multiscale feature fusion attention network for Alzheimer's disease prediction with structural MRI

Structural magnetic resonance imaging (sMRI) is a popular technique that is widely applied in Alzheimer's disease (AD) diagnosis. However, only a few structural atrophy areas in sMRI scans are highly associated with AD. The degree of atrophy in patients' brain tissues and the distribution of lesion areas differ among patients. Therefore, a key challenge in sMRI-based AD diagnosis is identifying discriminating atrophy features. Hence, we propose a multiplane and multiscale feature-level fusion attention (MPS-FFA) model. The model has three components, (1) A feature encoder uses a multiscale feature extractor with hybrid attention layers to simultaneously capture and fuse multiple pathological features in the sagittal, coronal, and axial planes. (2) A global attention classifier combines clinical scores and two global attention layers to evaluate the feature impact scores and balance the relative contributions of different feature blocks. (3) A feature similarity discriminator minimizes the feature similarities among heterogeneous labels to enhance the ability of the network to discriminate atrophy features. The MPS-FFA model provides improved interpretability for identifying discriminating features using feature visualization. The experimental results on the baseline sMRI scans from two databases confirm the effectiveness (e.g., accuracy and generalizability) of our method in locating pathological locations. The source code is available at https://github.com/LiuFei-AHU/MPSFFA.

Deciphering multiple sclerosis disability with deep learning attention maps on clinical MRI

The application of convolutional neural networks (CNNs) to MRI data has emerged as a promising approach to achieving unprecedented levels of accuracy when predicting the course of neurological conditions, including multiple sclerosis, by means of extracting image features not detectable through conventional methods. Additionally, the study of CNN-derived attention maps, which indicate the most relevant anatomical features for CNN-based decisions, has the potential to uncover key disease mechanisms leading to disability accumulation. From a cohort of patients prospectively followed up after a first demyelinating attack, we selected those with T1-weighted and T2-FLAIR brain MRI sequences available for image analysis and a clinical assessment performed within the following six months (N = 319). Patients were divided into two groups according to expanded disability status scale (EDSS) score: ≥3.0 and < 3.0. A 3D-CNN model predicted the class using whole-brain MRI scans as input. A comparison with a logistic regression (LR) model using volumetric measurements as explanatory variables and a validation of the CNN model on an independent dataset with similar characteristics (N = 440) were also performed. The layer-wise relevance propagation method was used to obtain individual attention maps. The CNN model achieved a mean accuracy of 79% and proved to be superior to the equivalent LR-model (77%). Additionally, the model was successfully validated in the independent external cohort without any re-training (accuracy = 71%). Attention-map analyses revealed the predominant role of frontotemporal cortex and cerebellum for CNN decisions, suggesting that the mechanisms leading to disability accrual exceed the mere presence of brain lesions or atrophy and probably involve how damage is distributed in the central nervous system.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

Prediction of conversion to dementia using interpretable machine learning in patients with amnestic mild cognitive impairment

Purpose

Amnestic mild cognitive impairment (aMCI) is a transitional state between normal aging and Alzheimer's disease (AD). However, not all aMCI patients are observed to convert to AD dementia. Therefore, developing a predictive algorithm for the conversion of aMCI to AD dementia is important. Parametric methods, such as logistic regression, have been developed; however, it is difficult to reflect complex patterns, such as non-linear relationships and interactions between variables. Therefore, this study aimed to improve the predictive power of aMCI patients' conversion to dementia by using an interpretable machine learning (IML) algorithm and to identify the factors that increase the risk of individual conversion to dementia in each patient.

Methods

We prospectively recruited 705 patients with aMCI who had been followed-up for at least 3 years after undergoing baseline neuropsychological tests at the Samsung Medical Center between 2007 and 2019. We used neuropsychological tests and apolipoprotein E (APOE) genotype data to develop a predictive algorithm. The model-building and validation datasets were composed of data of 565 and 140 patients, respectively. For global interpretation, four algorithms (logistic regression, random forest, support vector machine, and extreme gradient boosting) were compared. For local interpretation, individual conditional expectations (ICE) and SHapley Additive exPlanations (SHAP) were used to analyze individual patients.

Results

Among the four algorithms, the extreme gradient boost model showed the best performance, with an area under the receiver operating characteristic curve of 0.852 and an accuracy of 0.807. Variables, such as age, education, the scores of visuospatial and memory domains, the sum of boxes of the Clinical Dementia Rating scale, Mini-Mental State Examination, and APOE genotype were important features for creating the algorithm. Through ICE and SHAP analyses, it was also possible to interpret which variables acted as strong factors for each patient.

Conclusion

We were able to propose a predictive algorithm for each aMCI individual's conversion to dementia using the IML technique. This algorithm is expected to be useful in clinical practice and the research field, as it can suggest conversion with high accuracy and identify the degree of influence of risk factors for each patient.

Alzheimer's Disease Diagnosis With Brain Structural MRI Using Multiview-Slice Attention and 3D Convolution Neural Network

Numerous artificial intelligence (AI) based approaches have been proposed for automatic Alzheimer's disease (AD) prediction with brain structural magnetic resonance imaging (sMRI). Previous studies extract features from the whole brain or individual slices separately, ignoring the properties of multi-view slices and feature complementarity. For this reason, we present a novel AD diagnosis model based on the multiview-slice attention and 3D convolution neural network (3D-CNN). Specifically, we begin by extracting the local slice-level characteristic in various dimensions using multiple sub-networks. Then we proposed a slice-level attention mechanism to emphasize specific 2D-slices to exclude the redundancy features. After that, a 3D-CNN was employed to capture the global subject-level structural changes. Finally, all these 2D and 3D features were fused to obtain more discriminative representations. We conduct the experiments on 1,451 subjects from ADNI-1 and ADNI-2 datasets. Experimental results showed the superiority of our model over the state-of-the-art approaches regarding dementia classification. Specifically, our model achieves accuracy values of 91.1 and 80.1% on ADNI-1 for AD diagnosis and mild cognitive impairment (MCI) convention prediction, respectively.

Deep Learning in Large and Multi-Site Structural Brain MR Imaging Datasets

Large, multi-site, heterogeneous brain imaging datasets are increasingly required for the training, validation, and testing of advanced deep learning (DL)-based automated tools, including structural magnetic resonance (MR) image-based diagnostic and treatment monitoring approaches. When assembling a number of smaller datasets to form a larger dataset, understanding the underlying variability between different acquisition and processing protocols across the aggregated dataset (termed “batch effects”) is critical. The presence of variation in the training dataset is important as it more closely reflects the true underlying data distribution and, thus, may enhance the overall generalizability of the tool. However, the impact of batch effects must be carefully evaluated in order to avoid undesirable effects that, for example, may reduce performance measures. Batch effects can result from many sources, including differences in acquisition equipment, imaging technique and parameters, as well as applied processing methodologies. Their impact, both beneficial and adversarial, must be considered when developing tools to ensure that their outputs are related to the proposed clinical or research question (i.e., actual disease-related or pathological changes) and are not simply due to the peculiarities of underlying batch effects in the aggregated dataset. We reviewed applications of DL in structural brain MR imaging that aggregated images from neuroimaging datasets, typically acquired at multiple sites. We examined datasets containing both healthy control participants and patients that were acquired using varying acquisition protocols. First, we discussed issues around Data Access and enumerated the key characteristics of some commonly used publicly available brain datasets. Then we reviewed methods for correcting batch effects by exploring the two main classes of approaches: Data Harmonization that uses data standardization, quality control protocols or other similar algorithms and procedures to explicitly understand and minimize unwanted batch effects; and Domain Adaptation that develops DL tools that implicitly handle the batch effects by using approaches to achieve reliable and robust results. In this narrative review, we highlighted the advantages and disadvantages of both classes of DL approaches, and described key challenges to be addressed in future studies.

Identifying factors associated with central obesity in school students using artificial intelligence techniques

OBJECTIVES

We, in a large survey of school students from Beijing, aimed to identify the minimal number of promising factors associated with central obesity and the optimal machine-learning algorithm.

METHODS

Using a cluster sampling strategy, this cross-sectional survey was conducted in Beijing in early 2022 among students 6-14 years of age. Information was gleaned via online questionnaires and analyzed by the PyCharm and Python.

RESULTS

Data from 11,308 children were abstracted for analysis, and 3,970 of children had central obesity. Light gradient boosting machine (LGBM) outperformed the other 10 models. The accuracy, precision, recall, F1 score, area under the receiver operating characteristic of LGBM were 0.769982, 0.688312, 0.612323, 0.648098, and 0.825352, respectively. After a comprehensive evaluation, the minimal set involving top 6 important variables that can predict central obesity with descent performance was ascertained, including father's body mass index (BMI), mother's BMI, picky for foods, outdoor activity, screen, and sex. Validation using the deep-learning model indicated that prediction performance between variables in the minimal set and in the whole set was comparable.

CONCLUSIONS

We have identified and validated a minimal set of six important factors that can decently predict the risk of central obesity when using the optimal LGBM model relative to the whole set.

Factors Associated With Lower Respiratory Tract Infection Among Chinese Students Aged 6-14 Years

AIMS

We employed machine-learning methods to explore data from a large survey on students, with the goal of identifying and validating a thrifty panel of important factors associated with lower respiratory tract infection (LRTI).

METHODS

Cross-sectional cluster sampling was performed for a survey of students aged 6-14 years who attended primary or junior high school in Beijing within January, 2022. Data were collected via electronic questionnaires. Statistical analyses were completed using the PyCharm (Edition 2018.1 x64) and Python (Version 3.7.6).

RESULTS

Data from 11,308 students (5,527 girls and 5,781 boys) were analyzed, and 909 of them had LRTI with the prevalence of 8.01%. After a comprehensive evaluation, the Gaussian naive Bayes (gNB) algorithm outperformed the other machine-learning algorithms. The gNB algorithm had accuracy of 0.856, precision of 0.140, recall of 0.165, F1 score of 0.151, and area under the receiver operating characteristic curve (AUROC) of 0.652. Using the optimal gNB algorithm, top five important factors, including age, rhinitis, sitting time, dental caries, and food or drug allergy, had decent prediction performance. In addition, the top five factors had prediction performance comparable to all factors modeled. For example, under the sequential deep-learning model, the accuracy and loss were separately gauged at 92.26 and 25.62% when incorporating the top five factors, and 92.22 and 25.52% when incorporating all factors.

CONCLUSIONS

Our findings showed the top five important factors modeled by gNB algorithm can sufficiently represent all involved factors in predicting LRTI risk among Chinese students aged 6-14 years.

Multi-view prediction of Alzheimer's disease progression with end-to-end integrated framework

Alzheimer's disease is a common neurodegenerative brain disease that affects the elderly population worldwide. Its early automatic detection is vital for early intervention and treatment. A common solution is to perform future cognitive score prediction based on the baseline brain structural magnetic resonance image (MRI), which can directly infer the potential severity of disease. Recently, several studies have modelled disease progression by predicting the future brain MRI that can provide visual information of brain changes over time. Nevertheless, no studies explore the intra correlation of these two solutions, and it is unknown whether the predicted MRI can assist the prediction of cognitive score. Here, instead of independent prediction, we aim to predict disease progression in multi-view, i.e., predicting subject-specific changes of cognitive score and MRI volume concurrently. To achieve this, we propose an end-to-end integrated framework, where a regression model and a generative adversarial network are integrated together and then jointly optimized. Three integration strategies are exploited to unify these two models. Moreover, considering that some brain regions, such as hippocampus and middle temporal gyrus, could change significantly during the disease progression, a region-of-interest (ROI) mask and a ROI loss are introduced into the integrated framework to leverage this anatomical prior knowledge. Experimental results on the longitudinal Alzheimer's Disease Neuroimaging Initiative dataset demonstrated that the integrated framework outperformed the independent regression model for cognitive score prediction. And its performance can be further improved with the ROI loss for both cognitive score and MRI prediction.

THAN: task-driven hierarchical attention network for the diagnosis of mild cognitive impairment and Alzheimer's disease

BACKGROUND

To assist doctors to diagnose mild cognitive impairment (MCI) and Alzheimer's disease (AD) early and accurately, convolutional neural networks based on structural magnetic resonance imaging (sMRI) images have been developed and shown excellent performance. However, they are still limited in their capacity in extracting discriminative features because of large sMRI image volumes yet small lesion regions and the small number of sMRI images.

METHODS

We proposed a task-driven hierarchical attention network (THAN) taking advantage of the merits of patch-based and attention-based convolutional neural networks for MCI and AD diagnosis. THAN consists of an information sub-network and a hierarchical attention sub-network. In the information sub-network, an information map extractor, a patch-assistant module, and a mutual-boosting loss function are designed to generate a task-driven information map, which automatically highlights disease-related regions and their importance for final classification. In the hierarchical attention sub-network, a visual attention module and a semantic attention module are devised based on the information map to extract discriminative features for disease diagnosis.

RESULTS

Extensive experiments were conducted for four classification tasks: MCI versus (vs.) normal controls (NC), AD vs. NC, AD vs. MCI, and AD vs. MCI vs. NC. Results demonstrated that THAN attained the accuracy of 81.6% for MCI vs. NC, 93.5% for AD vs. NC, 80.8% for AD vs. MCI, and 62.9% for AD vs. MCI vs. NC. It outperformed advanced attention-based and patch-based methods. Moreover, information maps generated by the information sub-network could highlight the potential biomarkers of MCI and AD, such as the hippocampus and ventricles. Furthermore, when the visual and semantic attention modules were combined, the performance of the four tasks was highly improved.

CONCLUSIONS

The information sub-network can automatically highlight the disease-related regions. The hierarchical attention sub-network can extract discriminative visual and semantic features. Through the two sub-networks, THAN fully exploits the visual and semantic features of disease-related regions and meanwhile considers global features of sMRI images, which finally facilitate the diagnosis of MCI and AD.

Joint Landmark and Structure Learning for Automatic Evaluation of Developmental Dysplasia of the Hip

The ultrasound (US) screening of the infant hip is vital for early diagnosis of developmental dysplasia of the hip (DDH). The US diagnosis of DDH refers to measuring alpha and beta angles that quantify hip joint development. These two angles are calculated from key anatomical landmarks and structures of the hip. However, this measurement process is not trivial for sonographers and usually requires a thorough understanding of complex anatomical structures. In this study, we propose a multi-task framework to learn the relationships among landmarks and structures jointly and automatically evaluate DDH. Our multi-task networks are equipped with three novel modules. Firstly, we adopt Mask R-CNN as the basic framework to detect and segment key anatomical structures and add one landmark detection branch to form a new multi-task framework. Secondly, we propose a novel shape similarity loss to refine the incomplete anatomical structure prediction robustly and accurately. Thirdly, we further incorporate the landmark-structure consistent prior to ensure the consistency of the bony rim estimated from the segmented structure and the detected landmark. In our experiments, 1,231 US images of the infant hip from 632 patients are collected, of which 247 images from 126 patients are tested. The average errors in alpha and beta angles are 2.221 and 2.899. About 93% and 85% estimates of alpha and beta angles have errors less than 5 degrees, respectively. Experimental results demonstrate that the proposed method can accurately and robustly realize the automatic evaluation of DDH, showing great potential for clinical application.