Classification of negative and positive 18F-florbetapir brain PET studies in subjective cognitive decline patients using a convolutional neural network

A review of multimodal fusion-based deep learning for Alzheimer's disease

Alzheimer's Disease (AD) as one of the most prevalent neurodegenerative disorders worldwide, characterized by significant memory and cognitive decline in its later stages, severely impacting daily lives. Consequently, early diagnosis and accurate assessment are crucial for delaying disease progression. In recent years, multimodal imaging has gained widespread adoption in AD diagnosis and research, particularly the combined use of Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET). The complementarity of these modalities in structural and metabolic information offers a unique advantage for comprehensive disease understanding and precise diagnosis. With the rapid advancement of deep learning techniques, efficient fusion of MRI and PET multimodal data has emerged as a prominent research focus. This review systematically surveys the latest advancements in deep learning-based multimodal fusion of MRI and PET images for AD research, with a particular focus on studies published in the past five years (2021-2025). It first introduces the main sources of AD-related data, along with data preprocessing and feature extraction methods. Then, it summarizes performance metrics and multimodal fusion techniques. Next, it explores the application of various deep learning models and their variants in multimodal fusion tasks. Finally, it analyzes the key challenges currently faced in the field, including data scarcity and imbalance, inter-institutional data heterogeneity, etc., and discusses potential solutions and future research directions. This review aims to provide systematic guidance for researchers in the field of MRI and PET multimodal fusion, with the ultimate goal of advancing the development of early AD diagnosis and intervention strategies.

AmyloidPETNet: Classification of Amyloid Positivity in Brain PET Imaging Using End-to-End Deep Learning

Background Visual assessment of amyloid PET scans relies on the availability of radiologist expertise, whereas quantification of amyloid burden typically involves MRI for processing and analysis, which can be computationally expensive. Purpose To develop a deep learning model to classify minimally processed brain PET scans as amyloid positive or negative, evaluate its performance on independent data sets and different tracers, and compare it with human visual reads. Materials and Methods This retrospective study used 8476 PET scans (6722 patients) obtained from late 2004 to early 2023 that were analyzed across five different data sets. A deep learning model, AmyloidPETNet, was trained on 1538 scans from 766 patients, validated on 205 scans from 95 patients, and internally tested on 184 scans from 95 patients in the Alzheimer's Disease Neuroimaging Initiative (ADNI) fluorine 18 (18F) florbetapir (FBP) data set. It was tested on ADNI scans using different tracers and scans from independent data sets. Scan amyloid positivity was based on mean cortical standardized uptake value ratio cutoffs. To compare with model performance, each scan from both the Centiloid Project and a subset of the Anti-Amyloid Treatment in Asymptomatic Alzheimer's Disease (A4) study were visually interpreted with a confidence level (low, intermediate, high) of amyloid positivity/negativity. The area under the receiver operating characteristic curve (AUC) and other performance metrics were calculated, and Cohen κ was used to measure physician-model agreement. Results The model achieved an AUC of 0.97 (95% CI: 0.95, 0.99) on test ADNI 18F-FBP scans, which generalized well to 18F-FBP scans from the Open Access Series of Imaging Studies (AUC, 0.95; 95% CI: 0.93, 0.97) and the A4 study (AUC, 0.98; 95% CI: 0.98, 0.98). Model performance was high when applied to data sets with different tracers (AUC ≥ 0.97). Other performance metrics provided converging evidence. Physician-model agreement ranged from fair (Cohen κ = 0.39; 95% CI: 0.16, 0.60) on a sample of mostly equivocal cases from the A4 study to almost perfect (Cohen κ = 0.93; 95% CI: 0.86, 1.0) on the Centiloid Project. Conclusion The developed model was capable of automatically and accurately classifying brain PET scans as amyloid positive or negative without relying on experienced readers or requiring structural MRI. Clinical trial registration no. NCT00106899 © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Bryan and Forghani in this issue.

Automated detection of Alzheimer's disease: a multi-modal approach with 3D MRI and amyloid PET

Recent advances in deep learning and imaging technologies have revolutionized automated medical image analysis, especially in diagnosing Alzheimer's disease through neuroimaging. Despite the availability of various imaging modalities for the same patient, the development of multi-modal models leveraging these modalities remains underexplored. This paper addresses this gap by proposing and evaluating classification models using 2D and 3D MRI images and amyloid PET scans in uni-modal and multi-modal frameworks. Our findings demonstrate that models using volumetric data learn more effective representations than those using only 2D images. Furthermore, integrating multiple modalities enhances model performance over single-modality approaches significantly. We achieved state-of-the-art performance on the OASIS-3 cohort. Additionally, explainability analyses with Grad-CAM indicate that our model focuses on crucial AD-related regions for its predictions, underscoring its potential to aid in understanding the disease's causes.

Alzheimer's disease prediction algorithm based on de-correlation constraint and multi-modal feature interaction

Alzheimer's disease (AD) is a neurodegenerative disease characterized by various pathological changes. Utilizing multimodal data from Fluorodeoxyglucose positron emission tomography(FDG-PET) and Magnetic Resonance Imaging(MRI) of the brain can offer comprehensive information about the lesions from different perspectives and improve the accuracy of prediction. However, there are significant differences in the feature space of multimodal data. Commonly, the simple concatenation of multimodal features can cause the model to struggle in distinguishing and utilizing the complementary information between different modalities, thus affecting the accuracy of predictions. Therefore, we propose an AD prediction model based on de-correlation constraint and multi-modal feature interaction. This model consists of the following three parts: (1) The feature extractor employs residual connections and attention mechanisms to capture distinctive lesion features from FDG-PET and MRI data within their respective modalities. (2) The de-correlation constraint function enhances the model's capacity to extract complementary information from different modalities by reducing the feature similarity between them. (3) The mutual attention feature fusion module interacts with the features within and between modalities to enhance the modal-specific features and adaptively adjust the weights of these features based on information from other modalities. The experimental results on ADNI database demonstrate that the proposed model achieves a prediction accuracy of 86.79% for AD, MCI and NC, which is higher than the existing multi-modal AD prediction models.

Targeted examination of amyloid beta and tau protein accumulation via positron emission tomography for the differential diagnosis of Alzheimer's disease based on the A/T(N) research framework

Alzheimer's disease (AD) is one of the most common neurodegenerative diseases among the older population. Its main pathological features include the abnormal deposition of extracellular amyloid-β plaques and the intracellular neurofibrillary tangles of tau proteins. Its clinical presentation is complex. This review introduces the pathological processes in AD and other common neurodegenerative diseases. It then discusses the positron emission tomography (PET) probes that target amyloid-β plaques and tau proteins for diagnosing AD. According to the A/T(N) research framework, combined targeted amyloid-β and tau protein detection via PET to further improve the diagnostic accuracy of AD. In particular, the properties of the 18F-flortaucipir and 18F-MK6240 tracers-may be more beneficial in helping to differentiate AD from other common neurodegenerative diseases, such as dementia with Lewy bodies, Parkinson's disease dementia, and frontotemporal dementia. Furthermore, the A/T(N) research framework should be used as the clinical diagnosis model of AD in the future.

Real-world prediction of preclinical Alzheimer's disease with a deep generative model

Amyloid positivity is an early indicator of Alzheimer's disease and is necessary to determine the disease. In this study, a deep generative model is utilized to predict the amyloid positivity of cognitively normal individuals using proxy measures, such as structural MRI scans, demographic variables, and cognitive scores, instead of invasive direct measurements. Through its remarkable efficacy in handling imperfect datasets caused by missing data or labels, and imbalanced classes, the model outperforms previous studies and widely used machine learning approaches with an AUROC of 0.8609. Furthermore, this study illuminates the model's adaptability to diverse clinical scenarios, even when feature sets or diagnostic criteria differ from the training data. We identify the brain regions and variables that contribute most to classification, including the lateral occipital lobes, posterior temporal lobe, and APOE ϵ4 allele. Taking advantage of deep generative models, our approach can not only provide inexpensive, non-invasive, and accurate diagnostics for preclinical Alzheimer's disease, but also meet real-world requirements for clinical translation of a deep learning model, including transferability and interpretability.

Estimation of brain amyloid accumulation using deep learning in clinical [11C]PiB PET imaging

INTRODUCTION

Estimation of brain amyloid accumulation is valuable for evaluation of patients with cognitive impairment in both research and clinical routine. The development of high throughput and accurate strategies for the determination of amyloid status could be an important tool in patient selection for clinical trials and amyloid directed treatment. Here, we propose the use of deep learning to quantify amyloid accumulation using standardized uptake value ratio (SUVR) and classify amyloid status based on their PET images.

METHODS

A total of 1309 patients with cognitive impairment scanned with [¹¹C]PIB PET/CT or PET/MRI were included. Two convolutional neural networks (CNNs) for reading-based amyloid status and SUVR prediction were trained using 75% of the PET/CT data. The remaining PET/CT (n = 300) and all PET/MRI (n = 100) data was used for evaluation.

RESULTS

The prevalence of amyloid positive patients was 61%. The amyloid status classification model reproduced the expert reader's classification with 99% accuracy. There was a high correlation between reference and predicted SUVR (R² = 0.96). Both reference and predicted SUVR had an accuracy of 97% compared to expert classification when applying a predetermined SUVR threshold of 1.35 for binary classification of amyloid status.

CONCLUSION

The proposed CNN models reproduced both the expert classification and quantitative measure of amyloid accumulation in a large local dataset. This method has the potential to replace or simplify existing clinical routines and can facilitate fast and accurate classification well-suited for a high throughput pipeline.

Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review

Rational

Deep learning (DL) has demonstrated a remarkable performance in diagnostic imaging for various diseases and modalities and therefore has a high potential to be used as a clinical tool. However, current practice shows low deployment of these algorithms in clinical practice, because DL algorithms lack transparency and trust due to their underlying black-box mechanism. For successful employment, explainable artificial intelligence (XAI) could be introduced to close the gap between the medical professionals and the DL algorithms. In this literature review, XAI methods available for magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET) imaging are discussed and future suggestions are made.

Methods

PubMed, Embase.com and Clarivate Analytics/Web of Science Core Collection were screened. Articles were considered eligible for inclusion if XAI was used (and well described) to describe the behavior of a DL model used in MR, CT and PET imaging.

Results

A total of 75 articles were included of which 54 and 17 articles described post and ad hoc XAI methods, respectively, and 4 articles described both XAI methods. Major variations in performance is seen between the methods. Overall, post hoc XAI lacks the ability to provide class-discriminative and target-specific explanation. Ad hoc XAI seems to tackle this because of its intrinsic ability to explain. However, quality control of the XAI methods is rarely applied and therefore systematic comparison between the methods is difficult.

Conclusion

There is currently no clear consensus on how XAI should be deployed in order to close the gap between medical professionals and DL algorithms for clinical implementation. We advocate for systematic technical and clinical quality assessment of XAI methods. Also, to ensure end-to-end unbiased and safe integration of XAI in clinical workflow, (anatomical) data minimization and quality control methods should be included.

Comparing lesion and feature selections to predict progression in newly diagnosed DLBCL patients with FDG PET/CT radiomics features

PURPOSE

Biomarkers that can accurately predict outcome in DLBCL patients are urgently needed. Radiomics features extracted from baseline [¹⁸F]-FDG PET/CT scans have shown promising results. This study aims to investigate which lesion- and feature-selection approaches/methods resulted in the best prediction of progression after 2 years.

METHODS

A total of 296 patients were included. 485 radiomics features (n = 5 conventional PET, n = 22 morphology, n = 50 intensity, n = 408 texture) were extracted for all individual lesions and at patient level, where all lesions were aggregated into one VOI. 18 features quantifying dissemination were extracted at patient level. Several lesion selection approaches were tested (largest or hottest lesion, patient level [all with/without dissemination], maximum or median of all lesions) and compared to the predictive value of our previously published model. Several data reduction methods were applied (principal component analysis, recursive feature elimination (RFE), factor analysis, and univariate selection). The predictive value of all models was tested using a fivefold cross-validation approach with 50 repeats with and without oversampling, yielding the mean cross-validated AUC (CV-AUC). Additionally, the relative importance of individual radiomics features was determined.

RESULTS

Models with conventional PET and dissemination features showed the highest predictive value (CV-AUC: 0.72-0.75). Dissemination features had the highest relative importance in these models. No lesion selection approach showed significantly higher predictive value compared to our previous model. Oversampling combined with RFE resulted in highest CV-AUCs.

CONCLUSION

Regardless of the applied lesion selection or feature selection approach and feature reduction methods, patient level conventional PET features and dissemination features have the highest predictive value. Trial registration number and date: EudraCT: 2006-005174-42, 01-08-2008.

Deep Learning Analysis Using 18F-FDG PET/CT to Predict Occult Lymph Node Metastasis in Patients With Clinical N0 Lung Adenocarcinoma

Introduction

The aim of this work was to determine the feasibility of using a deep learning approach to predict occult lymph node metastasis (OLM) based on preoperative FDG-PET/CT images in patients with clinical node-negative (cN0) lung adenocarcinoma.

Materials and Methods

Dataset 1 (for training and internal validation) included 376 consecutive patients with cN0 lung adenocarcinoma from our hospital between May 2012 and May 2021. Dataset 2 (for prospective test) used 58 consecutive patients with cN0 lung adenocarcinoma from June 2021 to February 2022 at the same center. Three deep learning models: PET alone, CT alone, and combined model, were developed for the prediction of OLM. The performance of the models was evaluated on internal validation and prospective test in terms of accuracy, sensitivity, specificity, and areas under the receiver operating characteristic curve (AUCs).

Results

The combined model incorporating PET and CT showed the best performance, achieved an AUC of 0.81 [95% confidence interval (CI): 0.61, 1.00] in the prediction of OLM in internal validation set (n = 60) and an AUC of 0.87 (95% CI: 0.75, 0.99) in the prospective test set (n = 58). The model achieved 87.50% sensitivity, 80.00% specificity, and 81.00% accuracy in the internal validation set and achieved 75.00% sensitivity, 88.46% specificity, and 86.60% accuracy in the prospective test set.

Conclusion

This study presented a deep learning approach to enable the prediction of occult nodal involvement based on the PET/CT images before surgery in cN0 lung adenocarcinoma, which would help clinicians select patients who would be suitable for sublobar resection.

Spatial normalization and quantification approaches of PET imaging for neurological disorders

Quantification approaches of positron emission tomography (PET) imaging provide user-independent evaluation of pathophysiological processes in living brains, which have been strongly recommended in clinical diagnosis of neurological disorders. Most PET quantification approaches depend on spatial normalization of PET images to brain template; however, the spatial normalization and quantification approaches have not been comprehensively reviewed. In this review, we introduced and compared PET template-based and magnetic resonance imaging (MRI)-aided spatial normalization approaches. Tracer-specific and age-specific PET brain templates were surveyed between 1999 and 2021 for ¹⁸F-FDG, ¹¹C-PIB, ¹⁸F-Florbetapir, ¹⁸F-THK5317, and etc., as well as adaptive PET template methods. Spatial normalization-based PET quantification approaches were reviewed, including region-of-interest (ROI)-based and voxel-wise quantitative methods. Spatial normalization-based ROI segmentation approaches were introduced, including manual delineation on template, atlas-based segmentation, and multi-atlas approach. Voxel-wise quantification approaches were reviewed, including voxel-wise statistics and principal component analysis. Certain concerns and representative examples of clinical applications were provided for both ROI-based and voxel-wise quantification approaches. At last, a recipe for PET spatial normalization and quantification approaches was concluded to improve diagnosis accuracy of neurological disorders in clinical practice.

Optimization of Automatic Synthesis and Separation of [18F] AV-45 and Quality Control

Objective: Based on the Tracerlab FXF-N platform, a synthesis program and preparative high-performance liquid chromatography (HPLC) purification program edited by us can stably and repeatedly produce [18F] AV-45 without changing the process. The [18F] AV-45 produced meets the main indexes of radiopharmaceutical intravenous preparations.

Methods: The O-toluene sulfonated precursor (1 mg) was subjected to nucleophilic radiofluorination at 115°C in anhydrous dimethyl sulfoxide (DMSO), then the protective group was hydrolyzed by acid. The neutralized reaction mixture was purified through a preparative HPLC then formulated for injection using a C18 purification cartridge. This method yielded a relatively pure [18F] AV-45 product with high specific activity.

Results: Four consecutive radiochemical synthesis operations were carried out in this experiment; the average production time of [18F] AV-45 preparation was 60 min, the radiochemical yield was 14.8 ± 2.1% (n = 4), the radiochemical purity was greater than 95%, and the other important quality control indexes met the requirements of radioactive drugs for intravenous administration.

Conclusion: This experiment was based on the Tracerlab FXF-N platform with the synthesis program and preparative HPLC purification program edited by us. Through screening and optimization of the separation and purification system and the separation and analysis system, as well as automatic radiochemical synthesis and preparation quality control, intravenous [18F] AV-45 was successfully prepared.

3D Convolutional Neural Network-Based Denoising of Low-Count Whole-Body 18F-Fluorodeoxyglucose and 89Zr-Rituximab PET Scans

Acquisition time and injected activity of ¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) PET should ideally be reduced. However, this decreases the signal-to-noise ratio (SNR), which impairs the diagnostic value of these PET scans. In addition, ⁸⁹Zr-antibody PET is known to have a low SNR. To improve the diagnostic value of these scans, a Convolutional Neural Network (CNN) denoising method is proposed. The aim of this study was therefore to develop CNNs to increase SNR for low-count ¹⁸F-FDG and ⁸⁹Zr-antibody PET. Super-low-count, low-count and full-count ¹⁸F-FDG PET scans from 60 primary lung cancer patients and full-count ⁸⁹Zr-rituximab PET scans from five patients with non-Hodgkin lymphoma were acquired. CNNs were built to capture the features and to denoise the PET scans. Additionally, Gaussian smoothing (GS) and Bilateral filtering (BF) were evaluated. The performance of the denoising approaches was assessed based on the tumour recovery coefficient (TRC), coefficient of variance (COV; level of noise), and a qualitative assessment by two nuclear medicine physicians. The CNNs had a higher TRC and comparable or lower COV to GS and BF and was also the preferred method of the two observers for both ¹⁸F-FDG and ⁸⁹Zr-rituximab PET. The CNNs improved the SNR of low-count ¹⁸F-FDG and ⁸⁹Zr-rituximab PET, with almost similar or better clinical performance than the full-count PET, respectively. Additionally, the CNNs showed better performance than GS and BF.

Application of artificial intelligence in brain molecular imaging

Initial development of artificial Intelligence (AI) and machine learning (ML) dates back to the mid-twentieth century. A growing awareness of the potential for AI, as well as increases in computational resources, research, and investment are rapidly advancing AI applications to medical imaging and, specifically, brain molecular imaging. AI/ML can improve imaging operations and decision making, and potentially perform tasks that are not readily possible by physicians, such as predicting disease prognosis, and identifying latent relationships from multi-modal clinical information. The number of applications of image-based AI algorithms, such as convolutional neural network (CNN), is increasing rapidly. The applications for brain molecular imaging (MI) include image denoising, PET and PET/MRI attenuation correction, image segmentation and lesion detection, parametric image formation, and the detection/diagnosis of Alzheimer's disease and other brain disorders. When effectively used, AI will likely improve the quality of patient care, instead of replacing radiologists. A regulatory framework is being developed to facilitate AI adaptation for medical imaging.

The added value of PSMA PET/MR radiomics for prostate cancer staging

PURPOSE

To evaluate the performance of combined PET and multiparametric MRI (mpMRI) radiomics for the group-wise prediction of postsurgical Gleason scores (psGSs) in primary prostate cancer (PCa) patients.

METHODS

Patients with PCa, who underwent [68 Ga]Ga-PSMA-11 PET/MRI followed by radical prostatectomy, were included in this retrospective analysis (n = 101). Patients were grouped by psGS in three categories: ISUP grades 1-3, ISUP grade 4, and ISUP grade 5. mpMRI images included T1-weighted, T2-weighted, and apparent diffusion coefficient (ADC) map. Whole-prostate segmentations were performed on each modality, and image biomarker standardization initiative (IBSI)-compliant radiomic features were extracted. Nine support vector machine (SVM) models were trained: four single-modality radiomic models (PET, T1w, T2w, ADC); three PET + MRI double-modality models (PET + T1w, PET + T2w, PET + ADC), and two baseline models (one with patient data, one image-based) for comparison. A sixfold stratified cross-validation was performed, and balanced accuracies (bAcc) of the predictions of the best-performing models were reported and compared through Student's t-tests. The predictions of the best-performing model were compared against biopsy GS (bGS).

RESULTS

All radiomic models outperformed the baseline models. The best-performing (mean ± stdv [%]) single-modality model was the ADC model (76 ± 6%), although not significantly better (p > 0.05) than other single-modality models (T1w: 72 ± 3%, T2w: 73 ± 2%; PET: 75 ± 5%). The overall best-performing model combined PET + ADC radiomics (82 ± 5%). It significantly outperformed most other double-modality (PET + T1w: 74 ± 5%, p = 0.026; PET + T2w: 71 ± 4%, p = 0.003) and single-modality models (PET: p = 0.042; T1w: p = 0.002; T2w: p = 0.003), except the ADC-only model (p = 0.138). In this initial cohort, the PET + ADC model outperformed bGS overall (82.5% vs 72.4%) in the prediction of psGS.

CONCLUSION

All single- and double-modality models outperformed the baseline models, showing their potential in the prediction of GS, even with an unbalanced cohort. The best-performing model included PET + ADC radiomics, suggesting a complementary value of PSMA-PET and ADC radiomics.