Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification

Visualizing functional network connectivity differences using an explainable machine-learning method

Objective. Functional network connectivity (FNC) estimated from resting-state functional magnetic resonance imaging showed great information about the neural mechanism in different brain disorders. But previous research has mainly focused on standard statistical learning approaches to find FNC features separating patients from control. While machine learning models can improve classification accuracy, they often lack interpretability, making it difficult to understand how they arrive at their decisions.Approach. Explainable machine learning helps address this issue by identifying which features contribute most to the model's predictions. In this study, we introduce a novel framework leveraging SHapley Additive exPlanations (SHAPs) to identify crucial FNC features distinguishing between two distinct population classes.Main results. Initially, we validate our approach using synthetic data. Subsequently, applying our framework, we ascertain FNC biomarkers distinguishing between, controls and schizophrenia (SZ) patients with accuracy of 81.04% as well as middle aged adults and old aged adults with accuracy 71.38%, respectively, employing random forest, XGBoost, and CATBoost models.Significance. Our analysis underscores the pivotal role of the cognitive control network (CCN), subcortical network (SCN), and somatomotor network in discerning individuals with SZ from controls. In addition, our platform found CCN and SCN as the most important networks separating young adults from older.

Predicting the progression of MCI and Alzheimer's disease on structural brain integrity and other features with machine learning

Machine learning (ML) on structural MRI data shows high potential for classifying Alzheimer's disease (AD) progression, but the specific contribution of brain regions, demographics, and proteinopathy remains unclear. Using Alzheimer's Disease Neuroimaging Initiative (ADNI) data, we applied an extreme gradient-boosting algorithm and SHAP (SHapley Additive exPlanations) values to classify cognitively normal (CN) older adults, those with mild cognitive impairment (MCI) and AD dementia patients. Features included structural MRI, CSF status, demographics, and genetic data. Analyses comprised one cross-sectional multi-class classification (CN vs. MCI vs. AD dementia, n = 568) and two longitudinal binary-class classifications (CN-to-MCI converters vs. CN stable, n = 92; MCI-to-AD converters vs. MCI stable, n = 378). All classifications achieved 70-77% accuracy and 61-83% precision. Key features were CSF status, hippocampal volume, entorhinal thickness, and amygdala volume, with a clear dissociation: hippocampal properties contributed to the conversion to MCI, while the entorhinal cortex characterized the conversion to AD dementia. The findings highlight explainable, trajectory-specific insights into AD progression.

Multimodal MRI accurately identifies amyloid status in unbalanced cohorts in Alzheimer's disease continuum

Amyloid-β (Aβ) plaques in conjunction with hyperphosphorylated tau proteins in the form of neurofibrillary tangles are the two neuropathological hallmarks of Alzheimer's disease. It is well-known that the identification of individuals with Aβ positivity could enable early diagnosis. In this work, we aim at capturing the Aβ positivity status in an unbalanced cohort enclosing subjects at different disease stages, exploiting the underlying structural and connectivity disease-induced modulations as revealed by structural, functional, and diffusion MRI. Of note, due to the unbalanced cohort, the outcomes may be guided by those factors rather than amyloid accumulation. The partial views provided by each modality are integrated in the model, allowing to take full advantage of their complementarity in encoding the effects of the Aβ accumulation, leading to an accuracy of 0.762 ± 0.04. The specificity of the information brought by each modality is assessed by post hoc explainability analysis (guided backpropagation), highlighting the underlying structural and functional changes. Noteworthy, well-established biomarker key regions related to Aβ deposition could be identified by all modalities, including the hippocampus, thalamus, precuneus, and cingulate gyrus, witnessing in favor of the reliability of the method as well as its potential in shedding light on modality-specific possibly unknown Aβ deposition signatures.

InGSA: integrating generalized self-attention in CNN for Alzheimer's disease classification

Alzheimer's disease (AD) is an incurable neurodegenerative disorder that slowly impair the mental abilities. Early diagnosis, nevertheless, can greatly reduce the symptoms that are associated with the condition. Earlier techniques of diagnosing the AD from the MRI scans have been adopted by traditional machine learning technologies. However, such traditional methods involve depending on feature extraction that is usually complex, time-consuming, and requiring substantial effort from the medical personnel. Furthermore, these methods are usually not very specific as far as diagnosis is concerned. In general, traditional convolutional neural network (CNN) architectures have a problem with identifying AD. To this end, the developed framework consists of a new contrast enhancement approach, named haze-reduced local-global (HRLG). For multiclass AD classification, we introduce a global CNN-transformer model InGSA. The proposed InGSA is based on the InceptionV3 model which is pre-trained, and it encompasses an additional generalized self-attention (GSA) block at top of the network. This GSA module is capable of capturing the interaction not only in terms of the spatial relations within the feature space but also over the channel dimension it is capable of picking up fine detailing of the AD information while suppressing the noise. Furthermore, several GSA heads are used to exploit other dependency structures of global features as well. Our evaluation of InGSA on a two benchmark dataset, using various pre-trained networks, demonstrates the GSA's superior performance.

Explainable Artificial Intelligence in Neuroimaging of Alzheimer's Disease

Alzheimer's disease (AD) remains a significant global health challenge, affecting millions worldwide and imposing substantial burdens on healthcare systems. Advances in artificial intelligence (AI), particularly in deep learning and machine learning, have revolutionized neuroimaging-based AD diagnosis. However, the complexity and lack of interpretability of these models limit their clinical applicability. Explainable Artificial Intelligence (XAI) addresses this challenge by providing insights into model decision-making, enhancing transparency, and fostering trust in AI-driven diagnostics. This review explores the role of XAI in AD neuroimaging, highlighting key techniques such as SHAP, LIME, Grad-CAM, and Layer-wise Relevance Propagation (LRP). We examine their applications in identifying critical biomarkers, tracking disease progression, and distinguishing AD stages using various imaging modalities, including MRI and PET. Additionally, we discuss current challenges, including dataset limitations, regulatory concerns, and standardization issues, and propose future research directions to improve XAI's integration into clinical practice. By bridging the gap between AI and clinical interpretability, XAI holds the potential to refine AD diagnostics, personalize treatment strategies, and advance neuroimaging-based research.

Design and methods of the research unit 5187 PREACT (towards precision psychotherapy for non-respondent patients: from signatures to predictions to clinical utility) - a study protocol for a multicentre observational study in outpatient clinics

INTRODUCTION

Cognitive-behavioural therapy (CBT) works-but not equally well for all patients. Less than 50% of patients with internalising disorders achieve clinically meaningful improvement, with negative consequences for patients and healthcare systems. The research unit (RU) 5187 seeks to improve this situation by an in-depth investigation of the phenomenon of treatment non-response (TNR) to CBT. We aim to identify bio-behavioural signatures associated with TNR, develop predictive models applicable to individual patients and enhance the utility of predictive analytics by collecting a naturalistic cohort with high ecological validity for the outpatient sector.

METHODS AND ANALYSIS

The RU is composed of nine subprojects (SPs), spanning from clinical, machine learning and neuroimaging science and service projects to particular research questions on psychological, electrophysiological/autonomic, digital and neural signatures of TNR. The clinical study SP 1 comprises a four-centre, prospective-longitudinal observational trial where we recruit a cohort of 585 patients with a wide range of internalising disorders (specific phobia, social anxiety disorder, panic disorder, agoraphobia, generalised anxiety disorder, obsessive-compulsive disorder, post-traumatic stress disorder, and unipolar depressive disorders) using minimal exclusion criteria. Our experimental focus lies on emotion (dys)-regulation as a putative key mechanism of CBT and TNR. We use state-of-the-art machine learning methods to achieve single-patient predictions, incorporating pretrained convolutional neural networks for high-dimensional neuroimaging data and multiple kernel learning to integrate information from various modalities. The RU aims to advance precision psychotherapy by identifying emotion regulation-based biobehavioural markers of TNR, setting up a multilevel assessment for optimal predictors and using an ecologically valid sample to apply findings in diverse clinical settings, thereby addressing the needs of vulnerable patients.

ETHICS AND DISSEMINATION

The study has received ethical approval from the Institutional Ethics Committee of the Department of Psychology at Humboldt-Universität zu Berlin (approval no. 2021-01) and the Ethics Committee of Charité-Universitätsmedizin Berlin (approval no. EA1/186/22).Results will be disseminated through peer-reviewed journals and presentations at national and international conferences. Deidentified data and analysis scripts will be made available to researchers within the RU via a secure server, in line with ethical guidelines and participant consent. In compliance with European and German data protection regulations, patient data will not be publicly available through open science frameworks but may be shared with external researchers on reasonable request and under appropriate data protection agreements.

TRIAL REGISTRATION NUMBER

DRKS00030915.

Contrastive self-supervised learning for neurodegenerative disorder classification

Introduction

Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels.

Methods

We investigated if the SSL models can be applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network, trained with a contrastive loss, serves as the feature extractor that learns latent representations. The classification head is a single-layer perceptron that is trained to perform diagnostic group separation. We used N = 2,694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its phenotypes.

Results

Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV.

Conclusion

Our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable.

Intelligent skin disease prediction system using transfer learning and explainable artificial intelligence

Skin diseases impact millions of people around the world and pose a severe risk to public health. These diseases have a wide range of effects on the skin's structure, functionality, and appearance. Identifying and predicting skin diseases are laborious processes that require a complete physical examination, a review of the patient's medical history, and proper laboratory diagnostic testing. Additionally, it necessitates a significant number of histological and clinical characteristics for examination and subsequent treatment. As a disease's complexity and quantity of features grow, identifying and predicting it becomes more challenging. This research proposes a deep learning (DL) model utilizing transfer learning (TL) to quickly identify skin diseases like chickenpox, measles, and monkeypox. A pre-trained VGG16 is used for transfer learning. The VGG16 can identify and predict diseases more quickly by learning symptom patterns. Images of the skin from the four classes of chickenpox, measles, monkeypox, and normal are included in the dataset. The dataset is separated into training and testing. The experimental results performed on the dataset demonstrate that the VGG16 model can identify and predict skin diseases with 93.29% testing accuracy. However, the VGG16 model does not explain why and how the system operates because deep learning models are black boxes. Deep learning models' opacity stands in the way of their widespread application in the healthcare sector. In order to make this a valuable system for the health sector, this article employs layer-wise relevance propagation (LRP) to determine the relevance scores of each input. The identified symptoms provide valuable insights that could support timely diagnosis and treatment decisions for skin diseases.

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

This systematic literature review examines state-of-the-art Explainable Artificial Intelligence (XAI) methods applied to medical image analysis, discussing current challenges and future research directions, and exploring evaluation metrics used to assess XAI approaches. With the growing efficiency of Machine Learning (ML) and Deep Learning (DL) in medical applications, there's a critical need for adoption in healthcare. However, their "black-box" nature, where decisions are made without clear explanations, hinders acceptance in clinical settings where decisions have significant medicolegal consequences. Our review highlights the advanced XAI methods, identifying how they address the need for transparency and trust in ML/DL decisions. We also outline the challenges faced by these methods and propose future research directions to improve XAI in healthcare. This paper aims to bridge the gap between cutting-edge computational techniques and their practical application in healthcare, nurturing a more transparent, trustworthy, and effective use of AI in medical settings. The insights guide both research and industry, promoting innovation and standardisation in XAI implementation in healthcare.

AI for the prediction of early stages of Alzheimer's disease from neuroimaging biomarkers - A narrative review of a growing field

OBJECTIVES

The objectives of this narrative review are to summarize the current state of AI applications in neuroimaging for early Alzheimer's disease (AD) prediction and to highlight the potential of AI techniques in improving early AD diagnosis, prognosis, and management.

METHODS

We conducted a narrative review of studies using AI techniques applied to neuroimaging data for early AD prediction. We examined single-modality studies using structural MRI and PET imaging, as well as multi-modality studies integrating multiple neuroimaging techniques and biomarkers. Furthermore, they reviewed longitudinal studies that model AD progression and identify individuals at risk of rapid decline.

RESULTS

Single-modality studies using structural MRI and PET imaging have demonstrated high accuracy in classifying AD and predicting progression from mild cognitive impairment (MCI) to AD. Multi-modality studies, integrating multiple neuroimaging techniques and biomarkers, have shown improved performance and robustness compared to single-modality approaches. Longitudinal studies have highlighted the value of AI in modeling AD progression and identifying individuals at risk of rapid decline. However, challenges remain in data standardization, model interpretability, generalizability, clinical integration, and ethical considerations.

CONCLUSION

AI techniques applied to neuroimaging data have the potential to improve early AD diagnosis, prognosis, and management. Addressing challenges related to data standardization, model interpretability, generalizability, clinical integration, and ethical considerations is crucial for realizing the full potential of AI in AD research and clinical practice. Collaborative efforts among researchers, clinicians, and regulatory agencies are needed to develop reliable, robust, and ethical AI tools that can benefit AD patients and society.

A modified deep learning method for Alzheimer's disease detection based on the facial submicroscopic features in mice

Alzheimer's disease (AD) is a chronic disease among people aged 65 and older. As the aging population continues to grow at a rapid pace, AD has emerged as a pressing public health issue globally. Early detection of the disease is important, because increasing evidence has illustrated that early diagnosis holds the key to effective treatment of AD. In this work, we developed and refined a multi-layer cyclic Residual convolutional neural network model, specifically tailored to identify AD-related submicroscopic characteristics in the facial images of mice. Our experiments involved classifying the mice into two distinct groups: a normal control group and an AD group. Compared with the other deep learning models, the proposed model achieved a better detection performance in the dataset of the mouse experiment. The accuracy, sensitivity, specificity and precision for AD identification with our proposed model were as high as 99.78%, 100%, 99.65% and 99.44%, respectively. Moreover, the heat maps of AD correlation in the facial images of the mice were acquired with the class activation mapping algorithm. It was proven that the facial images contained AD-related submicroscopic features. Consequently, through our mouse experiments, we validated the feasibility and accuracy of utilizing a facial image-based deep learning model for AD identification. Therefore, the present study suggests the potential of using facial images for AD detection in humans through deep learning-based methods.

Explainable Machine Learning Models for Brain Diseases: Insights from a Systematic Review

In recent years, Artificial Intelligence (AI) methods, specifically Machine Learning (ML) models, have been providing outstanding results in different areas of knowledge, with the health area being one of its most impactful fields of application. However, to be applied reliably, these models must provide users with clear, simple, and transparent explanations about the medical decision-making process. This systematic review aims to investigate the use and application of explainability in ML models used in brain disease studies. A systematic search was conducted in three major bibliographic databases, Web of Science, Scopus, and PubMed, from January 2014 to December 2023. A total of 133 relevant studies were identified and analyzed out of a total of 682 found in the initial search, in which the explainability of ML models in the medical context was studied, identifying 11 ML models and 12 explainability techniques applied in the study of 20 brain diseases.

A Survey on Explainable Artificial Intelligence (XAI) Techniques for Visualizing Deep Learning Models in Medical Imaging

The combination of medical imaging and deep learning has significantly improved diagnostic and prognostic capabilities in the healthcare domain. Nevertheless, the inherent complexity of deep learning models poses challenges in understanding their decision-making processes. Interpretability and visualization techniques have emerged as crucial tools to unravel the black-box nature of these models, providing insights into their inner workings and enhancing trust in their predictions. This survey paper comprehensively examines various interpretation and visualization techniques applied to deep learning models in medical imaging. The paper reviews methodologies, discusses their applications, and evaluates their effectiveness in enhancing the interpretability, reliability, and clinical relevance of deep learning models in medical image analysis.

Advanced interpretable diagnosis of Alzheimer's disease using SECNN-RF framework with explainable AI

Early detection of Alzheimer's disease (AD) is vital for effective treatment, as interventions are most successful in the disease's early stages. Combining Magnetic Resonance Imaging (MRI) with artificial intelligence (AI) offers significant potential for enhancing AD diagnosis. However, traditional AI models often lack transparency in their decision-making processes. Explainable Artificial Intelligence (XAI) is an evolving field that aims to make AI decisions understandable to humans, providing transparency and insight into AI systems. This research introduces the Squeeze-and-Excitation Convolutional Neural Network with Random Forest (SECNN-RF) framework for early AD detection using MRI scans. The SECNN-RF integrates Squeeze-and-Excitation (SE) blocks into a Convolutional Neural Network (CNN) to focus on crucial features and uses Dropout layers to prevent overfitting. It then employs a Random Forest classifier to accurately categorize the extracted features. The SECNN-RF demonstrates high accuracy (99.89%) and offers an explainable analysis, enhancing the model's interpretability. Further exploration of the SECNN framework involved substituting the Random Forest classifier with other machine learning algorithms like Decision Tree, XGBoost, Support Vector Machine, and Gradient Boosting. While all these classifiers improved model performance, Random Forest achieved the highest accuracy, followed closely by XGBoost, Gradient Boosting, Support Vector Machine, and Decision Tree which achieved lower accuracy.

Four Transformer-Based Deep Learning Classifiers Embedded with an Attention U-Net-Based Lung Segmenter and Layer-Wise Relevance Propagation-Based Heatmaps for COVID-19 X-ray Scans

Background: Diagnosing lung diseases accurately is crucial for proper treatment. Convolutional neural networks (CNNs) have advanced medical image processing, but challenges remain in their accurate explainability and reliability. This study combines U-Net with attention and Vision Transformers (ViTs) to enhance lung disease segmentation and classification. We hypothesize that Attention U-Net will enhance segmentation accuracy and that ViTs will improve classification performance. The explainability methodologies will shed light on model decision-making processes, aiding in clinical acceptance. Methodology: A comparative approach was used to evaluate deep learning models for segmenting and classifying lung illnesses using chest X-rays. The Attention U-Net model is used for segmentation, and architectures consisting of four CNNs and four ViTs were investigated for classification. Methods like Gradient-weighted Class Activation Mapping plus plus (Grad-CAM++) and Layer-wise Relevance Propagation (LRP) provide explainability by identifying crucial areas influencing model decisions. Results: The results support the conclusion that ViTs are outstanding in identifying lung disorders. Attention U-Net obtained a Dice Coefficient of 98.54% and a Jaccard Index of 97.12%. ViTs outperformed CNNs in classification tasks by 9.26%, reaching an accuracy of 98.52% with MobileViT. An 8.3% increase in accuracy was seen while moving from raw data classification to segmented image classification. Techniques like Grad-CAM++ and LRP provided insights into the decision-making processes of the models. Conclusions: This study highlights the benefits of integrating Attention U-Net and ViTs for analyzing lung diseases, demonstrating their importance in clinical settings. Emphasizing explainability clarifies deep learning processes, enhancing confidence in AI solutions and perhaps enhancing clinical acceptance for improved healthcare results.

Contrastive Self-supervised Learning for Neurodegenerative Disorder Classification

Neurodegenerative diseases such as Alzheimer's disease (AD) or frontotemporal lobar degeneration (FTLD) involve specific loss of brain volume, detectable in vivo using T1-weighted MRI scans. Supervised machine learning approaches classifying neurodegenerative diseases require diagnostic-labels for each sample. However, it can be difficult to obtain expert labels for a large amount of data. Self-supervised learning (SSL) offers an alternative for training machine learning models without data-labels. We investigated if the SSL models can applied to distinguish between different neurodegenerative disorders in an interpretable manner. Our method comprises a feature extractor and a downstream classification head. A deep convolutional neural network trained in a contrastive self-supervised way serves as the feature extractor, learning latent representation, while the classifier head is a single-layer perceptron. We used N=2694 T1-weighted MRI scans from four data cohorts: two ADNI datasets, AIBL and FTLDNI, including cognitively normal controls (CN), cases with prodromal and clinical AD, as well as FTLD cases differentiated into its sub-types. Our results showed that the feature extractor trained in a self-supervised way provides generalizable and robust representations for the downstream classification. For AD vs. CN, our model achieves 82% balanced accuracy on the test subset and 80% on an independent holdout dataset. Similarly, the Behavioral variant of frontotemporal dementia (BV) vs. CN model attains an 88% balanced accuracy on the test subset. The average feature attribution heatmaps obtained by the Integrated Gradient method highlighted hallmark regions, i.e., temporal gray matter atrophy for AD, and insular atrophy for BV. In conclusion, our models perform comparably to state-of-the-art supervised deep learning approaches. This suggests that the SSL methodology can successfully make use of unannotated neuroimaging datasets as training data while remaining robust and interpretable.

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Background

Schizophrenia (SZ) is a psychiatric condition that adversely affects an individual's cognitive, emotional, and behavioral aspects. The etiology of SZ, although extensively studied, remains unclear, as multiple factors come together to contribute toward its development. There is a consistent body of evidence documenting the presence of structural and functional deviations in the brains of individuals with SZ. Moreover, the hereditary aspect of SZ is supported by the significant involvement of genomics markers. Therefore, the need to investigate SZ from a multi-modal perspective and develop approaches for improved detection arises.

Methods

Our proposed method employed a deep learning framework combining features from structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and genetic markers such as single nucleotide polymorphism (SNP). For sMRI, we used a pre-trained DenseNet to extract the morphological features. To identify the most relevant functional connections in fMRI and SNPs linked to SZ, we applied a 1-dimensional convolutional neural network (CNN) followed by layerwise relevance propagation (LRP). Finally, we concatenated these obtained features across modalities and fed them to the extreme gradient boosting (XGBoost) tree-based classifier to classify SZ from healthy control (HC).

Results

Experimental evaluation on clinical dataset demonstrated that, compared to the outcomes obtained from each modality individually, our proposed multi-modal approach performed classification of SZ individuals from HC with an improved accuracy of 79.01%.

Conclusion

We proposed a deep learning based framework that selects multi-modal (sMRI, fMRI and genetic) features efficiently and fuse them to obtain improved classification scores. Additionally, by using Explainable AI (XAI), we were able to pinpoint and validate significant functional network connections and SNPs that contributed the most toward SZ classification, providing necessary interpretation behind our findings.

Saliency-driven explainable deep learning in medical imaging: bridging visual explainability and statistical quantitative analysis

Deep learning shows great promise for medical image analysis but often lacks explainability, hindering its adoption in healthcare. Attribution techniques that explain model reasoning can potentially increase trust in deep learning among clinical stakeholders. In the literature, much of the research on attribution in medical imaging focuses on visual inspection rather than statistical quantitative analysis.In this paper, we proposed an image-based saliency framework to enhance the explainability of deep learning models in medical image analysis. We use adaptive path-based gradient integration, gradient-free techniques, and class activation mapping along with its derivatives to attribute predictions from brain tumor MRI and COVID-19 chest X-ray datasets made by recent deep convolutional neural network models.The proposed framework integrates qualitative and statistical quantitative assessments, employing Accuracy Information Curves (AICs) and Softmax Information Curves (SICs) to measure the effectiveness of saliency methods in retaining critical image information and their correlation with model predictions. Visual inspections indicate that methods such as ScoreCAM, XRAI, GradCAM, and GradCAM++ consistently produce focused and clinically interpretable attribution maps. These methods highlighted possible biomarkers, exposed model biases, and offered insights into the links between input features and predictions, demonstrating their ability to elucidate model reasoning on these datasets. Empirical evaluations reveal that ScoreCAM and XRAI are particularly effective in retaining relevant image regions, as reflected in their higher AUC values. However, SICs highlight variability, with instances of random saliency masks outperforming established methods, emphasizing the need for combining visual and empirical metrics for a comprehensive evaluation.The results underscore the importance of selecting appropriate saliency methods for specific medical imaging tasks and suggest that combining qualitative and quantitative approaches can enhance the transparency, trustworthiness, and clinical adoption of deep learning models in healthcare. This study advances model explainability to increase trust in deep learning among healthcare stakeholders by revealing the rationale behind predictions. Future research should refine empirical metrics for stability and reliability, include more diverse imaging modalities, and focus on improving model explainability to support clinical decision-making.

Explainability of three-dimensional convolutional neural networks for functional magnetic resonance imaging of Alzheimer's disease classification based on gradient-weighted class activation mapping

Currently, numerous studies focus on employing fMRI-based deep neural networks to diagnose neurological disorders such as Alzheimer's Disease (AD), yet only a handful have provided results regarding explainability. We address this gap by applying several prevalent explainability methods such as gradient-weighted class activation mapping (Grad-CAM) to an fMRI-based 3D-VGG16 network for AD diagnosis to improve the model's explainability. The aim is to explore the specific Region of Interest (ROI) of brain the model primarily focuses on when making predictions, as well as whether there are differences in these ROIs between AD and normal controls (NCs). First, we utilized multiple resting-state functional activity maps including ALFF, fALFF, ReHo, and VMHC to reduce the complexity of fMRI data, which differed from many studies that utilized raw fMRI data. Compared to methods utilizing raw fMRI data, this manual feature extraction approach may potentially alleviate the model's burden. Subsequently, 3D-VGG16 were employed for AD classification, where the final fully connected layers were replaced with a Global Average Pooling (GAP) layer, aimed at mitigating overfitting while preserving spatial information within the feature maps. The model achieved a maximum of 96.4% accuracy on the test set. Finally, several 3D CAM methods were employed to interpret the models. In the explainability results of the models with relatively high accuracy, the highlighted ROIs were primarily located in the precuneus and the hippocampus for AD subjects, while the models focused on the entire brain for NC. This supports current research on ROIs involved in AD. We believe that explaining deep learning models would not only provide support for existing research on brain disorders, but also offer important referential recommendations for the study of currently unknown etiologies.

Constructing personalized characterizations of structural brain aberrations in patients with dementia using explainable artificial intelligence

Deep learning approaches for clinical predictions based on magnetic resonance imaging data have shown great promise as a translational technology for diagnosis and prognosis in neurological disorders, but its clinical impact has been limited. This is partially attributed to the opaqueness of deep learning models, causing insufficient understanding of what underlies their decisions. To overcome this, we trained convolutional neural networks on structural brain scans to differentiate dementia patients from healthy controls, and applied layerwise relevance propagation to procure individual-level explanations of the model predictions. Through extensive validations we demonstrate that deviations recognized by the model corroborate existing knowledge of structural brain aberrations in dementia. By employing the explainable dementia classifier in a longitudinal dataset of patients with mild cognitive impairment, we show that the spatially rich explanations complement the model prediction when forecasting transition to dementia and help characterize the biological manifestation of disease in the individual brain. Overall, our work exemplifies the clinical potential of explainable artificial intelligence in precision medicine.

VespAI: a deep learning-based system for the detection of invasive hornets

The invasive hornet Vespa velutina nigrithorax is a rapidly proliferating threat to pollinators in Europe and East Asia. To effectively limit its spread, colonies must be detected and destroyed early in the invasion curve, however the current reliance upon visual alerts by the public yields low accuracy. Advances in deep learning offer a potential solution to this, but the application of such technology remains challenging. Here we present VespAI, an automated system for the rapid detection of V. velutina. We leverage a hardware-assisted AI approach, combining a standardised monitoring station with deep YOLOv5s architecture and a ResNet backbone, trained on a bespoke end-to-end pipeline. This enables the system to detect hornets in real-time-achieving a mean precision-recall score of ≥0.99-and send associated image alerts via a compact remote processor. We demonstrate the successful operation of a prototype system in the field, and confirm its suitability for large-scale deployment in future use cases. As such, VespAI has the potential to transform the way that invasive hornets are managed, providing a robust early warning system to prevent ingressions into new regions.

Secure federated learning for Alzheimer's disease detection

Machine Learning (ML) is considered a promising tool to aid and accelerate diagnosis in various medical areas, including neuroimaging. However, its success is set back by the lack of large-scale public datasets. Indeed, medical institutions possess a large amount of data; however, open-sourcing is prevented by the legal requirements to protect the patient's privacy. Federated Learning (FL) is a viable alternative that can overcome this issue. This work proposes training an ML model for Alzheimer's Disease (AD) detection based on structural MRI (sMRI) data in a federated setting. We implement two aggregation algorithms, Federated Averaging (FedAvg) and Secure Aggregation (SecAgg), and compare their performance with the centralized ML model training. We simulate heterogeneous environments and explore the impact of demographical (sex, age, and diagnosis) and imbalanced data distributions. The simulated heterogeneous environments allow us to observe these statistical differences' effect on the ML models trained using FL and highlight the importance of studying such differences when training ML models for AD detection. Moreover, as part of the evaluation, we demonstrate the increased privacy guarantees of FL with SecAgg via simulated membership inference attacks.

Sparse Deep Neural Network for Encoding and Decoding the Structural Connectome

Brain state classification by applying deep learning techniques on neuroimaging data has become a recent topic of research. However, unlike domains where the data is low dimensional or there are large number of available training samples, neuroimaging data is high dimensional and has few training samples. To tackle these issues, we present a sparse feedforward deep neural architecture for encoding and decoding the structural connectome of the human brain. We use a sparsely connected element-wise multiplication as the first hidden layer and a fixed transform layer as the output layer. The number of trainable parameters and the training time is significantly reduced compared to feedforward networks. We demonstrate superior performance of this architecture in encoding the structural connectome implicated in Alzheimer's disease (AD) and Parkinson's disease (PD) from DTI brain scans. For decoding, we propose recursive feature elimination (RFE) algorithm based on DeepLIFT, layer-wise relevance propagation (LRP), and Integrated Gradients (IG) algorithms to remove irrelevant features and thereby identify key biomarkers associated with AD and PD. We show that the proposed architecture reduces 45.1% and 47.1% of the trainable parameters compared to a feedforward DNN with an increase in accuracy by 2.6 % and 3.1% for cognitively normal (CN) vs AD and CN vs PD classification, respectively. We also show that the proposed RFE method leads to a further increase in accuracy by 2.1% and 4% for CN vs AD and CN vs PD classification, while removing approximately 90% to 95% irrelevant features. Furthermore, we argue that the biomarkers (i.e., key brain regions and connections) identified are consistent with previous literature. We show that relevancy score-based methods can yield high discriminative power and are suitable for brain decoding. We also show that the proposed approach led to a reduction in the number of trainable network parameters, an increase in classification accuracy, and a detection of brain connections and regions that were consistent with earlier studies.

Explaining decisions of a light-weight deep neural network for real-time coronary artery disease classification in magnetic resonance imaging

In certain healthcare settings, such as emergency or critical care units, where quick and accurate real-time analysis and decision-making are required, the healthcare system can leverage the power of artificial intelligence (AI) models to support decision-making and prevent complications. This paper investigates the optimization of healthcare AI models based on time complexity, hyper-parameter tuning, and XAI for a classification task. The paper highlights the significance of a lightweight convolutional neural network (CNN) for analysing and classifying Magnetic Resonance Imaging (MRI) in real-time and is compared with CNN-RandomForest (CNN-RF). The role of hyper-parameter is also examined in finding optimal configurations that enhance the model's performance while efficiently utilizing the limited computational resources. Finally, the benefits of incorporating the XAI technique (e.g. GradCAM and Layer-wise Relevance Propagation) in providing transparency and interpretable explanations of AI model predictions, fostering trust, and error/bias detection are explored. Our inference time on a MacBook laptop for 323 test images of size 100x100 is only 2.6 sec, which is merely 8 milliseconds per image while providing comparable classification accuracy with the ensemble model of CNN-RF classifiers. Using the proposed model, clinicians/cardiologists can achieve accurate and reliable results while ensuring patients' safety and answering questions imposed by the General Data Protection Regulation (GDPR). The proposed investigative study will advance the understanding and acceptance of AI systems in connected healthcare settings.

Prediction of estimated risk for bipolar disorder using machine learning and structural MRI features

BACKGROUND

Individuals with bipolar disorder are commonly correctly diagnosed a decade after symptom onset. Machine learning techniques may aid in early recognition and reduce the disease burden. As both individuals at risk and those with a manifest disease display structural brain markers, structural magnetic resonance imaging may provide relevant classification features.

METHODS

Following a pre-registered protocol, we trained linear support vector machine (SVM) to classify individuals according to their estimated risk for bipolar disorder using regional cortical thickness of help-seeking individuals from seven study sites (N = 276). We estimated the risk using three state-of-the-art assessment instruments (BPSS-P, BARS, EPIbipolar).

RESULTS

For BPSS-P, SVM achieved a fair performance of Cohen's κ of 0.235 (95% CI 0.11-0.361) and a balanced accuracy of 63.1% (95% CI 55.9-70.3) in the 10-fold cross-validation. In the leave-one-site-out cross-validation, the model performed with a Cohen's κ of 0.128 (95% CI -0.069 to 0.325) and a balanced accuracy of 56.2% (95% CI 44.6-67.8). BARS and EPIbipolar could not be predicted. In post hoc analyses, regional surface area, subcortical volumes as well as hyperparameter optimization did not improve the performance.

CONCLUSIONS

Individuals at risk for bipolar disorder, as assessed by BPSS-P, display brain structural alterations that can be detected using machine learning. The achieved performance is comparable to previous studies which attempted to classify patients with manifest disease and healthy controls. Unlike previous studies of bipolar risk, our multicenter design permitted a leave-one-site-out cross-validation. Whole-brain cortical thickness seems to be superior to other structural brain features.

Identifying subgroups of eating behavior traits unrelated to obesity using functional connectivity and feature representation learning

Eating behavior is highly heterogeneous across individuals and cannot be fully explained using only the degree of obesity. We utilized unsupervised machine learning and functional connectivity measures to explore the heterogeneity of eating behaviors measured by a self-assessment instrument using 424 healthy adults (mean ± standard deviation [SD] age = 47.07 ± 18.89 years; 67% female). We generated low-dimensional representations of functional connectivity using resting-state functional magnetic resonance imaging and estimated latent features using the feature representation capabilities of an autoencoder by nonlinearly compressing the functional connectivity information. The clustering approaches applied to latent features identified three distinct subgroups. The subgroups exhibited different levels of hunger traits, while their body mass indices were comparable. The results were replicated in an independent dataset consisting of 212 participants (mean ± SD age = 38.97 ± 19.80 years; 35% female). The model interpretation technique of integrated gradients revealed that the between-group differences in the integrated gradient maps were associated with functional reorganization in heteromodal association and limbic cortices and reward-related subcortical structures such as the accumbens, amygdala, and caudate. The cognitive decoding analysis revealed that these systems are associated with reward- and emotion-related systems. Our findings provide insights into the macroscopic brain organization of eating behavior-related subgroups independent of obesity.

Predictive modelling of brain disorders with magnetic resonance imaging: A systematic review of modelling practices, transparency, and interpretability in the use of convolutional neural networks

Brain disorders comprise several psychiatric and neurological disorders which can be characterized by impaired cognition, mood alteration, psychosis, depressive episodes, and neurodegeneration. Clinical diagnoses primarily rely on a combination of life history information and questionnaires, with a distinct lack of discriminative biomarkers in use for psychiatric disorders. Symptoms across brain conditions are associated with functional alterations of cognitive and emotional processes, which can correlate with anatomical variation; structural magnetic resonance imaging (MRI) data of the brain are therefore an important focus of research, particularly for predictive modelling. With the advent of large MRI data consortia (such as the Alzheimer's Disease Neuroimaging Initiative) facilitating a greater number of MRI-based classification studies, convolutional neural networks (CNNs)-deep learning models well suited to image processing tasks-have become increasingly popular for research into brain conditions. This has resulted in a myriad of studies reporting impressive predictive performances, demonstrating the potential clinical value of deep learning systems. However, methodologies can vary widely across studies, making them difficult to compare and/or reproduce, potentially limiting their clinical application. Here, we conduct a qualitative systematic literature review of 55 studies carrying out CNN-based predictive modelling of brain disorders using MRI data and evaluate them based on three principles-modelling practices, transparency, and interpretability. We propose several recommendations to enhance the potential for the integration of CNNs into clinical care.

A multimodal screening system for elderly neurological diseases based on deep learning

In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that capture action information with a much smaller data dimension. We also used voice data which are also effective indicators of neurological disorders. We designed a subnetwork for each protocol to extract features from landmarks or voice and a feature aggregator that combines all the information extracted from the protocols to make a final decision. Multitask learning was applied to screen two neurological diseases. To capture meaningful information about these human landmarks and voices, we applied various pre-trained models to extract preliminary features. The spatiotemporal characteristics of landmarks are extracted using a pre-trained graph neural network, and voice features are extracted using a pre-trained time-delay neural network. These extracted high-level features are then passed onto the subnetworks and an additional feature aggregator that are simultaneously trained. We also used various data augmentation techniques to overcome the shortage of data. Using a frame-length staticizer that considers the characteristics of the data, we can capture momentary tremors without wasting information. Finally, we examine the effectiveness of different protocols and different modalities (different body parts and voice) through extensive experiments. The proposed method achieves AUC scores of 0.802 for stroke and 0.780 for Parkinson's disease, which is effective for a screening system.

Artificial Cognition for Detection of Mental Disability: A Vision Transformer Approach for Alzheimer's Disease

Alzheimer's disease is a common neurological disorder and mental disability that causes memory loss and cognitive decline, presenting a major challenge to public health due to its impact on millions of individuals worldwide. It is crucial to diagnose and treat Alzheimer's in a timely manner to improve the quality of life of both patients and caregivers. In the recent past, machine learning techniques have showed potential in detecting Alzheimer's disease by examining neuroimaging data, especially Magnetic Resonance Imaging (MRI). This research proposes an attention-based mechanism that employs the vision transformer approach to detect Alzheimer's using MRI images. The presented technique applies preprocessing to the MRI images and forwards them to a vision transformer network for classification. This network is trained on the publicly available Kaggle dataset, and it illustrated impressive results with an accuracy of 99.06%, precision of 99.06%, recall of 99.14%, and F1-score of 99.1%. Furthermore, a comparative study is also conducted to evaluate the performance of the proposed method against various state-of-the-art techniques on diverse datasets. The proposed method demonstrated superior performance, outperforming other published methods when applied to the Kaggle dataset.

Evaluation of MRI-based machine learning approaches for computer-aided diagnosis of dementia in a clinical data warehouse

A variety of algorithms have been proposed for computer-aided diagnosis of dementia from anatomical brain MRI. These approaches achieve high accuracy when applied to research data sets but their performance on real-life clinical routine data has not been evaluated yet. The aim of this work was to study the performance of such approaches on clinical routine data, based on a hospital data warehouse, and to compare the results to those obtained on a research data set. The clinical data set was extracted from the hospital data warehouse of the Greater Paris area, which includes 39 different hospitals. The research set was composed of data from the Alzheimer's Disease Neuroimaging Initiative data set. In the clinical set, the population of interest was identified by exploiting the diagnostic codes from the 10th revision of the International Classification of Diseases that are assigned to each patient. We studied how the imbalance of the training sets, in terms of contrast agent injection and image quality, may bias the results. We demonstrated that computer-aided diagnosis performance was strongly biased upwards (over 17 percent points of balanced accuracy) by the confounders of image quality and contrast agent injection, a phenomenon known as the Clever Hans effect or shortcut learning. When these biases were removed, the performance was very poor. In any case, the performance was considerably lower than on the research data set. Our study highlights that there are still considerable challenges for translating dementia computer-aided diagnosis systems to clinical routine.

Explainable AI for Bioinformatics: Methods, Tools and Applications

Artificial intelligence (AI) systems utilizing deep neural networks and machine learning (ML) algorithms are widely used for solving critical problems in bioinformatics, biomedical informatics and precision medicine. However, complex ML models that are often perceived as opaque and black-box methods make it difficult to understand the reasoning behind their decisions. This lack of transparency can be a challenge for both end-users and decision-makers, as well as AI developers. In sensitive areas such as healthcare, explainability and accountability are not only desirable properties but also legally required for AI systems that can have a significant impact on human lives. Fairness is another growing concern, as algorithmic decisions should not show bias or discrimination towards certain groups or individuals based on sensitive attributes. Explainable AI (XAI) aims to overcome the opaqueness of black-box models and to provide transparency in how AI systems make decisions. Interpretable ML models can explain how they make predictions and identify factors that influence their outcomes. However, the majority of the state-of-the-art interpretable ML methods are domain-agnostic and have evolved from fields such as computer vision, automated reasoning or statistics, making direct application to bioinformatics problems challenging without customization and domain adaptation. In this paper, we discuss the importance of explainability and algorithmic transparency in the context of bioinformatics. We provide an overview of model-specific and model-agnostic interpretable ML methods and tools and outline their potential limitations. We discuss how existing interpretable ML methods can be customized and fit to bioinformatics research problems. Further, through case studies in bioimaging, cancer genomics and text mining, we demonstrate how XAI methods can improve transparency and decision fairness. Our review aims at providing valuable insights and serving as a starting point for researchers wanting to enhance explainability and decision transparency while solving bioinformatics problems. GitHub: https://github.com/rezacsedu/XAI-for-bioinformatics.

Artificial intelligence-based diagnosis of Alzheimer's disease with brain MRI images

Alzheimer's disease, a primary neurodegenerative condition, predominantly impacts the elderly and pre-elderly population. This progressive neurological disorder is characterized by an array of symptoms including memory loss, cognitive decline, and various physiological and psychological disturbances, significantly compromising the quality of life of patients and their caregivers. Recent advancements in Magnetic Resonance Imaging (MRI) technology have catalyzed research in AI-enhanced diagnostics for Alzheimer's disease, fostering optimism for early detection and timely interventions. This progress has paved the way for the development of sophisticated algorithms and models adept at analyzing complex brain imaging data, thereby augmenting diagnostic accuracy and efficiency. This advancement fuels optimism regarding the transformative potential of AI-driven diagnostics in revolutionizing Alzheimer's disease management, with the prospect of facilitating more effective treatment strategies and improved patient outcomes. The objective of this review is to provide a comprehensive overview of recent developments in deep learning methodologies applied to brain MRI images for the classification of various stages of Alzheimer's disease, with a particular emphasis on early diagnosis. Furthermore, this review underscores the limitations of current research, discussing potential challenges and future research directions in this dynamic field.

An Efficient Ensemble Approach for Alzheimer's Disease Detection Using an Adaptive Synthetic Technique and Deep Learning

Alzheimer's disease is an incurable neurological disorder that leads to a gradual decline in cognitive abilities, but early detection can significantly mitigate symptoms. The automatic diagnosis of Alzheimer's disease is more important due to the shortage of expert medical staff, because it reduces the burden on medical staff and enhances the results of diagnosis. A detailed analysis of specific brain disorder tissues is required to accurately diagnose the disease via segmented magnetic resonance imaging (MRI). Several studies have used the traditional machine-learning approaches to diagnose the disease from MRI, but manual extracted features are more complex, time-consuming, and require a huge amount of involvement from expert medical staff. The traditional approach does not provide an accurate diagnosis. Deep learning has automatic extraction features and optimizes the training process. The Magnetic Resonance Imaging (MRI) Alzheimer's disease dataset consists of four classes: mild demented (896 images), moderate demented (64 images), non-demented (3200 images), and very mild demented (2240 images). The dataset is highly imbalanced. Therefore, we used the adaptive synthetic oversampling technique to address this issue. After applying this technique, the dataset was balanced. The ensemble of VGG16 and EfficientNet was used to detect Alzheimer's disease on both imbalanced and balanced datasets to validate the performance of the models. The proposed method combined the predictions of multiple models to make an ensemble model that learned complex and nuanced patterns from the data. The input and output of both models were concatenated to make an ensemble model and then added to other layers to make a more robust model. In this study, we proposed an ensemble of EfficientNet-B2 and VGG-16 to diagnose the disease at an early stage with the highest accuracy. Experiments were performed on two publicly available datasets. The experimental results showed that the proposed method achieved 97.35% accuracy and 99.64% AUC for multiclass datasets and 97.09% accuracy and 99.59% AUC for binary-class datasets. We evaluated that the proposed method was extremely efficient and provided superior performance on both datasets as compared to previous methods.

Neuropsychiatric Disorder Subtyping Via Clustered Deep Learning Classifier Explanations. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083012 DOI: 10.1109/embc40787.2023.10340837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Identifying subtypes of neuropsychiatric disorders based on characteristics of their brain activity has tremendous potential to contribute to a better understanding of those disorders and to the development of new diagnostic and personalized treatment approaches. Many studies focused on neuropsychiatric disorders examine the interaction of brain networks over time using dynamic functional network connectivity (dFNC) extracted from resting-state functional magnetic resonance imaging data. Some of these studies involve the use of either deep learning classifiers or traditional clustering approaches, but usually not both. In this study, we present a novel approach for subtyping individuals with neuropsychiatric disorders within the context of schizophrenia (SZ). We trained an explainable deep learning classifier to differentiate between dFNC data from individuals with SZ and controls, obtaining a test accuracy of 79%. We next used cross-validation to obtain robust average explanations for SZ training participants across folds, identifying 5 SZ subtypes that each differed from controls in a distinct manner and that had different degrees of symptom severity. These subtypes specifically differed from one another in their interactions between the visual network and the subcortical, sensorimotor, and auditory networks and between the cerebellar network and the cognitive control and subcortical networks. Additionally, we uncovered statistically significant differences in negative symptom scores between the subtypes. It is our hope that the proposed novel subtyping approach will contribute to the improved understanding and characterization of SZ and other neuropsychiatric disorders.

Explainable AI in medical imaging: An overview for clinical practitioners – Saliency-based XAI approaches

Since recent achievements of Artificial Intelligence (AI) have proven significant success and promising results throughout many fields of application during the last decade, AI has also become an essential part of medical research. The improving data availability, coupled with advances in high-performance computing and innovative algorithms, has increased AI's potential in various aspects. Because AI rapidly reshapes research and promotes the development of personalized clinical care, alongside its implementation arises an urgent need for a deep understanding of its inner workings, especially in high-stake domains. However, such systems can be highly complex and opaque, limiting the possibility of an immediate understanding of the system's decisions. Regarding the medical field, a high impact is attributed to these decisions as physicians and patients can only fully trust AI systems when reasonably communicating the origin of their results, simultaneously enabling the identification of errors and biases. Explainable AI (XAI), becoming an increasingly important field of research in recent years, promotes the formulation of explainability methods and provides a rationale allowing users to comprehend the results generated by AI systems. In this paper, we investigate the application of XAI in medical imaging, addressing a broad audience, especially healthcare professionals. The content focuses on definitions and taxonomies, standard methods and approaches, advantages, limitations, and examples representing the current state of research regarding XAI in medical imaging. This paper focuses on saliency-based XAI methods, where the explanation can be provided directly on the input data (image) and which naturally are of special importance in medical imaging.

Interpretable machine learning for dementia: A systematic review

INTRODUCTION

Machine learning research into automated dementia diagnosis is becoming increasingly popular but so far has had limited clinical impact. A key challenge is building robust and generalizable models that generate decisions that can be reliably explained. Some models are designed to be inherently "interpretable," whereas post hoc "explainability" methods can be used for other models.

METHODS

Here we sought to summarize the state-of-the-art of interpretable machine learning for dementia.

RESULTS

We identified 92 studies using PubMed, Web of Science, and Scopus. Studies demonstrate promising classification performance but vary in their validation procedures and reporting standards and rely heavily on popular data sets.

DISCUSSION

Future work should incorporate clinicians to validate explanation methods and make conclusive inferences about dementia-related disease pathology. Critically analyzing model explanations also requires an understanding of the interpretability methods itself. Patient-specific explanations are also required to demonstrate the benefit of interpretable machine learning in clinical practice.

Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

Advances in artificial intelligence (AI), especially deep learning (DL), have facilitated magnetic resonance imaging (MRI) data analysis, enabling AI-assisted medical image diagnoses and prognoses. However, most of the DL models are considered as "black boxes". There is an unmet need to demystify DL models so domain experts can trust these high-performance DL models. This has resulted in a sub-domain of AI research called explainable artificial intelligence (XAI). In the last decade, many experts have dedicated their efforts to developing novel XAI methods that are competent at visualizing and explaining the logic behind data-driven DL models. However, XAI techniques are still in their infancy for medical MRI image analysis. This study aims to outline the XAI applications that are able to interpret DL models for MRI data analysis. We first introduce several common MRI data modalities. Then, a brief history of DL models is discussed. Next, we highlight XAI frameworks and elaborate on the principles of multiple popular XAI methods. Moreover, studies on XAI applications in MRI image analysis are reviewed across the tissues/organs of the human body. A quantitative analysis is conducted to reveal the insights of MRI researchers on these XAI techniques. Finally, evaluations of XAI methods are discussed. This survey presents recent advances in the XAI domain for explaining the DL models that have been utilized in MRI applications.

Higher performance for women than men in MRI-based Alzheimer's disease detection

INTRODUCTION

Although machine learning classifiers have been frequently used to detect Alzheimer's disease (AD) based on structural brain MRI data, potential bias with respect to sex and age has not yet been addressed. Here, we examine a state-of-the-art AD classifier for potential sex and age bias even in the case of balanced training data.

METHODS

Based on an age- and sex-balanced cohort of 432 subjects (306 healthy controls, 126 subjects with AD) extracted from the ADNI data base, we trained a convolutional neural network to detect AD in MRI brain scans and performed ten different random training-validation-test splits to increase robustness of the results. Classifier decisions for single subjects were explained using layer-wise relevance propagation.

RESULTS

The classifier performed significantly better for women (balanced accuracy [Formula: see text]) than for men ([Formula: see text]). No significant differences were found in clinical AD scores, ruling out a disparity in disease severity as a cause for the performance difference. Analysis of the explanations revealed a larger variance in regional brain areas for male subjects compared to female subjects.

DISCUSSION

The identified sex differences cannot be attributed to an imbalanced training dataset and therefore point to the importance of examining and reporting classifier performance across population subgroups to increase transparency and algorithmic fairness. Collecting more data especially among underrepresented subgroups and balancing the dataset are important but do not always guarantee a fair outcome.

Regional Deep Atrophy: a Self-Supervised Learning Method to Automatically Identify Regions Associated With Alzheimer's Disease Progression From Longitudinal MRI

Longitudinal assessment of brain atrophy, particularly in the hippocampus, is a well-studied biomarker for neurodegenerative diseases, such as Alzheimer's disease (AD). In clinical trials, estimation of brain progressive rates can be applied to track therapeutic efficacy of disease modifying treatments. However, most state-of-the-art measurements calculate changes directly by segmentation and/or deformable registration of MRI images, and may misreport head motion or MRI artifacts as neurodegeneration, impacting their accuracy. In our previous study, we developed a deep learning method DeepAtrophy that uses a convolutional neural network to quantify differences between longitudinal MRI scan pairs that are associated with time. DeepAtrophy has high accuracy in inferring temporal information from longitudinal MRI scans, such as temporal order or relative inter-scan interval. DeepAtrophy also provides an overall atrophy score that was shown to perform well as a potential biomarker of disease progression and treatment efficacy. However, DeepAtrophy is not interpretable, and it is unclear what changes in the MRI contribute to progression measurements. In this paper, we propose Regional Deep Atrophy (RDA), which combines the temporal inference approach from DeepAtrophy with a deformable registration neural network and attention mechanism that highlights regions in the MRI image where longitudinal changes are contributing to temporal inference. RDA has similar prediction accuracy as DeepAtrophy, but its additional interpretability makes it more acceptable for use in clinical settings, and may lead to more sensitive biomarkers for disease monitoring in clinical trials of early AD.

ICAM-Reg: Interpretable Classification and Regression With Feature Attribution for Mapping Neurological Phenotypes in Individual Scans

An important goal of medical imaging is to be able to precisely detect patterns of disease specific to individual scans; however, this is challenged in brain imaging by the degree of heterogeneity of shape and appearance. Traditional methods, based on image registration, historically fail to detect variable features of disease, as they utilise population-based analyses, suited primarily to studying group-average effects. In this paper we therefore take advantage of recent developments in generative deep learning to develop a method for simultaneous classification, or regression, and feature attribution (FA). Specifically, we explore the use of a VAE-GAN (variational autoencoder - general adversarial network) for translation called ICAM, to explicitly disentangle class relevant features, from background confounds, for improved interpretability and regression of neurological phenotypes. We validate our method on the tasks of Mini-Mental State Examination (MMSE) cognitive test score prediction for the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort, as well as brain age prediction, for both neurodevelopment and neurodegeneration, using the developing Human Connectome Project (dHCP) and UK Biobank datasets. We show that the generated FA maps can be used to explain outlier predictions and demonstrate that the inclusion of a regression module improves the disentanglement of the latent space. Our code is freely available on GitHub https://github.com/CherBass/ICAM.

Deep neural network heatmaps capture Alzheimer's disease patterns reported in a large meta-analysis of neuroimaging studies

Deep neural networks currently provide the most advanced and accurate machine learning models to distinguish between structural MRI scans of subjects with Alzheimer's disease and healthy controls. Unfortunately, the subtle brain alterations captured by these models are difficult to interpret because of the complexity of these multi-layer and non-linear models. Several heatmap methods have been proposed to address this issue and analyze the imaging patterns extracted from the deep neural networks, but no quantitative comparison between these methods has been carried out so far. In this work, we explore these questions by deriving heatmaps from Convolutional Neural Networks (CNN) trained using T1 MRI scans of the ADNI data set and by comparing these heatmaps with brain maps corresponding to Support Vector Machine (SVM) activation patterns. Three prominent heatmap methods are studied: Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG), and Guided Grad-CAM (GGC). Contrary to prior studies where the quality of heatmaps was visually or qualitatively assessed, we obtained precise quantitative measures by computing overlap with a ground-truth map from a large meta-analysis that combined 77 voxel-based morphometry (VBM) studies independently from ADNI. Our results indicate that all three heatmap methods were able to capture brain regions covering the meta-analysis map and achieved better results than SVM activation patterns. Among them, IG produced the heatmaps with the best overlap with the independent meta-analysis.

Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks

Artificial Intelligence (AI) techniques of deep learning have revolutionized the disease diagnosis with their outstanding image classification performance. In spite of the outstanding results, the widespread adoption of these techniques in clinical practice is still taking place at a moderate pace. One of the major hindrance is that a trained Deep Neural Networks (DNN) model provides a prediction, but questions about why and how that prediction was made remain unanswered. This linkage is of utmost importance for the regulated healthcare domain to increase the trust in the automated diagnosis system by the practitioners, patients and other stakeholders. The application of deep learning for medical imaging has to be interpreted with caution due to the health and safety concerns similar to blame attribution in the case of an accident involving autonomous cars. The consequences of both a false positive and false negative cases are far reaching for patients' welfare and cannot be ignored. This is exacerbated by the fact that the state-of-the-art deep learning algorithms comprise of complex interconnected structures, millions of parameters, and a 'black box' nature, offering little understanding of their inner working unlike the traditional machine learning algorithms. Explainable AI (XAI) techniques help to understand model predictions which help develop trust in the system, accelerate the disease diagnosis, and meet adherence to regulatory requirements. This survey provides a comprehensive review of the promising field of XAI for biomedical imaging diagnostics. We also provide a categorization of the XAI techniques, discuss the open challenges, and provide future directions for XAI which would be of interest to clinicians, regulators and model developers.

An Explainable Convolutional Neural Network for the Early Diagnosis of Alzheimer's Disease from 18F-FDG PET

Convolutional Neural Networks (CNN) which support the diagnosis of Alzheimer's Disease using 18F-FDG PET images are obtaining promising results; however, one of the main challenges in this domain is the fact that these models work as black-box systems. We developed a CNN that performs a multiclass classification task of volumetric 18F-FDG PET images, and we experimented two different post hoc explanation techniques developed in the field of Explainable Artificial Intelligence: Saliency Map (SM) and Layerwise Relevance Propagation (LRP). Finally, we quantitatively analyze the explanations returned and inspect their relationship with the PET signal. We collected 2552 scans from the Alzheimer's Disease Neuroimaging Initiative labeled as Cognitively Normal (CN), Mild Cognitive Impairment (MCI), and Alzheimer's Disease (AD) and we developed and tested a 3D CNN that classifies the 3D PET scans into its final clinical diagnosis. The model developed achieves, to the best of our knowledge, performances comparable with the relevant literature on the test set, with an average Area Under the Curve (AUC) for prediction of CN, MCI, and AD 0.81, 0.63, and 0.77 respectively. We registered the heatmaps with the Talairach Atlas to perform a regional quantitative analysis of the relationship between heatmaps and PET signals. With the quantitative analysis of the post hoc explanation techniques, we observed that LRP maps were more effective in mapping the importance metrics in the anatomic atlas. No clear relationship was found between the heatmap and the PET signal.

Channel Contribution in Deep Learning Based Automatic Sleep Scoring-How Many Channels Do We Need?

Machine learning based sleep scoring methods aim to automate the process of annotating polysomnograms with sleep stages. Although sleep signals of multiple modalities and channels should contain more information according to sleep guidelines, most multi-channel multi-modal models in the literature showed only a little performance improvement compared to single-channel EEG models and sometimes even failed to outperform them. In this paper, we investigate whether the high performance of single-channel EEG models can be attributed to specific model features in their deep learning architectures and to which extent multi-channel multi-modal models take the information from different channels of modalities into account. First, we transfer the model features from single-channel EEG models, such as combinations of small and large filters in CNNs, to multi-channel multi-modal models and measure their impacts. Second, we employ two explainability methods, the layer-wise relevance propagation as post-hoc and the embedded channel attention network as intrinsic interpretability methods, to measure the contribution of different channels on predictive performance. We find that i) single-channel model features can improve the performance of multi-channel multi-modal models and ii) multi-channel multi-modal models focus on one important channel per modality and use the remaining channels to complement the information of the focused channels. Our results suggest that more advanced methods for aggregating channel information using complementary information from other channels may improve sleep scoring performance for multi-channel multi-modal models.

Relevance-based channel selection in motor imagery brain-computer interface

Objective.Channel selection in the electroencephalogram (EEG)-based brain-computer interface (BCI) has been extensively studied for over two decades, with the goal being to select optimal subject-specific channels that can enhance the overall decoding efficacy of the BCI. With the emergence of deep learning (DL)-based BCI models, there arises a need for fresh perspectives and novel techniques to conduct channel selection. In this regard, subject-independent channel selection is relevant, since DL models trained using cross-subject data offer superior performance, and the impact of inherent inter-subject variability of EEG characteristics on subject-independent DL training is not yet fully understood.Approach.Here, we propose a novel methodology for implementing subject-independent channel selection in DL-based motor imagery (MI)-BCI, using layer-wise relevance propagation (LRP) and neural network pruning. Experiments were conducted using Deep ConvNet and 62-channel MI data from the Korea University EEG dataset.Main Results.Using our proposed methodology, we achieved a 61% reduction in the number of channels without any significant drop (p = 0.09) in subject-independent classification accuracy, due to the selection of highly relevant channels by LRP. LRP relevance-based channel selections provide significantly better accuracies compared to conventional weight-based selections while using less than 40% of the total number of channels, with differences in accuracies ranging from 5.96% to 1.72%. The performance of the adapted sparse-LRP model using only 16% of the total number of channels is similar to that of the adapted baseline model (p = 0.13). Furthermore, the accuracy of the adapted sparse-LRP model using only 35% of the total number of channels exceeded that of the adapted baseline model by 0.53% (p = 0.81). Analyses of channels chosen by LRP confirm the neurophysiological plausibility of selection, and emphasize the influence of motor, parietal, and occipital channels in MI-EEG classification.Significance.The proposed method addresses a traditional issue in EEG-BCI decoding, while being relevant and applicable to the latest developments in the field of BCI. We believe that our work brings forth an interesting and important application of model interpretability as a problem-solving technique.

Prediction of bone mineral density in CT using deep learning with explainability

Bone mineral density (BMD) is a key feature in diagnosing bone diseases. Although computational tomography (CT) is a common imaging modality, it seldom provides bone mineral density information in a clinic owing to technical difficulties. Thus, a dual-energy X-ray absorptiometry (DXA) is required to measure bone mineral density at the expense of additional radiation exposure. In this study, a deep learning framework was developed to estimate the bone mineral density from an axial cut of the L1 bone on computational tomography. As a result, the correlation coefficient between bone mineral density estimates and dual-energy X-ray absorptiometry bone mineral density was .90. When the samples were categorized into abnormal and normal groups using a standard (T-score = - 1.0 ), the maximum F1 score in the diagnostic test was .875. In addition, it was identified using explainable artificial intelligence techniques that the network intensively sees a local area spanning tissues around the vertebral foramen. This method is well suited as an auxiliary tool in clinical practice and as an automatic screener for identifying latent patients in computational tomography databases.

Layer-Wise Relevance Analysis for Motif Recognition in the Activation Pathway of the β2-Adrenergic GPCR Receptor

G-protein-coupled receptors (GPCRs) are cell membrane proteins of relevance as therapeutic targets, and are associated to the development of treatments for illnesses such as diabetes, Alzheimer's, or even cancer. Therefore, comprehending the underlying mechanisms of the receptor functional properties is of particular interest in pharmacoproteomics and in disease therapy at large. Their interaction with ligands elicits multiple molecular rearrangements all along their structure, inducing activation pathways that distinctly influence the cell response. In this work, we studied GPCR signaling pathways from molecular dynamics simulations as they provide rich information about the dynamic nature of the receptors. We focused on studying the molecular properties of the receptors using deep-learning-based methods. In particular, we designed and trained a one-dimensional convolution neural network and illustrated its use in a classification of conformational states: active, intermediate, or inactive, of the β2-adrenergic receptor when bound to the full agonist BI-167107. Through a novel explainability-oriented investigation of the prediction results, we were able to identify and assess the contribution of individual motifs (residues) influencing a particular activation pathway. Consequently, we contribute a methodology that assists in the elucidation of the underlying mechanisms of receptor activation-deactivation.

Discriminative patterns of white matter changes in Alzheimer's

Changes in structural connectivity of the Alzheimer's brain have not been widely studied utilizing cutting-edge methodologies. This study develops an efficient structural connectome-based convolutional neural network (CNN) to classify the AD and uses explanations of CNNs' choices in classification to pinpoint the discriminative changes in white matter connectivity in AD. A CNN architecture has been developed to classify normal control (NC) and AD subjects from the weighted structural connectome. Then, the CNN classification decision is visually analyzed using gradient-based localization techniques to identify the discriminative changes in white matter connectivity in Alzheimer's. The cortical regions involved in the identified discriminative structural connectivity changes in AD are highly covered in the temporal/subcortical regions. A specific pattern is identified in the discriminative changes in structural connectivity of AD, where the white matter changes are revealed within the temporal/subcortical regions and from the temporal/subcortical regions to the frontal and parietal regions in both left and right hemispheres.  The proposed approach has the potential to comprehensively analyze the discriminative structural connectivity differences in AD, change the way of detecting biomarkers, and help clinicians better understand the structural changes in AD and provide them with more confidence in automated diagnostic systems.

Challenges for machine learning in clinical translation of big data imaging studies

Combining deep learning image analysis methods and large-scale imaging datasets offers many opportunities to neuroscience imaging and epidemiology. However, despite these opportunities and the success of deep learning when applied to a range of neuroimaging tasks and domains, significant barriers continue to limit the impact of large-scale datasets and analysis tools. Here, we examine the main challenges and the approaches that have been explored to overcome them. We focus on issues relating to data availability, interpretability, evaluation, and logistical challenges and discuss the problems that still need to be tackled to enable the success of "big data" deep learning approaches beyond research.

Explainable AI: A review of applications to neuroimaging data

Deep neural networks (DNNs) have transformed the field of computer vision and currently constitute some of the best models for representations learned via hierarchical processing in the human brain. In medical imaging, these models have shown human-level performance and even higher in the early diagnosis of a wide range of diseases. However, the goal is often not only to accurately predict group membership or diagnose but also to provide explanations that support the model decision in a context that a human can readily interpret. The limited transparency has hindered the adoption of DNN algorithms across many domains. Numerous explainable artificial intelligence (XAI) techniques have been developed to peer inside the "black box" and make sense of DNN models, taking somewhat divergent approaches. Here, we suggest that these methods may be considered in light of the interpretation goal, including functional or mechanistic interpretations, developing archetypal class instances, or assessing the relevance of certain features or mappings on a trained model in a post-hoc capacity. We then focus on reviewing recent applications of post-hoc relevance techniques as applied to neuroimaging data. Moreover, this article suggests a method for comparing the reliability of XAI methods, especially in deep neural networks, along with their advantages and pitfalls.