Investigation of out-of-distribution detection across various models and training methodologies

Machine learning-based algorithms demonstrate impressive performance across numerous fields; however, they continue to suffer from certain limitations. Even sophisticated and precise algorithms often make erroneous predictions when implemented with datasets having different distributions compared to the training set. Out-of-distribution (OOD) detection, which distinguishes data with different distributions from that of the training set, is a critical research area necessary to overcome these limitations and create more reliable algorithms. The OOD issue, particularly concerning image data, has been extensively studied. However, recently developed OOD methods do not fulfill the expectation that OOD performance will increase as the accuracy of in-distribution classification improves. Our research presents a comprehensive study on OOD detection performance across multiple models and training methodologies to verify this phenomenon. Specifically, we explore various pre-trained models popular in the computer vision field with both old and new OOD detection methods. The experimental results highlight the performance disparity in existing OOD methods. Based on these observations, we introduce Trimmed Rank with Inverse softMax probability (TRIM), a remarkably simple yet effective method for model weights with newly developed training methods. The proposed method could serve as a potential tool for enhancing OOD detection performance owing to its promising results. The OOD performance of TRIM is highly compatible with the in-distribution accuracy model and may bridge the efforts on improving in-distribution accuracy to the ability to distinguish OOD data.

The explainable potential of coupling hybridized metaheuristics, XGBoost, and SHAP in revealing toluene behavior in the atmosphere

Toluene is a neurotoxic aromatic hydrocarbon and one of the major representatives of volatile organic compounds, known for its abundance, adverse health effects, and role in the formation of other atmospheric pollutants like ozone. This research introduces the enhanced version of the reptile search metaheuristics algorithm which has been utilized to tune the extreme gradient boosting hyperparameters, to investigate toluene atmospheric behavior patterns and interactions with other polluting species within defined environmental conditions. The study is based on a two-year database encompassing concentrations of inorganic gaseous contaminants every hour (NO, NO2, NOx, and O3), particulate matter fractions (PM1, PM2.5, and PM10), m,p-xylene, toluene, benzene, total non-methane hydrocarbons, and meteorological data. The experimental outcomes were validated against the results of extreme gradient boosting models optimized by seven other recent powerful metaheuristics algorithms. The best-performing model has been interpreted by employing Shapley additive explanations method. In the study, we have focused on the relationship between toluene and benzene, as its most important predictor, and provided a detailed description of environmental conditions which directed their interactions.

Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50

This study addresses the critical challenge of detecting brain tumors using MRI images, a pivotal task in medical diagnostics that demands high accuracy and interpretability. While deep learning has shown remarkable success in medical image analysis, there remains a substantial need for models that are not only accurate but also interpretable to healthcare professionals. The existing methodologies, predominantly deep learning-based, often act as black boxes, providing little insight into their decision-making process. This research introduces an integrated approach using ResNet50, a deep learning model, combined with Gradient-weighted Class Activation Mapping (Grad-CAM) to offer a transparent and explainable framework for brain tumor detection. We employed a dataset of MRI images, enhanced through data augmentation, to train and validate our model. The results demonstrate a significant improvement in model performance, with a testing accuracy of 98.52% and precision-recall metrics exceeding 98%, showcasing the model's effectiveness in distinguishing tumor presence. The application of Grad-CAM provides insightful visual explanations, illustrating the model's focus areas in making predictions. This fusion of high accuracy and explainability holds profound implications for medical diagnostics, offering a pathway towards more reliable and interpretable brain tumor detection tools.

CIME4R: Exploring iterative, AI-guided chemical reaction optimization campaigns in their parameter space

Chemical reaction optimization (RO) is an iterative process that results in large, high-dimensional datasets. Current tools allow for only limited analysis and understanding of parameter spaces, making it hard for scientists to review or follow changes throughout the process. With the recent emergence of using artificial intelligence (AI) models to aid RO, another level of complexity has been added. Helping to assess the quality of a model's prediction and understand its decision is critical to supporting human-AI collaboration and trust calibration. To address this, we propose CIME4R-an open-source interactive web application for analyzing RO data and AI predictions. CIME4R supports users in (i) comprehending a reaction parameter space, (ii) investigating how an RO process developed over iterations, (iii) identifying critical factors of a reaction, and (iv) understanding model predictions. This facilitates making informed decisions during the RO process and helps users to review a completed RO process, especially in AI-guided RO. CIME4R aids decision-making through the interaction between humans and AI by combining the strengths of expert experience and high computational precision. We developed and tested CIME4R with domain experts and verified its usefulness in three case studies. Using CIME4R the experts were able to produce valuable insights from past RO campaigns and to make informed decisions on which experiments to perform next. We believe that CIME4R is the beginning of an open-source community project with the potential to improve the workflow of scientists working in the reaction optimization domain. SCIENTIFIC CONTRIBUTION: To the best of our knowledge, CIME4R is the first open-source interactive web application tailored to the peculiar analysis requirements of reaction optimization (RO) campaigns. Due to the growing use of AI in RO, we developed CIME4R with a special focus on facilitating human-AI collaboration and understanding of AI models. We developed and evaluated CIME4R in collaboration with domain experts to verify its practical usefulness.

Determining risk and predictors of head and neck cancer treatment-related lymphedema: A clinicopathologic and dosimetric data mining approach using interpretable machine learning and ensemble feature selection

Background and purpose

The ability to determine the risk and predictors of lymphedema is vital in improving the quality of life for head and neck (HN) cancer patients. However, selecting robust features is challenging due to the multicollinearity and high dimensionality of radiotherapy (RT) data. This study aims to overcome these challenges using an ensemble feature selection technique with machine learning (ML).

Materials and methods

Thirty organs-at-risk, including bilateral cervical lymph node levels, were contoured, and dose-volume data were extracted from 76 HN treatment plans. Clinicopathologic data was collected. Ensemble feature selection was used to reduce the number of features. Using the reduced features as input to ML and competing risk models, internal and external lymphedema prediction capability was evaluated with the ML models, and time to lymphedema event and risk stratification were estimated using the risk models.

Results

Two ML models, XGBoost and random forest, exhibited robust prediction performance. They achieved average F1-scores and AUCs of 84 ± 3.3 % and 79 ± 11.9 % (external lymphedema), and 64 ± 12 % and 78 ± 7.9 % (internal lymphedema). Predictive ML and risk models identified common predictors, including bulky node involvement, high dose to various lymph node levels, and lymph nodes removed during surgery. At 180 days, removing 0-25, 26-50, and > 50 lymph nodes increased external lymphedema risk to 72.1 %, 95.6 %, and 57.7 % respectively (p = 0.01).

Conclusion

Our approach, involving the reduction of HN RT data dimensionality, resulted in effective ML models for HN lymphedema prediction. Predictive dosimetric features emerged from both predictive and competing risk models. Consistency with clinicopathologic features from other studies supports our methodology.

Stable feature selection utilizing Graph Convolutional Neural Network and Layer-wise Relevance Propagation for biomarker discovery in breast cancer

High-throughput technologies are becoming increasingly important in discovering prognostic biomarkers and in identifying novel drug targets. With Mammaprint, Oncotype DX, and many other prognostic molecular signatures breast cancer is one of the paradigmatic examples of the utility of high-throughput data to deliver prognostic biomarkers, that can be represented in a form of a rather short gene list. Such gene lists can be obtained as a set of features (genes) that are important for the decisions of a Machine Learning (ML) method applied to high-dimensional gene expression data. Several studies have identified predictive gene lists for patient prognosis in breast cancer, but these lists are unstable and have only a few genes in common. Instability of feature selection impedes biological interpretability: genes that are relevant for cancer pathology should be members of any predictive gene list obtained for the same clinical type of patients. Stability and interpretability of selected features can be improved by including information on molecular networks in ML methods. Graph Convolutional Neural Network (GCNN) is a contemporary deep learning approach applicable to gene expression data structured by a prior knowledge molecular network. Layer-wise Relevance Propagation (LRP) and SHapley Additive exPlanations (SHAP) are methods to explain individual decisions of deep learning models. We used both GCNN+LRP and GCNN+SHAP techniques to construct feature sets by aggregating individual explanations. We suggest a methodology to systematically and quantitatively analyze the stability, the impact on the classification performance, and the interpretability of the selected feature sets. We used this methodology to compare GCNN+LRP to GCNN+SHAP and to more classical ML-based feature selection approaches. Utilizing a large breast cancer gene expression dataset we show that, while feature selection with SHAP is useful in applications where selected features have to be impactful for classification performance, among all studied methods GCNN+LRP delivers the most stable (reproducible) and interpretable gene lists.

Learning active subspaces and discovering important features with Gaussian radial basis functions neural networks

Providing a model that achieves a strong predictive performance and is simultaneously interpretable by humans is one of the most difficult challenges in machine learning research due to the conflicting nature of these two objectives. To address this challenge, we propose a modification of the radial basis function neural network model by equipping its Gaussian kernel with a learnable precision matrix. We show that precious information is contained in the spectrum of the precision matrix that can be extracted once the training of the model is completed. In particular, the eigenvectors explain the directions of maximum sensitivity of the model revealing the active subspace and suggesting potential applications for supervised dimensionality reduction. At the same time, the eigenvectors highlight the relationship in terms of absolute variation between the input and the latent variables, thereby allowing us to extract a ranking of the input variables based on their importance to the prediction task enhancing the model interpretability. We conducted numerical experiments for regression, classification, and feature selection tasks, comparing our model against popular machine learning models, the state-of-the-art deep learning-based embedding feature selection techniques, and a transformer model for tabular data. Our results demonstrate that the proposed model does not only yield an attractive prediction performance compared to the competitors but also provides meaningful and interpretable results that potentially could assist the decision-making process in real-world applications. A PyTorch implementation of the model is available on GitHub at the following link.1.

Continued versus discontinued oxytocin stimulation in the active phase of labour (CONDISOX): individual management based on artificial intelligence - a secondary analysis

BACKGROUND

Current guidelines regarding oxytocin stimulation are not tailored to individuals as they are based on randomised controlled trials. The objective of the study was to develop an artificial intelligence (AI) model for individual prediction of the risk of caesarean delivery (CD) in women with a cervical dilatation of 6 cm after oxytocin stimulation for induced labour. The model included not only variables known when labour induction was initiated but also variables describing the course of the labour induction.

METHODS

Secondary analysis of data from the CONDISOX randomised controlled trial of discontinued vs. continued oxytocin infusion in the active phase of induced labour. Extreme gradient boosting (XGBoost) software was used to build the prediction model. To explain the impact of the predictors, we calculated Shapley additive explanation (SHAP) values and present a summary SHAP plot. A force plot was used to explain specifics about an individual's predictors that result in a change of the individual's risk output value from the population-based risk.

RESULTS

Among 1060 included women, 160 (15.1%) were delivered by CD. The XGBoost model found women who delivered vaginally were more likely to be parous, taller, to have a lower estimated birth weight, and to be stimulated with a lower amount of oxytocin. In 108 women (10% of 1060) the model favoured either continuation or discontinuation of oxytocin. For the remaining 90% of the women, the model found that continuation or discontinuation of oxytocin stimulation affected the risk difference of CD by less than 5% points.

CONCLUSION

In women undergoing labour induction, this AI model based on a secondary analysis of data from the CONDISOX trial may help predict the risk of CD and assist the mother and clinician in individual tailored management of oxytocin stimulation after reaching 6 cm of cervical dilation.

Validating the knowledge represented by a self-organizing map with an expert-derived knowledge structure

BACKGROUND

Professionals are reluctant to make use of machine learning results for tasks like curriculum development if they do not understand how the results were generated and what they mean. Visualizations of peer reviewed medical literature can summarize enormous amounts of information but are difficult to interpret. This article reports the validation of the meaning of a self-organizing map derived from the Medline/PubMed index of peer reviewed medical literature by its capacity to coherently summarize the references of a core psychiatric textbook.

METHODS

Reference lists from ten editions of Kaplan and Sadock's Comprehensive Textbook of Psychiatry were projected onto a self-organizing map trained on Medical Subject Headings annotating the complete set of peer reviewed medical research articles indexed in the Medline/PubMed database (MedSOM). K-means clustering was applied to references from every edition to examine the ability of the self-organizing map to coherently summarize the knowledge contained within the textbook.

RESULTS

MedSOM coherently clustered references into six psychiatric knowledge domains across ten editions (1967-2017). Clustering occurred at the abstract level of broad psychiatric practice including General/adult psychiatry, Child psychiatry, and Administrative psychiatry.

CONCLUSIONS

The uptake of visualizations of published medical literature by medical experts for purposes like curriculum development depends upon validation of the meaning of the visualizations. The current research demonstrates that a self-organizing map (MedSOM) can validate the stability and coherence of the references used to support the knowledge claims of a standard psychiatric textbook, linking the products of machine learning to a widely accepted standard of knowledge.

Expectation management in AI: A framework for understanding stakeholder trust and acceptance of artificial intelligence systems

As artificial intelligence systems gain traction, their trustworthiness becomes paramount to harness their benefits and mitigate risks. This study underscores the pressing need for an expectation management framework to align stakeholder anticipations before any system-related activities, such as data collection, modeling, or implementation. To this end, we introduce a comprehensive framework tailored to capture end-user expectations specifically for trustworthy artificial intelligence systems. To ensure its relevance and robustness, we validated the framework via semi-structured interviews, encompassing questions rooted in the framework's constructs and principles. These interviews engaged fourteen diverse end users across the healthcare and education sectors, including physicians, teachers, and students. Through a meticulous qualitative analysis of the interview transcripts, we unearthed pivotal themes and discerned varying perspectives among the interviewee groups. Ultimately, our framework stands as a pivotal tool, paving the way for in-depth discussions about user expectations, illuminating the significance of various system attributes, and spotlighting potential challenges that might jeopardize the system's efficacy.

Predicting levels of prolonged grief disorder symptoms during the COVID-19 pandemic: An integrated approach of classical data exploration, predictive machine learning, and explainable AI

BACKGROUND

Prior studies on Prolonged Grief Disorder (PGD) primarily employed classical approaches to link bereaved individuals' characteristics with PGD symptom levels. This study utilized machine learning to identify key factors influencing PGD symptoms during the COVID-19 pandemic.

METHODS

We analyzed data from 479 participants through an online survey, employing classical data exploration, predictive machine learning, and SHapley Additive exPlanations (SHAP) to determine key factors influencing PGD symptoms measured with the Traumatic Grief Inventory - Self Report (TGI-SR) from 19 variables, comparing five predictive models.

RESULTS

The classical approach identified eight variables associated with a possible PGD (TGI-SR score ≥ 59): unexpected causes of death, living alone, seeking professional support, taking anxiety and/or depression medications, using more grief services (telephone or online supports) and more confrontation-oriented coping strategies, and higher levels of depression and anxiety. Using machine learning techniques, the CatBoost algorithm provided the best predictive model of the TGI-SR score (r2 = 0.6479). The three variables influencing the most the level of PGD symptoms were anxiety, and levels of avoidance and confrontation coping strategies used.

CONCLUSIONS

This pioneering approach within the field of grief research enabled us to leverage the extensive dataset collected during the pandemic, facilitating a deeper comprehension of the predominant factors influencing the grieving process for individuals who experienced loss during this period.

LIMITATIONS

This study acknowledges self-selection bias, limited sample diversity, and suggests further research is needed to fully understand the predictors of PGD symptoms.

Predictive modeling for breast cancer classification in the context of Bangladeshi patients by use of machine learning approach with explainable AI

Breast cancer has rapidly increased in prevalence in recent years, making it one of the leading causes of mortality worldwide. Among all cancers, it is by far the most common. Diagnosing this illness manually requires significant time and expertise. Since detecting breast cancer is a time-consuming process, preventing its further spread can be aided by creating machine-based forecasts. Machine learning and Explainable AI are crucial in classification as they not only provide accurate predictions but also offer insights into how the model arrives at its decisions, aiding in the understanding and trustworthiness of the classification results. In this study, we evaluate and compare the classification accuracy, precision, recall, and F1 scores of five different machine learning methods using a primary dataset (500 patients from Dhaka Medical College Hospital). Five different supervised machine learning techniques, including decision tree, random forest, logistic regression, naive bayes, and XGBoost, have been used to achieve optimal results on our dataset. Additionally, this study applied SHAP analysis to the XGBoost model to interpret the model's predictions and understand the impact of each feature on the model's output. We compared the accuracy with which several algorithms classified the data, as well as contrasted with other literature in this field. After final evaluation, this study found that XGBoost achieved the best model accuracy, which is 97%.

Artificial intelligence and explanation: How, why, and when to explain black boxes

Artificial intelligence (AI) is infiltrating nearly all fields of science by storm. One notorious property that AI algorithms bring is their so-called black box character. In particular, they are said to be inherently unexplainable algorithms. Of course, such characteristics would pose a problem for the medical world, including radiology. The patient journey is filled with explanations along the way, from diagnoses to treatment, follow-up, and more. If we were to replace part of these steps with non-explanatory algorithms, we could lose grip on vital aspects such as finding mistakes, patient trust, and even the creation of new knowledge. In this article, we argue that, even for the darkest of black boxes, there is hope of understanding them. In particular, we compare the situation of understanding black box models to that of understanding the laws of nature in physics. In the case of physics, we are given a 'black box' law of nature, about which there is no upfront explanation. However, as current physical theories show, we can learn plenty about them. During this discussion, we present the process by which we make such explanations and the human role therein, keeping a solid focus on radiological AI situations. We will outline the AI developers' roles in this process, but also the critical role fulfilled by the practitioners, the radiologists, in providing a healthy system of continuous improvement of AI models. Furthermore, we explore the role of the explainable AI (XAI) research program in the broader context we describe.

Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment

Driver behavior is a critical factor in driving safety, making the development of sophisticated distraction classification methods essential. Our study presents a Distracted Driving Classification (DDC) approach utilizing a visual Large Language Model (LLM), named the Distracted Driving Language Model (DDLM). The DDLM introduces whole-body human pose estimation to isolate and analyze key postural features-head, right hand, and left hand-for precise behavior classification and better interpretability. Recognizing the inherent limitations of LLMs, particularly their lack of logical reasoning abilities, we have integrated a reasoning chain framework within the DDLM, allowing it to generate clear, reasoned explanations for its assessments. Tailored specifically with relevant data, the DDLM demonstrates enhanced performance, providing detailed, context-aware evaluations of driver behaviors and corresponding risk levels. Notably outperforming standard models in both zero-shot and few-shot learning scenarios, as evidenced by tests on the 100-Driver dataset, the DDLM stands out as an advanced tool that promises significant contributions to driving safety by accurately detecting and analyzing driving distractions.

Reliable water quality prediction and parametric analysis using explainable AI models

The consumption of water constitutes the physical health of most of the living species and hence management of its purity and quality is extremely essential as contaminated water has to potential to create adverse health and environmental consequences. This creates the dire necessity to measure, control and monitor the quality of water. The primary contaminant present in water is Total Dissolved Solids (TDS), which is hard to filter out. There are various substances apart from mere solids such as potassium, sodium, chlorides, lead, nitrate, cadmium, arsenic and other pollutants. The proposed work aims to provide the automation of water quality estimation through Artificial Intelligence and uses Explainable Artificial Intelligence (XAI) for the explanation of the most significant parameters contributing towards the potability of water and the estimation of the impurities. XAI has the transparency and justifiability as a white-box model since the Machine Learning (ML) model is black-box and unable to describe the reasoning behind the ML classification. The proposed work uses various ML models such as Logistic Regression, Support Vector Machine (SVM), Gaussian Naive Bayes, Decision Tree (DT) and Random Forest (RF) to classify whether the water is drinkable. The various representations of XAI such as force plot, test patch, summary plot, dependency plot and decision plot generated in SHAPELY explainer explain the significant features, prediction score, feature importance and justification behind the water quality estimation. The RF classifier is selected for the explanation and yields optimum Accuracy and F1-Score of 0.9999, with Precision and Re-call of 0.9997 and 0.998 respectively. Thus, the work is an exploratory analysis of the estimation and management of water quality with indicators associated with their significance. This work is an emerging research at present with a vision of addressing the water quality for the future as well.

Slideflow: deep learning for digital histopathology with real-time whole-slide visualization

Deep learning methods have emerged as powerful tools for analyzing histopathological images, but current methods are often specialized for specific domains and software environments, and few open-source options exist for deploying models in an interactive interface. Experimenting with different deep learning approaches typically requires switching software libraries and reprocessing data, reducing the feasibility and practicality of experimenting with new architectures. We developed a flexible deep learning library for histopathology called Slideflow, a package which supports a broad array of deep learning methods for digital pathology and includes a fast whole-slide interface for deploying trained models. Slideflow includes unique tools for whole-slide image data processing, efficient stain normalization and augmentation, weakly-supervised whole-slide classification, uncertainty quantification, feature generation, feature space analysis, and explainability. Whole-slide image processing is highly optimized, enabling whole-slide tile extraction at 40x magnification in 2.5 s per slide. The framework-agnostic data processing pipeline enables rapid experimentation with new methods built with either Tensorflow or PyTorch, and the graphical user interface supports real-time visualization of slides, predictions, heatmaps, and feature space characteristics on a variety of hardware devices, including ARM-based devices such as the Raspberry Pi.

Explicable prioritization of genetic variants by integration of rule-based and machine learning algorithms for diagnosis of rare Mendelian disorders

BACKGROUND

In the process of finding the causative variant of rare diseases, accurate assessment and prioritization of genetic variants is essential. Previous variant prioritization tools mainly depend on the in-silico prediction of the pathogenicity of variants, which results in low sensitivity and difficulty in interpreting the prioritization result. In this study, we propose an explainable algorithm for variant prioritization, named 3ASC, with higher sensitivity and ability to annotate evidence used for prioritization. 3ASC annotates each variant with the 28 criteria defined by the ACMG/AMP genome interpretation guidelines and features related to the clinical interpretation of the variants. The system can explain the result based on annotated evidence and feature contributions.

RESULTS

We trained various machine learning algorithms using in-house patient data. The performance of variant ranking was assessed using the recall rate of identifying causative variants in the top-ranked variants. The best practice model was a random forest classifier that showed top 1 recall of 85.6% and top 3 recall of 94.4%. The 3ASC annotates the ACMG/AMP criteria for each genetic variant of a patient so that clinical geneticists can interpret the result as in the CAGI6 SickKids challenge. In the challenge, 3ASC identified causal genes for 10 out of 14 patient cases, with evidence of decreased gene expression for 6 cases. Among them, two genes (HDAC8 and CASK) had decreased gene expression profiles confirmed by transcriptome data.

CONCLUSIONS

3ASC can prioritize genetic variants with higher sensitivity compared to previous methods by integrating various features related to clinical interpretation, including features related to false positive risk such as quality control and disease inheritance pattern. The system allows interpretation of each variant based on the ACMG/AMP criteria and feature contribution assessed using explainable AI techniques.

Explainable and Interpretable Machine Learning for Antimicrobial Stewardship: Opportunities and Challenges

There is growing interest in exploiting the advances in artificial intelligence and machine learning (ML) for improving and monitoring antimicrobial prescriptions in line with antimicrobial stewardship principles. Against this background, the concepts of interpretability and explainability are becoming increasingly essential to understanding how ML algorithms could predict antimicrobial resistance or recommend specific therapeutic agents, to avoid unintended biases related to the "black box" nature of complex models. In this commentary, we review and discuss some relevant topics on the use of ML algorithms for antimicrobial stewardship interventions, highlighting opportunities and challenges, with particular attention paid to interpretability and explainability of employed models. As in other fields of medicine, the exponential growth of artificial intelligence and ML indicates the potential for improving the efficacy of antimicrobial stewardship interventions, at least in part by reducing time-consuming tasks for overwhelmed health care personnel. Improving our knowledge about how complex ML models work could help to achieve crucial advances in promoting the appropriate use of antimicrobials, as well as in preventing antimicrobial resistance selection and dissemination.

Identifying EEG Biomarkers of Depression with Novel Explainable Deep Learning Architectures

Deep learning methods are increasingly being applied to raw electroencephalogram (EEG) data. However, if these models are to be used in clinical or research contexts, methods to explain them must be developed, and if these models are to be used in research contexts, methods for combining explanations across large numbers of models must be developed to counteract the inherent randomness of existing training approaches. Model visualization-based explainability methods for EEG involve structuring a model architecture such that its extracted features can be characterized and have the potential to offer highly useful insights into the patterns that they uncover. Nevertheless, model visualization-based explainability methods have been underexplored within the context of multichannel EEG, and methods to combine their explanations across folds have not yet been developed. In this study, we present two novel convolutional neural network-based architectures and apply them for automated major depressive disorder diagnosis. Our models obtain slightly lower classification performance than a baseline architecture. However, across 50 training folds, they find that individuals with MDD exhibit higher β power, potentially higher δ power, and higher brain-wide correlation that is most strongly represented within the right hemisphere. This study provides multiple key insights into MDD and represents a significant step forward for the domain of explainable deep learning applied to raw EEG. We hope that it will inspire future efforts that will eventually enable the development of explainable EEG deep learning models that can contribute both to clinical care and novel medical research discoveries.

A lightweight xAI approach to cervical cancer classification

Cervical cancer is caused in the vast majority of cases by the human papilloma virus (HPV) through sexual contact and requires a specific molecular-based analysis to be detected. As an HPV vaccine is available, the incidence of cervical cancer is up to ten times higher in areas without adequate healthcare resources. In recent years, liquid cytology has been used to overcome these shortcomings and perform mass screening. In addition, classifiers based on convolutional neural networks can be developed to help pathologists diagnose the disease. However, these systems always require the final verification of a pathologist to make a final diagnosis. For this reason, explainable AI techniques are required to highlight the most significant data to the healthcare professional, as it can be used to determine the confidence in the results and the areas of the image used for classification (allowing the professional to point out the areas he/she thinks are most important and cross-check them against those detected by the system in order to create incremental learning systems). In this work, a 4-phase optimization process is used to obtain a custom deep-learning classifier for distinguishing between 4 severity classes of cervical cancer with liquid-cytology images. The final classifier obtains an accuracy over 97% for 4 classes and 100% for 2 classes with execution times under 1 s (including the final report generation). Compared to previous works, the proposed classifier obtains better accuracy results with a lower computational cost.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

Shallow and deep learning classifiers in medical image analysis

An increasingly strong connection between artificial intelligence and medicine has enabled the development of predictive models capable of supporting physicians' decision-making. Artificial intelligence encompasses much more than machine learning, which nevertheless is its most cited and used sub-branch in the last decade. Since most clinical problems can be modeled through machine learning classifiers, it is essential to discuss their main elements. This review aims to give primary educational insights on the most accessible and widely employed classifiers in radiology field, distinguishing between "shallow" learning (i.e., traditional machine learning) algorithms, including support vector machines, random forest and XGBoost, and "deep" learning architectures including convolutional neural networks and vision transformers. In addition, the paper outlines the key steps for classifiers training and highlights the differences between the most common algorithms and architectures. Although the choice of an algorithm depends on the task and dataset dealing with, general guidelines for classifier selection are proposed in relation to task analysis, dataset size, explainability requirements, and available computing resources. Considering the enormous interest in these innovative models and architectures, the problem of machine learning algorithms interpretability is finally discussed, providing a future perspective on trustworthy artificial intelligence.Relevance statement The growing synergy between artificial intelligence and medicine fosters predictive models aiding physicians. Machine learning classifiers, from shallow learning to deep learning, are offering crucial insights for the development of clinical decision support systems in healthcare. Explainability is a key feature of models that leads systems toward integration into clinical practice. Key points • Training a shallow classifier requires extracting disease-related features from region of interests (e.g., radiomics).• Deep classifiers implement automatic feature extraction and classification.• The classifier selection is based on data and computational resources availability, task, and explanation needs.

Recent developments in machine learning modeling methods for hypertension treatment

Hypertension is the leading cause of cardiovascular complications. This review focuses on the advancements in medical artificial intelligence (AI) models aimed at individualized treatment for hypertension, with particular emphasis on the approach to time-series big data on blood pressure and the development of interpretable medical AI models. The digitalization of daily blood pressure records and the downsizing of measurement devices enable the accumulation and utilization of time-series data. As mainstream blood pressure data shift from snapshots to time series, the clinical significance of blood pressure variability will be clarified. The time-series blood pressure prediction model demonstrated the capability to forecast blood pressure variabilities with a reasonable degree of accuracy for up to four weeks in advance. In recent years, various explainable AI techniques have been proposed for different purposes of model interpretation. It is essential to select the appropriate technique based on the clinical aspects; for example, actionable path-planning techniques can present individualized intervention plans to efficiently improve outcomes such as hypertension. Despite considerable progress in this field, challenges remain, such as the need for the prospective validation of AI-driven interventions and the development of comprehensive systems that integrate multiple AI methods. Future research should focus on addressing these challenges and refining the AI models to ensure their practical applicability in real-world clinical settings. Furthermore, the implementation of interdisciplinary collaborations among AI experts, clinicians, and healthcare providers are crucial to further optimizing and validate AI-driven solutions for hypertension management.

Shedding light on ai in radiology: A systematic review and taxonomy of eye gaze-driven interpretability in deep learning

X-ray imaging plays a crucial role in diagnostic medicine. Yet, a significant portion of the global population lacks access to this essential technology due to a shortage of trained radiologists. Eye-tracking data and deep learning models can enhance X-ray analysis by mapping expert focus areas, guiding automated anomaly detection, optimizing workflow efficiency, and bolstering training methods for novice radiologists. However, the literature shows contradictory results regarding the usefulness of eye-tracking data in deep-learning architectures for abnormality detection. We argue that these discrepancies between studies in the literature are due to (a) the way eye-tracking data is (or is not) processed, (b) the types of deep learning architectures chosen, and (c) the type of application that these architectures will have. We conducted a systematic literature review using PRISMA to address these contradicting results. We analyzed 60 studies that incorporated eye-tracking data in a deep-learning approach for different application goals in radiology. We performed a comparative analysis to understand if eye gaze data contains feature maps that can be useful under a deep learning approach and whether they can promote more interpretable predictions. To the best of our knowledge, this is the first survey in the area that performs a thorough investigation of eye gaze data processing techniques and their impacts in different deep learning architectures for applications such as error detection, classification, object detection, expertise level analysis, fatigue estimation and human attention prediction in medical imaging data. Our analysis resulted in two main contributions: (1) taxonomy that first divides the literature by task, enabling us to analyze the value eye movement can bring for each case and build guidelines regarding architectures and gaze processing techniques adequate for each application, and (2) an overall analysis of how eye gaze data can promote explainability in radiology.

Evidence-based XAI: An empirical approach to design more effective and explainable decision support systems

This paper proposes a user study aimed at evaluating the impact of Class Activation Maps (CAMs) as an eXplainable AI (XAI) method in a radiological diagnostic task, the detection of thoracolumbar (TL) fractures from vertebral X-rays. In particular, we focus on two oft-neglected features of CAMs, that is granularity and coloring, in terms of what features, lower-level vs higher-level, should the maps highlight and adopting which coloring scheme, to bring better impact to the decision-making process, both in terms of diagnostic accuracy (that is effectiveness) and of user-centered dimensions, such as perceived confidence and utility (that is satisfaction), depending on case complexity, AI accuracy, and user expertise. Our findings show that lower-level features CAMs, which highlight more focused anatomical landmarks, are associated with higher diagnostic accuracy than higher-level features CAMs, particularly among experienced physicians. Moreover, despite the intuitive appeal of semantic CAMs, traditionally colored CAMs consistently yielded higher diagnostic accuracy across all groups. Our results challenge some prevalent assumptions in the XAI field and emphasize the importance of adopting an evidence-based and human-centered approach to design and evaluate AI- and XAI-assisted diagnostic tools. To this aim, the paper also proposes a hierarchy of evidence framework to help designers and practitioners choose the XAI solutions that optimize performance and satisfaction on the basis of the strongest evidence available or to focus on the gaps in the literature that need to be filled to move from opinionated and eminence-based research to one more based on empirical evidence and end-user work and preferences.

Designing explainable AI to improve human-AI team performance: A medical stakeholder-driven scoping review

The rise of complex AI systems in healthcare and other sectors has led to a growing area of research called Explainable AI (XAI) designed to increase transparency. In this area, quantitative and qualitative studies focus on improving user trust and task performance by providing system- and prediction-level XAI features. We analyze stakeholder engagement events (interviews and workshops) on the use of AI for kidney transplantation. From this we identify themes which we use to frame a scoping literature review on current XAI features. The stakeholder engagement process lasted over nine months covering three stakeholder group's workflows, determining where AI could intervene and assessing a mock XAI decision support system. Based on the stakeholder engagement, we identify four major themes relevant to designing XAI systems - 1) use of AI predictions, 2) information included in AI predictions, 3) personalization of AI predictions for individual differences, and 4) customizing AI predictions for specific cases. Using these themes, our scoping literature review finds that providing AI predictions before, during, or after decision-making could be beneficial depending on the complexity of the stakeholder's task. Additionally, expert stakeholders like surgeons prefer minimal to no XAI features, AI prediction, and uncertainty estimates for easy use cases. However, almost all stakeholders prefer to have optional XAI features to review when needed, especially in hard-to-predict cases. The literature also suggests that providing both system- and prediction-level information is necessary to build the user's mental model of the system appropriately. Although XAI features improve users' trust in the system, human-AI team performance is not always enhanced. Overall, stakeholders prefer to have agency over the XAI interface to control the level of information based on their needs and task complexity. We conclude with suggestions for future research, especially on customizing XAI features based on preferences and tasks.

Predicting and analyzing the algal population dynamics of a grass-type lake with explainable machine learning

Algal blooms, exacerbated by climate change and eutrophication, have emerged as a global concern. In this study, we introduce a novel interpretable machine learning (ML) workflow tailored for investigating the dynamics of algal populations in grass-type lakes, Liangzi lake. Utilizing seven ML methods and incorporating the covariance matrix adaptation evolution strategy (CMA-ES), we predict algal density across three distinct time periods, resulting in the construction of a total of 30 ML models. The CMA-ES-CatBoost model consistently demonstrates superior predictive accuracy and generalization capability across these periods. Through the collective validation of various interpretable tools, we identify water temperature and permanganate index as the two most critical water quality parameters (WQIs) influencing algal density in Liangzi Lake. Additionally, we quantify the independent and interactive effects of WQIs on algal density, pinpointing key thresholds and trends. Furthermore, we determine the minimum combination of WQIs that achieves near-optimal predictive performance, striking a balance between accuracy and cost-effectiveness. These findings offer a scientific and economically efficient foundation for governmental agencies to formulate strategies for water quality management and sustainable development.

A novel interpretable deep transfer learning combining diverse learnable parameters for improved T2D prediction based on single-cell gene regulatory networks

Accurate deep learning (DL) models to predict type 2 diabetes (T2D) are concerned not only with targeting the discrimination task but also with learning useful feature representation. However, existing DL tools are far from perfect and do not provide appropriate interpretation as a guideline to explain and promote superior performance in the target task. Therefore, we provide an interpretable approach for our presented deep transfer learning (DTL) models to overcome such drawbacks, working as follows. We utilize several pre-trained models including SEResNet152, and SEResNeXT101. Then, we transfer knowledge from pre-trained models via keeping the weights in the convolutional base (i.e., feature extraction part) while modifying the classification part with the use of Adam optimizer to deal with classifying healthy controls and T2D based on single-cell gene regulatory network (SCGRN) images. Another DTL models work in a similar manner but just with keeping weights of the bottom layers in the feature extraction unaltered while updating weights of consecutive layers through training from scratch. Experimental results on the whole 224 SCGRN images using five-fold cross-validation show that our model (TFeSEResNeXT101) achieving the highest average balanced accuracy (BAC) of 0.97 and thereby significantly outperforming the baseline that resulted in an average BAC of 0.86. Moreover, the simulation study demonstrated that the superiority is attributed to the distributional conformance of model weight parameters obtained with Adam optimizer when coupled with weights from a pre-trained model.

Explainable Model Using Shapley Additive Explanations Approach on Wound Infection after Wide Soft Tissue Sarcoma Resection: "Big Data" Analysis Based on Health Insurance Review and Assessment Service Hub

Background and Objectives: Soft tissue sarcomas represent a heterogeneous group of malignant mesenchymal tissues. Despite their low prevalence, soft tissue sarcomas present clinical challenges for orthopedic surgeons owing to their aggressive nature, and perioperative wound infections. However, the low prevalence of soft tissue sarcomas has hindered the availability of large-scale studies. This study aimed to analyze wound infections after wide resection in patients with soft tissue sarcomas by employing big data analytics from the Hub of the Health Insurance Review and Assessment Service (HIRA). Materials and Methods: Patients who underwent wide excision of soft tissue sarcomas between 2010 and 2021 were included. Data were collected from the HIRA database of approximately 50 million individuals' information in the Republic of Korea. The data collected included demographic information, diagnoses, prescribed medications, and surgical procedures. Random forest has been used to analyze the major associated determinants. A total of 10,906 observations with complete data were divided into training and validation sets in an 80:20 ratio (8773 vs. 2193 cases). Random forest permutation importance was employed to identify the major predictors of infection and Shapley Additive Explanations (SHAP) values were derived to analyze the directions of associations with predictors. Results: A total of 10,969 patients who underwent wide excision of soft tissue sarcomas were included. Among the study population, 886 (8.08%) patients had post-operative infections requiring surgery. The overall transfusion rate for wide excision was 20.67% (2267 patients). Risk factors among the comorbidities of each patient with wound infection were analyzed and dependence plots of individual features were visualized. The transfusion dependence plot reveals a distinctive pattern, with SHAP values displaying a negative trend for individuals without blood transfusions and a positive trend for those who received blood transfusions, emphasizing the substantial impact of blood transfusions on the likelihood of wound infection. Conclusions: Using the machine learning random forest model and the SHAP values, the perioperative transfusion, male sex, old age, and low SES were important features of wound infection in soft-tissue sarcoma patients.

Identifying Reproducibly Important EEG Markers of Schizophrenia with an Explainable Multi-Model Deep Learning Approach

The diagnosis of schizophrenia (SZ) can be challenging due to its diverse symptom presentation. As such, many studies have sought to identify diagnostic biomarkers of SZ using explainable machine learning methods. However, the generalizability of identified biomarkers in many machine learning-based studies is highly questionable given that most studies only analyze explanations from a small number of models. In this study, we present (1) a novel feature interaction-based explainability approach and (2) several new approaches for summarizing multi-model explanations. We implement our approach within the context of electroencephalogram (EEG) spectral power data. We further analyze both training and test set explanations with the goal of extracting generalizable insights from the models. Importantly, our analyses identify effects of SZ upon the α, β, and θ frequency bands, the left hemisphere of the brain, and interhemispheric interactions across a majority of folds. We hope that our analysis will provide helpful insights into SZ and inspire the development of robust approaches for identifying neuropsychiatric disorder biomarkers from explainable machine learning models.

Explaining decisions of a light-weight deep neural network for real-time coronary artery disease classification in magnetic resonance imaging

In certain healthcare settings, such as emergency or critical care units, where quick and accurate real-time analysis and decision-making are required, the healthcare system can leverage the power of artificial intelligence (AI) models to support decision-making and prevent complications. This paper investigates the optimization of healthcare AI models based on time complexity, hyper-parameter tuning, and XAI for a classification task. The paper highlights the significance of a lightweight convolutional neural network (CNN) for analysing and classifying Magnetic Resonance Imaging (MRI) in real-time and is compared with CNN-RandomForest (CNN-RF). The role of hyper-parameter is also examined in finding optimal configurations that enhance the model's performance while efficiently utilizing the limited computational resources. Finally, the benefits of incorporating the XAI technique (e.g. GradCAM and Layer-wise Relevance Propagation) in providing transparency and interpretable explanations of AI model predictions, fostering trust, and error/bias detection are explored. Our inference time on a MacBook laptop for 323 test images of size 100x100 is only 2.6 sec, which is merely 8 milliseconds per image while providing comparable classification accuracy with the ensemble model of CNN-RF classifiers. Using the proposed model, clinicians/cardiologists can achieve accurate and reliable results while ensuring patients' safety and answering questions imposed by the General Data Protection Regulation (GDPR). The proposed investigative study will advance the understanding and acceptance of AI systems in connected healthcare settings.

CAD-RADS scoring of coronary CT angiography with Multi-Axis Vision Transformer: A clinically-inspired deep learning pipeline

BACKGROUND AND OBJECTIVE

The standard non-invasive imaging technique used to assess the severity and extent of Coronary Artery Disease (CAD) is Coronary Computed Tomography Angiography (CCTA). However, manual grading of each patient's CCTA according to the CAD-Reporting and Data System (CAD-RADS) scoring is time-consuming and operator-dependent, especially in borderline cases. This work proposes a fully automated, and visually explainable, deep learning pipeline to be used as a decision support system for the CAD screening procedure. The pipeline performs two classification tasks: firstly, identifying patients who require further clinical investigations and secondly, classifying patients into subgroups based on the degree of stenosis, according to commonly used CAD-RADS thresholds.

METHODS

The pipeline pre-processes multiplanar projections of the coronary arteries, extracted from the original CCTAs, and classifies them using a fine-tuned Multi-Axis Vision Transformer architecture. With the aim of emulating the current clinical practice, the model is trained to assign a per-patient score by stacking the bi-dimensional longitudinal cross-sections of the three main coronary arteries along channel dimension. Furthermore, it generates visually interpretable maps to assess the reliability of the predictions.

RESULTS

When run on a database of 1873 three-channel images of 253 patients collected at the Monzino Cardiology Center in Milan, the pipeline obtained an AUC of 0.87 and 0.93 for the two classification tasks, respectively.

CONCLUSION

According to our knowledge, this is the first model trained to assign CAD-RADS scores learning solely from patient scores and not requiring finer imaging annotation steps that are not part of the clinical routine.

XL1R-Net: Explainable AI-driven improved L1-regularized deep neural architecture for NSCLC biomarker identification

BACKGROUND AND OBJECTIVE

Non-small cell lung cancer (NSCLC) exhibits intrinsic molecular heterogeneity, primarily driven by the mutation of specific biomarkers. Identification of these biomarkers would assist not only in distinguishing NSCLC into its major subtypes - Adenocarcinoma and Squamous Cell Carcinoma, but also in developing targeted therapy. Medical practitioners use one or more types of omic data to identify these biomarkers, copy number variation (CNV) being one such type. CNV provides a measure of genomic instability, which is considered a hallmark of carcinoma. However, the CNV data has not received much attention for biomarker identification. This paper aims to identify biomarkers for NSCLC using CNV data.

METHODS

An eXplainable AI (XAI)-driven L1-regularized deep learning architecture, XL1R-Net, is proposed that introduces a novel modification of the standard L1-regularized gradient descent algorithm to arrive at an improved deep neural classifier for NSCLC subtyping. Further, XAI-based feature identification has been used to leverage the trained classifier to uncover a set of twenty NCSLC-relevant biomarkers.

RESULTS

The identified biomarkers are evaluated based on their classification performance and clinical relevance. Using Multilayer Perceptron (MLP)-based model, a classification accuracy of 84.95% using 10-fold cross-validation is achieved. Moreover, the statistical significance test on the classification performance also revealed the superiority of the MLP model over the competitive machine learning models. Further, the publicly available Drug-Gene Interaction Database reveals twelve of the identified biomarkers as potentially druggable. The K-M Plotter tool was used to verify eighteen of the identified biomarkers with a high probability of predicting NSCLC patients' likelihood of survival. While nine of the identified biomarkers confirm the recent literature, five find mention in the OncoKB Gene List.

CONCLUSION

A set of seven novel biomarkers that have not been reported in the literature could be investigated for their potential contribution towards NSCLC therapy. Given NSCLC's genetic diversity, using only one omics data type may not adequately capture the tumor's complexity. Multiomics data and its integration with other sources will be examined in the future to better understand NSCLC heterogeneity.

CLARUS: An interactive explainable AI platform for manual counterfactuals in graph neural networks

BACKGROUND

Lack of trust in artificial intelligence (AI) models in medicine is still the key blockage for the use of AI in clinical decision support systems (CDSS). Although AI models are already performing excellently in systems medicine, their black-box nature entails that patient-specific decisions are incomprehensible for the physician. Explainable AI (XAI) algorithms aim to "explain" to a human domain expert, which input features influenced a specific recommendation. However, in the clinical domain, these explanations must lead to some degree of causal understanding by a clinician.

RESULTS

We developed the CLARUS platform, aiming to promote human understanding of graph neural network (GNN) predictions. CLARUS enables the visualisation of patient-specific networks, as well as, relevance values for genes and interactions, computed by XAI methods, such as GNNExplainer. This enables domain experts to gain deeper insights into the network and more importantly, the expert can interactively alter the patient-specific network based on the acquired understanding and initiate re-prediction or retraining. This interactivity allows us to ask manual counterfactual questions and analyse the effects on the GNN prediction.

CONCLUSION

We present the first interactive XAI platform prototype, CLARUS, that allows not only the evaluation of specific human counterfactual questions based on user-defined alterations of patient networks and a re-prediction of the clinical outcome but also a retraining of the entire GNN after changing the underlying graph structures. The platform is currently hosted by the GWDG on https://rshiny.gwdg.de/apps/clarus/.

Who might encounter hard-braking while speeding? Analysis for regular speeders using low-frequency taxi trajectories on arterial roads and explainable AI

Regular speeders are those who commit speeding recidivism during a period. Among their speeding behaviors, some occurring in specific scenarios may cause more hazards to road users. Therefore, there is a need to evaluate the driving risks if the regular speeders have different speeding propensities. This study considers speeding-related hard-braking events (SHEs) as a safety surrogate measure and recognizes the regular speeders who encounter at least one SHEs during the study period as risky individuals. To identify speeding behaviors and hard-braking events from low-frequency GPS trajectories, we compare the average travel speed between pairwise adjacent GPS points to the posted speed limit and examine the speed curve and the corresponding travel distance between these GPS points, respectively. Thereafter, a logistic model, XGBoost, and three 1D Convolutional Neural Networks (CNNs) including AlexNet CNN, Mini-AlexNet CNN, and Simple CNN are respectively developed to recognize the regular speeders who encountered SHEs based on their speeding propensities. The proposed Mini-AlexNet CNN achieves a global F1-score of 91% and recall of 90% on the testing data, which are superior to other models. Further, the study uses the Shapley Additive exPlanation (SHAP) framework to visually interpret the contribution of speeding propensities on SHE likelihood. It is found that speeding by 50% or greater for no more than 285 m is the most dangerous kind among all the speeding behaviors. Speeding on roads without bicycle lanes or on roads with roadside parking and excessive accesses increases the probability of encountering SHEs. Based on the analyses, we put forward tailored recommendations that aim to restrict hazard-related speeding behaviors rather than speeding behaviors of all kinds.

Towards interactive explanation-based nutrition virtual coaching systems

The awareness about healthy lifestyles is increasing, opening to personalized intelligent health coaching applications. A demand for more than mere suggestions and mechanistic interactions has driven attention to nutrition virtual coaching systems (NVC) as a bridge between human-machine interaction and recommender, informative, persuasive, and argumentation systems. NVC can rely on data-driven opaque mechanisms. Therefore, it is crucial to enable NVC to explain their doing (i.e., engaging the user in discussions (via arguments) about dietary solutions/alternatives). By doing so, transparency, user acceptance, and engagement are expected to be boosted. This study focuses on NVC agents generating personalized food recommendations based on user-specific factors such as allergies, eating habits, lifestyles, and ingredient preferences. In particular, we propose a user-agent negotiation process entailing run-time feedback mechanisms to react to both recommendations and related explanations. Lastly, the study presents the findings obtained by the experiments conducted with multi-background participants to evaluate the acceptability and effectiveness of the proposed system. The results indicate that most participants value the opportunity to provide feedback and receive explanations for recommendations. Additionally, the users are fond of receiving information tailored to their needs. Furthermore, our interactive recommendation system performed better than the corresponding traditional recommendation system in terms of effectiveness regarding the number of agreements and rounds.

Social media insights into spatio-temporal emotional responses to COVID-19 crisis

The Coronavirus pandemic has presented multifaceted challenges in urban emotional well-being and mental health management. Our study presents a spatio-temporal sentiment mining (STSM) framework to address these challenges, focusing on the space-time geography and environmental psychology. This framework analyzes the distribution and trends of 6 categories of public sentiments in Shanghai during the COVID-19 crisis, considering the potential urban spatial influencing factors. The research specifically draws on social media data temporally coinciding with the spread of COVID-19 and the pre-trained language model RoBERTa-wwm-ext to classify public sentiment, in order to characterize the distribution and trends of dominant urban sentiment under the influence of epidemic at different phases. The interactions between urban geospatial features and sentiments are further modelled and explained using LightGBM algorithm and SHapley Additive exPlanations (SHAP) technique. The experimental findings reveal the subtle yet dynamic impact of the urban environment on the long-term spatial variation and trends of public sentiment under the epidemic, with green spaces and socio-economic status emerging as significant factors. Regions with higher permanent population consumption demonstrated more positive sentiments, underscoring the significance of socio-economic factors in urban planning and public health policy. This research offers the most extensive analysis to date on the influence of urban characteristics on public sentiment during Shanghai's epidemic life cycle also lays the groundwork for applying the STSM framework in future crises beyond COVID-19.

Knowledge Graphs and Their Applications in Drug Discovery

Knowledge graphs represent information in the form of entities and relationships between those entities. Such a representation has multiple potential applications in drug discovery, including democratizing access to biomedical data, contextualizing or visualizing that data, and generating novel insights through the application of machine learning approaches. Knowledge graphs put data into context and therefore offer the opportunity to generate explainable predictions, which is a key topic in contemporary artificial intelligence. In this chapter, we outline some of the factors that need to be considered when constructing biomedical knowledge graphs, examine recent advances in mining such systems to gain insights for drug discovery, and identify potential future areas for further development.

Enlightening the path to NSCLC biomarkers: Utilizing the power of XAI-guided deep learning

BACKGROUND AND OBJECTIVE

The early diagnosis of Non-small cell lung cancer (NSCLC) is of prime importance to improve the patient's survivability and quality of life. Being a heterogeneous disease at the molecular and cellular level, the biomarkers responsible for the heterogeneity aid in distinguishing NSCLC into its prominent subtypes-adenocarcinoma and squamous cell carcinoma. Moreover, if identified, these biomarkers could pave the path to targeted therapy. Through this work, a novel explainable AI (XAI)-guided deep learning framework is proposed that assists in discovering a set of significant NSCLC-relevant biomarkers using methylation data.

METHODS

The proposed framework is divided into two blocks- the first block combines an autoencoder and a neural network to classify NSCLC instances. The second block utilizes various eXplainable AI (XAI) methods, namely IntegratedGradients, GradientSHAP, and DeepLIFT, to discover a set of seven significant biomarkers.

RESULTS

The classification performance of the biomarkers discovered using the proposed framework is evaluated by employing multiple machine learning algorithms, among which the Multilayer Perceptron (MLP) algorithm-based model outperforms others, yielding a 10-fold cross-validation accuracy of 91.53%. An improved accuracy of 96.37% is achieved by integrating RNA-Seq, CNV, and methylation data. On performing statistical analysis using the Friedman and Nemenyi tests, the MLP model is found to be significantly better than other machine learning-based models. Further, the clinical efficacy of the resultant biomarkers is established based on their potential druggability, the likelihood of predicting NSCLC patients' survival, gene-disease association, and biological pathways targeted by them. While the biomarkers C18orf18, CCNT2, THOP1, and TNPO2, are found potentially druggable, the biomarkers CCDC15, SNORA9, THOP1, and TNPO2 are found prognostically relevant. On further analysis, some of the discovered biomarkers are found to be associated with around 104 diseases. Moreover, five KEGG, ten Reactome, and three Wiki pathways are found to be triggered by the biomarkers discovered.

CONCLUSION

In summary, the proposed framework uncovers a set of clinically effective biomarkers that accurately classify NSCLC. As a future course of work, efforts would be made to combine a variety of omics data with histopathological data to unveil more precise biomarkers for devising personalized therapy.

Applying BERT and ChatGPT for Sentiment Analysis of Lyme Disease in Scientific Literature

This chapter presents a practical guide for conducting sentiment analysis using Natural Language Processing (NLP) techniques in the domain of tick-borne disease text. The aim is to demonstrate the process of how the presence of bias in the discourse surrounding chronic manifestations of the disease can be evaluated. The goal is to use a dataset of 5643 abstracts collected from scientific journals on the topic of chronic Lyme disease to demonstrate using Python, the steps for conducting sentiment analysis using pretrained language models and the process of validating the preliminary results using both interpretable machine learning tools, as well as a novel methodology of leveraging emerging state-of-the-art large language models like ChatGPT. This serves as a useful resource for researchers and practitioners interested in using NLP techniques for sentiment analysis in the medical domain.

Assessing the relevance of mental health factors in fibromyalgia severity: A data-driven case study using explainable AI

BACKGROUND AND OBJECTIVE

Fibromyalgia is a chronic disease that causes pain and affects patients' quality of life. Current treatments focus on pharmacological therapies for pain reduction. However, patients' psychological well-being is also affected, with depression and pain catastrophizing being common. This research addresses the clinicians' need to assess the influence of mental health factors on FM severity compared to pain factors.

METHODS

A co-development study between FM clinicians and data scientists analyzed data from 166 FM-diagnosed patients to assess the influence of mental health factors on FM severity in comparison to pain factors. The study used the Polysymptomatic Distress Scale (PDS) and Fibromyalgia Impact Questionnaire (FIQ) as FM severity indicators and collected 15 variables including regarding demographics, pain intensity perceived, and mental health factors. The team used an author's developed framework to identify the optimal FM severity classifier and explainability by selecting a number of features that lead to obtaining the best classification result. Machine learning classifiers employed in the framework were: decision trees, logistic regression, support vector machines, random forests, AdaBoost, extra trees, and RUSBoost. Explainability analyses were conducted using the following explainable AI techniques: SHapley Additive exPlanations (SHAP), Partial Dependence Plot (PDP), and Mean Decrease Impurity (MDI).

RESULTS

A balanced random forest with 6 features achieved the best performance with PDS (AUC_ROC, mean = 0.81, std = 0.07). Being FIQ the target variable, due to the imbalance in FM severity levels, a binary and a multiclass classification approaches were considered achieving the optimal performance, respectively, a logistic regression classifier (AUC_ROC, mean = 0.83, std = 0.08) with 6 selected features, and a random forest (AUC_ROC, mean = 0.91, std = 0.04) with 8 selected features. Next, the explainability analysis determined mental health factors were found to be more relevant than pain perceived factors for FM severity.

CONCLUSIONS

This study's findings, validated by clinicians, are potentially aligned with FM international guidelines that promote non-pharmacological interventions such as promoting mental well-being of FM patients.

Shedding light on the black box of a neural network used to detect prostate cancer in whole slide images by occlusion-based explainability

Diagnostic histopathology faces increasing demands due to aging populations and expanding healthcare programs. Semi-automated diagnostic systems employing deep learning methods are one approach to alleviate this pressure. The learning models for histopathology are inherently complex and opaque from the user's perspective. Hence different methods have been developed to interpret their behavior. However, relatively limited attention has been devoted to the connection between interpretation methods and the knowledge of experienced pathologists. The main contribution of this paper is a method for comparing morphological patterns used by expert pathologists to detect cancer with the patterns identified as important for inference of learning models. Given the patch-based nature of processing large-scale histopathological imaging, we have been able to show statistically that the VGG16 model could utilize all the structures that are observable by the pathologist, given the patch size and scan resolution. The results show that the neural network approach to recognizing prostatic cancer is similar to that of a pathologist at medium optical resolution. The saliency maps identified several prevailing histomorphological features characterizing carcinoma, e.g., single-layered epithelium, small lumina, and hyperchromatic nuclei with halo. A convincing finding was the recognition of their mimickers in non-neoplastic tissue. The method can also identify differences, i.e., standard patterns not used by the learning models and new patterns not yet used by pathologists. Saliency maps provide added value for automated digital pathology to analyze and fine-tune deep learning systems and improve trust in computer-based decisions.

Using Explainable Artificial Intelligence to Obtain Efficient Seizure-Detection Models Based on Electroencephalography Signals

Epilepsy is a condition that affects 50 million individuals globally, significantly impacting their quality of life. Epileptic seizures, a transient occurrence, are characterized by a spectrum of manifestations, including alterations in motor function and consciousness. These events impose restrictions on the daily lives of those affected, frequently resulting in social isolation and psychological distress. In response, numerous efforts have been directed towards the detection and prevention of epileptic seizures through EEG signal analysis, employing machine learning and deep learning methodologies. This study presents a methodology that reduces the number of features and channels required by simpler classifiers, leveraging Explainable Artificial Intelligence (XAI) for the detection of epileptic seizures. The proposed approach achieves performance metrics exceeding 95% in accuracy, precision, recall, and F1-score by utilizing merely six features and five channels in a temporal domain analysis, with a time window of 1 s. The model demonstrates robust generalization across the patient cohort included in the database, suggesting that feature reduction in simpler models-without resorting to deep learning-is adequate for seizure detection. The research underscores the potential for substantial reductions in the number of attributes and channels, advocating for the training of models with strategically selected electrodes, and thereby supporting the development of effective mobile applications for epileptic seizure detection.

Identifying predictive biomarkers for repetitive transcranial magnetic stimulation response in depression patients with explainability

Repetitive Transcranial Magnetic Stimulation (rTMS) is an evidence-based treatment for depression. However, the patterns of response to this treatment modality are inconsistent. Whilst many people see a significant reduction in the severity of their depression following rTMS treatment, some patients do not. To support and improve patient outcomes, recent work is exploring the possibility of using Machine Learning to predict rTMS treatment outcomes. Our proposed model is the first to combine functional magnetic resonance imaging (fMRI) connectivity with deep learning techniques to predict treatment outcomes before treatment starts. Furthermore, with the use of Explainable AI (XAI) techniques, we identify potential biomarkers that may discriminate between rTMS responders and non-responders. Our experiments utilize 200 runs of repeated bootstrap sampling on two rTMS datasets. We compare performances between our proposed feedforward deep neural network against existing methods, and compare the average accuracy, balanced accuracy and F1-score on a held-out test set. The results of these experiments show that our model outperforms existing methods with an average accuracy of 0.9423, balanced accuracy of 0.9423, and F1-score of 0.9461 in a sample of 61 patients. We found that functional connectivity measures between the Subgenual Anterior Cingulate Cortex and Centeral Opercular Cortex are a key determinant of rTMS treatment response. This knowledge provides psychiatrists with further information to explore the potential mechanisms of responses to rTMS treatment. Our developed prototype is ready to be deployed across large datasets in multiple centres and different countries.

Prediction of oxygen supplementation by a deep-learning model integrating clinical parameters and chest CT images in COVID-19

PURPOSE

As of March 2023, the number of patients with COVID-19 worldwide is declining, but the early diagnosis of patients requiring inpatient treatment and the appropriate allocation of limited healthcare resources remain unresolved issues. In this study we constructed a deep-learning (DL) model to predict the need for oxygen supplementation using clinical information and chest CT images of patients with COVID-19.

MATERIALS AND METHODS

We retrospectively enrolled 738 patients with COVID-19 for whom clinical information (patient background, clinical symptoms, and blood test findings) was available and chest CT imaging was performed. The initial data set was divided into 591 training and 147 evaluation data. We developed a DL model that predicted oxygen supplementation by integrating clinical information and CT images. The model was validated at two other facilities (n = 191 and n = 230). In addition, the importance of clinical information for prediction was assessed.

RESULTS

The proposed DL model showed an area under the curve (AUC) of 89.9% for predicting oxygen supplementation. Validation from the two other facilities showed an AUC > 80%. With respect to interpretation of the model, the contribution of dyspnea and the lactate dehydrogenase level was higher in the model.

CONCLUSIONS

The DL model integrating clinical information and chest CT images had high predictive accuracy. DL-based prediction of disease severity might be helpful in the clinical management of patients with COVID-19.

One label is all you need: Interpretable AI-enhanced histopathology for oncology

Artificial Intelligence (AI)-enhanced histopathology presents unprecedented opportunities to benefit oncology through interpretable methods that require only one overall label per hematoxylin and eosin (H&E) slide with no tissue-level annotations. We present a structured review of these methods organized by their degree of verifiability and by commonly recurring application areas in oncological characterization. First, we discuss morphological markers (tumor presence/absence, metastases, subtypes, grades) in which AI-identified regions of interest (ROIs) within whole slide images (WSIs) verifiably overlap with pathologist-identified ROIs. Second, we discuss molecular markers (gene expression, molecular subtyping) that are not verified via H&E but rather based on overlap with positive regions on adjacent tissue. Third, we discuss genetic markers (mutations, mutational burden, microsatellite instability, chromosomal instability) that current technologies cannot verify if AI methods spatially resolve specific genetic alterations. Fourth, we discuss the direct prediction of survival to which AI-identified histopathological features quantitatively correlate but are nonetheless not mechanistically verifiable. Finally, we discuss in detail several opportunities and challenges for these one-label-per-slide methods within oncology. Opportunities include reducing the cost of research and clinical care, reducing the workload of clinicians, personalized medicine, and unlocking the full potential of histopathology through new imaging-based biomarkers. Current challenges include explainability and interpretability, validation via adjacent tissue sections, reproducibility, data availability, computational needs, data requirements, domain adaptability, external validation, dataset imbalances, and finally commercialization and clinical potential. Ultimately, the relative ease and minimum upfront cost with which relevant data can be collected in addition to the plethora of available AI methods for outcome-driven analysis will surmount these current limitations and achieve the innumerable opportunities associated with AI-driven histopathology for the benefit of oncology.

Explainable machine learning for the prediction and assessment of complex drought impacts

Drought is a common and costly natural disaster with broad social, economic, and environmental impacts. Machine learning (ML) has been widely applied in scientific research because of its outstanding performance on predictive tasks. However, for practical applications like disaster monitoring and assessment, the cost of the models failure, especially false negative predictions, might significantly affect society. Stakeholders are not satisfied with or do not "trust" the predictions from a so-called black box. The explainability of ML models becomes progressively crucial in studying drought and its impacts. In this work, we propose an explainable ML pipeline using the XGBoost model and SHAP model based on a comprehensive database of drought impacts in the U.S. The XGBoost models significantly outperformed the baseline models in predicting the occurrence of multi-dimensional drought impacts derived from the text-based Drought Impact Reporter, attaining an average F2 score of 0.883 at the national level and 0.942 at the state level. The interpretation of the models at the state scale indicates that the Standardized Precipitation Index (SPI) and Standardized Temperature Index (STI) contribute significantly to predicting multi-dimensional drought impacts. The time scalar, importance, and relationships of the SPI and STI vary depending on the types of drought impacts and locations. The patterns between the SPI variables and drought impacts indicated by the SHAP values reveal an expected relationship in which negative SPI values positively contribute to complex drought impacts. The explainability based on the SPI variables improves the trustworthiness of the XGBoost models. Overall, this study reveals promising results in accurately predicting complex drought impacts and rendering the relationships between the impacts and indicators more interpretable. This study also reveals the potential of utilizing explainable ML for the general social good to help stakeholders better understand the multi-dimensional drought impacts at the regional level and motivate appropriate responses.

Explainable convolutional neural networks for assessing head and neck cancer histopathology

PURPOSE

Although neural networks have shown remarkable performance in medical image analysis, their translation into clinical practice remains difficult due to their lack of interpretability. An emerging field that addresses this problem is Explainable AI.

METHODS

Here, we aimed to investigate the ability of Convolutional Neural Networks (CNNs) to classify head and neck cancer histopathology. To this end, we manually annotated 101 histopathological slides of locally advanced head and neck squamous cell carcinoma. We trained a CNN to classify tumor and non-tumor tissue, and another CNN to semantically segment four classes - tumor, non-tumor, non-specified tissue, and background. We applied Explainable AI techniques, namely Grad-CAM and HR-CAM, to both networks and explored important features that contributed to their decisions.

RESULTS

The classification network achieved an accuracy of 89.9% on previously unseen data. Our segmentation network achieved a class-averaged Intersection over Union score of 0.690, and 0.782 for tumor tissue in particular. Explainable AI methods demonstrated that both networks rely on features agreeing with the pathologist's expert opinion.

CONCLUSION

Our work suggests that CNNs can predict head and neck cancer with high accuracy. Especially if accompanied by visual explanations, CNNs seem promising for assisting pathologists in the assessment of cancer sections.

Artificial intelligence-based preventive, personalized and precision medicine for cardiovascular disease/stroke risk assessment in rheumatoid arthritis patients: a narrative review

The challenges associated with diagnosing and treating cardiovascular disease (CVD)/Stroke in Rheumatoid arthritis (RA) arise from the delayed onset of symptoms. Existing clinical risk scores are inadequate in predicting cardiac events, and conventional risk factors alone do not accurately classify many individuals at risk. Several CVD biomarkers consider the multiple pathways involved in the development of atherosclerosis, which is the primary cause of CVD/Stroke in RA. To enhance the accuracy of CVD/Stroke risk assessment in the RA framework, a proposed approach involves combining genomic-based biomarkers (GBBM) derived from plasma and/or serum samples with innovative non-invasive radiomic-based biomarkers (RBBM), such as measurements of synovial fluid, plaque area, and plaque burden. This review presents two hypotheses: (i) RBBM and GBBM biomarkers exhibit a significant correlation and can precisely detect the severity of CVD/Stroke in RA patients. (ii) Artificial Intelligence (AI)-based preventive, precision, and personalized (aiP3) CVD/Stroke risk AtheroEdge™ model (AtheroPoint™, CA, USA) that utilizes deep learning (DL) to accurately classify the risk of CVD/stroke in RA framework. The authors conducted a comprehensive search using the PRISMA technique, identifying 153 studies that assessed the features/biomarkers of RBBM and GBBM for CVD/Stroke. The study demonstrates how DL models can be integrated into the AtheroEdge™-aiP3 framework to determine the risk of CVD/Stroke in RA patients. The findings of this review suggest that the combination of RBBM with GBBM introduces a new dimension to the assessment of CVD/Stroke risk in the RA framework. Synovial fluid levels that are higher than normal lead to an increase in the plaque burden. Additionally, the review provides recommendations for novel, unbiased, and pruned DL algorithms that can predict CVD/Stroke risk within a RA framework that is preventive, precise, and personalized.