Toward Explainable Artificial Intelligence for Precision Pathology

Toward efficient slide-level grading of liver biopsy via explainable deep learning framework

In the context of chronic liver diseases, where variability in progression necessitates early and precise diagnosis, this study addresses the limitations of traditional histological analysis and the shortcomings of existing deep learning approaches. A novel patch-level classification model employing multi-scale feature extraction and fusion was developed to enhance the grading accuracy and interpretability of liver biopsies, analyzing 1322 cases across various staining methods. The study also introduces a slide-level aggregation framework, comparing different diagnostic models, to efficiently integrate local histological information. Results from extensive validation show that the slide-level model consistently achieved high F1 scores, notably 0.9 for inflammatory activity and steatosis, and demonstrated rapid diagnostic capabilities with less than one minute per slide on average. The patch-level model also performed well, with an F1 score of 0.64 for ballooning and 0.99 for other indicators, and proved transferable to public datasets. The conclusion drawn is that the proposed analytical framework offers a reliable basis for the diagnosis and treatment of chronic liver diseases, with the added benefit of robust interpretability, suggesting its practical utility in clinical settings.

[Foundation models in pathology]

Foundation models prepare neural networks for applications in specific domains, such as speech applications or image analysis, through self-supervised pretraining. These models can be adapted for specific applications, such as histopathological diagnostics. While adaptation still requires supervised training, AI applications based on foundation models achieve significantly better prediction accuracy with fewer training data compared to conventional approaches. This article introduces the topic and provides an overview of foundation models in pathology.

A comparative study of explainability methods for whole slide classification of lymph node metastases using vision transformers

Recent advancements in deep learning have shown promise in enhancing the performance of medical image analysis. In pathology, automated whole slide imaging has transformed clinical workflows by streamlining routine tasks and diagnostic and prognostic support. However, the lack of transparency of deep learning models, often described as black boxes, poses a significant barrier to their clinical adoption. This study evaluates various explainability methods for Vision Transformers, assessing their effectiveness in explaining the rationale behind their classification predictions on histopathological images. Using a Vision Transformer trained on the publicly available CAMELYON16 dataset comprising of 399 whole slide images of lymph node metastases of patients with breast cancer, we conducted a comparative analysis of a diverse range of state-of-the-art techniques for generating explanations through heatmaps, including Attention Rollout, Integrated Gradients, RISE, and ViT-Shapley. Our findings reveal that Attention Rollout and Integrated Gradients are prone to artifacts, while RISE and particularly ViT-Shapley generate more reliable and interpretable heatmaps. ViT-Shapley also demonstrated faster runtime and superior performance in insertion and deletion metrics. These results suggest that integrating ViT-Shapley-based heatmaps into pathology reports could enhance trust and scalability in clinical workflows, facilitating the adoption of explainable artificial intelligence in pathology.

Current methods in explainable artificial intelligence and future prospects for integrative physiology

Explainable artificial intelligence (XAI) is gaining importance in physiological research, where artificial intelligence is now used as an analytical and predictive tool for many medical research questions. The primary goal of XAI is to make AI models understandable for human decision-makers. This can be achieved in particular through providing inherently interpretable AI methods or by making opaque models and their outputs transparent using post hoc explanations. This review introduces XAI core topics and provides a selective overview of current XAI methods in physiology. It further illustrates solved and discusses open challenges in XAI research using existing practical examples from the medical field. The article gives an outlook on two possible future prospects: (1) using XAI methods to provide trustworthy AI for integrative physiological research and (2) integrating physiological expertise about human explanation into XAI method development for useful and beneficial human-AI partnerships.

Development of a Disease Model for Predicting Postoperative Delirium Using Combined Blood Biomarkers

OBJECTIVE

Postoperative delirium, a common neurocognitive complication after surgery and anesthesia, requires early detection for potential intervention. Herein, we constructed a multidimensional postoperative delirium risk-prediction model incorporating multiple demographic parameters and blood biomarkers to enhance prediction accuracy.

METHODS

We included 555 patients undergoing radical surgery for colorectal cancer. Demographic characteristics and lipid profiles were collected preoperatively, and perioperative anesthesia and surgical conditions were recorded; blood biomarkers were measured before and after surgery. The 3D-CAM scale was used to assess postoperative delirium occurrence within 3 days after surgery. Patients were divided into the postoperative delirium (N = 100) and non-postoperative delirium (N = 455) groups. Based on machine learning, linear and nine non-linear models were developed and compared to select the optimal model. Shapley value-interpretation methods and mediation analysis were used to assess feature importance and interaction.

RESULTS

The median age of the participants was 65 years (interquartile range: 56-71 years; 57.8% male). Among the 10 machine-learning models, the random forest model performed the best (validation cohort, area under the receiver operating characteristic curve of 0.795 [0.704-0.885]). Lipid profile (total cholesterol, triglycerides, and trimethylamine-N-oxide) levels were identified as key postoperative delirium predictors. Mediation analysis further confirmed mediating effects among total cholesterol, trimethylamine-N-oxide, and postoperative delirium; a nomogram model was developed as a web-based tool for external validation and use by other clinicians.

INTERPRETATION

Blood biomarkers are crucial in predicting postoperative delirium and aid anesthesiologists in identifying its risks in a timely manner. This model facilitates personalized perioperative management and reduces the occurrence of postoperative delirium.

TRIAL REGISTRATION

ChiCTR2300075723.

Nongenerative Artificial Intelligence in Medicine: Advancements and Applications in Supervised and Unsupervised Machine Learning

The use of artificial intelligence (AI) within pathology and health care has advanced extensively. We have accordingly witnessed an increased adoption of various AI tools that are transforming our approach to clinical decision support, personalized medicine, predictive analytics, automation, and discovery. The familiar and more reliable AI tools that have been incorporated within health care thus far fall mostly under the nongenerative AI domain, which includes supervised and unsupervised machine learning (ML) techniques. This review article explores how such nongenerative AI methods, rooted in traditional rules-based systems, enhance diagnostic accuracy, efficiency, and consistency within medicine. Key concepts and the application of supervised learning models (ie, classification and regression) such as decision trees, support vector machines, linear and logistic regression, K-nearest neighbor, and neural networks are explained along with the newer landscape of neural network-based nongenerative foundation models. Unsupervised learning techniques, including clustering, dimensionality reduction, and anomaly detection, are also discussed for their roles in uncovering novel disease subtypes or identifying outliers. Technical details related to the application of nongenerative AI algorithms for analyzing whole slide images are also highlighted. The performance, explainability, and reliability of nongenerative AI models essential for clinical decision-making is also reviewed, as well as challenges related to data quality, model interpretability, and risk of data drift. An understanding of which AI-ML models to employ and which shortcomings need to be addressed is imperative to safely and efficiently leverage, integrate, and monitor these traditional AI tools in clinical practice and research.

Explainable artificial intelligence of DNA methylation-based brain tumor diagnostics

We have recently developed a machine learning classifier that enables fast, accurate, and affordable classification of brain tumors based on genome-wide DNA methylation profiles that is widely employed in the clinic. Neuro-oncology research would benefit greatly from understanding the underlying artificial intelligence decision process, which currently remains unclear. Here, we describe an interpretable framework to explain the classifier's decisions. We show that functional genomic regions of various sizes are predominantly employed to distinguish between different tumor classes, ranging from enhancers and CpG islands to large-scale heterochromatic domains. We detect a high degree of genomic redundancy, with many genes distinguishing individual tumor classes, explaining the robustness of the classifier and revealing potential targets for further therapeutic investigation. We anticipate that our resource will build up trust in machine learning in clinical settings, foster biomarker discovery and development of compact point-of-care assays, and enable further epigenome research of brain tumors. Our interpretable framework is accessible to the research community via an interactive web application ( https://hovestadtlab.shinyapps.io/shinyMNP/ ).

Towards an interpretable deep learning model of cancer

Cancer is a manifestation of dysfunctional cell states. It emerges from an interplay of intrinsic and extrinsic factors that disrupt cellular dynamics, including genetic and epigenetic alterations, as well as the tumor microenvironment. This complexity can make it challenging to infer molecular causes for treating the disease. This may be addressed by system-wide computer models of cells, as they allow rapid generation and testing of hypotheses that would be too slow or impossible to perform in the laboratory and clinic. However, so far, such models have been impeded by both experimental and computational limitations. In this perspective, we argue that they can now be achieved using deep learning algorithms to integrate omics data and prior knowledge of molecular networks. Such models would have many applications in precision oncology, e.g., for identifying drug targets and biomarkers, predicting resistance mechanisms and toxicity effects of drugs, or simulating cell-cell interactions in the microenvironment.

Education and Training Assessment and Artificial Intelligence. A Pragmatic Guide for Educators

The emergence of ChatGPT and similar new Generative AI tools has created concern about the validity of many current assessment methods in higher education, since learners might use these tools to complete those assessments. Here we review the current evidence on this issue and show that for assessments like essays and multiple-choice exams, these concerns are legitimate: ChatGPT can complete them to a very high standard, quickly and cheaply. We consider how to assess learning in alternative ways, and the importance of retaining assessments of foundational core knowledge. This evidence is considered from the perspective of current professional regulations covering the professional registration of Biomedical Scientists and their Health and Care Professions Council (HCPC) approved education providers, although it should be broadly relevant across higher education.

Artificial intelligence for diagnosis and predictive biomarkers in Non-Small cell lung cancer Patients: New promises but also new hurdles for the pathologist

The rapid development of artificial intelligence (AI) based tools in pathology laboratories has brought forward unlimited opportunities for pathologists. Promising AI applications used for accomplishing diagnostic, prognostic and predictive tasks are being developed at a high pace. This is notably true in thoracic oncology, given the significant and rapid therapeutic progress made recently for lung cancer patients. Advances have been based on drugs targeting molecular alterations, immunotherapies, and, more recently antibody-drug conjugates which are soon to be introduced. For over a decade, many proof-of-concept studies have explored the use of AI algorithms in thoracic oncology to improve lung cancer patient care. However, despite the enthusiasm in this domain, the set-up and use of AI algorithms in daily practice of thoracic pathologists has not been operative until now, due to several constraints. The purpose of this review is to describe the potential but also the current barriers of AI applications in routine thoracic pathology for non-small cell lung cancer patient care and to suggest practical solutions for rapid future implementation.

Decoding pan-cancer treatment outcomes using multimodal real-world data and explainable artificial intelligence

Despite advances in precision oncology, clinical decision-making still relies on limited variables and expert knowledge. To address this limitation, we combined multimodal real-world data and explainable artificial intelligence (xAI) to introduce AI-derived (AID) markers for clinical decision support. We used xAI to decode the outcome of 15,726 patients across 38 solid cancer entities based on 350 markers, including clinical records, image-derived body compositions, and mutational tumor profiles. xAI determined the prognostic contribution of each clinical marker at the patient level and identified 114 key markers that accounted for 90% of the neural network's decision process. Moreover, xAI enabled us to uncover 1,373 prognostic interactions between markers. Our approach was validated in an independent cohort of 3,288 patients with lung cancer from a US nationwide electronic health record-derived database. These results show the potential of xAI to transform the assessment of clinical variables and enable personalized, data-driven cancer care.

Convergence of evolving artificial intelligence and machine learning techniques in precision oncology

The confluence of new technologies with artificial intelligence (AI) and machine learning (ML) analytical techniques is rapidly advancing the field of precision oncology, promising to improve diagnostic approaches and therapeutic strategies for patients with cancer. By analyzing multi-dimensional, multiomic, spatial pathology, and radiomic data, these technologies enable a deeper understanding of the intricate molecular pathways, aiding in the identification of critical nodes within the tumor's biology to optimize treatment selection. The applications of AI/ML in precision oncology are extensive and include the generation of synthetic data, e.g., digital twins, in order to provide the necessary information to design or expedite the conduct of clinical trials. Currently, many operational and technical challenges exist related to data technology, engineering, and storage; algorithm development and structures; quality and quantity of the data and the analytical pipeline; data sharing and generalizability; and the incorporation of these technologies into the current clinical workflow and reimbursement models.

Analyzing Atomic Interactions in Molecules as Learned by Neural Networks

While machine learning (ML) models have been able to achieve unprecedented accuracies across various prediction tasks in quantum chemistry, it is now apparent that accuracy on a test set alone is not a guarantee for robust chemical modeling such as stable molecular dynamics (MD). To go beyond accuracy, we use explainable artificial intelligence (XAI) techniques to develop a general analysis framework for atomic interactions and apply it to the SchNet and PaiNN neural network models. We compare these interactions with a set of fundamental chemical principles to understand how well the models have learned the underlying physicochemical concepts from the data. We focus on the strength of the interactions for different atomic species, how predictions for intensive and extensive quantum molecular properties are made, and analyze the decay and many-body nature of the interactions with interatomic distance. Models that deviate too far from known physical principles produce unstable MD trajectories, even when they have very high energy and force prediction accuracy. We also suggest further improvements to the ML architectures to better account for the polynomial decay of atomic interactions.

Challenges and Opportunities in the Clinical Translation of High-Resolution Spatial Transcriptomics

Pathology has always been fueled by technological advances. Histology powered the study of tissue architecture at single-cell resolution and remains a cornerstone of clinical pathology today. In the last decade, next-generation sequencing has become informative for the targeted treatment of many diseases, demonstrating the importance of genome-scale molecular information for personalized medicine. Today, revolutionary developments in spatial transcriptomics technologies digitalize gene expression at subcellular resolution in intact tissue sections, enabling the computational analysis of cell types, cellular phenotypes, and cell-cell communication in routinely collected and archival clinical samples. Here we review how such molecular microscopes work, highlight their potential to identify disease mechanisms and guide personalized therapies, and provide guidance for clinical study design. Finally, we discuss remaining challenges to the swift translation of high-resolution spatial transcriptomics technologies and how integration of multimodal readouts and deep learning approaches is bringing us closer to a holistic understanding of tissue biology and pathology.

RSDCNet: An efficient and lightweight deep learning model for benign and malignant pathology detection in breast cancer

Background

Breast cancer is a leading malignant tumor among women globally, with its pathological classification into benign or malignant directly influencing treatment strategies and prognosis. Traditional diagnostic methods, reliant on manual interpretation, are not only time-intensive and subjective but also susceptible to variability based on the pathologist's expertise and workload. Consequently, the development of an efficient, automated, and precise pathological detection method is crucial.

Methods

This study introduces RSDCNet, an enhanced lightweight neural network architecture designed for the automatic detection of benign and malignant breast cancer pathology. Utilizing the BreakHis dataset, which comprises 9109 microscopic images of breast tumors including various differentiation levels of benign and malignant samples, RSDCNet integrates depthwise separable convolution and SCSE modules. This integration aims to reduce model parameters while enhancing key feature extraction capabilities, thereby achieving both lightweight design and high efficiency.

Results

RSDCNet demonstrated superior performance across multiple evaluation metrics in the classification task. The model achieved an accuracy of 0.9903, a recall of 0.9897, an F1 score of 0.9888, and a precision of 0.9879, outperforming established deep learning models such as EfficientNet, RegNet, HRNet, and ViT. Notably, RSDCNet's parameter count stood at just 1,199,662, significantly lower than HRNet's 19,254,102 and ViT's 85,800,194, highlighting its enhanced resource efficiency.

Conclusion

The RSDCNet model presented in this study excels in the efficient and accurate classification of benign and malignant breast cancer pathology. Compared to traditional methods and other leading models, RSDCNet not only reduces computational resource consumption but also offers improved feature extraction and clinical interpretability. This advancement provides substantial technical support for the intelligent diagnosis of breast cancer, paving the way for more effective treatment planning and prognosis assessment.

Artificial intelligence in surgical pathology - Where do we stand, where do we go?

Surgical and neuropathologists continuously search for new and disease-specific features, such as independent predictors of tumor prognosis or determinants of tumor entities and sub-entities. This is a task where artificial intelligence (AI)/machine learning (ML) systems could significantly contribute to help with tumor outcome prediction and the search for new diagnostic or treatment stratification biomarkers. AI systems are increasingly integrated into routine pathology workflows to improve accuracy, reproducibility, productivity and to reveal difficult-to-see features in complicated histological slides, including the quantification of important markers for tumor grading and staging. In this article, we review the infrastructure needed to facilitate digital and computational pathology. We address the barriers for its full deployment in the clinical setting and describe the use of AI in intraoperative or postoperative settings were frozen or formalin-fixed, paraffin-embedded materials are used. We also summarize quality assessment issues of slide digitization, new spatial biology approaches, and the determination of specific gene-expression from whole slide images. Finally, we highlight new innovative and future technologies, such as large language models, optical biopsies, and mass spectrometry imaging.

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

Explainable AI aims to overcome the black-box nature of complex ML models like neural networks by generating explanations for their predictions. Explanations often take the form of a heatmap identifying input features (e.g. pixels) that are relevant to the model's decision. These explanations, however, entangle the potentially multiple factors that enter into the overall complex decision strategy. We propose to disentangle explanations by extracting at some intermediate layer of a neural network, subspaces that capture the multiple and distinct activation patterns (e.g. visual concepts) that are relevant to the prediction. To automatically extract these subspaces, we propose two new analyses, extending principles found in PCA or ICA to explanations. These novel analyses, which we call principal relevant component analysis (PRCA) and disentangled relevant subspace analysis (DRSA), maximize relevance instead of e.g. variance or kurtosis. This allows for a much stronger focus of the analysis on what the ML model actually uses for predicting, ignoring activations or concepts to which the model is invariant. Our approach is general enough to work alongside common attribution techniques such as Shapley Value, Integrated Gradients, or LRP. Our proposed methods show to be practically useful and compare favorably to the state of the art as demonstrated on benchmarks and three use cases.

Artificial Intelligence in Uropathology

The global population is currently at unprecedented levels, with an estimated 7.8 billion people inhabiting the planet. We are witnessing a rise in cancer cases, attributed to improved control of cardiovascular diseases and a growing elderly population. While this has resulted in an increased workload for pathologists, it also presents an opportunity for advancement. The accurate classification of tumors and identification of prognostic and predictive factors demand specialized expertise and attention. Fortunately, the rapid progression of artificial intelligence (AI) offers new prospects in medicine, particularly in diagnostics such as image and surgical pathology. This article explores the transformative impact of AI in the field of uropathology, with a particular focus on its application in diagnosing, grading, and prognosticating various urological cancers. AI, especially deep learning algorithms, has shown significant potential in improving the accuracy and efficiency of pathology workflows. This comprehensive review is dedicated to providing an insightful overview of the primary data concerning the utilization of AI in diagnosing, predicting prognosis, and determining drug responses for tumors of the urinary tract. By embracing these advancements, we can look forward to improved outcomes and better patient care.

Applying interpretable machine learning in computational biology-pitfalls, recommendations and opportunities for new developments

Recent advances in machine learning have enabled the development of next-generation predictive models for complex computational biology problems, thereby spurring the use of interpretable machine learning (IML) to unveil biological insights. However, guidelines for using IML in computational biology are generally underdeveloped. We provide an overview of IML methods and evaluation techniques and discuss common pitfalls encountered when applying IML methods to computational biology problems. We also highlight open questions, especially in the era of large language models, and call for collaboration between IML and computational biology researchers.

AI-Assisted Rational Design and Activity Prediction of Biological Elements for Optimizing Transcription-Factor-Based Biosensors

The rational design, activity prediction, and adaptive application of biological elements (bio-elements) are crucial research fields in synthetic biology. Currently, a major challenge in the field is efficiently designing desired bio-elements and accurately predicting their activity using vast datasets. The advancement of artificial intelligence (AI) technology has enabled machine learning and deep learning algorithms to excel in uncovering patterns in bio-element data and predicting their performance. This review explores the application of AI algorithms in the rational design of bio-elements, activity prediction, and the regulation of transcription-factor-based biosensor response performance using AI-designed elements. We discuss the advantages, adaptability, and biological challenges addressed by the AI algorithms in various applications, highlighting their powerful potential in analyzing biological data. Furthermore, we propose innovative solutions to the challenges faced by AI algorithms in the field and suggest future research directions. By consolidating current research and demonstrating the practical applications and future potential of AI in synthetic biology, this review provides valuable insights for advancing both academic research and practical applications in biotechnology.

The recent advances in the approach of artificial intelligence (AI) towards drug discovery

Artificial intelligence (AI) has recently emerged as a unique developmental influence that is playing an important role in the development of medicine. The AI medium is showing the potential in unprecedented advancements in truth and efficiency. The intersection of AI has the potential to revolutionize drug discovery. However, AI also has limitations and experts should be aware of these data access and ethical issues. The use of AI techniques for drug discovery applications has increased considerably over the past few years, including combinatorial QSAR and QSPR, virtual screening, and denovo drug design. The purpose of this survey is to give a general overview of drug discovery based on artificial intelligence, and associated applications. We also highlighted the gaps present in the traditional method for drug designing. In addition, potential strategies and approaches to overcome current challenges are discussed to address the constraints of AI within this field. We hope that this survey plays a comprehensive role in understanding the potential of AI in drug discovery.

Application of 3D, 4D, 5D, and 6D bioprinting in cancer research: what does the future look like?

The application of three- and four-dimensional (3D/4D) printing in cancer research represents a significant advancement in understanding and addressing the complexities of cancer biology. 3D/4D materials provide more physiologically relevant environments compared to traditional two-dimensional models, allowing for a more accurate representation of the tumor microenvironment that enables researchers to study tumor progression, drug responses, and interactions with surrounding tissues under conditions similar to in vivo conditions. The dynamic nature of 4D materials introduces the element of time, allowing for the observation of temporal changes in cancer behavior and response to therapeutic interventions. The use of 3D/4D printing in cancer research holds great promise for advancing our understanding of the disease and improving the translation of preclinical findings to clinical applications. Accordingly, this review aims to briefly discuss 3D and 4D printing and their advantages and limitations in the field of cancer. Moreover, new techniques such as 5D/6D printing and artificial intelligence (AI) are also introduced as methods that could be used to overcome the limitations of 3D/4D printing and opened promising ways for the fast and precise diagnosis and treatment of cancer.

Effects of explainable artificial intelligence in neurology decision support

OBJECTIVE

Artificial intelligence (AI)-based decision support systems (DSS) are utilized in medicine but underlying decision-making processes are usually unknown. Explainable AI (xAI) techniques provide insight into DSS, but little is known on how to design xAI for clinicians. Here we investigate the impact of various xAI techniques on a clinician's interaction with an AI-based DSS in decision-making tasks as compared to a general population.

METHODS

We conducted a randomized, blinded study in which members of the Child Neurology Society and American Academy of Neurology were compared to a general population. Participants received recommendations from a DSS via a random assignment of an xAI intervention (decision tree, crowd sourced agreement, case-based reasoning, probability scores, counterfactual reasoning, feature importance, templated language, and no explanations). Primary outcomes included test performance and perceived explainability, trust, and social competence of the DSS. Secondary outcomes included compliance, understandability, and agreement per question.

RESULTS

We had 81 neurology participants with 284 in the general population. Decision trees were perceived as the more explainable by the medical versus general population (P < 0.01) and as more explainable than probability scores within the medical population (P < 0.001). Increasing neurology experience and perceived explainability degraded performance (P = 0.0214). Performance was not predicted by xAI method but by perceived explainability.

INTERPRETATION

xAI methods have different impacts on a medical versus general population; thus, xAI is not uniformly beneficial, and there is no one-size-fits-all approach. Further user-centered xAI research targeting clinicians and to develop personalized DSS for clinicians is needed.

Advancing Artificial Intelligence Integration Into the Pathology Workflow: Exploring Opportunities in Gastrointestinal Tract Biopsies

This review aims to present a comprehensive overview of the current landscape of artificial intelligence (AI) applications in the analysis of tubular gastrointestinal biopsies. These publications cover a spectrum of conditions, ranging from inflammatory ailments to malignancies. Moving beyond the conventional diagnosis based on hematoxylin and eosin-stained whole-slide images, the review explores additional implications of AI, including its involvement in interpreting immunohistochemical results, molecular subtyping, and the identification of cellular spatial biomarkers. Furthermore, the review examines how AI can contribute to enhancing the quality and control of diagnostic processes, introducing new workflow options, and addressing the limitations and caveats associated with current AI platforms in this context.

Diffeomorphic Counterfactuals With Generative Models

Counterfactuals can explain classification decisions of neural networks in a human interpretable way. We propose a simple but effective method to generate such counterfactuals. More specifically, we perform a suitable diffeomorphic coordinate transformation and then perform gradient ascent in these coordinates to find counterfactuals which are classified with great confidence as a specified target class. We propose two methods to leverage generative models to construct such suitable coordinate systems that are either exactly or approximately diffeomorphic. We analyze the generation process theoretically using Riemannian differential geometry and validate the quality of the generated counterfactuals using various qualitative and quantitative measures.

[Explainable artificial intelligence in pathology]

With the advancements in precision medicine, the demands on pathological diagnostics have increased, requiring standardized, quantitative, and integrated assessments of histomorphological and molecular pathological data. Great hopes are placed in artificial intelligence (AI) methods, which have demonstrated the ability to analyze complex clinical, histological, and molecular data for disease classification, biomarker quantification, and prognosis estimation. This paper provides an overview of the latest developments in pathology AI, discusses the limitations, particularly concerning the black box character of AI, and describes solutions to make decision processes more transparent using methods of so-called explainable AI (XAI).