Translational AI and Deep Learning in Diagnostic Pathology

Mitosis detection and classification for breast cancer diagnosis: What we know and what is next

Breast cancer is the second most deadly malignancy in women, behind lung cancer. Despite significant improvements in medical research, breast cancer is still accurately diagnosed with histological analysis. During this procedure, pathologists examine a physical sample for the presence of mitotic cells, or dividing cells. However, the high resolution of histopathology images and the difficulty of manually detecting tiny mitotic nuclei make it particularly challenging to differentiate mitotic cells from other types of cells. Numerous studies have addressed the detection and classification of mitosis, owing to increasing capacity and developments in automated approaches. The combination of machine learning and deep learning techniques has greatly revolutionized the process of identifying mitotic cells by offering automated, precise, and efficient solutions. In the last ten years, several pioneering methods have been presented, advancing towards practical applications in clinical settings. Unlike other forms of cancer, breast cancer and gliomas are categorized according to the number of mitotic divisions. Numerous papers have been published on techniques for identifying mitosis due to easy access to datasets and open competitions. Convolutional neural networks and other deep learning architectures can precisely identify mitotic cells, significantly decreasing the amount of labor that pathologists must perform. This article examines the techniques used over the past decade to identify and classify mitotic cells in histologically stained breast cancer hematoxylin and eosin images. Furthermore, we examine the benefits of current research techniques and predict forthcoming developments in the investigation of breast cancer mitosis, specifically highlighting machine learning and deep learning.

An Overview of Artificial Intelligence in Gynaecological Pathology Diagnostics

Background: The advent of artificial intelligence (AI) has revolutionised many fields in healthcare. More recently, it has garnered interest in terms of its potential applications in histopathology, where algorithms are increasingly being explored as adjunct technologies that can support pathologists in diagnosis, molecular typing and prognostication. While many research endeavours have focused on solid tumours, gynaecological malignancies have nevertheless been relatively overlooked. The aim of this review was therefore to provide a summary of the status quo in the field of AI in gynaecological pathology by encompassing malignancies throughout the entirety of the female reproductive tract rather than focusing on individual cancers. Methods: This narrative/scoping review explores the potential application of AI in whole slide image analysis in gynaecological histopathology, drawing on both findings from the research setting (where such technologies largely remain confined), and highlights any findings and/or applications identified and developed in other cancers that could be translated to this arena. Results: A particular focus is given to ovarian, endometrial, cervical and vulval/vaginal tumours. This review discusses different algorithms, their performance and potential applications. Conclusions: The effective application of AI tools is only possible through multidisciplinary co-operation and training.

An interpretable framework for gastric cancer classification using multi-channel attention mechanisms and transfer learning approach on histopathology images

The importance of gastric cancer (GC) and the role of deep learning techniques in categorizing GC histopathology images have recently increased. Identifying the drawbacks of traditional deep learning models, including lack of interpretability, inability to capture complex patterns, lack of adaptability, and sensitivity to noise. A multi-channel attention mechanism-based framework is proposed that can overcome the limitations of conventional deep learning models by dynamically focusing on relevant features, enhancing extraction, and capturing complex relationships in medical data. The proposed framework uses three different attention mechanism channels and convolutional neural networks to extract multichannel features during the classification process. The proposed framework's strong performance is confirmed by competitive experiments conducted on a publicly available Gastric Histopathology Sub-size Image Database, which yielded remarkable classification accuracies of 99.07% and 98.48% on the validation and testing sets, respectively. Additionally, on the HCRF dataset, the framework achieved high classification accuracy of 99.84% and 99.65% on the validation and testing sets, respectively. The effectiveness and interchangeability of the three channels are further confirmed by ablation and interchangeability experiments, highlighting the remarkable performance of the framework in GC histopathological image classification tasks. This offers an advanced and pragmatic artificial intelligence solution that addresses challenges posed by unique medical image characteristics for intricate image analysis. The proposed approach in artificial intelligence medical engineering demonstrates significant potential for enhancing diagnostic precision by achieving high classification accuracy and treatment outcomes.

AI in medical diagnosis: A contextualised study of patient motivations and concerns

Patients' reactions to the implementation of Artificial Intelligence (AI) in healthcare range from adverse to favourable. While AI holds the promise of revolutionising healthcare by enhancing, accelerating, and improving the precision of care services, our understanding of patients' reactions to these paradigm shifts remains limited. In particular, little is known about the extent to which patients are receptive to independently use AI-enabled applications for diagnosis. This research seeks to develop a holistic, context-specific model capturing both the negative and positive cognitive responses of patients utilising AI-powered diagnostic services. Employing a sequential mixed-methods approach, the study draws on Behavioural Reasoning Theory to decode patients' cognitive reactions, including their reasons for and reasons giants using such applications. The research begins with a qualitative exploration, analysing user reviews to identify context-specific barriers and motivators. Building on these qualitative insights, the model's empirical validity is tested through a quantitative phase involving survey data analysis. Our findings provide a nuanced understanding of the context-dependent factors shaping patients' cognitive responses to AI-enabled diagnostic services, offering valuable insights for the design and implementation of patient-centred AI solutions in healthcare.

Bridging the Clinical-Computational Transparency Gap in Digital Pathology

CONTEXT.—

Computational pathology combines clinical pathology with computational analysis, aiming to enhance diagnostic capabilities and improve clinical productivity. However, communication barriers between pathologists and developers often hinder the full realization of this potential.

OBJECTIVE.—

To propose a standardized framework that improves mutual understanding of clinical objectives and computational methodologies. The goal is to enhance the development and application of computer-aided diagnostic (CAD) tools.

DESIGN.—

This article suggests pivotal roles for pathologists and computer scientists in the CAD development process. It calls for increased understanding of computational terminologies, processes, and limitations among pathologists. Similarly, it argues that computer scientists should better comprehend the true use cases of the developed algorithms to avoid clinically meaningless metrics.

RESULTS.—

CAD tools improve pathology practice significantly. Some tools have even received US Food and Drug Administration approval. However, improved understanding of machine learning models among pathologists is essential to prevent misuse and misinterpretation. There is also a need for a more accurate representation of the algorithms' performance compared to that of pathologists.

CONCLUSIONS.—

A comprehensive understanding of computational and clinical paradigms is crucial for overcoming the translational gap in computational pathology. This mutual comprehension will improve patient care through more accurate and efficient disease diagnosis.

Pathology Foundation Models

Pathology plays a crucial role in diagnosing and evaluating patient tissue samples obtained via surgeries and biopsies. The advent of whole slide scanners and the development of deep learning technologies have considerably advanced this field, promoting extensive research and development in pathology artificial intelligence (AI). These advancements have contributed to reduced workload of pathologists and supported decision-making in treatment plans. Large-scale AI models, known as foundation models (FMs), are more accurate and applicable to various tasks than traditional AI. Such models have recently emerged and expanded their application scope in healthcare. Numerous FMs have been developed in pathology, with reported applications in various tasks, such as disease and rare cancer diagnoses, patient survival prognosis prediction, biomarker expression prediction, and scoring of the immunohistochemical expression intensity. However, several challenges persist in the clinical application of FMs, which healthcare professionals, as users, must be aware of. Research to address these challenges is ongoing. In the future, the development of generalist medical AI, which integrates pathology FMs with FMs from other medical domains, is expected to progress, effectively utilizing AI in real clinical settings to promote precision and personalized medicine.

Deep learning methods for improving the accuracy and efficiency of pathological image analysis

This study presents a novel integration of two advanced deep learning models, U-Net and EfficientNetV2, to achieve high-precision segmentation and rapid classification of pathological images. A key innovation is the development of a new heatmap generation algorithm, which leverages meticulous image preprocessing, data enhancement strategies, ensemble learning, attention mechanisms, and deep feature fusion techniques. This algorithm not only produces highly accurate and interpretatively rich heatmaps but also significantly improves the accuracy and efficiency of pathological image analysis. Unlike existing methods, our approach integrates these advanced techniques into a cohesive framework, enhancing its ability to reveal critical features in pathological images. Rigorous experimental validation demonstrated that our algorithm excels in key performance indicators such as accuracy, recall rate, and processing speed, underscoring its potential for broader applications in pathological image analysis and beyond.

Spatial heterogeneity of infiltrating immune cells in the tumor microenvironment of non-small cell lung cancer

Tumor-infiltrating lymphocytes (TILs) are essential components of the tumor microenvironment (TME) of non-small cell lung cancer (NSCLC). Still, it is difficult to describe due to their heterogeneity. In this study, five cell markers from NSCLC patients were analyzed. We segmented tumor cells (TCs) and TILs using Efficientnet-B3 and explored their quantitative information and spatial distribution. After that, we simulated multiplex immunohistochemistry (mIHC) by overlapping continuous single chromogenic IHCs slices. As a result, the proportion and the density of programmed cell death-ligand 1 (PD-L1)-positive TCs were the highest in the core. CD8+ T cells were the closest to the tumor (median distance: 41.71 μm), while PD-1+T cells were the most distant (median distance: 62.2μm), and our study found that most lymphocytes clustered together within the peritumoral range of 10-30 μm where cross-talk with TCs could be achieved. We also found that the classification of TME could be achieved using CD8+ T-cell density, which is correlated with the prognosis of patients. In addition, we achieved single chromogenic IHC slices overlap based on CD4-stained IHC slices. We explored the number and spatial distribution of cells in heterogeneous TME of NSCLC patients and achieved TME classification. We also found a way to show the co-expression of multiple molecules economically.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Present and Future Applications of Artificial Intelligence in Kidney Transplantation

Artificial intelligence (AI) has a wide and increasing range of applications across various sectors. In medicine, AI has already made an impact in numerous fields, rapidly transforming healthcare delivery through its growing applications in diagnosis, treatment and overall patient care. Equally, AI is swiftly and essentially transforming the landscape of kidney transplantation (KT), offering innovative solutions for longstanding problems that have eluded resolution through traditional approaches outside its spectrum. The purpose of this review is to explore the present and future applications of artificial intelligence in KT, with a focus on pre-transplant evaluation, surgical assistance, outcomes and post-transplant care. We discuss its great potential and the inevitable limitations that accompany these technologies. We conclude that by fostering collaboration between AI technologies and medical practitioners, we can pave the way for a future where advanced, personalised care becomes the standard in KT and beyond.

Computational modeling of tumor invasion from limited and diverse data in Glioblastoma

For diseases with high morbidity rates such as Glioblastoma Multiforme, the prognostic and treatment planning pipeline requires a comprehensive analysis of imaging, clinical, and molecular data. Many mutations have been shown to correlate strongly with the median survival rate and response to therapy of patients. Studies have demonstrated that these mutations manifest as specific visual biomarkers in tumor imaging modalities such as MRI. To minimize the number of invasive procedures on a patient and for the overall resource optimization for the prognostic and treatment planning process, the correlation of imaging and molecular features has garnered much interest. While the tumor mass is the most significant feature, the impacted tissue surrounding the tumor is also a significant biomarker contributing to the visual manifestation of mutations - which has not been studied as extensively. The pattern of tumor growth impacts the surrounding tissue accordingly, which is a reflection of tumor properties as well. Modeling how the tumor growth impacts the surrounding tissue can reveal important information about the patterns of tumor enhancement, which in turn has significant diagnostic and prognostic value. This paper presents the first work to automate the computational modeling of the impacted tissue surrounding the tumor using generative deep learning. The paper isolates and quantifies the impact of the Tumor Invasion (TI) on surrounding tissue based on change in mutation status, subsequently assessing its prognostic value. Furthermore, a TI Generative Adversarial Network (TI-GAN) is proposed to model the tumor invasion properties. Extensive qualitative and quantitative analyses, cross-dataset testing, and radiologist blind tests are carried out to demonstrate that TI-GAN can realistically model the tumor invasion under practical challenges of medical datasets such as limited data and high intra-class heterogeneity.

AI empowering traditional Chinese medicine?

For centuries, Traditional Chinese Medicine (TCM) has been a prominent treatment method in China, incorporating acupuncture, herbal remedies, massage, and dietary therapy to promote holistic health and healing. TCM has played a major role in drug discovery, with over 60% of small-molecule drugs approved by the FDA from 1981 to 2019 being derived from natural products. However, TCM modernization faces challenges such as data standardization and the complexity of TCM formulations. The establishment of comprehensive TCM databases has significantly improved the efficiency and accuracy of TCM research, enabling easier access to information on TCM ingredients and encouraging interdisciplinary collaborations. These databases have revolutionized TCM research, facilitating advancements in TCM modernization and patient care. In addition, advancements in AI algorithms and database data quality have accelerated progress in AI for TCM. The application of AI in TCM encompasses a wide range of areas, including herbal screening and new drug discovery, diagnostic and treatment principles, pharmacological mechanisms, network pharmacology, and the incorporation of innovative AI technologies. AI also has the potential to enable personalized medicine by identifying patterns and correlations in patient data, leading to more accurate diagnoses and tailored treatments. The potential benefits of AI for TCM are vast and diverse, promising continued progress and innovation in the field.

Medical Students' Perception Toward Using AI in Medical Education in the Kurdistan Region, Iraq: A Cross-Sectional Study

Background and aim AI is revolutionizing medical education by offering innovative tools and simulations that augment traditional teaching methods. This study explored the perceptions and expectations of medical students in the Kurdistan region, Iraq, regarding AI integration in medical education. Methods A cross-sectional online survey collected data from 224 medical students over four months. A descriptive analysis was conducted to present the student's attitudes.  Results In total, 224 medical students responded to the online survey. The majority of them were female (n=129; 57.6%), while 95 were male (42.4%). Additionally, most of the participants were in stage 4 (54 (24.1%); stage 1, 48 (21.4%); and stage 2, 43 (19.2%). In terms of measuring students' perceptions of AI integration in medical education, 186 (83%) of the students wanted to use smartphones and tablets, and 38 (17%) of them reported wanting hard copies. In addition, 112 (50%) of the medical students considered themselves experts in using AI and 98 (43.8%) did not know exactly what AI was used; however, only a few of them (6.3%) did not use AI. Few patients reported using Manikins instead of real patients (42 (18.8%)), while 140 (62.5%) reported that they could be used but not an alternative. Conclusion While many agree that digital tools and simulations are useful teaching tools, they are frequently viewed as adjunctive approaches. Better integration and training are required for the infrequent use of AI tools in medical education.

Clinical translation of nanomedicine with integrated digital medicine and machine learning interventions

Nanomaterials based therapeutics transform the ways of disease prevention, diagnosis and treatment with increasing sophistications in nanotechnology at a breakneck pace, but very few could reach to the clinic due to inconsistencies in preclinical studies followed by regulatory hinderances. To tackle this, integrating the nanomedicine discovery with digital medicine provide technologies as tools of specific biological activity measurement. Hence, overcome the redundancies in nanomedicine discovery by the on-site data acquisition and analytics through integrating intelligent sensors and artificial intelligence (AI) or machine learning (ML). Integrated AI/ML wearable sensors directly gather clinically relevant biochemical information from the subject's body and process data for physicians to make right clinical decision(s) in a time and cost-effective way. This review summarizes insights and recommend the infusion of actionable big data computation enabled sensors in burgeoning field of nanomedicine at academia, research institutes, and pharmaceutical industries, with a potential of clinical translation. Furthermore, many blind spots are present in modern clinically relevant computation, one of which could prevent ML-guided low-cost new nanomedicine development from being successfully translated into the clinic was also discussed.

Advanced image generation for cancer using diffusion models

Deep neural networks have significantly advanced the field of medical image analysis, yet their full potential is often limited by relatively small dataset sizes. Generative modeling, particularly through diffusion models, has unlocked remarkable capabilities in synthesizing photorealistic images, thereby broadening the scope of their application in medical imaging. This study specifically investigates the use of diffusion models to generate high-quality brain MRI scans, including those depicting low-grade gliomas, as well as contrast-enhanced spectral mammography (CESM) and chest and lung X-ray images. By leveraging the DreamBooth platform, we have successfully trained stable diffusion models utilizing text prompts alongside class and instance images to generate diverse medical images. This approach not only preserves patient anonymity but also substantially mitigates the risk of patient re-identification during data exchange for research purposes. To evaluate the quality of our synthesized images, we used the Fréchet inception distance metric, demonstrating high fidelity between the synthesized and real images. Our application of diffusion models effectively captures oncology-specific attributes across different imaging modalities, establishing a robust framework that integrates artificial intelligence in the generation of oncological medical imagery.

Expanding role and scope of artificial intelligence in the field of gastrointestinal pathology

Digital pathology (DP) and its subsidiaries including artificial intelligence (AI) are rapidly making inroads into the area of diagnostic anatomic pathology (AP) including gastrointestinal (GI) pathology. It is poised to revolutionize the field of diagnostic AP. Historically, AP has been slow to adopt digital technology, but this is changing rapidly, with many centers worldwide transitioning to DP. Coupled with advanced techniques of AI such as deep learning and machine learning, DP is likely to transform histopathology from a subjective field to an objective, efficient, and transparent discipline. AI is increasingly integrated into GI pathology, offering numerous advancements and improvements in overall diagnostic accuracy, efficiency, and patient care. Specifically, AI in GI pathology enhances diagnostic accuracy, streamlines workflows, provides predictive insights, integrates multimodal data, supports research, and aids in education and training, ultimately improving patient care and outcomes. This review summarized the latest developments in the role and scope of AI in AP with a focus on GI pathology. The main aim was to provide updates and create awareness among the pathology community.

DLCNBC-SA: a model for assessing axillary lymph node metastasis status in early breast cancer patients

Background

Axillary lymph node (ALN) status is a crucial prognostic indicator for breast cancer metastasis, with manual interpretation of whole slide images (WSIs) being the current standard practice. However, this method is subjective and time-consuming. Recent advancements in deep learning-based methods for medical image analysis have shown promise in improving clinical diagnosis. This study aims to leverage these technological advancements to develop a deep learning model based on features extracted from primary tumor biopsies for preoperatively identifying ALN metastasis in early-stage breast cancer patients with negative nodes.

Methods

We present DLCNBC-SA, a deep learning-based network specifically tailored for core needle biopsy and clinical data feature extraction, which integrates a self-attention mechanism (CNBC-SA). The proposed model consists of a feature extractor based on convolutional neural network (CNN) and an improved self-attention mechanism module, which can preserve the independence of features in WSIs for analysis and enhancement to provide rich feature representation. To validate the performance of the proposed model, we conducted comparative experiments and ablation studies using publicly available datasets, and verification was performed through quantitative analysis.

Results

The comparative experiment illustrates the superior performance of the proposed model in the task of binary classification of ALNs, as compared to alternative methods. Our method achieved outstanding performance [area under the curve (AUC): 0.882] in this task, significantly surpassing the state-of-the-art (SOTA) method on the same dataset (AUC: 0.862). The ablation experiment reveals that incorporating RandomRotation data augmentation technology and utilizing Adadelta optimizer can effectively enhance the performance of the proposed model.

Conclusions

The experimental results demonstrate that the model proposed in this paper outperforms the SOTA model on the same dataset, thereby establishing its reliability as an assistant for pathologists in analyzing WSIs of breast cancer. Consequently, it significantly enhances both the efficiency and accuracy of doctors during the diagnostic process.

Deep Learning-based Modeling for Preclinical Drug Safety Assessment

In drug development, assessing the toxicity of candidate compounds is crucial for successfully transitioning from preclinical research to early-stage clinical trials. Drug safety is typically assessed using animal models with a manual histopathological examination of tissue sections to characterize the dose-response relationship of the compound - a time-intensive process prone to inter-observer variability and predominantly involving tedious review of cases without abnormalities. Artificial intelligence (AI) methods in pathology hold promise to accelerate this assessment and enhance reproducibility and objectivity. Here, we introduce TRACE, a model designed for toxicologic liver histopathology assessment capable of tackling a range of diagnostic tasks across multiple scales, including situations where labeled data is limited. TRACE was trained on 15 million histopathology images extracted from 46,734 digitized tissue sections from 157 preclinical studies conducted on Rattus norvegicus. We show that TRACE can perform various downstream toxicology tasks spanning histopathological response assessment, lesion severity scoring, morphological retrieval, and automatic dose-response characterization. In an independent reader study, TRACE was evaluated alongside ten board-certified veterinary pathologists and achieved higher concordance with the consensus opinion than the average of the pathologists. Our study represents a substantial leap over existing computational models in toxicology by offering the first framework for accelerating and automating toxicological pathology assessment, promoting significant progress with faster, more consistent, and reliable diagnostic processes.

Comparative analysis of feature-based ML and CNN for binucleated erythroblast quantification in myelodysplastic syndrome patients using imaging flow cytometry data

Myelodysplastic syndrome is primarily characterized by dysplasia in the bone marrow (BM), presenting a challenge in consistent morphology interpretation. Accurate diagnosis through traditional slide-based analysis is difficult, necessitating a standardized objective technique. Over the past two decades, imaging flow cytometry (IFC) has proven effective in combining image-based morphometric analyses with high-parameter phenotyping. We have previously demonstrated the effectiveness of combining IFC with a feature-based machine learning algorithm to accurately identify and quantify rare binucleated erythroblasts (BNEs) in dyserythropoietic BM cells. However, a feature-based workflow poses challenges requiring software-specific expertise. Here we employ a Convolutional Neural Network (CNN) algorithm for BNE identification and differentiation from doublets and cells with irregular nuclear morphology in IFC data. We demonstrate that this simplified AI workflow, coupled with a powerful CNN algorithm, achieves comparable BNE quantification accuracy to manual and feature-based analysis with substantial time savings, eliminating workflow complexity. This streamlined approach holds significant clinical value, enhancing IFC accessibility for routine diagnostic purposes.

Multi-omics and artificial intelligence predict clinical outcomes of immunotherapy in non-small cell lung cancer patients

In recent years, various types of immunotherapy, particularly the use of immune checkpoint inhibitors targeting programmed cell death 1 or programmed death ligand 1 (PD-L1), have revolutionized the management and prognosis of non-small cell lung cancer. PD-L1 is frequently used as a biomarker for predicting the likely benefit of immunotherapy for patients. However, some patients receiving immunotherapy have high response rates despite having low levels of PD-L1. Therefore, the identification of this group of patients is extremely important to improve prognosis. The tumor microenvironment contains tumor, stromal, and infiltrating immune cells with its composition differing significantly within tumors, between tumors, and between individuals. The omics approach aims to provide a comprehensive assessment of each patient through high-throughput extracted features, promising a more comprehensive characterization of this complex ecosystem. However, features identified by high-throughput methods are complex and present analytical challenges to clinicians and data scientists. It is thus feasible that artificial intelligence could assist in the identification of features that are beyond human discernment as well as in the performance of repetitive tasks. In this paper, we review the prediction of immunotherapy efficacy by different biomarkers (genomic, transcriptomic, proteomic, microbiomic, and radiomic), together with the use of artificial intelligence and the challenges and future directions of these fields.

Responding to the healthcare workforce shortage: A scoping review exploring anatomical pathologists' professional identities over time

Anatomical pathology (AP) is an anatomy-centric medical specialty devoted to tissue-based diagnosis of disease. The field faces a current and predicted workforce shortage, likely increasing diagnostic wait times and delaying patient access to urgent treatment. A lack of AP exposure is proposed to preclude recruitment to the field, as medical students are afforded only a limited understanding of who a pathologist is and what they do (their professional identity/PI and role). Anatomical sciences educators may be well placed to increase student understanding of anatomical pathologists' PI features, but until features of anatomical pathologists' PI are understood, recommendations for anatomy educators are premature. Thus, this scoping review asked: "What are the professional identity features of anatomical pathologists reported in the literature, and how have these changed over time?" A six-stage scoping review was performed. Medline and PubMed, Global Health, and Embase were used to identify relevant studies (n = 74). Team-based framework analysis identified that features of anatomical pathologists' professional identity encompass five overarching themes: professional practice, views about the role, training and education, personal implications, and technology. Technology was identified as an important theme of anatomical pathologists' PI, as it intersected with many other PI feature themes, including diagnosis and collaboration. This review found that pathologists may sometimes perceive professional competition with technology, such as artificial intelligence. These findings suggest unique opportunities for integrating AP-specific PI features into anatomy teaching, which may foster student interest in AP, and potentially increase recruitment into the field.

Development of AI-Based Diagnostic Algorithm for Nasal Bone Fracture Using Deep Learning

Facial bone fractures are relatively common, with the nasal bone the most frequently fractured facial bone. Computed tomography is the gold standard for diagnosing such fractures. Most nasal bone fractures can be treated using a closed reduction. However, delayed diagnosis may cause nasal deformity or other complications that are difficult and expensive to treat. In this study, the authors developed an algorithm for diagnosing nasal fractures by learning computed tomography images of facial bones with artificial intelligence through deep learning. A significant concordance with human doctors' reading results of 100% sensitivity and 77% specificity was achieved. Herein, the authors report the results of a pilot study on the first stage of developing an algorithm for analyzing fractures in the facial bone.

Development of an automated estimation of foot process width using deep learning in kidney biopsies from patients with Fabry, minimal change, and diabetic kidney diseases

Podocyte injury plays a key role in pathogenesis of many kidney diseases with increased podocyte foot process width (FPW), an important measure of podocyte injury. Unfortunately, there is no consensus on the best way to estimate FPW and unbiased stereology, the current gold standard, is time consuming and not widely available. To address this, we developed an automated FPW estimation technique using deep learning. A U-Net architecture variant model was trained to semantically segment the podocyte-glomerular basement membrane interface and filtration slits. Additionally, we employed a post-processing computer vision approach to accurately estimate FPW. A custom segmentation utility was also created to manually classify these structures on digital electron microscopy (EM) images and to prepare a training dataset. The model was applied to EM images of kidney biopsies from 56 patients with Fabry disease, 15 with type 2 diabetes, 10 with minimal change disease, and 17 normal individuals. The results were compared with unbiased stereology measurements performed by expert technicians unaware of the clinical information. FPW measured by deep learning and by the expert technicians were highly correlated and not statistically different in any of the studied groups. A Bland-Altman plot confirmed interchangeability of the methods. FPW measurement time per biopsy was substantially reduced by deep learning. Thus, we have developed a novel validated deep learning model for FPW measurement on EM images. The model is accessible through a cloud-based application making calculation of this important biomarker more widely accessible for research and clinical applications.

Synergies and Challenges in the Preclinical and Clinical Implementation of Pathology Artificial Intelligence Applications

Recent introduction of digitalization in pathology has disrupted the field greatly with the potential to change the area immensely. Digital pathology has created the potential of applying advanced quantitative analysis and artificial intelligence (AI) to the domain. In this study, we present an overview of what pathology AI applications have the greatest potential of widespread adoption in the preclinical domain and subsequently, in the clinical setting. We also discuss the major challenges in AI adoption being faced by the field of digital and computational pathology. We review the research literature in the domain and present a detailed analysis of the most promising areas of digital and computational pathology AI research and identify applications that are likely to see the first adoptions of AI technology. Our analysis shows that certain areas and fields of application have received more attention and can potentially affect the field of digital and computational pathology more favorably, leading to the advancement of the field. We also present the main challenges that are faced by the field and provide a comparative analysis of various aspects that are likely to influence the field for the long term in the future.

Deep Learning for the Pathologic Diagnosis of Hepatocellular Carcinoma, Cholangiocarcinoma, and Metastatic Colorectal Cancer

Diagnosing primary liver cancers, particularly hepatocellular carcinoma (HCC) and cholangiocarcinoma (CC), is a challenging and labor-intensive process, even for experts, and secondary liver cancers further complicate the diagnosis. Artificial intelligence (AI) offers promising solutions to these diagnostic challenges by facilitating the histopathological classification of tumors using digital whole slide images (WSIs). This study aimed to develop a deep learning model for distinguishing HCC, CC, and metastatic colorectal cancer (mCRC) using histopathological images and to discuss its clinical implications. The WSIs from HCC, CC, and mCRC were used to train the classifiers. For normal/tumor classification, the areas under the curve (AUCs) were 0.989, 0.988, and 0.991 for HCC, CC, and mCRC, respectively. Using proper tumor tissues, the HCC/other cancer type classifier was trained to effectively distinguish HCC from CC and mCRC, with a concatenated AUC of 0.998. Subsequently, the CC/mCRC classifier differentiated CC from mCRC with a concatenated AUC of 0.995. However, testing on an external dataset revealed that the HCC/other cancer type classifier underperformed with an AUC of 0.745. After combining the original training datasets with external datasets and retraining, the classification drastically improved, all achieving AUCs of 1.000. Although these results are promising and offer crucial insights into liver cancer, further research is required for model refinement and validation.

RCKD: Response-Based Cross-Task Knowledge Distillation for Pathological Image Analysis

We propose a novel transfer learning framework for pathological image analysis, the Response-based Cross-task Knowledge Distillation (RCKD), which improves the performance of the model by pretraining it on a large unlabeled dataset guided by a high-performance teacher model. RCKD first pretrains a student model to predict the nuclei segmentation results of the teacher model for unlabeled pathological images, and then fine-tunes the pretrained model for the downstream tasks, such as organ cancer sub-type classification and cancer region segmentation, using relatively small target datasets. Unlike conventional knowledge distillation, RCKD does not require that the target tasks of the teacher and student models be the same. Moreover, unlike conventional transfer learning, RCKD can transfer knowledge between models with different architectures. In addition, we propose a lightweight architecture, the Convolutional neural network with Spatial Attention by Transformers (CSAT), for processing high-resolution pathological images with limited memory and computation. CSAT exhibited a top-1 accuracy of 78.6% on ImageNet with only 3M parameters and 1.08 G multiply-accumulate (MAC) operations. When pretrained by RCKD, CSAT exhibited average classification and segmentation accuracies of 94.2% and 0.673 mIoU on six pathological image datasets, which is 4% and 0.043 mIoU higher than EfficientNet-B0, and 7.4% and 0.006 mIoU higher than ConvNextV2-Atto pretrained on ImageNet, respectively.

Digitization of Pathology Labs: A Review of Lessons Learned

Pathology laboratories are increasingly using digital workflows. This has the potential of increasing laboratory efficiency, but the digitization process also involves major challenges. Several reports have been published describing the individual experiences of specific laboratories with the digitization process. However, a comprehensive overview of the lessons learned is still lacking. We provide an overview of the lessons learned for different aspects of the digitization process, including digital case management, digital slide reading, and computer-aided slide reading. We also cover metrics used for monitoring performance and pitfalls and corresponding values observed in practice. The overview is intended to help pathologists, information technology decision makers, and administrators to benefit from the experiences of others and to implement the digitization process in an optimal way to make their own laboratory future-proof.

Translation of tissue-based artificial intelligence into clinical practice: from discovery to adoption

Digital pathology (DP), or the digitization of pathology images, has transformed oncology research and cancer diagnostics. The application of artificial intelligence (AI) and other forms of machine learning (ML) to these images allows for better interpretation of morphology, improved quantitation of biomarkers, introduction of novel concepts to discovery and diagnostics (such as spatial distribution of cellular elements), and the promise of a new paradigm of cancer biomarkers. The application of AI to tissue analysis can take several conceptual approaches, within the domains of language modelling and image analysis, such as Deep Learning Convolutional Neural Networks, Multiple Instance Learning approaches, or the modelling of risk scores and their application to ML. The use of different approaches solves different problems within pathology workflows, including assistive applications for the detection and grading of tumours, quantification of biomarkers, and the delivery of established and new image-based biomarkers for treatment prediction and prognostic purposes. All these AI formats, applied to digital tissue images, are also beginning to transform our approach to clinical trials. In parallel, the novelty of DP/AI devices and the related computational science pipeline introduces new requirements for manufacturers to build into their design, development, regulatory and post-market processes, which may need to be taken into account when using AI applied to tissues in cancer discovery. Finally, DP/AI represents challenge to the way we accredit new diagnostic tools with clinical applicability, the understanding of which will allow cancer patients to have access to a new generation of complex biomarkers.

Artificial Intelligence Revolutionizing the Field of Medical Education

Medical education has ventured into a new arena of computer-assisted teaching powered by artificial intelligence (AI). In medical institutions, AI can serve as an intelligent tool facilitating the decision-making process effectively. AI can enhance teaching by assisting in developing new strategies for educators. Similarly, students also benefit from intelligent systems playing the role of competent teachers. Thus, AI-integrated medical education paves new opportunities for advanced teaching and learning experiences and improved outcomes. On the other hand, optical mark recognition and automated scoring are ways AI can also transform into a real-time assessor and evaluator in medical education. This review summarizes the AI tools and their application in medical teaching or learning, assessment, and administrative support. This article can aid medical institutes in planning and implementing AI according to the needs of the educators.

A novel cost function for nuclei segmentation and classification in imbalanced histopathology data-sets

Cancer is a major global health problem, causing millions of deaths yearly. Histopathological analysis plays a crucial role in detecting and diagnosing various types of cancer, enabling an accurate diagnosis to inform targeted treatment planning, allowing for better cancer staging, and ultimately improving prognosis. We aim to detect cancer earlier, which can ultimately help reduce mortality rates and enhance patients' quality of life. However, detecting and classifying rare cells is a key challenge for pathologists and researchers. Many histopathological data-sets contain imbalanced data, with only a few instances of rare cells whose unique morphological structures can impede early diagnosis efforts. Our model, SPNet, a spatially aware convolutional neural network, addresses this problem by employing a spatial data balancing technique, enhancing the classification of rare nuclei by 21.8 %. Since nuclei often cluster and exhibit patterns of the same class, SPNet's novel cost function targets spatial regions, resulting in a 1.9 % increase in the F1 classification of rare class types within the CoNSeP dataset. When integrated with a ResNet50-SE encoder, SPNet increases the mean F1 score for classifying all nuclei in the CoNSeP dataset by 4.3 %, compared to the benchmark set by the state-of-the-art HoVer-Net model. The potential integration of SPNet into existing medical devices could allow us to streamline diagnostic processes and minimise false negatives.

Artificial Intelligence in Digital Pathology for Bladder Cancer: Hype or Hope? A Systematic Review

Bladder cancer (BC) diagnosis and prediction of prognosis are hindered by subjective pathological evaluation, which may cause misdiagnosis and under-/over-treatment. Computational pathology (CPATH) can identify clinical outcome predictors, offering an objective approach to improve prognosis. However, a systematic review of CPATH in BC literature is lacking. Therefore, we present a comprehensive overview of studies that used CPATH in BC, analyzing 33 out of 2285 identified studies. Most studies analyzed regions of interest to distinguish normal versus tumor tissue and identify tumor grade/stage and tissue types (e.g., urothelium, stroma, and muscle). The cell's nuclear area, shape irregularity, and roundness were the most promising markers to predict recurrence and survival based on selected regions of interest, with >80% accuracy. CPATH identified molecular subtypes by detecting features, e.g., papillary structures, hyperchromatic, and pleomorphic nuclei. Combining clinicopathological and image-derived features improved recurrence and survival prediction. However, due to the lack of outcome interpretability and independent test datasets, robustness and clinical applicability could not be ensured. The current literature demonstrates that CPATH holds the potential to improve BC diagnosis and prediction of prognosis. However, more robust, interpretable, accurate models and larger datasets-representative of clinical scenarios-are needed to address artificial intelligence's reliability, robustness, and black box challenge.

Self-supervised deep learning for highly efficient spatial immunophenotyping

BACKGROUND

Efficient biomarker discovery and clinical translation depend on the fast and accurate analytical output from crucial technologies such as multiplex imaging. However, reliable cell classification often requires extensive annotations. Label-efficient strategies are urgently needed to reveal diverse cell distribution and spatial interactions in large-scale multiplex datasets.

METHODS

This study proposed Self-supervised Learning for Antigen Detection (SANDI) for accurate cell phenotyping while mitigating the annotation burden. The model first learns intrinsic pairwise similarities in unlabelled cell images, followed by a classification step to map learnt features to cell labels using a small set of annotated references. We acquired four multiplex immunohistochemistry datasets and one imaging mass cytometry dataset, comprising 2825 to 15,258 single-cell images to train and test the model.

FINDINGS

With 1% annotations (18-114 cells), SANDI achieved weighted F1-scores ranging from 0.82 to 0.98 across the five datasets, which was comparable to the fully supervised classifier trained on 1828-11,459 annotated cells (-0.002 to -0.053 of averaged weighted F1-score, Wilcoxon rank-sum test, P = 0.31). Leveraging the immune checkpoint markers stained in ovarian cancer slides, SANDI-based cell identification reveals spatial expulsion between PD1-expressing T helper cells and T regulatory cells, suggesting an interplay between PD1 expression and T regulatory cell-mediated immunosuppression.

INTERPRETATION

By striking a fine balance between minimal expert guidance and the power of deep learning to learn similarity within abundant data, SANDI presents new opportunities for efficient, large-scale learning for histology multiplex imaging data.

FUNDING

This study was funded by the Royal Marsden/ICR National Institute of Health Research Biomedical Research Centre.

Pathologists' first opinions on barriers and facilitators of computational pathology adoption in oncological pathology: an international study

Computational pathology (CPath) algorithms detect, segment or classify cancer in whole slide images, approaching or even exceeding the accuracy of pathologists. Challenges have to be overcome before these algorithms can be used in practice. We therefore aim to explore international perspectives on the future role of CPath in oncological pathology by focusing on opinions and first experiences regarding barriers and facilitators. We conducted an international explorative eSurvey and semi-structured interviews with pathologists utilizing an implementation framework to classify potential influencing factors. The eSurvey results showed remarkable variation in opinions regarding attitude, understandability and validation of CPath. Interview results showed that barriers focused on the quality of available evidence, while most facilitators concerned strengths of CPath. A lack of consensus was present for multiple factors, such as the determination of sufficient validation using CPath, the preferred function of CPath within the digital workflow and the timing of CPath introduction in pathology education. The diversity in opinions illustrates variety in influencing factors in CPath adoption. A next step would be to quantitatively determine important factors for adoption and initiate validation studies. Both should include clear case descriptions and be conducted among a more homogenous panel of pathologists based on sub specialization.

Cell projection plots: A novel visualization of bone marrow aspirate cytology

Deep models for cell detection have demonstrated utility in bone marrow cytology, showing impressive results in terms of accuracy and computational efficiency. However, these models have yet to be implemented in the clinical diagnostic workflow. Additionally, the metrics used to evaluate cell detection models are not necessarily aligned with clinical goals and targets. In order to address these issues, we introduce novel, automatically generated visual summaries of bone marrow aspirate specimens called cell projection plots (CPPs). Encompassing relevant biological patterns such as neutrophil maturation, CPPs provide a compact summary of bone marrow aspirate cytology. To gauge clinical relevance, CPPs were inspected by 3 hematopathologists, who decided whether corresponding diagnostic synopses matched with generated CPPs. Pathologists were able to match CPPs to the correct synopsis with a matching degree of 85%. Our finding suggests CPPs can represent clinically relevant information from bone marrow aspirate specimens and may be used to efficiently summarize bone marrow cytology to pathologists. CPPs could be a step toward human-centered implementation of artificial intelligence (AI) in hematopathology, and a basis for a diagnostic-support tool for digital pathology workflows.

Heterogeneity of the tumor immune microenvironment and clinical interventions

The tumor immune microenvironment (TIME) is broadly composed of various immune cells, and its heterogeneity is characterized by both immune cells and stromal cells. During the course of tumor formation and progression and anti-tumor treatment, the composition of the TIME becomes heterogeneous. Such immunological heterogeneity is not only present between populations but also exists on temporal and spatial scales. Owing to the existence of TIME, clinical outcomes can differ when a similar treatment strategy is provided to patients. Therefore, a comprehensive assessment of TIME heterogeneity is essential for developing precise and effective therapies. Facilitated by advanced technologies, it is possible to understand the complexity and diversity of the TIME and its influence on therapy responses. In this review, we discuss the potential reasons for TIME heterogeneity and the current approaches used to explore it. We also summarize clinical intervention strategies based on associated mechanisms or targets to control immunological heterogeneity.

Between Generating Noise and Generating Images: Noise in the Correct Frequency Improves the Quality of Synthetic Histopathology Images for Digital Pathology

Artificial intelligence and machine learning techniques have the promise to revolutionize the field of digital pathology. However, these models demand considerable amounts of data, while the availability of unbiased training data is limited. Synthetic images can augment existing datasets, to improve and validate AI algorithms. Yet, controlling the exact distribution of cellular features within them is still challenging. One of the solutions is harnessing conditional generative adversarial networks that take a semantic mask as an input rather than a random noise. Unlike other domains, outlining the exact cellular structure of tissues is hard, and most of the input masks depict regions of cell types. This is also the case for non-small cell lung cancer, the most common type of lung cancer. Deciding whether a patient would receive immunotherapy depends on quantifying regions of stained cells. However, using polygon-based masks introduce inherent artifacts within the synthetic images - due to the mismatch between the polygon size and the single-cell size. In this work, we show that introducing random single-pixel noise with the appropriate spatial frequency into a polygon semantic mask can dramatically improve the quality of the synthetic images. We used our platform to generate synthetic images of immunohistochemistry-treated lung biopsies. We test the quality of the images using a three-fold validation procedure. First, we show that adding the appropriate noise frequency yields 87% of the similarity metrics improvement that is obtained by adding the actual single-cell features. Second, we show that the synthetic images pass the Turing test. Finally, we show that adding these synthetic images to the train set improves AI performance in terms of PD-L1 semantic segmentation performances. Our work suggests a simple and powerful approach for generating synthetic data on demand to unbias limited datasets to improve the algorithms' accuracy and validate their robustness.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

Computational clustering reveals differentiated coronary artery calcium progression at prevalent levels of pulse wave velocity by classifying high-risk patients

Many studies found that increased arterial stiffness is significantly associated with the presence and progression of Coronary Calcium Score (CCS). However, none so far have used machine learning algorithms to improve their value. Therefore, this study aims to evaluate the association between carotid-femoral Pulse Wave Velocity (cfPWV) and CCS score through computational clustering. We conducted a retrospective cross-sectional study using data from a cardiovascular risk screening program that included 377 participants. We used an unsupervised clustering algorithm using age, weight, height, blood pressure, heart rate, and cfPWV as input variables. Differences between cluster groups were analyzed through Chi-square and T-student tests. The association between (i) cfPWV and age groups, (ii) log (CCS) and age groups, and (iii) cfPWV and log(CCS) were addressed through linear regression analysis. Clusters were labeled post hoc based on cardiovascular risk. A "higher-risk group" had significantly higher left (0.76 vs. 0.70 mm, P < 0.001) and right (0.71 vs. 0.66 mm, P = 0.003) intima-media thickness, CCS (42 vs. 4 Agatston units, P = 0.012), and ascending (3.40 vs. 3.20 cm, P < 0.001) and descending (2.60 vs. 2.37 cm, P < 0.001) aorta diameters. Association with age appeared linear for cfPWV and exponential for log (CCS). The progression of the log (CCS) and cfPWV through age groups was steeper in the "higher-risk group" than in the "lower-risk group". cfPWV strongly correlated with CCS, and CCS progression over cfPWV differed among clusters. This finding could improve PWV as a "gate-keeper" of CCS testing and potentially enhance cardiovascular risk stratification.

The convergence of traditional and digital biomarkers through AI-assisted biosensing: A new era in translational diagnostics?

Advances in consumer electronics, alongside the fields of microfluidics and nanotechnology have brought to the fore low-cost wearable/portable smart devices. Although numerous smart devices that track digital biomarkers have been successfully translated from bench-to-bedside, only a few follow the same fate when it comes to track traditional biomarkers. Current practices still involve laboratory-based tests, followed by blood collection, conducted in a clinical setting as they require trained personnel and specialized equipment. In fact, real-time, passive/active and robust sensing of physiological and behavioural data from patients that can feed artificial intelligence (AI)-based models can significantly improve decision-making, diagnosis and treatment at the point-of-procedure, by circumventing conventional methods of sampling, and in person investigation by expert pathologists, who are scarce in developing countries. This review brings together conventional and digital biomarker sensing through portable and autonomous miniaturized devices. We first summarise the technological advances in each field vs the current clinical practices and we conclude by merging the two worlds of traditional and digital biomarkers through AI/ML technologies to improve patient diagnosis and treatment. The fundamental role, limitations and prospects of AI in realizing this potential and enhancing the existing technologies to facilitate the development and clinical translation of "point-of-care" (POC) diagnostics is finally showcased.

Multimodal data fusion for cancer biomarker discovery with deep learning

Technological advances now make it possible to study a patient from multiple angles with high-dimensional, high-throughput multi-scale biomedical data. In oncology, massive amounts of data are being generated ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has significantly advanced the analysis of biomedical data. However, most approaches focus on single data modalities leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalised medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability, and standardisation of datasets.

Comparison of Ultra-Magnifying Endocytoscopic and Hematoxylin-Eosin-Stained Images of Lung Specimens

Endocytoscopy enables real-time observation of lesions at ultra-magnification. In the gastrointestinal and respiratory fields, endocytoscopic images are similar to hematoxylin-eosin-stained images. This study aimed to compare the nuclear features of pulmonary lesions in endocytoscopic and hematoxylin-eosin-stained images. We performed an endocytoscopy to observe resected specimens of normal lung tissue and lesions. Nuclear features were extracted using ImageJ. We analyzed five nuclear features: nuclear number per area, mean nucleus area, median circularity, coefficient of variation of roundness, and median Voronoi area. We conducted dimensionality reduction analyses for these features, followed by assessments of the inter-observer agreement among two pathologists and two pulmonologists to evaluate endocytoscopic videos. We analyzed the nuclear features of hematoxylin-eosin-stained and endocytoscopic images from 40 and 33 cases, respectively. Endocytoscopic and hematoxylin-eosin-stained images displayed a similar tendency for each feature, despite there being no correlation. Conversely, the dimensionality reduction analyses demonstrated similar distributions of normal lung and malignant clusters in both images, thus differentiating between the clusters. The diagnostic accuracy of the pathologists was 58.3% and 52.8% (κ-value 0.38, fair), and that of the pulmonologists was 50% and 47.2% (κ-value 0.33, fair). The five nuclear features of pulmonary lesions were similar in the endocytoscopic and hematoxylin-eosin-stained images.

Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images

BACKGROUND

Applying deep learning to digital histopathology is hindered by the scarcity of manually annotated datasets. While data augmentation can ameliorate this obstacle, its methods are far from standardized. Our aim was to systematically explore the effects of skipping data augmentation; applying data augmentation to different subsets of the whole dataset (training set, validation set, test set, two of them, or all of them); and applying data augmentation at different time points (before, during, or after dividing the dataset into three subsets). Different combinations of the above possibilities resulted in 11 ways to apply augmentation. The literature contains no such comprehensive systematic comparison of these augmentation ways.

RESULTS

Non-overlapping photographs of all tissues on 90 hematoxylin-and-eosin-stained urinary bladder slides were obtained. Then, they were manually classified as either inflammation (5948 images), urothelial cell carcinoma (5811 images), or invalid (3132 images; excluded). If done, augmentation was eight-fold by flipping and rotation. Four convolutional neural networks (Inception-v3, ResNet-101, GoogLeNet, and SqueezeNet), pre-trained on the ImageNet dataset, were fine-tuned to binary classify images of our dataset. This task was the benchmark for our experiments. Model testing performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. Model validation accuracy was also estimated. The best testing performance was achieved when augmentation was done to the remaining data after test-set separation, but before division into training and validation sets. This leaked information between the training and the validation sets, as evidenced by the optimistic validation accuracy. However, this leakage did not cause the validation set to malfunction. Augmentation before test-set separation led to optimistic results. Test-set augmentation yielded more accurate evaluation metrics with less uncertainty. Inception-v3 had the best overall testing performance.

CONCLUSIONS

In digital histopathology, augmentation should include both the test set (after its allocation), and the remaining combined training/validation set (before being split into separate training and validation sets). Future research should try to generalize our results.

Noninferiority of Artificial Intelligence-Assisted Analysis of Ki-67 and Estrogen/Progesterone Receptor in Breast Cancer Routine Diagnostics

Image analysis assistance with artificial intelligence (AI) has become one of the great promises over recent years in pathology, with many scientific studies being published each year. Nonetheless, and perhaps surprisingly, only few image AI systems are already in routine clinical use. A major reason for this is the missing validation of the robustness of many AI systems: beyond a narrow context, the large variability in digital images due to differences in preanalytical laboratory procedures, staining procedures, and scanners can be challenging for the subsequent image analysis. Resulting faulty AI analysis may bias the pathologist and contribute to incorrect diagnoses and, therefore, may lead to inappropriate therapy or prognosis. In this study, a pretrained AI assistance tool for the quantification of Ki-67, estrogen receptor (ER), and progesterone receptor (PR) in breast cancer was evaluated within a realistic study set representative of clinical routine on a total of 204 slides (72 Ki-67, 66 ER, and 66 PR slides). This represents the cohort with the largest image variance for AI tool evaluation to date, including 3 staining systems, 5 whole-slide scanners, and 1 microscope camera. These routine cases were collected without manual preselection and analyzed by 10 participant pathologists from 8 sites. Agreement rates for individual pathologists were found to be 87.6% for Ki-67 and 89.4% for ER/PR, respectively, between scoring with and without the assistance of the AI tool regarding clinical categories. Individual AI analysis results were confirmed by the majority of pathologists in 95.8% of Ki-67 cases and 93.2% of ER/PR cases. The statistical analysis provides evidence for high interobserver variance between pathologists (Krippendorff's α, 0.69) in conventional immunohistochemical quantification. Pathologist agreement increased slightly when using AI support (Krippendorff α, 0.72). Agreement rates of pathologist scores with and without AI assistance provide evidence for the reliability of immunohistochemical scoring with the support of the investigated AI tool under a large number of environmental variables that influence the quality of the diagnosed tissue images.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that sugest opportunities for future spatially aware WSI research using limited pathology datasets.

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.

Application of digital pathology and machine learning in the liver, kidney and lung diseases

The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.

Deepfake Histologic Images for Enhancing Digital Pathology

A pathologist's optical microscopic examination of thinly cut, stained tissue on glass slides prepared from a formalin-fixed paraffin-embedded tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of pathologists is dependent on their direct experience with common and rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task, and artificially synthesized histologic images can prove greatly beneficial. In this study, we present an approach to not only generate histologic images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves developing a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting the diagnostic ability of machine learning methods and their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data when training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images, and their analyses showed a similar level of interobserver agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the morphology of the complex microenvironment in such tissues can be reproduced. Finally, we present the ability for a user to generate deepfake histologic images using a simple markup of sematic labels.

An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS)

Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.

Whole Slide Images in Artificial Intelligence Applications in Digital Pathology: Challenges and Pitfalls

The use of digitized data in pathology research is rapidly increasing. The whole slide image (WSI) is an indispensable part of the visual examination of slides in digital pathology and artificial intelligence applications; therefore, the acquisition of WSI with the highest quality is essential. Unlike the conventional routine of pathology, the digital conversion of tissue slides and the differences in its use pose difficulties for pathologists. We categorized these challenges into three groups: before, during, and after the WSI acquisition. The problems before WSI acquisition are usually related to the quality of the glass slide and reflect all existing problems in the analytical process in pathology laboratories. WSI acquisition problems are dependent on the device used to produce the final image file. They may be related to the parts of the device that create an optical image or the hardware and software that enable digitization. Post-WSI acquisition issues are related to the final image file itself, which is the final form of this data, or the software and hardware that will use this file. Because of the digital nature of the data, most of the difficulties are related to the capabilities of the hardware or software. Being aware of the challenges and pitfalls of using digital pathology and AI will make pathologists' integration to the new technologies easier in their daily practice or research.

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.