Translational AI and Deep Learning in Diagnostic Pathology

SPIE-AAPM-NCI BreastPathQ challenge: an image analysis challenge for quantitative tumor cellularity assessment in breast cancer histology images following neoadjuvant treatment

Purpose: The breast pathology quantitative biomarkers (BreastPathQ) challenge was a grand challenge organized jointly by the International Society for Optics and Photonics (SPIE), the American Association of Physicists in Medicine (AAPM), the U.S. National Cancer Institute (NCI), and the U.S. Food and Drug Administration (FDA). The task of the BreastPathQ challenge was computerized estimation of tumor cellularity (TC) in breast cancer histology images following neoadjuvant treatment. Approach: A total of 39 teams developed, validated, and tested their TC estimation algorithms during the challenge. The training, validation, and testing sets consisted of 2394, 185, and 1119 image patches originating from 63, 6, and 27 scanned pathology slides from 33, 4, and 18 patients, respectively. The summary performance metric used for comparing and ranking algorithms was the average prediction probability concordance (PK) using scores from two pathologists as the TC reference standard. Results: Test PK performance ranged from 0.497 to 0.941 across the 100 submitted algorithms. The submitted algorithms generally performed well in estimating TC, with high-performing algorithms obtaining comparable results to the average interrater PK of 0.927 from the two pathologists providing the reference TC scores. Conclusions: The SPIE-AAPM-NCI BreastPathQ challenge was a success, indicating that artificial intelligence/machine learning algorithms may be able to approach human performance for cellularity assessment and may have some utility in clinical practice for improving efficiency and reducing reader variability. The BreastPathQ challenge can be accessed on the Grand Challenge website.

Digital/Computational Technology for Molecular Cytology Testing: A Short Technical Note with Literature Review

This short article describes the method of digital cytopathology using Z-stack scanning with or without extended focusing. This technology is suitable to observe such thick clusters as adenocarcinoma on cytologic specimens. Artificial intelligence (AI) has been applied to histological images, but its application on cytologic images is still limited. This article describes our attempt to apply AI technology to cytologic digital images. For molecular analysis, cytologic materials, such as smear, LBC, and cell blocks, have been successfully used for targeted single gene detection and multiplex gene analysis with next-generation sequencing. As a future perspective, the system can be connected to full automation by combining digital cytopathology with AI application to detect target cancer cells and to perform molecular analysis. The literature review is updated according to the subjects.

Feasibility of deep learning-based fully automated classification of microsatellite instability in tissue slides of colorectal cancer

High levels of microsatellite instability (MSI-H) occurs in about 15% of sporadic colorectal cancer (CRC) and is an important predictive marker for response to immune checkpoint inhibitors. To test the feasibility of a deep learning (DL)-based classifier as a screening tool for MSI status, we built a fully automated DL-based MSI classifier using pathology whole-slide images (WSIs) of CRCs. On small image patches of The Cancer Genome Atlas (TCGA) CRC WSI dataset, tissue/non-tissue, normal/tumor and MSS/MSI-H classifiers were applied sequentially for the fully automated prediction of the MSI status. The classifiers were also tested on an independent cohort. Furthermore, to test how the expansion of the training data affects the performance of the DL-based classifier, additional classifier trained on both TCGA and external datasets was tested. The areas under the receiver operating characteristic curves were 0.892 and 0.972 for the TCGA and external datasets, respectively, by a classifier trained on both datasets. The performance of the DL-based classifier was much better than that of previously reported histomorphology-based methods. We speculated that about 40% of CRC slides could be screened for MSI status without molecular testing by the DL-based classifier. These results demonstrated that the DL-based method has potential as a screening tool to discriminate molecular alteration in tissue slides.

Pathomics in urology

PURPOSE OF REVIEW

Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology.

RECENT FINDINGS

There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application.

SUMMARY

In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.

Role of AI and Histopathological Images in Detecting Prostate Cancer: A Survey

Prostate cancer is one of the most identified cancers and second most prevalent among cancer-related deaths of men worldwide. Early diagnosis and treatment are substantial to stop or handle the increase and spread of cancer cells in the body. Histopathological image diagnosis is a gold standard for detecting prostate cancer as it has different visual characteristics but interpreting those type of images needs a high level of expertise and takes too much time. One of the ways to accelerate such an analysis is by employing artificial intelligence (AI) through the use of computer-aided diagnosis (CAD) systems. The recent developments in artificial intelligence along with its sub-fields of conventional machine learning and deep learning provide new insights to clinicians and researchers, and an abundance of research is presented specifically for histopathology images tailored for prostate cancer. However, there is a lack of comprehensive surveys that focus on prostate cancer using histopathology images. In this paper, we provide a very comprehensive review of most, if not all, studies that handled the prostate cancer diagnosis using histopathological images. The survey begins with an overview of histopathological image preparation and its challenges. We also briefly review the computing techniques that are commonly applied in image processing, segmentation, feature selection, and classification that can help in detecting prostate malignancies in histopathological images.

Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification

BACKGROUND

Clinicians and pathologists traditionally use patient data in addition to clinical examination to support their diagnoses.

OBJECTIVES

We investigated whether a combination of histologic whole slides image (WSI) analysis based on convolutional neural networks (CNNs) and commonly available patient data (age, sex and anatomical site of the lesion) in a binary melanoma/nevus classification task could increase the performance compared with CNNs alone.

METHODS

We used 431 WSIs from two different laboratories and analysed the performance of classifiers that used the image or patient data individually or three common fusion techniques. Furthermore, we tested a naive combination of patient data and an image classifier: for cases interpreted as 'uncertain' (CNN output score <0.7), the decision of the CNN was replaced by the decision of the patient data classifier.

RESULTS

The CNN on its own achieved the best performance (mean ± standard deviation of five individual runs) with AUROC of 92.30% ± 0.23% and balanced accuracy of 83.17% ± 0.38%. While the classification performance was not significantly improved in general by any of the tested fusions, naive strategy of replacing the image classifier with the patient data classifier on slides with low output scores improved balanced accuracy to 86.72% ± 0.36%.

CONCLUSION

In most cases, the CNN on its own was so accurate that patient data integration did not provide any benefit. However, incorporating patient data for lesions that were classified by the CNN with low 'confidence' improved balanced accuracy.

Medical image analysis based on deep learning approach

Medical imaging plays a significant role in different clinical applications such as medical procedures used for early detection, monitoring, diagnosis, and treatment evaluation of various medical conditions. Basicsof the principles and implementations of artificial neural networks and deep learning are essential for understanding medical image analysis in computer vision. Deep Learning Approach (DLA) in medical image analysis emerges as a fast-growing research field. DLA has been widely used in medical imaging to detect the presence or absence of the disease. This paper presents the development of artificial neural networks, comprehensive analysis of DLA, which delivers promising medical imaging applications. Most of the DLA implementations concentrate on the X-ray images, computerized tomography, mammography images, and digital histopathology images. It provides a systematic review of the articles for classification, detection, and segmentation of medical images based on DLA. This review guides the researchers to think of appropriate changes in medical image analysis based on DLA.

Application of Single-Cell Approaches to Study Myeloproliferative Neoplasm Biology

Philadelphia-negative myeloproliferative neoplasms (MPNs) are an excellent tractable disease model of a number of aspects of human cancer biology, including genetic evolution, tissue-associated fibrosis, and cancer stem cells. In this review, we discuss recent insights into MPN biology gained from the application of a number of new single-cell technologies to study human disease, with a specific focus on single-cell genomics, single-cell transcriptomics, and digital pathology.

Artificial intelligence and computational pathology

Data processing and learning has become a spearhead for the advancement of medicine, with pathology and laboratory medicine has no exception. The incorporation of scientific research through clinical informatics, including genomics, proteomics, bioinformatics, and biostatistics, into clinical practice unlocks innovative approaches for patient care. Computational pathology is burgeoning subspecialty in pathology that promises a better-integrated solution to whole-slide images, multi-omics data, and clinical informatics. However, computational pathology faces several challenges, including the ability to integrate raw data from different sources, limitation of hardware processing capacity, and a lack of specific training programs, as well as issues on ethics and larger societal acceptable practices that are still solidifying. The establishment of the entire industry of computational pathology requires far-reaching changes of the three essential elements connecting patients and doctors: the local laboratory, the scan center, and the central cloud hub/portal for data processing and retrieval. Computational pathology, unlocked through information integration and advanced digital communication networks, has the potential to improve clinical workflow efficiency, diagnostic quality, and ultimately create personalized diagnosis and treatment plans for patients. This review describes clinical perspectives and discusses the statistical methods, clinical applications, potential obstacles, and future directions of computational pathology.

Artificial Intelligence in Pathology: From Prototype to Product

Modern image analysis techniques based on artificial intelligence (AI) have great potential to improve the quality and efficiency of diagnostic procedures in pathology and to detect novel biomarkers. Despite thousands of published research papers on applications of AI in pathology, hardly any research implementations have matured into commercial products for routine use. Bringing an AI solution for pathology to market poses significant technological, business, and regulatory challenges. In this paper, we provide a comprehensive overview and advice on how to meet these challenges. We outline how research prototypes can be turned into a product-ready state and integrated into the IT infrastructure of clinical laboratories. We also discuss business models for profitable AI solutions and reimbursement options for computer assistance in pathology. Moreover, we explain how to obtain regulatory approval so that AI solutions can be launched as in vitro diagnostic medical devices. Thus, this paper offers computer scientists, software companies, and pathologists a road map for transforming prototypes of AI solutions into commercial products.

Artificial intelligence and algorithmic computational pathology: an introduction with renal allograft examples

Whole slide imaging, which is an important technique in the field of digital pathology, has recently been the subject of increased interest and avenues for utilisation, and with more widespread whole slide image (WSI) utilisation, there will also be increased interest in and implementation of image analysis (IA) techniques. IA includes artificial intelligence (AI) and targeted or hypothesis-driven algorithms. In the overall pathology field, the number of citations related to these topics has increased in recent years. Renal pathology is one anatomical pathology subspecialty that has utilised WSIs and IA algorithms; it can be argued that renal transplant pathology could be particularly suited for whole slide imaging and IA, as renal transplant pathology is frequently classified by use of the semiquantitative Banff classification of renal allograft pathology. Hypothesis-driven/targeted algorithms have been used in the past for the assessment of a variety of features in the kidney (e.g. interstitial fibrosis, tubular atrophy, inflammation); in recent years, the amount of research has particularly increased in the area of AI/machine learning for the identification of glomeruli, for histological segmentation, and for other applications. Deep learning is the form of machine learning that is most often used for such AI approaches to the 'big data' of pathology WSIs, and deep learning methods such as artificial neural networks (ANNs)/convolutional neural networks (CNNs) are utilised. Unsupervised and supervised AI algorithms can be employed to accomplish image or semantic classification. In this review, AI and other IA algorithms applied to WSIs are discussed, and examples from renal pathology are covered, with an emphasis on renal transplant pathology.

Deep learning powers cancer diagnosis in digital pathology

Technological innovation has accelerated the pathological diagnostic process for cancer, especially in digitizing histopathology slides and incorporating deep learning-based approaches to mine the subvisual morphometric phenotypes for improving pathology diagnosis. In this perspective paper, we provide an overview on major deep learning approaches for digital pathology and discuss challenges and opportunities of such approaches to aid cancer diagnosis in digital pathology. In particular, the emerging graph neural network may further improve the performance and interpretability of deep learning in digital pathology.

Deep Learning-Based Image Classification in Differentiating Tufted Astrocytes, Astrocytic Plaques, and Neuritic Plaques

This study aimed to develop a deep learning-based image classification model that can differentiate tufted astrocytes (TA), astrocytic plaques (AP), and neuritic plaques (NP) based on images of tissue sections stained with phospho-tau immunohistochemistry. Phospho-tau-immunostained slides from the motor cortex were scanned at 20× magnification. An automated deep learning platform, Google AutoML, was used to create a model for distinguishing TA in progressive supranuclear palsy (PSP) from AP in corticobasal degeneration (CBD) and NP in Alzheimer disease (AD). A total of 1500 images of representative tau lesions were captured from 35 PSP, 27 CBD, and 33 AD patients. Of those, 1332 images were used for training, and 168 images for cross-validation. We tested the model using 100 additional test images taken from 20 patients of each disease. In cross-validation, precision and recall for each individual lesion type were 100% and 98.0% for TA, 98.5% and 98.5% for AP, and 98.0% and 100% for NP, respectively. In a test set, all images of TA and NP were correctly predicted. Only eleven images of AP were predicted to be TA or NP. Our data indicate the potential usefulness of deep learning-based image classification methods to assist in differential diagnosis of tauopathies.

Introduction to Artificial Intelligence and Machine Learning for Pathology

CONTEXT.—

Recent developments in machine learning have stimulated intense interest in software that may augment or replace human experts. Machine learning may impact pathology practice by offering new capabilities in analysis, interpretation, and outcomes prediction using images and other data. The principles of operation and management of machine learning systems are unfamiliar to pathologists, who anticipate a need for additional education to be effective as expert users and managers of the new tools.

OBJECTIVE.—

To provide a background on machine learning for practicing pathologists, including an overview of algorithms, model development, and performance evaluation; to examine the current status of machine learning in pathology and consider possible roles and requirements for pathologists in local deployment and management of machine learning systems; and to highlight existing challenges and gaps in deployment methodology and regulation.

DATA SOURCES.—

Sources include the biomedical and engineering literature, white papers from professional organizations, government reports, electronic resources, and authors' experience in machine learning. References were chosen when possible for accessibility to practicing pathologists without specialized training in mathematics, statistics, or software development.

CONCLUSIONS.—

Machine learning offers an array of techniques that in recent published results show substantial promise. Data suggest that human experts working with machine learning tools outperform humans or machines separately, but the optimal form for this combination in pathology has not been established. Significant questions related to the generalizability of machine learning systems, local site verification, and performance monitoring remain to be resolved before a consensus on best practices and a regulatory environment can be established.

Generalizability of Deep Learning System for the Pathologic Diagnosis of Various Cancers

The deep learning (DL)-based approaches in tumor pathology help to overcome the limitations of subjective visual examination from pathologists and improve diagnostic accuracy and objectivity. However, it is unclear how a DL system trained to discriminate normal/tumor tissues in a specific cancer could perform on other tumor types. Herein, we cross-validated the DL-based normal/tumor classifiers separately trained on the tissue slides of cancers from bladder, lung, colon and rectum, stomach, bile duct, and liver. Furthermore, we compared the differences between the classifiers trained on the frozen or formalin-fixed paraffin-embedded (FFPE) tissues. The Area under the curve (AUC) for the receiver operating characteristic (ROC) curve ranged from 0.982 to 0.999 when the tissues were analyzed by the classifiers trained on the same tissue preparation modalities and cancer types. However, the AUCs could drop to 0.476 and 0.439 when the classifiers trained for different tissue modalities and cancer types were applied. Overall, the optimal performance could be achieved only when the tissue slides were analyzed by the classifiers trained on the same preparation modalities and cancer types.

Development and Validation of a Deep Learning Model to Quantify Glomerulosclerosis in Kidney Biopsy Specimens

IMPORTANCE

A chronic shortage of donor kidneys is compounded by a high discard rate, and this rate is directly associated with biopsy specimen evaluation, which shows poor reproducibility among pathologists. A deep learning algorithm for measuring percent global glomerulosclerosis (an important predictor of outcome) on images of kidney biopsy specimens could enable pathologists to more reproducibly and accurately quantify percent global glomerulosclerosis, potentially saving organs that would have been discarded.

OBJECTIVE

To compare the performances of pathologists with a deep learning model on quantification of percent global glomerulosclerosis in whole-slide images of donor kidney biopsy specimens, and to determine the potential benefit of a deep learning model on organ discard rates.

DESIGN, SETTING, AND PARTICIPANTS

This prognostic study used whole-slide images acquired from 98 hematoxylin-eosin-stained frozen and 51 permanent donor biopsy specimen sections retrieved from 83 kidneys. Serial annotation by 3 board-certified pathologists served as ground truth for model training and for evaluation. Images of kidney biopsy specimens were obtained from the Washington University database (retrieved between June 2015 and June 2017). Cases were selected randomly from a database of more than 1000 cases to include biopsy specimens representing an equitable distribution within 0% to 5%, 6% to 10%, 11% to 15%, 16% to 20%, and more than 20% global glomerulosclerosis.

MAIN OUTCOMES AND MEASURES

Correlation coefficient (r) and root-mean-square error (RMSE) with respect to annotations were computed for cross-validated model predictions and on-call pathologists' estimates of percent global glomerulosclerosis when using individual and pooled slide results. Data were analyzed from March 2018 to August 2020.

RESULTS

The cross-validated model results of section images retrieved from 83 donor kidneys showed higher correlation with annotations (r = 0.916; 95% CI, 0.886-0.939) than on-call pathologists (r = 0.884; 95% CI, 0.825-0.923) that was enhanced when pooling glomeruli counts from multiple levels (r = 0.933; 95% CI, 0.898-0.956). Model prediction error for single levels (RMSE, 5.631; 95% CI, 4.735-6.517) was 14% lower than on-call pathologists (RMSE, 6.523; 95% CI, 5.191-7.783), improving to 22% with multiple levels (RMSE, 5.094; 95% CI, 3.972-6.301). The model decreased the likelihood of unnecessary organ discard by 37% compared with pathologists.

CONCLUSIONS AND RELEVANCE

The findings of this prognostic study suggest that this deep learning model provided a scalable and robust method to quantify percent global glomerulosclerosis in whole-slide images of donor kidneys. The model performance improved by analyzing multiple levels of a section, surpassing the capacity of pathologists in the time-sensitive setting of examining donor biopsy specimens. The results indicate the potential of a deep learning model to prevent erroneous donor organ discard.

Application of Artificial Intelligence in Medicine: An Overview

Artificial intelligence (AI) is a new technical discipline that uses computer technology to research and develop the theory, method, technique, and application system for the simulation, extension, and expansion of human intelligence. With the assistance of new AI technology, the traditional medical environment has changed a lot. For example, a patient's diagnosis based on radiological, pathological, endoscopic, ultrasonographic, and biochemical examinations has been effectively promoted with a higher accuracy and a lower human workload. The medical treatments during the perioperative period, including the preoperative preparation, surgical period, and postoperative recovery period, have been significantly enhanced with better surgical effects. In addition, AI technology has also played a crucial role in medical drug production, medical management, and medical education, taking them into a new direction. The purpose of this review is to introduce the application of AI in medicine and to provide an outlook of future trends.

Machine learning versus conventional clinical methods in guiding management of heart failure patients-a systematic review

Machine learning (ML) algorithms "learn" information directly from data, and their performance improves proportionally with the number of high-quality samples. The aim of our systematic review is to present the state of the art regarding the implementation of ML techniques in the management of heart failure (HF) patients. We manually searched MEDLINE and Cochrane databases as well the reference lists of the relevant review studies and included studies. Our search retrieved 122 relevant studies. These studies mainly refer to (a) the role of ML in the classification of HF patients into distinct categories which may require a different treatment strategy, (b) discrimination of HF patients from the healthy population or other diseases, (c) prediction of HF outcomes, (d) identification of HF patients from electronic records and identification of HF patients with similar characteristics who may benefit form a similar treatment strategy, (e) supporting the extraction of important data from clinical notes, and (f) prediction of outcomes in HF populations with implantable devices (left ventricular assist device, cardiac resynchronization therapy). We concluded that ML techniques may play an important role for the efficient construction of methodologies for diagnosis, management, and prediction of outcomes in HF patients.

Artificial intelligence technologies empowering identification of novel diagnostic molecular markers in gastric cancer

In recent clinical practice the molecular diagnostics have been significantly empowered and upgraded by the use of Artificial Intelligence and its assisted technologies. The use of Machine leaning and Deep Learning Neural network architectures have brought in a new dimension in clinical oncological research and development. These algorithm based software system with enhanced digital image analysis have emerged into a new branch of digital pathology and contributed immensely towards precision medicine and personal diagnostics. In India, gastric cancer is one of the most common cancers in males as well as in females. Various molecular biomarkers are associated with gastric cancer development and progression of which HER2 protein, a transmembrane tyrosine kinase (TK) receptor of epidermal growth factor receptors (EGFRs) family is of prime importance. The EGF receptor expression in gastric cancer is linked with its prognostics and theragnostics. These expressions are assessed by immunohistochemistry (IHC) and molecular techniques such as Fluorescence in-situ hybridization (FISH), as per recommendations for HER2 targeted immunotherapy. These have motivated the software giants like Google Inc. to produce innovative state of art technologies mimicking human traits such as learning and problem solving skill sets. This field is still under development and is slowly evolving and capturing global importance in recent times. A literature search on PubMed was performed to access updated information for this manuscript.

Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics

Purpose of Review

Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images.

Recent Findings

Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond.

Summary

Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.

The Value of Artificial Intelligence in Laboratory Medicine

OBJECTIVES

As laboratory medicine continues to undergo digitalization and automation, clinical laboratorians will likely be confronted with the challenges associated with artificial intelligence (AI). Understanding what AI is good for, how to evaluate it, what are its limitations, and how it can be implemented are not well understood. With a survey, we aimed to evaluate the thoughts of stakeholders in laboratory medicine on the value of AI in the diagnostics space and identify anticipated challenges and solutions to introducing AI.

METHODS

We conducted a web-based survey on the use of AI with participants from Roche's Strategic Advisory Network that included key stakeholders in laboratory medicine.

RESULTS

In total, 128 of 302 stakeholders responded to the survey. Most of the participants were medical practitioners (26%) or laboratory managers (22%). AI is currently used in the organizations of 15.6%, while 66.4% felt they might use it in the future. Most had an unsure attitude on what they would need to adopt AI in the diagnostics space. High investment costs, lack of proven clinical benefits, number of decision makers, and privacy concerns were identified as barriers to adoption. Education in the value of AI, streamlined implementation and integration into existing workflows, and research to prove clinical utility were identified as solutions needed to mainstream AI in laboratory medicine.

CONCLUSIONS

This survey demonstrates that specific knowledge of AI in the medical community is poor and that AI education is much needed. One strategy could be to implement new AI tools alongside existing tools.

Osteoid Metaplasia in Femoral Artery Plaques Is Associated With the Clinical Severity of Lower Extremity Artery Disease in Men

Lamellar metaplastic bone, osteoid metaplasia (OM), is found in atherosclerotic plaques, especially in the femoral arteries. In the carotid arteries, OM has been documented to be associated with plaque stability. This study investigated the clinical impact of OM load in femoral artery plaques of patients with lower extremity artery disease (LEAD) by using a deep learning-based image analysis algorithm. Plaques from 90 patients undergoing endarterectomy of the common femoral artery were collected and analyzed. After decalcification and fixation, 4-μm-thick longitudinal sections were stained with hematoxylin and eosin, digitized, and uploaded as whole-slide images on a cloud-based platform. A deep learning-based image analysis algorithm was trained to analyze the area percentage of OM in whole-slide images. Clinical data were extracted from electronic patient records, and the association with OM was analyzed. Fifty-one (56.7%) sections had OM. Females with diabetes had a higher area percentage of OM than females without diabetes. In male patients, the area percentage of OM inversely correlated with toe pressure and was significantly associated with severe symptoms of LEAD including rest pain, ulcer, or gangrene. According to our results, OM is a typical feature of femoral artery plaques and can be quantified using a deep learning-based image analysis method. The association of OM load with clinical features of LEAD appears to differ between male and female patients, highlighting the need for a gender-specific approach in the study of the mechanisms of atherosclerotic disease. In addition, the role of plaque characteristics in the treatment of atherosclerotic lesions warrants further consideration in the future.

Using Deep Learning Artificial Intelligence Algorithms to Verify N-Nitroso-N-Methylurea and Urethane Positive Control Proliferative Changes in Tg-RasH2 Mouse Carcinogenicity Studies

In Tg-rasH2 carcinogenicity mouse models, a positive control group is treated with a carcinogen such as urethane or N-nitroso-N-methylurea to test study validity based on the presence of the expected proliferative lesions in the transgenic mice. We hypothesized that artificial intelligence-based deep learning (DL) could provide decision support for the toxicologic pathologist by screening for the proliferative changes, verifying the expected pattern for the positive control groups. Whole slide images (WSIs) of the lungs, thymus, and stomach from positive control groups were used for supervised training of a convolutional neural network (CNN). A single pathologist annotated WSIs of normal and abnormal tissue regions for training the CNN-based supervised classifier using INHAND criteria. The algorithm was evaluated using a subset of tissue regions that were not used for training and then additional tissues were evaluated blindly by 2 independent pathologists. A binary output (proliferative classes present or not) from the pathologists was compared to that of the CNN classifier. The CNN model grouped proliferative lesion positive and negative animals at high concordance with the pathologists. This process simulated a workflow for review of these studies, whereby a DL algorithm could provide decision support for the pathologists in a nonclinical study.

A 'Real-Life' Experience on Automated Digital Image Analysis of FGFR2 Immunohistochemistry in Breast Cancer

We present here an assessment of a 'real-life' value of automated machine learning algorithm (AI) for examination of immunohistochemistry for fibroblast growth factor receptor-2 (FGFR2) in breast cancer (BC). Expression of FGFR2 in BC (n = 315) measured using a certified 3DHistech CaseViewer/QuantCenter software 2.3.0. was compared to the manual pathologic assessment in digital slides (PA). Results revealed: (i) substantial interrater agreement between AI and PA for dichotomized evaluation (Cohen's kappa = 0.61); (ii) strong correlation between AI and PA H-scores (Spearman r = 0.85, p < 0.001); (iii) a small constant error and a significant proportional error (Passing-Bablok regression y = 0.51 × X + 29.9, p < 0.001); (iv) discrepancies in H-score in cases of extreme (strongest/weakest) or heterogeneous FGFR2 expression and poor tissue quality. The time of AI was significantly longer (568 h) than that of the pathologist (32 h). This study shows that the described commercial machine learning algorithm can reliably execute a routine pathologic assessment, however, in some instances, human expertise is essential.

Current Trends in Artificial Intelligence Application for Endourology and Robotic Surgery

With the advent of electronic medical records and digitalization of health care over the past 2 decades, artificial intelligence (AI) has emerged as an enabling tool to manage complex datasets and deliver streamlined data-driven patient care. AI algorithms have the ability to extract meaningful signal from complex datasets through an iterative process akin to human learning. Through advancements over the past decade in deep learning, AI-driven innovations have accelerated applications in health care. Herein, the authors explore the development of these emerging AI technologies, focusing on the application of AI to endourology and robotic surgery.

The emergence of new trends in clinical laboratory diagnosis

Diagnostic processes typically rely on traditional and laborious methods, that are prone to human error, resulting in frequent misdiagnosis of diseases. Computational approaches are being increasingly used for more precise diagnosis of the clinical pathology, diagnosis of genetic and microbial diseases, and analysis of clinical chemistry data. These approaches are progressively used for improving the reliability of testing, resulting in reduced diagnostic errors. Artificial intelligence (AI)-based computational approaches mostly rely on training sets obtained from patient data stored in clinical databases. However, the use of AI is associated with several ethical issues, including patient privacy and data ownership. The capacity of AI-based mathematical models to interpret complex clinical data frequently leads to data bias and reporting of erroneous results based on patient data. In order to improve the reliability of computational approaches in clinical diagnostics, strategies to reduce data bias and analyzing real-life patient data need to be further refined.

Digital Versus Optical Diagnosis of Follicular Patterned Thyroid Lesions

OBJECTIVES

To study the concordance between pathologists in the diagnosis of follicular patterned thyroid lesions using both digital and conventional optical settings.

MATERIAL AND METHODS

Five pathologists reviewed 50 hematoxylin and eosin-stained slides of follicular patterned thyroid lesions using both digital (the D-Sight 2.0 scanner and navigator viewer) and conventional optical instruments with washout interval time.

RESULTS

The mean concordance rate with the ground truth (GT) was similar between conventional optical and digital observation (83.2 and 85.2%, respectively). The most frequent reason for diagnostic discordance with GT on both systems was the evaluation of nuclear features (69.1% for conventional optical observation and 59.4% for digital observation). The intraobserver diagnostic concordance mean was 86.8%. Time for digital observation (mean time per case = 2.9 ± 0.8 min) was higher than that for conventional optical observation (mean time per case = 2.0 ± 0.7 min). Interobserver correlation of measurements was higher in the digital observation than the conventional optical observation.

CONCLUSION

Conventional optical and digital observation settings showed a comparable accuracy for the diagnosis of follicular patterned thyroid nodules, as well as substantial intraobserver agreement and a significant improvement in the reproducibility of the measurements that support the use of digital diagnosis in thyroid pathology. The origins underlying the variability of the diagnosis were the same in both conventional optical microscopy and digital pathology systems.

Prediction of clinically actionable genetic alterations from colorectal cancer histopathology images using deep learning

BACKGROUND

Identifying genetic mutations in cancer patients have been increasingly important because distinctive mutational patterns can be very informative to determine the optimal therapeutic strategy. Recent studies have shown that deep learning-based molecular cancer subtyping can be performed directly from the standard hematoxylin and eosin (H&E) sections in diverse tumors including colorectal cancers (CRCs). Since H&E-stained tissue slides are ubiquitously available, mutation prediction with the pathology images from cancers can be a time- and cost-effective complementary method for personalized treatment.

AIM

To predict the frequently occurring actionable mutations from the H&E-stained CRC whole-slide images (WSIs) with deep learning-based classifiers.

METHODS

A total of 629 CRC patients from The Cancer Genome Atlas (TCGA-COAD and TCGA-READ) and 142 CRC patients from Seoul St. Mary Hospital (SMH) were included. Based on the mutation frequency in TCGA and SMH datasets, we chose APC, KRAS, PIK3CA, SMAD4, and TP53 genes for the study. The classifiers were trained with 360 × 360 pixel patches of tissue images. The receiver operating characteristic (ROC) curves and area under the curves (AUCs) for all the classifiers were presented.

RESULTS

The AUCs for ROC curves ranged from 0.693 to 0.809 for the TCGA frozen WSIs and from 0.645 to 0.783 for the TCGA formalin-fixed paraffin-embedded WSIs. The prediction performance can be enhanced with the expansion of datasets. When the classifiers were trained with both TCGA and SMH data, the prediction performance was improved.

CONCLUSION

APC, KRAS, PIK3CA, SMAD4, and TP53 mutations can be predicted from H&E pathology images using deep learning-based classifiers, demonstrating the potential for deep learning-based mutation prediction in the CRC tissue slides.

PyHIST: A Histological Image Segmentation Tool

The development of increasingly sophisticated methods to acquire high-resolution images has led to the generation of large collections of biomedical imaging data, including images of tissues and organs. Many of the current machine learning methods that aim to extract biological knowledge from histopathological images require several data preprocessing stages, creating an overhead before the proper analysis. Here we present PyHIST (https://github.com/manuel-munoz-aguirre/PyHIST), an easy-to-use, open source whole slide histological image tissue segmentation and preprocessing command-line tool aimed at tile generation for machine learning applications. From a given input image, the PyHIST pipeline i) optionally rescales the image to a different resolution, ii) produces a mask for the input image which separates the background from the tissue, and iii) generates individual image tiles with tissue content.

Histopathology images are an essential tool to assess and quantify tissue composition and its relationship to disease. The digitization of slides and the decreasing costs of computation and data storage have fueled the development of new computational methods, especially in the field of machine learning. These methods seek to make use of the histopathological patterns encoded in these slides with the aim of aiding clinicians in healthcare decision-making, as well as researchers in tissue biology. However, in order to prepare digital slides for usage in machine learning applications, researchers usually need to develop custom scripts from scratch in order to reshape the image data in a way that is suitable to train a model, slowing down the development process. With PyHIST, we provide a toolbox for researchers that work in the intersection of machine learning, biology and histology to effortlessly preprocess whole slide images into image tiles in a standardized manner for usage in external applications.

Novel Pathologic Factors for Risk Stratification of Gastric "Indefinite for Dysplasia" Lesions

Methods

In total, 123 IND cases with final diagnoses of cancer (29.3%), high-grade dysplasia (6.5%), low-grade dysplasia (11.4%), and nonneoplasm (52.8%) were randomly divided into test set (n = 27) and validation set (n = 96). By the image analysis, size, pleomorphism, hyperchromasia, irregularity of nuclei, and ratios of structural atypia area (SAA) to total IND area were measured in the test set. Using the validation set, consensus meetings were held for the evaluation of pathologic factors that predict the final diagnosis.

Results

By image analysis, the only ratio of SAA to total IND area was associated with the final diagnosis (p < 0.001). In the consensus meeting for validation, the nuclear factors, except loss of nuclear polarity (p = 0.004–0.026), could not predict the final diagnosis. Conversely, most structural factors could predict the final diagnosis. In particular, SAA > 25% was the most powerful predictive factor. We proposed criteria of risk stratification by using SAA > 25%, loss of surface maturation (LOSM), and loss of nuclear polarity (LONP) (Malignancy rate; Category 0: SAA ≤ 25% without LOSM and LONP; 0%, Category 1: SAA ≤ 25% with any of LOSM or LONP; 15.2%–16.7%, Category 2: SAA > 25% without LOSM and LONP; 44.4%–50.0%, Category 3: SAA > 25% with any of LOSM or LONP 54.5%–55.6%).

Conclusions

Structural atypia was more helpful than nuclear atypia and SAA > 25% was the most powerful predictor for the diagnosis of INDs of the stomach. We propose shortening the follow-up period to six months for Category 1, endoscopic resection for Category 2 and 3, postresection follow-up periods of one year for Category 2, and six months for Category 3.

Artificial intelligence (AI) and big data in cancer and precision oncology

Artificial intelligence (AI) and machine learning have significantly influenced many facets of the healthcare sector. Advancement in technology has paved the way for analysis of big datasets in a cost- and time-effective manner. Clinical oncology and research are reaping the benefits of AI. The burden of cancer is a global phenomenon. Efforts to reduce mortality rates requires early diagnosis for effective therapeutic interventions. However, metastatic and recurrent cancers evolve and acquire drug resistance. It is imperative to detect novel biomarkers that induce drug resistance and identify therapeutic targets to enhance treatment regimes. The introduction of the next generation sequencing (NGS) platforms address these demands, has revolutionised the future of precision oncology. NGS offers several clinical applications that are important for risk predictor, early detection of disease, diagnosis by sequencing and medical imaging, accurate prognosis, biomarker identification and identification of therapeutic targets for novel drug discovery. NGS generates large datasets that demand specialised bioinformatics resources to analyse the data that is relevant and clinically significant. Through these applications of AI, cancer diagnostics and prognostic prediction are enhanced with NGS and medical imaging that delivers high resolution images. Regardless of the improvements in technology, AI has some challenges and limitations, and the clinical application of NGS remains to be validated. By continuing to enhance the progression of innovation and technology, the future of AI and precision oncology show great promise.

Automated feature detection in dental periapical radiographs by using deep learning

OBJECTIVE

The aim of this study was to investigate automated feature detection, segmentation, and quantification of common findings in periapical radiographs (PRs) by using deep learning (DL)-based computer vision techniques.

STUDY DESIGN

Caries, alveolar bone recession, and interradicular radiolucencies were labeled on 206 digital PRs by 3 specialists (2 oral pathologists and 1 endodontist). The PRs were divided into "Training and Validation" and "Test" data sets consisting of 176 and 30 PRs, respectively. Multiple transformations of image data were used as input to deep neural networks during training. Outcomes of existing and purpose-built DL architectures were compared to identify the most suitable architecture for automated analysis.

RESULTS

The U-Net architecture and its variant significantly outperformed Xnet and SegNet in all metrics. The overall best performing architecture on the validation data set was "U-Net+Densenet121" (mean intersection over union [mIoU] = 0.501; Dice coefficient = 0.569). Performance of all architectures degraded on the "Test" data set; "U-Net" delivered the best performance (mIoU = 0.402; Dice coefficient = 0.453). Interradicular radiolucencies were the most difficult to segment.

CONCLUSIONS

DL has potential for automated analysis of PRs but warrants further research. Among existing off-the-shelf architectures, U-Net and its variants delivered the best performance. Further performance gains can be obtained via purpose-built architectures and a larger multicentric cohort.

A bird's-eye view of deep learning in bioimage analysis

Deep learning of artificial neural networks has become the de facto standard approach to solving data analysis problems in virtually all fields of science and engineering. Also in biology and medicine, deep learning technologies are fundamentally transforming how we acquire, process, analyze, and interpret data, with potentially far-reaching consequences for healthcare. In this mini-review, we take a bird's-eye view at the past, present, and future developments of deep learning, starting from science at large, to biomedical imaging, and bioimage analysis in particular.

Prediction of outcome of treatment of acute severe ulcerative colitis using principal component analysis and artificial intelligence

Background and Aim

About 15% patients with acute severe ulcerative colitis (UC) fail to respond to medical treatment and may require colectomy. An early prediction of response may help the treating team and the patients and their family to prepare for alternative treatment options.

Methods

Data of 263 patients (mean age 37.0 ± 14.0-years, 176, 77% male) with acute severe UC admitted during a 12-year period were used to study predictors of response using univariate analysis, multivariate linear principal component analysis (PCA), and nonlinear artificial neural network (ANN).

Results

Of 263 patients, 231 (87.8%) responded to the initial medical treatment that included oral prednisolone (n = 14, 5.3%), intravenous (IV) hydrocortisone (n = 238, 90.5%), IV cyclosporine (n = 9, 3.4%), and inflixmab (n = 2, 0.7%), and 28 (10.6%) did not respond and the remaining 4 (1.5%) died, all of whom did were also nonresponders. Nonresponding patients had to stay longer in the hospital and died more often. On univariate analysis, the presence of complications, the need for use of cyclosporin, lower Hb, platelets, albumin, serum potassium, and higher C-reactive protein were predictors of nonresponse. Hb and albumin were strong predictive factors on both PCA and ANN. Though the nonlinear modeling using ANN had a good predictive accuracy for the response, its accuracy for predicting nonresponse was lower.

Conclusion

It is possible to predict the response to medical treatment in patients with UC using linear and nonlinear modeling technique. Serum albumin and Hb are strong predictive factors.

Introduction to digital pathology and computer-aided pathology

Digital pathology (DP) is no longer an unfamiliar term for pathologists, but it is still difficult for many pathologists to understand the engineering and mathematics concepts involved in DP. Computer-aided pathology (CAP) aids pathologists in diagnosis. However, some consider CAP a threat to the existence of pathologists and are skeptical of its clinical utility. Implementation of DP is very burdensome for pathologists because technical factors, impact on workflow, and information technology infrastructure must be considered. In this paper, various terms related to DP and computer-aided pathologic diagnosis are defined, current applications of DP are discussed, and various issues related to implementation of DP are outlined. The development of computer-aided pathologic diagnostic tools and their limitations are also discussed.