ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Information-incorporated sparse hierarchical cancer heterogeneity analysis

Cancer heterogeneity analysis is essential for precision medicine. Most of the existing heterogeneity analyses only consider a single type of data and ignore the possible sparsity of important features. In cancer clinical practice, it has been suggested that two types of data, pathological imaging and omics data, are commonly collected and can produce hierarchical heterogeneous structures, in which the refined sub-subgroup structure determined by omics features can be nested in the rough subgroup structure determined by the imaging features. Moreover, sparsity pursuit has extraordinary significance and is more challenging for heterogeneity analysis, because the important features may not be the same in different subgroups, which is ignored by the existing heterogeneity analyses. Fortunately, rich information from previous literature (for example, those deposited in PubMed) can be used to assist feature selection in the present study. Advancing from the existing analyses, in this study, we propose a novel sparse hierarchical heterogeneity analysis framework, which can integrate two types of features and incorporate prior knowledge to improve feature selection. The proposed approach has satisfactory statistical properties and competitive numerical performance. A TCGA real data analysis demonstrates the practical value of our approach in analyzing data heterogeneity and sparsity.

A new model using deep learning to predict recurrence after surgical resection of lung adenocarcinoma

This study aimed to develop a deep learning (DL) model for predicting the recurrence risk of lung adenocarcinoma (LUAD) based on its histopathological features. Clinicopathological data and whole slide images from 164 LUAD cases were collected and used to train DL models with an ImageNet pre-trained efficientnet-b2 architecture, densenet201, and resnet152. The models were trained to classify each image patch into high-risk or low-risk groups, and the case-level result was determined by multiple instance learning with final FC layer's features from a model from all patches. Analysis of the clinicopathological and genetic characteristics of the model-based risk group was performed. For predicting recurrence, the model had an area under the curve score of 0.763 with 0.750, 0.633 and 0.680 of sensitivity, specificity, and accuracy in the test set, respectively. High-risk cases for recurrence predicted by the model (HR group) were significantly associated with shorter recurrence-free survival and a higher stage (both, p < 0.001). The HR group was associated with specific histopathological features such as poorly differentiated components, complex glandular pattern components, tumor spread through air spaces, and a higher grade. In the HR group, pleural invasion, necrosis, and lymphatic invasion were more frequent, and the size of the invasion was larger (all, p < 0.001). Several genetic mutations, including TP53 (p = 0.007) mutations, were more frequently found in the HR group. The results of stages I-II were similar to those of the general cohort. DL-based model can predict the recurrence risk of LUAD and identify the presence of the TP53 gene mutation by analyzing histopathologic features.

PathoGraph: An Attention-Based Graph Neural Network Capable of Prognostication Based on CD276 Labelling of Malignant Glioma Cells

Computerized methods have been developed that allow quantitative morphological analyses of whole slide images (WSIs), e.g., of immunohistochemical stains. The latter are attractive because they can provide high-resolution data on the distribution of proteins in tissue. However, many immunohistochemical results are complex because the protein of interest occurs in multiple locations (in different cells and also extracellularly). We have recently established an artificial intelligence framework, PathoFusion which utilises a bifocal convolutional neural network (BCNN) model for detecting and counting arbitrarily definable morphological structures. We have now complemented this model by adding an attention-based graph neural network (abGCN) for the advanced analysis and automated interpretation of such data. Classical convolutional neural network (CNN) models suffer from limitations when handling global information. In contrast, our abGCN is capable of creating a graph representation of cellular detail from entire WSIs. This abGCN method combines attention learning with visualisation techniques that pinpoint the location of informative cells and highlight cell-cell interactions. We have analysed cellular labelling for CD276, a protein of great interest in cancer immunology and a potential marker of malignant glioma cells/putative glioma stem cells (GSCs). We are especially interested in the relationship between CD276 expression and prognosis. The graphs permit predicting individual patient survival on the basis of GSC community features. Our experiments lay a foundation for the use of the BCNN-abGCN tool chain in automated diagnostic prognostication using immunohistochemically labelled histological slides, but the method is essentially generic and potentially a widely usable tool in medical research and AI based healthcare applications.

Toward Explainable Artificial Intelligence for Precision Pathology

The rapid development of precision medicine in recent years has started to challenge diagnostic pathology with respect to its ability to analyze histological images and increasingly large molecular profiling data in a quantitative, integrative, and standardized way. Artificial intelligence (AI) and, more precisely, deep learning technologies have recently demonstrated the potential to facilitate complex data analysis tasks, including clinical, histological, and molecular data for disease classification; tissue biomarker quantification; and clinical outcome prediction. This review provides a general introduction to AI and describes recent developments with a focus on applications in diagnostic pathology and beyond. We explain limitations including the black-box character of conventional AI and describe solutions to make machine learning decisions more transparent with so-called explainable AI. The purpose of the review is to foster a mutual understanding of both the biomedical and the AI side. To that end, in addition to providing an overview of the relevant foundations in pathology and machine learning, we present worked-through examples for a better practical understanding of what AI can achieve and how it should be done.

Clustering on hierarchical heterogeneous data with prior pairwise relationships

BACKGROUND

Clustering is a fundamental problem in statistics and has broad applications in various areas. Traditional clustering methods treat features equally and ignore the potential structure brought by the characteristic difference of features. Especially in cancer diagnosis and treatment, several types of biological features are collected and analyzed together. Treating these features equally fails to identify the heterogeneity of both data structure and cancer itself, which leads to incompleteness and inefficacy of current anti-cancer therapies.

OBJECTIVES

In this paper, we propose a clustering framework based on hierarchical heterogeneous data with prior pairwise relationships. The proposed clustering method fully characterizes the difference of features and identifies potential hierarchical structure by rough and refined clusters.

RESULTS

The refined clustering further divides the clusters obtained by the rough clustering into different subtypes. Thus it provides a deeper insight of cancer that can not be detected by existing clustering methods. The proposed method is also flexible with prior information, additional pairwise relationships of samples can be incorporated to help to improve clustering performance. Finally, well-grounded statistical consistency properties of our proposed method are rigorously established, including the accurate estimation of parameters and determination of clustering structures.

CONCLUSIONS

Our proposed method achieves better clustering performance than other methods in simulation studies, and the clustering accuracy increases with prior information incorporated. Meaningful biological findings are obtained in the analysis of lung adenocarcinoma with clinical imaging data and omics data, showing that hierarchical structure produced by rough and refined clustering is necessary and reasonable.

Histopathology images-based deep learning prediction of prognosis and therapeutic response in small cell lung cancer

Small cell lung cancer (SCLC) is a highly aggressive subtype of lung cancer characterized by rapid tumor growth and early metastasis. Accurate prediction of prognosis and therapeutic response is crucial for optimizing treatment strategies and improving patient outcomes. In this study, we conducted a deep-learning analysis of Hematoxylin and Eosin (H&E) stained histopathological images using contrastive clustering and identified 50 intricate histomorphological phenotype clusters (HPCs) as pathomic features. We identified two of 50 HPCs with significant prognostic value and then integrated them into a pathomics signature (PathoSig) using the Cox regression model. PathoSig showed significant risk stratification for overall survival and disease-free survival and successfully identified patients who may benefit from postoperative or preoperative chemoradiotherapy. The predictive power of PathoSig was validated in independent multicenter cohorts. Furthermore, PathoSig can provide comprehensive prognostic information beyond the current TNM staging system and molecular subtyping. Overall, our study highlights the significant potential of utilizing histopathology images-based deep learning in improving prognostic predictions and evaluating therapeutic response in SCLC. PathoSig represents an effective tool that aids clinicians in making informed decisions and selecting personalized treatment strategies for SCLC patients.

Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology

Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.

Deep learning of cell spatial organizations identifies clinically relevant insights in tissue images

Recent advancements in tissue imaging techniques have facilitated the visualization and identification of various cell types within physiological and pathological contexts. Despite the emergence of cell-cell interaction studies, there is a lack of methods for evaluating individual spatial interactions. In this study, we introduce Ceograph, a cell spatial organization-based graph convolutional network designed to analyze cell spatial organization (for example,. the cell spatial distribution, morphology, proximity, and interactions) derived from pathology images. Ceograph identifies key cell spatial organization features by accurately predicting their influence on patient clinical outcomes. In patients with oral potentially malignant disorders, our model highlights reduced structural concordance and increased closeness in epithelial substrata as driving features for an elevated risk of malignant transformation. In lung cancer patients, Ceograph detects elongated tumor nuclei and diminished stroma-stroma closeness as biomarkers for insensitivity to EGFR tyrosine kinase inhibitors. With its potential to predict various clinical outcomes, Ceograph offers a deeper understanding of biological processes and supports the development of personalized therapeutic strategies.

AI/ML advances in non-small cell lung cancer biomarker discovery

Lung cancer is the leading cause of cancer deaths among both men and women, representing approximately 25% of cancer fatalities each year. The treatment landscape for non-small cell lung cancer (NSCLC) is rapidly evolving due to the progress made in biomarker-driven targeted therapies. While advancements in targeted treatments have improved survival rates for NSCLC patients with actionable biomarkers, long-term survival remains low, with an overall 5-year relative survival rate below 20%. Artificial intelligence/machine learning (AI/ML) algorithms have shown promise in biomarker discovery, yet NSCLC-specific studies capturing the clinical challenges targeted and emerging patterns identified using AI/ML approaches are lacking. Here, we employed a text-mining approach and identified 215 studies that reported potential biomarkers of NSCLC using AI/ML algorithms. We catalogued these studies with respect to BEST (Biomarkers, EndpointS, and other Tools) biomarker sub-types and summarized emerging patterns and trends in AI/ML-driven NSCLC biomarker discovery. We anticipate that our comprehensive review will contribute to the current understanding of AI/ML advances in NSCLC biomarker research and provide an important catalogue that may facilitate clinical adoption of AI/ML-derived biomarkers.

Enhancing diagnosis of Hirschsprung's disease using deep learning from histological sections of post pull-through specimens: preliminary results

PURPOSE

Accurate histological diagnosis in Hirschsprung disease (HD) is challenging, due to its complexity and potential for errors. In this study, we present an artificial intelligence (AI)-based method designed to identify ganglionic cells and hypertrophic nerves in HD histology.

METHODS

Formalin-fixed samples were used and an expert pathologist and a surgeon annotated these slides on a web-based platform, identifying ganglionic cells and nerves. Images were partitioned into square sections, augmented through data manipulation techniques and used to develop two distinct U-net models: one for detecting ganglionic cells and normal nerves; the other to recognise hypertrophic nerves.

RESULTS

The study included 108 annotated samples, resulting in 19,600 images after data augmentation and manually segmentation. Subsequently, 17,655 slides without target elements were excluded. The algorithm was trained using 1945 slides (930 for model 1 and 1015 for model 2) with 1556 slides used for training the supervised network and 389 for validation. The accuracy of model 1 was found to be 92.32%, while model 2 achieved an accuracy of 91.5%.

CONCLUSION

The AI-based U-net technique demonstrates robustness in detecting ganglion cells and nerves in HD. The deep learning approach has the potential to standardise and streamline HD diagnosis, benefiting patients and aiding in training of pathologists.

Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes for prediction of prognosis in resected colon cancer

Tumor-infiltrating lymphocytes (TIL) have been suggested as an important prognostic marker in colorectal cancer, but assessment usually requires additional tissue processing and interpretational efforts. The aim of this study is to assess the clinical significance of artificial intelligence (AI)-powered spatial TIL analysis using only a hematoxylin and eosin (H&E)-stained whole-slide image (WSI) for the prediction of prognosis in stage II-III colon cancer treated with surgery and adjuvant therapy. In this retrospective study, we used Lunit SCOPE IO, an AI-powered H&E WSI analyzer, to assess intratumoral TIL (iTIL) and tumor-related stromal TIL (sTIL) densities from WSIs of 289 patients. The patients with confirmed recurrences had significantly lower sTIL densities (mean sTIL density 630.2/mm2 in cases with confirmed recurrence vs. 1021.3/mm2 in no recurrence, p < 0.001). Additionally, significantly higher recurrence rates were observed in patients having sTIL or iTIL in the lower quartile groups. Risk groups defined as high-risk (both iTIL and sTIL in the lowest quartile groups), low-risk (sTIL higher than the median), or intermediate-risk (not high- or low-risk) were predictive of recurrence and were independently associated with clinical outcomes after adjusting for other clinical factors. AI-powered TIL analysis can provide prognostic information in stage II/III colon cancer in a practical manner.

Deep Learning for Predicting Effect of Neoadjuvant Therapies in Non-Small Cell Lung Carcinomas With Histologic Images

Neoadjuvant therapies are used for locally advanced non-small cell lung carcinomas, whereby pathologists histologically evaluate the effect using resected specimens. Major pathological response (MPR) has recently been used for treatment evaluation and as an economical survival surrogate; however, interobserver variability and poor reproducibility are often noted. The aim of this study was to develop a deep learning (DL) model to predict MPR from hematoxylin and eosin-stained tissue images and to validate its utility for clinical use. We collected data on 125 primary non-small cell lung carcinoma cases that were resected after neoadjuvant therapy. The cases were randomly divided into 55 for training/validation and 70 for testing. A total of 261 hematoxylin and eosin-stained slides were obtained from the maximum tumor beds, and whole slide images were prepared. We used a multiscale patch model that can adaptively weight multiple convolutional neural networks trained with different field-of-view images. We performed 3-fold cross-validation to evaluate the model. During testing, we compared the percentages of viable tumor evaluated by annotator pathologists (reviewed data), those evaluated by nonannotator pathologists (primary data), and those predicted by the DL-based model using 2-class confusion matrices and receiver operating characteristic curves and performed a survival analysis between MPR-achieved and non-MPR cases. In cross-validation, accuracy and mean F1 score were 0.859 and 0.805, respectively. During testing, accuracy and mean F1 score with reviewed data and those with primary data were 0.986, 0.985, 0.943, and 0.943, respectively. The areas under the receiver operating characteristic curve with reviewed and primary data were 0.999 and 0.978, respectively. The disease-free survival of MPR-achieved cases with reviewed and primary data was significantly better than that of the non-MPR cases (P<.001 and P=.001), and that predicted by the DL-based model was almost identical (P=.005). The DL model may support pathologist evaluations and can offer accurate determinations of MPR in patients.

Exploration and Enhancement of Classifiers in the Detection of Lung Cancer from Histopathological Images

Lung cancer is a prevalent malignancy that impacts individuals of all genders and is often diagnosed late due to delayed symptoms. To catch it early, researchers are developing algorithms to study lung cancer images. The primary objective of this work is to propose a novel approach for the detection of lung cancer using histopathological images. In this work, the histopathological images underwent preprocessing, followed by segmentation using a modified approach of KFCM-based segmentation and the segmented image intensity values were dimensionally reduced using Particle Swarm Optimization (PSO) and Grey Wolf Optimization (GWO). Algorithms such as KL Divergence and Invasive Weed Optimization (IWO) are used for feature selection. Seven different classifiers such as SVM, KNN, Random Forest, Decision Tree, Softmax Discriminant, Multilayer Perceptron, and BLDC were used to analyze and classify the images as benign or malignant. Results were compared using standard metrics, and kappa analysis assessed classifier agreement. The Decision Tree Classifier with GWO feature extraction achieved good accuracy of 85.01% without feature selection and hyperparameter tuning approaches. Furthermore, we present a methodology to enhance the accuracy of the classifiers by employing hyperparameter tuning algorithms based on Adam and RAdam. By combining features from GWO and IWO, and using the RAdam algorithm, the Decision Tree classifier achieves the commendable accuracy of 91.57%.

Lung Cancer Classification in Histopathology Images Using Multiresolution Efficient Nets

Histopathological images are very effective for investigating the status of various biological structures and diagnosing diseases like cancer. In addition, digital histopathology increases diagnosis precision and provides better image quality and more detail for the pathologist with multiple viewing options and team annotations. As a result of the benefits above, faster treatment is available, increasing therapy success rates and patient recovery and survival chances. However, the present manual examination of these images is tedious and time-consuming for pathologists. Therefore, reliable automated techniques are needed to effectively classify normal and malignant cancer images. This paper applied a deep learning approach, namely, EfficientNet and its variants from B0 to B7. We used different image resolutions for each model, from 224 × 224 pixels to 600 × 600 pixels. We also applied transfer learning and parameter tuning techniques to improve the results and overcome the overfitting problem. We collected the dataset from the Lung and Colon Cancer Histopathological Image LC25000 image dataset. The dataset acquisition consists of 25,000 histopathology images of five classes (lung adenocarcinoma, lung squamous cell carcinoma, benign lung tissue, colon adenocarcinoma, and colon benign tissue). Then, we performed preprocessing on the dataset to remove the noisy images and bring them into a standard format. The model's performance was evaluated in terms of classification accuracy and loss. We have achieved good accuracy results for all variants; however, the results of EfficientNetB2 stand excellent, with an accuracy of 97% for 260 × 260 pixels resolution images.

H&E image analysis pipeline for quantifying morphological features

Detecting cell types from histopathological images is essential for various digital pathology applications. However, large number of cells in whole-slide images (WSIs) necessitates automated analysis pipelines for efficient cell type detection. Herein, we present hematoxylin and eosin (H&E) Image Processing pipeline (HEIP) for automatied analysis of scanned H&E-stained slides. HEIP is a flexible and modular open-source software that performs preprocessing, instance segmentation, and nuclei feature extraction. To evaluate the performance of HEIP, we applied it to extract cell types from ovarian high-grade serous carcinoma (HGSC) patient WSIs. HEIP showed high precision in instance segmentation, particularly for neoplastic and epithelial cells. We also show that there is a significant correlation between genomic ploidy values and morphological features, such as major axis of the nucleus.

Deep Learning for Lung Cancer Diagnosis, Prognosis and Prediction Using Histological and Cytological Images: A Systematic Review

Lung cancer is one of the deadliest cancers worldwide, with a high incidence rate, especially in tobacco smokers. Lung cancer accurate diagnosis is based on distinct histological patterns combined with molecular data for personalized treatment. Precise lung cancer classification from a single H&E slide can be challenging for a pathologist, requiring most of the time additional histochemical and special immunohistochemical stains for the final pathology report. According to WHO, small biopsy and cytology specimens are the available materials for about 70% of lung cancer patients with advanced-stage unresectable disease. Thus, the limited available diagnostic material necessitates its optimal management and processing for the completion of diagnosis and predictive testing according to the published guidelines. During the new era of Digital Pathology, Deep Learning offers the potential for lung cancer interpretation to assist pathologists' routine practice. Herein, we systematically review the current Artificial Intelligence-based approaches using histological and cytological images of lung cancer. Most of the published literature centered on the distinction between lung adenocarcinoma, lung squamous cell carcinoma, and small cell lung carcinoma, reflecting the realistic pathologist's routine. Furthermore, several studies developed algorithms for lung adenocarcinoma predominant architectural pattern determination, prognosis prediction, mutational status characterization, and PD-L1 expression status estimation.

Deep Learning of Cell Spatial Organizations Identifies Clinically Relevant Insights in Tissue Images

Recent advancements in tissue imaging techniques have facilitated the visualization and identification of various cell types within physiological and pathological contexts. Despite the emergence of cell-cell interaction studies, there is a lack of methods for evaluating individual spatial interactions. In this study, we introduce Ceograph, a novel cell spatial organization-based graph convolutional network designed to analyze cell spatial organization (i.e. the cell spatial distribution, morphology, proximity, and interactions) derived from pathology images. Ceograph identifies key cell spatial organization features by accurately predicting their influence on patient clinical outcomes. In patients with oral potentially malignant disorders, our model highlights reduced structural concordance and increased closeness in epithelial substrata as driving features for an elevated risk of malignant transformation. In lung cancer patients, Ceograph detects elongated tumor nuclei and diminished stroma-stroma closeness as biomarkers for insensitivity to EGFR tyrosine kinase inhibitors. With its potential to predict various clinical outcomes, Ceograph offers a deeper understanding of biological processes and supports the development of personalized therapeutic strategies.

High Expression of Microtubule-associated Protein TBCB Predicts Adverse Outcome and Immunosuppression in Acute Myeloid Leukemia

Acute myeloid leukemia (AML) is a devastating blood cancer with high heterogeneity and ill-fated outcome. Despite numerous advances in AML treatment, the prognosis remains poor for a significant proportion of patients. Consequently, it is necessary to accurately and comprehensively identify biomarkers as soon as possible to enhance the efficacy of diagnosis, prognosis and treatment of AML. In this study, we aimed to identify prognostic markers of AML by analyzing the cohorts from TCGA-LAML database and GEO microarray datasets. Interestingly, the transcriptional level of microtubule-associated protein TBCB in AML patients was noticeably increased when compared with normal individuals, and this was verified in two independent cohorts (GSE9476 and GSE13159) and with our AML patients. Furthermore, univariate and multivariate regression analysis revealed that high TBCB expression was an independent poor prognostic factor for AML. GO and GSEA enrichment analysis hinted that immune-related signaling pathways were enriched in up-regulated DEGs between two populations separated by the median expression level of TBCB. By constructing a protein-protein interaction network, we obtained six hub genes, all of which are immune-related molecules, and their expression levels were positively linked to that of TBCB. In addition, the high expression of three hub genes was significantly associated with a poor prognosis in AML. Moreover, we found that the tumor microenvironment in AML with high TBCB expression tended to be infiltrated by NK cells, especially CD56^bright NK cells. The transcriptional levels of NK cell inhibitory receptors and their ligands were positively related to that of TBCB, and their high expression levels also predicted poor prognosis in AML. Notably, we found that the down-regulation of TBCB suppressed cell proliferation in AML cell lines by enhancing the apoptosis and cell cycle arrest. Finally, drug sensitivity prediction illustrated that cells with high TBCB expression were more responsive to ATRA and midostaurin but resistant to cytarabine, dasatinib, and imatinib. In conclusion, our findings shed light on the feasibility of TBCB as a potential predictor of poor outcome and to be an alternative target of treatment in AML.

SPARTIN: a Bayesian method for the quantification and characterization of cell type interactions in spatial pathology data

Introduction: The acquisition of high-resolution digital pathology imaging data has sparked the development of methods to extract context-specific features from such complex data. In the context of cancer, this has led to increased exploration of the tumor microenvironment with respect to the presence and spatial composition of immune cells. Spatial statistical modeling of the immune microenvironment may yield insights into the role played by the immune system in the natural development of cancer as well as downstream therapeutic interventions. Methods: In this paper, we present SPatial Analysis of paRtitioned Tumor-Immune imagiNg (SPARTIN), a Bayesian method for the spatial quantification of immune cell infiltration from pathology images. SPARTIN uses Bayesian point processes to characterize a novel measure of local tumor-immune cell interaction, Cell Type Interaction Probability (CTIP). CTIP allows rigorous incorporation of uncertainty and is highly interpretable, both within and across biopsies, and can be used to assess associations with genomic and clinical features. Results: Through simulations, we show SPARTIN can accurately distinguish various patterns of cellular interactions as compared to existing methods. Using SPARTIN, we characterized the local spatial immune cell infiltration within and across 335 melanoma biopsies and evaluated their association with genomic, phenotypic, and clinical outcomes. We found that CTIP was significantly (negatively) associated with deconvolved immune cell prevalence scores including CD8+ T-Cells and Natural Killer cells. Furthermore, average CTIP scores differed significantly across previously established transcriptomic classes and significantly associated with survival outcomes. Discussion: SPARTIN provides a general framework for investigating spatial cellular interactions in high-resolution digital histopathology imaging data and its associations with patient level characteristics. The results of our analysis have potential implications relevant to both treatment and prognosis in the context of Skin Cutaneous Melanoma. The R-package for SPARTIN is available at https://github.com/bayesrx/SPARTIN along with a visualization tool for the images and results at: https://nateosher.github.io/SPARTIN.

A review of open-source image analysis tools for mammalian cell culture: algorithms, features and implementations

Cell culture is undeniably important for multiple scientific applications, including pharmaceuticals, transplants, and cosmetics. However, cell culture involves multiple manual steps, such as regularly analyzing cell images for their health and morphology. Computer scientists have developed algorithms to automate cell imaging analysis, but they are not widely adopted by biologists, especially those lacking an interactive platform. To address the issue, we compile and review existing open-source cell image processing tools that provide interactive interfaces for management and prediction tasks. We highlight the prediction tools that can detect, segment, and track different mammalian cell morphologies across various image modalities and present a comparison of algorithms and unique features of these tools, whether they work locally or in the cloud. This would guide non-experts to determine which is best suited for their purposes and, developers to acknowledge what is worth further expansion. In addition, we provide a general discussion on potential implementations of the tools for a more extensive scope, which guides the reader to not restrict them to prediction tasks only. Finally, we conclude the article by stating new considerations for the development of interactive cell imaging tools and suggesting new directions for future research.

Digital Quantification of Tumor Cellularity as a Novel Prognostic Feature of Non-Small Cell Lung Carcinoma

Non-small cell lung carcinoma is currently staged based on the size and involvement of other structures. Tumor size may be a surrogate measure of the total number of tumor cells. A recently revised reporting system for adenocarcinoma incorporates high-risk histologic patterns, which may have increased cellular density. Modern digital image analysis tools can be utilized to automate the quantification of cells. In this study, we tested the hypothesis that tumor cellularity can be used as a novel prognostic tool for lung cancer. Digital slides from The Cancer Genome Atlas lung adenocarcinoma (ADC) data set (n = 213) and lung squamous cell carcinoma (SCC) data set (n = 90) were obtained and analyzed using QuPath. The number of tumor cells was normalized with the surface area of the tumor to provide a measure of tumor cell density. Tumor cellularity was calculated by multiplying the size of the tumor with the cell density. Major histologic patterns and grade were compared with the tumor density of the lung ADC and lung SCC cases. The overall and progression-free survival were compared between groups of high and low tumor cellularity. High-grade histologic patterns in the ADC and SCC cases were associated with greater tumor densities compared with low-grade patterns. Cases with lower tumor cellularity had improved overall and progression-free survival compared with cases with higher cellularity. These results support tumor cellularity as a novel prognostic tool for non-small cell lung carcinoma that considers tumor stage and grade elements.

Integration of artificial intelligence in lung cancer: Rise of the machine

The goal of oncology is to provide the longest possible survival outcomes with the therapeutics that are currently available without sacrificing patients' quality of life. In lung cancer, several data points over a patient's diagnostic and treatment course are relevant to optimizing outcomes in the form of precision medicine, and artificial intelligence (AI) provides the opportunity to use available data from molecular information to radiomics, in combination with patient and tumor characteristics, to help clinicians provide individualized care. In doing so, AI can help create models to identify cancer early in diagnosis and deliver tailored therapy on the basis of available information, both at the time of diagnosis and in real time as they are undergoing treatment. The purpose of this review is to summarize the current literature in AI specific to lung cancer and how it applies to the multidisciplinary team taking care of these complex patients.

Application of digital pathology and machine learning in the liver, kidney and lung diseases

The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.

Multimodal Lung Cancer Subtyping Using Deep Learning Neural Networks on Whole Slide Tissue Images and MALDI MSI

Artificial intelligence (AI) has shown potential for facilitating the detection and classification of tumors. In patients with non-small cell lung cancer, distinguishing between the most common subtypes, adenocarcinoma (ADC) and squamous cell carcinoma (SqCC), is crucial for the development of an effective treatment plan. This task, however, may still present challenges in clinical routine. We propose a two-modality, AI-based classification algorithm to detect and subtype tumor areas, which combines information from matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) data and digital microscopy whole slide images (WSIs) of lung tissue sections. The method consists of first detecting areas with high tumor cell content by performing a segmentation of the hematoxylin and eosin-stained (H&E-stained) WSIs, and subsequently classifying the tumor areas based on the corresponding MALDI MSI data. We trained the algorithm on six tissue microarrays (TMAs) with tumor samples from N = 232 patients and used 14 additional whole sections for validation and model selection. Classification accuracy was evaluated on a test dataset with another 16 whole sections. The algorithm accurately detected and classified tumor areas, yielding a test accuracy of 94.7% on spectrum level, and correctly classified 15 of 16 test sections. When an additional quality control criterion was introduced, a 100% test accuracy was achieved on sections that passed the quality control (14 of 16). The presented method provides a step further towards the inclusion of AI and MALDI MSI data into clinical routine and has the potential to reduce the pathologist's work load. A careful analysis of the results revealed specific challenges to be considered when training neural networks on data from lung cancer tissue.

Hierarchical cancer heterogeneity analysis based on histopathological imaging features

In cancer research, supervised heterogeneity analysis has important implications. Such analysis has been traditionally based on clinical/demographic/molecular variables. Recently, histopathological imaging features, which are generated as a byproduct of biopsy, have been shown as effective for modeling cancer outcomes, and a handful of supervised heterogeneity analysis has been conducted based on such features. There are two types of histopathological imaging features, which are extracted based on specific biological knowledge and using automated imaging processing software, respectively. Using both types of histopathological imaging features, our goal is to conduct the first supervised cancer heterogeneity analysis that satisfies a hierarchical structure. That is, the first type of imaging features defines a rough structure, and the second type defines a nested and more refined structure. A penalization approach is developed, which has been motivated by but differs significantly from penalized fusion and sparse group penalization. It has satisfactory statistical and numerical properties. In the analysis of lung adenocarcinoma data, it identifies a heterogeneity structure significantly different from the alternatives and has satisfactory prediction and stability performance.

An Open-Source AI Framework for the Analysis of Single Cells in Whole-Slide Images with a Note on CD276 in Glioblastoma

Routine examination of entire histological slides at cellular resolution poses a significant if not insurmountable challenge to human observers. However, high-resolution data such as the cellular distribution of proteins in tissues, e.g., those obtained following immunochemical staining, are highly desirable. Our present study extends the applicability of the PathoFusion framework to the cellular level. We illustrate our approach using the detection of CD276 immunoreactive cells in glioblastoma as an example. Following automatic identification by means of PathoFusion's bifocal convolutional neural network (BCNN) model, individual cells are automatically profiled and counted. Only discriminable cells selected through data filtering and thresholding were segmented for cell-level analysis. Subsequently, we converted the detection signals into the corresponding heatmaps visualizing the distribution of the detected cells in entire whole-slide images of adjacent H&E-stained sections using the Discrete Wavelet Transform (DWT). Our results demonstrate that PathoFusion is capable of autonomously detecting and counting individual immunochemically labelled cells with a high prediction performance of 0.992 AUC and 97.7% accuracy. The data can be used for whole-slide cross-modality analyses, e.g., relationships between immunochemical signals and anaplastic histological features. PathoFusion has the potential to be applied to additional problems that seek to correlate heterogeneous data streams and to serve as a clinically applicable, weakly supervised system for histological image analyses in (neuro)pathology.

The state of the art for artificial intelligence in lung digital pathology

Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial intelligence in clinical applications for lung cancer: diagnosis, treatment and prognosis

Artificial Intelligence (AI) is a branch of computer science that includes research in robotics, language recognition, image recognition, natural language processing, and expert systems. AI is poised to change medical practice, and oncology is not an exception to this trend. As the matter of fact, lung cancer has the highest morbidity and mortality worldwide. The leading cause is the complexity of associating early pulmonary nodules with neoplastic changes and numerous factors leading to strenuous treatment choice and poor prognosis. AI can effectively enhance the diagnostic efficiency of lung cancer while providing optimal treatment and evaluating prognosis, thereby reducing mortality. This review seeks to provide an overview of AI relevant to all the fields of lung cancer. We define the core concepts of AI and cover the basics of the functioning of natural language processing, image recognition, human-computer interaction and machine learning. We also discuss the most recent breakthroughs in AI technologies and their clinical application regarding diagnosis, treatment, and prognosis in lung cancer. Finally, we highlight the future challenges of AI in lung cancer and its impact on medical practice.

The pathological risk score: A new deep learning-based signature for predicting survival in cervical cancer

PURPOSE

To develop and validate a deep learning-based pathological risk score (RS) with an aim of predicting patients' prognosis to investigate the potential association between the information within the whole slide image (WSI) and cervical cancer prognosis.

METHODS

A total of 251 patients with the International Federation of Gynecology and Obstetrics (FIGO) Stage IA1-IIA2 cervical cancer who underwent surgery without any preoperative treatment were enrolled in this study. Both the clinical characteristics and WSI of each patient were collected. To construct a prognosis-associate RS, high-dimensional pathological features were extracted using a convolutional neural network with an autoencoder. With the score threshold selected by X-tile, Kaplan-Meier survival analysis was applied to verify the prediction performance of RS in overall survival (OS) and disease-free survival (DFS) in both the training and testing datasets, as well as different clinical subgroups.

RESULTS

For the OS and DFS prediction in the testing cohort, RS showed a Harrell's concordance index of higher than 0.700, while the areas under the curve (AUC) achieved up to 0.800 in the same cohort. Furthermore, Kaplan-Meier survival analysis demonstrated that RS was a potential prognostic factor, even in different datasets or subgroups. It could further distinguish the survival differences after clinicopathological risk stratification.

CONCLUSION

In the present study, we developed an effective signature in cervical cancer for prognosis prediction and patients' stratification in OS and DFS.

Bayesian hierarchical finite mixture of regression for histopathological imaging-based cancer data analysis

Cancer is heterogeneous, and for seemingly similar cancer patients, the associations between an outcome/phenotype and covariates can be different. To describe such differences, finite mixture of regression (FMR) and other modeling techniques have been developed. "Classic" FMR analysis has usually been based on clinical, demographic, and molecular variables. More recently, histopathological imaging data-which is a byproduct of biopsy and enjoys broader data availability and higher cost-effectiveness-has been increasingly used in cancer modeling, although it is noted that its application to cancer FMR analysis still remains limited. In this article, we further advance cancer FMR analysis based on histopathological imaging data. Significantly advancing from the existing analyses under heterogeneity and homogeneity, our goal is to simultaneously use two types of histopathological imaging features, which are extracted based on domain-specific biomedical knowledge and using automated signal processing software, respectively. A significant modeling/methodological advancement is that, to reflect the "increased resolution" of the second type of imaging features over the first type, we impose a hierarchy in the mixture structures. An effective and flexible Bayesian approach is proposed. Simulation shows its competitiveness over several alternatives. The TCGA lung cancer data is analyzed, and interesting heterogeneous structures different from using the alternatives are found. Overall, this study provides a new venue for FMR analysis for cancer and other complex diseases.

Lung Cancer Computational Biology and Resources

Comprehensive clinical, pathological, and molecular data, when appropriately integrated with advanced computational approaches, are transforming the way we characterize and study lung cancer. Clinically, cancer registry and publicly available historical clinical trial data enable retrospective analyses to examine how socioeconomic factors, patient demographics, and cancer characteristics affect treatment and outcome. Pathologically, digital pathology and artificial intelligence are revolutionizing histopathological image analyses, not only with improved efficiency and accuracy, but also by extracting additional information for prognostication and tumor microenvironment characterization. Genetically and molecularly, individual patient tumors and preclinical models of lung cancer are profiled by various high-throughput platforms to characterize the molecular properties and functional liabilities. The resulting multi-omics data sets and their interrogation facilitate both basic research mechanistic studies and translation of the findings into the clinic. In this review, we provide a list of resources and tools potentially valuable for lung cancer basic and translational research. Importantly, we point out pitfalls and caveats when performing computational analyses of these data sets and provide a vision of future computational biology developments that will aid lung cancer translational research.

A narrative review of artificial intelligence-assisted histopathologic diagnosis and decision-making for non-small cell lung cancer: achievements and limitations

Objective

To summarize the current evidence regarding the applications, workflow, and limitations of artificial intelligence (AI) in the management of patients pathologically-diagnosed with lung cancer.

Background

Lung cancer is one of the most common cancers and the leading cause of cancer-related deaths worldwide. AI technologies have been applied to daily medical workflow and have achieved an excellent performance in predicting histopathologic subtypes, analyzing gene mutation profiles, and assisting in clinical decision-making for lung cancer treatment. More advanced deep learning for classifying pathologic images with minimal human interactions has been developed in addition to the conventional machine learning scheme.

Methods

Studies were identified by searching databases, including PubMed, EMBASE, Web of Science, and Cochrane Library, up to February 2021 without language restrictions.

Conclusions

A number of studies have evaluated AI pipelines and confirmed that AI is robust and efficacious in lung cancer diagnosis and decision-making, demonstrating that AI models are a useful tool for assisting oncologists in health management. Although several limitations that pose an obstacle for the widespread use of AI schemes persist, the unceasing refinement of AI techniques is poised to overcome such problems. Thus, AI technology is a promising tool for use in diagnosing and managing lung cancer.

Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology

Recent advances in computational and algorithmic power are evolving the field of medical imaging rapidly. In cancer research, many new directions are sought to characterize patients with additional imaging features derived from radiology and pathology images. The emerging field of Computational Pathology targets the high-throughput extraction and analysis of the spatial distribution of cells from digital histopathology images. The associated morphological and architectural features allow researchers to quantify and characterize new imaging biomarkers for cancer diagnosis, prognosis, and treatment decisions. However, while the image feature space grows, exploration and analysis become more difficult and ineffective. There is a need for dedicated interfaces for interactive data manipulation and visual analysis of computational pathology and clinical data. For this purpose, we present IIComPath, a visual analytics approach that enables clinical researchers to formulate hypotheses and create computational pathology pipelines involving cohort construction, spatial analysis of image-derived features, and cohort analysis. We demonstrate our approach through use cases that investigate the prognostic value of current diagnostic features and new computational pathology biomarkers.

Disease type detection in lung and colon cancer images using the complement approach of inefficient sets

Lung and colon cancers are deadly diseases that can develop simultaneously in organs and adversely affect human life in some special cases. Although the frequency of simultaneous occurrence of these two types of cancer is unlikely, there is a high probability of metastasis between the two organs if not diagnosed early. Traditionally, specialists have to go through a lengthy and complicated process to examine histopathological images and diagnose cancer cases; yet, it is now possible to achieve this process faster with the available technological possibilities. In this study, artificial intelligence-supported model and optimization methods were used to realize the classification of lung and colon cancers' histopathological images. The used dataset has five classes of histopathological images consisting of two colon cancer classes and three lung cancer classes. In the proposed approach, the image classes were trained from scratch with the DarkNet-19 model, which is one of the deep learning models. In the feature set extracted from the DarkNet-19 model, selection of the inefficient features was performed by using Equilibrium and Manta Ray Foraging optimization algorithms. Then, the set containing the inefficient features was distinguished from the rest of the set features, creating an efficient feature set (complementary rule insets). The efficient features obtained by the two used optimization algorithms were combined and classified with the Support Vector Machine (SVM) method. The overall accuracy rate obtained in the classification process was 99.69%. Based on the outcomes of this study, it has been observed that using the complementary method together with some optimization methods improved the classification performance of the dataset.

HISTOPATHOLOGY IMAGE REGISTRATION BY INTEGRATED TEXTURE AND SPATIAL PROXIMITY BASED LANDMARK SELECTION AND MODIFICATION

Three-dimensional (3D) digital pathology has been emerging for next-generation tissue based cancer research. To enable such histopathology image volume analysis, serial histopathology slides need to be well aligned. In this paper, we propose a histopathology image registration fine tuning method with integrated landmark evaluations by texture and spatial proximity measures. Representative anatomical structures and image corner features are first detected as landmark candidates. Next, we identify strong and modify weak matched landmarks by leveraging image texture features and landmark spatial proximity measures. Both qualitative and quantitative results of extensive experiments demonstrate that our proposed method is robust and can further enhance registration accuracy of our previously registered image set by 31.15% (correlation), 4.88% (mutual information), and 41.02% (mean squared error), respectively. The promising experimental results suggest that our method can be used as a fine tuning module to further boost registration accuracy, a premise of histology spatial and morphology analysis in an information-lossless 3D tissue space for cancer research.

Deep Learning of Histopathology Images at the Single Cell Level

The tumor immune microenvironment (TIME) encompasses many heterogeneous cell types that engage in extensive crosstalk among the cancer, immune, and stromal components. The spatial organization of these different cell types in TIME could be used as biomarkers for predicting drug responses, prognosis and metastasis. Recently, deep learning approaches have been widely used for digital histopathology images for cancer diagnoses and prognoses. Furthermore, some recent approaches have attempted to integrate spatial and molecular omics data to better characterize the TIME. In this review we focus on machine learning-based digital histopathology image analysis methods for characterizing tumor ecosystem. In this review, we will consider three different scales of histopathological analyses that machine learning can operate within: whole slide image (WSI)-level, region of interest (ROI)-level, and cell-level. We will systematically review the various machine learning methods in these three scales with a focus on cell-level analysis. We will provide a perspective of workflow on generating cell-level training data sets using immunohistochemistry markers to "weakly-label" the cell types. We will describe some common steps in the workflow of preparing the data, as well as some limitations of this approach. Finally, we will discuss future opportunities of integrating molecular omics data with digital histopathology images for characterizing tumor ecosystem.

DeepRePath: Identifying the Prognostic Features of Early-Stage Lung Adenocarcinoma Using Multi-Scale Pathology Images and Deep Convolutional Neural Networks

Simple Summary

Pathology images are vital for understanding solid cancers. In this study, we created DeepRePath using multi-scale pathology images with two-channel deep learning to predict the prognosis of patients with early-stage lung adenocarcinoma (LUAD). DeepRePath demonstrated that it could predict the recurrence of early-stage LUAD with average area under the curve scores of 0.77 and 0.76 in cohort I and cohort II (external validation set), respectively. Pathological features found to be associated with a high probability of recurrence included tumor necrosis, discohesive tumor cells, and atypical nuclei. In conclusion, DeepRePath can improve the treatment modality for patients with early-stage LUAD through recurrence prediction.

Abstract

The prognosis of patients with lung adenocarcinoma (LUAD), especially early-stage LUAD, is dependent on clinicopathological features. However, its predictive utility is limited. In this study, we developed and trained a DeepRePath model based on a deep convolutional neural network (CNN) using multi-scale pathology images to predict the prognosis of patients with early-stage LUAD. DeepRePath was pre-trained with 1067 hematoxylin and eosin-stained whole-slide images of LUAD from the Cancer Genome Atlas. DeepRePath was further trained and validated using two separate CNNs and multi-scale pathology images of 393 resected lung cancer specimens from patients with stage I and II LUAD. Of the 393 patients, 95 patients developed recurrence after surgical resection. The DeepRePath model showed average area under the curve (AUC) scores of 0.77 and 0.76 in cohort I and cohort II (external validation set), respectively. Owing to low performance, DeepRePath cannot be used as an automated tool in a clinical setting. When gradient-weighted class activation mapping was used, DeepRePath indicated the association between atypical nuclei, discohesive tumor cells, and tumor necrosis in pathology images showing recurrence. Despite the limitations associated with a relatively small number of patients, the DeepRePath model based on CNNs with transfer learning could predict recurrence after the curative resection of early-stage LUAD using multi-scale pathology images.

Deep Learning for the Classification of Non-Hodgkin Lymphoma on Histopathological Images

Simple Summary

Histopathological examination of lymph node (LN) specimens allows the detection of hematological diseases. The identification and the classification of lymphoma, a blood cancer with a manifestation in LNs, are difficult and require many years of training, as well as additional expensive investigations. Today, artificial intelligence (AI) can be used to support the pathologist in identifying abnormalities in LN specimens. In this article, we trained and optimized an AI algorithm to automatically detect two common lymphoma subtypes that require different therapies using normal LN parenchyma as a control. The balanced accuracy in an independent test cohort was above 95%, which means that the vast majority of cases were classified correctly and only a few cases were misclassified. We applied specific methods to explain which parts of the image were important for the AI algorithm and to ensure a reliable result. Our study shows that classifications of lymphoma subtypes is possible with high accuracy. We think that routine histopathological applications for AI should be pursued.

Abstract

The diagnosis and the subtyping of non-Hodgkin lymphoma (NHL) are challenging and require expert knowledge, great experience, thorough morphological analysis, and often additional expensive immunohistological and molecular methods. As these requirements are not always available, supplemental methods supporting morphological-based decision making and potentially entity subtyping are required. Deep learning methods have been shown to classify histopathological images with high accuracy, but data on NHL subtyping are limited. After annotation of histopathological whole-slide images and image patch extraction, we trained and optimized an EfficientNet convolutional neuronal network algorithm on 84,139 image patches from 629 patients and evaluated its potential to classify tumor-free reference lymph nodes, nodal small lymphocytic lymphoma/chronic lymphocytic leukemia, and nodal diffuse large B-cell lymphoma. The optimized algorithm achieved an accuracy of 95.56% on an independent test set including 16,960 image patches from 125 patients after the application of quality controls. Automatic classification of NHL is possible with high accuracy using deep learning on histopathological images and routine diagnostic applications should be pursued.

Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study

BACKGROUND

Targeted therapy and immunotherapy put forward higher demands for accurate lung cancer classification, as well as benign versus malignant disease discrimination. Digital whole slide images (WSIs) witnessed the transition from traditional histopathology to computational approaches, arousing a hype of deep learning methods for histopathological analysis. We aimed at exploring the potential of deep learning models in the identification of lung cancer subtypes and cancer mimics from WSIs.

METHODS

We initially obtained 741 WSIs from the First Affiliated Hospital of Sun Yat-sen University (SYSUFH) for the deep learning model development, optimization, and verification. Additional 318 WSIs from SYSUFH, 212 from Shenzhen People's Hospital, and 422 from The Cancer Genome Atlas were further collected for multi-centre verification. EfficientNet-B5- and ResNet-50-based deep learning methods were developed and compared using the metrics of recall, precision, F1-score, and areas under the curve (AUCs). A threshold-based tumour-first aggregation approach was proposed and implemented for the label inferencing of WSIs with complex tissue components. Four pathologists of different levels from SYSUFH reviewed all the testing slides blindly, and the diagnosing results were used for quantitative comparisons with the best performing deep learning model.

RESULTS

We developed the first deep learning-based six-type classifier for histopathological WSI classification of lung adenocarcinoma, lung squamous cell carcinoma, small cell lung carcinoma, pulmonary tuberculosis, organizing pneumonia, and normal lung. The EfficientNet-B5-based model outperformed ResNet-50 and was selected as the backbone in the classifier. Tested on 1067 slides from four cohorts of different medical centres, AUCs of 0.970, 0.918, 0.963, and 0.978 were achieved, respectively. The classifier achieved high consistence to the ground truth and attending pathologists with high intraclass correlation coefficients over 0.873.

CONCLUSIONS

Multi-cohort testing demonstrated our six-type classifier achieved consistent and comparable performance to experienced pathologists and gained advantages over other existing computational methods. The visualization of prediction heatmap improved the model interpretability intuitively. The classifier with the threshold-based tumour-first label inferencing method exhibited excellent accuracy and feasibility in classifying lung cancers and confused nonneoplastic tissues, indicating that deep learning can resolve complex multi-class tissue classification that conforms to real-world histopathological scenarios.

Automatic ganglion cell detection for improving the efficiency and accuracy of hirschprung disease diagnosis

Histopathologic diagnosis of Hirschsprung's disease (HSCR) is time consuming and requires expertise. The use of artificial intelligence (AI) in digital pathology is actively researched and may improve the diagnosis of HSCR. The purpose of this research was to develop an algorithm capable of identifying ganglion cells in digital pathology slides and implement it as an assisting tool for the pathologist in the diagnosis of HSCR. Ninety five digital pathology slides were used for the construction and training of the algorithm. Fifty cases suspected for HSCR (727 slides) were used as a validation cohort. Image sets suspected to contain ganglion cells were chosen by the algorithm and then reviewed and scored by five pathologists, one HSCR expert and 4 non-experts. The algorithm was able to identify ganglion cells with 96% sensitivity and 99% specificity (in normal colon) as well as to correctly identify a case previously misdiagnosed as non-HSCR. The expert was able to achieve perfectly accurate diagnoses based solely on the images suggested by the algorithm, with over 95% time saved. Non-experts would require expert consultation in 20–58% of the cases to achieve similar results. The use of AI in the diagnosis of HSCR can greatly reduce the time and effort required for diagnosis and improve accuracy.

Molecular biomarkers in early stage lung cancer

Low dose computed tomography (LDCT) screening, together with the recent advances in targeted and immunotherapies, have shown to improve non-small cell lung cancer (NSCLC) survival. Furthermore, screening has increased the number of early stage-detected tumors, allowing for surgical resection and multimodality treatments when needed. The need for improved sensitivity and specificity of NSCLC screening has led to increased interest in combining clinical and radiological data with molecular data. The development of biomarkers is poised to refine inclusion criteria for LDCT screening programs. Biomarkers may also be useful to better characterize the risk of indeterminate nodules found in the course of screening or to refine prognosis and help in the management of screening detected tumors. The clinical implications of these biomarkers are still being investigated and whether or not biomarkers will be included in further decision-making algorithms in the context of screening and early lung cancer management still needs to be determined. However, it seems clear that there is much room for improvement even in early stage lung cancer disease-free survival (DFS) rates; thus, biomarkers may be the key to refine risk-stratification and treatment of these patients. Clinicians’ capacity to register, integrate, and analyze all the available data in both high risk individuals and early stage NSCLC patients will lead to a better understanding of the disease’s mechanisms, and will have a direct impact in diagnosis, treatment, and follow up of these patients. In this review, we aim to summarize all the available data regarding the role of biomarkers in LDCT screening and early stage NSCLC from a multidisciplinary perspective. We have highlighted clinical implications, the need to combine risk stratification, clinical data, radiomics, molecular information and artificial intelligence in order to improve clinical decision-making, especially regarding early diagnostics and adjuvant therapy. We also discuss current and future perspectives for biomarker implementation in routine clinical practice.

Reimagining T Staging Through Artificial Intelligence and Machine Learning Image Processing Approaches in Digital Pathology

Tumor stage and grade, visually assessed by pathologists from evaluation of pathology images in conjunction with radiographic imaging techniques, have been linked to outcome, progression, and survival for a number of cancers. The gold standard of staging in oncology has been the TNM (tumor-node-metastasis) staging system. Though histopathological grading has shown prognostic significance, it is subjective and limited by interobserver variability even among experienced surgical pathologists. Recently, artificial intelligence (AI) approaches have been applied to pathology images toward diagnostic-, prognostic-, and treatment prediction-related tasks in cancer. AI approaches have the potential to overcome the limitations of conventional TNM staging and tumor grading approaches, providing a direct prognostic prediction of disease outcome independent of tumor stage and grade. Broadly speaking, these AI approaches involve extracting patterns from images that are then compared against previously defined disease signatures. These patterns are typically categorized as either (1) handcrafted, which involve domain-inspired attributes, such as nuclear shape, or (2) deep learning (DL)-based representations, which tend to be more abstract. DL approaches have particularly gained considerable popularity because of the minimal domain knowledge needed for training, mostly only requiring annotated examples corresponding to the categories of interest. In this article, we discuss AI approaches for digital pathology, especially as they relate to disease prognosis, prediction of genomic and molecular alterations in the tumor, and prediction of treatment response in oncology. We also discuss some of the potential challenges with validation, interpretability, and reimbursement that must be addressed before widespread clinical deployment. The article concludes with a brief discussion of potential future opportunities in the field of AI for digital pathology and oncology.

A narrative review of digital pathology and artificial intelligence: focusing on lung cancer

The emergence of whole slide imaging technology allows for pathology diagnosis on a computer screen. The applications of digital pathology are expanding, from supporting remote institutes suffering from a shortage of pathologists to routine use in daily diagnosis including that of lung cancer. Through practice and research large archival databases of digital pathology images have been developed that will facilitate the development of artificial intelligence (AI) methods for image analysis. Currently, several AI applications have been reported in the field of lung cancer; these include the segmentation of carcinoma foci, detection of lymph node metastasis, counting of tumor cells, and prediction of gene mutations. Although the integration of AI algorithms into clinical practice remains a significant challenge, we have implemented tumor cell count for genetic analysis, a helpful application for routine use. Our experience suggests that pathologists often overestimate the contents of tumor cells, and the use of AI-based analysis increases the accuracy and makes the tasks less tedious. However, there are several difficulties encountered in the practical use of AI in clinical diagnosis. These include the lack of sufficient annotated data for the development and validation of AI systems, the explainability of black box AI models, such as those based on deep learning that offer the most promising performance, and the difficulty in defining the ground truth data for training and validation owing to inherent ambiguity in most applications. All of these together present significant challenges in the development and clinical translation of AI methods in the practice of pathology. Additional research on these problems will help in resolving the barriers to the clinical use of AI. Helping pathologists in developing knowledge of the working and limitations of AI will benefit the use of AI in both diagnostics and research.

Histopathological imaging features- versus molecular measurements-based cancer prognosis modeling

For lung and many other cancers, prognosis is essentially important, and extensive modeling has been carried out. Cancer is a genetic disease. In the past 2 decades, diverse molecular data (such as gene expressions and DNA mutations) have been analyzed in prognosis modeling. More recently, histopathological imaging data, which is a "byproduct" of biopsy, has been suggested as informative for prognosis. In this article, with the TCGA LUAD and LUSC data, we examine and directly compare modeling lung cancer overall survival using gene expressions versus histopathological imaging features. High-dimensional penalization methods are adopted for estimation and variable selection. Our findings include that gene expressions have slightly better prognostic performance, and that most of the gene expressions are weakly correlated imaging features. This study may provide additional insight into utilizing the two types of important data in cancer prognosis modeling and into lung cancer overall survival.