Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features

[Clinical Study of Artificial Intelligence-assisted Diagnosis System in Predicting the Invasive Subtypes of Early-stage Lung Adenocarcinoma Appearing as Pulmonary Nodules]

BACKGROUND

Lung cancer is the cancer with the highest mortality at home and abroad at present. The detection of lung nodules is a key step to reducing the mortality of lung cancer. Artificial intelligence-assisted diagnosis system presents as the state of the art in the area of nodule detection, differentiation between benign and malignant and diagnosis of invasive subtypes, however, a validation with clinical data is necessary for further application. Therefore, the aim of this study is to evaluate the effectiveness of artificial intelligence-assisted diagnosis system in predicting the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules.

METHODS

Clinical data of 223 patients with early-stage lung adenocarcinoma appearing as pulmonary nodules admitted to the Lanzhou University Second Hospital from January 1st, 2016 to December 31th, 2021 were retrospectively analyzed, which were divided into invasive adenocarcinoma group (n=170) and non-invasive adenocarcinoma group (n=53), and the non-invasive adenocarcinoma group was subdivided into minimally invasive adenocarcinoma group (n=31) and preinvasive lesions group (n=22). The malignant probability and imaging characteristics of each group were compared to analyze their predictive ability for the invasive subtypes of early-stage lung adenocarcinoma. The concordance between qualitative diagnostic results of artificial intelligence-assisted diagnosis of the invasive subtypes of early-stage lung adenocarcinoma and postoperative pathology was then analyzed.

RESULTS

In different invasive subtypes of early-stage lung adenocarcinoma, the mean CT value of pulmonary nodules (P<0.001), diameter (P<0.001), volume (P<0.001), malignant probability (P<0.001), pleural retraction sign (P<0.001), lobulation (P<0.001), spiculation (P<0.001) were significantly different. At the same time, it was also found that with the increased invasiveness of different invasive subtypes of early-stage lung adenocarcinoma, the proportion of dominant signs of each group gradually increased. On the issue of binary classification, the sensitivity, specificity, and area under the curve (AUC) values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 81.76%, 92.45% and 0.871 respectively. On the issue of three classification, the accuracy, recall rate, F1 score, and AUC values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 83.86%, 85.03%, 76.46% and 0.879 respectively.

CONCLUSIONS

Artificial intelligence-assisted diagnosis system could predict the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules, and has a certain predictive value. With the optimization of algorithms and the improvement of data, it may provide guidance for individualized treatment of patients.

Fast Segmentation of Metastatic Foci in H&E Whole-Slide Images for Breast Cancer Diagnosis

Breast cancer is the leading cause of death for women globally. In clinical practice, pathologists visually scan over enormous amounts of gigapixel microscopic tissue slide images, which is a tedious and challenging task. In breast cancer diagnosis, micro-metastases and especially isolated tumor cells are extremely difficult to detect and are easily neglected because tiny metastatic foci might be missed in visual examinations by medical doctors. However, the literature poorly explores the detection of isolated tumor cells, which could be recognized as a viable marker to determine the prognosis for T1NoMo breast cancer patients. To address these issues, we present a deep learning-based framework for efficient and robust lymph node metastasis segmentation in routinely used histopathological hematoxylin−eosin-stained (H−E) whole-slide images (WSI) in minutes, and a quantitative evaluation is conducted using 188 WSIs, containing 94 pairs of H−E-stained WSIs and immunohistochemical CK(AE1/AE3)-stained WSIs, which are used to produce a reliable and objective reference standard. The quantitative results demonstrate that the proposed method achieves 89.6% precision, 83.8% recall, 84.4% F1-score, and 74.9% mIoU, and that it performs significantly better than eight deep learning approaches, including two recently published models (v3_DCNN and Xception-65), and three variants of Deeplabv3+ with three different backbones, namely, U-Net, SegNet, and FCN, in precision, recall, F1-score, and mIoU (p<0.001). Importantly, the proposed system is shown to be capable of identifying tiny metastatic foci in challenging cases, for which there are high probabilities of misdiagnosis in visual inspection, while the baseline approaches tend to fail in detecting tiny metastatic foci. For computational time comparison, the proposed method takes 2.4 min for processing a WSI utilizing four NVIDIA Geforce GTX 1080Ti GPU cards and 9.6 min using a single NVIDIA Geforce GTX 1080Ti GPU card, and is notably faster than the baseline methods (4-times faster than U-Net and SegNet, 5-times faster than FCN, 2-times faster than the 3 different variants of Deeplabv3+, 1.4-times faster than v3_DCNN, and 41-times faster than Xception-65).

Deep SED-Net with interactive learning for multiple testicular cell types segmentation and cell composition analysis in mouse seminiferous tubules

The development of mouse spermatozoa is a continuous process from spermatogonia, spermatocytes, spermatids to mature sperm. Those developing germ cells (spermatogonia, spermatocyte, spermatids) together with supporting Sertoli cells are all enclosed inside seminiferous tubules of the testis, their identification is key to testis histology and pathology analysis. Automated segmentation of all these cells is a challenging task because of their dynamical changes in different stages. The accurate segmentation of testicular cells is critical in developing computerized spermatogenesis staging. In this paper, we present a novel segmentation model, SED-Net, which incorporates a Squeeze-and-Excitation (SE) module and a Dense unit. The SE module optimizes and obtains features from different channels, whereas the Dense unit uses fewer parameters to enhance the use of features. A human-in-the-loop strategy, named deep interactive learning, is developed to achieve better segmentation performance while reducing the workload of manual annotation and time consumption. Across a cohort of 274 seminiferous tubules from Stages VI to VIII, the SED-Net achieved a pixel accuracy of 0.930, a mean pixel accuracy of 0.866, a mean intersection over union of 0.710, and a frequency weighted intersection over union of 0.878, respectively, in terms of four types of testicular cell segmentation. There is no significant difference between manual annotated tubules and segmentation results by SED-Net in cell composition analysis for tubules from Stages VI to VIII. In addition, we performed cell composition analysis on 2346 segmented seminiferous tubule images from 12 segmented testicular section results. The results provided quantitation of cells of various testicular cell types across 12 stages. The rule reflects the cell variation tendency across 12 stages during development of mouse spermatozoa. The method could enable us to not only analyze cell morphology and staging during the development of mouse spermatozoa but also potientially could be applied to the study of reproductive diseases such as infertility.

Machine learning application in personalised lung cancer recurrence and survivability prediction

Machine learning is an important artificial intelligence technique that is widely applied in cancer diagnosis and detection. More recently, with the rise of personalised and precision medicine, there is a growing trend towards machine learning applications for prognosis prediction. However, to date, building reliable prediction models of cancer outcomes in everyday clinical practice is still a hurdle. In this work, we integrate genomic, clinical and demographic data of lung adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) patients from The Cancer Genome Atlas (TCGA) and introduce copy number variation (CNV) and mutation information of 15 selected genes to generate predictive models for recurrence and survivability. We compare the accuracy and benefits of three well-established machine learning algorithms: decision tree methods, neural networks and support vector machines. Although the accuracy of predictive models using the decision tree method has no significant advantage, the tree models reveal the most important predictors among genomic information (e.g. KRAS, EGFR, TP53), clinical status (e.g. TNM stage and radiotherapy) and demographics (e.g. age and gender) and how they influence the prediction of recurrence and survivability for both early stage LUAD and LUSC. The machine learning models have the potential to help clinicians to make personalised decisions on aspects such as follow-up timeline and to assist with personalised planning of future social care needs.

Progress on deep learning in digital pathology of breast cancer: a narrative review

BACKGROUND AND OBJECTIVE

Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology.

METHODS

A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied.

KEY CONTENT AND FINDINGS

DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches.

CONCLUSIONS

Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.

Cancer Chemotherapy via Natural Bioactive Compounds

BACKGROUND

Cancer-induced mortality is increasingly prevalent globally which skyrocketed the necessity to discover new/novel safe and effective anticancer drugs. Cancer is characterized by the continuous multiplication of cells in the human which is unable to control. Scientific research is drawing its attention towards naturally-derived bioactive compounds as they have fewer side effects compared to the current synthetic drugs used for chemotherapy.

OBJECTIVE

Drugs isolated from natural sources and their role in the manipulation of epigenetic markers in cancer are discussed briefly in this review article.

METHODS

With advancing medicinal plant biotechnology and microbiology in the past century, several anticancer phytomedicines were developed. Modern pharmacopeia contains at least 25% herbal-based remedy including clinically used anticancer drugs. These drugs mainly include the podophyllotoxin derivatives vinca alkaloids, curcumin, mistletoe plant extracts, taxanes, camptothecin, combretastatin, and others including colchicine, artesunate, homoharringtonine, ellipticine, roscovitine, maytanasin, tapsigargin,andbruceantin.

RESULTS

Compounds (psammaplin, didemnin, dolastin, ecteinascidin,and halichondrin) isolated from marine sources and animals such as microalgae, cyanobacteria, heterotrophic bacteria, invertebrates. They have been evaluated for their anticancer activity on cells and experimental animal models and used chemotherapy.Drug induced manipulation of epigenetic markers plays an important role in the treatment of cancer.

CONCLUSION

The development of a new drug from isolated bioactive compounds of plant sources has been a feasible way to lower the toxicity and increase their effectiveness against cancer. Potential anticancer therapeutic leads obtained from various ethnomedicinal plants, foods, marine, and microorganisms are showing effective yet realistically safe pharmacological activity. This review will highlight important plant-based bioactive compounds like curcumin, stilbenes, terpenes, other polyphenolic phyto-compounds, and structurally related families that are used to prevent/ ameliorate cancer. However, a contribution from all possible fields of science is still a prerequisite for discovering safe and effective anticancer drugs.

A Deep Learning Model for Prostate Adenocarcinoma Classification in Needle Biopsy Whole-Slide Images Using Transfer Learning

The histopathological diagnosis of prostate adenocarcinoma in needle biopsy specimens is of pivotal importance for determining optimum prostate cancer treatment. Since diagnosing a large number of cases containing 12 core biopsy specimens by pathologists using a microscope is time-consuming manual system and limited in terms of human resources, it is necessary to develop new techniques that can rapidly and accurately screen large numbers of histopathological prostate needle biopsy specimens. Computational pathology applications that can assist pathologists in detecting and classifying prostate adenocarcinoma from whole-slide images (WSIs) would be of great benefit for routine pathological practice. In this paper, we trained deep learning models capable of classifying needle biopsy WSIs into adenocarcinoma and benign (non-neoplastic) lesions. We evaluated the models on needle biopsy, transurethral resection of the prostate (TUR-P), and The Cancer Genome Atlas (TCGA) public dataset test sets, achieving an ROC-AUC up to 0.978 in needle biopsy test sets and up to 0.9873 in TCGA test sets for adenocarcinoma.

Automatic Recognition of Ragged Red Fibers in Muscle Biopsy from Patients with Mitochondrial Disorders

Mitochondrial dysfunction is considered to be a major cause of primary mitochondrial myopathy in children and adults, as reduced mitochondrial respiration and morphological changes such as ragged red fibers (RRFs) are observed in muscle biopsies. However, it is also possible to hypothesize the role of mitochondrial dysfunction in aging muscle or in secondary mitochondrial dysfunctions. The recognition of true histological patterns of mitochondrial myopathy can avoid unnecessary genetic investigations. The aim of our study was to develop and validate machine-learning methods for RRF detection in light microscopy images of skeletal muscle tissue. We used image sets of 489 color images captured from representative areas of Gomori’s trichrome-stained tissue retrieved from light microscopy images at a 20× magnification. We compared the performance of random forest, gradient boosting machine, and support vector machine classifiers. Our results suggested that the advent of scanning technologies, combined with the development of machine-learning models for image classification, make neuromuscular disorders’ automated diagnostic systems a concrete possibility.

An Aging-Related Gene Signature-Based Model for Risk Stratification and Prognosis Prediction in Lung Squamous Carcinoma

Aging is an inevitable process characterized by a decline in many physiological activities, and has been known as a significant risk factor for many kinds of malignancies, but there are few studies about aging-related genes (ARGs) in lung squamous carcinoma (LUSC). We designed this study to explore the prognostic value of ARGs and establish an ARG-based prognosis signature for LUSC patients. RNA-sequencing and corresponding clinicopathological data of patients with LUSC were downloaded from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO). The ARG risk signature was developed on the basis of results of LASSO and multivariate Cox analysis in the TCGA training dataset (n = 492). Furthermore, the GSE73403 dataset (n = 69) validated the prognostic performance of this ARG signature. Immunohistochemistry (IHC) staining was used to verify the expression of the ARGs in the signature. A five ARG-based signature, including A2M, CHEK2, ELN, FOS, and PLAU, was constructed in the TCGA dataset, and stratified patients into low- and high-risk groups with significantly different overall survival (OS) rates. The ARG risk score remained to be considered as an independent indicator of OS in the multivariate Cox regression model for LUSC patients. Then, a prognostic nomogram incorporating the ARG risk score with T-, N-, and M-classification was established. It achieved a good discriminative ability with a C-index of 0.628 (95% confidence interval [CI]: 0.586-0.671) in the TCGA cohort and 0.648 (95% CI: 0.535-0.762) in the GSE73403 dataset. Calibration curves displayed excellent agreement between the actual observations and the nomogram-predicted survival. The IHC staining discovered that these five ARGs were overexpression in LUSC tissues. Besides, the immune infiltration analysis in the TCGA cohort represented a distinctly differentiated infiltration of anti-tumor immune cells between the low- and high-risk groups. We identified a novel ARG-related prognostic signature, which may serve as a potential biomarker for individualized survival predictions and personalized therapeutic recommendation of anti-tumor immunity for patients with LUSC.

Bayesian hierarchical finite mixture of regression for histopathological imaging-based cancer data analysis

Cancer is heterogeneous, and for seemingly similar cancer patients, the associations between an outcome/phenotype and covariates can be different. To describe such differences, finite mixture of regression (FMR) and other modeling techniques have been developed. "Classic" FMR analysis has usually been based on clinical, demographic, and molecular variables. More recently, histopathological imaging data-which is a byproduct of biopsy and enjoys broader data availability and higher cost-effectiveness-has been increasingly used in cancer modeling, although it is noted that its application to cancer FMR analysis still remains limited. In this article, we further advance cancer FMR analysis based on histopathological imaging data. Significantly advancing from the existing analyses under heterogeneity and homogeneity, our goal is to simultaneously use two types of histopathological imaging features, which are extracted based on domain-specific biomedical knowledge and using automated signal processing software, respectively. A significant modeling/methodological advancement is that, to reflect the "increased resolution" of the second type of imaging features over the first type, we impose a hierarchy in the mixture structures. An effective and flexible Bayesian approach is proposed. Simulation shows its competitiveness over several alternatives. The TCGA lung cancer data is analyzed, and interesting heterogeneous structures different from using the alternatives are found. Overall, this study provides a new venue for FMR analysis for cancer and other complex diseases.

On the Automation of Radiomics-Based Identification and Characterization of NSCLC

Proper detection and accurate characterization of Non-Small Cell Lung Cancer (NSCLC) are an open challenge in the imaging field. Biomedical imaging is fundamental in lung cancer assessment and offers the possibility of calculating predictive biomarkers impacting patients' management. Within this context, radiomics, which consists of extracting quantitative features from digital images, shows encouraging results for clinical applications, but the sub-optimal standardization of the procedure and the lack of definitive results are still a concern in the field. For these reasons, this work proposes the design and development of LuCIFEx, a fully-automated pipeline for non-invasive in-vivo characterization of NSCLC, aiming to speed up the analysis process and enable an early diagnosis of the tumor.LuCIFEx pipeline relies on routinely acquired [18F]FDG-PET/CT images for the automatic segmentation of the cancer lesion, allowing the computation of accurate radiomic features, then employed for cancer characterization through Machine Learning algorithms. The proposed multi-stage segmentation process can identify the lesion with a mean accuracy of 94.2±5.0%. Finally, the proposed data analysis pipeline demonstrates the potential of PET/CT features for the automatic recognition of lung metastases and NSCLC histological subtypes, while highlighting the main current limitations of the radiomic approach.

Molecular Pathology of Lung Cancer

This overview of the molecular pathology of lung cancer includes a review of the most salient molecular alterations of the genome, transcriptome, and the epigenome. The insights provided by the growing use of next-generation sequencing (NGS) in lung cancer will be discussed, and interrelated concepts such as intertumor heterogeneity, intratumor heterogeneity, tumor mutational burden, and the advent of liquid biopsy will be explored. Moreover, this work describes how the evolving field of molecular pathology refines the understanding of different histologic phenotypes of non-small-cell lung cancer (NSCLC) and the underlying biology of small-cell lung cancer. This review will provide an appreciation for how ongoing scientific findings and technologic advances in molecular pathology are crucial for development of biomarkers, therapeutic agents, clinical trials, and ultimately improved patient care.

Detection of ovarian cancer via the spectral fingerprinting of quantum-defect-modified carbon nanotubes in serum by machine learning

Serum biomarkers are often insufficiently sensitive or specific to facilitate cancer screening or diagnostic testing. In ovarian cancer, the few established serum biomarkers are highly specific, yet insufficiently sensitive to detect early-stage disease and to impact the mortality rates of patients with this cancer. Here we show that a 'disease fingerprint' acquired via machine learning from the spectra of near-infrared fluorescence emissions of an array of carbon nanotubes functionalized with quantum defects detects high-grade serous ovarian carcinoma in serum samples from symptomatic individuals with 87% sensitivity at 98% specificity (compared with 84% sensitivity at 98% specificity for the current best clinical screening test, which uses measurements of cancer antigen 125 and transvaginal ultrasonography). We used 269 serum samples to train and validate several machine-learning classifiers for the discrimination of patients with ovarian cancer from those with other diseases and from healthy individuals. The predictive values of the best classifier could not be attained via known protein biomarkers, suggesting that the array of nanotube sensors responds to unidentified serum biomarkers.

Recent Advances of Deep Learning for Computational Histopathology: Principles and Applications

With the remarkable success of digital histopathology, we have witnessed a rapid expansion of the use of computational methods for the analysis of digital pathology and biopsy image patches. However, the unprecedented scale and heterogeneous patterns of histopathological images have presented critical computational bottlenecks requiring new computational histopathology tools. Recently, deep learning technology has been extremely successful in the field of computer vision, which has also boosted considerable interest in digital pathology applications. Deep learning and its extensions have opened several avenues to tackle many challenging histopathological image analysis problems including color normalization, image segmentation, and the diagnosis/prognosis of human cancers. In this paper, we provide a comprehensive up-to-date review of the deep learning methods for digital H&E-stained pathology image analysis. Specifically, we first describe recent literature that uses deep learning for color normalization, which is one essential research direction for H&E-stained histopathological image analysis. Followed by the discussion of color normalization, we review applications of the deep learning method for various H&E-stained image analysis tasks such as nuclei and tissue segmentation. We also summarize several key clinical studies that use deep learning for the diagnosis and prognosis of human cancers from H&E-stained histopathological images. Finally, online resources and open research problems on pathological image analysis are also provided in this review for the convenience of researchers who are interested in this exciting field.

A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images

Simple Summary

In this pilot study, we aimed to investigate the use of deep learning for the classification of whole-slide images of liquid-based cytology specimens into neoplastic and non-neoplastic. To do so, we used a large training and test sets. Overall, the model achieved good classification performance in classifying whole-slide images, demonstrating the promising potential use of such models for aiding the screening processes for cervical cancer.

Abstract

Liquid-based cytology (LBC) for cervical cancer screening is now more common than the conventional smears, which when digitised from glass slides into whole-slide images (WSIs), opens up the possibility of artificial intelligence (AI)-based automated image analysis. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to develop new computational techniques that can automatically and rapidly diagnose a large amount of specimens without delay, which would be of great benefit for clinical laboratories and hospitals. The goal of this study was to investigate the use of a deep learning model for the classification of WSIs of LBC specimens into neoplastic and non-neoplastic. To do so, we used a dataset of 1605 cervical WSIs. We evaluated the model on three test sets with a combined total of 1468 WSIs, achieving ROC AUCs for WSI diagnosis in the range of 0.89–0.96, demonstrating the promising potential use of such models for aiding screening processes.

Prediction of lymph node metastasis in early colorectal cancer based on histologic images by artificial intelligence

Risk evaluation of lymph node metastasis (LNM) for endoscopically resected submucosal invasive (T1) colorectal cancers (CRC) is critical for determining therapeutic strategies, but interobserver variability for histologic evaluation remains a major problem. To address this issue, we developed a machine-learning model for predicting LNM of T1 CRC without histologic assessment. A total of 783 consecutive T1 CRC cases were randomly split into 548 training and 235 validation cases. First, we trained convolutional neural networks (CNN) to extract cancer tile images from whole-slide images, then re-labeled these cancer tiles with LNM status for re-training. Statistical parameters of the tile images based on the probability of primary endpoints were assembled to predict LNM in cases with a random forest algorithm, and defined its predictive value as random forest score. We evaluated the performance of case-based prediction models for both training and validation datasets with area under the receiver operating characteristic curves (AUC). The accuracy for classifying cancer tiles was 0.980. Among cancer tiles, the accuracy for classifying tiles that were LNM-positive or LNM-negative was 0.740. The AUCs of the prediction models in the training and validation sets were 0.971 and 0.760, respectively. CNN judged the LNM probability by considering histologic tumor grade.

A new systematic model for analysis and a hybrid fuzzy multimodality model for lung tumor prediction

Lung cancer is one of the most commonly occurring diseases that ranked in the top of the present survey. Advancements in the medical field enable non-invasive methods of computerised diagnosis procedures and detection processes. Deep learning methods are already in evaluation by keeping the deep analysis on improving segmentation accuracy and prediction accuracy etc. The classification of tumour type depends on the quality of segmentation work and feature mappings. In this paper, we developed a robust model that classifies the types of tumours with improved accuracy but is also capable of detecting the early stages of cancer by detecting the unique hidden points of the image intensity in the lung images, etc. The system is comprised of a novel relative convergence technique for feature extraction technique to extract the infected area and its characteristic pixels to evaluate a unique feature mapping vector. The MSB feature mapping vectors are analysed with Hybrid Regress Fuzzy Net. The final result on whether a tumour is present in the CT image or normal depends on the three individual decisions made by the three algorithms mentioned. The accuracy of each algorithm is also considered for the probable decision-making. The performance measure of the entire proposed Hybrid Regress Net is evaluated through Accuracy, Precision, Recall and F1Score etc.

Identification of Tissue-Specific Gene Clusters Induced by DNA Demethylation in Lung Adenocarcinoma: More Than Germline Genes

Simple Summary

Loss of DNA methylation is often observed in human tumors, but how this epigenetic alteration impacts the transcriptome of cancer cells remains largely undefined. So far, DNA hypomethylation in tumors has been associated with aberrant activation of a germline-specific gene expression program. Here, we exploited transcriptomic and methylomic datasets of lung adenocarcinoma to investigate the possibility that other gene expression programs also become ectopically activated in hypomethylated tumors. Remarkably, we found that DNA hypomethylation in lung adenocarcinoma is associated with ectopic activation of not only germline-specific genes, but also gene clusters displaying specific expression in the gastrointestinal tract, or in stratified epithelia. Interestingly, expression of genes in this latter group was of prognostic value. Together, our study brings novel insight into the transcriptomic changes associated with DNA hypomethylation in tumors, and is an incentive to explore the value of hypomethylated DNA sequences as cancer biomarkers.

Abstract

Genome-wide loss of DNA methylation is commonly observed in human cancers, but its impact on the tumor transcriptome remains ill-defined. Previous studies demonstrated that this epigenetic alteration causes aberrant activation of a germline-specific gene expression program. Here, we examined if DNA hypomethylation in tumors also leads to de-repression of gene clusters with other tissue specificities. To this end, we explored transcriptomic and methylomic datasets from human lung adenocarcinoma (LUAD) cell lines, normal lung, and lung alveolar type II cells, considered as the origin of LUAD. Interestingly, DNA demethylation in LUAD cell lines was associated with activation of not only germline-specific (CG) genes, but also gene clusters displaying specific expression in the gastrointestinal tract (GI), or in stratified epithelia (SE). Consistently, genes from all three clusters showed highly specific patterns of promoter methylation among normal tissues and cell types, and were generally sensitive to induction by a DNA demethylating agent. Analysis of TCGA datasets confirmed that demethylation and activation of CG, GI and SE genes also occurs in vivo in LUAD tumor tissues, in association with global genome hypomethylation. For genes of the GI cluster, we demonstrated that HNF4A is a necessary factor for transcriptional activation following promoter demethylation. Interestingly, expression of several SE genes, in particular FAM83A, correlated with both tumor grade and reduced patient survival. Together, our study uncovers novel cell-type specific gene clusters that become aberrantly activated in LUAD tumors in association with genome-wide hypomethylation.

Obtaining spatially resolved tumor purity maps using deep multiple instance learning in a pan-cancer study

Tumor purity is the percentage of cancer cells within a tissue section. Pathologists estimate tumor purity to select samples for genomic analysis by manually reading hematoxylin-eosin (H&E)-stained slides, which is tedious, time consuming, and prone to inter-observer variability. Besides, pathologists' estimates do not correlate well with genomic tumor purity values, which are inferred from genomic data and accepted as accurate for downstream analysis. We developed a deep multiple instance learning model predicting tumor purity from H&E-stained digital histopathology slides. Our model successfully predicted tumor purity in eight The Cancer Genome Atlas (TCGA) cohorts and a local Singapore cohort. The predictions were highly consistent with genomic tumor purity values. Thus, our model can be utilized to select samples for genomic analysis, which will help reduce pathologists' workload and decrease inter-observer variability. Furthermore, our model provided tumor purity maps showing the spatial variation within sections. They can help better understand the tumor microenvironment.

•

MIL model successfully predicts a sample's tumor purity from histopathology slides

•

MIL model learns to spatially resolve tumor purity from sample-level labels

•

Tumor purity varies spatially within a sample

•

Pathologists’ region selection is vital for correct percentage tumor nuclei estimation

Given some big data and coarse-level labels, extracting fine-level information is a demanding yet rewarding challenge in data science. This study develops a machine learning model utilizing big data and exploiting coarse-level labels to reveal fine-level details within the data. Although it can be applied to different data science tasks with enormous data and coarse labels, we applied it to a computational histopathology task with gigapixel histopathology slides and sample-level labels. Specifically, the model revealed spatial resolution of tumor purity within histopathology slides using only sample-level genomic tumor purity values during training. This can also be extended to other omics features, providing precious information about cancer biology and promising personalized, precision medicine. Such studies are of great clinical importance in discovering imaging biomarkers and better understanding the tumor microenvironment.

Cervical Cancer Metastasis and Recurrence Risk Prediction Based on Deep Convolutional Neural Network

Background:

Evaluating the risk of metastasis and recurrence of a cervical cancer patient is critical for appropriate adjuvant therapy. However, current risk assessment models usually involve the testing of tens to thousands of genes from patients’ tissue samples, which is expensive and timeconsuming. Therefore, computer-aided diagnosis and prognosis prediction based on Hematoxylin and Eosin (H&E) pathological images have received much attention recently.

Objective:

The prognosis of whether patients will have metastasis and recurrence can support accurate treatment for patients in advance and help reduce patient loss. It is also important for guiding treatment after surgery to be able to quickly and accurately predict the risk of metastasis and recurrence of a cervical cancer patient.

Method:

To address this problem, we propose a hybrid method. Transfer learning is used to extract features, and it is combined with traditional machine learning in order to analyze and determine whether patients have the risks of metastasis and recurrence. First, the proposed model retrieved relevant patches using a color-based method from H&E pathological images, which were then subjected to image preprocessing steps such as image normalization and color homogenization. Based on the labeled patched images, the Xception model with good classification performance was selected, and deep features of patched pathological images were automatically extracted with transfer learning. After that, the extracted features were combined to train a random forest model to predict the label of a new patched image. Finally, a majority voting method was developed to predict the metastasis and recurrence risk of a patient based on the predictions of patched images from the whole-slide H&E image.

Results:

In our experiment, the proposed model yielded an area under the receiver operating characteristic curve of 0.82 for the whole-slide image. The experimental results showed that the high-level features extracted by the deep convolutional neural network from the whole-slide image can be used to predict the risk of recurrence and metastasis after surgical resection and help identify patients who might receive additional benefit from adjuvant therapy.

Conclusion:

This paper explored the feasibility of predicting the risk of metastasis and recurrence from cervical cancer whole slide H&E images through deep learning and random forest methods.

HistoLens: A generalizable tool for increasing accessibility and interpretability of quantitative analyses in digital pathology

The incorporation of automated computational tools has a great amount of potential to positively influence the field of pathology. However, pathologists and regulatory agencies are reluctant to trust the output of complex models such as Convolutional Neural Networks (CNNs) due to their usual implementation as black-box tools. Increasing the interpretability of quantitative analyses is a critical line of research in order to increase the adoption of modern Machine Learning (ML) pipelines in clinical environments. Towards that goal, we present HistoLens, a Graphical User Interface (GUI) designed to facilitate quantitative assessments of datasets of annotated histological compartments. Additionally, we introduce the use of hand-engineered feature visualizations to highlight regions within each structure that contribute to particular feature values. These feature visualizations can then be paired with feature hierarchy determinations in order to view which regions within an image are significant to a particular sub-group within the dataset. As a use case, we analyzed a dataset of old and young mouse kidney sections with glomeruli annotated. We highlight some of the functional components within HistoLens that allow non-computational experts to efficiently navigate a new dataset as well as allowing for easier transition to downstream computational analyses.

Lung Cancer Computational Biology and Resources

Comprehensive clinical, pathological, and molecular data, when appropriately integrated with advanced computational approaches, are transforming the way we characterize and study lung cancer. Clinically, cancer registry and publicly available historical clinical trial data enable retrospective analyses to examine how socioeconomic factors, patient demographics, and cancer characteristics affect treatment and outcome. Pathologically, digital pathology and artificial intelligence are revolutionizing histopathological image analyses, not only with improved efficiency and accuracy, but also by extracting additional information for prognostication and tumor microenvironment characterization. Genetically and molecularly, individual patient tumors and preclinical models of lung cancer are profiled by various high-throughput platforms to characterize the molecular properties and functional liabilities. The resulting multi-omics data sets and their interrogation facilitate both basic research mechanistic studies and translation of the findings into the clinic. In this review, we provide a list of resources and tools potentially valuable for lung cancer basic and translational research. Importantly, we point out pitfalls and caveats when performing computational analyses of these data sets and provide a vision of future computational biology developments that will aid lung cancer translational research.

From What to Why, the Growing Need for a Focus Shift Toward Explainability of AI in Digital Pathology

While it is impossible to deny the performance gains achieved through the incorporation of deep learning (DL) and other artificial intelligence (AI)-based techniques in pathology, minimal work has been done to answer the crucial question of why these algorithms predict what they predict. Tracing back classification decisions to specific input features allows for the quick identification of model bias as well as providing additional information toward understanding underlying biological mechanisms. In digital pathology, increasing the explainability of AI models would have the largest and most immediate impact for the image classification task. In this review, we detail some considerations that should be made in order to develop models with a focus on explainability.

Artificial intelligence-assisted decision making for prognosis and drug efficacy prediction in lung cancer patients: a narrative review

Objective

In this review, we aim to present frontier studies in patients with lung cancer as it related to artificial intelligence (AI)-assisted decision-making and summarize the latest advances, challenges and future trend in this field.

Background

Despite increasing survival rate in cancer patients over the last decades, lung cancer remains one of the leading causes of death worldwide. The early diagnosis, accurate evaluation and individualized treatment are vital approaches to improve the survival rate of patients with lung cancer. Thus, decision making based on these approaches requires accuracy and efficiency beyond manpower. Recent advances in AI and precision medicine have provided a fertile environment for the development of AI-based models. These models have the potential to assist radiologists and oncologists in detecting lung cancer, predicting prognosis and developing personalized treatment plans for better outcomes of the patients.

Methods

We searched literature from 2000 through July 31^th, 2021 in Medline/PubMed, the Web of Science, the Cochrane Library, ACM Digital Library, INSPEC and EMBASE. Key words such as “artificial intelligence”, “AI”, “deep learning”, “lung cancer”, “NSCLC”, “SCLC” were combined to identify related literatures. These literatures were then selected by two independent authors. Articles chosen by only one author will be examined by another author to determine whether this article was relative and valuable. The selected literatures were read by all authors and discussed to draw reliable conclusions.

Conclusions

AI, especially for those based on deep learning and radiomics, is capable of assisting clinical decision making from many aspects, for its quantitatively interpretation of patients’ information and its potential to deal with the dynamics, individual differences and heterogeneity of lung cancer. Hopefully, remaining problems such as insufficient data and poor interpretability may be solved to put AI-based models into clinical practice.

A narrative review of artificial intelligence-assisted histopathologic diagnosis and decision-making for non-small cell lung cancer: achievements and limitations

Objective

To summarize the current evidence regarding the applications, workflow, and limitations of artificial intelligence (AI) in the management of patients pathologically-diagnosed with lung cancer.

Background

Lung cancer is one of the most common cancers and the leading cause of cancer-related deaths worldwide. AI technologies have been applied to daily medical workflow and have achieved an excellent performance in predicting histopathologic subtypes, analyzing gene mutation profiles, and assisting in clinical decision-making for lung cancer treatment. More advanced deep learning for classifying pathologic images with minimal human interactions has been developed in addition to the conventional machine learning scheme.

Methods

Studies were identified by searching databases, including PubMed, EMBASE, Web of Science, and Cochrane Library, up to February 2021 without language restrictions.

Conclusions

A number of studies have evaluated AI pipelines and confirmed that AI is robust and efficacious in lung cancer diagnosis and decision-making, demonstrating that AI models are a useful tool for assisting oncologists in health management. Although several limitations that pose an obstacle for the widespread use of AI schemes persist, the unceasing refinement of AI techniques is poised to overcome such problems. Thus, AI technology is a promising tool for use in diagnosing and managing lung cancer.

A deep learning model for breast ductal carcinoma in situ classification in whole slide images

The pathological differential diagnosis between breast ductal carcinoma in situ (DCIS) and invasive ductal carcinoma (IDC) is of pivotal importance for determining optimum cancer treatment(s) and clinical outcomes. Since conventional diagnosis by pathologists using microscopes is limited in terms of human resources, it is necessary to develop new techniques that can rapidly and accurately diagnose large numbers of histopathological specimens. Computational pathology tools which can assist pathologists in detecting and classifying DCIS and IDC from whole slide images (WSIs) would be of great benefit for routine pathological diagnosis. In this paper, we trained deep learning models capable of classifying biopsy and surgical histopathological WSIs into DCIS, IDC, and benign. We evaluated the models on two independent test sets (n= 1382, n= 548), achieving ROC areas under the curves (AUCs) up to 0.960 and 0.977 for DCIS and IDC, respectively.

An Expandable Informatics Framework for Enhancing Central Cancer Registries with Digital Pathology Specimens, Computational Imaging Tools, and Advanced Mining Capabilities

BACKGROUND

Population-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI's Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features).

MATERIALS AND METHODS

As part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated.

RESULTS

Our team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics.

CONCLUSION

To meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.

Assessment of deep learning assistance for the pathological diagnosis of gastric cancer

Previous studies on deep learning (DL) applications in pathology have focused on pathologist-versus-algorithm comparisons. However, DL will not replace the breadth and contextual knowledge of pathologists; rather, only through their combination may the benefits of DL be achieved. A fully crossed multireader multicase study was conducted to evaluate DL assistance with pathologists' diagnosis of gastric cancer. A total of 110 whole-slide images (WSI) (50 malignant and 60 benign) were interpreted by 16 board-certified pathologists with or without DL assistance, with a washout period between sessions. DL-assisted pathologists achieved a higher area under receiver operating characteristic curve (ROC-AUC) (0.911 vs. 0.863, P = 0.003) than unassisted in interpreting the 110 WSIs. Pathologists with DL assistance demonstrated higher sensitivity in detection of gastric cancer than without (90.63% vs. 82.75%, P = 0.010). No significant difference was observed in specificity with or without deep learning assistance (78.23% vs. 79.90%, P = 0.468). The average review time per WSI was shortened with DL assistance than without (22.68 vs. 26.37 second, P = 0.033). Our results demonstrated that DL assistance indeed improved pathologists' accuracy and efficiency in gastric cancer diagnosis and further boosted the acceptance of this new technique.

Spatial analysis of histology in 3D: quantification and visualization of organ and tumor level tissue environment

Histological changes in tissue are of primary importance in pathological research and diagnosis. Automated histological analysis requires ability to computationally separate pathological alterations from normal tissue. Conventional histopathological assessments are performed from individual tissue sections, leading to the loss of three-dimensional context of the tissue. Yet, the tissue context and spatial determinants are critical in several pathologies, such as in understanding growth patterns of cancer in its local environment. Here, we develop computational methods for visualization and quantitative assessment of histopathological alterations in three dimensions. First, we reconstruct the 3D representation of the whole organ from serial sectioned tissue. Then, we proceed to analyze the histological characteristics and regions of interest in 3D. As our example cases, we use whole slide images representing hematoxylin-eosin stained whole mouse prostates in a Pten+/- mouse prostate tumor model. We show that quantitative assessment of tumor sizes, shapes, and separation between spatial locations within the organ enable characterizing and grouping tumors. Further, we show that 3D visualization of tissue with computationally quantified features provides an intuitive way to observe tissue pathology. Our results underline the heterogeneity in composition and cellular organization within individual tumors. As an example, we show how prostate tumors have nuclear density gradients indicating areas of tumor growth directions and reflecting varying pressure from the surrounding tissue. The methods presented here are applicable to any tissue and different types of pathologies. This work provides a proof-of-principle for gaining a comprehensive view from histology by studying it quantitatively in 3D.

OUP accepted manuscript

Background

Materials and Methods

Results

Conclusion

OUP accepted manuscript

Motivation

Cancer survival prediction can greatly assist clinicians in planning patient treatments and improving their life quality. Recent evidence suggests the fusion of multimodal data, such as genomic data and pathological images, is crucial for understanding cancer heterogeneity and enhancing survival prediction. As a powerful multimodal fusion technique, Kronecker product has shown its superiority in predicting survival. However, this technique introduces a large number of parameters that may lead to high computational cost and a risk of overfitting, thus limiting its applicability and improvement in performance. Another limitation of existing approaches using Kronecker product is that they only mine relations for one single time to learn multimodal representation and therefore face significant challenges in deeply mining rich information from multimodal data for accurate survival prediction.

Results

To address the above limitations, we present a novel hierarchical multimodal fusion approach named HFBSurv by employing factorized bilinear model to fuse genomic and image features step by step. Specifically, with a multiple fusion strategy HFBSurv decomposes the fusion problem into different levels and each of them integrates and passes information progressively from the low level to the high level, thus leading to the more specialized fusion procedure and expressive multimodal representation. In this hierarchical framework, both modality-specific and cross-modality attentional factorized bilinear modules are designed to not only capture and quantify complex relations from multimodal data, but also dramatically reduce computational complexity. Extensive experiments demonstrate that our method performs an effective hierarchical fusion of multimodal data and achieves consistently better performance than other methods for survival prediction.

Availability and implementation

HFBSurv is freely available at https://github.com/Liruiqing-ustc/HFBSurv.

Supplementary information

Supplementary data are available at Bioinformatics online.

Terahertz toroidal metasurface biosensor for sensitive distinction of lung cancer cells

Lung cancer is the most frequently life-threatening disease and the prominent cause of cancer-related mortality among human beings worldwide, where poor early diagnosis and expensive detection costs are considered as significant reasons. Here, we try to tackle this issue by proposing a novel label-free and low-cost strategy for rapid detection and distinction of lung cancer cells relying on plasmonic toroidal metasurfaces at terahertz frequencies. Three disjoint regions are displayed in identifiable intensity-frequency diagram, which could directly help doctors determine the type of lung cancer cells for clinical treatment. The metasurface is generated by two mirrored gold split ring resonators with subwavelength sizes. When placing analytes on the metasurface, apparent shifts of both the resonance frequency and the resonance depth can be observed in the terahertz transmission spectra. The theoretical sensitivity of the biosensor over the reflective index reaches as high as 485.3 GHz/RIU. Moreover, the proposed metasurface shows high angular stability for oblique incident angle from 0 to 30°, where the maximum resonance frequency shift is less than 0.66% and the maximum transmittance variation keeps below 1.33%. To experimentally verify the sensing strategy, three types of non-small cell lung cancer cells (Calu-1, A427, and 95D) are cultured with different concentrations and their terahertz transmission spectra are measured with the proposed metasurface biosensor. The two-dimensional fingerprint diagram considering both the frequency and transmittance variations of the toroidal resonance dip is obtained, where the curves for different cells are completely separated with each other. This implies that we can directly distinguish the type of the analyte cells and its concentration by only single spectral measurement. We envisage that the proposed strategy has potential for clinical diagnosis and significantly expands the capabilities of plasmonic metamaterials in biological detection.

Nanoscale delivery of phytochemicals targeting CRISPR/Cas9 for cancer therapy

BACKGROUND

With growing global prevalence, cancer is a major cause of disease-related deaths. The understanding of the fundamental tumor pathology has contributed to the development of agents targeting oncogenic signaling pathways. Although these agents have increased survival for defined cancers, the therapeutic choices are still limited due to the development of drug resistance. CRISPR/Cas9 is a powerful new technology in cancer therapy by facilitating the identification of novel treatment targets and development of cell-based treatment strategies.

PURPOSE

We focused on applications of the CRISPR/Cas9 system in cancer therapy and discuss nanoscale delivery of cytotoxic phytochemical targeting the CRISPR/Cas9 system.

RESULTS

Genome engineering has been significantly accelerated by the advancement of the CRISPR/Cas9 technique. Phytochemicals play a key role in treating cancer by targeting various mechanisms and pathways.

CONCLUSIONS

The use of CRISPR/Cas9 for nanoscale delivery of phytochemicals opens new avenues in cancer therapy. One of the main obstacles in the clinical application of CRISPR/Cas9 is safe and efficient delivery. As viral delivery methods have certain drawbacks, there is an urgent need to develop non-viral delivery systems for therapeutic applications.

Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a multicentre observational study

BACKGROUND

Accurate prediction of tumour response to neoadjuvant chemoradiotherapy enables personalised perioperative therapy for locally advanced rectal cancer. We aimed to develop and validate an artificial intelligence radiopathomics integrated model to predict pathological complete response in patients with locally advanced rectal cancer using pretreatment MRI and haematoxylin and eosin (H&E)-stained biopsy slides.

METHODS

In this multicentre observational study, eligible participants who had undergone neoadjuvant chemoradiotherapy followed by radical surgery were recruited, with their pretreatment pelvic MRI (T2-weighted imaging, contrast-enhanced T1-weighted imaging, and diffusion-weighted imaging) and whole slide images of H&E-stained biopsy sections collected for annotation and feature extraction. The RAdioPathomics Integrated preDiction System (RAPIDS) was constructed by machine learning on the basis of three feature sets associated with pathological complete response: radiomics MRI features, pathomics nucleus features, and pathomics microenvironment features from a retrospective training cohort. The accuracy of RAPIDS for the prediction of pathological complete response in locally advanced rectal cancer was verified in two retrospective external validation cohorts and further validated in a multicentre, prospective observational study (ClinicalTrials.gov, NCT04271657). Model performances were evaluated using area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

FINDINGS

Between Sept 25, 2009, and Nov 3, 2017, 303 patients were retrospectively recruited in the training cohort, 480 in validation cohort 1, and 150 in validation cohort 2; 100 eligible patients were enrolled in the prospective study between Jan 10 and June 10, 2020. RAPIDS had favourable accuracy for the prediction of pathological complete response in the training cohort (AUC 0·868 [95% CI 0·825-0·912]), and in validation cohort 1 (0·860 [0·828-0·892]) and validation cohort 2 (0·872 [0·810-0·934]). In the prospective validation study, RAPIDS had an AUC of 0·812 (95% CI 0·717-0·907), sensitivity of 0·888 (0·728-0·999), specificity of 0·740 (0·593-0·886), NPV of 0·929 (0·862-0·995), and PPV of 0·512 (0·313-0·710). RAPIDS also significantly outperformed single-modality prediction models (AUC 0·630 [0·507-0·754] for the pathomics microenvironment model, 0·716 [0·580-0·852] for the radiomics MRI model, and 0·733 [0·620-0·845] for the pathomics nucleus model; all p<0·0001).

INTERPRETATION

RAPIDS was able to predict pathological complete response to neoadjuvant chemoradiotherapy based on pretreatment radiopathomics images with high accuracy and robustness and could therefore provide a novel tool to assist in individualised management of locally advanced rectal cancer.

FUNDING

National Natural Science Foundation of China; Youth Innovation Promotion Association of the Chinese Academy of Sciences.

Evaluating Cancer-Related Biomarkers Based on Pathological Images: A Systematic Review

Many diseases are accompanied by changes in certain biochemical indicators called biomarkers in cells or tissues. A variety of biomarkers, including proteins, nucleic acids, antibodies, and peptides, have been identified. Tumor biomarkers have been widely used in cancer risk assessment, early screening, diagnosis, prognosis, treatment, and progression monitoring. For example, the number of circulating tumor cell (CTC) is a prognostic indicator of breast cancer overall survival, and tumor mutation burden (TMB) can be used to predict the efficacy of immune checkpoint inhibitors. Currently, clinical methods such as polymerase chain reaction (PCR) and next generation sequencing (NGS) are mainly adopted to evaluate these biomarkers, which are time-consuming and expansive. Pathological image analysis is an essential tool in medical research, disease diagnosis and treatment, functioning by extracting important physiological and pathological information or knowledge from medical images. Recently, deep learning-based analysis on pathological images and morphology to predict tumor biomarkers has attracted great attention from both medical image and machine learning communities, as this combination not only reduces the burden on pathologists but also saves high costs and time. Therefore, it is necessary to summarize the current process of processing pathological images and key steps and methods used in each process, including: (1) pre-processing of pathological images, (2) image segmentation, (3) feature extraction, and (4) feature model construction. This will help people choose better and more appropriate medical image processing methods when predicting tumor biomarkers.

Using a convolutional neural network for classification of squamous and non-squamous non-small cell lung cancer based on diagnostic histopathology HES images

Histological stratification in metastatic non-small cell lung cancer (NSCLC) is essential to properly guide therapy. Morphological evaluation remains the basis for subtyping and is completed by additional immunohistochemistry labelling to confirm the diagnosis, which delays molecular analysis and utilises precious sample. Therefore, we tested the capacity of convolutional neural networks (CNNs) to classify NSCLC based on pathologic HES diagnostic biopsies. The model was estimated with a learning cohort of 132 NSCLC patients and validated on an external validation cohort of 65 NSCLC patients. Based on image patches, a CNN using InceptionV3 architecture was trained and optimized to classify NSCLC between squamous and non-squamous subtypes. Accuracies of 0.99, 0.87, 0.85, 0.85 was reached in the training, validation and test sets and in the external validation cohort. At the patient level, the CNN model showed a capacity to predict the tumour histology with accuracy of 0.73 and 0.78 in the learning and external validation cohorts respectively. Selecting tumour area using virtual tissue micro-array improved prediction, with accuracy of 0.82 in the external validation cohort. This study underlines the capacity of CNN to predict NSCLC subtype with good accuracy and to be applied to small pathologic samples without annotation.

AI in spotting high-risk characteristics of medical imaging and molecular pathology

Medical imaging provides a comprehensive perspective and rich information for disease diagnosis. Combined with artificial intelligence technology, medical imaging can be further mined for detailed pathological information. Many studies have shown that the macroscopic imaging characteristics of tumors are closely related to microscopic gene, protein and molecular changes. In order to explore the function of artificial intelligence algorithms in in-depth analysis of medical imaging information, this paper reviews the articles published in recent years from three perspectives: medical imaging analysis method, clinical applications and the development of medical imaging in the direction of pathological molecular prediction. We believe that AI-aided medical imaging analysis will be extensively contributing to precise and efficient clinical decision.

Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning

Objective.Subtype classification plays a guiding role in the clinical diagnosis and treatment of non-small-cell lung cancer (NSCLC). However, due to the gigapixel of whole slide images (WSIs) and the absence of definitive morphological features, most automatic subtype classification methods for NSCLC require manually delineating the regions of interest (ROIs) on WSIs.Approach.In this paper, a weakly supervised framework is proposed for accurate subtype classification while freeing pathologists from pixel-level annotation. With respect to the characteristics of histopathological images, we design a two-stage structure with ROI localization and subtype classification. We first develop a method called multi-resolution expectation-maximization convolutional neural network (MR-EM-CNN) to locate ROIs for subsequent subtype classification. The EM algorithm is introduced to select the discriminative image patches for training a patch-wise network, with only WSI-wise labels available. A multi-resolution mechanism is designed for fine localization, similar to the coarse-to-fine process of manual pathological analysis. In the second stage, we build a novel hierarchical attention multi-scale network (HMS) for subtype classification. HMS can capture multi-scale features flexibly driven by the attention module and implement hierarchical features interaction.Results.Experimental results on the 1002-patient Cancer Genome Atlas dataset achieved an AUC of 0.9602 in the ROI localization and an AUC of 0.9671 for subtype classification.Significance.The proposed method shows superiority compared with other algorithms in the subtype classification of NSCLC. The proposed framework can also be extended to other classification tasks with WSIs.

Weakly Supervised Deep Ordinal Cox Model for Survival Prediction From Whole-Slide Pathological Images

Whole-Slide Histopathology Image (WSI) is generally considered the gold standard for cancer diagnosis and prognosis. Given the large inter-operator variation among pathologists, there is an imperative need to develop machine learning models based on WSIs for consistently predicting patient prognosis. The existing WSI-based prediction methods do not utilize the ordinal ranking loss to train the prognosis model, and thus cannot model the strong ordinal information among different patients in an efficient way. Another challenge is that a WSI is of large size (e.g., 100,000-by-100,000 pixels) with heterogeneous patterns but often only annotated with a single WSI-level label, which further complicates the training process. To address these challenges, we consider the ordinal characteristic of the survival process by adding a ranking-based regularization term on the Cox model and propose a weakly supervised deep ordinal Cox model (BDOCOX) for survival prediction from WSIs. Here, we generate amounts of bags from WSIs, and each bag is comprised of the image patches representing the heterogeneous patterns of WSIs, which is assumed to match the WSI-level labels for training the proposed model. The effectiveness of the proposed method is well validated by theoretical analysis as well as the prognosis and patient stratification results on three cancer datasets from The Cancer Genome Atlas (TCGA).

Artificial intelligence in thoracic surgery: a narrative review

OBJECTIVE

The aim of this article is to review the current applications of artificial intelligence in thoracic surgery, from diagnosis and pulmonary disease management, to preoperative risk-assessment, surgical planning, and outcomes prediction.

BACKGROUND

Artificial intelligence implementation in healthcare settings is rapidly growing, though its widespread use in clinical practice is still limited. The employment of machine learning algorithms in thoracic surgery is wide-ranging, including all steps of the clinical pathway.

METHODS

We performed a narrative review of the literature on Scopus, PubMed and Cochrane databases, including all the relevant studies published in the last ten years, until March 2021.

CONCLUSION

Machine learning methods are promising encouraging results throughout the key issues of thoracic surgery, both clinical, organizational, and educational. Artificial intelligence-based technologies showed remarkable efficacy to improve the perioperative evaluation of the patient, to assist the decision-making process, to enhance the surgical performance, and to optimize the operating room scheduling. Still, some concern remains about data supply, protection, and transparency, thus further studies and specific consensus guidelines are needed to validate these technologies for daily common practice.

KEYWORDS

Artificial intelligence (AI); thoracic surgery; machine learning; lung resection; perioperative medicine.

A Polarization-Imaging-Based Machine Learning Framework for Quantitative Pathological Diagnosis of Cervical Precancerous Lesions

Polarization images encode high resolution microstructural information even at low resolution. We propose a framework combining polarization imaging and traditional microscopy imaging, constructing a dual-modality machine learning framework that is not only accurate but also generalizable and interpretable. We demonstrate the viability of our proposed framework using the cervical intraepithelial neoplasia grading task, providing a polarimetry feature parameter to quantitatively characterize microstructural variations with lesion progression in hematoxylin-eosin-stained pathological sections of cervical precancerous tissues. By taking advantages of polarization imaging techniques and machine learning methods, the model enables interpretable and quantitative diagnosis of cervical precancerous lesion cases with improved sensitivity and accuracy in a low-resolution and wide-field system. The proposed framework applies routine image-analysis technology to identify the macro-structure and segment the target region in H&E-stained pathological images, and then employs emerging polarization method to extract the micro-structure information of the target region, which intends to expand the boundary of the current image-heavy digital pathology, bringing new possibilities for quantitative medical diagnosis.

Integrative multiomics-histopathology analysis for breast cancer classification

Histopathologic evaluation of biopsy slides is a critical step in diagnosing and subtyping breast cancers. However, the connections between histology and multi-omics status have never been systematically explored or interpreted. We developed weakly supervised deep learning models over hematoxylin-and-eosin-stained slides to examine the relations between visual morphological signal, clinical subtyping, gene expression, and mutation status in breast cancer. We first designed fully automated models for tumor detection and pathology subtype classification, with the results validated in independent cohorts (area under the receiver operating characteristic curve ≥ 0.950). Using only visual information, our models achieved strong predictive performance in estrogen/progesterone/HER2 receptor status, PAM50 status, and TP53 mutation status. We demonstrated that these models learned lymphocyte-specific morphological signals to identify estrogen receptor status. Examination of the PAM50 cohort revealed a subset of PAM50 genes whose expression reflects cancer morphology. This work demonstrates the utility of deep learning-based image models in both clinical and research regimes, through its ability to uncover connections between visual morphology and genetic statuses.

Machine learning-based pathomics signature could act as a novel prognostic marker for patients with clear cell renal cell carcinoma

BACKGROUND

Traditional histopathology performed by pathologists through naked eyes is insufficient for accurate survival prediction of clear cell renal cell carcinoma (ccRCC).

METHODS

A total of 483 whole slide images (WSIs) data from three patient cohorts were retrospectively analyzed. We performed machine learning algorithm to identify optimal digital pathological features and constructed machine learning-based pathomics signature (MLPS) for ccRCC patients. Prognostic performance of the prognostic model was also verified in two independent validation cohorts.

RESULTS

MLPS could significantly distinguish ccRCC patients with high survival risk, with hazard ratio of 15.05, 4.49 and 1.65 in three independent cohorts, respectively. Cox regression analysis revealed that the MLPS could act as an independent prognostic factor for ccRCC patients. Integration nomogram based on MLPS, tumour stage system and tumour grade system improved the current survival prediction accuracy for ccRCC patients, with area under curve value of 89.5%, 90.0%, 88.5% and 85.9% for 1-, 3-, 5- and 10-year disease-free survival prediction.

DISCUSSION

The machine learning-based pathomics signature could act as a novel prognostic marker for patients with ccRCC. Nevertheless, prospective studies with multicentric patient cohorts are still needed for further verifications.

An attention-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in whole slide images

Melanoma is an aggressive neoplasm responsible for the majority of deaths from skin cancer. Specifically, spitzoid melanocytic tumors are one of the most challenging melanocytic lesions due to their ambiguous morphological features. The gold standard for its diagnosis and prognosis is the analysis of skin biopsies. In this process, dermatopathologists visualize skin histology slides under a microscope, in a highly time-consuming and subjective task. In the last years, computer-aided diagnosis (CAD) systems have emerged as a promising tool that could support pathologists in daily clinical practice. Nevertheless, no automatic CAD systems have yet been proposed for the analysis of spitzoid lesions. Regarding common melanoma, no system allows both the selection of the tumor region and the prediction of the benign or malignant form in the diagnosis. Motivated by this, we propose a novel end-to-end weakly supervised deep learning model, based on inductive transfer learning with an improved convolutional neural network (CNN) to refine the embedding features of the latent space. The framework is composed of a source model in charge of finding the tumor patch-level patterns, and a target model focuses on the specific diagnosis of a biopsy. The latter retrains the backbone of the source model through a multiple instance learning workflow to obtain the biopsy-level scoring. To evaluate the performance of the proposed methods, we performed extensive experiments on a private skin database with spitzoid lesions. Test results achieved an accuracy of 0.9231 and 0.80 for the source and the target models, respectively. In addition, the heat map findings are directly in line with the clinicians' medical decision and even highlight, in some cases, patterns of interest that were overlooked by the pathologist.

Deep Learning Models for Poorly Differentiated Colorectal Adenocarcinoma Classification in Whole Slide Images Using Transfer Learning

Colorectal poorly differentiated adenocarcinoma (ADC) is known to have a poor prognosis as compared with well to moderately differentiated ADC. The frequency of poorly differentiated ADC is relatively low (usually less than 5% among colorectal carcinomas). Histopathological diagnosis based on endoscopic biopsy specimens is currently the most cost effective method to perform as part of colonoscopic screening in average risk patients, and it is an area that could benefit from AI-based tools to aid pathologists in their clinical workflows. In this study, we trained deep learning models to classify poorly differentiated colorectal ADC from Whole Slide Images (WSIs) using a simple transfer learning method. We evaluated the models on a combination of test sets obtained from five distinct sources, achieving receiver operating characteristic curve (ROC) area under the curves (AUCs) up to 0.95 on 1799 test cases.

Generative Adversarial Networks in Digital Pathology and Histopathological Image Processing: A Review

Digital pathology is gaining prominence among the researchers with developments in advanced imaging modalities and new technologies. Generative adversarial networks (GANs) are a recent development in the field of artificial intelligence and since their inception, have boosted considerable interest in digital pathology. GANs and their extensions have opened several ways to tackle many challenging histopathological image processing problems such as color normalization, virtual staining, ink removal, image enhancement, automatic feature extraction, segmentation of nuclei, domain adaptation and data augmentation. This paper reviews recent advances in histopathological image processing using GANs with special emphasis on the future perspectives related to the use of such a technique. The papers included in this review were retrieved by conducting a keyword search on Google Scholar and manually selecting the papers on the subject of H&E stained digital pathology images for histopathological image processing. In the first part, we describe recent literature that use GANs in various image preprocessing tasks such as stain normalization, virtual staining, image enhancement, ink removal, and data augmentation. In the second part, we describe literature that use GANs for image analysis, such as nuclei detection, segmentation, and feature extraction. This review illustrates the role of GANs in digital pathology with the objective to trigger new research on the application of generative models in future research in digital pathology informatics.

Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine

Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.