Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features

Deciphering the Morphology of Tumor-Stromal Features in Invasive Breast Cancer Using Artificial Intelligence

Tumor-associated stroma in breast cancer (BC) is complex and exhibits a high degree of heterogeneity. To date, no standardized assessment method has been established. Artificial intelligence (AI) could provide an objective morphologic assessment of tumors and stroma, with the potential to identify new features not discernible by visual microscopy. In this study, we used AI to assess the clinical significance of (1) stroma-to-tumor ratio (S:TR) and (2) the spatial arrangement of stromal cells, tumor cell density, and tumor burden in BC. Whole-slide images of a large cohort (n = 1968) of well-characterized luminal BC cases were examined. Region and cell-level annotation was performed, and supervised deep learning models were applied for automated quantification of tumor and stromal features. S:TR was calculated in terms of surface area and cell count ratio, and the S:TR heterogeneity and spatial distribution were also assessed. Tumor cell density and tumor size were used to estimate tumor burden. Cases were divided into discovery (n = 1027) and test (n = 941) sets for validation of the findings. In the whole cohort, the stroma-to-tumor mean surface area ratio was 0.74, and stromal cell density heterogeneity score was high (0.7/1). BC with high S:TR showed features characteristic of good prognosis and longer patient survival in both the discovery and test sets. Heterogeneous spatial distribution of S:TR areas was predictive of worse outcome. Higher tumor burden was associated with aggressive tumor behavior and shorter survival and was an independent predictor of worse outcome (BC-specific survival; hazard ratio: 1.7, P = .03, 95% CI, 1.04-2.83 and distant metastasis-free survival; hazard ratio: 1.64, P = .04, 95% CI, 1.01-2.62) superior to absolute tumor size. The study concludes that AI provides a tool to assess major and subtle morphologic stromal features in BC with prognostic implications. Tumor burden is more prognostically informative than tumor size.

Research for accurate auxiliary diagnosis of lung cancer based on intracellular fluorescent fingerprint information

The distinctions in pathological types and genetic subtypes of lung cancer have a direct impact on the choice of treatment choices and clinical prognosis in clinical practice. This study used pathological histological sections of surgically removed or biopsied tumor tissue from 36 patients. Based on a small sample size, millions of spectral data points were extracted to investigate the feasibility of employing intracellular fluorescent fingerprint information to diagnose the pathological types and mutational status of lung cancer. The intracellular fluorescent fingerprint information revealed the EGFR gene mutation characteristics in lung cancer, and the area under the curve (AUC) value for the optimal model was 0.98. For the classification of lung cancer pathological types, the macro average AUC value for the ensemble-learning model was 0.97. Our research contributes new idea for pathological diagnosis of lung cancer and offers a quick, easy, and accurate auxiliary diagnostic approach.

Histopathology of Ganglion Cells in the Proximal Resected Bowel Correlates With the Clinical Outcome in Hirschsprung Disease: A Pilot Study

INTRODUCTION

Intraoperative leveling biopsy by identifying ganglion cells is crucial to determine surgical margin during surgery for Hirschsprung disease (HSCR). The anastomosis should be performed at least 5 cm proximal to the ganglionic segment to prevent transition zone pull-through. However, the length of the transition zone could be much longer than expected and the histological evaluation of the entire circumference of the proximal margin is recommended, which is time-consuming and not applicable for leveling biopsy. We found that the histopathologic features of ganglion cells varied in the examined bowel specimens and demonstrated a pattern similar to immature and degenerated neuron cells. We assumed that the histopathologic features of ganglion cells in the proximal resected bowel were associated with the clinical outcome and might guide the leveling biopsy. In this study, we described a histopathologic grade of ganglion cells based on the degree of maturity and degeneration. We assessed the correlation between the histopathological grade of ganglion cells in the proximal surgical margin and clinical outcome.

METHODS

Three hundred fifty seven patients with HSCR treated between 2013 and 2020 were included. The ganglion cells were divided into six grades based on the histopathologic features in frozen sections. Medical records and detailed histopathologic results of intraoperative frozen sections were reviewed. Follow-up data were collected to evaluate clinical outcomes. The pediatric incontinence and constipation scoring system was used to predict bowel function.

RESULTS

The histopathologic results of proximal resected bowel from 357 HSCR patients were presented as follows: Grade I in 52 patients (14.6%), Grade II in 186 patients (52.1%), Grade III in 107 patients (30.0%), and Grade IV in 12 patients (3.4%). The median follow-up time was 46.8 mo (13.0-97.6 mo). The histopathologic grade of ganglion cells from the proximal resected margin was significantly related to postoperative constipation problems and the incidence of Hirschsprung-associated enterocolitis. The results from the pediatric incontinence and constipation scoring system indicated a positive correlation between better postoperative bowel function and lower histopathologic grade of ganglion cells.

CONCLUSIONS

This pilot study showed an association between the histopathologic features of ganglion cells in the proximal surgical margin and the clinical outcome. It may provide additional information for intraoperative pathologic consultation in leveling biopsy to prevent insufficient resection of the affected colon. A prospective study is warranted to validate these findings before clinical application.

A prospective observational multicentric clinical trial to evaluate microscopic examination of acid-fast bacilli in sputum by artificial intelligence-based microscopy system

Microscopy-based tuberculosis (TB) diagnosis i.e., Ziehl-Neelsen (ZN) stained smear screening still remains the primary diagnostic method in resource poor and high TB burden countries, however itrequires considerable experience and is bound to human errors. In remote areas, wherever expert microscopist is not available, timely diagnosis at initial level is not possible. Artificial intelligence (AI)-based microscopy may be a solution to this problem. A prospective observational multi-centric clinical trial to evaluate microscopic examination of acid-fast bacilli (AFB) in sputum by the AI based system was done in three hospitals in Northern India. Sputum samples from 400 clinically suspected cases of pulmonary tuberculosis were collected from three centres. Ziehl-Neelsen staining of smears was done. All the smears were observed by 3 microscopist and the AI based microscopy system. AI based microscopy was found to have a sensitivity, specificity, positive predictive value, negative predictive value and diagnostic accuracy of 89.25%, 92.15%, 75.45%, 96.94%, 91.53% respectively. AI based sputum microscopy has an acceptable degree of accuracy, PPV, NPV, specificity and sensitivity and thus may be used as a screening tool for the diagnosis of pulmonary tuberculosis.

Attention-based generative adversarial networks improve prognostic outcome prediction of cancer from multimodal data

The prediction of prognostic outcome is critical for the development of efficient cancer therapeutics and potential personalized medicine. However, due to the heterogeneity and diversity of multimodal data of cancer, data integration and feature selection remain a challenge for prognostic outcome prediction. We proposed a deep learning method with generative adversarial network based on sequential channel-spatial attention modules (CSAM-GAN), a multimodal data integration and feature selection approach, for accomplishing prognostic stratification tasks in cancer. Sequential channel-spatial attention modules equipped with an encoder-decoder are applied for the input features of multimodal data to accurately refine selected features. A discriminator network was proposed to make the generator and discriminator learning in an adversarial way to accurately describe the complex heterogeneous information of multiple modal data. We conducted extensive experiments with various feature selection and classification methods and confirmed that the CSAM-GAN via the multilayer deep neural network (DNN) classifier outperformed these baseline methods on two different multimodal data sets with miRNA expression, mRNA expression and histopathological image data: lower-grade glioma and kidney renal clear cell carcinoma. The CSAM-GAN via the multilayer DNN classifier bridges the gap between heterogenous multimodal data and prognostic outcome prediction.

Enhancing Kidney Transplant Care through the Integration of Chatbot

Kidney transplantation is a critical treatment option for end-stage kidney disease patients, offering improved quality of life and increased survival rates. However, the complexities of kidney transplant care necessitate continuous advancements in decision making, patient communication, and operational efficiency. This article explores the potential integration of a sophisticated chatbot, an AI-powered conversational agent, to enhance kidney transplant practice and potentially improve patient outcomes. Chatbots and generative AI have shown promising applications in various domains, including healthcare, by simulating human-like interactions and generating contextually appropriate responses. Noteworthy AI models like ChatGPT by OpenAI, BingChat by Microsoft, and Bard AI by Google exhibit significant potential in supporting evidence-based research and healthcare decision making. The integration of chatbots in kidney transplant care may offer transformative possibilities. As a clinical decision support tool, it could provide healthcare professionals with real-time access to medical literature and guidelines, potentially enabling informed decision making and improved knowledge dissemination. Additionally, the chatbot has the potential to facilitate patient education by offering personalized and understandable information, addressing queries, and providing guidance on post-transplant care. Furthermore, under clinician or transplant pharmacist supervision, it has the potential to support post-transplant care and medication management by analyzing patient data, which may lead to tailored recommendations on dosages, monitoring schedules, and potential drug interactions. However, to fully ascertain its effectiveness and safety in these roles, further studies and validation are required. Its integration with existing clinical decision support systems may enhance risk stratification and treatment planning, contributing to more informed and efficient decision making in kidney transplant care. Given the importance of ethical considerations and bias mitigation in AI integration, future studies may evaluate long-term patient outcomes, cost-effectiveness, user experience, and the generalizability of chatbot recommendations. By addressing these factors and potentially leveraging AI capabilities, the integration of chatbots in kidney transplant care holds promise for potentially improving patient outcomes, enhancing decision making, and fostering the equitable and responsible use of AI in healthcare.

Artificial Intelligence in Digital Pathology for Bladder Cancer: Hype or Hope? A Systematic Review

Bladder cancer (BC) diagnosis and prediction of prognosis are hindered by subjective pathological evaluation, which may cause misdiagnosis and under-/over-treatment. Computational pathology (CPATH) can identify clinical outcome predictors, offering an objective approach to improve prognosis. However, a systematic review of CPATH in BC literature is lacking. Therefore, we present a comprehensive overview of studies that used CPATH in BC, analyzing 33 out of 2285 identified studies. Most studies analyzed regions of interest to distinguish normal versus tumor tissue and identify tumor grade/stage and tissue types (e.g., urothelium, stroma, and muscle). The cell's nuclear area, shape irregularity, and roundness were the most promising markers to predict recurrence and survival based on selected regions of interest, with >80% accuracy. CPATH identified molecular subtypes by detecting features, e.g., papillary structures, hyperchromatic, and pleomorphic nuclei. Combining clinicopathological and image-derived features improved recurrence and survival prediction. However, due to the lack of outcome interpretability and independent test datasets, robustness and clinical applicability could not be ensured. The current literature demonstrates that CPATH holds the potential to improve BC diagnosis and prediction of prognosis. However, more robust, interpretable, accurate models and larger datasets-representative of clinical scenarios-are needed to address artificial intelligence's reliability, robustness, and black box challenge.

Comparison of deep learning-based recurrence-free survival with random survival forest and Cox proportional hazard models in Stage-I NSCLC patients

BACKGROUND

The curative treatment for Stage I non-small cell lung cancer (NSCLC) is surgical resection. Even for Stage I patients, the probability of recurrence after curative treatment is around 20%.

METHODS

In this retrospective study, we included 268 operated Stage I NSCLC patients between January 2008 and June 2018 to analyze the prognostic factors (pathological stage, histological type, number of sampled mediastinal lymph node stations, type of resection, SUVmax of the lesion) that may affect relapse with three different methods, Cox proportional hazard (CoxPH), random survival forest (RSF), DeepSurv, and to compare the performance of these methods with Harrell's C-index. The dataset was randomly split into two sets, training and test sets.

RESULTS

In the training set, DeepSurv showed the best performance among the three models, the C-index of the training set was 0.832, followed by RSF (0.675) and CoxPH (0.672). In the test set, RSF showed the best performance among the three models, followed by DeepSurv with 0.677 and CoxPH methods with 0.625.

CONCLUSION

In conclusion, machine-learning techniques can be useful in predicting recurrence for lung cancer and guide clinicians both in choosing the adjuvant treatment options and best follow-up programs.

Machine learning in computational histopathology: Challenges and opportunities

Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.

Bibliometric analysis of the global scientific production on machine learning applied to different cancer types

Cancer disease is one of the main causes of death in the world, with million annual cases in the last decades. The need to find a cure has stimulated the search for efficient treatments and diagnostic procedures. One of the most promising tools that has emerged against cancer in recent years is machine learning (ML), which has raised a huge number of scientific papers published in a relatively short period of time. The present study analyzes global scientific production on ML applied to the most relevant cancer types through various bibliometric indicators. We find that over 30,000 studies have been published so far and observe that cancers with the highest number of published studies using ML (breast, lung, and colon cancer) are those with the highest incidence, being the USA and China the main scientific producers on the subject. Interestingly, the role of China and Japan in stomach cancer is correlated with the number of cases of this cancer type in Asia (78% of the worldwide cases). Knowing the countries and institutions that most study each area can be of great help for improving international collaborations between research groups and countries. Our analysis shows that medical and computer science journals lead the number of publications on the subject and could be useful for researchers in the field. Finally, keyword co-occurrence analysis suggests that ML-cancer research trends are focused not only on the use of ML as an effective diagnostic method, but also for the improvement of radiotherapy- and chemotherapy-based treatments.

FAM3L: Feature-Aware Multi-Modal Metric Learning for Integrative Survival Analysis of Human Cancers

Survival analysis is to estimate the survival time for an individual or a group of patients, which is a valid solution for cancer treatments. Recent studies suggested that the integrative analysis of histopathological images and genomic data can better predict the survival of cancer patients than simply using single bio-marker, for different bio-markers may provide complementary information. However, for the given multi-modal data that may contain irrelevant or redundant features, it is still challenge to design a distance metric that can simultaneously discover significant features and measure the difference of survival time among different patients. To solve this issue, we propose a Feature-Aware Multi-modal Metric Learning method (FAM3L), which not only learns the metric for distance constraints on patients' survival time, but also identifies important images and genomic features for survival analysis. Specifically, for each modality of data, we firstly design one feature-aware metric that can be decoupled into a traditional distance metric and a diagonal weight for important feature identification. Then, in order to explore the complex correlation across multiple modality data, we apply Hilbert-Schmidt Independence Criterion (HSIC) to jointly learn multiple metrics. Finally, based on the learned distance metrics, we apply the Cox proportional hazards model for prognosis prediction. We evaluate the performance of our proposed FAM3L method on three cancer cohorts derived from The Cancer Genome Atlas (TCGA), the experimental results demonstrate that our method can not only achieve superior performance for cancer prognosis, but also identify meaningful image and genomic features correlating strongly with cancer survival.

Novel tools for early diagnosis and precision treatment based on artificial intelligence

Lung cancer has the highest mortality rate among all cancers in the world. Hence, early diagnosis and personalized treatment plans are crucial to improving its 5-year survival rate. Chest computed tomography (CT) serves as an essential tool for lung cancer screening, and pathology images are the gold standard for lung cancer diagnosis. However, medical image evaluation relies on manual labor and suffers from missed diagnosis or misdiagnosis, and physician heterogeneity. The rapid development of artificial intelligence (AI) has brought a whole novel opportunity for medical task processing, demonstrating the potential for clinical application in lung cancer diagnosis and treatment. AI technologies, including machine learning and deep learning, have been deployed extensively for lung nodule detection, benign and malignant classification, and subtype identification based on CT images. Furthermore, AI plays a role in the non-invasive prediction of genetic mutations and molecular status to provide the optimal treatment regimen, and applies to the assessment of therapeutic efficacy and prognosis of lung cancer patients, enabling precision medicine to become a reality. Meanwhile, histology-based AI models assist pathologists in typing, molecular characterization, and prognosis prediction to enhance the efficiency of diagnosis and treatment. However, the leap to extensive clinical application still faces various challenges, such as data sharing, standardized label acquisition, clinical application regulation, and multimodal integration. Nevertheless, AI holds promising potential in the field of lung cancer to improve cancer care.

Integrative models of histopathological images and multi-omics data predict prognosis in endometrial carcinoma

Objective

This study aimed to predict the molecular features of endometrial carcinoma (EC) and the overall survival (OS) of EC patients using histopathological imaging.

Methods

The patients from The Cancer Genome Atlas (TCGA) were separated into the training set (n = 215) and test set (n = 214) in proportion of 1:1. By analyzing quantitative histological image features and setting up random forest model verified by cross-validation, we constructed prognostic models for OS. The model performance is evaluated with the time-dependent receiver operating characteristics (AUC) over the test set.

Results

Prognostic models based on histopathological imaging features (HIF) predicted OS in the test set (5-year AUC = 0.803). The performance of combining histopathology and omics transcends that of genomics, transcriptomics, or proteomics alone. Additionally, multi-dimensional omics data, including HIF, genomics, transcriptomics, and proteomics, attained the largest AUCs of 0.866, 0.869, and 0.856 at years 1, 3, and 5, respectively, showcasing the highest discrepancy in survival (HR = 18.347, 95% CI [11.09-25.65], p < 0.001).

Conclusions

The results of this experiment indicated that the complementary features of HIF could improve the prognostic performance of EC patients. Moreover, the integration of HIF and multi-dimensional omics data might ameliorate survival prediction and risk stratification in clinical practice.

Machine learning for cryosection pathology predicts the 2021 WHO classification of glioma

BACKGROUND

Timely and accurate intraoperative cryosection evaluations remain the gold standard for guiding surgical treatments for gliomas. However, the tissue-freezing process often generates artifacts that make histologic interpretation difficult. In addition, the 2021 WHO Classification of Tumors of the Central Nervous System incorporates molecular profiles in the diagnostic categories, so standard visual evaluation of cryosections alone cannot completely inform diagnoses based on the new classification system.

METHODS

To address these challenges, we develop the context-aware Cryosection Histopathology Assessment and Review Machine (CHARM) using samples from 1,524 glioma patients from three different patient populations to systematically analyze cryosection slides.

FINDINGS

Our CHARM models successfully identified malignant cells (AUROC = 0.98 ± 0.01 in the independent validation cohort), distinguished isocitrate dehydrogenase (IDH)-mutant tumors from wild type (AUROC = 0.79-0.82), classified three major types of molecularly defined gliomas (AUROC = 0.88-0.93), and identified the most prevalent subtypes of IDH-mutant tumors (AUROC = 0.89-0.97). CHARM further predicts clinically important genetic alterations in low-grade glioma, including ATRX, TP53, and CIC mutations, CDKN2A/B homozygous deletion, and 1p/19q codeletion via cryosection images.

CONCLUSIONS

Our approaches accommodate the evolving diagnostic criteria informed by molecular studies, provide real-time clinical decision support, and will democratize accurate cryosection diagnoses.

FUNDING

Supported in part by the National Institute of General Medical Sciences grant R35GM142879, the Google Research Scholar Award, the Blavatnik Center for Computational Biomedicine Award, the Partners' Innovation Discovery Grant, and the Schlager Family Award for Early Stage Digital Health Innovations.

E2EFP-MIL: End-to-end and high-generalizability weakly supervised deep convolutional network for lung cancer classification from whole slide image

Efficient and accurate distinction of histopathological subtype of lung cancer is quite critical for the individualized treatment. So far, artificial intelligence techniques have been developed, whose performance yet remained debatable on more heterogenous data, hindering their clinical deployment. Here, we propose an end-to-end, well-generalized and data-efficient weakly supervised deep learning-based method. The method, end-to-end feature pyramid deep multi-instance learning model (E2EFP-MIL), contains an iterative sampling module, a trainable feature pyramid module and a robust feature aggregation module. E2EFP-MIL uses end-to-end learning to extract generalized morphological features automatically and identify discriminative histomorphological patterns. This method is trained with 1007 whole slide images (WSIs) of lung cancer from TCGA, with AUCs of 0.95-0.97 in test sets. We validated E2EFP-MIL in 5 real-world external heterogenous cohorts including nearly 1600 WSIs from both United States and China with AUCs of 0.94-0.97, and found that 100-200 training images are enough to achieve an AUC of >0.9. E2EFP-MIL overperforms multiple state-of-the-art MIL-based methods with high accuracy and low hardware requirements. Excellent and robust results prove generalizability and effectiveness of E2EFP-MIL in clinical practice. Our code is available at https://github.com/raycaohmu/E2EFP-MIL.

Recent advances of pathomics in colorectal cancer diagnosis and prognosis

Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.

Deep Learning of Cell Spatial Organizations Identifies Clinically Relevant Insights in Tissue Images

Recent advancements in tissue imaging techniques have facilitated the visualization and identification of various cell types within physiological and pathological contexts. Despite the emergence of cell-cell interaction studies, there is a lack of methods for evaluating individual spatial interactions. In this study, we introduce Ceograph, a novel cell spatial organization-based graph convolutional network designed to analyze cell spatial organization (i.e. the cell spatial distribution, morphology, proximity, and interactions) derived from pathology images. Ceograph identifies key cell spatial organization features by accurately predicting their influence on patient clinical outcomes. In patients with oral potentially malignant disorders, our model highlights reduced structural concordance and increased closeness in epithelial substrata as driving features for an elevated risk of malignant transformation. In lung cancer patients, Ceograph detects elongated tumor nuclei and diminished stroma-stroma closeness as biomarkers for insensitivity to EGFR tyrosine kinase inhibitors. With its potential to predict various clinical outcomes, Ceograph offers a deeper understanding of biological processes and supports the development of personalized therapeutic strategies.

A Novel Hybrid Approach for Classifying Osteosarcoma Using Deep Feature Extraction and Multilayer Perceptron

Osteosarcoma is the most common type of bone cancer that tends to occur in teenagers and young adults. Due to crowded context, inter-class similarity, inter-class variation, and noise in H&E-stained (hematoxylin and eosin stain) histology tissue, pathologists frequently face difficulty in osteosarcoma tumor classification. In this paper, we introduced a hybrid framework for improving the efficiency of three types of osteosarcoma tumor (nontumor, necrosis, and viable tumor) classification by merging different types of CNN-based architectures with a multilayer perceptron (MLP) algorithm on the WSI (whole slide images) dataset. We performed various kinds of preprocessing on the WSI images. Then, five pre-trained CNN models were trained with multiple parameter settings to extract insightful features via transfer learning, where convolution combined with pooling was utilized as a feature extractor. For feature selection, a decision tree-based RFE was designed to recursively eliminate less significant features to improve the model generalization performance for accurate prediction. Here, a decision tree was used as an estimator to select the different features. Finally, a modified MLP classifier was employed to classify binary and multiclass types of osteosarcoma under the five-fold CV to assess the robustness of our proposed hybrid model. Moreover, the feature selection criteria were analyzed to select the optimal one based on their execution time and accuracy. The proposed model achieved an accuracy of 95.2% for multiclass classification and 99.4% for binary classification. Experimental findings indicate that our proposed model significantly outperforms existing methods; therefore, this model could be applicable to support doctors in osteosarcoma diagnosis in clinics. In addition, our proposed model is integrated into a web application using the FastAPI web framework to provide a real-time prediction.

Deep learning classification of uveal melanoma based on histopathological images and identification of a novel indicator for prognosis of patients

BACKGROUND

Deep learning has been extensively used in digital histopathology. The purpose of this study was to test deep learning (DL) algorithms for predicting the vital status of whole-slide image (WSI) of uveal melanoma (UM).

METHODS

We developed a deep learning model (Google-net) to predict the vital status of UM patients from histopathological images in TCGA-UVM cohort and validated it in an internal cohort. The histopathological DL features extracted from the model and then were applied to classify UM patients into two subtypes. The differences between two subtypes in clinical outcomes, tumor mutation, and microenvironment, and probability of drug therapeutic response were investigated further.

RESULTS

We observed that the developed DL model can achieve a high accuracy of >  = 90% for patches and WSIs prediction. Using 14 histopathological DL features, we successfully classified UM patients into Cluster1 and Cluster2 subtypes. Compared to Cluster2, patients in the Cluster1 subtype have a poor survival outcome, increased expression levels of immune-checkpoint genes, higher immune-infiltration of CD8 + T cell and CD4 + T cells, and more sensitivity to anti-PD-1 therapy. Besides, we established and verified prognostic histopathological DL-signature and gene-signature which outperformed the traditional clinical features. Finally, a well-performed nomogram combining the DL-signature and gene-signature was constructed to predict the mortality of UM patients.

CONCLUSIONS

Our findings suggest that DL model can accurately predict vital status in UM patents just using histopathological images. We found out two subgroups based on histopathological DL features, which may in favor of immunotherapy and chemotherapy. Finally, a well-performing nomogram that combines DL-signature and gene-signature was constructed to give a more straightforward and reliable prognosis for UM patients in treatment and management.

Predicting the prognosis of HER2-positive breast cancer patients by fusing pathological whole slide images and clinical features using multiple instance learning

In 2022, breast cancer will become an important factor affecting women's public health and HER2 positivity for approximately 15-20$ \% $ invasive breast cancer cases. Follow-up data for HER2-positive patients are rare, and research on prognosis and auxiliary diagnosis is still limited. In light of the findings obtained from the analysis of clinical features, we have developed a novel multiple instance learning (MIL) fusion model that integrates hematoxylin-eosin (HE) pathological images and clinical features to accurately predict the prognostic risk of patients. Specifically, we segmented the HE pathology images of patients into patches, clustered them by K-means, aggregated them into a bag feature-level representation through graph attention networks (GATs) and multihead attention networks, and fused them with clinical features to predict the prognosis of patients. We divided West China Hospital (WCH) patients (n = 1069) into a training cohort and internal validation cohort and used The Cancer Genome Atlas (TCGA) patients (n = 160) as an external test cohort. The 3-fold average C-index of the proposed OS-based model was 0.668, the C-index of the WCH test set was 0.765, and the C-index of the TCGA independent test set was 0.726. By plotting the Kaplan-Meier curve, the fusion feature (P = 0.034) model distinguished high- and low-risk groups more accurately than clinical features (P = 0.19). The MIL model can directly analyze a large number of unlabeled pathological images, and the multimodal model is more accurate than the unimodal models in predicting Her2-positive breast cancer prognosis based on large amounts of data.

Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients

Histopathologic assessment is indispensable for diagnosing colorectal cancer (CRC). However, manual evaluation of the diseased tissues under the microscope cannot reliably inform patient prognosis or genomic variations crucial for treatment selections. To address these challenges, we develop the Multi-omics Multi-cohort Assessment (MOMA) platform, an explainable machine learning approach, to systematically identify and interpret the relationship between patients' histologic patterns, multi-omics, and clinical profiles in three large patient cohorts (n = 1888). MOMA successfully predicts the overall survival, disease-free survival (log-rank test P-value<0.05), and copy number alterations of CRC patients. In addition, our approaches identify interpretable pathology patterns predictive of gene expression profiles, microsatellite instability status, and clinically actionable genetic alterations. We show that MOMA models are generalizable to multiple patient populations with different demographic compositions and pathology images collected from distinctive digitization methods. Our machine learning approaches provide clinically actionable predictions that could inform treatments for colorectal cancer patients.

Recent Advances in Genome-Editing Technology with CRISPR/Cas9 Variants and Stimuli-Responsive Targeting Approaches within Tumor Cells: A Future Perspective of Cancer Management

The innovative advances in transforming clustered regularly interspaced short palindromic repeats-associated protein 9 (CRISPR/Cas9) into different variants have taken the art of genome-editing specificity to new heights. Allosteric modulation of Cas9-targeting specificity by sgRNA sequence alterations and protospacer adjacent motif (PAM) modifications have been a good lesson to learn about specificity and activity scores in different Cas9 variants. Some of the high-fidelity Cas9 variants have been ranked as Sniper-Cas9, eSpCas9 (1.1), SpCas9-HF1, HypaCas9, xCas9, and evoCas9. However, the selection of an ideal Cas9 variant for a given target sequence remains a challenging task. A safe and efficient delivery system for the CRISPR/Cas9 complex at tumor target sites faces considerable challenges, and nanotechnology-based stimuli-responsive delivery approaches have significantly contributed to cancer management. Recent innovations in nanoformulation design, such as pH, glutathione (GSH), photo, thermal, and magnetic responsive systems, have modernized the art of CRISPR/Cas9 delivery approaches. These nanoformulations possess enhanced cellular internalization, endosomal membrane disruption/bypass, and controlled release. In this review, we aim to elaborate on different CRISPR/Cas9 variants and advances in stimuli-responsive nanoformulations for the specific delivery of this endonuclease system. Furthermore, the critical constraints of this endonuclease system on clinical translations towards the management of cancer and prospects are described.

GraphLSurv: A scalable survival prediction network with adaptive and sparse structure learning for histopathological whole-slide images

BACKGROUND AND OBJECTIVE

Predicting patients' survival from gigapixel Whole-Slide Images (WSIs) has always been a challenging task. To learn effective WSI representations for survival prediction, existing deep learning methods have explored utilizing graphs to describe the complex structure inner WSIs, where graph node is respective to WSI patch. However, these graphs are often densely-connected or static, leading to some redundant or missing patch correlations. Moreover, these methods cannot be directly scaled to the very-large WSI with more than 10,000 patches. To address these, this paper proposes a scalable graph convolution network, GraphLSurv, which can efficiently learn adaptive and sparse structures to better characterize WSIs for survival prediction.

METHODS

GraphLSurv has three highlights in methodology: (1) it generates adaptive and sparse structures for patches so that latent patch correlations could be captured and adjusted dynamically according to prediction tasks; (2) based on the generated structure and a given graph, GraphLSurv further aggregates local microenvironmental cues into a non-local embedding using the proposed hybrid message passing network; (3) to make this network suitable for very large-scale graphs, it adopts an anchor-based technique to reduce theorical computation complexity.

RESULTS

The experiments on 2268 WSIs show that GraphLSurv achieves a concordance-index of 0.66132 and 0.68348, with an improvement of 3.79% and 3.41% compared to existing methods, on NLST and TCGA-BRCA, respectively.

CONCLUSIONS

GraphLSurv could often perform better than previous methods, which suggests that GraphLSurv could provide an important and effective means for WSI survival prediction. Moreover, this work empirically shows that adaptive and sparse structures could be more suitable than static or dense ones for modeling WSIs.

Early Triage of Critically Ill Adult Patients With Mushroom Poisoning: Machine Learning Approach

BACKGROUND

Early triage of patients with mushroom poisoning is essential for administering precise treatment and reducing mortality. To our knowledge, there has been no established method to triage patients with mushroom poisoning based on clinical data.

OBJECTIVE

The purpose of this work was to construct a triage system to identify patients with mushroom poisoning based on clinical indicators using several machine learning approaches and to assess the prediction accuracy of these strategies.

METHODS

In all, 567 patients were collected from 5 primary care hospitals and facilities in Enshi, Hubei Province, China, and divided into 2 groups; 322 patients from 2 hospitals were used as the training cohort, and 245 patients from 3 hospitals were used as the test cohort. Four machine learning algorithms were used to construct the triage model for patients with mushroom poisoning. Performance was assessed using the area under the receiver operating characteristic curve (AUC), decision curve, sensitivity, specificity, and other representative statistics. Feature contributions were evaluated using Shapley additive explanations.

RESULTS

Among several machine learning algorithms, extreme gradient boosting (XGBoost) showed the best discriminative ability in 5-fold cross-validation (AUC=0.83, 95% CI 0.77-0.90) and the test set (AUC=0.90, 95% CI 0.83-0.96). In the test set, the XGBoost model had a sensitivity of 0.93 (95% CI 0.81-0.99) and a specificity of 0.79 (95% CI 0.73-0.85), whereas the physicians' assessment had a sensitivity of 0.86 (95% CI 0.72-0.95) and a specificity of 0.66 (95% CI 0.59-0.73).

CONCLUSIONS

The 14-factor XGBoost model for the early triage of mushroom poisoning can rapidly and accurately identify critically ill patients and will possibly serve as an important basis for the selection of treatment options and referral of patients, potentially reducing patient mortality and improving clinical outcomes.

Predicting peritoneal recurrence in gastric cancer with serosal invasion using a pathomics nomogram

Peritoneal recurrence is the most frequent and lethal recurrence pattern in gastric cancer (GC) with serosal invasion after radical surgery. However, current evaluation methods are not adequate for predicting peritoneal recurrence in GC with serosal invasion. Emerging evidence shows that pathomics analyses could be advantageous for risk stratification and outcome prediction. Herein, we propose a pathomics signature composed of multiple pathomics features extracted from digital hematoxylin and eosin-stained images. We found that the pathomics signature was significantly associated with peritoneal recurrence. A competing-risk pathomics nomogram including carbohydrate antigen 19-9 level, depth of invasion, lymph node metastasis, and pathomics signature was developed for predicting peritoneal recurrence. The pathomics nomogram had favorable discrimination and calibration. Thus, the pathomics signature is a predictive indicator of peritoneal recurrence, and the pathomics nomogram may provide a helpful reference for predicting an individual's risk in peritoneal recurrence of GC with serosal invasion.

An automatic entropy method to efficiently mask histology whole-slide images

Tissue segmentation of histology whole-slide images (WSI) remains a critical task in automated digital pathology workflows for both accurate disease diagnosis and deep phenotyping for research purposes. This is especially challenging when the tissue structure of biospecimens is relatively porous and heterogeneous, such as for atherosclerotic plaques. In this study, we developed a unique approach called 'EntropyMasker' based on image entropy to tackle the fore- and background segmentation (masking) task in histology WSI. We evaluated our method on 97 high-resolution WSI of human carotid atherosclerotic plaques in the Athero-Express Biobank Study, constituting hematoxylin and eosin and 8 other staining types. Using multiple benchmarking metrics, we compared our method with four widely used segmentation methods: Otsu's method, Adaptive mean, Adaptive Gaussian and slideMask and observed that our method had the highest sensitivity and Jaccard similarity index. We envision EntropyMasker to fill an important gap in WSI preprocessing, machine learning image analysis pipelines, and enable disease phenotyping beyond the field of atherosclerosis.

Improving the robustness and stability of a machine learning model for breast cancer prognosis through the use of multi-modal classifiers

Breast cancer is a deadly disease with a high mortality rate among PAN cancers. The advancements in biomedical information retrieval techniques have been beneficial in developing early prognosis and diagnosis systems for cancer patients. These systems provide the oncologist with plenty of information from several modalities to make the correct and feasible treatment plan for breast cancer patients and protect them from unnecessary therapies and their toxic side effects. The cancer patient's related information can be collected using various modalities like clinical, copy number variation, DNA-methylation, microRNA sequencing, gene expression, and histopathological whole slide images. High dimensionality and heterogeneity in these modalities demand the development of some intelligent systems to understand related features to the prognosis and diagnosis of diseases and make correct predictions. In this work, we have studied some end-to-end systems having two main components : (a) dimensionality reduction techniques applied to original features from different modalities and (b) classification techniques applied to the fusion of reduced feature vectors from different modalities for automatic predictions of breast cancer patients into two categories: short-time and long-time survivors. Principal component analysis (PCA) and variational auto-encoders (VAEs) are used as the dimensionality reduction techniques, followed by support vector machines (SVM) or random forest as the machine learning classifiers. The study utilizes raw, PCA, and VAE extracted features of the TCGA-BRCA dataset from six different modalities as input to the machine learning classifiers. We conclude this study by suggesting that adding more modalities to the classifiers provides complementary information to the classifier and increases the stability and robustness of the classifiers. In this study, the multimodal classifiers have not been validated on primary data prospectively.

Modelling blood flow in patients with heart valve disease using deep learning: A computationally efficient method to expand diagnostic capabilities in clinical routine

Introduction

The computational modelling of blood flow is known to provide vital hemodynamic parameters for diagnosis and treatment-support for patients with valvular heart disease. However, most diagnosis/treatment-support solutions based on flow modelling proposed utilize time- and resource-intensive computational fluid dynamics (CFD) and are therefore difficult to implement into clinical practice. In contrast, deep learning (DL) algorithms provide results quickly with little need for computational power. Thus, modelling blood flow with DL instead of CFD may substantially enhances the usability of flow modelling-based diagnosis/treatment support in clinical routine. In this study, we propose a DL-based approach to compute pressure and wall-shear-stress (WSS) in the aorta and aortic valve of patients with aortic stenosis (AS).

Methods

A total of 103 individual surface models of the aorta and aortic valve were constructed from computed tomography data of AS patients. Based on these surface models, a total of 267 patient-specific, steady-state CFD simulations of aortic flow under various flow rates were performed. Using this simulation data, an artificial neural network (ANN) was trained to compute spatially resolved pressure and WSS using a centerline-based representation. An unseen test subset of 23 cases was used to compare both methods.

Results

ANN and CFD-based computations agreed well with a median relative difference between both methods of 6.0% for pressure and 4.9% for wall-shear-stress. Demonstrating the ability of DL to compute clinically relevant hemodynamic parameters for AS patients, this work presents a possible solution to facilitate the introduction of modelling-based treatment support into clinical practice.

A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images

Background and objectives

The detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature.

Methods

A dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves.

Results

21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique.

Conclusions

The presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.

Proposal of SVM Utility Kernel for Breast Cancer Survival Estimation

The advancement of medical research in the field of cancer prognosis and diagnosis using various modalities has put oncologists under tremendous stress. The complexity and heterogeneity involved in multiple modalities and their significantly varied clinical outcomes make it difficult to analyze the disease and provide the correct treatment. Breast cancer is the major concern among all cancers worldwide, specifically for females. To help oncologists and cancer patients, research for breast cancer survival estimation has been proposed. It ranges from complex deep neural networks to simple and interpretable architectures. We propose a utility kernel for a support vector machine (SVM) in this article. It is a simple yet powerful function, which performs better than other popular machine learning algorithms and deep neural networks in the task of breast cancer survival prediction using the TCGA-BRCA dataset. This study validates the proposed utility kernel using four different modalities (gene expression, copy number variation, clinical, and histopathological tissue images) and their multi-modal combinations. The SVM based on our utility kernel empirically proves its efficacy by achieving the highest value on various performance measures, whereas advanced deep neural networks fail to train on small and highly imbalanced breast cancer data.

Development and Interpretation of a Clinicopathological-Based Model for the Identification of Microsatellite Instability in Colorectal Cancer

Chemotherapy is not recommended for patients with deficient mismatch repair (dMMR) in colorectal cancer (CRC); therefore, assessing the status of MMR is crucial for the selection of subsequent treatment. This study is aimed at building predictive models to accurately and rapidly identify dMMR. A retrospective analysis was performed at Wuhan Union Hospital between May 2017 and December 2019 based on the clinicopathological data of patients with CRC. The variables were subjected to collinearity, least absolute shrinkage and selection operator (LASSO) regression, and random forest (RF) feature screening analyses. Four sets of machine learning models (extreme gradient boosting (XGBoost), support vector machine (SVM), naive Bayes (NB), and RF) and a conventional logistic regression (LR) model were built for model training and testing. Receiver operating characteristic (ROC) curves were plotted to evaluate the predictive performance of the developed models. In total, 2279 patients were included in the study and were randomly divided into either the training or test group. Twelve clinicopathological features were incorporated into the development of the predictive models. The area under curve (AUC) values of the five predictive models were 0.8055 for XGBoost, 0.8174 for SVM, 0.7424 for NB, 8584 for RF, and 0.7835 for LR (Delong test, P value < 0.05). The results showed that the RF model exhibited the best recognition ability and outperformed the conventional LR method in identifying dMMR and proficient MMR (pMMR). Our predictive models based on routine clinicopathological data can significantly improve the diagnostic performance of dMMR and pMMR. The four machine learning models outperformed the conventional LR model.

A Heuristic Machine Learning-Based Optimization Technique to Predict Lung Cancer Patient Survival

Cancer has been a significant threat to human health and well-being, posing the biggest obstacle in the history of human sickness. The high death rate in cancer patients is primarily due to the complexity of the disease and the wide range of clinical outcomes. Increasing the accuracy of the prediction is equally crucial as predicting the survival rate of cancer patients, which has become a key issue of cancer research. Many models have been suggested at the moment. However, most of them simply use single genetic data or clinical data to construct prediction models for cancer survival. There is a lot of emphasis in present survival studies on determining whether or not a patient will survive five years. The personal issue of how long a lung cancer patient will survive remains unanswered. The proposed technique Naive Bayes and SSA is estimating the overall survival time with lung cancer. Two machine learning challenges are derived from a single customized query. To begin with, determining whether a patient will survive for more than five years is a simple binary question. The second step is to develop a five-year survival model using regression analysis. When asked to forecast how long a lung cancer patient would survive within five years, the mean absolute error (MAE) of this technique's predictions is accurate within a month. Several biomarker genes have been associated with lung cancers. The accuracy, recall, and precision achieved from this algorithm are 98.78%, 98.4%, and 98.6%, respectively.

Self-supervised learning-based Multi-Scale feature Fusion Network for survival analysis from whole slide images

Understanding prognosis and mortality is critical for evaluating the treatment plan of patients. Advances in digital pathology and deep learning techniques have made it practical to perform survival analysis in whole slide images (WSIs). Current methods are usually based on a multi-stage framework which includes patch sampling, feature extraction and prediction. However, the random patch sampling strategy is highly unstable and prone to sampling non-ROI. Feature extraction typically relies on hand-crafted features or convolutional neural networks (CNNs) pre-trained on ImageNet, while the artificial error or domain gaps may affect the survival prediction performance. Besides, the limited information representation of local sampling patches will create a bottleneck limitation on the effectiveness of prediction. To address the above challenges, we propose a novel patch sampling strategy based on image information entropy and construct a Multi-Scale feature Fusion Network (MSFN) based on self-supervised feature extractor. Specifically, we adopt image information entropy as a criterion to select representative sampling patches, thereby avoiding the noise interference caused by random to blank regions. Meanwhile, we pretrain the feature extractor utilizing self-supervised learning mechanism to improve the efficiency of feature extraction. Furthermore, a global-local feature fusion prediction network based on the attention mechanism is constructed to improve the survival prediction effect of WSIs with comprehensive multi-scale information representation. The proposed method is validated by adequate experiments and achieves competitive results on both of the most popular WSIs survival analysis datasets, TCGA-GBM and TCGA-LUSC. Code and trained models are made available at: https://github.com/Mercuriiio/MSFN.

Artificial intelligence in lung cancer diagnosis and prognosis: Current application and future perspective

Lung cancer is one of the malignant tumors with the highest incidence and mortality in the world. The overall five-year survival rate of lung cancer is relatively lower than many leading cancers. Early diagnosis and prognosis of lung cancer are essential to improve the patient's survival rate. With artificial intelligence (AI) approaches widely applied in lung cancer, early diagnosis and prediction have achieved excellent performance in recent years. This review summarizes various types of AI algorithm applications in lung cancer, including natural language processing (NLP), machine learning and deep learning, and reinforcement learning. In addition, we provides evidence regarding the application of AI in lung cancer diagnostic and clinical prognosis. This review aims to elucidate the value of AI in lung cancer diagnosis and prognosis as the novel screening decision-making for the precise treatment of lung cancer patients.

Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy

Membranous nephropathy is one of the most prevalent conditions responsible for nephrotic syndrome in adults. It is clinically nonspecific and mainly diagnosed by kidney biopsy pathology, with three prevalent techniques: light microscopy, electron microscopy, and immunofluorescence microscopy. Manual observation of glomeruli one by one under the microscope is very time-consuming, and there are certain differences in the observation results between physicians. This study makes use of whole-slide images scanned by a light microscope as well as immunofluorescence images to classify patients with membranous nephropathy. The framework mainly includes a glomerular segmentation module, a confidence coefficient extraction module, and a multi-modal fusion module. This framework first identifies and segments the glomerulus from whole-slide images and immunofluorescence images, and then a glomerular classifier is trained to extract the features of each glomerulus. The results are then combined to produce the final diagnosis. The results of the experiments show that the F1-score of image classification results obtained by combining two kinds of features, which can reach 97.32%, is higher than those obtained by using only light-microscopy-observed images or immunofluorescent images, which reach 92.76% and 93.20%, respectively. Experiments demonstrate that considering both WSIs and immunofluorescence images is effective in improving the diagnosis of membranous nephropathy.

Non-Association of Driver Alterations in PTEN with Differential Gene Expression and Gene Methylation in IDH1 Wildtype Glioblastomas

During oncogenesis, alterations in driver genes called driver alterations (DAs) modulate the transcriptome, methylome and proteome through oncogenic signaling pathways. These modulatory effects of any DA may be analyzed by examining differentially expressed mRNAs (DEMs), differentially methylated genes (DMGs) and differentially expressed proteins (DEPs) between tumor samples with and without that DA. We aimed to analyze these modulations with 12 common driver genes in Isocitrate Dehydrogenase 1 wildtype glioblastomas (IDH1-W-GBs). Using Cbioportal, groups of tumor samples with and without DAs in these 12 genes were generated from the IDH1-W-GBs available from "The Cancer Genomics Atlas Firehose Legacy Study Group" (TCGA-FL-SG) on Glioblastomas (GBs). For all 12 genes, samples with and without DAs were compared for DEMs, DMGs and DEPs. We found that DAs in PTEN were unassociated with any DEM or DMG in contrast to DAs in all other drivers, which were associated with several DEMs and DMGs. This contrasting PTEN-related property of being unassociated with differential gene expression or methylation in IDH1-W-GBs was unaffected by concurrent DAs in other common drivers or by the types of DAs affecting PTEN. From the lists of DEMs and DMGs associated with some common drivers other than PTEN, enriched gene ontology terms and insights into the co-regulatory effects of these drivers on the transcriptome were obtained. The findings from this study can improve our understanding of the molecular mechanisms underlying gliomagenesis with potential therapeutic benefits.

Using deep learning to predict survival outcome in non-surgical cervical cancer patients based on pathological images

PURPOSE

We analyzed clinical features and the representative HE-stained pathologic images to predict 5-year overall survival via the deep-learning approach in cervical cancer patients in order to assist oncologists in designing the optimal treatment strategies.

METHODS

The research retrospectively collected 238 non-surgical cervical cancer patients treated with radiochemotherapy from 2014 to 2017. These patients were randomly divided into the training set (n = 165) and test set (n = 73). Then, we extract deep features after segmenting the HE-stained image into patches of size 224 × 224. A Lasso-Cox model was constructed with clinical data to predict 5-year OS. C-index evaluated this model performance with 95% CI, calibration curve, and ROC.

RESULTS

Based on multivariate analysis, 2 of 11 clinical characteristics (C-index 0.68) and 2 of 2048 pathomic features (C-index 0.74) and clinical-pathomic model (C-index 0.83) of nomograms predict 5-year survival in the training set, respectively. In test set, compared with the pathomic and clinical characteristics used alone, the clinical-pathomic model had an AUC of 0.750 (95% CI 0.540-0.959), the clinical predictor model had an AUC of 0.729 (95% CI 0.551-0.909), and the pathomic model AUC was 0.703 (95% CI 0.487-0.919). Based on appropriate nomogram scores, we divided patients into high-risk and low-risk groups, and Kaplan-Meier survival probability curves for both groups showed statistical differences.

CONCLUSION

We built a clinical-pathomic model to predict 5-year OS in non-surgical cervical cancer patients, which may be a promising method to improve the precision of personalized therapy.

Detection of Necrosis in Digitised Whole-Slide Images for Better Grading of Canine Soft-Tissue Sarcomas Using Machine-Learning

The definitive diagnosis of canine soft-tissue sarcomas (STSs) is based on histological assessment of formalin-fixed tissues. Assessment of parameters, such as degree of differentiation, necrosis score and mitotic score, give rise to a final tumour grade, which is important in determining prognosis and subsequent treatment modalities. However, grading discrepancies are reported to occur in human and canine STSs, which can result in complications regarding treatment plans. The introduction of digital pathology has the potential to help improve STS grading via automated determination of the presence and extent of necrosis. The detected necrotic regions can be factored in the grading scheme or excluded before analysing the remaining tissue. Here we describe a method to detect tumour necrosis in histopathological whole-slide images (WSIs) of STSs using machine learning. Annotated areas of necrosis were extracted from WSIs and the patches containing necrotic tissue fed into a pre-trained DenseNet161 convolutional neural network (CNN) for training, testing and validation. The proposed CNN architecture reported favourable results, with an overall validation accuracy of 92.7% for necrosis detection which represents the number of correctly classified data instances over the total number of data instances. The proposed method, when vigorously validated represents a promising tool to assist pathologists in evaluating necrosis in canine STS tumours, by increasing efficiency, accuracy and reducing inter-rater variation.

Inference of core needle biopsy whole slide images requiring definitive therapy for prostate cancer

BACKGROUND

Prostate cancer is often a slowly progressive indolent disease. Unnecessary treatments from overdiagnosis are a significant concern, particularly low-grade disease. Active surveillance has being considered as a risk management strategy to avoid potential side effects by unnecessary radical treatment. In 2016, American Society of Clinical Oncology (ASCO) endorsed the Cancer Care Ontario (CCO) Clinical Practice Guideline on active surveillance for the management of localized prostate cancer.

METHODS

Based on this guideline, we developed a deep learning model to classify prostate adenocarcinoma into indolent (applicable for active surveillance) and aggressive (necessary for definitive therapy) on core needle biopsy whole slide images (WSIs). In this study, we trained deep learning models using a combination of transfer, weakly supervised, and fully supervised learning approaches using a dataset of core needle biopsy WSIs (n=1300). In addition, we performed an inter-rater reliability evaluation on the WSI classification.

RESULTS

We evaluated the models on a test set (n=645), achieving ROC-AUCs of 0.846 for indolent and 0.980 for aggressive. The inter-rater reliability evaluation showed s-scores in the range of 0.10 to 0.95, with the lowest being on the WSIs with both indolent and aggressive classification by the model, and the highest on benign WSIs.

CONCLUSION

The results demonstrate the promising potential of deployment in a practical prostate adenocarcinoma histopathological diagnostic workflow system.

Application of digital pathology and machine learning in the liver, kidney and lung diseases

The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.

Deep learning-based predictions of clear and eosinophilic phenotypes in clear cell renal cell carcinoma

We have recently shown that histological phenotypes focusing on clear and eosinophilic cytoplasm in clear cell renal cell carcinoma (ccRCC) correlated with prognosis and the response to angiogenesis inhibition and checkpoint blockade. This study aims to objectively show the diagnostic utility of clear or eosinophilic phenotypes of ccRCC by developing an artificial intelligence (AI) model using the TCGA-ccRCC dataset and to demonstrate if the clear or eosinophilic predicted phenotypes correlate with pathological factors and gene signatures associated with angiogenesis and cancer immunity. Before the development of the AI model, histological evaluation using hematoxylin and eosin whole-slide images of the TCGA-ccRCC cohort (n = 435) was performed by a urologic pathologist. The AI model was developed as follows. First, the highest-grade area on each whole slide image was captured for image processing. Second, the selected regions were cropped into tiles. Third, the AI model was trained using transfer learning on a deep convolutional neural network, and clear or eosinophilic predictions were scaled as AI scores. Next, we verified the AI model using a validation cohort (n = 95). Finally, we evaluated the accuracy of the prognostic predictions of the AI model and revealed that the AI model detected clear and eosinophilic phenotypes with high accuracy. The AI model stratified the patients' outcomes, and the predicted eosinophilic phenotypes correlated with adverse clinicopathological characteristics and high immune-related gene signatures. In conclusion, the AI-based histologic subclassification accurately predicted clear or eosinophilic phenotypes of ccRCC, allowing for consistently reproducible stratification for prognostic and therapeutic stratification.

Lung Cancer Subtype Diagnosis by Fusing Image-Genomics Data and Hybrid Deep Networks

Accurate diagnosis of cancer subtypes is crucial for precise treatment, because different cancer subtypes are involved with different pathology and require different therapies. Although deep learning techniques have made great success in computer vision and other fields, they do not work well on Lung cancer subtype diagnosis, due to the distinction of slide images between different cancer subtypes is ambiguous. Furthermore, they often over-fit to high-dimensional genomics data with limited samples, and do not fuse the image and genomics data in a sensible way. In this paper, we propose a hybrid deep network based approach LungDIG for Lung cancer subtype Diagnosis by fusing Image-Genomics data. LungDIG first tiles the tissue slide image into small patches and extracts the patch-level features by fine-tuning an Inception-V3 model. Since the patches may contain some false positives in non-diagnostic regions, it further designs a patch-level feature combination strategy to integrate the extracted patch features and maintain the diversity between different cancer subtypes. At the same time, it extracts the genomics features from Copy Number Variation data by an attention based nonlinear extractor. Next, it fuses the image and genomics features by an attention based multilayer perceptron (MLP) to diagnose cancer subtype. Experiments on TCGA lung cancer data show that LungDIG can not only achieve higher accuracy for cancer subtype diagnosis than state-of-the-art methods, but also have a high authenticity and good interpretability.

Recurrence risk stratification for locally advanced cervical cancer using multi-modality transformer network

Objectives

Recurrence risk evaluation is clinically significant for patients with locally advanced cervical cancer (LACC). We investigated the ability of transformer network in recurrence risk stratification of LACC based on computed tomography (CT) and magnetic resonance (MR) images.

Methods

A total of 104 patients with pathologically diagnosed LACC between July 2017 and December 2021 were enrolled in this study. All patients underwent CT and MR scanning, and their recurrence status was identified by the biopsy. We randomly divided patients into training cohort (48 cases, non-recurrence: recurrence = 37: 11), validation cohort (21 cases, non-recurrence: recurrence = 16: 5), and testing cohort (35 cases, non-recurrence: recurrence = 27: 8), upon which we extracted 1989, 882 and 315 patches for model's development, validation and evaluation, respectively. The transformer network consisted of three modality fusion modules to extract multi-modality and multi-scale information, and a fully-connected module to perform recurrence risk prediction. The model's prediction performance was assessed by six metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, f1-score, sensitivity, specificity and precision. Univariate analysis with F-test and T-test were conducted for statistical analysis.

Results

The proposed transformer network is superior to conventional radiomics methods and other deep learning networks in both training, validation and testing cohorts. Particularly, in testing cohort, the transformer network achieved the highest AUC of 0.819 ± 0.038, while four conventional radiomics methods and two deep learning networks got the AUCs of 0.680 ± 0.050, 0.720 ± 0.068, 0.777 ± 0.048, 0.691 ± 0.103, 0.743 ± 0.022 and 0.733 ± 0.027, respectively.

Conclusions

The multi-modality transformer network showed promising performance in recurrence risk stratification of LACC and may be used as an effective tool to help clinicians make clinical decisions.

Anticancer activity of Pseudomonas aeruginosa derived peptide with iRGD in colon cancer therapy

Objectives

Colon cancer is well-known as a life-threatening disease. Since the current treatment modalities for this type of cancer are powerful yet face some limitations, finding novel treatments is required to achieve better outcomes with fewer side effects. Here we investigated the therapeutic potential of Azurin-p28 alone or along with iRGD (Ac-CRGDKGPDC-amide) as a tumor-penetrating peptide and 5-fluorouracil (5-FU) for colon cancer.

Materials and Methods

Inhibitory effect of p28 with or without iRGD/5-FU was studied in CT26 and HT29, as well as the xenograft animal model of cancer. The effect of p28 alone or along with iRGD/5-FU on cell migration, apoptotic activity, and cell cycle of the cell lines was assessed. Level of the BAX and BCL2 genes, tumor suppressor genes [(p53 and collagen type-Iα1 (COL1A1), collagen type-Iα2 (COL1A2)] were assessed by quantitative RT-PCR.

Results

These findings show that using p28 with or without iRGD and 5-FU raised the level of p53 and BAX but decreased BCL2, compared with control and 5-FU groups in tissues of the tumor, which result in raising the apoptosis.

Conclusion

It seems that p28 may be used as a new therapeutic approach in colon cancer therapy that can enhance the anti-tumor effect of 5-FU.

Multiclass prediction of different dementia syndromes based on multi-centric volumetric MRI imaging

INTRODUCTION

Dementia syndromes can be difficult to diagnose. We aimed at building a classifier for multiple dementia syndromes using magnetic resonance imaging (MRI).

METHODS

Atlas-based volumetry was performed on T1-weighted MRI data of 426 patients and 51 controls from the multi-centric German Research Consortium of Frontotemporal Lobar Degeneration including patients with behavioral variant frontotemporal dementia, Alzheimer's disease, the three subtypes of primary progressive aphasia, i.e., semantic, logopenic and nonfluent-agrammatic variant, and the atypical parkinsonian syndromes progressive supranuclear palsy and corticobasal syndrome. Support vector machine classification was used to classify each patient group against controls (binary classification) and all seven diagnostic groups against each other in a multi-syndrome classifier (multiclass classification).

RESULTS

The binary classification models reached high prediction accuracies between 71 and 95% with a chance level of 50%. Feature importance reflected disease-specific atrophy patterns. The multi-syndrome model reached accuracies of more than three times higher than chance level but was far from 100%. Multi-syndrome model performance was not homogenous across dementia syndromes, with better performance in syndromes characterized by regionally specific atrophy patterns. Whereas diseases generally could be classified vs controls more correctly with increasing severity and duration, differentiation between diseases was optimal in disease-specific windows of severity and duration.

DISCUSSION

Results suggest that automated methods applied to MR imaging data can support physicians in diagnosis of dementia syndromes. It is particularly relevant for orphan diseases beside frequent syndromes such as Alzheimer's disease.

CAMR: cross-aligned multimodal representation learning for cancer survival prediction

MOTIVATION

Accurately predicting cancer survival is crucial for helping clinicians to plan appropriate treatments, which largely improves the life quality of cancer patients and spares the related medical costs. Recent advances in survival prediction methods suggest that integrating complementary information from different modalities, e.g. histopathological images and genomic data, plays a key role in enhancing predictive performance. Despite promising results obtained by existing multimodal methods, the disparate and heterogeneous characteristics of multimodal data cause the so-called modality gap problem, which brings in dramatically diverse modality representations in feature space. Consequently, detrimental modality gaps make it difficult for comprehensive integration of multimodal information via representation learning and therefore pose a great challenge to further improvements of cancer survival prediction.

RESULTS

To solve the above problems, we propose a novel method called cross-aligned multimodal representation learning (CAMR), which generates both modality-invariant and -specific representations for more accurate cancer survival prediction. Specifically, a cross-modality representation alignment learning network is introduced to reduce modality gaps by effectively learning modality-invariant representations in a common subspace, which is achieved by aligning the distributions of different modality representations through adversarial training. Besides, we adopt a cross-modality fusion module to fuse modality-invariant representations into a unified cross-modality representation for each patient. Meanwhile, CAMR learns modality-specific representations which complement modality-invariant representations and therefore provides a holistic view of the multimodal data for cancer survival prediction. Comprehensive experiment results demonstrate that CAMR can successfully narrow modality gaps and consistently yields better performance than other survival prediction methods using multimodal data.

AVAILABILITY AND IMPLEMENTATION

CAMR is freely available at https://github.com/wxq-ustc/CAMR.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

The value of artificial intelligence in the diagnosis of lung cancer: A systematic review and meta-analysis

Lung cancer is a common malignant tumor disease with high clinical disability and death rates. Currently, lung cancer diagnosis mainly relies on manual pathology section analysis, but the low efficiency and subjective nature of manual film reading can lead to certain misdiagnoses and omissions. With the continuous development of science and technology, artificial intelligence (AI) has been gradually applied to imaging diagnosis. Although there are reports on AI-assisted lung cancer diagnosis, there are still problems such as small sample size and untimely data updates. Therefore, in this study, a large amount of recent data was included, and meta-analysis was used to evaluate the value of AI for lung cancer diagnosis. With the help of STATA16.0, the value of AI-assisted lung cancer diagnosis was assessed by specificity, sensitivity, negative likelihood ratio, positive likelihood ratio, diagnostic ratio, and plotting the working characteristic curves of subjects. Meta-regression and subgroup analysis were used to investigate the value of AI-assisted lung cancer diagnosis. The results of the meta-analysis showed that the combined sensitivity of the AI-aided diagnosis system for lung cancer diagnosis was 0.87 [95% CI (0.82, 0.90)], specificity was 0.87 [95% CI (0.82, 0.91)] (CI stands for confidence interval.), the missed diagnosis rate was 13%, the misdiagnosis rate was 13%, the positive likelihood ratio was 6.5 [95% CI (4.6, 9.3)], the negative likelihood ratio was 0.15 [95% CI (0.11, 0.21)], a diagnostic ratio of 43 [95% CI (24, 76)] and a sum of area under the combined subject operating characteristic (SROC) curve of 0.93 [95% CI (0.91, 0.95)]. Based on the results, the AI-assisted diagnostic system for CT (Computerized Tomography), imaging has considerable diagnostic accuracy for lung cancer diagnosis, which is of significant value for lung cancer diagnosis and has greater feasibility of realizing the extension application in the field of clinical diagnosis.

Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors

Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists' annotation of region of interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) - based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.

Tumor-identification method for predicting recurrence of early-stage lung adenocarcinoma using digital pathology images by machine learning

Lung cancer is one of the cancers with the highest morbidity and mortality in the world. Recurrence often occurs even after complete resection of early-stage lung cancer, and prediction of recurrence after resection is clinically important. However, the pathological characteristics of the recurrence of pathological stage IB lung adenocarcinoma (LAIB) have not yet been elucidated. Therefore, the problem is what type of histological image of lung adenocarcinoma recurs, and it is important to examine the histological image of recurrence. We attempted to predict recurrence of early lung adenocarcinoma after resection on the basis of digital pathological images of hematoxylin and eosin-stained specimens and machine learning applying a convolutional neural network. We constructed a model that extracts the features of two-color spaces and a switching model that automatically switches between our extraction model and one that extracts the features of one-color space for each image. We then developed a tumor-identification method for predicting the presence or absence of LAIB recurrence using these models. We conducted an experiment involving 55 patients with LAIB who underwent surgical resection to evaluate the proposed method. The proposed method determined LAIB recurrence with an accuracy of 84.8%. The use of digital pathology and machine learning can be used for highly accurate prediction of LAIB recurrence after surgical resection. The proposed method has the potential for objective postoperative follow-up observation.