Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study

The therapeutic potential of Ribes orientale in hypertension: Improving electrolyte balance, enhancing antioxidant activity, and modulating the muscarinic pathway

Cardiovascular diseases like hypertension require diuretics, but their effectiveness and side effects limit their use. Alternatives like Ribes orientale extract are being investigated for potential diuretic. In acute diuretic action, the extract and fractions of Ribes orientale were investigated in saline-loaded rats at doses of 12.5, 25, and 50mg/kg, p.o., while prolonged diuretic activity was also performed by administration of the most potent and significant dose of Ribes orientale butanol fraction (ROBF) for 7days. ROBF diuretic action was confirmed through parameters like urine volume, electrolyte concentration, pH and conductivity. The study found that 50mg/kg of ROBF had the most potent and significant diuretic activity, like furosemide (10mg/kg). It also produced significant natriuresis and insignificant kaliuresis in electrolyte excretion. ROBF increased diuretic activity significantly from the first and 7th days. In the mechanistic study, it has been evaluated that nitric oxide and prostaglandins pathways are not involved in the diuretic effect of ROBF, while cholinergic pathway might be responsible for the diuretic effect of ROBF. Hematological, renal function tests and histopathological studies confirmed the safety of Ribes orientale at 50mg/kg. Results showed slight changes in blood count parameters and no significant changes in creatinine and blood urea nitrogen levels compared to the negative control. Furthermore, downregulation of aldosterone by ROBF prolonged activity enhances the local bradykinin availability that results in marked diuretic and natriuretic activity. It is evident from the decreased level of malondialdehyde (MDA) that Ribes orientale reduces the oxidative stress while increased level of superoxide dismutase (SOD) depicts improving level of antioxidant activity. We conclude that the butanol fraction of Ribes orientale roots possesses a considerable diuretic effect. Therefore, more study is needed to identify the exact mechanism of action and the real efficacy of Ribes orientale as diuretic.

Diagnosis of Lung Cancer Using Endobronchial Ultrasonography Image Based on Multi-Scale Image and Multi-Feature Fusion Framework

Lung cancer is the leading cause of cancer-related deaths globally and ranks among the most common cancer types. Given its low overall five-year survival rate, early diagnosis and timely treatment are essential to improving patient outcomes. In recent years, advances in computer technology have enabled artificial intelligence to make groundbreaking progress in imaging-based lung cancer diagnosis. The primary aim of this study is to develop a computer-aided diagnosis (CAD) system for lung cancer using endobronchial ultrasonography (EBUS) images and deep learning algorithms to facilitate early detection and improve patient survival rates. We propose M3-Net, which is a multi-branch framework that integrates multiple features through an attention-based mechanism, enhancing diagnostic performance by providing more comprehensive information for lung cancer assessment. The framework was validated on a dataset of 95 patient cases, including 13 benign and 82 malignant cases. The dataset comprises 1140 EBUS images, with 540 images used for training, and 300 images each for the validation and test sets. The evaluation yielded the following results: accuracy of 0.76, F1-score of 0.75, AUC of 0.83, PPV of 0.80, NPV of 0.75, sensitivity of 0.72, and specificity of 0.80. These findings indicate that the proposed attention-based multi-feature fusion framework holds significant potential in assisting with lung cancer diagnosis.

Externally validated and clinically useful machine learning algorithms to support patient-related decision-making in oncology: a scoping review

BACKGROUND

This scoping review systematically maps externally validated machine learning (ML)-based models in cancer patient care, quantifying their performance, and clinical utility, and examining relationships between models, cancer types, and clinical decisions. By synthesizing evidence, this study identifies, strengths, limitations, and areas requiring further research.

METHODS

The review followed the Joanna Briggs Institute's methodology, Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines, and the Population, Concept, and Context mnemonic. Searches were conducted across Embase, IEEE Xplore, PubMed, Scopus, and Web of Science (January 2014-September 2022), targeting English-language quantitative studies in Q1 journals (SciMago Journal and Country Ranking > 1) that used ML to evaluate clinical outcomes for human cancer patients with commonly available data. Eligible models required external validation, clinical utility assessment, and performance metric reporting. Studies involving genetics, synthetic patients, plants, or animals were excluded. Results were presented in tabular, graphical, and descriptive form.

RESULTS

From 4023 deduplicated abstracts and 636 full-text reviews, 56 studies (2018-2022) met the inclusion criteria, covering diverse cancer types and applications. Convolutional neural networks were most prevalent, demonstrating high performance, followed by gradient- and decision tree-based algorithms. Other algorithms, though underrepresented, showed promise. Lung and digestive system cancers were most frequently studied, focusing on diagnosis and outcome predictions. Most studies were retrospective and multi-institutional, primarily using image-based data, followed by text-based and hybrid approaches. Clinical utility assessments involved 499 clinicians and 12 tools, indicating improved clinician performance with AI assistance and superior performance to standard clinical systems.

DISCUSSION

Interest in ML-based clinical decision-making has grown in recent years alongside increased multi-institutional collaboration. However, small sample sizes likely impacted data quality and generalizability. Persistent challenges include limited international validation across ethnicities, inconsistent data sharing, disparities in validation metrics, and insufficient calibration reporting, hindering model comparison reliability.

CONCLUSION

Successful integration of ML in oncology decision-making requires standardized data and methodologies, larger sample sizes, greater transparency, and robust validation and clinical utility assessments.

OTHER

Financed by FCT-Fundação para a Ciência e a Tecnologia (Portugal, project LA/P/0063/2020, grant 2021.09040.BD) as part of CSS's Ph.D. This work was not registered.

MultiSCCHisto-Net-KD: A deep network for multi-organ explainable squamous cell carcinoma diagnosis with knowledge distillation

Squamous cell carcinoma is a prevalent cancer type that affects various organs in the human body. Manual analysis for detecting squamous cell carcinoma in histopathological images is time-consuming and may be subjective. Squamous cell carcinoma diagnosis is typically based on the differences in the architectural arrangement of squamous epithelial layers and the presence of keratinization. However, the existing literature has predominantly concentrated on identifying cellular irregularities with high magnification images and considering specific organs of squamous cell carcinoma origin. In contrast, relatively little attention has been given to recognizing structural abnormalities observable at low magnification images. In this paper, we consider squamous cell carcinoma histopathological images across different organs of origin captured at low magnification and these images are gathered from various centers to develop a robust model. We propose a novel deep neural network model (MultiSCCHisto-Net) that can detect squamous cell carcinoma of any organ irrespective of organ of origin. In addition, deep neural networks used for histopathological image analysis typically have many parameters, making them computationally expensive. To address this research gap, we incorporate knowledge distillation, which compresses knowledge from a complex teacher model (MultiSCCHisto-Net) into a smaller student model (MultiSCCHisto-Net-KD) while preserving performance and enhancing the generalization of the student model by learning from the teacher's intermediate layer features. Moreover, an explainable deep learning technique called gradient-weighted class activation mapping is incorporated to highlight the image areas that help to classify the sample into particular classes. This explainability significantly enhances our confidence in the proposed model outcomes. We evaluate the model's robustness using private multi-centric and publicly available datasets. Our results show that accuracy rates of 97% and 93% are achieved on private and public datasets, respectively, surpassing the performance of state-of-the-art models.

Not seeing the trees for the forest. The impact of neighbours on graph-based configurations in histopathology

BACKGROUND

Deep learning (DL) has set new standards in cancer diagnosis, significantly enhancing the accuracy of automated classification of whole slide images (WSIs) derived from biopsied tissue samples. To enable DL models to process these large images, WSIs are typically divided into thousands of smaller tiles, each containing 10-50 cells. Multiple Instance Learning (MIL) is a commonly used approach, where WSIs are treated as bags comprising numerous tiles (instances) and only bag-level labels are provided during training. The model learns from these broad labels to extract more detailed, instance-level insights. However, biopsied sections often exhibit high intra- and inter-phenotypic heterogeneity, presenting a significant challenge for classification. To address this, many graph-based methods have been proposed, where each WSI is represented as a graph with tiles as nodes and edges defined by specific spatial relationships.

RESULTS

In this study, we investigate how different graph configurations, varying in connectivity and neighborhood structure, affect the performance of MIL models. We developed a novel pipeline, K-MIL, to evaluate the impact of contextual information on cell classification performance. By incorporating neighboring tiles into the analysis, we examined whether contextual information improves or impairs the network's ability to identify patterns and features critical for accurate classification. Our experiments were conducted on two datasets: COLON cancer and UCSB datasets.

CONCLUSIONS

Our results indicate that while incorporating more spatial context information generally improves model accuracy at both the bag and tile levels, the improvement at the tile level is not linear. In some instances, increasing spatial context leads to misclassification, suggesting that more context is not always beneficial. This finding highlights the need for careful consideration when incorporating spatial context information in digital pathology classification tasks.

A Novel Hybrid Model for Automatic Non-Small Cell Lung Cancer Classification Using Histopathological Images

Background/Objectives: Despite recent advances in research, cancer remains a significant public health concern and a leading cause of death. Among all cancer types, lung cancer is the most common cause of cancer-related deaths, with most cases linked to non-small cell lung cancer (NSCLC). Accurate classification of NSCLC subtypes is essential for developing treatment strategies. Medical professionals regard tissue biopsy as the gold standard for the identification of lung cancer subtypes. However, since biopsy images have very high resolutions, manual examination is time-consuming and depends on the pathologist's expertise. Methods: In this study, we propose a hybrid model to assist pathologists in the classification of NSCLC subtypes from histopathological images. This model processes deep, textural and contextual features obtained by using EfficientNet-B0, local binary pattern (LBP) and vision transformer (ViT) encoder as feature extractors, respectively. In the proposed method, each feature matrix is flattened separately and then combined to form a comprehensive feature vector. The feature vector is given as input to machine learning classifiers to identify the NSCLC subtype. Results: We set up 13 different training scenarios to test 4 different classifiers: support vector machine (SVM), logistic regression (LR), light gradient boosting machine (LightGBM) and extreme gradient boosting (XGBoost). Among these scenarios, we obtained the highest classification accuracy (99.87%) with the combination of EfficientNet-B0 + LBP + ViT Encoder + SVM. The proposed hybrid model significantly enhanced the classification accuracy of NSCLC subtypes. Conclusions: The integration of deep, textural, and contextual features assisted the model in capturing subtle information from the images, thereby reducing the risk of misdiagnosis and facilitating more effective treatment planning.

Next-generation lung cancer pathology: Development and validation of diagnostic and prognostic algorithms

Non-small cell lung cancer (NSCLC) is one of the most common malignant tumors. In this study, we develop a clinically useful computational pathology platform for NSCLC that can be a foundation for multiple downstream applications and provide immediate value for patient care optimization and individualization. We train the primary multi-class tissue segmentation algorithm on a substantial, high-quality, manually annotated dataset of whole-slide images with lung adenocarcinoma and squamous cell carcinomas. We investigate two downstream applications. NSCLC subtyping algorithm is trained and validated using a large, multi-institutional (n = 6), multi-scanner (n = 5), international cohort of NSCLC cases (slides/patients 4,097/1,527). Moreover, we develop four AI-derived, fully explainable, quantitative, prognostic parameters (based on tertiary lymphoid structure and necrosis assessment) and validate them for different clinical endpoints. The computational platform enables the high-precision, quantitative analysis of H&E-stained slides. The developed prognostic parameters facilitate robust and independent risk stratification of patients with NSCLC.

Leveraging immuno-fluorescence data to reduce pathologist annotation requirements in lung tumor segmentation using deep learning

The main bottleneck in training a robust tumor segmentation algorithm for non-small cell lung cancer (NSCLC) on H&E is generating sufficient ground truth annotations. Various approaches for generating tumor labels to train a tumor segmentation model was explored. A large dataset of low-cost low-accuracy panCK-based annotations was used to pre-train the model and determine the minimum required size of the expensive but highly accurate pathologist annotations dataset. PanCK pre-training was compared to foundation models and various architectures were explored for model backbone. Proper study design and sample procurement for training a generalizable model that captured variations in NSCLC H&E was studied. H&E imaging was performed on 112 samples (three centers, two scanner types, different staining and imaging protocols). Attention U-Net architecture was trained using the large panCK-based annotations dataset (68 samples, total area 10,326 [mm2]) followed by fine-tuning using a small pathologist annotations dataset (80 samples, total area 246 [mm2]). This approach resulted in mean intersection over union (mIoU) of 82% [77 87]. Using panCK pretraining provided better performance compared to foundation models and allowed for 70% reduction in pathologist annotations with no drop in performance. Study design ensured model generalizability over variations on H&E where performance was consistent across centers, scanners, and subtypes.

Pathology diagnosis of intraoperative frozen thyroid lesions assisted by deep learning

BACKGROUND

Thyroid cancer is a common thyroid malignancy. The majority of thyroid lesion needs intraoperative frozen pathology diagnosis, which provides important information for precision operation. As digital whole slide images (WSIs) develop, deep learning methods for histopathological classification of the thyroid gland (paraffin sections) have achieved outstanding results. Our current study is to clarify whether deep learning assists pathology diagnosis for intraoperative frozen thyroid lesions or not.

METHODS

We propose an artificial intelligence-assisted diagnostic system for frozen thyroid lesions that applies prior knowledge in tandem with a dichotomous judgment of whether the lesion is cancerous or not and a quadratic judgment of the type of cancerous lesion to categorize the frozen thyroid lesions into five categories: papillary thyroid carcinoma, medullary thyroid carcinoma, anaplastic thyroid carcinoma, follicular thyroid tumor, and non-cancerous lesion. We obtained 4409 frozen digital pathology sections (WSI) of thyroid from the First Affiliated Hospital of Sun Yat-sen University (SYSUFH) to train and test the model, and the performance was validated by a six-fold cross validation, 101 papillary microcarcinoma sections of thyroid were used to validate the system's sensitivity, and 1388 WSIs of thyroid were used for the evaluation of the external dataset. The deep learning models were compared in terms of several metrics such as accuracy, F1 score, recall, precision and AUC (Area Under Curve).

RESULTS

We developed the first deep learning-based frozen thyroid diagnostic classifier for histopathological WSI classification of papillary carcinoma, medullary carcinoma, follicular tumor, anaplastic carcinoma, and non-carcinoma lesion. On test slides, the system had an accuracy of 0.9459, a precision of 0.9475, and an AUC of 0.9955. In the papillary carcinoma test slides, the system was able to accurately predict even lesions as small as 2 mm in diameter. Tested with the acceleration component, the cut processing can be performed in 346.12 s and the visual inference prediction results can be obtained in 98.61 s, thus meeting the time requirements for intraoperative diagnosis. Our study employs a deep learning approach for high-precision classification of intraoperative frozen thyroid lesion distribution in the clinical setting, which has potential clinical implications for assisting pathologists and precision surgery of thyroid lesions.

Artificial Intelligence Recognition Model Using Liquid-Based Cytology Images to Discriminate Malignancy and Histological Types of Non-Small-Cell Lung Cancer

INTRODUCTION

Artificial intelligence image recognition has applications in clinical practice. The purpose of this study was to develop an automated image classification model for lung cancer cytology using a deep learning convolutional neural network (DCNN).

METHODS

Liquid-based cytology samples from 8 normal parenchymal (N), 22 adenocarcinoma (ADC), and 15 squamous cell carcinoma (SQCC) surgical specimens were prepared, and 45 Papanicolaou-stained slides were scanned using whole-slide imaging. The final dataset of 9,141 patches consisted of 2,737 N, 4,756 ADC, and 1,648 SQCC samples. Densenet-121 was used as the DCNN to classify N versus malignant (ADC+SQCC) and ADC versus SQCC images. AdamW optimizer and 5-fold cross-validation were used in the training.

RESULTS

For malignancy prediction, the sensitivity, specificity, and accuracy were 0.97, 0.85, and 0.94, respectively, in the patch-level classification, and 0.92, 0.88, and 0.91, respectively, in the case-level classification. For SQCC prediction, the sensitivity, specificity, and accuracy were 0.86, 0.91, and 0.90, respectively, in the patch-level classification and 0.73, 0.82, and 0.78, respectively, in the case-level classification.

CONCLUSION

The DCNN model performed excellently in predicting malignancy and histological types of lung cancer. This model may be useful for predicting cytopathological diagnosis in clinical situations by reinforcing training.

Prediction of non-muscle invasive bladder cancer recurrence using deep learning of pathology image

We aimed to build a deep learning-based pathomics model to predict the early recurrence of non-muscle-infiltrating bladder cancer (NMIBC) in this work. A total of 147 patients from Xuzhou Central Hospital were enrolled as the training cohort, and 63 patients from Suqian Affiliated Hospital of Xuzhou Medical University were enrolled as the test cohort. Based on two consecutive phases of patch level prediction and WSI-level predictione, we built a pathomics model, with the initial model developed in the training cohort and subjected to transfer learning, and then the test cohort was validated for generalization. The features extracted from the visualization model were used for model interpretation. After migration learning, the area under the receiver operating characteristic curve for the deep learning-based pathomics model in the test cohort was 0.860 (95% CI 0.752-0.969), with good agreement between the migration training cohort and the test cohort in predicting recurrence, and the predicted values matched well with the observed values, with p values of 0.667766 and 0.140233 for the Hosmer-Lemeshow test, respectively. The good clinical application was observed using a decision curve analysis method. We developed a deep learning-based pathomics model showed promising performance in predicting recurrence within one year in NMIBC patients. Including 10 state prediction NMIBC recurrence group pathology features be visualized, which may be used to facilitate personalized management of NMIBC patients to avoid ineffective or unnecessary treatment for the benefit of patients.

Deep learning-based detection and semi-quantitative model for spread through air spaces (STAS) in lung adenocarcinoma

Tumor spread through air spaces (STAS) is a distinctive metastatic pattern affecting prognosis in lung adenocarcinoma (LUAD) patients. Several challenges are associated with STAS detection, including misdetection, low interobserver agreement, and lack of quantitative analysis. In this research, a total of 489 digital whole slide images (WSIs) were collected. The deep learning-based STAS detection model, named STASNet, was constructed to calculate semi-quantitative parameters associated with STAS density and distance. STASNet demonstrated an accuracy of 0.93 for STAS detection at the tiles level and had an AUC of 0.72-0.78 for determining the STAS status at the WSI level. Among the semi-quantitative parameters, T10S, combined with the spatial location information, significantly stratified stage I LUAD patients on disease-free survival. Additionally, STASNet was deployed into a real-time pathological diagnostic environment, which boosted the STAS detection rate and led to the identification of three easily misidentified types of occult STAS.

A multi-classifier system integrated by clinico-histology-genomic analysis for predicting recurrence of papillary renal cell carcinoma

Integrating genomics and histology for cancer prognosis demonstrates promise. Here, we develop a multi-classifier system integrating a lncRNA-based classifier, a deep learning whole-slide-image-based classifier, and a clinicopathological classifier to accurately predict post-surgery localized (stage I-III) papillary renal cell carcinoma (pRCC) recurrence. The multi-classifier system demonstrates significantly higher predictive accuracy for recurrence-free survival (RFS) compared to the three single classifiers alone in the training set and in both validation sets (C-index 0.831-0.858 vs. 0.642-0.777, p < 0.05). The RFS in our multi-classifier-defined high-risk stage I/II and grade 1/2 groups is significantly worse than in the low-risk stage III and grade 3/4 groups (p < 0.05). Our multi-classifier system is a practical and reliable predictor for recurrence of localized pRCC after surgery that can be used with the current staging system to more accurately predict disease course and inform strategies for individualized adjuvant therapy.

Deep Learning Histology for Prediction of Lymph Node Metastases and Tumor Regression after Neoadjuvant FLOT Therapy of Gastroesophageal Adenocarcinoma

BACKGROUND

The aim of this study was to establish a deep learning prediction model for neoadjuvant FLOT chemotherapy response. The neural network utilized clinical data and visual information from whole-slide images (WSIs) of therapy-naïve gastroesophageal cancer biopsies.

METHODS

This study included 78 patients from the University Hospital of Cologne and 59 patients from the University Hospital of Heidelberg used as external validation.

RESULTS

After surgical resection, 33 patients from Cologne (42.3%) were ypN0 and 45 patients (57.7%) were ypN+, while 23 patients from Heidelberg (39.0%) were ypN0 and 36 patients (61.0%) were ypN+ (p = 0.695). The neural network had an accuracy of 92.1% to predict lymph node metastasis and the area under the curve (AUC) was 0.726. A total of 43 patients from Cologne (55.1%) had less than 50% residual vital tumor (RVT) compared to 34 patients from Heidelberg (57.6%, p = 0.955). The model was able to predict tumor regression with an error of ±14.1% and an AUC of 0.648.

CONCLUSIONS

This study demonstrates that visual features extracted by deep learning from therapy-naïve biopsies of gastroesophageal adenocarcinomas correlate with positive lymph nodes and tumor regression. The results will be confirmed in prospective studies to achieve early allocation of patients to the most promising treatment.

A Radiological-Radiomics model for differentiation between minimally invasive adenocarcinoma and invasive adenocarcinoma less than or equal to 3 cm: A two-center retrospective study

OBJECTIVE

To develop a Radiological-Radiomics (R-R) combined model for differentiation between minimal invasive adenocarcinoma (MIA) and invasive adenocarcinoma (IA) of lung adenocarcinoma (LUAD) and evaluate its predictive performance.

METHODS

The clinical, pathological, and imaging data of a total of 509 patients (522 lesions) with LUAD diagnosed by surgical pathology from 2 medical centres were retrospectively collected, with 392 patients (402 lesions) from center 1 trained and validated using a five-fold cross-validation method, and 117 patients (120 lesions) from center 2 serving as an independent external test set. The least absolute shrinkage and selection operator (LASSO) method was utilized to filter features. Logistic regression was used to construct three models for predicting IA, namely, Radiological model, Radiomics model, and R-R model. Also, receiver operating curve curves (ROCs) were plotted, generating corresponding area under the curve (AUC), sensitivity, specificity, and accuracy.

RESULTS

The R-R model for IA prediction achieved an AUC of 0.918 (95 % CI: 0.889-0.947), a sensitivity of 80.3 %, a specificity of 88.2 %, and an accuracy of 82.1 % in the training set. In the validation set, this model exhibited an AUC of 0.906 (95 % CI: 0.842-0.970), a sensitivity of 79.9 %, a specificity of 88.1 %, and an accuracy of 81.8 %. In the external test set, the AUC was 0.894 (95 % CI: 0.824-0.964), a sensitivity of 84.8 %, a specificity of 78.6 %, and an accuracy of 83.3 %.

CONCLUSION

The R-R model showed excellent diagnostic performance in differentiating MIA and IA, which can provide a certain reference for clinical diagnosis and surgical treatment plans.

Prediction of metastases in confusing mediastinal lymph nodes based on flourine-18 fluorodeoxyglucose (18F-FDG) positron emission tomography/computed tomography (PET/CT) imaging using machine learning

Background

For patient management and prognosis, accurate assessment of mediastinal lymph node (LN) status is essential. This study aimed to use machine learning approaches to assess the status of confusing LNs in the mediastinum using positron emission tomography/computed tomography (PET/CT) images; the results were then compared with the diagnostic conclusions of nuclear medicine physicians.

Methods

A total of 509 confusing mediastinal LNs that had undergone pathological assessment or follow-up from 320 patients from three centres were retrospectively included in the study. LNs from centres I and II were randomised into a training cohort (N=324) and an internal validation cohort (N=81), while those from centre III patients formed an external validation cohort (N=104). Various parameters measured from PET and CT images and extracted radiomics and deep learning features were used to construct PET/CT-parameter, radiomics, and deep learning models, respectively. Model performance was compared with the diagnostic results of nuclear medicine physicians using the area under the curve (AUC), sensitivity, specificity, and decision curve analysis (DCA).

Results

The coupled model of gradient boosting decision tree-logistic regression (GBDT-LR) incorporating radiomic features showed AUCs of 92.2% [95% confidence interval (CI), 0.890-0.953], 84.6% (95% CI, 0.761-0.930) and 84.6% (95% CI, 0.770-0.922) across the three cohorts. It significantly outperformed the deep learning model, the parametric PET/CT model and the physician's diagnosis. DCA demonstrated the clinical usefulness of the GBDT-LR model.

Conclusions

The presented GBDT-LR model performed well in evaluating confusing mediastinal LNs in both internal and external validation sets. It not only crossed radiometric features but also avoided overfitting.

A dual data stream hybrid neural network for classifying pathological images of lung adenocarcinoma

Lung cancer has seriously threatened human health due to its high lethality and morbidity. Lung adenocarcinoma, in particular, is one of the most common subtypes of lung cancer. Pathological diagnosis is regarded as the gold standard for cancer diagnosis. However, the traditional manual screening of lung cancer pathology images is time consuming and error prone. Computer-aided diagnostic systems have emerged to solve this problem. Current research methods are unable to fully exploit the beneficial features inherent within patches, and they are characterized by high model complexity and significant computational effort. In this study, a deep learning framework called Multi-Scale Network (MSNet) is proposed for the automatic detection of lung adenocarcinoma pathology images. MSNet is designed to efficiently harness the valuable features within data patches, while simultaneously reducing model complexity, computational demands, and storage space requirements. The MSNet framework employs a dual data stream input method. In this input method, MSNet combines Swin Transformer and MLP-Mixer models to address global information between patches and the local information within each patch. Subsequently, MSNet uses the Multilayer Perceptron (MLP) module to fuse local and global features and perform classification to output the final detection results. In addition, a dataset of lung adenocarcinoma pathology images containing three categories is created for training and testing the MSNet framework. Experimental results show that the diagnostic accuracy of MSNet for lung adenocarcinoma pathology images is 96.55 %. In summary, MSNet has high classification performance and shows effectiveness and potential in the classification of lung adenocarcinoma pathology images.

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.

Predicting meningioma grades and pathologic marker expression via deep learning

OBJECTIVES

To establish a deep learning (DL) model for predicting tumor grades and expression of pathologic markers of meningioma.

METHODS

A total of 1192 meningioma patients from two centers who underwent surgical resection between September 2018 and December 2021 were retrospectively included. The pathological data and post-contrast T1-weight images for each patient were collected. The patients from institute I were subdivided into training, validation, and testing sets, while the patients from institute II served as the external testing cohort. The fine-tuned ResNet50 model based on transfer learning was adopted to classify WHO grade in the whole cohort and predict Ki-67 index, H3K27me3, and progesterone receptor (PR) status of grade 1 meningiomas. The predictive performance was evaluated by the accuracy and loss curve, confusion matrix, receiver operating characteristic curve (ROC), and area under curve (AUC).

RESULTS

The DL prediction model for each label achieved high predictive performance in two cohorts. For WHO grade prediction, the area under the curve (AUC) was 0.966 (95%CI 0.957-0.975) in the internal testing set and 0.669 (95%CI 0.643-0.695) in the external validation cohort. The AUC in predicting Ki-67 index, H3K27me3, and PR status were 0.905 (95%CI 0.895-0.915), 0.773 (95%CI 0.760-0.786), and 0.771 (95%CI 0.750-0.792) in the internal testing set and 0.591 (95%CI 0.562-0.620), 0.658 (95%CI 0.648-0.668), and 0.703 (95%CI 0.674-0.732) in the external validation cohort, respectively.

CONCLUSION

DL models can preoperatively predict meningioma grades and pathologic marker expression with favorable predictive performance.

CLINICAL RELEVANCE STATEMENT

Our DL model could predict meningioma grades and expression of pathologic markers and identify high-risk patients with WHO grade 1 meningioma, which would suggest a more aggressive operative intervention preoperatively and a more frequent follow-up schedule postoperatively.

KEY POINTS

WHO grades and some pathologic markers of meningioma were associated with therapeutic strategies and clinical outcomes. A deep learning-based approach was employed to develop a model for predicting meningioma grades and the expression of pathologic markers. Preoperative prediction of meningioma grades and the expression of pathologic markers was beneficial for clinical decision-making.

DeepTree: Pathological Image Classification Through Imitating Tree-Like Strategies of Pathologists

Digitization of pathological slides has promoted the research of computer-aided diagnosis, in which artificial intelligence analysis of pathological images deserves attention. Appropriate deep learning techniques in natural images have been extended to computational pathology. Still, they seldom take into account prior knowledge in pathology, especially the analysis process of lesion morphology by pathologists. Inspired by the diagnosis decision of pathologists, we design a novel deep learning architecture based on tree-like strategies called DeepTree. It imitates pathological diagnosis methods, designed as a binary tree structure, to conditionally learn the correlation between tissue morphology, and optimizes branches to finetune the performance further. To validate and benchmark DeepTree, we build a dataset of frozen lung cancer tissues and design experiments on a public dataset of breast tumor subtypes and our dataset. Results show that the deep learning architecture based on tree-like strategies makes the pathological image classification more accurate, transparent, and convincing. Simultaneously, prior knowledge based on diagnostic strategies yields superior representation ability compared to alternative methods. Our proposed methodology helps improve the trust of pathologists in artificial intelligence analysis and promotes the practical clinical application of pathology-assisted diagnosis.

AnnoPRO: a strategy for protein function annotation based on multi-scale protein representation and a hybrid deep learning of dual-path encoding

Protein function annotation has been one of the longstanding issues in biological sciences, and various computational methods have been developed. However, the existing methods suffer from a serious long-tail problem, with a large number of GO families containing few annotated proteins. Herein, an innovative strategy named AnnoPRO was therefore constructed by enabling sequence-based multi-scale protein representation, dual-path protein encoding using pre-training, and function annotation by long short-term memory-based decoding. A variety of case studies based on different benchmarks were conducted, which confirmed the superior performance of AnnoPRO among available methods. Source code and models have been made freely available at: https://github.com/idrblab/AnnoPRO and https://zenodo.org/records/10012272.

Histopathology images-based deep learning prediction of prognosis and therapeutic response in small cell lung cancer

Small cell lung cancer (SCLC) is a highly aggressive subtype of lung cancer characterized by rapid tumor growth and early metastasis. Accurate prediction of prognosis and therapeutic response is crucial for optimizing treatment strategies and improving patient outcomes. In this study, we conducted a deep-learning analysis of Hematoxylin and Eosin (H&E) stained histopathological images using contrastive clustering and identified 50 intricate histomorphological phenotype clusters (HPCs) as pathomic features. We identified two of 50 HPCs with significant prognostic value and then integrated them into a pathomics signature (PathoSig) using the Cox regression model. PathoSig showed significant risk stratification for overall survival and disease-free survival and successfully identified patients who may benefit from postoperative or preoperative chemoradiotherapy. The predictive power of PathoSig was validated in independent multicenter cohorts. Furthermore, PathoSig can provide comprehensive prognostic information beyond the current TNM staging system and molecular subtyping. Overall, our study highlights the significant potential of utilizing histopathology images-based deep learning in improving prognostic predictions and evaluating therapeutic response in SCLC. PathoSig represents an effective tool that aids clinicians in making informed decisions and selecting personalized treatment strategies for SCLC patients.

A user-friendly deep learning application for accurate lung cancer diagnosis

BACKGROUND

Accurate diagnosis and subsequent delineated treatment planning require the experience of clinicians in the handling of their case numbers. However, applying deep learning in image processing is useful in creating tools that promise faster high-quality diagnoses, but the accuracy and precision of 3-D image processing from 2-D data may be limited by factors such as superposition of organs, distortion and magnification, and detection of new pathologies. The purpose of this research is to use radiomics and deep learning to develop a tool for lung cancer diagnosis.

METHODS

This study applies radiomics and deep learning in the diagnosis of lung cancer to help clinicians accurately analyze the images and thereby provide the appropriate treatment planning. 86 patients were recruited from Bach Mai Hospital, and 1012 patients were collected from an open-source database. First, deep learning has been applied in the process of segmentation by U-NET and cancer classification via the use of the DenseNet model. Second, the radiomics were applied for measuring and calculating diameter, surface area, and volume. Finally, the hardware also was designed by connecting between Arduino Nano and MFRC522 module for reading data from the tag. In addition, the displayed interface was created on a web platform using Python through Streamlit.

RESULTS

The applied segmentation model yielded a validation loss of 0.498, a train loss of 0.27, a cancer classification validation loss of 0.78, and a training accuracy of 0.98. The outcomes of the diagnostic capabilities of lung cancer (recognition and classification of lung cancer from chest CT scans) were quite successful.

CONCLUSIONS

The model provided means for storing and updating patients' data directly on the interface which allowed the results to be readily available for the health care providers. The developed system will improve clinical communication and information exchange. Moreover, it can manage efforts by generating correlated and coherent summaries of cancer diagnoses.

Artificial intelligence-based recurrence prediction outperforms classical histopathological methods in pulmonary adenocarcinoma biopsies

INTRODUCTION

Between 10 and 50% of early-stage lung adenocarcinoma patients experience local or distant recurrence. Histological parameters such as a solid or micropapillary growth pattern are well-described risk factors for recurrence. However, not every patient presenting with such a pattern will develop recurrence. Designing a model which can more accurately predict recurrence on small biopsy samples can aid the stratification of patients for surgery, (neo-)adjuvant therapy, and follow-up.

MATERIAL AND METHODS

In this study, a statistical model on biopsies fed with histological data from early and advanced-stage lung adenocarcinomas was developed to predict recurrence after surgical resection. Additionally, a convolutional neural network (CNN)-based artificial intelligence (AI) classification model, named AI-based Lung Adenocarcinoma Recurrence Predictor (AILARP), was trained to predict recurrence, with an ImageNet pre-trained EfficientNet that was fine-tuned on lung adenocarcinoma biopsies using transfer learning. Both models were validated using the same biopsy dataset to ensure that an accurate comparison was demonstrated.

RESULTS

The statistical model had an accuracy of 0.49 for all patients when using histology data only. The AI classification model yielded a test accuracy of 0.70 and 0.82 and an area under the curve (AUC) of 0.74 and 0.87 on patch-wise and patient-wise hematoxylin and eosin (H&E) stained whole slide images (WSIs), respectively.

CONCLUSION

AI classification outperformed the traditional clinical approach for recurrence prediction on biopsies by a fair margin. The AI classifier may stratify patients according to their recurrence risk, based only on small biopsies. This model warrants validation in a larger lung biopsy cohort.

Deep Learning-Based Knee MRI Classification for Common Peroneal Nerve Palsy with Foot Drop

Foot drop can have a variety of causes, including the common peroneal nerve (CPN) injuries, and is often difficult to diagnose. We aimed to develop a deep learning-based algorithm that can classify foot drop with CPN injury in patients with knee MRI axial images only. In this retrospective study, we included 945 MR image data from foot drop patients confirmed with CPN injury in electrophysiologic tests (n = 42), and 1341 MR image data with non-traumatic knee pain (n = 107). Data were split into training, validation, and test datasets using a 8:1:1 ratio. We used a convolution neural network-based algorithm (EfficientNet-B5, ResNet152, VGG19) for the classification between the CPN injury group and the others. Performance of each classification algorithm used the area under the receiver operating characteristic curve (AUC). In classifying CPN MR images and non-CPN MR images, EfficientNet-B5 had the highest performance (AUC = 0.946), followed by the ResNet152 and the VGG19 algorithms. On comparison of other performance metrics including precision, recall, accuracy, and F1 score, EfficientNet-B5 had the best performance of the three algorithms. In a saliency map, the EfficientNet-B5 algorithm focused on the nerve area to detect CPN injury. In conclusion, deep learning-based analysis of knee MR images can successfully differentiate CPN injury from other etiologies in patients with foot drop.

A multi-center performance assessment for automated histopathological classification and grading of glioma using whole slide images

Accurate pathological classification and grading of gliomas is crucial in clinical diagnosis and treatment. The application of deep learning techniques holds promise for automated histological pathology diagnosis. In this study, we collected 733 whole slide images from four medical centers, of which 456 were used for model training, 150 for internal validation, and 127 for multi-center testing. The study includes 5 types of common gliomas. A subtask-guided multi-instance learning image-to-label training pipeline was employed. The pipeline leveraged "patch prompting" for the model to converge with reasonable computational cost. Experiments showed that an overall accuracy of 0.79 in the internal validation dataset. The performance on the multi-center testing dataset showed an overall accuracy to 0.73. The findings suggest a minor yet acceptable performance decrease in multi-center data, demonstrating the model's strong generalizability and establishing a robust foundation for future clinical applications.

Improved artificial intelligence discrimination of minor histological populations by supplementing with color-adjusted images

Despite the dedicated research of artificial intelligence (AI) for pathological images, the construction of AI applicable to histopathological tissue subtypes, is limited by insufficient dataset collection owing to disease infrequency. Here, we present a solution involving the addition of supplemental tissue array (TA) images that are adjusted to the tonality of the main data using a cycle-consistent generative adversarial network (CycleGAN) to the training data for rare tissue types. F1 scores of rare tissue types that constitute < 1.2% of the training data were significantly increased by improving recall values after adding color-adjusted TA images constituting < 0.65% of total training patches. The detector also enabled the equivalent discrimination of clinical images from two distinct hospitals and the capability was more increased following color-correction of test data before AI identification (F1 score from 45.2 ± 27.1 to 77.1 ± 10.3, p < 0.01). These methods also classified intraoperative frozen sections, while excessive supplementation paradoxically decreased F1 scores. These results identify strategies for building an AI that preserves the imbalance between training data with large differences in actual disease frequencies, which is important for constructing AI for practical histopathological classification.

Deep Learning for Predicting Effect of Neoadjuvant Therapies in Non-Small Cell Lung Carcinomas With Histologic Images

Neoadjuvant therapies are used for locally advanced non-small cell lung carcinomas, whereby pathologists histologically evaluate the effect using resected specimens. Major pathological response (MPR) has recently been used for treatment evaluation and as an economical survival surrogate; however, interobserver variability and poor reproducibility are often noted. The aim of this study was to develop a deep learning (DL) model to predict MPR from hematoxylin and eosin-stained tissue images and to validate its utility for clinical use. We collected data on 125 primary non-small cell lung carcinoma cases that were resected after neoadjuvant therapy. The cases were randomly divided into 55 for training/validation and 70 for testing. A total of 261 hematoxylin and eosin-stained slides were obtained from the maximum tumor beds, and whole slide images were prepared. We used a multiscale patch model that can adaptively weight multiple convolutional neural networks trained with different field-of-view images. We performed 3-fold cross-validation to evaluate the model. During testing, we compared the percentages of viable tumor evaluated by annotator pathologists (reviewed data), those evaluated by nonannotator pathologists (primary data), and those predicted by the DL-based model using 2-class confusion matrices and receiver operating characteristic curves and performed a survival analysis between MPR-achieved and non-MPR cases. In cross-validation, accuracy and mean F1 score were 0.859 and 0.805, respectively. During testing, accuracy and mean F1 score with reviewed data and those with primary data were 0.986, 0.985, 0.943, and 0.943, respectively. The areas under the receiver operating characteristic curve with reviewed and primary data were 0.999 and 0.978, respectively. The disease-free survival of MPR-achieved cases with reviewed and primary data was significantly better than that of the non-MPR cases (P<.001 and P=.001), and that predicted by the DL-based model was almost identical (P=.005). The DL model may support pathologist evaluations and can offer accurate determinations of MPR in patients.

Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance

The pathology is decisive for disease diagnosis but relies heavily on experienced pathologists. In recent years, there has been growing interest in the use of artificial intelligence in pathology (AIP) to enhance diagnostic accuracy and efficiency. However, the impressive performance of deep learning-based AIP in laboratory settings often proves challenging to replicate in clinical practice. As the data preparation is important for AIP, the paper has reviewed AIP-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. An in-depth analysis of data preparation methods is conducted, encompassing the acquisition of pathological tissue slides, data cleaning, screening, and subsequent digitization. Expert review, image annotation, dataset division for model training and validation are also discussed. Furthermore, we delve into the reasons behind the challenges in reproducing the high performance of AIP in clinical settings and present effective strategies to enhance AIP's clinical performance. The robustness of AIP depends on a randomized collection of representative disease slides, incorporating rigorous quality control and screening, correction of digital discrepancies, reasonable annotation, and sufficient data volume. Digital pathology is fundamental in clinical-grade AIP, and the techniques of data standardization and weakly supervised learning methods based on whole slide image (WSI) are effective ways to overcome obstacles of performance reproduction. The key to performance reproducibility lies in having representative data, an adequate amount of labeling, and ensuring consistency across multiple centers. Digital pathology for clinical diagnosis, data standardization and the technique of WSI-based weakly supervised learning will hopefully build clinical-grade AIP.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Predicting Neoadjuvant Chemotherapy Response and High-Grade Serous Ovarian Cancer From CT Images in Ovarian Cancer with Multitask Deep Learning: A Multicenter Study

RATIONALE AND OBJECTIVES

Accurate prediction neoadjuvant chemotherapy (NACT) response in ovarian cancer (OC) is essential for personalized medicine. We aimed to develop and validate a deep learning (DL) model based on pretreatment contrast-enhanced CT (CECT) images for predicting NACT responses and classifying high-grade serous ovarian cancer (HGSOC) to identify patients who may benefit from NACT.

MATERIALS AND METHODS

This multicenter study, which contained both retrospective and prospective studies, included consecutive OC patients (n = 757) from three hospitals. Using WHO RECIST 1.1 for the reference standard, a total of 587 women with 1761 images were included in the training and validation sets, 67 women with 201 images were included in the prospective sets, and 103 women with 309 images were included in the external sets. A multitask DL model based on the multiperiod CT image was developed to predict NACT response and HGSOC.

RESULTS

Logistic regression analysis showed that peritoneal invasion, retinal invasion, and inguinal lymph node metastasis were independent predictors. The DL achieved promising segmentation performances with DICEmean= 0.83 (range: 0.78-0.87). For predicting NACT response, the DL model combined with clinical risk factors obtained area under the receiver operating characteristic curve (AUCs) of 0.87 (0.83-0.89), 0.88 (0.86-0.91), 0.86 (0.82-0.89), and 0.79 (0.75-0.82) in the training, validation, prospective, and external sets, respectively. The AUCs were 0.91 (0.87-0.94), 0.89 (0.86-0.91), 0.80 (0.76-0.84), and 0.80 (0.75-0.85) in four sets in HGSOC classification.

CONCLUSION

The multitask DL model developed using multiperiod CT images exhibited a promising performance for predicting NACT response and HGSOC with OC, which could provide valuable information for individualized treatment.

Novel tools for early diagnosis and precision treatment based on artificial intelligence

Lung cancer has the highest mortality rate among all cancers in the world. Hence, early diagnosis and personalized treatment plans are crucial to improving its 5-year survival rate. Chest computed tomography (CT) serves as an essential tool for lung cancer screening, and pathology images are the gold standard for lung cancer diagnosis. However, medical image evaluation relies on manual labor and suffers from missed diagnosis or misdiagnosis, and physician heterogeneity. The rapid development of artificial intelligence (AI) has brought a whole novel opportunity for medical task processing, demonstrating the potential for clinical application in lung cancer diagnosis and treatment. AI technologies, including machine learning and deep learning, have been deployed extensively for lung nodule detection, benign and malignant classification, and subtype identification based on CT images. Furthermore, AI plays a role in the non-invasive prediction of genetic mutations and molecular status to provide the optimal treatment regimen, and applies to the assessment of therapeutic efficacy and prognosis of lung cancer patients, enabling precision medicine to become a reality. Meanwhile, histology-based AI models assist pathologists in typing, molecular characterization, and prognosis prediction to enhance the efficiency of diagnosis and treatment. However, the leap to extensive clinical application still faces various challenges, such as data sharing, standardized label acquisition, clinical application regulation, and multimodal integration. Nevertheless, AI holds promising potential in the field of lung cancer to improve cancer care.

Deep Learning for Lung Cancer Diagnosis, Prognosis and Prediction Using Histological and Cytological Images: A Systematic Review

Lung cancer is one of the deadliest cancers worldwide, with a high incidence rate, especially in tobacco smokers. Lung cancer accurate diagnosis is based on distinct histological patterns combined with molecular data for personalized treatment. Precise lung cancer classification from a single H&E slide can be challenging for a pathologist, requiring most of the time additional histochemical and special immunohistochemical stains for the final pathology report. According to WHO, small biopsy and cytology specimens are the available materials for about 70% of lung cancer patients with advanced-stage unresectable disease. Thus, the limited available diagnostic material necessitates its optimal management and processing for the completion of diagnosis and predictive testing according to the published guidelines. During the new era of Digital Pathology, Deep Learning offers the potential for lung cancer interpretation to assist pathologists' routine practice. Herein, we systematically review the current Artificial Intelligence-based approaches using histological and cytological images of lung cancer. Most of the published literature centered on the distinction between lung adenocarcinoma, lung squamous cell carcinoma, and small cell lung carcinoma, reflecting the realistic pathologist's routine. Furthermore, several studies developed algorithms for lung adenocarcinoma predominant architectural pattern determination, prognosis prediction, mutational status characterization, and PD-L1 expression status estimation.

E2EFP-MIL: End-to-end and high-generalizability weakly supervised deep convolutional network for lung cancer classification from whole slide image

Efficient and accurate distinction of histopathological subtype of lung cancer is quite critical for the individualized treatment. So far, artificial intelligence techniques have been developed, whose performance yet remained debatable on more heterogenous data, hindering their clinical deployment. Here, we propose an end-to-end, well-generalized and data-efficient weakly supervised deep learning-based method. The method, end-to-end feature pyramid deep multi-instance learning model (E2EFP-MIL), contains an iterative sampling module, a trainable feature pyramid module and a robust feature aggregation module. E2EFP-MIL uses end-to-end learning to extract generalized morphological features automatically and identify discriminative histomorphological patterns. This method is trained with 1007 whole slide images (WSIs) of lung cancer from TCGA, with AUCs of 0.95-0.97 in test sets. We validated E2EFP-MIL in 5 real-world external heterogenous cohorts including nearly 1600 WSIs from both United States and China with AUCs of 0.94-0.97, and found that 100-200 training images are enough to achieve an AUC of >0.9. E2EFP-MIL overperforms multiple state-of-the-art MIL-based methods with high accuracy and low hardware requirements. Excellent and robust results prove generalizability and effectiveness of E2EFP-MIL in clinical practice. Our code is available at https://github.com/raycaohmu/E2EFP-MIL.

Noninvasive imaging-based machine learning algorithm to identify progressive disease in advanced hepatocellular carcinoma receiving second-line systemic therapy

The aim of this study was to predict tyrosine kinase inhibitors (TKI) plus anti-PD-1 antibodies (TKI-PD-1) efficacy as second-line treatment in advanced hepatocellular carcinoma (HCC) using radiomics analysis. From November 2018 to November 2019, a total of 55 patients were included. Radiomic features were obtained from the CT images before treatment and filtered using intraclass correlation coefficients (ICCs) and least absolute shrinkage and selection operator (LASSO) methods. Subsequently, ten prediction algorithms were developed and validated based on radiomic characteristics. The accuracy of the constructed model was measured through area under the receiver operating characteristic curve (AUC) analysis; survival analysis was performed via Kaplan-Meier and Cox regression analyses. Overall, 18 (32.7%) out of 55 patients had progressive disease. Through ICCs and LASSO, ten radiomic features were entered into the algorithm construction and validation. Ten machine learning algorithms showed different accuracies, with the support vector machine (SVM) model having the highest AUC value of 0.933 in the training cohort and 0.792 in the testing cohort. The radiomic features were associated with overall survival. In conclsion, the SVM algorithm is a useful method to predict TKI-PD-1 efficacy in patients with advanced HCC using images taken prior to treatment.

A Weakly Supervised Deep Learning Model and Human-Machine Fusion for Accurate Grading of Renal Cell Carcinoma from Histopathology Slides

(1) Background: The Fuhrman grading (FG) system is widely used in the management of clear cell renal cell carcinoma (ccRCC). However, it is affected by observer variability and irreproducibility in clinical practice. We aimed to use a deep learning multi-class model called SSL-CLAM to assist in diagnosing the FG status of ccRCC patients using digitized whole slide images (WSIs). (2) Methods: We recruited 504 eligible ccRCC patients from The Cancer Genome Atlas (TCGA) cohort and obtained 708 hematoxylin and eosin-stained WSIs for the development and internal validation of the SSL-CLAM model. Additionally, we obtained 445 WSIs from 188 ccRCC eligible patients in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort as an independent external validation set. A human-machine fusion approach was used to validate the added value of the SSL-CLAM model for pathologists. (3) Results: The SSL-CLAM model successfully diagnosed the five FG statuses (Grade-0, 1, 2, 3, and 4) of ccRCC, and achieved AUCs of 0.917 and 0.887 on the internal and external validation sets, respectively, outperforming a junior pathologist. For the normal/tumor classification (Grade-0, Grade-1/2/3/4) task, the SSL-CLAM model yielded AUCs close to 1 on both the internal and external validation sets. The SSL-CLAM model achieved a better performance for the two-tiered FG (Grade-0, Grade-1/2, and Grade-3/4) task, with AUCs of 0.936 and 0.915 on the internal and external validation sets, respectively. The human-machine diagnostic performance was superior to that of the SSL-CLAM model, showing promising prospects. In addition, the high-attention regions of the SSL-CLAM model showed that with an increasing FG status, the cell nuclei in the tumor region become larger, with irregular contours and increased cellular pleomorphism. (4) Conclusions: Our findings support the feasibility of using deep learning and human-machine fusion methods for FG classification on WSIs from ccRCC patients, which may assist pathologists in making diagnostic decisions.

Association of collagen deep learning classifier with prognosis and chemotherapy benefits in stage II-III colon cancer

The current tumor-node-metastasis staging system does not provide sufficient prognostic prediction or adjuvant chemotherapy benefit information for stage II-III colon cancer (CC) patients. Collagen in the tumor microenvironment affects the biological behaviors and chemotherapy response of cancer cells. Hence, in this study, we proposed a collagen deep learning (collagenDL) classifier based on the 50-layer residual network model for predicting disease-free survival (DFS) and overall survival (OS). The collagenDL classifier was significantly associated with DFS and OS (P < 0.001). The collagenDL nomogram, integrating the collagenDL classifier and three clinicopathologic predictors, improved the prediction performance, which showed satisfactory discrimination and calibration. These results were independently validated in the internal and external validation cohorts. In addition, high-risk stage II and III CC patients with high-collagenDL classifier, rather than low-collagenDL classifier, exhibited a favorable response to adjuvant chemotherapy. In conclusion, the collagenDL classifier could predict prognosis and adjuvant chemotherapy benefits in stage II-III CC patients.

Interpretable deep learning survival predictive tool for small cell lung cancer

Background

Small cell lung cancer (SCLC) is an aggressive and almost universally lethal neoplasm. There is no accurate predictive method for its prognosis. Artificial intelligence deep learning may bring new hope.

Methods

By searching the Surveillance, Epidemiology, and End Results database (SEER), 21,093 patients' clinical data were eventually included. Data were then divided into two groups (train dataset/test dataset). The train dataset (diagnosed in 2010-2014, N = 17,296) was utilized to conduct a deep learning survival model, validated by itself and the test dataset (diagnosed in 2015, N = 3,797) in parallel. According to clinical experience, age, sex, tumor site, T, N, M stage (7th American Joint Committee on Cancer TNM stage), tumor size, surgery, chemotherapy, radiotherapy, and history of malignancy were chosen as predictive clinical features. The C-index was the main indicator to evaluate model performance.

Results

The predictive model had a 0.7181 C-index (95% confidence intervals, CIs, 0.7174-0.7187) in the train dataset and a 0.7208 C-index (95% CIs, 0.7202-0.7215) in the test dataset. These indicated that it had a reliable predictive value on OS for SCLC, so it was then packaged as a Windows software which is free for doctors, researchers, and patients to use.

Conclusion

The interpretable deep learning survival predictive tool for small cell lung cancer developed by this study had a reliable predictive value on their overall survival. More biomarkers may help improve the prognostic predictive performance of small cell lung cancer.

A collaborative workflow between pathologists and deep learning for the evaluation of tumour cellularity in lung adenocarcinoma

AIMS

The reporting of tumour cellularity in cancer samples has become a mandatory task for pathologists. However, the estimation of tumour cellularity is often inaccurate. Therefore, we propose a collaborative workflow between pathologists and artificial intelligence (AI) models to evaluate tumour cellularity in lung cancer samples and propose a protocol to apply it to routine practice.

METHODS AND RESULTS

We developed a quantitative model of lung adenocarcinoma that was validated and tested on 50 cases, and a collaborative workflow where pathologists could access the AI results and adjust their original tumour cellularity scores (adjusted-score) that we tested on 151 cases. The adjusted-score was validated by comparing them with a ground truth established by manual annotation of haematoxylin and eosin slides with reference to immunostains with thyroid transcription factor-1 and napsin A. For training, validation, testing the AI and testing the collaborative workflow, we used 40, 10, 50 and 151 whole slide images of lung adenocarcinoma, respectively. The sensitivity and specificity of tumour segmentation were 97 and 87%, respectively, and the accuracy of nuclei recognition was 99%. One pathologist's visually estimated scores were compared to the adjusted-score, and the pathologist's scores were altered in 87% of cases. Comparison with the ground truth revealed that the adjusted-score was more precise than the pathologists' scores (P < 0.05).

CONCLUSION

We proposed a collaborative workflow between AI and pathologists as a model to improve daily practice and enhance the prediction of tumour cellularity for genetic tests.

A novel transfer-learning based physician-level general and subtype classifier for non-small cell lung cancer

Confirming histological patterns of lung carcinoma is important for determining the prognosis and the next steps of treatment for a patient. Confirming the histologic patterns (subtype) of lung adenocarcinoma is important for determining the prognosis and treatment options for a patient. The task is challenging, and often requires the input of experienced pathologists, who by themselves lack interobserver concordance. A computer-aided diagnosis holds the potential to accelerate the time to diagnosis. As many adenocarcinoma tissue samples contain multiple histologic patterns, accurate computer-aided diagnosis requires annotations manually labeled by pathologists. We propose a method that merges weak supervised learning and Integrated Learning using Transfer Learning using two datasets: The Cancer Genome Atlas (TCGA), and the Clinical Proteomic Tumor Analysis Consortium (CPTAC) to reduce the need for manual annotation by a pathologist while maintaining accuracy. Whole-slide images (WSI) are first determined to be either adenocarcinoma or squamous cell carcinoma, then further identify the subtypes by generating weak classifiers for each subtype, then using integrated learning to create a strong classifier. Our model was evaluated with independent datasets from the CPTAC dataset and a dataset from a private hospital. It can achieve AUC values of 0.86, 0.91, 0.82, 0.77, 0.96, 0.98 in Acinar, LPA, Micropapillary, Papillary, Solid, and Normal, respectively.

Multi-modality artificial intelligence in digital pathology

In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin-eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors' work and discusses the opportunities and challenges of AI.

Artificial intelligence in histopathology: enhancing cancer research and clinical oncology

Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology.

Automatic CD30 scoring method for whole slide images of primary cutaneous CD30+ lymphoproliferative diseases

AIMS

Deep-learning methods for scoring biomarkers are an active research topic. However, the superior performance of many studies relies on large datasets collected from clinical samples. In addition, there are fewer studies on immunohistochemical marker assessment for dermatological diseases. Accordingly, we developed a method for scoring CD30 based on convolutional neural networks for a few primary cutaneous CD30+ lymphoproliferative disorders and used this method to evaluate other biomarkers.

METHODS

A multipatch spatial attention mechanism and conditional random field algorithm were used to fully fuse tumour tissue characteristics on immunohistochemical slides and alleviate the few sample feature deficits. We trained and tested 28 CD30+ immunohistochemical whole slide images (WSIs), evaluated them with a performance index, and compared them with the diagnoses of senior dermatologists. Finally, the model's performance was further demonstrated on the publicly available Yale HER2 cohort.

RESULTS

Compared with the diagnoses by senior dermatologists, this method can better locate the tumour area and reduce the misdiagnosis rate. The prediction of CD3 and Ki-67 validated the model's ability to identify other biomarkers.

CONCLUSIONS

In this study, using a few immunohistochemical WSIs, our model can accurately identify CD30, CD3 and Ki-67 markers. In addition, the model could be applied to additional tumour identification tasks to aid pathologists in diagnosis and benefit clinical evaluation.

Histopathological Tissue Segmentation of Lung Cancer with Bilinear CNN and Soft Attention

Automatic tissue segmentation in whole-slide images (WSIs) is a critical task in hematoxylin and eosin- (H&E-) stained histopathological images for accurate diagnosis and risk stratification of lung cancer. Patch classification and stitching the classification results can fast conduct tissue segmentation of WSIs. However, due to the tumour heterogeneity, large intraclass variability and small interclass variability make the classification task challenging. In this paper, we propose a novel bilinear convolutional neural network- (Bilinear-CNN-) based model with a bilinear convolutional module and a soft attention module to tackle this problem. This method investigates the intraclass semantic correspondence and focuses on the more distinguishable features that make feature output variations relatively large between interclass. The performance of the Bilinear-CNN-based model is compared with other state-of-the-art methods on the histopathological classification dataset, which consists of 107.7 k patches of lung cancer. We further evaluate our proposed algorithm on an additional dataset from colorectal cancer. Extensive experiments show that the performance of our proposed method is superior to that of previous state-of-the-art ones and the interpretability of our proposed method is demonstrated by Grad-CAM.

Breast Cancer Molecular Subtype Prediction on Pathological Images with Discriminative Patch Selection and Multi-Instance Learning

Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient’s paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable block sampling error is risky due to the tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&amp;E pathological whole slide images (WSI) using the AI method is useful and critical to assist pathologists to pre-screen proper paraffin block for IHC. It is a challenging task since only WSI-level labels of molecular subtypes from IHC can be obtained without detailed local region information. Gigapixel WSIs are divided into a huge amount of patches to be computationally feasible for deep learning, while with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selection and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&amp;E WSIs. Firstly, co-teaching strategy using two networks was adopted to learn molecular subtype representations and filter out some noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating local patch with global slide constraint information was used to fine-tune MIL framework on obtained discriminative patches and further improve the prediction performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed AI method and our models outperformed even senior pathologists, which has the potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic.

Classification of subtypes including LCNEC in lung cancer biopsy slides using convolutional neural network from scratch

Identifying the lung carcinoma subtype in small biopsy specimens is an important part of determining a suitable treatment plan but is often challenging without the help of special and/or immunohistochemical stains. Pathology image analysis that tackles this issue would be helpful for diagnoses and subtyping of lung carcinoma. In this study, we developed AI models to classify multinomial patterns of lung carcinoma; ADC, LCNEC, SCC, SCLC, and non-neoplastic lung tissue based on convolutional neural networks (CNN or ConvNet). Four CNNs that were pre-trained using transfer learning and one CNN built from scratch were used to classify patch images from pathology whole-slide images (WSIs). We first evaluated the diagnostic performance of each model in the test sets. The Xception model and the CNN built from scratch both achieved the highest performance with a macro average AUC of 0.90. The CNN built from scratch model obtained a macro average AUC of 0.97 on the dataset of four classes excluding LCNEC, and 0.95 on the dataset of three subtypes of lung carcinomas; NSCLC, SCLC, and non-tumor, respectively. Of particular note is that the relatively simple CNN built from scratch may be an approach for pathological image analysis.