Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features

MIF: Multi-Shot Interactive Fusion Model for Cancer Survival Prediction Using Pathological Image and Genomic Data

Accurate cancer survival prediction is crucial for oncologists to determine therapeutic plan, which directly influences the treatment efficacy and survival outcome of patient. Recently, multimodal fusion-based prognostic methods have demonstrated effectiveness for survival prediction by fusing diverse cancer-related data from different medical modalities, e.g., pathological images and genomic data. However, these works still face significant challenges. First, most approaches attempt multimodal fusion by simple one-shot fusion strategy, which is insufficient to explore complex interactions underlying in highly disparate multimodal data. Second, current methods for investigating multimodal interactions face the capability-efficiency dilemma, which is the difficult balance between powerful modeling capability and applicable computational efficiency, thus impeding effective multimodal fusion. In this study, to encounter these challenges, we propose an innovative multi-shot interactive fusion method named MIF for precise survival prediction by utilizing pathological and genomic data. Particularly, a novel multi-shot fusion framework is introduced to promote multimodal fusion by decomposing it into successive fusing stages, thus delicately integrating modalities in a progressive way. Moreover, to address the capacity-efficiency dilemma, various affinity-based interactive modules are introduced to synergize the multi-shot framework. Specifically, by harnessing comprehensive affinity information as guidance for mining interactions, the proposed interactive modules can efficiently generate low-dimensional discriminative multimodal representations. Extensive experiments on different cancer datasets unravel that our method not only successfully achieves state-of-the-art performance by performing effective multimodal fusion, but also possesses high computational efficiency compared to existing survival prediction methods.

Deep learning radiopathomics predicts targeted therapy sensitivity in EGFR-mutant lung adenocarcinoma

BACKGROUND

Ttyrosine kinase inhibitors (TKIs) represent the standard first-line treatment for patients with epidermal growth factor receptor (EGFR)-mutant lung adenocarcinoma. However, not all patients with EGFR mutations respond to TKIs. This study aims to develop a deep learning radiological-pathological-clinical (DLRPC) model that integrates computed tomography (CT) images, hematoxylin and eosin (H&E)-stained aspiration biopsy samples, and clinical data to predict the response in EGFR-mutant lung adenocarcinoma patients undergoing TKIs treatment.

METHODS

We retrospectively analyzed data from 214 lung adenocarcinoma patients who received TKIs treatment from two medical centers between September 2013 and June 2023. The DLRPC model leverages paired CT, pathological images and clinical data, incorporating a clinical-based attention mask to further explore the cross-modality associations. To evaluate its diagnostic performance, we compared the DLRPC model against single-modality models and a decision level fusion model based on Dempster-Shafer theory. Model performances metrics, including area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), were used for evaluation. The Delong test assessed statistically significantly differences in AUC among models.

RESULTS

The DLRPC model demonstrated strong performance, achieving an AUC value of 0.8424. It outperformed the single-modality models (AUC = 0.6894, 0.7753, 0.8052 for CT model, pathology model and clinical model, respectively. P < 0.05). Additionally, the DLRPC model surpassed the decision level fusion model (AUC = 0.8132, P < 0.05).

CONCLUSION

The DLRPC model effectively predicts the response of EGFR-mutant lung adenocarcinoma patients to TKIs, providing a promising tool for personalized treatment decisions in lung cancer management.

Artificial intelligence entering the pathology arena in oncology: current applications and future perspectives

BACKGROUND

Artificial intelligence (AI) is rapidly transforming the fields of pathology and oncology, offering novel opportunities for advancing diagnosis, prognosis, and treatment of cancer.

METHODS

Through a systematic review-based approach, the representatives from the European Society for Medical Oncology (ESMO) Precision Oncology Working Group (POWG) and international experts identified studies in pathology and oncology that applied AI-based algorithms for tumour diagnosis, molecular biomarker detection, and cancer prognosis assessment. These findings were synthesised to provide a comprehensive overview of current AI applications and future directions in cancer pathology.

RESULTS

The integration of AI tools in digital pathology is markedly improving the accuracy and efficiency of image analysis, allowing for automated tumour detection and classification, identification of prognostic molecular biomarkers, and prediction of treatment response and patient outcomes. Several barriers for the adoption of AI in clinical workflows, such as data availability, explainability, and regulatory considerations, still persist. There are currently no prognostic or predictive AI-based biomarkers supported by level IA or IB evidence. The ongoing advancements in AI algorithms, particularly foundation models, generalist models and transformer-based deep learning, offer immense promise for the future of cancer research and care. AI is also facilitating the integration of multi-omics data, leading to more precise patient stratification and personalised treatment strategies.

CONCLUSIONS

The application of AI in pathology is poised to not only enhance the accuracy and efficiency of cancer diagnosis and prognosis but also facilitate the development of personalised treatment strategies. Although barriers to implementation remain, ongoing research and development in this field coupled with addressing ethical and regulatory considerations will likely lead to a future where AI plays an integral role in cancer management and precision medicine. The continued evolution and adoption of AI in pathology and oncology are anticipated to reshape the landscape of cancer care, heralding a new era of precision medicine and improved patient outcomes.

Integration of histopathological image features and multi-dimensional omics data in predicting molecular features and survival in glioblastoma

Objectives

Glioblastoma (GBM) is a highly malignant brain tumor with complex molecular mechanisms. Histopathological images provide valuable morphological information of tumors. This study aims to evaluate the predictive potential of quantitative histopathological image features (HIF) for molecular characteristics and overall survival (OS) in GBM patients by integrating HIF with multi-omics data.

Methods

We included 439 GBM patients with eligible histopathological images and corresponding genetic data from The Cancer Genome Atlas (TCGA). A total of 550 image features were extracted from the histopathological images. Machine learning algorithms were employed to identify molecular characteristics, with random forest (RF) models demonstrating the best predictive performance. Predictive models for OS were constructed based on HIF using RF. Additionally, we enrolled tissue microarrays of 67 patients as an external validation set. The prognostic histopathological image features (PHIF) were identified using two machine learning algorithms, and prognosis-related gene modules were discovered through WGCNA.

Results

The RF-based OS prediction model achieved significant prognostic accuracy (5-year AUC = 0.829). Prognostic models were also developed using single-omics, the integration of HIF and single-omics (HIF + genomics, HIF + transcriptomics, HIF + proteomics), and all features (multi-omics). The multi-omics model achieved the best prediction performance (1-, 3- and 5-year AUCs of 0.820, 0.926 and 0.878, respectively).

Conclusion

Our study indicated a certain prognostic value of HIF, and the integrated multi-omics model may enhance the prognostic prediction of GBM, offering improved accuracy and robustness for clinical application.

Advanced hybrid deep learning model for enhanced evaluation of osteosarcoma histopathology images

Background

Recent advances in machine learning are transforming medical image analysis, particularly in cancer detection and classification. Techniques such as deep learning, especially convolutional neural networks (CNNs) and vision transformers (ViTs), are now enabling the precise analysis of complex histopathological images, automating detection, and enhancing classification accuracy across various cancer types. This study focuses on osteosarcoma (OS), the most common bone cancer in children and adolescents, which affects the long bones of the arms and legs. Early and accurate detection of OS is essential for improving patient outcomes and reducing mortality. However, the increasing prevalence of cancer and the demand for personalized treatments create challenges in achieving precise diagnoses and customized therapies.

Methods

We propose a novel hybrid model that combines convolutional neural networks (CNN) and vision transformers (ViT) to improve diagnostic accuracy for OS using hematoxylin and eosin (H&E) stained histopathological images. The CNN model extracts local features, while the ViT captures global patterns from histopathological images. These features are combined and classified using a Multi-Layer Perceptron (MLP) into four categories: non-tumor (NT), non-viable tumor (NVT), viable tumor (VT), and non-viable ratio (NVR).

Results

Using the Cancer Imaging Archive (TCIA) dataset, the model achieved an accuracy of 99.08%, precision of 99.10%, recall of 99.28%, and an F1-score of 99.23%. This is the first successful four-class classification using this dataset, setting a new benchmark in OS research and offering promising potential for future diagnostic advancements.

Multiscale transformers and multi-attention mechanism networks for pathological nuclei segmentation

Pathology nuclei segmentation is crucial of computer-aided diagnosis in pathology. However, due to the high density, complex backgrounds, and blurred cell boundaries, it makes pathology cell segmentation still a challenging problem. In this paper, we propose a network model for pathology image segmentation based on a multi-scale Transformer multi-attention mechanism. To solve the problem that the high density of cell nuclei and the complexity of the background make it difficult to extract features, a dense attention module is embedded in the encoder, which improves the learning of the target cell information to minimize target information loss; Additionally, to solve the problem of poor segmentation accuracy due to the blurred cell boundaries, the Multi-scale Transformer Attention module is embedded between encoder and decoder, improving the transfer of the boundary feature information and makes the segmented cell boundaries more accurate. Experimental results on MoNuSeg, GlaS and CoNSeP datasets demonstrate the network's superior accuracy.

Multi-modal radiomics features to predict overall survival of locally advanced esophageal cancer after definitive chemoradiotherapy

PURPOSE

To establish prediction models to predict 2-year overall survival (OS) and stratify patients with different risks based on radiomics features extracted from magnetic resonance imaging (MRI) and computed tomography (CT) before definite chemoradiotherapy (dCRT) in locally advanced esophageal squamous cell carcinoma (ESCC).

METHODS

Patients with locally advanced ESCC were recruited. We extracted 547 radiomics features from MRI and CT images. The least absolute shrinkage and selection operator (LASSO) for COX algorithm was used to obtain features highly correlated with survival outcomes in the training cohort. Based on MRI, CT, and the hybrid image data, three prediction models were built. The predictive performance of the radiomics models was evaluated in the training cohort and verified in the validation cohort using AUC values.

RESULTS

A total of 192 patients were included and randomized into the training and validation cohorts. In predicting 2-year OS, the AUCs of the CT-based model were 0.733 and 0.654 for the training and validation sets. The MRI radiomics-based model was observed with similar AUCs of 0.750 and 0.686 in the training and validation sets. The AUC values of hybrid model combining MRI and CT radiomics features in predicting 2-year OS were 0.792 and 0.715 in the training and validation cohorts. It showed significant differences in 2-year OS in the high-risk and low-risk groups divided by the best cutoff value in the hybrid radiomics-based model.

CONCLUSIONS

The hybrid radiomics-based model demontrated the best performance of predicting 2-year OS and can differentiate the high-risk and low-risk patients.

Public Awareness of and Attitudes Toward the Use of AI in Pathology Research and Practice: Mixed Methods Study

BACKGROUND

The last decade has witnessed major advances in the development of artificial intelligence (AI) technologies for use in health care. One of the most promising areas of research that has potential clinical utility is the use of AI in pathology to aid cancer diagnosis and management. While the value of using AI to improve the efficiency and accuracy of diagnosis cannot be underestimated, there are challenges in the development and implementation of such technologies. Notably, questions remain about public support for the use of AI to assist in pathological diagnosis and for the use of health care data, including data obtained from tissue samples, to train algorithms.

OBJECTIVE

This study aimed to investigate public awareness of and attitudes toward AI in pathology research and practice.

METHODS

A nationally representative, cross-sectional, web-based mixed methods survey (N=1518) was conducted to assess the UK public's awareness of and views on the use of AI in pathology research and practice. Respondents were recruited via Prolific, an online research platform. To be eligible for the study, participants had to be aged >18 years, be UK residents, and have the capacity to express their own opinion. Respondents answered 30 closed-ended questions and 2 open-ended questions. Sociodemographic information and previous experience with cancer were collected. Descriptive and inferential statistics were used to analyze quantitative data; qualitative data were analyzed thematically.

RESULTS

Awareness was low, with only 23.19% (352/1518) of the respondents somewhat or moderately aware of AI being developed for use in pathology. Most did not support a diagnosis of cancer (908/1518, 59.82%) or a diagnosis based on biomarkers (694/1518, 45.72%) being made using AI only. However, most (1478/1518, 97.36%) supported diagnoses made by pathologists with AI assistance. The adjusted odds ratio (aOR) for supporting AI in cancer diagnosis and management was higher for men (aOR 1.34, 95% CI 1.02-1.75). Greater awareness (aOR 1.25, 95% CI 1.10-1.42), greater trust in data security and privacy protocols (aOR 1.04, 95% CI 1.01-1.07), and more positive beliefs (aOR 1.27, 95% CI 1.20-1.36) also increased support, whereas identifying more risks reduced the likelihood of support (aOR 0.80, 95% CI 0.73-0.89). In total, 3 main themes emerged from the qualitative data: bringing the public along, the human in the loop, and more hard evidence needed, indicating conditional support for AI in pathology with human decision-making oversight, robust measures for data handling and protection, and evidence for AI benefit and effectiveness.

CONCLUSIONS

Awareness of AI's potential use in pathology was low, but attitudes were positive, with high but conditional support. Challenges remain, particularly among women, regarding AI use in cancer diagnosis and management. Apprehension persists about the access to and use of health care data by private organizations.

A deep-learning model for predicting tyrosine kinase inhibitor response from histology in gastrointestinal stromal tumor

Over 90% of gastrointestinal stromal tumors (GISTs) harbor mutations in KIT or PDGFRA that can predict response to tyrosine kinase inhibitor (TKI) therapies, as recommended by NCCN (National Comprehensive Cancer Network) guidelines. However, gene sequencing for mutation testing is expensive and time-consuming and is susceptible to a variety of preanalytical factors. To overcome the challenges associated with genetic screening by sequencing, in the current study we developed an artificial intelligence-based deep-learning （DL） model that uses convolutional neural networks (CNN) to analyze digitized hematoxylin and eosin staining in tumor histological sections to predict potential response to imatinib or avapritinib treatment in GIST patients. Assessment with an independent testing set showed that our DL  model could predict imatinib sensitivity with an area under the curve (AUC) of 0.902 in case-wise analysis and 0.807 in slide-wise analysis. Case-level AUCs for predicting imatinib-dose-adjustment cases, avapritinib-sensitive cases, and wildtype GISTs were 0.920, 0.958, and 0.776, respectively, while slide-level AUCs for these respective groups were 0.714, 0.922, and 0.886, respectively. Our model showed comparable or better prediction of actual response to TKI than sequencing-based screening (accuracy 0.9286 versus 0.8929; DL model versus sequencing), while predictions of nonresponse to imatinib/avapritinib showed markedly higher accuracy than sequencing (0.7143 versus 0.4286). These results demonstrate the potential of a DL model to improve predictions of treatment response to TKI therapy from histology in GIST patients. © 2025 The Pathological Society of Great Britain and Ireland.

Spatiotemporal feature learning for actin dynamics

The social amoeba Dictyostelium discoideum is a standard model system for studying cell motility and formation of biological patterns. D. discoideum cells form protrusions and migrate via cytoskeletal reorganization driven by coordinated waves of actin polymerization and depolymerization. Assembly and disassembly of actin filaments are regulated by a complex network of biochemical reactions, exhibiting sensitivity to external physical cues such as stiffness, composition and surface topography of the extracellular matrix, as well as the presence of external electric fields. In this study, we investigate whether the cellular microenvironment, and in particular the presence of electric fields and the nano-topography type, can be directly inferred from images or videos of actin waves. We employ three machine learning techniques to analyze the resulting videos: dictionary learning, scattering transforms, and optical flow. We predict the type of the extracellular environment by observing actin waves frame-by-frame and identifying key visual features that help classify cell motion by the microenvironment type. Our analysis reveals that the decomposition of static images into an adaptive basis of visual primitives provides a robust approach to classifying cells by the nano-topography type. In contrast, predicting whether cells are moving under the influence of an external electric field requires tracking of stable cellular features such as corners and edges over a period of time. We expect our computational approach to be useful in many settings where non-trivial collective dynamics is observed with the help of fluorescent labeling and video microscopy.

Integrating radiomics, pathomics, and biopsy-adapted immunoscore for predicting distant metastasis in locally advanced rectal cancer

BACKGROUND

This study aimed to develop and validate a nomogram that utilized macro- and microscopic tumor characteristics at baseline, including radiomics, pathomics, and biopsy-adapted immunoscore (ISB), to accurately predict distant metastasis (DM) in patients with locally advanced rectal cancer (LARC) who underwent neoadjuvant chemoradiotherapy (nCRT).

MATERIALS AND METHODS

In total, 201 patients with LARC (91 months of median follow-up) were enrolled. Radiomics features were extracted from apparent diffusion coefficient maps and T2-weighted images. Pathomics features including global pattern (features of the entire image) and local pattern (features of the tumor nuclei) were extracted from whole-slide images of hematoxylin-eosin-stained biopsy specimens. ISB was calculated from the densities of CD3+ and CD8+ T cells in the tumor region using immunohistochemistry on biopsy specimens. The construction of a predictive model was carried out using the least absolute shrinkage and selection operator-Cox analysis, with performance metrics including the area under the curve (AUC) and concordance index (C-index) utilized for evaluation.

RESULTS

Compared with patients with moderate and high ISB, patients with low ISB exhibited significantly higher risk scores for radiomics and pathomics signatures. The nomogram showed respective C-indexes of 0.902 and 0.848 for 5-year DM-free survival in the training and test sets, along with corresponding AUC values of 0.950 and 0.872. Patients could be efficiently categorized into low- and high-risk groups for developing DM using the nomogram.

CONCLUSIONS

The nomogram integrating macroscopic radiological information and microscopic pathological information is effective for risk stratification at baseline in LARC treated with nCRT.

Machine Learning-Based Pathomics Model Predicts Angiopoietin-2 Expression and Prognosis in Hepatocellular Carcinoma

Angiopoietin-2 (ANGPT2) shows promise as prognostic marker and therapeutic target in hepatocellular carcinoma (HCC). However, assessing ANGPT2 expression and prognostic potential using histopathology images viewed with naked eye is challenging. Herein, machine learning was employed to develop a pathomics model for analyzing histopathology images to predict ANGPT2 status. HCC cases obtained from The Cancer Genome Atlas (TCGA-HCC; n = 267) were randomly assigned to the training or testing set, and cases from a single center were employed as a validation set (n = 91). In the TCGA-HCC cohort, the group with high ANGPT2 expression had a significantly lower overall survival compared with the group with low ANGPT2. Histopathologic features in the training set were extracted, screened, and incorporated into a gradient-boosting machine model that generated a pathomics score, which successfully predicted ANGPT2 expression in the three data sets and showed remarkable risk stratification for overall survival in both the TCGA-HCC (P < 0.0001) and single-center cohorts (P = 0.001). Multivariate analysis suggested that the pathomics score could serve as a predictor of prognosis (P < 0.001). Bioinformatics analysis illustrated a distinction in tumor growth and development related gene-enriched pathways, vascular endothelial growth factor-related gene expression, and immune cell infiltration between high and low pathomics scores. This study indicates that the use of histopathology image features can enhance the prediction of molecular status and prognosis in HCC. The integration of image features with machine learning may improve prognosis prediction in HCC.

Multi-omics models for predicting prognosis in non-small cell lung cancer patients following chemotherapy and radiotherapy: A multi-center study

BACKGROUND AND PURPOSE

Quantifying tumor heterogeneity from various dimensions is crucial for precise treatment. This study aimed to develop and validate multi-omics models based on the computed tomography images, pathological images, dose and clinical information to predict treatment response and overall survival of non-small cell lung cancer (NSCLC) patients undergoing chemotherapy and radiotherapy.

MATERIALS AND METHODS

This retrospective study included 220 NSCLC patients from three centers. Following feature extraction and selection, single-omics and multi-omics models were built for treatment response and overall survival prediction. The performance of treatment response models was evaluated using the area under the curve (AUC) and box plots. For overall survival analysis, the model's evaluation included AUC, concordance index (C-index), Kaplan-Meier curves, and calibration curves. Shapley values were used to assess the contribution of different features to multi-omics models.

RESULTS

Multi-omics models consistently exhibited superior discriminative ability compared to single-omics models in predicting both treatment response and overall survival. For treatment response, the three all-modality models achieved AUC values of 0.87, 0.91, and 0.82 in the external validation set, respectively. In overall survival analysis, the three all-modality models demonstrated AUC values and C-index of 0.73/0.72, 0.80/0.77, 0.79/0.78 in the external validation set, respectively.

CONCLUSION

Multi-omics prediction models demonstrated superior predictive ability with robustness and interpretability. By predicting treatment response and overall survival in NSCLC patients, these models have the potential to assist clinician optimizing treatment plans, supporting individualized treatment strategies, improving the tumor control probability and prolonging the patients' survival.

A High-Resolution Digital Pathological Image Staining Style Transfer Model Based on Gradient Guidance

Digital pathology images have long been regarded as the gold standard for cancer diagnosis in clinical medicine. A highly generalized digital pathological image diagnosis system can provide strong support for cancer diagnosis, help to improve the diagnostic efficiency and accuracy of doctors, and has important research value. The whole slide image of different centers can lead to very large staining differences due to different scanners and dyes, which pose a challenge to the generalization performance of the model application in multi-center data testing. In order to achieve the normalization of multi-center data, this paper proposes a style transfer algorithm based on an adversarial generative network for high-resolution images. The gradient-guided dye migration model proposed in this paper introduces a gradient-enhanced regularized term in the loss function design of the algorithm. A style transfer algorithm was applied to the source data, and the diagnostic performance of the multi-example learning model based on the domain data was significantly improved by validation in the pathological image datasets of two centers. The proposed method improved the AUC of the best classification model from 0.8856 to 0.9243, and another set of experiments improved the AUC from 0.8012 to 0.8313.

Bayesian Modeling of Cancer Outcomes Using Genetic Variables Assisted by Pathological Imaging Data

With the increasing maturity of genetic profiling, an essential and routine task in cancer research is to model disease outcomes/phenotypes using genetic variables. Many methods have been successfully developed. However, oftentimes, empirical performance is unsatisfactory because of a "lack of information." In cancer research and clinical practice, a source of information that is broadly available and highly cost-effective comes from pathological images, which are routinely collected for definitive diagnosis and staging. In this article, we consider a Bayesian approach for selecting relevant genetic variables and modeling their relationships with a cancer outcome/phenotype. We propose borrowing information from (manually curated, low-dimensional) pathological imaging features via reinforcing the same selection results for the cancer outcome and imaging features. We further develop a weighting strategy to accommodate the scenario where information borrowing may not be equally effective for all subjects. Computation is carefully examined. Simulations demonstrate competitive performance of the proposed approach. We analyze TCGA (The Cancer Genome Atlas) LUAD (lung adenocarcinoma) data, with overall survival and gene expressions being the outcome and genetic variables, respectively. Findings different from the alternatives and with sound properties are made.

Identifying invasiveness to aid lung adenocarcinoma diagnosis using deep learning and pathomics

Most classification efforts for primary subtypes of lung adenocarcinoma (LUAD) have not yet been integrated into clinical practice. This study explores the feasibility of combining deep learning and pathomics to identify tumor invasiveness in LUAD patients, highlighting its potential clinical value in assisting junior and intermediate pathologists. We retrospectively analyzed whole slide image (WSI) data from 289 patients with surgically resected ground-glass nodules (GGNs). First, three ResNet deep learning models were used to identify tumor regions. Second, features from the best-performing model were extracted to build pathomics using machine learning classifiers. Third, the accuracy of pathomics in predicting tumor invasiveness was compared with junior and intermediate pathologists' diagnoses. Performance was evaluated using the area under receiver operator characteristic curve (AUC). On the test cohort, ResNet18 achieved the highest AUC (0.956) and sensitivity (0.832) in identifying tumor areas, with an accuracy of 0.904; Random Forest provided high accuracy and AUC values of 0.814 and 0.807 in assessing tumor invasiveness. Pathology assistance improved diagnostic accuracy for junior and intermediate pathologists, with AUC values increasing from 0.547 to 0.759 and 0.656 to 0.769. This study suggests that deep learning and pathomics can enhance diagnostic accuracy, offering valuable support to pathologists.

Artificial intelligence for diagnosis and predictive biomarkers in Non-Small cell lung cancer Patients: New promises but also new hurdles for the pathologist

The rapid development of artificial intelligence (AI) based tools in pathology laboratories has brought forward unlimited opportunities for pathologists. Promising AI applications used for accomplishing diagnostic, prognostic and predictive tasks are being developed at a high pace. This is notably true in thoracic oncology, given the significant and rapid therapeutic progress made recently for lung cancer patients. Advances have been based on drugs targeting molecular alterations, immunotherapies, and, more recently antibody-drug conjugates which are soon to be introduced. For over a decade, many proof-of-concept studies have explored the use of AI algorithms in thoracic oncology to improve lung cancer patient care. However, despite the enthusiasm in this domain, the set-up and use of AI algorithms in daily practice of thoracic pathologists has not been operative until now, due to several constraints. The purpose of this review is to describe the potential but also the current barriers of AI applications in routine thoracic pathology for non-small cell lung cancer patient care and to suggest practical solutions for rapid future implementation.

Quantifying Nuclear Structures of Digital Pathology Images Across Cancers Using Transport-Based Morphometry

Alterations in nuclear morphology are useful adjuncts and even diagnostic tools used by pathologists in the diagnosis and grading of many tumors, particularly malignant tumors. Large datasets such as TCGA and the Human Protein Atlas, in combination with emerging machine learning and statistical modeling methods, such as feature extraction and deep learning techniques, can be used to extract meaningful knowledge from images of nuclei, particularly from cancerous tumors. Here, we describe a new technique based on the mathematics of optimal transport for modeling the information content related to nuclear chromatin structure directly from imaging data. In contrast to other techniques, our method represents the entire information content of each nucleus relative to a template nucleus using a transport-based morphometry (TBM) framework. We demonstrate that the model is robust to different staining patterns and imaging protocols, and can be used to discover meaningful and interpretable information within and across datasets and cancer types. In particular, we demonstrate morphological differences capable of distinguishing nuclear features along the spectrum from benign to malignant categories of tumors across different cancer tissue types, including tumors derived from liver parenchyma, thyroid gland, lung mesothelium, and skin epithelium. We believe these proof-of-concept calculations demonstrate that the TBM framework can provide the quantitative measurements necessary for performing meaningful comparisons across a wide range of datasets and cancer types that can potentially enable numerous cancer studies, technologies, and clinical applications and help elevate the role of nuclear morphometry into a more quantitative science.

MultiSCCHisto-Net-KD: A deep network for multi-organ explainable squamous cell carcinoma diagnosis with knowledge distillation

Squamous cell carcinoma is a prevalent cancer type that affects various organs in the human body. Manual analysis for detecting squamous cell carcinoma in histopathological images is time-consuming and may be subjective. Squamous cell carcinoma diagnosis is typically based on the differences in the architectural arrangement of squamous epithelial layers and the presence of keratinization. However, the existing literature has predominantly concentrated on identifying cellular irregularities with high magnification images and considering specific organs of squamous cell carcinoma origin. In contrast, relatively little attention has been given to recognizing structural abnormalities observable at low magnification images. In this paper, we consider squamous cell carcinoma histopathological images across different organs of origin captured at low magnification and these images are gathered from various centers to develop a robust model. We propose a novel deep neural network model (MultiSCCHisto-Net) that can detect squamous cell carcinoma of any organ irrespective of organ of origin. In addition, deep neural networks used for histopathological image analysis typically have many parameters, making them computationally expensive. To address this research gap, we incorporate knowledge distillation, which compresses knowledge from a complex teacher model (MultiSCCHisto-Net) into a smaller student model (MultiSCCHisto-Net-KD) while preserving performance and enhancing the generalization of the student model by learning from the teacher's intermediate layer features. Moreover, an explainable deep learning technique called gradient-weighted class activation mapping is incorporated to highlight the image areas that help to classify the sample into particular classes. This explainability significantly enhances our confidence in the proposed model outcomes. We evaluate the model's robustness using private multi-centric and publicly available datasets. Our results show that accuracy rates of 97% and 93% are achieved on private and public datasets, respectively, surpassing the performance of state-of-the-art models.

Azurin a potent anticancer and antimicrobial agent isolated from a novel Pseudomonas aeruginosa strain

Azurin, a bacterial blue-copper protein, has garnered significant attention as a potential anticancer drug in recent years. Among twenty Pseudomonas aeruginosa isolates, we identified one isolate that demonstrated potent and remarkable azurin synthesis using the VITEK 2 system and 16S rRNA sequencing. The presence of the azurin gene was confirmed in the genomic DNA using specific oligonucleotide primers, and azurin expression was also detected in the synthesized cDNA, which revealed that the azurin expression is active. Furthermore, crude azurin protein was extracted, precipitated using 70% ammonium sulfate, dialyzed, and subjected to purification using carboxymethyl-Sephadex in affinity chromatography as a cheap method for purification. The partially purified azurin protein was characterized using polyacrylamide gel electrophoresis, energy-dispersive X-ray spectroscopy, Fourier-transform infrared spectroscopy, and nuclear magnetic resonance spectroscopy. Notably, qualitative elemental analysis by EDX showed the presence of copper and sulfur, corresponding to the copper-core and disulfide-bridge, respectively, in the purified azurin fraction. Moreover, FTIR spectroscopy revealed characteristic amide I and II absorption peaks (1500-1700 cm- 1), revealing the possible secondary structure of azurin. The results of NMR revealed the presence of characteristic amino acids such as methionine and cysteine, which confirmed the EDX results for sulfur-containing amino acids. Purified azurin exhibited antimicrobial activity against Staphylococcus aureus, Bacillus subtilis, Escherichia coli, and Klebsiella pneumoniae. Additionally, its anticancer properties were determined using the MTT assay and cell cycle analysis, revealing a preference for inhibiting the MCF7 breast cancer cell line where breast cancer is most common in Egypt. Overall, the research findings suggest that the local isolate, P. aeruginosa strain 105, could be a potential source of azurin protein for incorporation into cancer treatment strategies.

IDH-mutant glioma risk stratification via whole slide images: Identifying pathological feature associations

This article aims to develop and validate a pathological prognostic model for predicting prognosis in patients with isocitrate dehydrogenase (IDH)-mutant gliomas and reveal the biological underpinning of the prognostic pathological features. The pathomic model was constructed based on whole slide images (WSIs) from a training set (N = 486) and evaluated on internal validation set (N = 209), HPPH validation set (N = 54), and TCGA validation set (N = 352). Biological implications of PathScore and individual pathomic features were identified by pathogenomics set (N = 100). The WSI-based pathological signature was an independent prognostic factor. Incorporating the pathological features into a clinical model resulted in a pathological-clinical model that predicted survival better than either the pathological model or clinical model alone. Ten categories of pathways (metabolism, proliferation, immunity, DNA damage response, disease, migrate, protein modification, synapse, transcription and translation, and complex cellular functions) were significantly correlated with the WSI-based pathological features.

More than meets the eye: predicting adrenocortical carcinoma outcomes with pathomics

BACKGROUND

Adrenocortical carcinoma (ACC) is a rare, aggressive malignancy with high recurrence rates and poor prognosis. Current prognostic models are inadequate, highlighting the need for innovative diagnostic tools. Pathomics, which utilizes computer algorithms to analyze whole-slide images, offers a promising approach to enhance prognostic models for ACC.

METHODS

A retrospective cohort of 159 patients who underwent radical adrenalectomy between 2002 and 2019 was analyzed. Patients were divided into training (N = 111) and validation (N = 48) cohorts. Pathomics features were extracted using an unsupervised segmentation method. A pathomics signature (PSACC) was developed through LASSO-Cox regression, incorporating 5 specific pathomics features.

RESULTS

The PSACC showed a strong correlation with ACC prognosis. In the training cohort, the hazard ratio was 3.380 (95% CI, 1.687-6.772, P < .001), and in the validation cohort, it was 3.904 (95% CI, 1.039-14.669, P < .001). A comprehensive nomogram integrating PSACC and M stage significantly outperformed the conventional clinicopathological model in prediction accuracy, with concordance indexes of 0.779 versus 0.668 in the training cohort (P = .002) and 0.752 versus 0.603 in the validation cohort (P = .003).

CONCLUSIONS

The development of a pathomics-based nomogram for ACC presents a superior prognostic tool, enhancing personalized clinical decision making. This study highlights the potential of pathomics in refining prognostic models for complex malignancies like ACC, with implications for improving prognosis prediction and guiding treatment strategies in clinical practice.

Semi-supervised ViT knowledge distillation network with style transfer normalization for colorectal liver metastases survival prediction

Colorectal liver metastases (CLM) affect almost half of all colon cancer patients and the response to systemic chemotherapy plays a crucial role in patient survival. While oncologists typically use tumor grading scores, such as tumor regression grade (TRG), to establish an accurate prognosis on patient outcomes, including overall survival (OS) and time-to-recurrence (TTR), these traditional methods have several limitations. They are subjective, time-consuming, and require extensive expertise, which limits their scalability and reliability. Additionally, existing approaches for prognosis prediction using machine learning mostly rely on radiological imaging data, but recently histological images have been shown to be relevant for survival predictions by allowing to fully capture the complex microenvironmental and cellular characteristics of the tumor. To address these limitations, we propose an end-to-end approach for automated prognosis prediction using histology slides stained with Hematoxylin and Eosin (H&E) and Hematoxylin Phloxine Saffron (HPS). We first employ a Generative Adversarial Network (GAN) for slide normalization to reduce staining variations and improve the overall quality of the images that are used as input to our prediction pipeline. We propose a semi-supervised model to perform tissue classification from sparse annotations, producing segmentation and feature maps. Specifically, we use an attention-based approach that weighs the importance of different slide regions in producing the final classification results. Finally, we exploit the extracted features for the metastatic nodules and surrounding tissue to train a prognosis model. In parallel, we train a vision Transformer model in a knowledge distillation framework to replicate and enhance the performance of the prognosis prediction. We evaluate our approach on an in-house clinical dataset of 258 CLM patients, achieving superior performance compared to other comparative models with a c-index of 0.804 (0.014) for OS and 0.735 (0.016) for TTR, as well as on two public datasets. The proposed approach achieves an accuracy of 86.9% to 90.3% in predicting TRG dichotomization. For the 3-class TRG classification task, the proposed approach yields an accuracy of 78.5% to 82.1%, outperforming the comparative methods. Our proposed pipeline can provide automated prognosis for pathologists and oncologists, and can greatly promote precision medicine progress in managing CLM patients.

Machine learning methods for histopathological image analysis: Updates in 2024

The combination of artificial intelligence and digital pathology has emerged as a transformative force in healthcare and biomedical research. As an update to our 2018 review, this review presents comprehensive analysis of machine learning applications in histopathological image analysis, with focus on the developments since 2018. We highlight significant advances that have expanded the technical capabilities and practical applications of computational pathology. The review examines progress in addressing key challenges in the field as follows: processing of gigapixel whole slide images, insufficient labeled data, multidimensional analysis, domain shifts across institutions, and interpretability of machine learning models. We evaluate emerging trends, such as foundation models and multimodal integration, that are reshaping the field. Overall, our review highlights the potential of machine learning in enhancing both routine pathological analysis and scientific discovery in pathology. By providing this comprehensive overview, this review aims to guide researchers and clinicians in understanding the current state of the pathology image analysis field and its future trajectory.

RiskPath: Explainable deep learning for multistep biomedical prediction in longitudinal data

Many diseases are the end outcomes of multifactorial risks that interact and increment over months or years. Timeseries AI methods have attracted increasing interest given their ability to operate on native timeseries data to predict disease outcomes. Instantiating such models in risk stratification tools has proceeded more slowly, in part limited by factors such as structural complexity, model size and explainability. Here, we present RiskPath, an explainable AI toolbox that offers advanced timeseries methods and additional functionality relevant to risk stratification use cases in classic and emerging longitudinal cohorts. Theoretically-informed optimization is integrated in prediction to specify optimal model topology or explore performance-complexity tradeoffs. Accompanying modules allow the user to map the changing importance of predictors over the disease course, visualize the most important antecedent time epochs contributing to disease risk or remove predictors to construct compact models for clinical applications with minimal performance impact.

Multimodal integration to identify the invasion status of lung adenocarcinoma intraoperatively

Evaluating the invasiveness of lung adenocarcinoma is crucial for determining the appropriate surgical strategy, impacting postoperative outcomes. This study developed a multimodality model combining radiomics, intraoperative frozen section (FS) pathology, and clinical indicators to predict invasion status. The study enrolled 1,424 patients from two hospitals, divided into multimodal training, radiology testing, and pathology testing cohorts. A prospective validation cohort of 114 patients was selected between March and May 2023. The radiomics + pathology + clinical indicators multimodality model (multi-RPC model) achieved an area under the curve (AUC) of 0.921 (95% confidence interval [CI] 0.899-0.939) in the multimodal training cohort and 0.939 (95% CI 0.878-0.975) in the validation cohort, outperforming single- and dual-modality models. The multi-RPC model's predictive accuracy of 0.860 (95% CI 0.782-0.918) suggests that it could significantly reduce inappropriate surgical procedures, enhancing precision oncology by integrating multimodal information to guide surgical decisions.

Role of artificial intelligence in cancer detection using protein p53: A Review

Normal cell development and prevention of tumor formation rely on the tumor-suppressor protein p53. This crucial protein is produced from the Tp53 gene, which encodes the p53 protein. The p53 protein plays a vital role in regulating cell growth, DNA repair, and apoptosis (programmed cell death), thereby maintaining the integrity of the genome and preventing the formation of tumors. Since p53 was discovered 43 years ago, many researchers have clarified its functions in the development of tumors. With the support of the protein p53 and targeted artificial intelligence modeling, it will be possible to detect cancer and tumor activity at an early stage. This will open up new research opportunities. In this review article, a comprehensive analysis was conducted on different machine learning techniques utilized in conjunction with the protein p53 to predict and speculate cancer. The study examined the types of data incorporated and evaluated the performance of these techniques. The aim was to provide a thorough understanding of the effectiveness of machine learning in predicting and speculating cancer using the protein p53.

NETosis Genes and Pathomic Signature: A Novel Prognostic Marker for Ovarian Serous Cystadenocarcinoma

To evaluate the prognostic significance and molecular mechanism of NETosis markers in ovarian serous cystadenocarcinoma (OSC), we constructed a machine learning-based pathomic model utilizing hematoxylin and eosin (H&E) slides. We analyzed 333 patients with OSC from The Cancer Genome Atlas for prognostic-related neutrophil extracellular trap formation (NETosis) genes through bioinformatics analysis. Pathomic features were extracted from 54 cases with complete pathological images, genetic matrices, and clinical information. Two pathomic prognostic models were constructed using support vector machine (SVM) and logistic regression (LR) algorithms. Additionally, we established a predictive scoring system that integrated pathomic scores based on the NETcluster subtypes and clinical signature. We identified four NETosis genes significantly correlated with OSC prognosis, which were functionally associated with immune response, somatic mutations, tumor invasion, and metastasis. Five robust pathomic features were selected for overall survival prediction. The LR and SVM pathomic models demonstrated strong predictive performance for the NETcluster subtype classification through five-fold cross-validation. Time-dependent ROC analysis revealed excellent prognostic capability of the LR pathomic model's score for the overall survival (AUC values of 0.658, 0.761, and 0.735 at 36, 48, and 60 months, respectively), further validated by Kaplan-Meier analysis. The expression levels of NETosis genes greatly affected OSC patients' prognoses. The pathomic analysis of H&E slide pathological images provides an effective approach for predicting both NETcluster subtype and overall survival in OSC patients.

Advancements in Artificial Intelligence-Enhanced Imaging Diagnostics for the Management of Liver Disease-Applications and Challenges in Personalized Care

Liver disease can significantly impact life expectancy, making early diagnosis and therapeutic intervention critical challenges in medical care. Imaging diagnostics play a crucial role in diagnosing and managing liver diseases. Recently, the application of artificial intelligence (AI) in medical imaging analysis has become indispensable in healthcare. AI, trained on vast datasets of medical images, has sometimes demonstrated diagnostic accuracy that surpasses that of human experts. AI-assisted imaging diagnostics are expected to contribute significantly to the standardization of diagnostic quality. Furthermore, AI has the potential to identify image features that are imperceptible to humans, thereby playing an essential role in clinical decision-making. This capability enables physicians to make more accurate diagnoses and develop effective treatment strategies, ultimately improving patient outcomes. Additionally, AI is anticipated to become a powerful tool in personalized medicine. By integrating individual patient imaging data with clinical information, AI can propose optimal plans for treatment, making it an essential component in the provision of the most appropriate care for each patient. Current reports highlight the advantages of AI in managing liver diseases. As AI technology continues to evolve, it is expected to advance personalized diagnostics and treatments and contribute to overall improvements in healthcare quality.

ChatGPT-Assisted Machine Learning for Chronic Disease Classification and Prediction: A Developmental and Validation Study

Background Chronic diseases such as chronic kidney disease (CKD), chronic liver disease (CLD), tuberculosis (TB), dementia, and heart disease are global health concerns of significant importance, representing major causes of morbidity and mortality worldwide. Early diagnosis and interventions are critical to improve patient outcomes and reduce healthcare costs. Methods This prospective observational study analyzed clinical data from 270 patients (calculated using G*Power 3.1.9.7 analysis (Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany), α = 0.05, power = 0.80), with 260 (96.3%) completing the protocol. The cohort comprised 149 (55.2%) males and 121 (44.8%) females, distributed across CKD (n=55, 21.2%), CLD (n=52, 20.0%), TB (n=51, 19.6%), dementia (n=50, 19.2%), and heart disease (n=52, 20.0%). Three ML models were employed with ChatGPT version 3.5 assistance (OpenAI, San Francisco, CA, USA) in feature selection and hyperparameter optimization: logistic regression, random forest, and support vector machines. Model performance was evaluated using accuracy, sensitivity, specificity, precision, recall, F1-score, and AUC-ROC metrics. Ten-fold cross-validation was applied to ensure robustness. Results The random forest model demonstrated superior performance, achieving the highest accuracy in predicting CKD (47/55, 85.3%, p < 0.001, sensitivity 45/55, 82.5%, specificity 48/55, 87.2%) and heart disease (46/52, 88.2%, p < 0.001, sensitivity 45/52, 85.7%, specificity 47/52, 90.1%). Logistic regression effectively predicted TB (41/51, 80.1%, p < 0.01) and dementia (41/50, 82.4%, p < 0.01). Key predictive parameters included hemoglobin (median 10.2 g/dL, IQR 8.4-12.6) and erythrocyte sedimentation rate (median 42.0 mm/hr, IQR 20.0-65.0). Model validation showed high consistency, with positive acid-fast bacilli in 40/51 (78.4%) TB cases and characteristic radiological findings in 43/51 (84.3%) cases. Conclusion ML algorithms, particularly random forest, show promising potential in chronic disease classification and prediction. The integration of ChatGPT enhanced model development through optimized feature selection and hyperparameter tuning. Future research should focus on external validation through multi-center studies and prospective clinical trials.

Role of artificial intelligence in digital pathology for gynecological cancers

The diagnosis of cancer is typically based on histopathological sections or biopsies on glass slides. Artificial intelligence (AI) approaches have greatly enhanced our ability to extract quantitative information from digital histopathology images as a rapid growth in oncology data. Gynecological cancers are major diseases affecting women's health worldwide. They are characterized by high mortality and poor prognosis, underscoring the critical importance of early detection, treatment, and identification of prognostic factors. This review highlights the various clinical applications of AI in gynecological cancers using digitized histopathology slides. Particularly, deep learning models have shown promise in accurately diagnosing, classifying histopathological subtypes, and predicting treatment response and prognosis. Furthermore, the integration with transcriptomics, proteomics, and other multi-omics techniques can provide valuable insights into the molecular features of diseases. Despite the considerable potential of AI, substantial challenges remain. Further improvements in data acquisition and model optimization are required, and the exploration of broader clinical applications, such as the biomarker discovery, need to be explored.

Embracing Large Language Models for Adult Life Support Learning

Background It is recognised that large language models (LLMs) may aid medical education by supporting the understanding of explanations behind answers to multiple choice questions. This study aimed to evaluate the efficacy of LLM chatbots ChatGPT and Bard in answering an Intermediate Life Support pre-course multiple choice question (MCQs) test developed by the Resuscitation Council UK focused on managing deteriorating patients and identifying causes and treating cardiac arrest. We assessed the accuracy of responses and quality of explanations to evaluate the utility of the chatbots. Methods The performance of the AI chatbots ChatGPT-3.5 and Bard were assessed on their ability to choose the correct answer and provide clear comprehensive explanations in answering MCQs developed by the Resuscitation Council UK for their Intermediate Life Support Course. Ten MCQs were tested with a total score of 40, with one point scored for each accurate response to each statement a-d. In a separate scoring, questions were scored out of 1 if all sub-statements a-d were correct, to give a total score out of 10 for the test. The explanations provided by the AI chatbots were evaluated by three qualified physicians as per a rating scale from 0-3 for each overall question and median rater scores calculated and compared. The Fleiss multi-rater kappa (κ) was used to determine the score agreement among the three raters. Results When scoring each overall question to give a total score out of 10, Bard outperformed ChatGPT although the difference was not significant (p=0.37). Furthermore, there was no statistically significant difference in the performance of ChatGPT compared to Bard when scoring each sub-question separately to give a total score out of 40 (p=0.26). The qualities of explanations were similar for both LLMs. Importantly, despite answering certain questions incorrectly, both AI chatbots provided some useful correct information in their explanations of the answers to these questions. The Fleiss multi-rater kappa was 0.899 (p<0.001) for ChatGPT and 0.801 (p<0.001) for Bard. Conclusions The performances of both Bard and ChatGPT were similar in answering the MCQs with similar scores achieved. Notably, despite having access to data across the web, neither of the LLMs answered all questions accurately. This suggests that there is still learning required of AI models in medical education.

Artificial intelligence: A new tool in the pathologist's armamentarium for the diagnosis of IBD

Inflammatory bowel diseases (IBD) are classified into two entities, namely Crohn's disease (CD) and ulcerative colitis (UC), which differ in disease trajectories, genetics, epidemiological, clinical, endoscopic, and histopathological aspects. As no single golden standard modality for diagnosing IBD exists, the differential diagnosis among UC, CD, and non-IBD involves a multidisciplinary approach, considering professional groups that include gastroenterologists, endoscopists, radiologists, and pathologists. In this context, histological examination of endoscopic or surgical specimens plays a fundamental role. Nevertheless, in differentiating IBD from non-IBD colitis, the histopathological evaluation of the morphological lesions is limited by sampling and subjective human judgment, leading to potential diagnostic discrepancies. To overcome these limitations, artificial intelligence (AI) techniques are emerging to enable automated analysis of medical images with advantages in accuracy, precision, and speed of investigation, increasing interest in the histological analysis of gastrointestinal inflammation. This review aims to provide an overview of the most recent knowledge and advances in AI methods, summarizing its applications in the histopathological analysis of endoscopic biopsies from IBD patients, and discussing its strengths and limitations in daily clinical practice.

Machine learning model and nomogram to predict the risk of heart failure hospitalization in peritoneal dialysis patients

INTRODUCTION

The study presented here aimed to establish a predictive model for heart failure (HF) and all-cause mortality in peritoneal dialysis (PD) patients with machine learning (ML) algorithm.

METHODS

We retrospectively included 1006 patients who initiated PD from 2010 to 2016. XGBoost, random forest (RF), and AdaBoost were used to train models for assessing risk for 1-year and 5-year HF hospitalization and mortality. The performance was validated using fivefold cross-validation. The optimal ML algorithm was used to construct the models to predictive the risk of the HF and all-cause mortality. The prediction performance of ML methods and Cox regression was compared.

RESULTS

Over a median follow-up of 49 months. Two hundred and ninety-eight patients developed HF required hospitalization; 199 patients died during the follow-up. The RF model (AUC = 0.853) was the best performing model for predicting HF, and the XGBoost model (AUC = 0.871) was the best model for predicting mortality. Baseline moderate or severe renal disease, systolic blood pressure (SBP), body mass index (BMI), age, Charlson Comorbidity Index (CCI) score were strongly associated with HF hospitalization, whereas age, CCI score, creatinine, age, high-density lipoprotein cholesterol (HDL-C), total cholesterol, baseline estimated glomerular filtration rate (eGFR) were the most significant predictors of mortality. For all the above endpoints, the ML models demonstrated better discrimination than Cox regression.

CONCLUSIONS

We developed and validated a novel method to predict the risk factors of HF and all-cause mortality that integrates readily available clinical, laboratory, and electrocardiographic variables to predict the risk of HF among PD patients.

Deep learning analysis of histopathological images predicts immunotherapy prognosis and reveals tumour microenvironment features in non-small cell lung cancer

BACKGROUND

Non-small cell lung cancer (NSCLC) is one of the leading causes of cancer mortality worldwide. Immune checkpoint inhibitors (ICIs) have emerged as a crucial treatment option for patients with advanced NSCLC. However, only a subset of patients experience clinical benefit from ICIs. Therefore, identifying biomarkers that can predict response to ICIs is imperative for optimising patient selection.

METHODS

Hematoxylin and eosin (H&E) images of NSCLC patients were obtained from the local cohort (n = 106) and The Cancer Genome Atlas (TCGA) (n = 899). We developed an ICI-related pathological prognostic signature (ir-PPS) based on H&E stained histopathology images to predict prognosis in NSCLC patients treated with ICIs using deep learning. To accomplish this, we employed a modified ResNet model (ResNet18-PG), a widely-used deep learning architecture well-known for its effectiveness in handling complex image recognition tasks. Our modifications include a progressive growing strategy to improve the stability of model training and the use of the AdamW optimiser, which enhances the optimisation process by adjusting the learning rate based on training dynamics.

RESULTS

The deep learning model, ResNet18-PG, achieved an area under the receiver operating characteristic curve (AUC) of 0.918 and a recall of 0.995 on the local cohort. The ir-PPS effectively risk-stratified NSCLC patients. Patients in the low-risk group (n = 40) had significantly improved progression-free survival (PFS) after ICI treatment compared to those in the high-risk group (n = 66, log-rank P = 0.004, hazard ratio (HR) = 3.65, 95%CI: 1.75-7.60). The ir-PPS demonstrated good discriminatory power for predicting 6-month PFS (AUC = 0.750), 12-month PFS (AUC = 0.677), and 18-month PFS (AUC = 0.662). The low-risk group exhibited increased expression of immune checkpoint molecules, cytotoxicity-related genes, an elevated abundance of tumour-infiltrating lymphocytes, and enhanced activity in immune stimulatory pathways.

CONCLUSIONS

The ir-PPS signature derived from H&E images using deep learning could predict ICIs prognosis in NSCLC patients. The ir-PPS provides a novel imaging biomarker that may help select optimal candidates for ICIs therapy in NSCLC.

Predicting BRCA mutation and stratifying targeted therapy response using multimodal learning: a multicenter study

BACKGROUND

The status of BRCA1/2 genes plays a crucial role in the treatment decision-making process for multiple cancer types. However, due to high costs and limited resources, a demand for BRCA1/2 genetic testing among patients is currently unmet. Notably, not all patients with BRCA1/2 mutations achieve favorable outcomes with poly (ADP-ribose) polymerase inhibitors (PARPi), indicating the necessity for risk stratification. In this study, we aimed to develop and validate a multimodal model for predicting BRCA1/2 gene status and prognosis with PARPi treatment.

METHODS

We included 1695 slides from 1417 patients with ovarian, breast, prostate, and pancreatic cancers across three independent cohorts. Using a self-attention mechanism, we constructed a multi-instance attention model (MIAM) to detect BRCA1/2 gene status from hematoxylin and eosin (H&E) pathological images. We further combined tissue features from the MIAM model, cell features, and clinical factors (the MIAM-C model) to predict BRCA1/2 mutations and progression-free survival (PFS) with PARPi therapy. Model performance was evaluated using area under the curve (AUC) and Kaplan-Meier analysis. Morphological features contributing to MIAM-C were analyzed for interpretability.

RESULTS

Across the four cancer types, MIAM-C outperformed the deep learning-based MIAM in identifying the BRCA1/2 genotype. Interpretability analysis revealed that high-attention regions included high-grade tumors and lymphocytic infiltration, which correlated with BRCA1/2 mutations. Notably, high lymphocyte ratios appeared characteristic of BRCA1/2 mutations. Furthermore, MIAM-C predicted PARPi therapy response (log-rank p < 0.05) and served as an independent prognostic factor for patients with BRCA1/2-mutant ovarian cancer (p < 0.05, hazard ratio:0.4, 95% confidence interval: 0.16-0.99).

CONCLUSIONS

The MIAM-C model accurately detected BRCA1/2 gene status and effectively stratified prognosis for patients with BRCA1/2 mutations.

AI based diagnostics product design for osteosarcoma cells microscopy imaging of bone cancer patients using CA-MobileNet V3

Objective

The incidence of osteosarcoma (OS) is low, but primary malignant bone tumors rank third among the causes of death in cancer patients under the age of 20. Currently, analysis of cellular structure and tumor morphology through microscopic images remains one of the main diagnostic methods for osteosarcoma. However, this completely manual approach is tedious, time-consuming, and difficult to diagnose accurately due to the similarities in certain characteristics of malignant and benign tumors.

Methods

Leveraging the potential of artificial intelligence (AI) in assessing and classifying images, this study explored a modified CA-MobileNet V3 model that was embedded into innovative microscope products to enhance the microscope's feature extraction capabilities and help reduce misclassification during diagnosis.

Results

The intelligent recognition model method introduced in this paper has significant advantages in retrieval and classification of osteosarcoma cells and other cell types. Compared with models such as ShuffleNet V2, EfficientNet V2, Mobilenet V3 (without transfer learning), TL-MobileNet V3 (with transfer learning), etc., the model size is only 5.33 MB, is a lightweight model, and the accuracy of the improved model reached 98.69 %. In addition, the artificial intelligence microscope (AIM) with integrated design based on this model can also help improve diagnostic efficiency.

Conclusion

The innovative method of the CA-MobileNet V3 automatic classification model based on deep learning provides an efficient and reliable solution for the pathological diagnosis of osteosarcoma. This study contributes to medical image analysis and provides doctors with an accurate and valuable tool for microscopic diagnosis. It also promotes the advancement of artificial intelligence in medical imaging technology.

Use of a Pathomics Nomogram to Predict Postoperative Liver Metastasis in Patients with Stage III Colorectal Cancer

BACKGROUND

Approximately 25% of patients with stage III colorectal cancer experience liver metastasis after radical resection; however, there is currently a lack of methods to predict liver metastasis. This study aims to develop and validate a pathomics nomogram to predict liver metastasis in patients with stage III colorectal cancer.

METHODS

A total of 318 enrolled patients were divided into three cohorts: a training cohort (n = 139), a validation cohort (n = 69), and an external cohort (n = 110). A competitive risk nomogram was established by the pathomics signature and clinicopathological characteristics and assessed by calibration, discrimination, and clinical usefulness.

RESULTS

A significant correlation between the pathomics signature and liver metastasis in stage III colorectal cancer was found. Multivariate Fine-Gray analysis indicated that preoperative carcinoembryonic antigen level, postoperative chemotherapy, and pathomics signature were independent predictors of liver metastasis. A competitive risk nomogram was developed to predict liver metastasis in patients with stage III colorectal cancer. The predicting nomogram shows good discrimination and calibration, with C-indexes of 0.811 (95% confidence interval [CI] 0.651-0.971), 0.759 (95% CI 0.531-0.987), and 0.845 (95% CI 0.641-0.999), with area under the receiver operating characteristic (AUROC) curves at 5 years of 0.833 (95% CI 0.742-0.925), 0.760 (95% CI 0.652-0.893), and 0.812 (95% CI 0.692-0.931) in the training, validation, and external cohorts, respectively. Compared with the clinicopathological nomogram, the nomogram combined with the pathomics signature had better performance (AUROC 0.823 [95% CI 0.764-0.881] vs. 0.678 [95% CI 0.606-0.751]; p < 0.001).

CONCLUSIONS

The pathomics signature is a predictive indicator for liver metastasis in patients with stage III colorectal cancer, and the integrated nomogram can be used to predict liver metastasis better than the clinicopathological nomogram alone.

Artificial Intelligence Applications in Lymphoma Diagnosis and Management: Opportunities, Challenges, and Future Directions

Lymphoma, a heterogeneous group of blood cancers, presents significant diagnostic and therapeutic challenges due to its complex subtypes and variable clinical outcomes. Artificial intelligence (AI) has emerged as a promising tool to enhance the accuracy and efficiency of lymphoma pathology. This review explores the potential of AI in lymphoma diagnosis, classification, prognosis prediction, and treatment planning, as well as addressing the challenges and future directions in this rapidly evolving field.

Machine learning models including patient-reported outcome data in oncology: a systematic literature review and analysis of their reporting quality

PURPOSE

To critically examine the current state of machine learning (ML) models including patient-reported outcome measure (PROM) scores in cancer research, by investigating the reporting quality of currently available studies and proposing areas of improvement for future use of ML in the field.

METHODS

PubMed and Web of Science were systematically searched for publications of studies on patients with cancer applying ML models with PROM scores as either predictors or outcomes. The reporting quality of applied ML models was assessed utilizing an adapted version of the MI-CLAIM (Minimum Information about CLinical Artificial Intelligence Modelling) checklist. The key variables of the checklist are study design, data preparation, model development, optimization, performance, and examination. Reproducibility and transparency complement the reporting quality criteria.

RESULTS

The literature search yielded 1634 hits, of which 52 (3.2%) were eligible. Thirty-six (69.2%) publications included PROM scores as a predictor and 32 (61.5%) as an outcome. Results of the reporting quality appraisal indicate a potential for improvement, especially in the areas of model examination. According to the standards of the MI-CLAIM checklist, the reporting quality of ML models in included studies proved to be low. Only nine (17.3%) publications present a discussion about the clinical applicability of the developed model and reproducibility and only three (5.8%) provide a code to reproduce the model and the results.

CONCLUSION

The herein performed critical examination of the status quo of the application of ML models including PROM scores in published oncological studies allowed the identification of areas of improvement for reporting and future use of ML in the field.

Artificial intelligence in healthcare: a mastery

There is a vast development of artificial intelligence (AI) in recent years. Computational technology, digitized data collection and enormous advancement in this field have allowed AI applications to penetrate the core human area of specialization. In this review article, we describe current progress achieved in the AI field highlighting constraints on smooth development in the field of medical AI sector, with discussion of its implementation in healthcare from a commercial, regulatory and sociological standpoint. Utilizing sizable multidimensional biological datasets that contain individual heterogeneity in genomes, functionality and milieu, precision medicine strives to create and optimize approaches for diagnosis, treatment methods and assessment. With the arise of complexity and expansion of data in the health-care industry, AI can be applied more frequently. The main application categories include indications for diagnosis and therapy, patient involvement and commitment and administrative tasks. There has recently been a sharp rise in interest in medical AI applications due to developments in AI software and technology, particularly in deep learning algorithms and in artificial neural network (ANN). In this overview, we enlisted the major categories of issues that AI systems are ideally equipped to resolve followed by clinical diagnostic tasks. It also includes a discussion of the future potential of AI, particularly for risk prediction in complex diseases, and the difficulties, constraints and biases that must be meticulously addressed for the effective delivery of AI in the health-care sector.

Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification

It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.

Pathomics signatures and cuproptosis-related genes signatures for prediction of prognosis in patients with hepatocellular carcinoma

Background

Hepatocellular carcinoma (HCC) is a common malignant tumor with high heterogeneity and poor prognosis, so early prediction and treatment are still difficult. Cuproptosis is a newly discovered type of programmed cell death that has been shown to be closely related to the occurrence and progression of HCC. Cancer morphology is influenced by genetic drivers, and computational pathology methods typically use tissue images such as entire slide images as input to predict clinical or genetic features. Therefore, the comprehensive analysis of pathological features and genomic data provides a feasible way to explore the potential mechanism of the tumor. The objective of this study was to develop a prediction model for HCC prognosis based on the pathomics signatures (PS) and the genomics signatures (GS).

Methods

A dataset comprising 315 HCC patients was randomly divided into a training set (n=200) and a validation set (n=115). Prognostic models related to PS and GS were constructed by univariate and multivariate Cox regression analyses and least absolute shrinkage and selection operator (LASSO) regression analysis. Kaplan-Meier survival analysis, receiver operating characteristic (ROC) curve, univariate and multivariate Cox analyses, and nomogram were used to evaluate the predictive performance of the prognostic model. The prognostic value of the model was internally validated.

Results

A prognostic model incorporating clinical features, PS, and GS was developed using Cox regression analysis and LASSO regression analyses. Kaplan-Meier survival analysis revealed statistically significant differences in survival time between high-risk and low-risk subgroups in both the training and validation datasets (PS: P=0.003 and <0.001, respectively; GS: P=0.008 and 0.004, respectively). The time-dependent ROC curve showed favorable predictive value for survival in both the training and validation sets. The area under the ROC curves at 1, 3, and 5 years was 0.750, 0.830, and 0.870 in the training set, and 0.780, 0.810, and 0.760 in the validation set, respectively. A nomogram model based on the risk model score could effectively predict the survival probability of HCC patients. The calibration curves further demonstrated the good predictive capability of the nomogram model.

Conclusions

The prognostic model incorporating PS and GS could effectively predict the prognosis of HCC patients.

Concordance in the estimation of tumor percentage in non-small cell lung cancer using digital pathology

The incorporation of digital pathology in clinical practice will require the training of pathologists in digital skills. Our study aimed to assess the reliability among pathologists in determining tumor percentage in whole slide images (WSI) of non-small cell lung cancer (NSCLC) using digital image analysis, and study how the results correlate with the molecular findings. Pathologists from nine centers were trained to quantify epithelial tumor cells, tumor-associated stromal cells, and non-neoplastic cells from NSCLC WSI using QuPath. Then, we conducted two consecutive ring trials. In the first trial, analyzing four WSI, reliability between pathologists in the assessment of tumor cell percentage was poor (intraclass correlation coefficient (ICC) 0.09). After performing the first ring trial pathologists received feedback. The second trial, comprising 10 WSI with paired next-generation sequencing results, also showed poor reliability (ICC 0.24). Cases near the recommended 20% visual threshold for molecular techniques exhibited higher values with digital analysis. In the second ring trial reliability slightly improved and human errors were reduced from 5.6% to 1.25%. Most discrepancies arose from subjective tasks, such as the annotation process, suggesting potential improvement with future artificial intelligence solutions.

A random survival forest-based pathomics signature classifies immunotherapy prognosis and profiles TIME and genomics in ES-SCLC patients

BACKGROUND

Small cell lung cancer (SCLC) is a highly aggressive neuroendocrine tumor with high mortality, and only a limited subset of extensive-stage SCLC (ES-SCLC) patients demonstrate prolonged survival under chemoimmunotherapy, which warrants the exploration of reliable biomarkers. Herein, we built a machine learning-based model using pathomics features extracted from hematoxylin and eosin (H&E)-stained images to classify prognosis and explore its potential association with genomics and TIME.

METHODS

We retrospectively recruited ES-SCLC patients receiving first-line chemoimmunotherapy at Nanjing Jinling Hospital between April 2020 and August 2023. Digital H&E-stained whole-slide images were acquired, and targeted next-generation sequencing, programmed death ligand-1 staining, and multiplex immunohistochemical staining for immune cells were performed on a subset of patients. A random survival forest (RSF) model encompassing clinical and pathomics features was established to predict overall survival. The function of putative genes was assessed via single-cell RNA sequencing.

RESULTS AND CONCLUSION

During the median follow-up period of 12.12 months, 118 ES-SCLC patients receiving first-line immunotherapy were recruited. The RSF model utilizing three pathomics features and liver metastases, bone metastases, smoking status, and lactate dehydrogenase, could predict the survival of first-line chemoimmunotherapy in patients with ES-SCLC with favorable discrimination and calibration. Underlyingly, the higher RSF-Score potentially indicated more infiltration of CD8+ T cells in the stroma as well as a greater probability of MCL-1 amplification and EP300 mutation. At the single-cell level, MCL-1 was associated with TNFA-NFKB signaling and apoptosis-related processes. Hopefully, this noninvasive model could act as a biomarker for immunotherapy, potentially facilitating precision medicine in the management of ES-SCLC.

Dual-stream multi-dependency graph neural network enables precise cancer survival analysis

Histopathology image-based survival prediction aims to provide a precise assessment of cancer prognosis and can inform personalized treatment decision-making in order to improve patient outcomes. However, existing methods cannot automatically model the complex correlations between numerous morphologically diverse patches in each whole slide image (WSI), thereby preventing them from achieving a more profound understanding and inference of the patient status. To address this, here we propose a novel deep learning framework, termed dual-stream multi-dependency graph neural network (DM-GNN), to enable precise cancer patient survival analysis. Specifically, DM-GNN is structured with the feature updating and global analysis branches to better model each WSI as two graphs based on morphological affinity and global co-activating dependencies. As these two dependencies depict each WSI from distinct but complementary perspectives, the two designed branches of DM-GNN can jointly achieve the multi-view modeling of complex correlations between the patches. Moreover, DM-GNN is also capable of boosting the utilization of dependency information during graph construction by introducing the affinity-guided attention recalibration module as the readout function. This novel module offers increased robustness against feature perturbation, thereby ensuring more reliable and stable predictions. Extensive benchmarking experiments on five TCGA datasets demonstrate that DM-GNN outperforms other state-of-the-art methods and offers interpretable prediction insights based on the morphological depiction of high-attention patches. Overall, DM-GNN represents a powerful and auxiliary tool for personalized cancer prognosis from histopathology images and has great potential to assist clinicians in making personalized treatment decisions and improving patient outcomes.

A pathology foundation model for cancer diagnosis and prognosis prediction

Histopathology image evaluation is indispensable for cancer diagnoses and subtype classification. Standard artificial intelligence methods for histopathology image analyses have focused on optimizing specialized models for each diagnostic task1,2. Although such methods have achieved some success, they often have limited generalizability to images generated by different digitization protocols or samples collected from different populations3. Here, to address this challenge, we devised the Clinical Histopathology Imaging Evaluation Foundation (CHIEF) model, a general-purpose weakly supervised machine learning framework to extract pathology imaging features for systematic cancer evaluation. CHIEF leverages two complementary pretraining methods to extract diverse pathology representations: unsupervised pretraining for tile-level feature identification and weakly supervised pretraining for whole-slide pattern recognition. We developed CHIEF using 60,530 whole-slide images spanning 19 anatomical sites. Through pretraining on 44 terabytes of high-resolution pathology imaging datasets, CHIEF extracted microscopic representations useful for cancer cell detection, tumour origin identification, molecular profile characterization and prognostic prediction. We successfully validated CHIEF using 19,491 whole-slide images from 32 independent slide sets collected from 24 hospitals and cohorts internationally. Overall, CHIEF outperformed the state-of-the-art deep learning methods by up to 36.1%, showing its ability to address domain shifts observed in samples from diverse populations and processed by different slide preparation methods. CHIEF provides a generalizable foundation for efficient digital pathology evaluation for patients with cancer.

Sequential Patching Lattice for Image Classification and Enquiry: Streamlining Digital Pathology Image Processing

Digital pathology and the integration of artificial intelligence (AI) models have revolutionized histopathology, opening new opportunities. With the increasing availability of whole-slide images (WSIs), demand is growing for efficient retrieval, processing, and analysis of relevant images from vast biomedical archives. However, processing WSIs presents challenges due to their large size and content complexity. Full computer digestion of WSIs is impractical, and processing all patches individually is prohibitively expensive. In this article, we propose an unsupervised patching algorithm, Sequential Patching Lattice for Image Classification and Enquiry (SPLICE). This novel approach condenses a histopathology WSI into a compact set of representative patches, forming a collage of WSI while minimizing redundancy. SPLICE prioritizes patch quality and uniqueness by sequentially analyzing a WSI and selecting nonredundant representative features. In search and match applications, SPLICE showed improved accuracy, reduced computation time, and storage requirements compared with existing state-of-the-art methods. As an unsupervised method, SPLICE effectively reduced storage requirements for representing tissue images by 50%. This reduction can enable numerous algorithms in computational pathology to operate much more efficiently, paving the way for accelerated adoption of digital pathology.

Leveraging State-of-the-Art AI Algorithms in Personalized Oncology: From Transcriptomics to Treatment

BACKGROUND

Continuous breakthroughs in computational algorithms have positioned AI-based models as some of the most sophisticated technologies in the healthcare system. AI shows dynamic contributions in advancing various medical fields involving data interpretation and monitoring, imaging screening and diagnosis, and treatment response and survival prediction. Despite advances in clinical oncology, more effort must be employed to tailor therapeutic plans based on each patient's unique transcriptomic profile within the precision/personalized oncology frame. Furthermore, the standard analysis method is not compatible with the comprehensive deciphering of significant data streams, thus precluding the prediction of accurate treatment options.

METHODOLOGY

We proposed a novel approach that includes obtaining different tumour tissues and preparing RNA samples for comprehensive transcriptomic interpretation using specifically trained, programmed, and optimized AI-based models for extracting large data volumes, refining, and analyzing them. Next, the transcriptomic results will be scanned against an expansive drug library to predict the response of each target to the tested drugs. The obtained target-drug combination/s will be then validated using in vitro and in vivo experimental models. Finally, the best treatment combination option/s will be introduced to the patient. We also provided a comprehensive review discussing AI models' recent innovations and implementations to aid in molecular diagnosis and treatment planning.

RESULTS

The expected transcriptomic analysis generated by the AI-based algorithms will provide an inclusive genomic profile for each patient, containing statistical and bioinformatics analyses, identification of the dysregulated pathways, detection of the targeted genes, and recognition of molecular biomarkers. Subjecting these results to the prediction and pairing AI-based processes will result in statistical graphs presenting each target's likely response rate to various treatment options. Different in vitro and in vivo investigations will further validate the selection of the target drug/s pairs.

CONCLUSIONS

Leveraging AI models will provide more rigorous manipulation of large-scale datasets on specific cancer care paths. Such a strategy would shape treatment according to each patient's demand, thus fortifying the avenue of personalized/precision medicine. Undoubtedly, this will assist in improving the oncology domain and alleviate the burden of clinicians in the coming decade.

Breast cancer prognosis through the use of multi-modal classifiers: current state of the art and the way forward

We present a survey of the current state-of-the-art in breast cancer detection and prognosis. We analyze the evolution of Artificial Intelligence-based approaches from using just uni-modal information to multi-modality for detection and how such paradigm shift facilitates the efficacy of detection, consistent with clinical observations. We conclude that interpretable AI-based predictions and ability to handle class imbalance should be considered priority.