The Emergence of Pathomics

Applying image features of proximal paracancerous tissues in predicting prognosis of patients with hepatocellular carcinoma

BACKGROUND

Most of the methods using digital pathological image for predicting Hepatocellular carcinoma (HCC) prognosis have not considered paracancerous tissue microenvironment (PTME), which are potentially important for tumour initiation and metastasis. This study aimed to identify roles of image features of PTME in predicting prognosis and tumour recurrence of HCC patients.

METHODS

We collected whole slide images (WSIs) of 146 HCC patients from Sun Yat-sen Memorial Hospital (SYSM dataset). For each WSI, five types of regions of interests (ROIs) in PTME and tumours were manually annotated. These ROIs were used to construct a Lasso Cox survival model for predicting the prognosis of HCC patients. To make the model broadly useful, we established a deep learning method to automatically segment WSIs, and further used it to construct a prognosis prediction model. This model was tested by the samples of 225 HCC patients from the Cancer Genome Atlas Liver Hepatocellular Carcinoma (TCGA-LIHC).

RESULTS

In predicting prognosis of the HCC patients, using the image features of manually annotated ROIs in PTME achieved C-index 0.668 in the SYSM testing dataset, which is higher than the C-index 0.648 reached by the model only using image features of tumours. Integrating ROIs of PTME and tumours achieved C-index 0.693 in the SYSM testing dataset. The model using automatically segmented ROIs of PTME and tumours achieved C-index of 0.665 (95% CI: 0.556-0.774) in the TCGA-LIHC samples, which is better than the widely used methods, WSISA (0.567), DeepGraphSurv (0.593), and SeTranSurv (0.642). Finally, we found the Texture SumAverage Skew HV on immune cell infiltration and Texture related features on desmoplastic reaction are the most important features of PTME in predicting HCC prognosis. We additionally used the model in prediction HCC recurrence for patients from SYSM-training, SYSM-testing, and TCGA-LIHC datasets, indicating the important roles of PTME in the prediction.

CONCLUSIONS

Our results indicate image features of PTME is critical for improving the prognosis prediction of HCC. Moreover, the image features related with immune cell infiltration and desmoplastic reaction of PTME are the most important factors associated with prognosis of HCC.

Shaping the future of oral cancer diagnosis: advances in salivary proteomics

INTRODUCTION

Saliva has gained increasing attention in the quest for disease biomarkers. Because it is a biological fluid that can be collected is an easy, painless, and safe way, it has been increasingly studied for the identification of oral cancer biomarkers. This is particularly important because oral cancer is often diagnosed at late stages with a poor prognosis.

AREAS COVERED

The review addresses the evolution of the experimental approaches used in salivary proteomics studies of oral cancer over the years and outlines advantages and pitfalls related to each one. In addition, examines the current landscape of oral cancer biomarker discovery and translation focusing on salivary proteomic studies. This discussion is based on an extensive literature search (PubMed, Scopus and Google Scholar).

EXPERT OPINION

The introduction of mass spectrometry has revolutionized the study of salivary proteomics. In the future, the focus will be on refining existing methods and introducing powerful experimental techniques such as mass spectrometry with selected reaction monitoring, which, despite their effectiveness, are still underutilized due to their high cost. In addition, conducting studies with larger cohorts and establishing standardized protocols for salivary proteomics are key challenges that need to be addressed in the coming years.

Machine learning-based pathomics signature of histology slides as a novel prognostic indicator in primary central nervous system lymphoma

PURPOSE

To develop and validate a pathomics signature for predicting the outcomes of Primary Central Nervous System Lymphoma (PCNSL).

METHODS

In this study, 132 whole-slide images (WSIs) of 114 patients with PCNSL were enrolled. Quantitative features of hematoxylin and eosin (H&E) stained slides were extracted using CellProfiler. A pathomics signature was established and validated. Cox regression analysis, receiver operating characteristic (ROC) curves, Calibration, decision curve analysis (DCA), and net reclassification improvement (NRI) were performed to assess the significance and performance.

RESULTS

In total, 802 features were extracted using a fully automated pipeline. Six machine-learning classifiers demonstrated high accuracy in distinguishing malignant neoplasms. The pathomics signature remained a significant factor of overall survival (OS) and progression-free survival (PFS) in the training cohort (OS: HR 7.423, p < 0.001; PFS: HR 2.143, p = 0.022) and independent validation cohort (OS: HR 4.204, p = 0.017; PFS: HR 3.243, p = 0.005). A significantly lower response rate to initial treatment was found in high Path-score group (19/35, 54.29%) as compared to patients in the low Path-score group (16/70, 22.86%; p < 0.001). The DCA and NRI analyses confirmed that the nomogram showed incremental performance compared with existing models. The ROC curve demonstrated a relatively sensitive and specific profile for the nomogram (1-, 2-, and 3-year AUC = 0.862, 0.932, and 0.927, respectively).

CONCLUSION

As a novel, non-invasive, and convenient approach, the newly developed pathomics signature is a powerful predictor of OS and PFS in PCNSL and might be a potential predictive indicator for therapeutic response.

tRigon: an R package and Shiny App for integrative (path-)omics data analysis

BACKGROUND

Pathomics facilitates automated, reproducible and precise histopathology analysis and morphological phenotyping. Similar to molecular omics, pathomics datasets are high-dimensional, but also face large outlier variability and inherent data missingness, making quick and comprehensible data analysis challenging. To facilitate pathomics data analysis and interpretation as well as support a broad implementation we developed tRigon (Toolbox foR InteGrative (path-)Omics data aNalysis), a Shiny application for fast, comprehensive and reproducible pathomics analysis.

RESULTS

tRigon is available via the CRAN repository ( https://cran.r-project.org/web/packages/tRigon ) with its source code available on GitLab ( https://git-ce.rwth-aachen.de/labooratory-ai/trigon ). The tRigon package can be installed locally and its application can be executed from the R console via the command 'tRigon::run_tRigon()'. Alternatively, the application is hosted online and can be accessed at https://labooratory.shinyapps.io/tRigon . We show fast computation of small, medium and large datasets in a low- and high-performance hardware setting, indicating broad applicability of tRigon.

CONCLUSIONS

tRigon allows researchers without coding abilities to perform exploratory feature analyses of pathomics and non-pathomics datasets on their own using a variety of hardware.

[Advances in computational quantitative nephropathology]

BACKGROUND

Semiquantitative histological scoring systems are frequently used in nephropathology. In computational nephropathology, the focus is on generating quantitative data from histology (so-called pathomics). Several recent studies have collected such data using next-generation morphometry (NGM) based on segmentations by artificial neural networks and investigated their usability for various clinical or diagnostic purposes.

AIM

To present an overview of the current state of studies regarding renal pathomics and to identify current challenges and potential solutions.

MATERIALS AND METHODS

Due to the literature restriction (maximum of 30 references), studies were selected based on a database search that processed as much data as possible, used innovative methodologies, and/or were ideally multicentric in design.

RESULTS AND DISCUSSION

Pathomics studies in the kidney have impressively demonstrated that morphometric data are useful clinically (for example, for prognosis assessment) and translationally. Further development of NGM requires overcoming some challenges, including better standardization and generation of prospective evidence.

Artificial intelligence and machine learning for clinical pharmacology

Artificial intelligence (AI) will impact many aspects of clinical pharmacology, including drug discovery and development, clinical trials, personalized medicine, pharmacogenomics, pharmacovigilance and clinical toxicology. The rapid progress of AI in healthcare means clinical pharmacologists should have an understanding of AI and its implementation in clinical practice. As with any new therapy or health technology, it is imperative that AI tools are subject to robust and stringent evaluation to ensure that they enhance clinical practice in a safe and equitable manner. This review serves as an introduction to AI for the clinical pharmacologist, highlighting current applications, aspects of model development and issues surrounding evaluation and deployment. The aim of this article is to empower clinical pharmacologists to embrace and lead on the safe and effective use of AI within healthcare.

Clinical application of machine learning-based pathomics signature of gastric atrophy

Background

The diagnosis of gastric atrophy is highly subjective, and we aimed to establish a model of gastric atrophy based on pathological features to improve diagnostic consistency.

Methods

We retrospectively collected the HE-stained pathological slides of gastric biopsies and used CellProfiler software for image segmentation and feature extraction of ten representative images for each sample. Subsequently, we employed the Least absolute shrinkage and selection operator (LASSO) to select features and different machine learning (ML) algorithms to construct the diagnostic models for gastric atrophy.

Results

We selected 289 gastric biopsy specimens for training, testing, and external validation. We extracted 464 pathological features and screened ten features by LASSO to establish the diagnostic model for moderate-to-severe atrophy. The range of area under the curve (AUC) for various machine learning algorithms was 0.835-1.000 in the training set, 0.786-0.949 in the testing set, and 0.689-0.818 in the external validation set. LR model had the highest AUC value, with 0.900 (95% CI: 0.852-0.947) in the training set, 0.901 (95% CI: 0.807-0.996) in the testing set, and 0.818 (95% CI: 0.714-0.923) in the external validation set. The atrophy pathological score based on the LR model was associated with endoscopic atrophy grading (Z=-2.478, P=0.013) and gastric cancer (GC) (OR=5.70, 95% CI: 2.63-12.33, P<0.001).

Conclusion

The ML model based on pathological features could improve the diagnostic consistency of gastric atrophy, which is also associated with endoscopic atrophy grading and GC.

Quantitative assessment of H&E staining for pathology: development and clinical evaluation of a novel system

BACKGROUND

Staining tissue samples to visualise cellular detail and tissue structure is at the core of pathology diagnosis, but variations in staining can result in significantly different appearances of the tissue sample. While the human visual system is adept at compensating for stain variation, with the growth of digital imaging in pathology, the impact of this variation can be more profound. Despite the ubiquity of haematoxylin and eosin staining in clinical practice worldwide, objective quantification is not yet available. We propose a method for quantitative haematoxylin and eosin stain assessment to facilitate quality assurance of histopathology staining, enabling truly quantitative quality control and improved standardisation.

METHODS

The stain quantification method comprises conventional microscope slides with a stain-responsive biopolymer film affixed to one side, called stain assessment slides. The stain assessment slides were characterised with haematoxylin and eosin, and implemented in one clinical laboratory to quantify variation levels.

RESULTS

Stain assessment slide stain uptake increased linearly with duration of haematoxylin and eosin staining (r = 0.99), and demonstrated linearly comparable staining to samples of human liver tissue (r values 0.98-0.99). Laboratory implementation of this technique quantified intra- and inter-instrument variation of staining instruments at one point in time and across a five-day period.

CONCLUSION

The proposed method has been shown to reliably quantify stain uptake, providing an effective laboratory quality control method for stain variation. This is especially important for whole slide imaging and the future development of artificial intelligence in digital pathology.

Standardizing digital biobanks: integrating imaging, genomic, and clinical data for precision medicine

Advancements in data acquisition and computational methods are generating a large amount of heterogeneous biomedical data from diagnostic domains such as clinical imaging, pathology, and next-generation sequencing (NGS), which help characterize individual differences in patients. However, this information needs to be available and suitable to promote and support scientific research and technological development, supporting the effective adoption of the precision medicine approach in clinical practice. Digital biobanks can catalyze this process, facilitating the sharing of curated and standardized imaging data, clinical, pathological and molecular data, crucial to enable the development of a comprehensive and personalized data-driven diagnostic approach in disease management and fostering the development of computational predictive models. This work aims to frame this perspective, first by evaluating the state of standardization of individual diagnostic domains and then by identifying challenges and proposing a possible solution towards an integrative approach that can guarantee the suitability of information that can be shared through a digital biobank. Our analysis of the state of the art shows the presence and use of reference standards in biobanks and, generally, digital repositories for each specific domain. Despite this, standardization to guarantee the integration and reproducibility of the numerical descriptors generated by each domain, e.g. radiomic, pathomic and -omic features, is still an open challenge. Based on specific use cases and scenarios, an integration model, based on the JSON format, is proposed that can help address this problem. Ultimately, this work shows how, with specific standardization and promotion efforts, the digital biobank model can become an enabling technology for the comprehensive study of diseases and the effective development of data-driven technologies at the service of precision medicine.

Deep learning-based automated pipeline for blood vessel detection and distribution analysis in multiplexed prostate cancer images

Introduction: Prostate cancer is a highly heterogeneous disease, presenting varying levels of aggressiveness and response to treatment. Angiogenesis is one of the hallmarks of cancer, providing oxygen and nutrient supply to tumors. Micro vessel density has previously been correlated with higher Gleason score and poor prognosis. Manual segmentation of blood vessels (BVs) In microscopy images is challenging, time consuming and may be prone to inter-rater variabilities. In this study, an automated pipeline is presented for BV detection and distribution analysis in multiplexed prostate cancer images. Methods: A deep learning model was trained to segment BVs by combining CD31, CD34 and collagen IV images. In addition, the trained model was used to analyze the size and distribution patterns of BVs in relation to disease progression in a cohort of prostate cancer patients (N = 215). Results: The model was capable of accurately detecting and segmenting BVs, as compared to ground truth annotations provided by two reviewers. The precision (P), recall (R) and dice similarity coefficient (DSC) were equal to 0.93 (SD 0.04), 0.97 (SD 0.02) and 0.71 (SD 0.07) with respect to reviewer 1, and 0.95 (SD 0.05), 0.94 (SD 0.07) and 0.70 (SD 0.08) with respect to reviewer 2, respectively. BV count was significantly associated with 5-year recurrence (adjusted p = 0.0042), while both count and area of blood vessel were significantly associated with Gleason grade (adjusted p = 0.032 and 0.003 respectively). Discussion: The proposed methodology is anticipated to streamline and standardize BV analysis, offering additional insights into the biology of prostate cancer, with broad applicability to other cancers.

The Pulse of AI: Implementation of Artificial Intelligence in Healthcare and its Potential Hazards

In this editorial, we explore the existing utilization of artificial intelligence (AI) within the healthcare industry, examining both its scope and potential harms if implemented and relied upon on a broader scale. Collaboration among corporations, government bodies, policymakers, and medical experts is essential to address potential concerns, ensuring smooth AI integration into healthcare systems.

Spatial multi-omics: novel tools to study the complexity of cardiovascular diseases

Spatial multi-omic studies have emerged as a promising approach to comprehensively analyze cells in tissues, enabling the joint analysis of multiple data modalities like transcriptome, epigenome, proteome, and metabolome in parallel or even the same tissue section. This review focuses on the recent advancements in spatial multi-omics technologies, including novel data modalities and computational approaches. We discuss the advancements in low-resolution and high-resolution spatial multi-omics methods which can resolve up to 10,000 of individual molecules at subcellular level. By applying and integrating these techniques, researchers have recently gained valuable insights into the molecular circuits and mechanisms which govern cell biology along the cardiovascular disease spectrum. We provide an overview of current data analysis approaches, with a focus on data integration of multi-omic datasets, highlighting strengths and weaknesses of various computational pipelines. These tools play a crucial role in analyzing and interpreting spatial multi-omics datasets, facilitating the discovery of new findings, and enhancing translational cardiovascular research. Despite nontrivial challenges, such as the need for standardization of experimental setups, data analysis, and improved computational tools, the application of spatial multi-omics holds tremendous potential in revolutionizing our understanding of human disease processes and the identification of novel biomarkers and therapeutic targets. Exciting opportunities lie ahead for the spatial multi-omics field and will likely contribute to the advancement of personalized medicine for cardiovascular diseases.

Open and reusable deep learning for pathology with WSInfer and QuPath

Digital pathology has seen a proliferation of deep learning models in recent years, but many models are not readily reusable. To address this challenge, we developed WSInfer: an open-source software ecosystem designed to streamline the sharing and reuse of deep learning models for digital pathology. The increased access to trained models can augment research on the diagnostic, prognostic, and predictive capabilities of digital pathology.

Establishment of a pathomic-based machine learning model to predict CD276 (B7-H3) expression in colon cancer

CD276 is a promising prognostic indicator and an attractive therapeutic target in various malignancies. However, current methods for CD276 detection are time-consuming and expensive, limiting extensive studies and applications of CD276. We aimed to develop a pathomic model for CD276 prediction from H&E-stained pathological images, and explore the underlying mechanism of the pathomic features by associating the pathomic model with transcription profiles. A dataset of colon adenocarcinoma (COAD) patients was retrieved from the Cancer Genome Atlas (TCGA) database. The dataset was divided into the training and validation sets according to the ratio of 8:2 by a stratified sampling method. Using the gradient boosting machine (GBM) algorithm, we established a pathomic model to predict CD276 expression in COAD. Univariate and multivariate Cox regression analyses were conducted to assess the predictive performance of the pathomic model for overall survival in COAD. Gene Set Enrichment Analysis (GESA) was performed to explore the underlying biological mechanisms of the pathomic model. The pathomic model formed by three pathomic features for CD276 prediction showed an area under the curve (AUC) of 0.833 (95%CI: 0.784-0.882) in the training set and 0.758 (95%CI: 0.637-0.878) in the validation set, respectively. The calibration curves and Hosmer-Lemeshow goodness of fit test showed that the prediction probability of high/low expression of CD276 was in favorable agreement with the real situation in both the training and validation sets (P=0.176 and 0.255, respectively). The DCA curves suggested that the pathomic model acquired high clinical benefit. All the subjects were categorized into high pathomic score (PS) (PS-H) and low PS (PS-L) groups according to the cutoff value of PS. Univariate and multivariate Cox regression analysis indicated that PS was a risk factor for overall survival in COAD. Furthermore, through GESA analysis, we found several immune and inflammatory-related pathways and genes were associated with the pathomic model. We constructed a pathomics-based machine learning model for CD276 prediction directly from H&E-stained images in COAD. Through integrated analysis of the pathomic model and transcriptomics, the interpretability of the pathomic model provide a theoretical basis for further hypothesis and experimental research.

The modified Polsby-Popper score, a novel quantitative histomorphological biomarker and its potential to predict lymph node positivity and cancer-specific survival in oral tongue squamous cell carcinoma

BACKGROUND

The significance of different histological spreading patterns of tumor tissue in oral tongue squamous cell carcinoma (TSCC) is well known. Our aim was to construct a numeric parameter on a continuous scale, that is, the modified Polsby-Popper (MPP) score, to describe the aggressiveness of tumor growth and infiltration, with the potential to analyze hematoxylin and eosin-stained whole slide images (WSIs) in an automated manner. We investigated the application of the MPP score in predicting survival and cervical lymph node metastases as well as in determining patients at risk in the context of different surgical margin scenarios.

METHODS

We developed a semiautomated image analysis pipeline to detect areas belonging to the tumor tissue compartment. Perimeter and area measurements of all detected tissue regions were derived, and a specific mathematical formula was applied to reflect the perimeter/area ratio in a comparable, observer-independent manner across digitized WSIs. We demonstrated the plausibility of the MPP score by correlating it with well-established clinicopathologic parameters. We then performed survival analysis to assess the relevance of the MPP score, with an emphasis on different surgical margin scenarios. Machine learning models were developed to assess the relevance of the MPP score in predicting survival and occult cervical nodal metastases.

RESULTS

The MPP score was associated with unfavorable tumor growth and infiltration patterns, the presence of lymph node metastases, the extracapsular spread of tumor cells, and higher tumor thickness. Higher MPP scores were associated with worse overall survival (OS) and tongue carcinoma-specific survival (TCSS), both when assessing all pT-categories and pT1-pT2 categories only; moreover, higher MPP scores were associated with a significantly worse TCSS in cases where a cancer-free surgical margin of <5 mm could be achieved on the main surgical specimen. This discriminatory capacity remained constant when examining pT1-pT2 categories only. Importantly, the MPP score could successfully define cases at risk in terms of metastatic disease in pT1-pT2 cancer where tumor thickness failed to exhibit a significant predictive value. Machine learning (ML) models incorporating the MPP score could predict the 5-year TCSS efficiently. Furthermore, we demonstrated that machine learning models that predict occult cervical lymph node involvement can benefit from including the MPP score.

CONCLUSIONS

We introduced an objective, quantifiable, and observer-independent parameter, the MPP score, representing the aggressiveness of tumor growth and infiltration in TSCC. We showed its prognostic relevance especially in pT1-pT2 category TSCC, and its possible use in ML models predicting TCSS and occult lymph node metastases.

Pathomic Features Reveal Immune and Molecular Evolution From Lung Preneoplasia to Invasive Adenocarcinoma

Recent statistics on lung cancer, including the steady decline of advanced diseases and the dramatically increasing detection of early-stage diseases and indeterminate pulmonary nodules, mark the significance of a comprehensive understanding of early lung carcinogenesis. Lung adenocarcinoma (ADC) is the most common histologic subtype of lung cancer, and atypical adenomatous hyperplasia is the only recognized preneoplasia to ADC, which may progress to adenocarcinoma in situ (AIS) and minimally invasive adenocarcinoma (MIA) and eventually to invasive ADC. Although molecular evolution during early lung carcinogenesis has been explored in recent years, the progress has been significantly hindered, largely due to insufficient materials from ADC precursors. Here, we employed state-of-the-art deep learning and artificial intelligence techniques to robustly segment and recognize cells on routinely used hematoxylin and eosin histopathology images and extracted 9 biology-relevant pathomic features to decode lung preneoplasia evolution. We analyzed 3 distinct cohorts (Japan, China, and United States) covering 98 patients, 162 slides, and 669 regions of interest, including 143 normal, 129 atypical adenomatous hyperplasia, 94 AIS, 98 MIA, and 205 ADC. Extracted pathomic features revealed progressive increase of atypical epithelial cells and progressive decrease of lymphocytic cells from normal to AAH, AIS, MIA, and ADC, consistent with the results from tissue-consuming and expensive molecular/immune profiling. Furthermore, pathomics analysis manifested progressively increasing cellular intratumor heterogeneity along with the evolution from normal lung to invasive ADC. These findings demonstrated the feasibility and substantial potential of pathomics in studying lung cancer carcinogenesis directly from the low-cost routine hematoxylin and eosin staining.

Clinical Application of Digital and Computational Pathology in Renal Cell Carcinoma: A Systematic Review

CONTEXT

Computational pathology is a new interdisciplinary field that combines traditional pathology with modern technologies such as digital imaging and machine learning to better understand the diagnosis, prognosis, and natural history of many diseases.

OBJECTIVE

To provide an overview of digital and computational pathology and its current and potential applications in renal cell carcinoma (RCC).

EVIDENCE ACQUISITION

A systematic review of the English-language literature was conducted using the PubMed, Web of Science, and Scopus databases in December 2022 according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PROSPERO ID: CRD42023389282). Risk of bias was assessed according to the Prediction Model Study Risk of Bias Assessment Tool.

EVIDENCE SYNTHESIS

In total, 20 articles were included in the review. All the studies used a retrospective design, and all digital pathology techniques were implemented retrospectively. The studies were classified according to their primary objective: detection, tumor characterization, and patient outcome. Regarding the transition to clinical practice, several studies showed promising potential. However, none presented a comprehensive assessment of clinical utility and implementation. Notably, there was substantial heterogeneity for both the strategies used for model building and the performance metrics reported.

CONCLUSIONS

This review highlights the vast potential of digital and computational pathology for the detection, classification, and assessment of oncological outcomes in RCC. Preliminary work in this field has yielded promising results. However, these models have not yet reached a stage where they can be integrated into routine clinical practice.

PATIENT SUMMARY

Computational pathology combines traditional pathology and technologies such as digital imaging and artificial intelligence to improve diagnosis of disease and identify prognostic factors and new biomarkers. The number of studies exploring its potential in kidney cancer is rapidly increasing. However, despite the surge in research activity, computational pathology is not yet ready for widespread routine use.

Revolutionizing Digital Pathology With the Power of Generative Artificial Intelligence and Foundation Models

Digital pathology has transformed the traditional pathology practice of analyzing tissue under a microscope into a computer vision workflow. Whole-slide imaging allows pathologists to view and analyze microscopic images on a computer monitor, enabling computational pathology. By leveraging artificial intelligence (AI) and machine learning (ML), computational pathology has emerged as a promising field in recent years. Recently, task-specific AI/ML (eg, convolutional neural networks) has risen to the forefront, achieving above-human performance in many image-processing and computer vision tasks. The performance of task-specific AI/ML models depends on the availability of many annotated training datasets, which presents a rate-limiting factor for AI/ML development in pathology. Task-specific AI/ML models cannot benefit from multimodal data and lack generalization, eg, the AI models often struggle to generalize to new datasets or unseen variations in image acquisition, staining techniques, or tissue types. The 2020s are witnessing the rise of foundation models and generative AI. A foundation model is a large AI model trained using sizable data, which is later adapted (or fine-tuned) to perform different tasks using a modest amount of task-specific annotated data. These AI models provide in-context learning, can self-correct mistakes, and promptly adjust to user feedback. In this review, we provide a brief overview of recent advances in computational pathology enabled by task-specific AI, their challenges and limitations, and then introduce various foundation models. We propose to create a pathology-specific generative AI based on multimodal foundation models and present its potentially transformative role in digital pathology. We describe different use cases, delineating how it could serve as an expert companion of pathologists and help them efficiently and objectively perform routine laboratory tasks, including quantifying image analysis, generating pathology reports, diagnosis, and prognosis. We also outline the potential role that foundation models and generative AI can play in standardizing the pathology laboratory workflow, education, and training.

Revolutionizing Oral Cancer Detection: An Approach Using Aquila and Gorilla Algorithms Optimized Transfer Learning-Based CNNs

The early detection of oral cancer is pivotal for improving patient survival rates. However, the high cost of manual initial screenings poses a challenge, especially in resource-limited settings. Deep learning offers an enticing solution by enabling automated and cost-effective screening. This study introduces a groundbreaking empirical framework designed to revolutionize the accurate and automatic classification of oral cancer using microscopic histopathology slide images. This innovative system capitalizes on the power of convolutional neural networks (CNNs), strengthened by the synergy of transfer learning (TL), and further fine-tuned using the novel Aquila Optimizer (AO) and Gorilla Troops Optimizer (GTO), two cutting-edge metaheuristic optimization algorithms. This integration is a novel approach, addressing bias and unpredictability issues commonly encountered in the preprocessing and optimization phases. In the experiments, the capabilities of well-established pre-trained TL models, including VGG19, VGG16, MobileNet, MobileNetV3Small, MobileNetV2, MobileNetV3Large, NASNetMobile, and DenseNet201, all initialized with 'ImageNet' weights, were harnessed. The experimental dataset consisted of the Histopathologic Oral Cancer Detection dataset, which includes a 'normal' class with 2494 images and an 'OSCC' (oral squamous cell carcinoma) class with 2698 images. The results reveal a remarkable performance distinction between the AO and GTO, with the AO consistently outperforming the GTO across all models except for the Xception model. The DenseNet201 model stands out as the most accurate, achieving an astounding average accuracy rate of 99.25% with the AO and 97.27% with the GTO. This innovative framework signifies a significant leap forward in automating oral cancer detection, showcasing the tremendous potential of applying optimized deep learning models in the realm of healthcare diagnostics. The integration of the AO and GTO in our CNN-based system not only pushes the boundaries of classification accuracy but also underscores the transformative impact of metaheuristic optimization techniques in the field of medical image analysis.

Immune modulation during anti-cancer radio(immuno)therapy

Cancer can affect all human organs and tissues and ranks as a prominent cause of death as well as an obstruction to increasing life expectancy. A notable breakthrough in oncology has been the inclusion of the immune system in fighting cancer, potentially prolonging life and providing long-term benefits. The concept of "immunotherapy" has been discussed from the 19th and early 20th centuries by Wilhelm Busch, William B. Coley and Paul Ehrlich. This involves distinct approaches, including vaccines, non-specific cytokines and adoptive cell therapies. However, despite the advances made in recent years, questions on how to select the best therapeutic options or how to select the best combinations to improve clinical outcomes are still relevant for scientists and clinicians. More than half of cancer patients receive radiotherapy (RT) as part of their treatment. With the advances in RT and immunotherapy approaches, it is reasonable to consider how to enhance immunotherapy with radiation and vice versa, and to investigate whether combinations of these therapies would be beneficial. In this chapter, we will discuss how the immune system responds to cancer cells and different cancer therapies with a focus on combination of RT and immunotherapy (radioimmunotherapy, RIT).

Recent advances of pathomics in colorectal cancer diagnosis and prognosis

Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.

Identification and Analysis of Imaging-Genomic Signatures to Study Recurrence in Breast Cancers

One of the main causes of breast cancer related death is its recurrence. In this study, we investigate the association of gene expression and pathological image features to understand breast cancer recurrence. A total of 172 breast cancer patient data was downloaded from the TCGA-BRCA database. The dataset contained diagnostic whole slide images and RNA-seq data of 80 recurrent and 92 disease-free breast cancer patients. We performed genomic analysis on RNA-seq data to obtain the hub genes related to recurrent breast cancer. We extracted relevant pathomic features from histopathology images. The discriminative ability of the hub genes and pathomic features were evaluated using machine learning classifiers. We used Spearman rank correlation analysis to find statistically significant association between gene expression and pathomic features. We identified that, genes which are related to breast cancer progression is significantly associated (adjusted p-value<0.05) with several pathomic features.Clinical Relevance- Histopathology is the gold standard for cancer detection. It provides us with cellular level information. A strong association between a pathomic feature and a gene expression will help clinicians understand the cellular and molecular mechanism of cancer for better prognosis.

Vaccines in Breast Cancer: Challenges and Breakthroughs

Breast cancer is a problem for women's health globally. Early detection techniques come in a variety of forms ranging from local to systemic and from non-invasive to invasive. The treatment of cancer has always been challenging despite the availability of a wide range of therapeutics. This is either due to the variable behaviour and heterogeneity of the proliferating cells and/or the individual's response towards the treatment applied. However, advancements in cancer biology and scientific technology have changed the course of the cancer treatment approach. This current review briefly encompasses the diagnostics, the latest and most recent breakthrough strategies and challenges, and the limitations in fighting breast cancer, emphasising the development of breast cancer vaccines. It also includes the filed/granted patents referring to the same aspects.

The slow-paced digital evolution of pathology: lights and shadows from a multifaceted board

Objective

The digital revolution in pathology represents an invaluable resource fto optimise costs, reduce the risk of error and improve patient care, even though it is still adopted in a minority of laboratories. Barriers include concerns about initial costs, lack of confidence in using whole slide images for primary diagnosis, and lack of guidance on transition. To address these challenges and develop a programme to facilitate the introduction of digital pathology (DP) in Italian pathology departments, a panel discussion was set up to identify the key points to be considered.

Methods

On 21 July 2022, an initial conference call was held on Zoom to identify the main issues to be discussed during the face-to-face meeting. The final summit was divided into four different sessions: (I) the definition of DP, (II) practical applications of DP, (III) the use of AI in DP, (IV) DP and education.

Results

Essential requirements for the implementation of DP are a fully tracked and automated workflow, selection of the appropriate scanner based on the specific needs of each department, and a strong commitment combined with coordinated teamwork (pathologists, technicians, biologists, IT service and industries). This could reduce human error, leading to the application of AI tools for diagnosis, prognosis and prediction. Open challenges are the lack of specific regulations for virtual slide storage and the optimal storage solution for large volumes of slides.

Conclusion

Teamwork is key to DP transition, including close collaboration with industry. This will ease the transition and help bridge the gap that currently exists between many labs and full digitisation. The ultimate goal is to improve patient care.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Multimodal data fusion for cancer biomarker discovery with deep learning

Technological advances now make it possible to study a patient from multiple angles with high-dimensional, high-throughput multi-scale biomedical data. In oncology, massive amounts of data are being generated ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has significantly advanced the analysis of biomedical data. However, most approaches focus on single data modalities leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalised medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability, and standardisation of datasets.

Tumor Cellularity Assessment of Breast Histopathological Slides via Instance Segmentation and Pathomic Features Explainability

The segmentation and classification of cell nuclei are pivotal steps in the pipelines for the analysis of bioimages. Deep learning (DL) approaches are leading the digital pathology field in the context of nuclei detection and classification. Nevertheless, the features that are exploited by DL models to make their predictions are difficult to interpret, hindering the deployment of such methods in clinical practice. On the other hand, pathomic features can be linked to an easier description of the characteristics exploited by the classifiers for making the final predictions. Thus, in this work, we developed an explainable computer-aided diagnosis (CAD) system that can be used to support pathologists in the evaluation of tumor cellularity in breast histopathological slides. In particular, we compared an end-to-end DL approach that exploits the Mask R-CNN instance segmentation architecture with a two steps pipeline, where the features are extracted while considering the morphological and textural characteristics of the cell nuclei. Classifiers that are based on support vector machines and artificial neural networks are trained on top of these features in order to discriminate between tumor and non-tumor nuclei. Afterwards, the SHAP (Shapley additive explanations) explainable artificial intelligence technique was employed to perform a feature importance analysis, which led to an understanding of the features processed by the machine learning models for making their decisions. An expert pathologist validated the employed feature set, corroborating the clinical usability of the model. Even though the models resulting from the two-stage pipeline are slightly less accurate than those of the end-to-end approach, the interpretability of their features is clearer and may help build trust for pathologists to adopt artificial intelligence-based CAD systems in their clinical workflow. To further show the validity of the proposed approach, it has been tested on an external validation dataset, which was collected from IRCCS Istituto Tumori "Giovanni Paolo II" and made publicly available to ease research concerning the quantification of tumor cellularity.

A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images

Background and objectives

The detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature.

Methods

A dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves.

Results

21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique.

Conclusions

The presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.

Predicting peritoneal recurrence in gastric cancer with serosal invasion using a pathomics nomogram

Peritoneal recurrence is the most frequent and lethal recurrence pattern in gastric cancer (GC) with serosal invasion after radical surgery. However, current evaluation methods are not adequate for predicting peritoneal recurrence in GC with serosal invasion. Emerging evidence shows that pathomics analyses could be advantageous for risk stratification and outcome prediction. Herein, we propose a pathomics signature composed of multiple pathomics features extracted from digital hematoxylin and eosin-stained images. We found that the pathomics signature was significantly associated with peritoneal recurrence. A competing-risk pathomics nomogram including carbohydrate antigen 19-9 level, depth of invasion, lymph node metastasis, and pathomics signature was developed for predicting peritoneal recurrence. The pathomics nomogram had favorable discrimination and calibration. Thus, the pathomics signature is a predictive indicator of peritoneal recurrence, and the pathomics nomogram may provide a helpful reference for predicting an individual's risk in peritoneal recurrence of GC with serosal invasion.

An advanced molecular medicine case report of a rare human tumor using genomics, pathomics, and radiomics

Background: Pulmonary Sclerosing Pneumocytoma (PSP) is a rare tumor of the lung with a low malignant potential that primarily affects females. Initial studies of PSP focused primarily on analyzing features uncovered using conventional X-ray or CT imaging. In recent years, because of the widespread use of next-generation sequencing (NGS), the study of PSP at the molecular-level has emerged. Methods: Analytical approaches involving genomics, radiomics, and pathomics were performed. Genomics studies involved both DNA and RNA analyses. DNA analyses included the patient's tumor and germline tissues and involved targeted panel sequencing and copy number analyses. RNA analyses included tumor and adjacent normal tissues and involved studies covering expressed mutations, differential gene expression, gene fusions and molecular pathways. Radiomics approaches were utilized on clinical imaging studies and pathomics techniques were applied to tumor whole slide images. Results: A comprehensive molecular profiling endeavor involving over 50 genomic analyses corresponding to 16 sequencing datasets of this rare neoplasm of the lung were generated along with detailed radiomic and pathomic analyses to reveal insights into the etiology and molecular behavior of the patient's tumor. Driving mutations (AKT1) and compromised tumor suppression pathways (TP53) were revealed. To ensure the accuracy and reproducibility of this study, a software infrastructure and methodology known as NPARS, which encapsulates NGS and associated data, open-source software libraries and tools including versions, and reporting features for large and complex genomic studies was used. Conclusion: Moving beyond descriptive analyses towards more functional understandings of tumor etiology, behavior, and improved therapeutic predictability requires a spectrum of quantitative molecular medicine approaches and integrations. To-date this is the most comprehensive study of a patient with PSP, which is a rare tumor of the lung. Detailed radiomic, pathomic and genomic molecular profiling approaches were performed to reveal insights regarding the etiology and molecular behavior. In the event of recurrence, a rational therapy plan is proposed based on the uncovered molecular findings.

Basic Elements of Artificial Intelligence Tools in the Diagnosis of Cutaneous Melanoma

Cutaneous melanoma (CM) incidence has dramatically increased in the last years. Early diagnosis is of paramount importance in terms of prognosis. Artificial Intelligence (AI) tools are being proposed for clinicians and pathologists as an adjunct support in the diagnostic process. We described herein an overview of the most important parameters that a potential AI tool should take into consideration in histopathology to evaluate a skin lesion. First of all, recognition of a melanocytic or non-melanocytic nature. Furthermore, melanocytic lesions should be stratified according to at least four parameters: silhouette and asymmetry; identification and spatial distribution of the cells; mitosis count; presence of ulceration. According to the number of parameters the AI tools might stratify the risk of CM and prioritize the pathologist's work.

Radio-pathomic approaches in pediatric neuro-oncology: Opportunities and challenges

With medical software platforms moving to cloud environments with scalable storage and computing, the translation of predictive artificial intelligence (AI) models to aid in clinical decision-making and facilitate personalized medicine for cancer patients is becoming a reality. Medical imaging, namely radiologic and histologic images, has immense analytical potential in neuro-oncology, and models utilizing integrated radiomic and pathomic data may yield a synergistic effect and provide a new modality for precision medicine. At the same time, the ability to harness multi-modal data is met with challenges in aggregating data across medical departments and institutions, as well as significant complexity in modeling the phenotypic and genotypic heterogeneity of pediatric brain tumors. In this paper, we review recent pathomic and integrated pathomic, radiomic, and genomic studies with clinical applications. We discuss current challenges limiting translational research on pediatric brain tumors and outline technical and analytical solutions. Overall, we propose that to empower the potential residing in radio-pathomics, systemic changes in cross-discipline data management and end-to-end software platforms to handle multi-modal data sets are needed, in addition to embracing modern AI-powered approaches. These changes can improve the performance of predictive models, and ultimately the ability to advance brain cancer treatments and patient outcomes through the development of such models.

Bladder Cancer Radiation Oncology of the Future: Prognostic Modelling, Radiomics, and Treatment Planning With Artificial Intelligence

Machine learning (ML) and artificial intelligence (AI) have demonstrated potential to improve the care of radiation oncology patients. Here we review recent advances applicable to the care of bladder cancer, with an eye towards studies that may suggest next steps in clinical implementation. Algorithms have been applied to clinical records, pathology, and radiology data to generate accurate predictive models for prognosis and clinical outcomes. AI has also shown increasing utility for auto-contouring and efficient creation of workflows involving multiple treatment plans. As technologies progress towards routine clinical use for bladder cancer patients, we also discuss emerging methods to improve interpretability and reliability of algorithms.

Prognostic and predictive value of a pathomics signature in gastric cancer

The current tumour-node-metastasis (TNM) staging system alone cannot provide adequate information for prognosis and adjuvant chemotherapy benefits in patients with gastric cancer (GC). Pathomics, which is based on the development of digital pathology, is an emerging field that might improve clinical management. Herein, we propose a pathomics signature (PS_GC) that is derived from multiple pathomics features of haematoxylin and eosin-stained slides. We find that the PS_GC is an independent predictor of prognosis. A nomogram incorporating the PS_GC and TNM staging system shows significantly improved accuracy in predicting the prognosis compared to the TNM staging system alone. Moreover, in stage II and III GC patients with a low PS_GC (but not in those with a high PS_GC), satisfactory chemotherapy benefits are observed. Therefore, the PS_GC could serve as a prognostic predictor in patients with GC and might be a potential predictive indicator for decision-making regarding adjuvant chemotherapy.

A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer

Lung cancer accounts for more deaths worldwide than any other cancer disease. In order to provide patients with the most effective treatment for these aggressive tumours, multimodal learning is emerging as a new and promising field of research that aims to extract complementary information from the data of different modalities for prognostic and predictive purposes. This knowledge could be used to optimise current treatments and maximise their effectiveness. To predict overall survival, in this work, we investigate the use of multimodal learning on the CLARO dataset, which includes CT images and clinical data collected from a cohort of non-small-cell lung cancer patients. Our method allows the identification of the optimal set of classifiers to be included in the ensemble in a late fusion approach. Specifically, after training unimodal models on each modality, it selects the best ensemble by solving a multiobjective optimisation problem that maximises both the recognition performance and the diversity of the predictions. In the ensemble, the labels of each sample are assigned using the majority voting rule. As further validation, we show that the proposed ensemble outperforms the models learning a single modality, obtaining state-of-the-art results on the task at hand.

The relationship between radiomics and pathomics in Glioblastoma patients: Preliminary results from a cross-scale association study

Glioblastoma multiforme (GBM) typically exhibits substantial intratumoral heterogeneity at both microscopic and radiological resolution scales. Diffusion Weighted Imaging (DWI) and dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) are two functional MRI techniques that are commonly employed in clinic for the assessment of GBM tumor characteristics. This work presents initial results aiming at determining if radiomics features extracted from preoperative ADC maps and post-contrast T1 (T1C) images are associated with pathomic features arising from H&E digitized pathology images. 48 patients from the public available CPTAC-GBM database, for which both radiology and pathology images were available, were involved in the study. 91 radiomics features were extracted from ADC maps and post-contrast T1 images using PyRadiomics. 65 pathomic features were extracted from cell detection measurements from H&E images. Moreover, 91 features were extracted from cell density maps of H&E images at four different resolutions. Radiopathomic associations were evaluated by means of Spearman's correlation (ρ) and factor analysis. p values were adjusted for multiple correlations by using a false discovery rate adjustment. Significant cross-scale associations were identified between pathomics and ADC, both considering features (n = 186, 0.45 < ρ < 0.74 in absolute value) and factors (n = 5, 0.48 < ρ < 0.54 in absolute value). Significant but fewer ρ values were found concerning the association between pathomics and radiomics features (n = 53, 0.5 < ρ < 0.65 in absolute value) and factors (n = 2, ρ = 0.63 and ρ = 0.53 in absolute value). The results of this study suggest that cross-scale associations may exist between digital pathology and ADC and T1C imaging. This can be useful not only to improve the knowledge concerning GBM intratumoral heterogeneity, but also to strengthen the role of radiomics approach and its validation in clinical practice as "virtual biopsy", introducing new insights for omics integration toward a personalized medicine approach.

Fusing hand-crafted and deep-learning features in a convolutional neural network model to identify prostate cancer in pathology images

Prostate cancer can be diagnosed by prostate biopsy using transectal ultrasound guidance. The high number of pathology images from biopsy tissues is a burden on pathologists, and analysis is subjective and susceptible to inter-rater variability. The use of machine learning techniques could make prostate histopathology diagnostics more precise, consistent, and efficient overall. This paper presents a new classification fusion network model that was created by fusing eight advanced image features: seven hand-crafted features and one deep-learning feature. These features are the scale-invariant feature transform (SIFT), speeded up robust feature (SURF), oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) (ORB) of local features, shape and texture features of the cell nuclei, the histogram of oriented gradients (HOG) feature of the cavities, a color feature, and a convolution deep-learning feature. Matching, integrated, and fusion networks are the three essential components of the proposed deep-learning network. The integrated network consists of both a backbone and an additional network. When classifying 1100 prostate pathology images using this fusion network with different backbones (ResNet-18/50, VGG-11/16, and DenseNet-121/201), we discovered that the proposed model with the ResNet-18 backbone achieved the best performance in terms of the accuracy (95.54%), specificity (93.64%), and sensitivity (97.27%) as well as the area under the receiver operating characteristic curve (98.34%). However, each of the assessment criteria for these separate features had a value lower than 90%, which demonstrates that the suggested model combines differently derived characteristics in an effective manner. Moreover, a Grad-CAM++ heatmap was used to observe the differences between the proposed model and ResNet-18 in terms of the regions of interest. This map showed that the proposed model was better at focusing on cancerous cells than ResNet-18. Hence, the proposed classification fusion network, which combines hand-crafted features and a deep-learning feature, is useful for computer-aided diagnoses based on pathology images of prostate cancer. Because of the similarities in the feature engineering and deep learning for different types of pathology images, the proposed method could be used for other pathology images, such as those of breast, thyroid cancer.

Advantages of manual and automatic computer-aided compared to traditional histopathological diagnosis of melanoma: A pilot study

BACKGROUND

Cutaneous malignant melanoma (CMM) accounts for the highest mortality rate among all skin cancers. Traditional histopathologic diagnosis may be limited by the pathologists' subjectivity. Second-opinion strategies and multidisciplinary consultations are usually performed to overcome this issue. An available solution in the future could be the use of automated solutions based on a computational algorithm that could help the pathologist in everyday practice. The aim of this pilot study was to investigate the potential diagnostic aid of a machine-based algorithm in the histopathologic diagnosis of CMM.

METHODS

We retrospectively examined excisional biopsies of 50 CMM and 20 benign congenital compound nevi. Hematoxylin and eosin (H&E) stained WSI were reviewed independently by two expert dermatopathologists. A fully automated pipeline for WSI processing to support the estimation and prioritization of the melanoma areas was developed.

RESULTS

The spatial distribution of the nuclei in the sample provided a multi-scale overview of the tumor. A global overview of the lesion's silhouette was achieved and, by increasing the magnification, the topological distribution of the nuclei and the most informative areas of interest for the CMM diagnosis were identified and highlighted. These silhouettes allow the histopathologist to discriminate between nevus and CMM with an accuracy of 96% without any extra information.

CONCLUSION

In this study we proposed an easy-to-use model that produces segmentations of CMM silhouettes at fine detail level.

A Cascaded Deep Learning Framework for Segmentation of Nuclei in Digital Histology Images

Accurate segmentation of nuclei is an essential step in analysis of digital histology images for diagnostic and prognostic applications. Despite recent advances in automated frameworks for nuclei segmentation, this task is still challenging. Specifically, detecting small nuclei in large-scale histology images and delineating the border of touching nuclei accurately is a complicated task even for advanced deep neural networks. In this study, a cascaded deep learning framework is proposed to segment nuclei accurately in digitized microscopy images of histology slides. A U-Net based model with customized pixel-wised weighted loss function is adapted in the proposed framework, followed by a U-Net based model with VGG16 backbone and a soft Dice loss function. The model was pretrained on the Post-NAT-BRCA public dataset before training and independent evaluation on the MoNuSeg dataset. The cascaded model could outperform the other state-of-the-art models with an AJI of 0.72 and a F1-score of 0.83 on the MoNuSeg test set.

The state of the art for artificial intelligence in lung digital pathology

Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Software tools and platforms in Digital Pathology: a review for clinicians and computer scientists

At the end of the twentieth century, a new technology was developed that allowed an entire tissue section to be scanned on an objective slide. Originally called virtual microscopy, this technology is now known as Whole Slide Imaging (WSI). WSI presents new challenges for reading, visualization, storage, and analysis. For this reason, several technologies have been developed to facilitate the handling of these images. In this paper, we analyze the most widely used technologies in the field of digital pathology, ranging from specialized libraries for the reading of these images to complete platforms that allow reading, visualization, and analysis. Our aim is to provide the reader, whether a pathologist or a computational scientist, with the knowledge to choose the technologies to use for new studies, development, or research.

Chronic Lymphocytic Leukemia Progression Diagnosis with Intrinsic Cellular Patterns via Unsupervised Clustering

Simple Summary

Distinguishing between chronic lymphocytic leukemia (CLL), accelerated CLL (aCLL), and full-blown transformation to diffuse large B-cell lymphoma (Richter transformation; RT) has significant clinical implications. Identifying cellular phenotypes via unsupervised clustering provides the most robust analytic performance in analyzing digitized pathology slides. This study serves as a proof of concept that using an unsupervised machine learning scheme can enhance diagnostic accuracy.

Abstract

Identifying the progression of chronic lymphocytic leukemia (CLL) to accelerated CLL (aCLL) or transformation to diffuse large B-cell lymphoma (Richter transformation; RT) has significant clinical implications as it prompts a major change in patient management. However, the differentiation between these disease phases may be challenging in routine practice. Unsupervised learning has gained increased attention because of its substantial potential in data intrinsic pattern discovery. Here, we demonstrate that cellular feature engineering, identifying cellular phenotypes via unsupervised clustering, provides the most robust analytic performance in analyzing digitized pathology slides (accuracy = 0.925, AUC = 0.978) when compared to alternative approaches, such as mixed features, supervised features, unsupervised/mixed/supervised feature fusion and selection, as well as patch-based convolutional neural network (CNN) feature extraction. We further validate the reproducibility and robustness of unsupervised feature extraction via stability and repeated splitting analysis, supporting its utility as a diagnostic aid in identifying CLL patients with histologic evidence of disease progression. The outcome of this study serves as proof of principle using an unsupervised machine learning scheme to enhance the diagnostic accuracy of the heterogeneous histology patterns that pathologists might not easily see.

MRI-Based Radiomic Features Help Identify Lesions and Predict Histopathological Grade of Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is the most common form of liver cancer. Radiomics is a promising tool that may increase the value of magnetic resonance imaging (MRI) in the management of HCC. The purpose of our study is to develop an MRI-based radiomics approach to preoperatively detect HCC and predict its histological grade. Thirty-eight HCC patients at staging who underwent axial T2-weighted and dynamic contrast-enhanced MRI (DCE-MRI) were considered. Three-dimensional volumes of interest (VOIs) were manually placed on HCC lesions and normal hepatic tissue (HT) on arterial phase post-contrast images. Radiomic features from T2 images and arterial, portal and tardive post-contrast images from DCE-MRI were extracted by using Pyradiomics. Feature selection was performed using correlation filter, Wilcoxon-rank sum test and mutual information. Predictive models were constructed for HCC differentiation with respect to HT and HCC histopathologic grading used at each step an imbalance-adjusted bootstrap resampling (IABR) on 1000 samples. Promising results were obtained from radiomic prediction models, with best AUCs ranging from 71% to 96%. Radiomics MRI based on T2 and DCE-MRI revealed promising results concerning both HCC detection and grading. It may be a suitable tool for personalized treatment of HCC patients and could also be used to develop new prognostic biomarkers useful for HCC assessment without the need for invasive procedures.

Unsupervised Segmentation in NSCLC: How to Map the Output of Unsupervised Segmentation to Meaningful Histological Labels by Linear Combination?

Background: Segmentation is, in many Pathomics projects, an initial step. Usually, in supervised settings, well-annotated and large datasets are required. Regarding the rarity of such datasets, unsupervised learning concepts appear to be a potential solution. Against this background, we tested for a small dataset on lung cancer tissue microarrays (TMA) if a model (i) first can be in a previously published unsupervised setting and (ii) secondly can be modified and retrained to produce meaningful labels, and (iii) we finally compared this approach to standard segmentation models. Methods: (ad i) First, a convolutional neuronal network (CNN) segmentation model is trained in an unsupervised fashion, as recently described by Kanezaki et al. (ad ii) Second, the model is modified by adding a remapping block and is retrained on an annotated dataset in a supervised setting. (ad iii) Third, the segmentation results are compared to standard segmentation models trained on the same dataset. Results: (ad i–ii) By adding an additional mapping-block layer and by retraining, models previously trained in an unsupervised manner can produce meaningful labels. (ad iii) The segmentation quality is inferior to standard segmentation models trained on the same dataset. Conclusions: Unsupervised training in combination with subsequent supervised training offers for histological images here no benefit.

Imaging-based Machine-learning Models to Predict Clinical Outcomes and Identify Biomarkers in Pancreatic Cancer: A Scoping Review

OBJECTIVE

To perform a scoping review of imaging-based machine-learning models to predict clinical outcomes and identify biomarkers in patients with PDAC.

SUMMARY OF BACKGROUND DATA

Patients with PDAC could benefit from better selection for systemic and surgical therapy. Imaging-based machine-learning models may improve treatment selection.

METHODS

A scoping review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses-scoping review guidelines in the PubMed and Embase databases (inception-October 2020). The review protocol was prospectively registered (open science framework registration: m4cyx). Included were studies on imaging-based machine-learning models for predicting clinical outcomes and identifying biomarkers for PDAC. The primary outcome was model performance. An area under the curve (AUC) of ≥0.75, or a P-value of ≤0.05, was considered adequate model performance. Methodological study quality was assessed using the modified radiomics quality score.

RESULTS

After screening 1619 studies, 25 studies with 2305 patients fulfilled the eligibility criteria. All but 1 study was published in 2019 and 2020. Overall, 23/25 studies created models using radiomics features, 1 study quantified vascular invasion on computed tomography, and one used histopathological data. Nine models predicted clinical outcomes with AUC measures of 0.78-0.95, and C-indices of 0.65-0.76. Seventeen models identified biomarkers with AUC measures of 0.68-0.95. Adequate model performance was reported in 23/25 studies. The methodological quality of the included studies was suboptimal, with a median modified radiomics quality score score of 7/36.

CONCLUSIONS

The use of imaging-based machine-learning models to predict clinical outcomes and identify biomarkers in patients with PDAC is increasingly rapidly. Although these models mostly have good performance scores, their methodological quality should be improved.

Selecting Candidates for Organ-Preserving Strategies After Neoadjuvant Chemoradiotherapy for Rectal Cancer: Development and Validation of a Model Integrating MRI Radiomics and Pathomics

BACKGROUND

Histopathologic evaluation after surgery is the gold standard to evaluate treatment response to neoadjuvant chemoradiotherapy (nCRT) in locally advanced rectal cancer (LARC). However, it cannot be used to guide organ-preserving strategies due to poor timeliness.

PURPOSE

To develop and validate a multiscale model incorporating radiomics and pathomics features for predicting pathological good response (pGR) of down-staging to stage ypT0-1N0 after nCRT.

STUDY TYPE

Retrospective.

POPULATION

A total of 153 patients (median age, 55 years; 109 men; 107 training group; 46 validation group) with clinicopathologically confirmed LARC.

FIELD STRENGTH/SEQUENCE

A 3.0-T; fast spin echo T₂ -weighted and single-shot EPI diffusion-weighted images.

ASSESSMENT

The differences in clinicoradiological variables between pGR and non-pGR groups were assessed. Pretreatment and posttreatment radiomics signatures, and pathomics signature were constructed. A multiscale pGR prediction model was established. The predictive performance of the model was evaluated and compared to that of the clinicoradiological model.

STATISTICAL TESTS

The χ² test, Fisher's exact test, t-test, the minimum redundancy maximum relevance algorithm, the least absolute shrinkage and selection operator logistic regression algorithm, regression analysis, receiver operating characteristic curve (ROC) analysis, Delong method. P < 0.05 indicated a significant difference.

RESULTS

Pretreatment radiomics signature (odds ratio [OR] = 2.53; 95% CI: 1.58-4.66), posttreatment radiomics signature (OR = 9.59; 95% CI: 3.04-41.46), and pathomics signature (OR = 3.14; 95% CI: 1.40-8.31) were independent factors for predicting pGR. The multiscale model presented good predictive performance with areas under the curve (AUC) of 0.93 (95% CI: 0.88-0.98) and 0.90 (95% CI: 0.78-1.00) in the training and validation groups, those were significantly higher than that of the clinicoradiological model with AUCs of 0.69 (95% CI: 0.55-0.82) and 0.68 (95% CI: 0.46-0.91) in both groups.

DATA CONCLUSION

A model incorporating radiomics and pathomics features effectively predicted pGR after nCRT in patients with LARC.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 4.

Machine Learning: A New Prospect in Multi-Omics Data Analysis of Cancer

Cancer is defined as a large group of diseases that is associated with abnormal cell growth, uncontrollable cell division, and may tend to impinge on other tissues of the body by different mechanisms through metastasis. What makes cancer so important is that the cancer incidence rate is growing worldwide which can have major health, economic, and even social impacts on both patients and the governments. Thereby, the early cancer prognosis, diagnosis, and treatment can play a crucial role at the front line of combating cancer. The onset and progression of cancer can occur under the influence of complicated mechanisms and some alterations in the level of genome, proteome, transcriptome, metabolome etc. Consequently, the advent of omics science and its broad research branches (such as genomics, proteomics, transcriptomics, metabolomics, and so forth) as revolutionary biological approaches have opened new doors to the comprehensive perception of the cancer landscape. Due to the complexities of the formation and development of cancer, the study of mechanisms underlying cancer has gone beyond just one field of the omics arena. Therefore, making a connection between the resultant data from different branches of omics science and examining them in a multi-omics field can pave the way for facilitating the discovery of novel prognostic, diagnostic, and therapeutic approaches. As the volume and complexity of data from the omics studies in cancer are increasing dramatically, the use of leading-edge technologies such as machine learning can have a promising role in the assessments of cancer research resultant data. Machine learning is categorized as a subset of artificial intelligence which aims to data parsing, classification, and data pattern identification by applying statistical methods and algorithms. This acquired knowledge subsequently allows computers to learn and improve accurate predictions through experiences from data processing. In this context, the application of machine learning, as a novel computational technology offers new opportunities for achieving in-depth knowledge of cancer by analysis of resultant data from multi-omics studies. Therefore, it can be concluded that the use of artificial intelligence technologies such as machine learning can have revolutionary roles in the fight against cancer.

Artificial intelligence in cancer diagnostics and therapy: current perspectives

Artificial intelligence (AI) has found its way into every sphere of human life including the field of medicine. Detection of cancer might be AI's most altruistic and convoluted challenge to date in the field of medicine. Embedding AI into various aspects of cancer diagnostics would be of immense use in dealing with the tedious, repetitive, time-consuming job of lesion detection, remove opportunities for human error, and cut costs and time. This would be of great value in cancer screening programs. By using AI algorithms, data from digital images from radiology and pathology that are imperceptible to the human eye can be identified (radiomics and pathomics). Correlating radiomics and pathomics with clinico-demographic-therapy-morbidity-mortality profiles will lead to a greater understanding of cancers. Specific imaging phenotypes have been found to be associated with specific gene-determined molecular pathways involved in cancer pathogenesis (radiogenomics). All these developments would not only help to personalize oncologic practice but also lead to the development of new imaging biomarkers. AI algorithms in oncoimaging and oncopathology will broadly have the following uses: cancer screening (detection of lesions), characterization and grading of tumors, and clinical decision-making and prognostication. However, AI cannot be a foolproof panacea nor can it supplant the role of humans. It can however be a powerful and useful complement to human insights and deeper understanding. Multiple issues like standardization, validity, ethics, privacy, finances, legal liability, training, accreditation, etc., need to be overcome before the vast potential of AI in diagnostic oncology can be fully harnessed.