Computer extracted gland features from H&E predicts prostate cancer recurrence comparably to a genomic companion diagnostic test: a large multi-site study

An integrated radiology-pathology machine learning classifier for outcome prediction following radical prostatectomy: Preliminary findings

Objectives

To evaluate the added benefit of integrating features from pre-treatment MRI (radiomics) and digitized post-surgical pathology slides (pathomics) in prostate cancer (PCa) patients for prognosticating outcomes post radical-prostatectomy (RP) including a) rising prostate specific antigen (PSA), and b) extraprostatic-extension (EPE).

Methods

Multi-institutional data (N = 58) of PCa patients who underwent pre-treatment 3-T MRI prior to RP were included in this retrospective study. Radiomic and pathomic features were extracted from PCa regions on MRI and RP specimens delineated by expert clinicians. On training set (D1, N = 44), Cox Proportional-Hazards models MR, MP and MRaP were trained using radiomics, pathomics, and their combination, respectively, to prognosticate rising PSA (PSA > 0.03 ng/mL). Top features from MRaP were used to train a model to predict EPE on D1 and test on external dataset (D2, N = 14). C-index, Kalplan-Meier curves were used for survival analysis, and area under ROC (AUC) was used for EPE. MRaP was compared with the existing post-treatment risk-calculator, CAPRA (MC).

Results

Patients had median follow-up of 34 months. MRaP (c-index = 0.685 ± 0.05) significantly outperformed MR (c-index = 0.646 ± 0.05), MP (c-index = 0.631 ± 0.06) and MC (c-index = 0.601 ± 0.071) (p < 0.0001). Cross-validated Kaplan-Meier curves showed significant separation among risk groups for rising PSA for MRaP (p < 0.005, Hazard Ratio (HR) = 11.36) as compared to MR (p = 0.64, HR = 1.33), MP (p = 0.19, HR = 2.82) and MC (p = 0.10, HR = 3.05). Integrated radio-pathomic model MRaP (AUC = 0.80) outperformed MR (AUC = 0.57) and MP (AUC = 0.76) in predicting EPE on external-data (D2).

Conclusions

Results from this preliminary study suggest that a combination of radiomic and pathomic features can better predict post-surgical outcomes (rising PSA and EPE) compared to either of them individually as well as extant prognostic nomogram (CAPRA).

Demographic bias in misdiagnosis by computational pathology models

Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.

External Validation of a Digital Pathology-based Multimodal Artificial Intelligence Architecture in the NRG/RTOG 9902 Phase 3 Trial

BACKGROUND

Accurate risk stratification is critical to guide management decisions in localized prostate cancer (PCa). Previously, we had developed and validated a multimodal artificial intelligence (MMAI) model generated from digital histopathology and clinical features. Here, we externally validate this model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial.

OBJECTIVE

To externally validate the MMAI model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial.

DESIGN, SETTING, AND PARTICIPANTS

Our validation cohort included 318 localized high-risk PCa patients from NRG/RTOG 9902 with available histopathology (337 [85%] of the 397 patients enrolled into the trial had available slides, of which 19 [5.6%] failed due to poor image quality).

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

Two previously locked prognostic MMAI models were validated for their intended endpoint: distant metastasis (DM) and PCa-specific mortality (PCSM). Individual clinical factors and the number of National Comprehensive Cancer Network (NCCN) high-risk features served as comparators. Subdistribution hazard ratio (sHR) was reported per standard deviation increase of the score with corresponding 95% confidence interval (CI) using Fine-Gray or Cox proportional hazards models.

RESULTS AND LIMITATIONS

The DM and PCSM MMAI algorithms were significantly and independently associated with the risk of DM (sHR [95% CI] = 2.33 [1.60-3.38], p < 0.001) and PCSM, respectively (sHR [95% CI] = 3.54 [2.38-5.28], p < 0.001) when compared against other prognostic clinical factors and NCCN high-risk features. The lower 75% of patients by DM MMAI had estimated 5- and 10-yr DM rates of 4% and 7%, and the highest quartile had average 5- and 10-yr DM rates of 19% and 32%, respectively (p < 0.001). Similar results were observed for the PCSM MMAI algorithm.

CONCLUSIONS

We externally validated the prognostic ability of MMAI models previously developed among men with localized high-risk disease. MMAI prognostic models further risk stratify beyond the clinical and pathological variables for DM and PCSM in a population of men already at a high risk for disease progression. This study provides evidence for consistent validation of our deep learning MMAI models to improve prognostication and enable more informed decision-making for patient care.

PATIENT SUMMARY

This paper presents a novel approach using images from pathology slides along with clinical variables to validate artificial intelligence (computer-generated) prognostic models. When implemented, clinicians can offer a more personalized and tailored prognostic discussion for men with localized prostate cancer.

Quantitative Histomorphometric Features of Prostate Cancer Predict Patients Who Biochemically Recur Following Prostatectomy

Prostate cancer is the most commonly diagnosed cancer in men, accounting for 27% of the new male cancer diagnoses in 2022. If organ-confined, removal of the prostate through radical prostatectomy is considered curative; however, distant metastases may occur, resulting in a poor patient prognosis. This study sought to determine whether quantitative pathomic features of prostate cancer differ in patients who biochemically experience biological recurrence after surgery. Whole-mount prostate histology from 78 patients was analyzed for this study. In total, 614 slides were hematoxylin and eosin stained and digitized to produce whole slide images (WSI). Regions of differing Gleason patterns were digitally annotated by a genitourinary fellowship-trained pathologist, and high-resolution tiles were extracted from each annotated region of interest for further analysis. Individual glands within the prostate were identified using automated image processing algorithms, and histomorphometric features were calculated on a per-tile basis and across WSI and averaged by patients. Tiles were organized into cancer and benign tissues. Logistic regression models were fit to assess the predictive value of the calculated pathomic features across tile groups and WSI; additionally, models using clinical information were used for comparisons. Logistic regression classified each pathomic feature model at accuracies >80% with areas under the curve of 0.82, 0.76, 0.75, and 0.72 for all tiles, cancer only, noncancer only, and across WSI. This was comparable with standard clinical information, Gleason Grade Groups, and CAPRA score, which achieved similar accuracies but areas under the curve of 0.80, 0.77, and 0.70, respectively. This study demonstrates that the use of quantitative pathomic features calculated from digital histology of prostate cancer may provide clinicians with additional information beyond the traditional qualitative pathologist assessment. Further research is warranted to determine possible inclusion in treatment guidance.

Nondestructive 3D pathology with analysis of nuclear features for prostate cancer risk assessment

Prostate cancer treatment decisions rely heavily on subjective visual interpretation [assigning Gleason patterns or International Society of Urological Pathology (ISUP) grade groups] of limited numbers of two-dimensional (2D) histology sections. Under this paradigm, interobserver variance is high, with ISUP grades not correlating well with outcome for individual patients, and this contributes to the over- and undertreatment of patients. Recent studies have demonstrated improved prognostication of prostate cancer outcomes based on computational analyses of glands and nuclei within 2D whole slide images. Our group has also shown that the computational analysis of three-dimensional (3D) glandular features, extracted from 3D pathology datasets of whole intact biopsies, can allow for improved recurrence prediction compared to corresponding 2D features. Here we seek to expand on these prior studies by exploring the prognostic value of 3D shape-based nuclear features in prostate cancer (e.g. nuclear size, sphericity). 3D pathology datasets were generated using open-top light-sheet (OTLS) microscopy of 102 cancer-containing biopsies extracted ex vivo from the prostatectomy specimens of 46 patients. A deep learning-based workflow was developed for 3D nuclear segmentation within the glandular epithelium versus stromal regions of the biopsies. 3D shape-based nuclear features were extracted, and a nested cross-validation scheme was used to train a supervised machine classifier based on 5-year biochemical recurrence (BCR) outcomes. Nuclear features of the glandular epithelium were found to be more prognostic than stromal cell nuclear features (area under the ROC curve [AUC] = 0.72 versus 0.63). 3D shape-based nuclear features of the glandular epithelium were also more strongly associated with the risk of BCR than analogous 2D features (AUC = 0.72 versus 0.62). The results of this preliminary investigation suggest that 3D shape-based nuclear features are associated with prostate cancer aggressiveness and could be of value for the development of decision-support tools. © 2023 The Pathological Society of Great Britain and Ireland.

Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology

Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.

Multi-candidate immunohistochemical markers to assess radiation response and prognosis in prostate cancer: results from the CHHiP trial of radiotherapy fractionation

BACKGROUND

Protein markers of cellular proliferation, hypoxia, apoptosis, cell cycle checkpoints, growth factor signalling and inflammation in localised prostate tumours have previously shown prognostic ability. A translational substudy within the CHHiP trial of radiotherapy fractionation evaluated whether these could improve prediction of prognosis and assist treatment stratification following either conventional or hypofractionated radiotherapy.

METHODS

Using case:control methodology, patients with biochemical or clinical failure after radiotherapy (BCR) were matched to patients without recurrence according to established prognostic factors (Gleason score, presenting PSA, tumour-stage) and fractionation schedule. Immunohistochemical (IHC) staining of diagnostic biopsy sections was performed and scored for HIF1α, Bcl-2, Ki67, Geminin, p16, p53, p-chk1 and PTEN. Univariable and multivariable conditional logistic regression models, adjusted for matching strata and age, estimated the prognostic value of each IHC biomarker, including interaction terms to determine BCR prediction according to fractionation.

FINDINGS

IHC results were available for up to 336 tumours. PTEN, Geminin, mean Ki67 and max Ki67 were prognostic after adjusting for multiple comparisons and were fitted in a multivariable model (n = 212, 106 matched pairs). Here, PTEN and Geminin showed significant prediction of prognosis. No marker predicted BCR according to fractionation.

INTERPRETATION

Geminin or Ki67, and PTEN, predicted response to radiotherapy independently of established prognostic factors. These results provide essential independent external validation of previous findings and confirm a role for these markers in treatment stratification.

FUNDING

Cancer Research UK (BIDD) grant (A12518), Cancer Research UK (C8262/A7253), Department of Health, Prostate Cancer UK, Movember Foundation, NIHR Biomedical Research Centre at Royal Marsden/ICR.

Computational pathology in ovarian cancer

Histopathologic evaluations of tissue sections are key to diagnosing and managing ovarian cancer. Pathologists empirically assess and integrate visual information, such as cellular density, nuclear atypia, mitotic figures, architectural growth patterns, and higher-order patterns, to determine the tumor type and grade, which guides oncologists in selecting appropriate treatment options. Latent data embedded in pathology slides can be extracted using computational imaging. Computers can analyze digital slide images to simultaneously quantify thousands of features, some of which are visible with a manual microscope, such as nuclear size and shape, while others, such as entropy, eccentricity, and fractal dimensions, are quantitatively beyond the grasp of the human mind. Applications of artificial intelligence and machine learning tools to interpret digital image data provide new opportunities to explore and quantify the spatial organization of tissues, cells, and subcellular structures. In comparison to genomic, epigenomic, transcriptomic, and proteomic patterns, morphologic and spatial patterns are expected to be more informative as quantitative biomarkers of complex and dynamic tumor biology. As computational pathology is not limited to visual data, nuanced subvisual alterations that occur in the seemingly “normal” pre-cancer microenvironment could facilitate research in early cancer detection and prevention. Currently, efforts to maximize the utility of computational pathology are focused on integrating image data with other -omics platforms that lack spatial information, thereby providing a new way to relate the molecular, spatial, and microenvironmental characteristics of cancer. Despite a dire need for improvements in ovarian cancer prevention, early detection, and treatment, the ovarian cancer field has lagged behind other cancers in the application of computational pathology. The intent of this review is to encourage ovarian cancer research teams to apply existing and/or develop additional tools in computational pathology for ovarian cancer and actively contribute to advancing this important field.

Multiplex Immunofluorescence and the Digital Image Analysis Workflow for Evaluation of the Tumor Immune Environment in Translational Research

A robust understanding of the tumor immune environment has important implications for cancer diagnosis, prognosis, research, and immunotherapy. Traditionally, immunohistochemistry (IHC) has been regarded as the standard method for detecting proteins in situ, but this technique allows for the evaluation of only one cell marker per tissue sample at a time. However, multiplexed imaging technologies enable the multiparametric analysis of a tissue section at the same time. Also, through the curation of specific antibody panels, these technologies enable researchers to study the cell subpopulations within a single immunological cell group. Thus, multiplexed imaging gives investigators the opportunity to better understand tumor cells, immune cells, and the interactions between them. In the multiplexed imaging technology workflow, once the protocol for a tumor immune micro environment study has been defined, histological slides are digitized to produce high-resolution images in which regions of interest are selected for the interrogation of simultaneously expressed immunomarkers (including those co-expressed by the same cell) by using an image analysis software and algorithm. Most currently available image analysis software packages use similar machine learning approaches in which tissue segmentation first defines the different components that make up the regions of interest and cell segmentation, then defines the different parameters, such as the nucleus and cytoplasm, that the software must utilize to segment single cells. Image analysis tools have driven dramatic evolution in the field of digital pathology over the past several decades and provided the data necessary for translational research and the discovery of new therapeutic targets. The next step in the growth of digital pathology is optimization and standardization of the different tasks in cancer research, including image analysis algorithm creation, to increase the amount of data generated and their accuracy in a short time as described herein. The aim of this review is to describe this process, including an image analysis algorithm creation for multiplex immunofluorescence analysis, as an essential part of the optimization and standardization of the different processes in cancer research, to increase the amount of data generated and their accuracy in a short time.

Predicting biochemical recurrence of prostate cancer with artificial intelligence

Background

The first sign of metastatic prostate cancer after radical prostatectomy is rising PSA levels in the blood, termed biochemical recurrence. The prediction of recurrence relies mainly on the morphological assessment of prostate cancer using the Gleason grading system. However, in this system, within-grade morphological patterns and subtle histopathological features are currently omitted, leaving a significant amount of prognostic potential unexplored.

Methods

To discover additional prognostic information using artificial intelligence, we trained a deep learning system to predict biochemical recurrence from tissue in H&E-stained microarray cores directly. We developed a morphological biomarker using convolutional neural networks leveraging a nested case-control study of 685 patients and validated on an independent cohort of 204 patients. We use concept-based explainability methods to interpret the learned tissue patterns.

Results

The biomarker provides a strong correlation with biochemical recurrence in two sets (n = 182 and n = 204) from separate institutions. Concept-based explanations provided tissue patterns interpretable by pathologists.

Conclusions

These results show that the model finds predictive power in the tissue beyond the morphological ISUP grading.

To determine the prognosis of patients with prostate cancer, several clinical factors are taken into account. One of these is the cancer grade, assigned by a pathologist based on the cancer’s appearance under a microscope. The grade ranges from 1 to 5, where 5 is the most aggressive tumour type. This study explored whether deep learning—a technique in which computer software learns patterns from multiple examples—can learn to predict the risk of patients’ cancers recurring from microscopic images of the tumours. We show, on two clinical datasets from different institutions, that such a system can help to better predict prognosis, beyond the information provided by grade alone. In the future, this type of method could help clinicians to predict the prognosis of individual prostate cancer patients.

Pinckaers et al. develop a deep learning system to predict biochemical recurrence in prostate cancer patients treated with radical prostatectomy. The authors’ morphological biomarker provides predictive power beyond traditional Gleason grading, based on analysis of two clinical datasets from different institutions.

Prostate Cancer Risk Stratification via Nondestructive 3D Pathology with Deep Learning-Assisted Gland Analysis

Prostate cancer treatment planning is largely dependent upon examination of core-needle biopsies. The microscopic architecture of the prostate glands forms the basis for prognostic grading by pathologists. Interpretation of these convoluted three-dimensional (3D) glandular structures via visual inspection of a limited number of two-dimensional (2D) histology sections is often unreliable, which contributes to the under- and overtreatment of patients. To improve risk assessment and treatment decisions, we have developed a workflow for nondestructive 3D pathology and computational analysis of whole prostate biopsies labeled with a rapid and inexpensive fluorescent analogue of standard hematoxylin and eosin (H&E) staining. This analysis is based on interpretable glandular features and is facilitated by the development of image translation-assisted segmentation in 3D (ITAS3D). ITAS3D is a generalizable deep learning-based strategy that enables tissue microstructures to be volumetrically segmented in an annotation-free and objective (biomarker-based) manner without requiring immunolabeling. As a preliminary demonstration of the translational value of a computational 3D versus a computational 2D pathology approach, we imaged 300 ex vivo biopsies extracted from 50 archived radical prostatectomy specimens, of which, 118 biopsies contained cancer. The 3D glandular features in cancer biopsies were superior to corresponding 2D features for risk stratification of patients with low- to intermediate-risk prostate cancer based on their clinical biochemical recurrence outcomes. The results of this study support the use of computational 3D pathology for guiding the clinical management of prostate cancer. SIGNIFICANCE: An end-to-end pipeline for deep learning-assisted computational 3D histology analysis of whole prostate biopsies shows that nondestructive 3D pathology has the potential to enable superior prognostic stratification of patients with prostate cancer.

Computationally Derived Cribriform Area Index from Prostate Cancer Hematoxylin and Eosin Images Is Associated with Biochemical Recurrence Following Radical Prostatectomy and Is Most Prognostic in Gleason Grade Group 2

BACKGROUND

The presence of invasive cribriform adenocarcinoma (ICC), an expanse of cells containing punched-out lumina uninterrupted by stroma, in radical prostatectomy (RP) specimens has been associated with biochemical recurrence (BCR). However, ICC identification has only moderate inter-reviewer agreement.

OBJECTIVE

To investigate quantitative machine-based assessment of the extent and prognostic utility of ICC, especially within individual Gleason grade groups.

DESIGN, SETTING, AND PARTICIPANTS

A machine learning approach was developed for ICC segmentation using 70 RP patients and validated in a cohort of 749 patients from four sites whose median year of surgery was 2007 and with median follow-up of 28 mo. ICC was segmented on one representative hematoxylin and eosin RP slide per patient and the fraction of tumor area composed of ICC, the cribriform area index (CAI), was measured.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

The association between CAI and BCR was measured in terms of the concordance index (c index) and hazard ratio (HR).

RESULTS AND LIMITATIONS

CAI was correlated with BCR (c index 0.62) in the validation set of 411 patients with ICC morphology, especially those with Gleason grade group 2 cancer (n = 192; c index 0.66), and was less prognostic when patients without ICC were included (c index 0.54). A doubling of CAI in the group with ICC morphology was prognostic after controlling for Gleason grade, surgical margin positivity, preoperative prostate-specific antigen level, pathological T stage, and age (HR 1.19, 95% confidence interval 1.03-1.38; p = 0.018).

CONCLUSIONS

Automated image analysis and machine learning could provide an objective, quantitative, reproducible, and high-throughput method of quantifying ICC area. The performance of CAI for grade group 2 cancer suggests that for patients with little Gleason 4 pattern, the ICC fraction has a strong prognostic role.

PATIENT SUMMARY

Machine-based measurement of a specific cell pattern (cribriform; sieve-like, with lots of spaces) in images of prostate specimens could improve risk stratification for patients with prostate cancer. In the future, this could help in expanding the criteria for active surveillance.