Histostitcher™: An informatics software platform for reconstructing whole-mount prostate histology using the extensible imaging platform framework

An integrated radiology-pathology machine learning classifier for outcome prediction following radical prostatectomy: Preliminary findings

Objectives

To evaluate the added benefit of integrating features from pre-treatment MRI (radiomics) and digitized post-surgical pathology slides (pathomics) in prostate cancer (PCa) patients for prognosticating outcomes post radical-prostatectomy (RP) including a) rising prostate specific antigen (PSA), and b) extraprostatic-extension (EPE).

Methods

Multi-institutional data (N = 58) of PCa patients who underwent pre-treatment 3-T MRI prior to RP were included in this retrospective study. Radiomic and pathomic features were extracted from PCa regions on MRI and RP specimens delineated by expert clinicians. On training set (D1, N = 44), Cox Proportional-Hazards models MR, MP and MRaP were trained using radiomics, pathomics, and their combination, respectively, to prognosticate rising PSA (PSA > 0.03 ng/mL). Top features from MRaP were used to train a model to predict EPE on D1 and test on external dataset (D2, N = 14). C-index, Kalplan-Meier curves were used for survival analysis, and area under ROC (AUC) was used for EPE. MRaP was compared with the existing post-treatment risk-calculator, CAPRA (MC).

Results

Patients had median follow-up of 34 months. MRaP (c-index = 0.685 ± 0.05) significantly outperformed MR (c-index = 0.646 ± 0.05), MP (c-index = 0.631 ± 0.06) and MC (c-index = 0.601 ± 0.071) (p < 0.0001). Cross-validated Kaplan-Meier curves showed significant separation among risk groups for rising PSA for MRaP (p < 0.005, Hazard Ratio (HR) = 11.36) as compared to MR (p = 0.64, HR = 1.33), MP (p = 0.19, HR = 2.82) and MC (p = 0.10, HR = 3.05). Integrated radio-pathomic model MRaP (AUC = 0.80) outperformed MR (AUC = 0.57) and MP (AUC = 0.76) in predicting EPE on external-data (D2).

Conclusions

Results from this preliminary study suggest that a combination of radiomic and pathomic features can better predict post-surgical outcomes (rising PSA and EPE) compared to either of them individually as well as extant prognostic nomogram (CAPRA).

Reconstructing virtual large slides can improve the accuracy and consistency of tumor bed evaluation for breast cancer after neoadjuvant therapy

BACKGROUND

To explore whether the "WSI Stitcher", a program we developed for reconstructing virtual large slide through whole slide imaging fragments stitching, can improve the efficiency and consistency of pathologists in evaluating the tumor bed after neoadjuvant treatment of breast cancer compared with the conventional methods through stack splicing of physical slides.

METHODS

This study analyzed the advantages of using software-assisted methods to evaluate the tumor bed after neoadjuvant treatment of breast cancer. This new method is to use "WSI Stitcher" to stitch all the WSI fragments together to reconstruct a virtual large slide and evaluate the tumor bed with the help of the built-in ruler and tumor proportion calculation functions.

RESULTS

Compared with the conventional method, the evaluation time of the software-assisted method was shortened by 35%(P < 0.001). In the process of tumor bed assessment after neoadjuvant treatment of breast cancer, the software-assisted method has higher intraclass correlation coefficient when measuring the length (0.994 versus 0.934), width (0.992 versus 0.927), percentage of residual tumor cells (0.947 versus 0.878), percentage of carcinoma in situ (0.983 versus 0.881) and RCB index(0.997 versus 0.772). The software-assisted method has higher kappa values when evaluating tumor staging(0.901 versus 0.687) and RCB grading (0.963 versus 0.857).

CONCLUSION

The "WSI Stitcher" is an effective tool to help pathologists with the assessment of breast cancer after neoadjuvant treatment.

ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate

Magnetic resonance imaging (MRI) is an increasingly important tool for the diagnosis and treatment of prostate cancer. However, interpretation of MRI suffers from high inter-observer variability across radiologists, thereby contributing to missed clinically significant cancers, overdiagnosed low-risk cancers, and frequent false positives. Interpretation of MRI could be greatly improved by providing radiologists with an answer key that clearly shows cancer locations on MRI. Registration of histopathology images from patients who had radical prostatectomy to pre-operative MRI allows such mapping of ground truth cancer labels onto MRI. However, traditional MRI-histopathology registration approaches are computationally expensive and require careful choices of the cost function and registration hyperparameters. This paper presents ProsRegNet, a deep learning-based pipeline to accelerate and simplify MRI-histopathology image registration in prostate cancer. Our pipeline consists of image preprocessing, estimation of affine and deformable transformations by deep neural networks, and mapping cancer labels from histopathology images onto MRI using estimated transformations. We trained our neural network using MR and histopathology images of 99 patients from our internal cohort (Cohort 1) and evaluated its performance using 53 patients from three different cohorts (an additional 12 from Cohort 1 and 41 from two public cohorts). Results show that our deep learning pipeline has achieved more accurate registration results and is at least 20 times faster than a state-of-the-art registration algorithm. This important advance will provide radiologists with highly accurate prostate MRI answer keys, thereby facilitating improvements in the detection of prostate cancer on MRI. Our code is freely available at https://github.com/pimed//ProsRegNet.

Whole Slide Imaging: Technology and Applications

Pathology has benefited from advanced innovation with novel technology to implement a digital solution. Whole slide imaging is a disruptive technology where glass slides are scanned to produce digital images. There have been significant advances in whole slide scanning hardware and software that have allowed for ready access of whole slide images. The digital images, or whole slide images, can be viewed comparable to glass slides in a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Worldwide usage of whole slide imaging has grown significantly. Pathology regulatory organizations (ie, College of American Pathologists) have put forth guidelines for clinical validation, and the US Food and Drug Administration have also approved whole slide imaging for primary diagnosis. This article will review the digital pathology ecosystem and discuss clinical and nonclinical applications of its use.

Statistical Analysis of Survival Models Using Feature Quantification on Prostate Cancer Histopathological Images

Background

Grading of prostatic adenocarcinoma is based on the Gleason scoring system and the more recently established prognostic grade groups. Typically, prostate cancer grading is performed by pathologists based on the morphology of the tumor on hematoxylin and eosin (H and E) slides. In this study, we investigated the histopathological image features with various survival models and attempted to study their correlations.

Methods

Three texture methods (speeded-up robust features, histogram of oriented gradient, and local binary pattern) and two convolutional neural network (CNN)-based methods were applied to quantify histopathological image features. Five survival models were assessed on those image features in the context with other prostate clinical prognostic factors, including primary and secondary Gleason patterns, prostate-specific antigen levels, age, and clinical tumor stages.

Results

Based on statistical comparisons among different image features with survival models, image features from CNN-based method with a recurrent neural network called CNN-long-short-term memory provided the highest hazard ratio of prostate cancer recurrence under Cox regression with an elastic net penalty.

Conclusions

This approach outperformed the other image quantification methods listed above. Using this approach, patient outcomes were highly correlated with the histopathological image features of the tissue samples. In future studies, we plan to investigate the potential use of this approach for predicting recurrence in a wider range of cancer types.

Biomass Pretreatment and Enzymatic Hydrolysis Dynamics Analysis Based on Particle Size Imaging

Parameters such as pretreatment method, enzyme type and concentration, determine the conversion efficiency of biomass' cellulose and hemicellulose to glucose and mainly xylose in biomass-based fuel production. Chemical quantification of these processes offers no information on the effect of enzymatic hydrolysis (EH) on particle morphology. We report on the development of a microscopy method for imaging pretreated biomass particles at different EH stages. The method was based on acquiring large field of view images, typically 20×10 mm2 containing thousands of particles. Morphology of particles with lengths between 2 μm and 5 mm could be visualized and analyzed. The particle length distribution of corn stover samples, pretreated with increasing amounts of sulfuric acid at different EH stages, was measured. Particle size was shown to be dependent on pretreatment severity and EH time. The methodology developed could offer an alternative method for characterization of EH of biomass for second generation biofuels and visualization of recalcitrant structures.

Recurrence analysis on prostate cancer patients with Gleason score 7 using integrated histopathology whole-slide images and genomic data through deep neural networks

Prostate cancer is the most common nonskin-related cancer, affecting one in seven men in the United States. Gleason score, a sum of the primary and secondary Gleason patterns, is one of the best predictors of prostate cancer outcomes. Recently, significant progress has been made in molecular subtyping prostate cancer through the use of genomic sequencing. It has been established that prostate cancer patients presented with a Gleason score 7 show heterogeneity in both disease recurrence and survival. We built a unified system using publicly available whole-slide images and genomic data of histopathology specimens through deep neural networks to identify a set of computational biomarkers. Using a survival model, the experimental results on the public prostate dataset showed that the computational biomarkers extracted by our approach had hazard ratio as 5.73 and C -index as 0.74, which were higher than standard clinical prognostic factors and other engineered image texture features. Collectively, the results of this study highlight the important role of neural network analysis of prostate cancer and the potential of such approaches in other precision medicine applications.

Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: Preliminary findings

Translation of radiomics into the clinic may require a more comprehensive understanding of the underlying morphologic tissue characteristics they reflect. In the context of prostate cancer (PCa), some studies have correlated gross histological measurements of gland lumen, epithelium, and nuclei with disease appearance on MRI. Quantitative histomorphometry (QH), like radiomics for radiologic images, is the computer based extraction of features for describing tumor morphology on digitized tissue images. In this work, we attempt to establish the histomorphometric basis for radiomic features for prostate cancer by (1) identifying the radiomic features from T2w MRI most discriminating of low vs. intermediate/high Gleason score, (2) identifying QH features correlated with the most discriminating radiomic features previously identified, and (3) evaluating the discriminative ability of QH features found to be correlated with spatially co-localized radiomic features. On a cohort of 36 patients (23 for training, 13 for validation), Gabor texture features were identified as being most predictive of Gleason grade on MRI (AUC of 0.69) and gland lumen shape features were identified as the most predictive QH features (AUC = 0.75). Our results suggest that the PCa grade discriminability of Gabor features is a consequence of variations in gland shape and morphology at the tissue level.

Deformable image registration between pathological images and MR image via an optical macro image

Computed tomography (CT) and magnetic resonance (MR) imaging have been widely used for visualizing the inside of the human body. However, in many cases, pathological diagnosis is conducted through a biopsy or resection of an organ to evaluate the condition of tissues as definitive diagnosis. To provide more advanced information onto CT or MR image, it is necessary to reveal the relationship between tissue information and image signals. We propose a registration scheme for a set of PT images of divided specimens and a 3D-MR image by reference to an optical macro image (OM image) captured by an optical camera. We conducted a fundamental study using a resected human brain after the death of a brain cancer patient. We constructed two kinds of registration processes using the OM image as the base for both registrations to make conversion parameters between the PT and MR images. The aligned PT images had shapes similar to the OM image. On the other hand, the extracted cross-sectional MR image was similar to the OM image. From these resultant conversion parameters, the corresponding region on the PT image could be searched and displayed when an arbitrary pixel on the MR image was selected. The relationship between the PT and MR images of the whole brain can be analyzed using the proposed method. We confirmed that same regions between the PT and MR images could be searched and displayed using resultant information obtained by the proposed method. In terms of the accuracy of proposed method, the TREs were 0.56±0.39mm and 0.87±0.42mm. We can analyze the relationship between tissue information and MR signals using the proposed method.

Computer-aided Detection of Prostate Cancer with MRI: Technology and Applications

One in six men will develop prostate cancer in his lifetime. Early detection and accurate diagnosis of the disease can improve cancer survival and reduce treatment costs. Recently, imaging of prostate cancer has greatly advanced since the introduction of multiparametric magnetic resonance imaging (mp-MRI). Mp-MRI consists of T2-weighted sequences combined with functional sequences including dynamic contrast-enhanced MRI, diffusion-weighted MRI, and magnetic resonance spectroscopy imaging. Because of the big data and variations in imaging sequences, detection can be affected by multiple factors such as observer variability and visibility and complexity of the lesions. To improve quantitative assessment of the disease, various computer-aided detection systems have been designed to help radiologists in their clinical practice. This review paper presents an overview of literatures on computer-aided detection of prostate cancer with mp-MRI, which include the technology and its applications. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application.

AutoStitcher: An Automated Program for Efficient and Robust Reconstruction of Digitized Whole Histological Sections from Tissue Fragments

In applications involving large tissue specimens that have been sectioned into smaller tissue fragments, manual reconstruction of a “pseudo whole-mount” histological section (PWMHS) can facilitate (a) pathological disease annotation, and (b) image registration and correlation with radiological images. We have previously presented a program called HistoStitcher, which allows for more efficient manual reconstruction than general purpose image editing tools (such as Photoshop). However HistoStitcher is still manual and hence can be laborious and subjective, especially when doing large cohort studies. In this work we present AutoStitcher, a novel automated algorithm for reconstructing PWMHSs from digitized tissue fragments. AutoStitcher reconstructs (“stitches”) a PWMHS from a set of 4 fragments by optimizing a novel cost function that is domain-inspired to ensure (i) alignment of similar tissue regions, and (ii) contiguity of the prostate boundary. The algorithm achieves computational efficiency by performing reconstruction in a multi-resolution hierarchy. Automated PWMHS reconstruction results (via AutoStitcher) were quantitatively and qualitatively compared to manual reconstructions obtained via HistoStitcher for 113 prostate pathology sections. Distances between corresponding fiducials placed on each of the automated and manual reconstruction results were between 2.7%–3.2%, reflecting their excellent visual similarity.

Development of a 2D Image Reconstruction and Viewing System for Histological Images from Multiple Tissue Blocks: Towards High-Resolution Whole-Organ 3D Histological Images

High-resolution 3D histology image reconstruction of the whole brain organ starts from reconstructing the high-resolution 2D histology images of a brain slice. In this paper, we introduced a method to automatically align the histology images of thin tissue sections cut from the multiple paraffin-embedded tissue blocks of a brain slice. For this method, we employed template matching and incorporated an optimization technique to further improve the accuracy of the 2D reconstructed image. In the template matching, we used the gross image of the brain slice as a reference to the reconstructed 2D histology image of the slice, while in the optimization procedure, we utilized the Jaccard index as the metric of the reconstruction accuracy. The results of our experiment on the initial 3 different whole-brain tissue slices showed that while the method works, it is also constrained by tissue deformations introduced during the tissue processing and slicing. The size of the reconstructed high-resolution 2D histology image of a brain slice is huge, and designing an image viewer that makes particularly efficient use of the computing power of a standard computer used in our laboratories is of interest. We also present the initial implementation of our 2D image viewer system in this paper.

Customizing Laboratory Information Systems: Closing the Functionality Gap

Highly customizable laboratory information systems help to address great variations in laboratory workflows, typical in Pathology. Often, however, built-in customization tools are not sufficient to add all of the desired functionality and improve systems interoperability. Emerging technologies and advances in medicine often create a void in functionality that we call a functionality gap. These gaps have distinct characteristics—a persuasive need to change the way a pathology group operates, the general availability of technology to address the missing functionality, the absence of this technology from your laboratory information system, and inability of built-in customization tools to address it. We emphasize the pervasive nature of these gaps, the role of pathology informatics in closing them, and suggest methods on how to achieve that. We found that a large number of the papers in the Journal of Pathology Informatics are concerned with these functionality gaps, and an even larger proportion of electronic posters and abstracts presented at the Pathology Informatics Summit conference each year deal directly with these unmet needs in pathology practice. A rapid, continuous, and sustainable approach to closing these gaps is critical for Pathology to provide the highest quality of care, adopt new technologies, and meet regulatory and financial challenges. The key element of successfully addressing functionality gaps is gap ownership—the ability to control the entire pathology information infrastructure with access to complementary systems and components. In addition, software developers with detailed domain expertise, equipped with right tools and methodology can effectively address these needs as they emerge.

Identifying in vivo DCE MRI markers associated with microvessel architecture and gleason grades of prostate cancer

BACKGROUND

To identify computer extracted in vivo dynamic contrast enhanced (DCE) MRI markers associated with quantitative histomorphometric (QH) characteristics of microvessels and Gleason scores (GS) in prostate cancer.

METHODS

This study considered retrospective data from 23 biopsy confirmed prostate cancer patients who underwent 3 Tesla multiparametric MRI before radical prostatectomy (RP). Representative slices from RP specimens were stained with vascular marker CD31. Tumor extent was mapped from RP sections onto DCE MRI using nonlinear registration methods. Seventy-seven microvessel QH features and 18 DCE MRI kinetic features were extracted and evaluated for their ability to distinguish low from intermediate and high GS. The effect of temporal sampling on kinetic features was assessed and correlations between those robust to temporal resolution and microvessel features discriminative of GS were examined.

RESULTS

A total of 12 microvessel architectural features were discriminative of low and intermediate/high grade tumors with area under the receiver operating characteristic curve (AUC) > 0.7. These features were most highly correlated with mean washout gradient (WG) (max rho = -0.62). Independent analysis revealed WG to be moderately robust to temporal resolution (intraclass correlation coefficient [ICC] = 0.63) and WG variance, which was poorly correlated with microvessel features, to be predictive of low grade tumors (AUC = 0.77). Enhancement ratio was the most robust (ICC = 0.96) and discriminative (AUC = 0.78) kinetic feature but was moderately correlated with microvessel features (max rho = -0.52).

CONCLUSION

Computer extracted features of prostate DCE MRI appear to be correlated with microvessel architecture and may be discriminative of low versus intermediate and high GS.

Building of a composite virtual slide from contiguous tissue samples

Background

Currently available microscope slide scanners produce whole slide images at various resolutions from histological sections. Nevertheless, acquisition area and so visualization of large tissue samples are limited by the standardized size of glass slides, used daily in pathology departments. The proposed solution has been developed to build composite virtual slides from images of large tumor fragments.

Materials and methods

Images of HES or immunostained histological sections of carefully labeled fragments from a representative slice of breast carcinoma were acquired with a digital slide scanner at a magnification of 20×. The tiling program involves three steps: the straightening of tissue fragment images using polynomial interpolation method, and the building and assembling of strips of contiguous tissue sample whole slide images in × and y directions. The final image is saved in a pyramidal BigTiff file format. The program has been tested on several tumor slices. A correlation quality control has been done on five images artificially cut.

Results

Sixty tumor slices from twenty surgical specimens, cut into two to twenty six pieces, were reconstructed. A median of 98.71% is obtained by computing the correlation coefficients between native and reconstructed images for quality control.

Conclusions

The proposed method is efficient and able to adapt itself to daily work conditions of classical pathology laboratories.