Prostate histopathology: learning tissue component histograms for cancer detection and classification

Relevance of Prostatic Fluid on the Apparent Diffusion Coefficient: An Inversion Recovery Diffusion-Weighted Imaging Investigation

OBJECTIVES

Diffusion-weighted imaging (DWI) is pivotal for prostate magnetic resonance imaging. This is rooted in the generally reduced apparent diffusion coefficient (ADC) observed in prostate cancer in comparison to healthy prostate tissue. This difference originates from microstructural tissue composition changes, including a potentially decreased fluid-containing lumen volume. This study explored the nature of the observed ADC contrast in prostate tissue through inversion recovery-prepared DWI examinations that generated varying levels of fluid suppression.

MATERIALS AND METHODS

This institutional review board-approved, single-center, prospective study was conducted from 2023 to 2024; all participants underwent magnetic resonance imaging including DWI with b-values of 50 and 800 s/mm 2 at 16 inversion times (TI; 60-4000 milliseconds). The measured ADC was interpreted with a 2-compartment model (compartments: tissue and fluid). Descriptive statistics were computed for all analyzed parameters.

RESULTS

Twelve healthy male volunteers (45 ± 17 years) and 1 patient with prostate adenocarcinoma (66 years) were evaluated. The ADC map appearance depended heavily on the TI, and we observed a feature-rich ADC(TI) curve. The ADC in the transition zone (TZ) of healthy volunteers increased between TI = 60 milliseconds and approximately 1100 milliseconds, then dropped drastically before increasing again, stabilizing at a very high TI. This effect was greatly reduced in the patient's prostate cancer lesion. The 2-compartment model described this behavior well. After the inversion, tissue magnetization recovers faster, decreasing its signal contribution in absolute terms and resulting in an increase in the ADC. At the tipping point, the total magnetization is zero at b = 0, when the positive tissue magnetization and still-inverted fluid magnetization cancel out. A small diffusion encoding leads to a positive signal, thus generating an infinite ADC. After the tipping point, the fluid magnetization remains negative and thereby reduces the ADC.

CONCLUSIONS

Prostate fluid appears to contribute significantly to prostate ADCs. Its contribution could be adjusted by choosing an appropriate inversion recovery preparation, potentially enhancing contrast for prostate cancer lesions.

MoMA: Momentum contrastive learning with multi-head attention-based knowledge distillation for histopathology image analysis

There is no doubt that advanced artificial intelligence models and high quality data are the keys to success in developing computational pathology tools. Although the overall volume of pathology data keeps increasing, a lack of quality data is a common issue when it comes to a specific task due to several reasons including privacy and ethical issues with patient data. In this work, we propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model, to overcome such issues in computational pathology. Specifically, we employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data and distill relevant knowledge via momentum contrastive learning with multi-head attention mechanism, which provides consistent and context-aware feature representations. This enables the target model to assimilate informative representations of the teacher model while seamlessly adapting to the unique nuances of the target data. The proposed method is rigorously evaluated across different scenarios where the teacher model was trained on the same, relevant, and irrelevant classification tasks with the target model. Experimental results demonstrate the accuracy and robustness of our approach in transferring knowledge to different domains and tasks, outperforming other related methods. Moreover, the results provide a guideline on the learning strategy for different types of tasks and scenarios in computational pathology.

Pros and Cons of High-Performance Gradient Enabled Short-TE Prostate DWI: A Prospective Study

OBJECTIVES

Recent advances in high-performance gradient technology have enabled shorter echo times (TEs) for diffusion-weighted prostate MRI. Short TE may improve the conspicuity of the usually T2 hypointense lesions but may also influence the diagnostic performance of the apparent diffusion coefficient (ADC) due to a changed weighting of subcompartments, including prostate fluid and tissues. The purpose of this study was to evaluate the influence of TE on prostate diffusion-weighted images with respect to lesion conspicuity and diagnostic performance of the ADC.

MATERIALS AND METHODS

This institutional review board-approved prospective monocentric study included n = 55 (mean age 69 ± 9 years) patients undergoing clinically indicated prostate MRI on two 3 T MRI scanners with high-performance gradients. Diffusion-weighted imaging (DWI) was performed with an echo-planar sequence at 2 different TEs, 41 ms and 70 ms, with b-values of 50 s/mm2 and 800 s/mm2. Computed DWI was generated for a b-value of 1400 s/mm2. The lesion conspicuity and image quality were rated by 3 independent readers with a 5-point Likert scale and tested with the Wilcoxon rank sum test. Lesion ADCs were recorded, and their ability to detect significant lesions (Gleason score >6) was assessed with a receiver operator curve analysis.

RESULTS

Among the participants, n = 24 had clinically significant prostate cancer. The image quality at b = 1400 s/mm2 was rated significantly higher at TE = 41 ms than at TE = 70 ms (mean Likert score ± standard deviation for TE = 41 ms vs TE = 70 ms: R1: 4.06 ± 0.68 vs 3.02 ± 0.59; R2: 4.09 ± 0.82 vs 3.26 ± 0.67; R3: 4.16 ± 0.71 vs 3.18 ± 0.70; for all P's < 0.001). The lesion conspicuity at b = 1400 s/mm2 was rated higher at TE = 41 ms than at TE = 70 ms (mean Likert score ± standard deviation for TE = 41 ms vs TE = 70 ms: R1: 4.55 ± 0.66 vs 4.46 ± 0.72, P = 0.17; R2: 4.64 ± 0.59 vs 4.53 ± 0.63, P = 0.03; R3: 4.53 ± 0.66 vs 4.28 ± 0.80, P = 0.01). However, the ADC-based area under the curve for lesion characterization decreased from 0.80 at TE = 70 ms to 0.70 at TE = 41 ms (P = 0.07).

CONCLUSIONS

Shortening TE to 41 ms in prostate DWI increases lesion conspicuity on high b-value images; however, it negatively impacts the diagnostic performance of the ADC.

Automating Prostate Cancer Grading: A Novel Deep Learning Framework for Automatic Prostate Cancer Grade Assessment using Classification and Segmentation

Prostate Cancer (PCa) is the second most common cancer in men and affects more than a million people each year. Grading prostate cancer is based on the Gleason grading system, a subjective and labor-intensive method for evaluating prostate tissue samples. The variability in diagnostic approaches underscores the urgent need for more reliable methods. By integrating deep learning technologies and developing automated systems, diagnostic precision can be improved, and human error minimized. The present work introduces a three-stage framework-based innovative deep-learning system for assessing PCa severity using the PANDA challenge dataset. After a meticulous selection process, 2699 usable cases were narrowed down from the initial 5160 cases after extensive data cleaning. There are three stages in the proposed framework: classification of PCa grades using deep neural networks (DNNs), segmentation of PCa grades, and computation of International Society for Urological Pathology (ISUP) grades using machine learning classifiers. Four classes of patches were classified and segmented (benign, Gleason 3, Gleason 4, and Gleason 5). Patch sampling at different sizes (500 × 500 and 1000 × 1000 pixels) was used to optimize the classification and segmentation processes. The segmentation performance of the proposed network is enhanced by a Self-organized operational neural network (Self-ONN) based DeepLabV3 architecture. Based on these predictions, the distribution percentages of each cancer grade within the whole slide images (WSI) were calculated. These features were then concatenated into machine learning classifiers to predict the final ISUP PCa grade. EfficientNet_b0 achieved the highest F1-score of 83.83% for classification, while DeepLabV3 + architecture based on self-ONN and EfficientNet encoder achieved the highest Dice Similarity Coefficient (DSC) score of 84.9% for segmentation. Using the RandomForest (RF) classifier, the proposed framework achieved a quadratic weighted kappa (QWK) score of 0.9215. Deep learning frameworks are being developed to grade PCa automatically and have shown promising results. In addition, it provides a prospective approach to a prognostic tool that can produce clinically significant results efficiently and reliably. Further investigations are needed to evaluate the framework's adaptability and effectiveness across various clinical scenarios.

Transfer Learning with Pretrained Convolutional Neural Network for Automated Gleason Grading of Prostate Cancer Tissue Microarrays

Background

The Gleason grading system has been the most effective prediction for prostate cancer patients. This grading system provides this possibility to assess prostate cancer's aggressiveness and then constitutes an important factor for stratification and therapeutic decisions. However, determining Gleason grade requires highly-trained pathologists and is time-consuming and tedious, and suffers from inter-pathologist variability. To remedy these limitations, this paper introduces an automatic methodology based on transfer learning with pretrained convolutional neural networks (CNNs) for automatic Gleason grading of prostate cancer tissue microarray (TMA).

Methods

Fifteen pretrained (CNNs): Efficient Nets (B0-B5), NasNetLarge, NasNetMobile, InceptionV3, ResNet-50, SeResnet 50, Xception, DenseNet121, ResNext50, and inception_resnet_v2 were fine-tuned on a dataset of prostate carcinoma TMA images. Six pathologists separately identified benign and cancerous areas for each prostate TMA image by allocating benign, 3, 4, or 5 Gleason grade for 244 patients. The dataset was labeled by these pathologists and majority vote was applied on pixel-wise annotations to obtain a unified label.

Results

Results showed the NasnetLarge architecture is the best model among them in the classification of prostate TMA images of 244 patients with accuracy of 0.93 and area under the curve of 0.98.

Conclusion

Our study can act as a highly trained pathologist to categorize the prostate cancer stages with more objective and reproducible results.

Artificial Intelligence and Pathomics: Prostate Cancer

Artificial intelligence (AI) has the potential to transform pathologic diagnosis and cancer patient management as a predictive and prognostic biomarker. AI-based systems can be used to examine digitally scanned histopathology slides and differentiate benign from malignant cells and low from high grade. Deep learning models can analyze patient data from individual or multimodal combinations and identify patterns to be used to predict the response to different therapeutic options, the risk of recurrence or progression, and the prognosis of the newly diagnosed patient. AI-based models will improve treatment planning for patients with prostate cancer and improve the efficiency and cost-effectiveness of the pathology laboratory.

A systematic review and meta-analysis of artificial intelligence diagnostic accuracy in prostate cancer histology identification and grading

BACKGROUND

Artificial intelligence (AI) is a promising tool in pathology, including cancer diagnosis, subtyping, grading, and prognostic prediction.

METHODS

The aim of the study is to assess AI application in prostate cancer (PCa) histology. We carried out a systematic literature search in 3 databases. Primary outcome was AI accuracy in differentiating between PCa and benign hyperplasia. Secondary outcomes were AI accuracy in determining Gleason grade and agreement among AI and pathologists.

RESULTS

Our final sample consists of 24 studies conducted from 2007 to 2021. They aggregate data from roughly 8000 cases of prostate biopsy and 458 cases of radical prostatectomy (RP). Sensitivity for PCa diagnostic exceeded 90% and ranged from 87% to 100%, and specificity varied from 68% to 99%. Overall accuracy ranged from 83.7% to 98.3% with AUC reaching 0.99. The meta-analysis using the Mantel-Haenszel method showed pooled sensitivity of 0.96 with I2 = 80.7% and pooled specificity of 0.95 with I2 = 86.1%. Pooled positive likehood ratio was 15.3 with I2 = 87.3% and negative - was 0.04 with I2 = 78.6%. SROC (symmetric receiver operating characteristics) curve represents AUC = 0.99. For grading the accuracy of AI was lower: sensitivity for Gleason grading ranged from 77% to 87%, and specificity from 82% to 90%.

CONCLUSIONS

The accuracy of AI for PCa identification and grading is comparable to expert pathologists. This is a promising approach which has several possible clinical applications resulting in expedite and optimize pathology reports. AI introduction into common practice may be limited by difficult and time-consuming convolutional neural network training and tuning.

Whole slide image representation in bone marrow cytology

One of the goals of AI-based computational pathology is to generate compact representations of whole slide images (WSIs) that capture the essential information needed for diagnosis. While such approaches have been applied to histopathology, few applications have been reported in cytology. Bone marrow aspirate cytology is the basis for key clinical decisions in hematology. However, visual inspection of aspirate specimens is a tedious and complex process subject to variation in interpretation, and hematopathology expertise is scarce. The ability to generate a compact representation of an aspirate specimen may form the basis for clinical decision-support tools in hematology. In this study, we leverage our previously published end-to-end AI-based system for counting and classifying cells from bone marrow aspirate WSIs, which enables the direct use of individual cells as inputs rather than WSI patches. We then construct bags of individual cell features from each WSI, and apply multiple instance learning to extract their vector representations. To evaluate the quality of our representations, we conducted WSI retrieval and classification tasks. Our results show that we achieved a mAP@10 of 0.58 ±0.02 in WSI-level image retrieval, surpassing the random-retrieval baseline of 0.39 ±0.1. Furthermore, we predicted five diagnostic labels for individual aspirate WSIs with a weighted-average F1 score of 0.57 ±0.03 using a k-nearest-neighbors (k-NN) model, outperforming guessing using empirical class prior probabilities (0.26 ±0.02). We present the first example of exploring trainable mechanisms to generate compact, slide-level representations in bone marrow cytology with deep learning. This method has the potential to summarize complex semantic information in WSIs toward improved diagnostics in hematology, and may eventually support AI-assisted computational pathology approaches.

Lung cancer lesion detection in histopathology images using graph-based sparse PCA network

Early detection of lung cancer is critical for improvement of patient survival. To address the clinical need for efficacious treatments, genetically engineered mouse models (GEMM) have become integral in identifying and evaluating the molecular underpinnings of this complex disease that may be exploited as therapeutic targets. Assessment of GEMM tumor burden on histopathological sections performed by manual inspection is both time consuming and prone to subjective bias. Therefore, an interplay of needs and challenges exists for computer-aided diagnostic tools, for accurate and efficient analysis of these histopathology images. In this paper, we propose a simple machine learning approach called the graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E). Our method comprises four steps: 1) cascaded graph-based sparse PCA, 2) PCA binary hashing, 3) block-wise histograms, and 4) support vector machine (SVM) classification. In our proposed architecture, graph-based sparse PCA is employed to learn the filter banks of the multiple stages of a convolutional network. This is followed by PCA hashing and block histograms for indexing and pooling. The meaningful features extracted from this GS-PCA are then fed to an SVM classifier. We evaluate the performance of the proposed algorithm on H&E slides obtained from an inducible K-ras^G12D lung cancer mouse model using precision/recall rates, F_β-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC) and show that our algorithm is efficient and provides improved detection accuracy compared to existing algorithms.

An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS)

Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.

Applications of Digital and Computational Pathology and Artificial Intelligence in Genitourinary Pathology Diagnostics

As machine learning (ML) solutions for genitourinary pathology image analysis are fostered by a progressively digitized laboratory landscape, these integrable modalities usher in a revolution in histopathological diagnosis. As technology advances, limitations stymying clinical artificial intelligence (AI) will not be extinguished without thorough validation and interrogation of ML tools by pathologists and regulatory bodies alike. ML solutions deployed in clinical settings for applications in prostate pathology yield promising results. Recent breakthroughs in clinical artificial intelligence for genitourinary pathology demonstrate unprecedented generalizability, heralding prospects for a future in which AI-driven assistive solutions may be seen as laboratory faculty, rather than novelty.

Artificial intelligence techniques for neuropathological diagnostics and research

Artificial intelligence (AI) research began in theoretical neurophysiology, and the resulting classical paper on the McCulloch-Pitts mathematical neuron was written in a psychiatry department almost 80 years ago. However, the application of AI in digital neuropathology is still in its infancy. Rapid progress is now being made, which prompted this article. Human brain diseases represent distinct system states that fall outside the normal spectrum. Many differ not only in functional but also in structural terms, and the morphology of abnormal nervous tissue forms the traditional basis of neuropathological disease classifications. However, only a few countries have the medical specialty of neuropathology, and, given the sheer number of newly developed histological tools that can be applied to the study of brain diseases, a tremendous shortage of qualified hands and eyes at the microscope is obvious. Similarly, in neuroanatomy, human observers no longer have the capacity to process the vast amounts of connectomics data. Therefore, it is reasonable to assume that advances in AI technology and, especially, whole-slide image (WSI) analysis will greatly aid neuropathological practice. In this paper, we discuss machine learning (ML) techniques that are important for understanding WSI analysis, such as traditional ML and deep learning, introduce a recently developed neuropathological AI termed PathoFusion, and present thoughts on some of the challenges that must be overcome before the full potential of AI in digital neuropathology can be realized.

A deep learning network for Gleason grading of prostate biopsies using EfficientNet

Objectives

The most crucial part in the diagnosis of cancer is severity grading. Gleason’s score is a widely used grading system for prostate cancer. Manual examination of the microscopic images and grading them is tiresome and consumes a lot of time. Hence to automate the Gleason grading process, a novel deep learning network is proposed in this work.

Methods

In this work, a deep learning network for Gleason grading of prostate cancer is proposed based on EfficientNet architecture. It applies a compound scaling method to balance the dimensions of the underlying network. Also, an additional attention branch is added to EfficientNet-B7 for precise feature weighting.

Result

To the best of our knowledge, this is the first work that integrates an additional attention branch with EfficientNet architecture for Gleason grading. The proposed models were trained using H&E-stained samples from prostate cancer Tissue Microarrays (TMAs) in the Harvard Dataverse dataset.

Conclusions

The proposed network was able to outperform the existing methods and it achieved an Kappa score of 0.5775.

Histological image color normalization using a skewed normal distribution mixed model

Color variation between histological images may influence the performance of computer-aided histological image analysis. Therefore, among the most essential and challenging tasks in histological image analysis are the reduction of the color variation between images and the preservation of the histological information contained in the images. In recent years, many methods have been introduced with respect to the color normalization of histological images. In this study, we introduce a new clustering method referred to as the skewed normal distribution mixed model clustering algorithm. Realizing that the color distribution of hue values approximates the combination of several skewed normal distributions, we propose to use the skewed normal distribution mixture model to analyze the hue distribution. The proposed skewed normal distribution mixture model clustering algorithm includes saturation-weighted hue histograms because it takes into account the saturation and hue information of a particular histogram image, which can diminish the influence of achromatic pixels. Finally, we conducted extensive experiments based on three data sets and compared them with commonly used color normalization methods. The experiments show that the proposed algorithm has better performance in stain separation and color normalization compared to other methods.

Fast whole-slide cartography in colon cancer histology using superpixels and CNN classification

Purpose: Automatic outlining of different tissue types in digitized histological specimen provides a basis for follow-up analyses and can potentially guide subsequent medical decisions. The immense size of whole-slide-images (WSIs), however, poses a challenge in terms of computation time. In this regard, the analysis of nonoverlapping patches outperforms pixelwise segmentation approaches but still leaves room for optimization. Furthermore, the division into patches, regardless of the biological structures they contain, is a drawback due to the loss of local dependencies. Approach: We propose to subdivide the WSI into coherent regions prior to classification by grouping visually similar adjacent pixels into superpixels. Afterward, only a random subset of patches per superpixel is classified and patch labels are combined into a superpixel label. We propose a metric for identifying superpixels with an uncertain classification and evaluate two medical applications, namely tumor area and invasive margin estimation and tumor composition analysis. Results: The algorithm has been developed on 159 hand-annotated WSIs of colon resections and its performance is compared with an analysis without prior segmentation. The algorithm shows an average speed-up of 41% and an increase in accuracy from 93.8% to 95.7%. By assigning a rejection label to uncertain superpixels, we further increase the accuracy by 0.4%. While tumor area estimation shows high concordance to the annotated area, the analysis of tumor composition highlights limitations of our approach. Conclusion: By combining superpixel segmentation and patch classification, we designed a fast and accurate framework for whole-slide cartography that is AI-model agnostic and provides the basis for various medical endpoints.

A statistical, voxelised model of prostate cancer for biologically optimised radiotherapy

Background and purpose

Radiation therapy (RT) is commonly indicated for treatment of prostate cancer (PC). Biologicallyoptimised RT for PC may improve disease-free survival. This requires accurate spatial localisation and characterisation of tumour lesions. We aimed to generate a statistical, voxelised biological model to complement in vivomultiparametric MRI data to facilitate biologically-optimised RT.

Material and methods

Ex vivo prostate MRI and histopathological imaging were acquired for 63 PC patients. These data were co-registered to derive three-dimensional distributions of graded tumour lesions and cell density. Novel registration processes were used to map these data to a common reference geometry. Voxelised statistical models of tumour probability and cell density were generated to create the PC biological atlas. Cell density models were analysed using the Kullback-Leibler divergence to compare normal vs. lognormal approximations to empirical data.

Results

A reference geometry was constructed using ex vivo MRI space, patient data were deformably registered using a novel anatomy-guided process. Substructure correspondence was maintained using peripheral zone definitions to address spatial variability in prostate anatomy between patients. Three distinct approaches to interpolation were designed to map contours, tumour annotations and cell density maps from histology into ex vivo MRI space. Analysis suggests a log-normal model provides a more consistent representation of cell density when compared to a linear-normal model.

Conclusion

A biological model has been created that combines spatial distributions of tumour characteristics from a population into three-dimensional, voxelised, statistical models. This tool will be used to aid the development of biologically-optimised RT for PC patients.

Detection of Structure Characteristics and Its Discontinuity Based Field Programmable Gate Array Processor in Cancer Cell by Wavelet Transform

Digital clinical histopathology is one of the crucial techniques for precise cancer cell diagnosing in modern medicine. The Synovial Sarcoma (SS) cancer cell patterns seem to be a spindle shaped cell (SSC) structure and it is very difficult to identify the exact oval shaped cell structure through pathologist’s eye perception. Meanwhile, there is necessitating for monitoring and securing the successful and effective image data processing in the the huge network data which is also a complex one. A field programmable Gate Array (FPGA) was regarded as a necessary one for this. In this work, based on FPGA a Cancer Cell classification is made for the regulation and execution. Hence, mathematically the SSC regularity structures and its discontinuities are measured by the holder exponent (HE) function. In this research work, HE values have been determined by Wavelet Transform Modulus Maxima (WTMM) and Wavelet Leader (WL) methods with basis function of Haar wavelet based on FPGA Processor. The quantitative parameters such as Mean of Asymptotic Discontinuity (MAD), Mean of Removable Discontinuity (MRD) and Number of Discontinuity Points (NDPs) have been considered to determine the prediction of discontinuity detection between WTMM and WL methods. With the help of receiver operating characteristics (ROC) curve, the significant difference of discontinuity detection performance between both the methods has been analyzed. From the experimental results, it is clear that the WL method is more practically feasible and it gives satisfactory performance, in terms of sensitivity and specificity percentage values, which are 80.56% and 59.46%, respectively in the blue color components of the SNR 20 dB noise image.

Joint categorical and ordinal learning for cancer grading in pathology images

Cancer grading in pathology image analysis is one of the most critical tasks since it is related to patient outcomes and treatment planning. Traditionally, it has been considered a categorical problem, ignoring the natural ordering among the cancer grades, i.e., the higher the grade is, the more aggressive it is, and the worse the outcome is. Herein, we propose a joint categorical and ordinal learning framework for cancer grading in pathology images. The approach simultaneously performs both categorical classification and ordinal classification and aims to leverage the distinctive features from the two tasks. Moreover, we propose a new loss function for the ordinal classification task that offers an improved contrast between the correctly classified examples and misclassified examples. The proposed method is evaluated on multiple collections of colorectal and prostate pathology images that underwent different acquisition and processing procedures. Both quantitative and qualitative assessments of the experimental results confirm the effectiveness and robustness of the proposed method in comparison to other competing methods. The results suggest that the proposed approach could permit improved histopathologic analysis of cancer grades in pathology images.

Multi-scale binary pattern encoding network for cancer classification in pathology images

Multi-scale approaches have been widely studied in pathology image analysis. These offer an ability to characterize tissues in an image at various scales, in which the tissues may appear differently. Many of such methods have focused on extracting multi-scale hand-crafted features and applied them to various tasks in pathology image analysis. Even, several deep learning methods explicitly adopt the multi-scale approaches. However, most of these methods simply merge the multi-scale features together or adopt the coarse-to-fine/fine-to-coarse strategy, which uses the features one at a time in a sequential manner. Utilizing the multi-scale features in a cooperative and discriminative fashion, the learning capabilities could be further improved. Herein, we propose a multi-scale approach that can identify and leverage the patterns of the multiple scales within a deep neural network and provide the superior capability of cancer classification. The patterns of the features across multiple scales are encoded as a binary pattern code and further converted to a decimal number, which can be easily embedded in the current framework of the deep neural networks. To evaluate the proposed method, multiple sets of pathology images are employed. Under the various experimental settings, the proposed method is systematically assessed and shows an improved classification performance in comparison to other competing methods.

The 2019 Genitourinary Pathology Society (GUPS) White Paper on Contemporary Grading of Prostate Cancer

CONTEXT.—

Controversies and uncertainty persist in prostate cancer grading.

OBJECTIVE.—

To update grading recommendations.

DATA SOURCES.—

Critical review of the literature along with pathology and clinician surveys.

CONCLUSIONS.—

Percent Gleason pattern 4 (%GP4) is as follows: (1) report %GP4 in needle biopsy with Grade Groups (GrGp) 2 and 3, and in needle biopsy on other parts (jars) of lower grade in cases with at least 1 part showing Gleason score (GS) 4 + 4 = 8; and (2) report %GP4: less than 5% or less than 10% and 10% increments thereafter. Tertiary grade patterns are as follows: (1) replace "tertiary grade pattern" in radical prostatectomy (RP) with "minor tertiary pattern 5 (TP5)," and only use in RP with GrGp 2 or 3 with less than 5% Gleason pattern 5; and (2) minor TP5 is noted along with the GS, with the GrGp based on the GS. Global score and magnetic resonance imaging (MRI)-targeted biopsies are as follows: (1) when multiple undesignated cores are taken from a single MRI-targeted lesion, an overall grade for that lesion is given as if all the involved cores were one long core; and (2) if providing a global score, when different scores are found in the standard and the MRI-targeted biopsy, give a single global score (factoring both the systematic standard and the MRI-targeted positive cores). Grade Groups are as follows: (1) Grade Groups (GrGp) is the terminology adopted by major world organizations; and (2) retain GS 3 + 5 = 8 in GrGp 4. Cribriform carcinoma is as follows: (1) report the presence or absence of cribriform glands in biopsy and RP with Gleason pattern 4 carcinoma. Intraductal carcinoma (IDC-P) is as follows: (1) report IDC-P in biopsy and RP; (2) use criteria based on dense cribriform glands (>50% of the gland is composed of epithelium relative to luminal spaces) and/or solid nests and/or marked pleomorphism/necrosis; (3) it is not necessary to perform basal cell immunostains on biopsy and RP to identify IDC-P if the results would not change the overall (highest) GS/GrGp part per case; (4) do not include IDC-P in determining the final GS/GrGp on biopsy and/or RP; and (5) "atypical intraductal proliferation (AIP)" is preferred for an intraductal proliferation of prostatic secretory cells which shows a greater degree of architectural complexity and/or cytological atypia than typical high-grade prostatic intraepithelial neoplasia, yet falling short of the strict diagnostic threshold for IDC-P. Molecular testing is as follows: (1) Ki67 is not ready for routine clinical use; (2) additional studies of active surveillance cohorts are needed to establish the utility of PTEN in this setting; and (3) dedicated studies of RNA-based assays in active surveillance populations are needed to substantiate the utility of these expensive tests in this setting. Artificial intelligence and novel grading schema are as follows: (1) incorporating reactive stromal grade, percent GP4, minor tertiary GP5, and cribriform/intraductal carcinoma are not ready for adoption in current practice.

A Multi-Channel and Multi-Spatial Attention Convolutional Neural Network for Prostate Cancer ISUP Grading

Prostate cancer (PCa) is one of the most prevalent cancers worldwide. As the demand for prostate biopsies increases, a worldwide shortage and an uneven geographical distribution of proficient pathologists place a strain on the efficacy of pathological diagnosis. Deep learning (DL) is able to automatically extract features from whole-slide images of prostate biopsies annotated by skilled pathologists and to classify the severity of PCa. A whole-slide image of biopsies has many irrelevant features that weaken the performance of DL models. To enable DL models to focus more on cancerous tissues, we propose a Multi-Channel and Multi-Spatial (MCMS) Attention module that can be easily plugged into any backbone CNN to enhance feature extraction. Specifically, MCMS learns a channel attention vector to assign weights to channels in the feature map by pooling from multiple attention branches with different reduction ratios; similarly, it also learns a spatial attention matrix to focus on more relevant areas of the image, by pooling from multiple convolutional layers with different kernel sizes. The model is verified on the most extensive multi-center PCa dataset that consists of 11,000 H&E-stained histopathology whole-slide images. Experimental results demonstrate that an MCMS-assisted CNN can effectively boost prediction performance in accuracy (ACC) and quadratic weighted kappa (QWK), compared with prior studies. The proposed model and results can serve as a credible benchmark for future research in automated PCa grading.

Pathomics in urology

PURPOSE OF REVIEW

Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology.

RECENT FINDINGS

There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application.

SUMMARY

In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

Discriminative dictionary learning algorithm with pairwise local constraints for histopathological image classification

Histopathological image contains rich pathological information that is valued for the aided diagnosis of many diseases such as cancer. An important issue in histopathological image classification is how to learn a high-quality discriminative dictionary due to diverse tissue pattern, a variety of texture, and different morphologies structure. In this paper, we propose a discriminative dictionary learning algorithm with pairwise local constraints (PLCDDL) for histopathological image classification. Inspired by the one-to-one mapping between dictionary atom and profile, we learn a pair of discriminative graph Laplacian matrices that are less sensitive to noise or outliers to capture the locality and discriminating information of data manifold by utilizing the local geometry information of category-specific dictionaries rather than input data. Furthermore, graph-based pairwise local constraints are designed and incorporated into the original dictionary learning model to effectively encode the locality consistency with intra-class samples and the locality inconsistency with inter-class samples. Specifically, we learn the discriminative localities for representations by jointly optimizing both the intra-class locality and inter-class locality, which can significantly improve the discriminability and robustness of dictionary. Extensive experiments on the challenging datasets verify that the proposed PLCDDL algorithm can achieve a better classification accuracy and powerful robustness compared with the state-of-the-art dictionary learning methods. Graphical abstract The proposed PLCDDL algorithm. 1) A pair of graph Laplacian matrices are first learned based on the class-specific dictionaries. 2) Graph-based pairwise local constraints are designed to transfer the locality for coding coefficients. 3) Class-specific dictionaries can be further updated.

Precision Medicine, Artificial Intelligence, and Genomic Markers in Urology. Do we need to Tailor our Clinical Practice?

Precision medicine plays a key role in urological oncology practice nowadays, with the breakthrough of the poly (ADP-ribose) polymerase inhibitors (PARPi), which play a critical role in different DNA damage repair (DDR) pathways, the immune checkpoint inhibitors, the genomic expression profiles and current genome manipulation-directed targeted therapy. Information and technology (IT) are set to change the way we assess and treat patients and should be reviewed and discussed. The aim of the present article is to demonstrate a detailed revision on precision medicine, including novel therapeutic targets, genomic markers, genomic stratification of urological patients, and the top-notch technological breakthroughs that could change our clinical practice

We performed a review of the literature in four different databases (PubMed, Embase, Lilacs, and Scielo) on any information concerning prostate, bladder, kidney and urothelial cancer novel treatments with PARPi, immune checkpoint inhibitors (ICIs), targeted therapy with fibroblast growth factor receptor inhibitors (FGFRi), and theranostics with prostate-specific membrane antigen (PSMA) targeted monoclonal antibodies. Artificial intelligence, machine learning, and deep learning algorithm in urological practice were also part of the search. We included all articles written in English, published within the past 7 years, that discussed outstanding therapies and genomics in urological cancer and artificial intelligence applied to urology. Meanwhile, we excluded articles with lack of a clear methodology and written in any other language than English.

One-hundred and twenty-six articles of interest were found; of these, 65 articles that presented novel treatments of urological neoplasms, discussed precision medicine, genomic expression profiles and biomarkers in urology, and latest deep learning and machine learning algorithms as well as the use of artificial intelligence in urological practice were selected. A critical review of the literature is presented in the present article.

Urology is a constantly changing specialty with a wide range of therapeutic breakthroughs, a huge understanding of the genomic expression profiles for each urological cancer and a tendency to use cutting-edge technology to treat our patients. All of these major developments must be analyzed objectively, taking into account costs to the health systems, risks and benefits to the patients, and the legal background that comes with them. A critical analysis of these new technologies and pharmacological breakthroughs should be made before considering changing our clinical practice. Nowadays, research needs to be strengthened to help us improve results in assessing and treating our patients.

The Application of Artificial Intelligence in Prostate Cancer Management—What Improvements Can Be Expected? A Systematic Review

Artificial Intelligence (AI) is progressively remodeling our daily life. A large amount of information from “big data” now enables machines to perform predictions and improve our healthcare system. AI has the potential to reshape prostate cancer (PCa) management thanks to growing applications in the field. The purpose of this review is to provide a global overview of AI in PCa for urologists, pathologists, radiotherapists, and oncologists to consider future changes in their daily practice. A systematic review was performed, based on PubMed MEDLINE, Google Scholar, and DBLP databases for original studies published in English from January 2009 to January 2019 relevant to PCa, AI, Machine Learning, Artificial Neural Networks, Convolutional Neural Networks, and Natural-Language Processing. Only articles with full text accessible were considered. A total of 1008 articles were reviewed, and 48 articles were included. AI has potential applications in all fields of PCa management: analysis of genetic predispositions, diagnosis in imaging, and pathology to detect PCa or to differentiate between significant and non-significant PCa. AI also applies to PCa treatment, whether surgical intervention or radiotherapy, skills training, or assessment, to improve treatment modalities and outcome prediction. AI in PCa management has the potential to provide a useful role by predicting PCa more accurately, using a multiomic approach and risk-stratifying patients to provide personalized medicine.

Using hyperspectral imaging automatic classification of gastric cancer grading with a shallow residual network

The gastric cancer grading of patients determines their clinical treatment plan. We use hyperspectral imaging (HSI) gastric cancer section data to automatically classify the three different cancer grades (low grade, intermediate grade, and high grade) and healthy tissue. This paper proposed the use of HSI data combined with a shallow residual network (SR-Net) as the classifier. We collected hyperspectral data from gastric sections of 30 participants, with the wavelength range of hyperspectral data being 374 nm to 990 nm. We compared the classification results between hyperspectral data and color images. The results show that using hyperspectral data and a SR-Net an average classification accuracy of 91.44% could be achieved, which is 13.87% higher than that of the color image. In addition, we applied a modified SR-Net incorporated direct down-sampling, asymmetric filters, and global average pooling to reduce the parameters and floating-point operations. Compared with the regular residual network with the same number of blocks, the floating-point operations of a SR-Net are one order of magnitude less. The experimental results show that hyperspectral data with a SR-Net can achieve cutting-edge performance with minimum computational cost and therefore have potential in the study of gastric cancer grading.

Automatic cancer detection on digital histopathology images of mid-gland radical prostatectomy specimens

Purpose: Automatic cancer detection on radical prostatectomy (RP) sections facilitates graphical and quantitative surgical pathology reporting, which can potentially benefit postsurgery follow-up care and treatment planning. It can also support imaging validation studies using a histologic reference standard and pathology research studies. This problem is challenging due to the large sizes of digital histopathology whole-mount whole-slide images (WSIs) of RP sections and staining variability across different WSIs. Approach: We proposed a calibration-free adaptive thresholding algorithm, which compensates for staining variability and yields consistent tissue component maps (TCMs) of the nuclei, lumina, and other tissues. We used and compared three machine learning methods for classifying each cancer versus noncancer region of interest (ROI) throughout each WSI: (1) conventional machine learning methods and 14 texture features extracted from TCMs, (2) transfer learning with pretrained AlexNet fine-tuned by TCM ROIs, and (3) transfer learning with pretrained AlexNet fine-tuned with raw image ROIs. Results: The three methods yielded areas under the receiver operating characteristic curve of 0.96, 0.98, and 0.98, respectively, in leave-one-patient-out cross validation using 1.3 million ROIs from 286 mid-gland whole-mount WSIs from 68 patients. Conclusion: Transfer learning with the use of TCMs demonstrated state-of-the-art overall performance and is more stable with respect to sample size across different tissue types. For the tissue types involving Gleason 5 (most aggressive) cancer, it achieved the best performance compared to the other tested methods. This tool can be translated to clinical workflow to assist graphical and quantitative pathology reporting for surgical specimens upon further multicenter validation.

Histologic tissue components provide major cues for machine learning-based prostate cancer detection and grading on prostatectomy specimens

Automatically detecting and grading cancerous regions on radical prostatectomy (RP) sections facilitates graphical and quantitative pathology reporting, potentially benefitting post-surgery prognosis, recurrence prediction, and treatment planning after RP. Promising results for detecting and grading prostate cancer on digital histopathology images have been reported using machine learning techniques. However, the importance and applicability of those methods have not been fully investigated. We computed three-class tissue component maps (TCMs) from the images, where each pixel was labeled as nuclei, lumina, or other. We applied seven different machine learning approaches: three non-deep learning classifiers with features extracted from TCMs, and four deep learning, using transfer learning with the 1) TCMs, 2) nuclei maps, 3) lumina maps, and 4) raw images for cancer detection and grading on whole-mount RP tissue sections. We performed leave-one-patient-out cross-validation against expert annotations using 286 whole-slide images from 68 patients. For both cancer detection and grading, transfer learning using TCMs performed best. Transfer learning using nuclei maps yielded slightly inferior overall performance, but the best performance for classifying higher-grade cancer. This suggests that 3-class TCMs provide the major cues for cancer detection and grading primarily using nucleus features, which are the most important information for identifying higher-grade cancer.

Circular Clustering in Fuzzy Approximation Spaces for Color Normalization of Histological Images

One of the foremost and challenging tasks in hematoxylin and eosin stained histological image analysis is to reduce color variation present among images, which may significantly affect the performance of computer-aided histological image analysis. In this regard, the paper introduces a new rough-fuzzy circular clustering algorithm for stain color normalization. It judiciously integrates the merits of both fuzzy and rough sets. While the theory of rough sets deals with uncertainty, vagueness, and incompleteness in stain class definition, fuzzy set handles the overlapping nature of histochemical stains. The proposed circular clustering algorithm works on a weighted hue histogram, which considers both saturation and local neighborhood information of the given image. A new dissimilarity measure is introduced to deal with the circular nature of hue values. Some new quantitative measures are also proposed to evaluate the color constancy after normalization. The performance of the proposed method, along with a comparison with other state-of-the-art methods, is demonstrated on several publicly available standard data sets consisting of hematoxylin and eosin stained histological images.

A new era: artificial intelligence and machine learning in prostate cancer

Artificial intelligence (AI) - the ability of a machine to perform cognitive tasks to achieve a particular goal based on provided data - is revolutionizing and reshaping our health-care systems. The current availability of ever-increasing computational power, highly developed pattern recognition algorithms and advanced image processing software working at very high speeds has led to the emergence of computer-based systems that are trained to perform complex tasks in bioinformatics, medical imaging and medical robotics. Accessibility to 'big data' enables the 'cognitive' computer to scan billions of bits of unstructured information, extract the relevant information and recognize complex patterns with increasing confidence. Computer-based decision-support systems based on machine learning (ML) have the potential to revolutionize medicine by performing complex tasks that are currently assigned to specialists to improve diagnostic accuracy, increase efficiency of throughputs, improve clinical workflow, decrease human resource costs and improve treatment choices. These characteristics could be especially helpful in the management of prostate cancer, with growing applications in diagnostic imaging, surgical interventions, skills training and assessment, digital pathology and genomics. Medicine must adapt to this changing world, and urologists, oncologists, radiologists and pathologists, as high-volume users of imaging and pathology, need to understand this burgeoning science and acknowledge that the development of highly accurate AI-based decision-support applications of ML will require collaboration between data scientists, computer researchers and engineers.

Generative Adversarial Networks for Facilitating Stain-Independent Supervised and Unsupervised Segmentation: A Study on Kidney Histology

A major challenge in the field of segmentation in digital pathology is given by the high effort for manual data annotations in combination with many sources introducing variability in the image domain. This requires methods that are able to cope with variability without requiring to annotate a large amount of samples for each characteristic. In this paper, we develop approaches based on adversarial models for image-to-image translation relying on unpaired training. Specifically, we propose approaches for stain-independent supervised segmentation relying on image-to-image translation for obtaining an intermediate representation. Furthermore, we develop a fully-unsupervised segmentation approach exploiting image-to-image translation to convert from the image to the label domain. Finally, both approaches are combined to obtain optimum performance in unsupervised segmentation independent of the characteristics of the underlying stain. Experiments on patches showing kidney histology proof that stain-translation can be performed highly effectively and can be used for domain adaptation to obtain independence of the underlying stain. It is even capable of facilitating the underlying segmentation task, thereby boosting the accuracy if an appropriate intermediate stain is selected. Combining domain adaptation with unsupervised segmentation finally showed the most significant improvements.

A new optical density granulometry-based descriptor for the classification of prostate histological images using shallow and deep Gaussian processes

BACKGROUND AND OBJECTIVE

Prostate cancer is one of the most common male tumors. The increasing use of whole slide digital scanners has led to an enormous interest in the application of machine learning techniques to histopathological image classification. Here we introduce a novel family of morphological descriptors which, extracted in the appropriate image space and combined with shallow and deep Gaussian process based classifiers, improves early prostate cancer diagnosis.

METHOD

We decompose the acquired RGB image in its RGB and optical density hematoxylin and eosin components. Then, we define two novel granulometry-based descriptors which work in both, RGB and optical density, spaces but perform better when used on the latter. In this space they clearly encapsulate knowledge used by pathologists to identify cancer lesions. The obtained features become the inputs to shallow and deep Gaussian process classifiers which achieve an accurate prediction of cancer.

RESULTS

We have used a real and unique dataset. The dataset is composed of 60 Whole Slide Images. For a five fold cross validation, shallow and deep Gaussian Processes obtain area under ROC curve values higher than 0.98. They outperform current state of the art patch based shallow classifiers and are very competitive to the best performing deep learning method. Models were also compared on 17 Whole Slide test Images using the FROC curve. With the cost of one false positive, the best performing method, the one layer Gaussian process, identifies 83.87% (sensitivity) of all annotated cancer in the Whole Slide Image. This result corroborates the quality of the extracted features, no more than a layer is needed to achieve excellent generalization results.

CONCLUSION

Two new descriptors to extract morphological features from histological images have been proposed. They collect very relevant information for cancer detection. From these descriptors, shallow and deep Gaussian Processes are capable of extracting the complex structure of prostate histological images. The new space/descriptor/classifier paradigm outperforms state-of-art shallow classifiers. Furthermore, despite being much simpler, it is competitive to state-of-art CNN architectures both on the proposed SICAPv1 database and on an external database.

Unsupervised Domain Adaptation for Classification of Histopathology Whole-Slide Images

Computational image analysis is one means for evaluating digitized histopathology specimens that can increase the reproducibility and reliability with which cancer diagnoses are rendered while simultaneously providing insight as to the underlying mechanisms of disease onset and progression. A major challenge that is confronted when analyzing samples that have been prepared at disparate laboratories and institutions is that the algorithms used to assess the digitized specimens often exhibit heterogeneous staining characteristics because of slight differences in incubation times and the protocols used to prepare the samples. Unfortunately, such variations can render a prediction model learned from one batch of specimens ineffective for characterizing an ensemble originating from another site. In this work, we propose to adopt unsupervised domain adaptation to effectively transfer the discriminative knowledge obtained from any given source domain to the target domain without requiring any additional labeling or annotation of images at the target site. In this paper, our team investigates the use of two approaches for performing the adaptation: (1) color normalization and (2) adversarial training. The adversarial training strategy is implemented through the use of convolutional neural networks to find an invariant feature space and Siamese architecture within the target domain to add a regularization that is appropriate for the entire set of whole-slide images. The adversarial adaptation results in significant classification improvement compared with the baseline models under a wide range of experimental settings.

Artificial intelligence at the intersection of pathology and radiology in prostate cancer

Pathologic grading plays a key role in prostate cancer risk stratification and treatment selection, traditionally assessed from systemic core needle biopsies sampled throughout the prostate gland. Multiparametric magnetic resonance imaging (mpMRI) has become a well-established clinical tool for detecting and localizing prostate cancer. However, both pathologic and radiologic assessment suffer from poor reproducibility among readers. Artificial intelligence (AI) methods show promise in aiding the detection and assessment of imaging-based tasks, dependent on the curation of high-quality training sets. This review provides an overview of recent advances in AI applied to mpMRI and digital pathology in prostate cancer which enable advanced characterization of disease through combined radiology-pathology assessment.

Path R-CNN for Prostate Cancer Diagnosis and Gleason Grading of Histological Images

Prostate cancer is the most common and second most deadly form of cancer in men in the United States. The classification of prostate cancers based on Gleason grading using histological images is important in risk assessment and treatment planning for patients. Here, we demonstrate a new region-based convolutional neural network framework for multi-task prediction using an epithelial network head and a grading network head. Compared with a single-task model, our multi-task model can provide complementary contextual information, which contributes to better performance. Our model is achieved a state-of-the-art performance in epithelial cells detection and Gleason grading tasks simultaneously. Using fivefold cross-validation, our model is achieved an epithelial cells detection accuracy of 99.07% with an average area under the curve of 0.998. As for Gleason grading, our model is obtained a mean intersection over union of 79.56% and an overall pixel accuracy of 89.40%.

Comparison of Artificial Intelligence Techniques to Evaluate Performance of a Classifier for Automatic Grading of Prostate Cancer From Digitized Histopathologic Images

IMPORTANCE

Proper evaluation of the performance of artificial intelligence techniques in the analysis of digitized medical images is paramount for the adoption of such techniques by the medical community and regulatory agencies.

OBJECTIVES

To compare several cross-validation (CV) approaches to evaluate the performance of a classifier for automatic grading of prostate cancer in digitized histopathologic images and compare the performance of the classifier when trained using data from 1 expert and multiple experts.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study used tissue microarray data (333 cores) from 231 patients who underwent radical prostatectomy at the Vancouver General Hospital between June 27, 1997, and June 7, 2011. Digitized images of tissue cores were annotated by 6 pathologists for 4 classes (benign and Gleason grades 3, 4, and 5) between December 12, 2016, and October 5, 2017. Patches of 192 µm2 were extracted from these images. There was no overlap between patches. A deep learning classifier based on convolutional neural networks was trained to predict a class label from among the 4 classes (benign and Gleason grades 3, 4, and 5) for each image patch. The classification performance was evaluated in leave-patches-out CV, leave-cores-out CV, and leave-patients-out 20-fold CV. The analysis was performed between November 15, 2018, and January 1, 2019.

MAIN OUTCOMES AND MEASURES

The classifier performance was evaluated by its accuracy, sensitivity, and specificity in detection of cancer (benign vs cancer) and in low-grade vs high-grade differentiation (Gleason grade 3 vs grades 4-5). The statistical significance analysis was performed using the McNemar test. The agreement level between pathologists and the classifier was quantified using a quadratic-weighted κ statistic.

RESULTS

On 333 tissue microarray cores from 231 participants with prostate cancer (mean [SD] age, 63.2 [6.3] years), 20-fold leave-patches-out CV resulted in mean (SD) accuracy of 97.8% (1.2%), sensitivity of 98.5% (1.0%), and specificity of 97.5% (1.2%) for classifying benign patches vs cancerous patches. By contrast, 20-fold leave-patients-out CV resulted in mean (SD) accuracy of 85.8% (4.3%), sensitivity of 86.3% (4.1%), and specificity of 85.5% (7.2%). Similarly, 20-fold leave-cores-out CV resulted in mean (SD) accuracy of 86.7% (3.7%), sensitivity of 87.2% (4.0%), and specificity of 87.7% (5.5%). Results of McNemar tests showed that the leave-patches-out CV accuracy, sensitivity, and specificity were significantly higher than those for both leave-patients-out CV and leave-cores-out CV. Similar results were observed for classifying low-grade cancer vs high-grade cancer. When trained on a single expert, the overall agreement in grading between pathologists and the classifier ranged from 0.38 to 0.58; when trained using the majority vote among all experts, it was 0.60.

CONCLUSIONS AND RELEVANCE

Results of this study suggest that in prostate cancer classification from histopathologic images, patch-wise CV and single-expert training and evaluation may lead to a biased estimation of classifier's performance. To allow reproducibility and facilitate comparison between automatic classification methods, studies in the field should evaluate their performance using patient-based CV and multiexpert data. Some of these conclusions may be generalizable to other histopathologic applications and to other applications of machine learning in medicine.

An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies

Automated Gleason grading is an important preliminary step for quantitative histopathological feature extraction. Different from the traditional task of classifying small pre-selected homogeneous regions, semantic segmentation provides pixel-wise Gleason predictions across an entire slide. Deep learning-based segmentation models can automatically learn visual semantics from data, which alleviates the need for feature engineering. However, performance of deep learning models is limited by the scarcity of large-scale fully annotated datasets, which can be both expensive and time-consuming to create. One way to address this problem is to leverage external weakly labeled datasets to augment models trained on the limited data. In this paper, we developed an expectation maximization-based approach constrained by an approximated prior distribution in order to extract useful representations from a large number of weakly labeled images generated from low-magnification annotations. This method was utilized to improve the performance of a model trained on a limited fully annotated dataset. Our semi-supervised approach trained with 135 fully annotated and 1800 weakly annotated tiles achieved a mean Jaccard Index of 49.5% on an independent test set, which was 14% higher than the initial model trained only on the fully annotated dataset.

Characterization of prostate microstructure using water diffusion and NMR relaxation

For many pathologies, early structural tissue changes occur at the cellular level, on the scale of micrometers or tens of micrometers. Magnetic resonance imaging (MRI) is a powerful non-invasive imaging tool used for medical diagnosis, but its clinical hardware is incapable of reaching the cellular length scale directly. In spite of this limitation, microscopic tissue changes in pathology can potentially be captured indirectly, from macroscopic imaging characteristics, by studying water diffusion. Here we focus on water diffusion and NMR relaxation in the human prostate, a highly heterogeneous organ at the cellular level. We present a physical picture of water diffusion and NMR relaxation in the prostate tissue, that is comprised of a densely-packed cellular compartment (composed of stroma and epithelium), and a luminal compartment with almost unrestricted water diffusion. Transverse NMR relaxation is used to identify fast and slow T ₂ components, corresponding to these tissue compartments, and to disentangle the luminal and cellular compartment contributions to the temporal evolution of the overall water diffusion coefficient. Diffusion in the luminal compartment falls into the short-time surface-to-volume (S/V) limit, indicating that only a small fraction of water molecules has time to encounter the luminal walls of healthy tissue; from the S/V ratio, the average lumen diameter averaged over three young healthy subjects is measured to be 217.7±188.7 μm. Conversely, the diffusion in the cellular compartment is highly restricted and anisotropic, consistent with the fibrous character of the stromal tissue. Diffusion transverse to these fibers is well described by the random permeable barrier model (RPBM), as confirmed by the dynamical exponent ϑ = 1/2 for approaching the long-time limit of diffusion, and the corresponding structural exponent p = -1 in histology. The RPBM-derived fiber diameter and membrane permeability were 19.8±8.1 μm and 0.044±0.045 μm/ms, respectively, in agreement with known values from tissue histology and membrane biophysics. Lastly, we revisited 38 prostate cancer cases from a recently published study, and found the same dynamical exponent ϑ = 1/2 of diffusion in tumors and benign regions. Our results suggest that a multi-parametric MRI acquisition combined with biophysical modeling may be a powerful non-invasive complement to prostate cancer grading, potentially foregoing biopsies.

Absolute cosine-based SVM-RFE feature selection method for prostate histopathological grading

OBJECTIVE

Feature selection (FS) methods are widely used in grading and diagnosing prostate histopathological images. In this context, FS is based on the texture features obtained from the lumen, nuclei, cytoplasm and stroma, all of which are important tissue components. However, it is difficult to represent the high-dimensional textures of these tissue components. To solve this problem, we propose a new FS method that enables the selection of features with minimal redundancy in the tissue components.

METHODOLOGY

We categorise tissue images based on the texture of individual tissue components via the construction of a single classifier and also construct an ensemble learning model by merging the values obtained by each classifier. Another issue that arises is overfitting due to the high-dimensional texture of individual tissue components. We propose a new FS method, SVM-RFE(AC), that integrates a Support Vector Machine-Recursive Feature Elimination (SVM-RFE) embedded procedure with an absolute cosine (AC) filter method to prevent redundancy in the selected features of the SV-RFE and an unoptimised classifier in the AC.

RESULTS

We conducted experiments on H&E histopathological prostate and colon cancer images with respect to three prostate classifications, namely benign vs. grade 3, benign vs. grade 4 and grade 3 vs. grade 4. The colon benchmark dataset requires a distinction between grades 1 and 2, which are the most difficult cases to distinguish in the colon domain. The results obtained by both the single and ensemble classification models (which uses the product rule as its merging method) confirm that the proposed SVM-RFE(AC) is superior to the other SVM and SVM-RFE-based methods.

CONCLUSION

We developed an FS method based on SVM-RFE and AC and successfully showed that its use enabled the identification of the most crucial texture feature of each tissue component. Thus, it makes possible the distinction between multiple Gleason grades (e.g. grade 3 vs. grade 4) and its performance is far superior to other reported FS methods.

Differentiation among prostate cancer patients with Gleason score of 7 using histopathology whole-slide image and genomic data

Prostate cancer is the most common non-skin related cancer affecting 1 in 7 men in the United States. Treatment of patients with prostate cancer still remains a difficult decision-making process that requires physicians to balance clinical benefits, life expectancy, comorbidities, and treatment-related side effects. Gleason score (a sum of the primary and secondary Gleason patterns) solely based on morphological prostate glandular architecture has shown as one of the best predictors of prostate cancer outcome. Significant progress has been made on molecular subtyping prostate cancer delineated through the increasing use of gene sequencing. Prostate cancer patients with Gleason score of 7 show heterogeneity in recurrence and survival outcomes. Therefore, we propose to assess the correlation between histopathology images and genomic data with disease recurrence in prostate tumors with a Gleason 7 score to identify prognostic markers. In the study, we identify image biomarkers within tissue WSIs by modeling the spatial relationship from automatically created patches as a sequence within WSI by adopting a recurrence network model, namely long short-term memory (LSTM). Our preliminary results demonstrate that integrating image biomarkers from CNN with LSTM and genomic pathway scores, is more strongly correlated with patients recurrence of disease compared to standard clinical markers and engineered image texture features. The study further demonstrates that prostate cancer patients with Gleason score of 4+3 have a higher risk of disease progression and recurrence compared to prostate cancer patients with Gleason score of 3+4.

An improved model for prostate diffusion incorporating the results of Monte Carlo simulations of diffusion in the cellular compartment

The purpose of this work was to refine a previously published model of prostate diffusion by incorporating improved estimates of cellular diffusivity obtained by Monte Carlo simulation. Stromal and epithelial cell size and intracellular volume fraction in different grades of cancer were determined from histological images. Diffusion in different mixtures of cells, corresponding to different tumor grades, was simulated and cellular apparent diffusion coefficient and kurtosis values determined. These values were incorporated into the previously published model of prostate diffusion and model predictions compared with values found in the literature. Stromal cell radius and intracellular volume fraction were 3.74 ± 0.96 μm and 13 ± 3% respectively in normal peripheral zone (PZ), and were similar in all grades of cancer. Epithelial cell radius and intracellular volume fraction were 3.40 ± 0.15 μm and 45 ± 5% respectively in normal PZ, rising to 4.75 ± 0.20 μm and 70 ± 8% in high grade cancer. Cellular apparent diffusion coefficient and kurtosis were 1.02 μm² ms^-1 and 0.58 respectively in normal PZ, and 0.61 μm² ms^-1 and 1.15 in high grade cancer (variation in simulation values are less than 0.1%). Agreement between model predictions and measurements were good, with a mean square error of 0.22 μm² ms^-1 . Incorporation of cellular diffusion coefficient and kurtosis values obtained by Monte Carlo simulation into a model of prostate diffusion gives good agreement with published results.

Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features

BACKGROUND

Histopathology image analysis is a gold standard for cancer recognition and diagnosis. Automatic analysis of histopathology images can help pathologists diagnose tumor and cancer subtypes, alleviating the workload of pathologists. There are two basic types of tasks in digital histopathology image analysis: image classification and image segmentation. Typical problems with histopathology images that hamper automatic analysis include complex clinical representations, limited quantities of training images in a dataset, and the extremely large size of singular images (usually up to gigapixels). The property of extremely large size for a single image also makes a histopathology image dataset be considered large-scale, even if the number of images in the dataset is limited.

RESULTS

In this paper, we propose leveraging deep convolutional neural network (CNN) activation features to perform classification, segmentation and visualization in large-scale tissue histopathology images. Our framework transfers features extracted from CNNs trained by a large natural image database, ImageNet, to histopathology images. We also explore the characteristics of CNN features by visualizing the response of individual neuron components in the last hidden layer. Some of these characteristics reveal biological insights that have been verified by pathologists. According to our experiments, the framework proposed has shown state-of-the-art performance on a brain tumor dataset from the MICCAI 2014 Brain Tumor Digital Pathology Challenge and a colon cancer histopathology image dataset.

CONCLUSIONS

The framework proposed is a simple, efficient and effective system for histopathology image automatic analysis. We successfully transfer ImageNet knowledge as deep convolutional activation features to the classification and segmentation of histopathology images with little training data. CNN features are significantly more powerful than expert-designed features.

AIIMDs: An Integrated Framework of Automatic Idiopathic Inflammatory Myopathy Diagnosis for Muscle

Idiopathic inflammatory myopathy (IIM) is a common skeletal muscle disease that relates to weakness and inflammation of muscle. Early diagnosis and prognosis of different types of IIMs will guide the effective treatment. Interpretation of digitized images of the cross-section muscle biopsy, which is currently done manually, provides the most reliable diagnostic information. With the increasing volume of images, the management and manual interpretation of the digitized muscle images suffer from low efficiency and high interobserver variabilities. In order to address these problems, we propose the first complete framework of automatic IIM diagnosis system for the management and interpretation of digitized skeletal muscle histopathology images. The proposed framework consists of several key components: (1) Automatic cell segmentation, perimysium annotation, and nuclei detection; (2) histogram-based feature extraction and quantification; (3) content-based image retrieval to search and retrieve similar cases in the database for comparative study; and (4) majority voting-based classification to provide decision support for computer-aided clinical diagnosis. Experiments show that the proposed diagnosis system provides efficient and robust interpretation of the digitized muscle image and computer-aided diagnosis of IIM.

Multiview boosting digital pathology analysis of prostate cancer

BACKGROUND AND OBJECTIVE

Various digital pathology tools have been developed to aid in analyzing tissues and improving cancer pathology. The multi-resolution nature of cancer pathology, however, has not been fully analyzed and utilized. Here, we develop an automated, cooperative, and multi-resolution method for improving prostate cancer diagnosis.

METHODS

Digitized tissue specimen images are obtained from 5 tissue microarrays (TMAs). The TMAs include 70 benign and 135 cancer samples (TMA1), 74 benign and 89 cancer samples (TMA2), 70 benign and 115 cancer samples (TMA3), 79 benign and 82 cancer samples (TMA4), and 72 benign and 86 cancer samples (TMA5). The tissue specimen images are segmented using intensity- and texture-based features. Using the segmentation results, a number of morphological features from lumens and epithelial nuclei are computed to characterize tissues at different resolutions. Applying a multiview boosting algorithm, tissue characteristics, obtained from differing resolutions, are cooperatively combined to achieve accurate cancer detection.

RESULTS

In segmenting prostate tissues, the multiview boosting method achieved≥ 0.97 AUC using TMA1. For detecting cancers, the multiview boosting method achieved an AUC of 0.98 (95% CI: 0.97-0.99) as trained on TMA2 and tested on TMA3, TMA4, and TMA5. The proposed method was superior to single-view approaches, utilizing features from a single resolution or merging features from all the resolutions. Moreover, the performance of the proposed method was insensitive to the choice of the training dataset. Trained on TMA3, TMA4, and TMA5, the proposed method obtained an AUC of 0.97 (95% CI: 0.96-0.98), 0.98 (95% CI: 0.96-0.99), and 0.97 (95% CI: 0.96-0.98), respectively.

CONCLUSIONS

The multiview boosting method is capable of integrating information from multiple resolutions in an effective and efficient fashion and identifying cancers with high accuracy. The multiview boosting method holds a great potential for improving digital pathology tools and research.

Automatic extraction of cell nuclei from H&E-stained histopathological images

Extraction of cell nuclei from hematoxylin and eosin (H&E)-stained histopathological images is an essential preprocessing step in computerized image analysis for disease detection, diagnosis, and prognosis. We present an automated cell nuclei segmentation approach that works with H&E-stained images. A color deconvolution algorithm was first applied to the image to get the hematoxylin channel. Using a morphological operation and thresholding technique on the hematoxylin channel image, candidate target nuclei and background regions were detected, which were then used as markers for a marker-controlled watershed transform segmentation algorithm. Moreover, postprocessing was conducted to split the touching nuclei. For each segmented region from the previous steps, the regional maximum value positions were identified as potential nuclei centers. These maximum values were further grouped into [Formula: see text]-clusters, and the locations within each cluster were connected with the minimum spanning tree technique. Then, these connected positions were utilized as new markers for a watershed segmentation approach. The final number of nuclei at each region was determined by minimizing an objective function that iterated all of the possible [Formula: see text]-values. The proposed method was applied to the pathological images of the tumor tissues from The Cancer Genome Atlas study. Experimental results show that the proposed method can lead to promising results in terms of segmentation accuracy and separation of touching nuclei.