An optimized image analysis algorithm for detecting nuclear signals in digital whole slides for histopathology

Image Analysis Using the Fluorescence Imaging of Nuclear Staining (FINS) Algorithm

Finding appropriate image analysis techniques for a particular purpose can be difficult. In the context of the analysis of immunocytochemistry images, where the key information lies in the number of nuclei containing co-localised fluorescent signals from a marker of interest, researchers often opt to use manual counting techniques because of the paucity of available tools. Here, we present the development and validation of the Fluorescence Imaging of Nuclear Staining (FINS) algorithm for the quantification of fluorescent signals from immunocytochemically stained cells. The FINS algorithm is based on a variational segmentation of the nuclear stain channel and an iterative thresholding procedure to count co-localised fluorescent signals from nuclear proteins in other channels. We present experimental results comparing the FINS algorithm to the manual counts of seven researchers across a dataset of three human primary cell types which are immunocytochemically stained for a nuclear marker (DAPI), a biomarker of cellular proliferation (Ki67), and a biomarker of DNA damage (γH2AX). The quantitative performance of the algorithm is analysed in terms of consistency with the manual count data and acquisition time. The FINS algorithm produces data consistent with that achieved by manual counting but improves the process by reducing subjectivity and time. The algorithm is simple to use, based on software that is omnipresent in academia, and allows data review with its simple, intuitive user interface. We hope that, as the FINS tool is open-source and is custom-built for this specific application, it will streamline the analysis of immunocytochemical images.

Comparison of TERT and 5-Hydroxymethylcytocine immunohistochemistry in various thyroid carcinomas

Telomerase reverse transcriptase (TERT) promoter mutation is associated with an aggressive clinical course in thyroid carcinomas. Therefore, detection of TERT promoter mutation is essential for proper patient management. 5-Hydroxymethylcytosine (5hmC) is an epigenetic marker involved in the DNA demethylation pathway, and its loss has been observed in various tumors. Loss of 5hmC has also been reported in thyroid carcinomas and is presented as a possible predictive biomarker for TERT promoter mutation and worse prognosis. This study evaluated the expression of TERT and 5hmC by immunohistochemistry (IHC) in 105 patients (44 in the TERT mutant group and 61 in the TERT wild group) with various thyroid carcinomas. H-scores were calculated using an image analyzer. The median H-scores of TERT IHC were significantly higher in the TERT mutant group than in the TERT wild group (47.15 vs. 9.80). The sensitivity and specificity of TERT IHC for predicting TERT promoter mutations were 65.9 and 65.7 %, respectively. Regardless of TERT promoter mutation status, the 5hmC H-scores were markedly lower in all subtypes of thyroid carcinomas compared to those in their normal counterparts. Significant differences in 5hmC H-scores were observed between N0 and N1 in total thyroid carcinomas, but not within the papillary thyroid carcinoma subgroup. In conclusion, TERT and 5hmC IHC have limitations in predicting the presence of TERT promoter mutations. The expression of 5hmC was downregulated in various thyroid carcinomas compared to that in normal and benign lesions, but comprehensive further studies are required to elucidate the role of 5hmC in thyroid carcinomas.

Myelofibrosis progression grading based on type I and type III collagen and fibrillin 1 expression boosted by whole slide image analysis

AIMS

The progression of primary myelofibrosis is characterised by ongoing extracellular matrix deposition graded based on 'reticulin' and 'collagen' fibrosis, as revealed by Gomori's silver impregnation. Here we studied the expression of the major extracellular matrix proteins of fibrosis in relation to diagnostic silver grading supported by image analysis.

METHODS AND RESULTS

By using automated immunohistochemistry, in this study we demonstrate that the expression of both types I and III collagens and fibrillin 1 by bone marrow stromal cells can reveal the extracellular matrix scaffolding in line with myelofibrosis progression as classified by silver grading. 'Reticulin' fibrosis indicated by type III collagen expression and 'collagen' fibrosis featured by type I collagen expression were parallel, rather than sequential, events. This is line with the proposed role of type III collagen in regulating type I collagen fibrillogenesis. The uniformly strong fibrillin 1 immune signals offered the best inter-rater agreements and the highest statistical correlations with silver grading of the three markers, which was robustly confirmed by automated whole slide digital image analysis using a machine learning-based algorithm. The progressive up-regulation of fibrillin 1 during myelofibrosis may result from a negative feedback loop as fibrillin microfibrils sequester TGF-β, the major promoter of fibrosis. This can also reduce TGF-β-induced RANKL levels, which would stimulate osteoclastogenesis and thus can support osteosclerosis in advanced myelofibrosis.

CONCLUSIONS

Through the in-situ detection of these extracellular matrix proteins, our results verify the molecular pathobiology of fibrosis during myelofibrosis progression. In particular, fibrillin 1 immunohistochemistry, with or without image analysis, can complement diagnostic silver grading at decent cell morphology.

Prognostic value and computer image analysis of p53 in mantle cell lymphoma

P53 prognostic cut-off values differ between studies of mantle cell lymphoma (MCL), and its immunohistochemistry (IHC) interpretation is still based on semiquantitative estimation, which might be inaccurate. This study aimed to investigate the optimal cut-off value for p53 in predicting prognosis of patients with MCL and the possible use of computer image analysis to identify the positive rate of p53. We calculated p53 positive rate using QuPath software and compared it with the data obtained by manual counting and semiquantitative estimation. Survival curves were generated by using the Youden index and the Kaplan-Meier method. The chi-squared (χ2) test was used to compare MIPI, Ann Arbor stage, and cell morphology with p53. Spearman rank correlation test and Bland-Altman analysis were used to compare manual counting, computer image analysis and semiquantitative estimation, as well as the consistency between different observers. The optimal cut-off value of p53 for predicting prognosis was 20% in MCL patients. Patients with p53 ≥ 20% had a significantly worse overall survival (OS) than those with p53 < 20% (P < 0.0001). MCL patients with MIPI intermediate to high risk, Ann Arbor stage III-IV, and blastoid/pleomorphic variant cell morphology had more p53 ≥ 20%. There was a strong correlation between computer image analysis and manual counting of p53 from the same areas in MCL tissues (Spearman's rho = 0.966, P < 0.0001). The results of computer analysis are completely consistent between observers, and computer image analysis of Ki-67 can predict the prognosis of MCL patients. MCL patients with p53 ≥ 20% had a shorter OS and a tendency for MIPI intermediate to high risk, Ann Arbor stage III-IV, and blastoid/pleomorphic variant. Computer image analysis could determine the actual positive rate of p53 and Ki-67 and is a more attractive alternative than semiquantitative estimation in MCL.

How the variability between computer-assisted analysis procedures evaluating immune markers can influence patients' outcome prediction

Differences between computer-assisted image analysis (CAI) algorithms may cause discrepancies in the identification of immunohistochemically stained immune biomarkers in biopsies of breast cancer patients. These discrepancies have implications for their association with disease outcome. This study aims to compare three CAI procedures (A, B and C) to measure positive marker areas in post-neoadjuvant chemotherapy biopsies of patients with triple-negative breast cancer (TNBC) and to explore the differences in their performance in determining the potential association with relapse in these patients. A total of 3304 digital images of biopsy tissue obtained from 118 TNBC patients were stained for seven immune markers using immunohistochemistry (CD4, CD8, FOXP3, CD21, CD1a, CD83, HLA-DR) and were analyzed with procedures A, B and C. The three methods measure the positive pixel markers in the total tissue areas. The extent of agreement between paired CAI procedures, a principal component analysis (PCA) and Cox multivariate analysis was assessed. Comparisons of paired procedures showed close agreement for most of the immune markers at low concentration. The probability of differences between the paired procedures B/C and B/A was generally higher than those observed in C/A. The principal component analysis, largely based on data from CD8, CD1a and HLA-DR, identified two groups of patients with a significantly lower probability of relapse than the others. The multivariate regression models showed similarities in the factors associated with relapse for procedures A and C, as opposed to those obtained with procedure B. General agreement among the results of CAI procedures would not guarantee that the same predictive breast cancer markers were consistently identified. These results highlight the importance of developing additional strategies to improve the sensitivity of CAI procedures.

Image-Based Method to Quantify Decellularization of Tissue Sections

Tissue decellularization is typically assessed through absorbance-based DNA quantification after tissue digestion. This method has several disadvantages, namely its destructive nature and inadequacy in experimental situations where tissue is scarce. Here, we present an image processing algorithm for quantitative analysis of DNA content in (de)cellularized tissues as a faster, simpler and more comprehensive alternative. Our method uses local entropy measurements of a phase contrast image to create a mask, which is then applied to corresponding nuclei labelled (UV) images to extract average fluorescence intensities as an estimate of DNA content. The method can be used on native or decellularized tissue to quantify DNA content, thus allowing quantitative assessment of decellularization procedures. We confirm that our new method yields results in line with those obtained using the standard DNA quantification method and that it is successful for both lung and heart tissues. We are also able to accurately obtain a timeline of decreasing DNA content with increased incubation time with a decellularizing agent. Finally, the identified masks can also be applied to additional fluorescence images of immunostained proteins such as collagen or elastin, thus allowing further image-based tissue characterization.

Detection of Ki67 Hot-Spots of Invasive Breast Cancer Based on Convolutional Neural Networks Applied to Mutual Information of H&E and Ki67 Whole Slide Images

Ki67 hot-spot detection and its evaluation in invasive breast cancer regions play a significant role in routine medical practice. The quantification of cellular proliferation assessed by Ki67 immunohistochemistry is an established prognostic and predictive biomarker that determines the choice of therapeutic protocols. In this paper, we present three deep learning-based approaches to automatically detect and quantify Ki67 hot-spot areas by means of the Ki67 labeling index. To this end, a dataset composed of 100 whole slide images (WSIs) belonging to 50 breast cancer cases (Ki67 and H&E WSI pairs) was used. Three methods based on CNN classification were proposed and compared to create the tumor proliferation map. The best results were obtained by applying the CNN to the mutual information acquired from the color deconvolution of both the Ki67 marker and the H&E WSIs. The overall accuracy of this approach was 95%. The agreement between the automatic Ki67 scoring and the manual analysis is promising with a Spearman’s ρ correlation of 0.92. The results illustrate the suitability of this CNN-based approach for detecting hot-spots areas of invasive breast cancer in WSI.

Automatic Human Dendritic Cells Segmentation Using K-Means Clustering and Chan-Vese Active Contour Model

BACKGROUND AND OBJECTIVE

Nowadays, the number of pathologies related to food are multiplied. Mycotoxins are one of the most severe food contaminants that cause serious effects on the human health. Therefore, it is necessary to develop an assessment tool for evaluating their impact on the immune response. Recently, a new investigational method using human dendritic cells was endorsed by biologists. Nevertheless, analysis of the morphological features and the behavior of these cells remains merely visual. In addition, this manual analysis is difficult and time-consuming. Here, we focus mainly on automating the evaluation process by using advanced image processing technology.

METHODS

An automatic segmentation approach of microscopic dendritic cell images is developed to provide a fast and objective evaluation. First, a combination of K-means clustering and mathematical morphology is used to detect dendritic cells. Second, a region-based Chan-Vese active contour model is used to segment the detected cells more precisely. Finally, dendritic cells are extracted by a filtering based on eccentricity measure.

RESULTS

The proposed scheme is tested on an actual dataset containing 421 microscopic dendritic cell images. The experimental results show high conformity between the results of the proposed scheme and ground-truth elaborated by biological expert. Moreover, a comparative study with other state-of-art segmentation schemes demonstrates the efficiency of the proposed method. It gives the highest average accuracy rate (99.42 %) compared to recent studied approaches.

CONCLUSIONS

The proposed image segmentation method for morphological analysis of dendrite inhibition can consistently be used as an assessment tool for biologists to facilitate the evaluation of serious health impacts of mycotoxins.

Artificial intelligence as the next step towards precision pathology

Pathology is the cornerstone of cancer care. The need for accuracy in histopathologic diagnosis of cancer is increasing as personalized cancer therapy requires accurate biomarker assessment. The appearance of digital image analysis holds promise to improve both the volume and precision of histomorphological evaluation. Recently, machine learning, and particularly deep learning, has enabled rapid advances in computational pathology. The integration of machine learning into routine care will be a milestone for the healthcare sector in the next decade, and histopathology is right at the centre of this revolution. Examples of potential high-value machine learning applications include both model-based assessment of routine diagnostic features in pathology, and the ability to extract and identify novel features that provide insights into a disease. Recent groundbreaking results have demonstrated that applications of machine learning methods in pathology significantly improves metastases detection in lymph nodes, Ki67 scoring in breast cancer, Gleason grading in prostate cancer and tumour-infiltrating lymphocyte (TIL) scoring in melanoma. Furthermore, deep learning models have also been demonstrated to be able to predict status of some molecular markers in lung, prostate, gastric and colorectal cancer based on standard HE slides. Moreover, prognostic (survival outcomes) deep neural network models based on digitized HE slides have been demonstrated in several diseases, including lung cancer, melanoma and glioma. In this review, we aim to present and summarize the latest developments in digital image analysis and in the application of artificial intelligence in diagnostic pathology.

Diagnosis of Liver Neoplasms by Computational and Statistical Image Analysis

Background

Distinguishing well-differentiated hepatocellular carcinoma (WD-HCC), hepatocellular adenoma (HA) and non-neoplastic liver tissue (NNLT) solely on morphology is often challenging. The purpose of this study was to evaluate the use of computational image analysis to distinguish WD-HCC, HA and NNLT.

Methods

Seventy-seven cases comprising of WD-HCC (n = 26), HA (n = 23) and NNLT (n = 28) were retrieved and reviewed. A total of 485 hematoxylin and eosin (H&E) photomicrographs (× 400, 0.09 µm²) of WD-HCC (n = 183), HA (n = 173), NNLT (n = 129) and nine whole-slide scans (three of each diagnosis) were obtained, color deconvoluted and digitally transformed. Quantitative data including nuclear density, nuclear sphericity, nuclear perimeter, and nuclear eccentricity from each image were acquired. The data were analyzed by one-way analysis of variance (ANOVA) with Tukey post hoc test, followed by unsupervised and supervised (Chi-square automatic interaction detection (CHAID)) cluster analysis.

Results

Unsupervised cluster analysis identified three well defined clusters of WD-HCC, HA and NNLT. Employing the four most discriminating nuclear features, supervised analysis was performed on a training set of 383 images, and validated on the remaining 102 test images. The analysis identified WD-HCC (sensitivity 100%, specificity 98%), HA (sensitivity 71%, specificity 85%) and NNLT (sensitivity 70%, specificity 86%). An analysis of whole-slide images identified WD-HCC with sensitivity and specificity of 100%.

Conclusions

We have successfully demonstrated that computational image analysis of nuclear features can differentiate WD-HCC from non-malignant liver with high accuracy, and can be used to assist in the histopathological diagnosis of hepatocellular carcinoma.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Comparison between two programs for image analysis, machine learning and subsequent classification

In the early 1950s, flow cytometry was developed as the first method for automated quantitative cellular analysis. In the early 1990s, the first equipment for image cytometry (laser scanning cytometry, LSC) became commercially available. As flow cytometry was considered the gold standard, various studies found that the results of flow cytometry and LSC generated comparable results. One of the first programs for image analysis that included morphological parameters was ImageJ, published in 1997. One of the newer programs for image analysis that is not limited to fluorescence images is the free software CellProfiler. In 2008, the same group published a new software, CellProfiler Analyst. One part of CellProfiler Analyst is a supervised machine-learning-based classifier that allows users to conduct imaging-based diagnoses, e.g., cellular diagnosis based on morphology. Another relatively new, free software for image analysis is QuPath. The aim of the present study was to compare two free programs for conducting image analysis, CellProfiler and QuPath, and the subsequent classification based on machine learning. For this study, images of renal tissue were analyzed, and the identified objects were classified. The same images were loaded in both software programs. Advanced statistical analysis was used to compare the two methods. The Bland-Altman assay showed that all of the differences were within the mean ± 1.96 * standard deviation, i.e., the differences are normally distributed, and the software programs are comparable. For the analyzed samples (renal tissue stained with HIF and TUNEL), the use of QuPath was easier because it offers image analysis without a previous processing of the images (e.g., conversion to grayscale, inverted intensities) and an unsupervised machine learning process.

Ki67 reproducibility using digital image analysis: an inter-platform and inter-operator study

Ki67 expression has been a valuable prognostic variable in breast cancer, but has not seen broad adoption due to lack of standardization between institutions. Automation could represent a solution. Here we investigate the reproducibility of Ki67 measurement between three image analysis platforms with supervised classifiers performed by the same operator, by multiple operators, and finally we compare their accuracy in prognostic potential. Two breast cancer patient cohorts were used for this study. The standardization was done with the 30 cases of ER+ breast cancer that were used in phase 3 of International Ki67 in Breast Cancer Working Group initiatives where blocks were centrally cut and stained for Ki67. The outcome cohort was from 149 breast cancer cases from the Yale Pathology archives. A tissue microarray was built from representative tissue blocks with median follow-up of 120 months. The Mib-1 antibody (Dako) was used to detect Ki67 (dilution 1:100). HALO (IndicaLab), QuantCenter (3DHistech), and QuPath (open source software) digital image analysis (DIA) platforms were used to evaluate Ki67 expression. Intraclass correlation coefficient (ICC) was used to measure reproducibility. Between-DIA platform reproducibility was excellent (ICC: 0.933, CI: 0.879-0.966). Excellent reproducibility was found between all DIA platforms and the reference standard Ki67 values of Spectrum Webscope (QuPath-Spectrum Webscope ICC: 0.970, CI: 0.936-0.986; HALO-Spectrum Webscope ICC: 0.968, CI: 0.933-0.985; QuantCenter-Spectrum Webscope ICC: 0.964, CI: 0.919-0.983). All platforms showed excellent intra-DIA reproducibility (QuPath ICC: 0.992, CI: 0.986-0.996; HALO ICC: 0.972, CI: 0.924-0.988; QuantCenter ICC: 0.978, CI: 0.932-0.991). Comparing each DIA against outcome, the hazard ratios were similar. The inter-operator reproducibility was particularly high (ICC: 0.962-0.995). Our results showed outstanding reproducibility both within and between-DIA platforms, including one freely available DIA platform (QuPath). We also found the platforms essentially indistinguishable with respect to prediction of breast cancer patient outcome. Results justify multi-institutional DIA studies to assess clinical utility.

Laboratory Computer Performance in a Digital Pathology Environment: Outcomes from a Single Institution

Background:

In an effort to provide improved user experience and system reliability at a moderate cost, our department embarked on targeted upgrades of a total of 87 computers over a period of 3 years. Upgrades came in three forms: (i) replacement of the computer with newer architecture, (ii) replacement of the computer's hard drive with a solid-state drive (SSD), or (iii) replacement of the computer with newer architecture and a SSD.

Methods:

We measured the impact of each form of upgrade on a set of pathology-relevant tasks that fell into three categories: standard use, whole-slide navigation, and whole-slide analysis. We used time to completion of a task as the primary variable of interest.

Results:

We found that for most tasks, the SSD upgrade had a greater impact than the upgrade in architecture. This effect was especially prominent for whole-slide viewing, likely due to the way in which most whole-slide viewers cached image tiles. However, other tasks, such as whole-slide image analysis, often relied less on disk input or output and were instead more sensitive to the computer architecture.

Conclusions:

Based on our experience, we suggest that SSD upgrades are viewed in some settings as a viable alternative to complete computer replacement and recommend that computer replacements in a digital pathology setting are accompanied by an upgrade to SSDs.

Pathological Features of Staphylococcus aureus Induced Mastitis in Dairy Cows and Isobaric-Tags-for-Relative-and-Absolute-Quantitation Proteomic Analyses

In part as a result of the production of an enterotoxin, Staphylococcus aureus is a highly infectious pathogen and is a considerable threat to food hygiene and safety. Clinical mastitis models were established by S. aureus nipple-tube perfusion. The influence of mastitis on the mammary-gland-tissue proteomic profile was investigated using isobaric tags for relative and absolute quantitation. In this study, healthy and mastitic tissues from different mammary-gland areas of the same dairy cows were screened, and differentially expressed proteins were identified. Bioinformatic analysis identified proteins related to the inflammation and immunization of dairy cows. Histology, immunoblotting, and immunohistochemical-staining analyses were used to determine the expression of PGLYRP1 and PTX3 proteins in the acquired mammary-gland-tissue samples. PGLYRP1 and PTX3 in mastitic mammary glands may be associated with tissue damage and immune responses to late stages of infection. This further contributes to the understanding of the molecular theory of the treatment of mastitis caused by S. aureus.

Protocol for HER2 FISH Using a Non-cross-linking, Formalin-free Tissue Fixative to Combine Advantages of Cryo-preservation and Formalin Fixation

Morphologic assessment of formalin-fixed, paraffin-embedded (FFPE) tissue samples has been the gold standard for cancer diagnostics for decades due to its excellent preservation of morphology. Personalized medicine increasingly provides individually adapted and targeted therapies for characterized individual diseases enabled by combined morphological and molecular analytical technologies and diagnostics. Performance of morphologic and molecular assays from the same FFPE specimen is challenging because of the negative impact of formalin due to chemical modification and cross-linking of nucleic acids and proteins. A non-cross-linking, formalin-free tissue fixative has been recently developed to fulfil both requirements, i.e., to preserve morphology like FFPE and biomolecules like cryo-preservation. Since FISH is often required in combination with histopathology and molecular diagnostics, we tested the applicability of FISH protocols on tissues treated with this new fixative. We found that formalin post-fixation of histological sections of non-cross-linking, formalin-free and paraffin-embedded (NCFPE) breast cancer tissue generated equivalent results to those with FFPE tissue in human epidermal growth factor receptor 2 (HER2) FISH analysis. This protocol describes how a FISH assay originally developed and validated for FFPE tissue can be used for NCFPE tissues by a simple post-fixation step of histological sections.