Blind color decomposition of histological images

All you need is data preparation: A systematic review of image harmonization techniques in Multi-center/device studies for medical support systems

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) models trained on multi-centric and multi-device studies can provide more robust insights and research findings compared to single-center studies. However, variability in acquisition protocols and equipment can introduce inconsistencies that hamper the effective pooling of multi-source datasets. This systematic review evaluates strategies for image harmonization, which standardizes appearances to enable reliable AI analysis of multi-source medical imaging.

METHODS

A literature search using PRISMA guidelines was conducted to identify relevant papers published between 2013 and 2023 analyzing multi-centric and multi-device medical imaging studies that utilized image harmonization approaches.

RESULTS

Common image harmonization techniques included grayscale normalization (improving classification accuracy by up to 24.42 %), resampling (increasing the percentage of robust radiomics features from 59.5 % to 89.25 %), and color normalization (enhancing AUC by up to 0.25 in external test sets). Initially, mathematical and statistical methods dominated, but machine and deep learning adoption has risen recently. Color imaging modalities like digital pathology and dermatology have remained prominent application areas, though harmonization efforts have expanded to diverse fields including radiology, nuclear medicine, and ultrasound imaging. In all the modalities covered by this review, image harmonization improved AI performance, with increasing of up to 24.42 % in classification accuracy and 47 % in segmentation Dice scores.

CONCLUSIONS

Continued progress in image harmonization represents a promising strategy for advancing healthcare by enabling large-scale, reliable analysis of integrated multi-source datasets using AI. Standardizing imaging data across clinical settings can help realize personalized, evidence-based care supported by data-driven technologies while mitigating biases associated with specific populations or acquisition protocols.

Multi-Stage Classification-Based Deep Learning for Gleason System Grading Using Histopathological Images

In this work, we introduced an automated diagnostic system for Gleason system grading and grade groups (GG) classification using whole slide images (WSIs) of digitized prostate biopsy specimens (PBSs). Our system first classifies the Gleason pattern (GP) from PBSs and then identifies the Gleason score (GS) and GG. We developed a comprehensive DL-based approach to develop a grading pipeline system for the digitized PBSs and consider GP as a classification problem (not segmentation) compared to current research studies (deals with as a segmentation problem). A multilevel binary classification was implemented to enhance the segmentation accuracy for GP. Also, we created three levels of analysis (pyramidal levels) to extract different types of features. Each level has four shallow binary CNN to classify five GP labels. A majority fusion is applied for each pixel that has a total of 39 labeled images to create the final output for GP. The proposed framework is trained, validated, and tested on 3080 WSIs of PBS. The overall diagnostic accuracy for each CNN is evaluated using several metrics: precision (PR), recall (RE), and accuracy, which are documented by the confusion matrices.The results proved our system's potential for classifying all five GP and, thus, GG. The overall accuracy for the GG is evaluated using two metrics, PR and RE. The grade GG results are between 50% to 92% for RE and 50% to 92% for PR. Also, a comparison between our CNN architecture and the standard CNN (ResNet50) highlights our system's advantage. Finally, our deep-learning system achieved an agreement with the consensus grade groups.

AI based prostate analysis system trained without human supervision to predict patient outcome from tissue samples

In order to plan the best treatment for prostate cancer patients, the aggressiveness of the tumor is graded based on visual assessment of tissue biopsies according to the Gleason scale. Recently, a number of AI models have been developed that can be trained to do this grading as well as human pathologists. But the accuracy of the AI grading will be limited by the accuracy of the subjective “ground truth” Gleason grades used for the training. We have trained an AI to predict patient outcome directly based on image analysis of a large biobank of tissue samples with known outcome without input of any human knowledge about cancer grading. The model has shown similar and in some cases better ability to predict patient outcome on an independent test-set than expert pathologists doing the conventional grading.

Breast cancer histopathological images recognition based on two-stage nuclei segmentation strategy

Pathological examination is the gold standard for breast cancer diagnosis. The recognition of histopathological images of breast cancer has attracted a lot of attention in the field of medical image processing. In this paper, on the base of the Bioimaging 2015 dataset, a two-stage nuclei segmentation strategy, that is, a method of watershed segmentation based on histopathological images after stain separation, is proposed to make the dataset recognized to be the carcinoma and non-carcinoma recognition. Firstly, stain separation is performed on breast cancer histopathological images. Then the marker-based watershed segmentation method is used for images obtained from stain separation to achieve the nuclei segmentation target. Next, the completed local binary pattern is used to extract texture features from the nuclei regions (images after nuclei segmentation), and color features were extracted by using the color auto-correlation method on the stain-separated images. Finally, the two kinds of features were fused and the support vector machine was used for carcinoma and non-carcinoma recognition. The experimental results show that the two-stage nuclei segmentation strategy proposed in this paper has significant advantages in the recognition of carcinoma and non-carcinoma on breast cancer histopathological images, and the recognition accuracy arrives at 91.67%. The proposed method is also applied to the ICIAR 2018 dataset to realize the automatic recognition of carcinoma and non-carcinoma, and the recognition accuracy arrives at 92.50%.

A Method for Unsupervised Semi-Quantification of Inmunohistochemical Staining with Beta Divergences

In many research laboratories, it is essential to determine the relative expression levels of some proteins of interest in tissue samples. The semi-quantitative scoring of a set of images consists of establishing a scale of scores ranging from zero or one to a maximum number set by the researcher and assigning a score to each image that should represent some predefined characteristic of the IHC staining, such as its intensity. However, manual scoring depends on the judgment of an observer and therefore exposes the assessment to a certain level of bias. In this work, we present a fully automatic and unsupervised method for comparative biomarker quantification in histopathological brightfield images. The method relies on a color separation method that discriminates between two chromogens expressed as brown and blue colors robustly, independent of color variation or biomarker expression level. For this purpose, we have adopted a two-stage stain separation approach in the optical density space. First, a preliminary separation is performed using a deconvolution method in which the color vectors of the stains are determined after an eigendecomposition of the data. Then, we adjust the separation using the non-negative matrix factorization method with beta divergences, initializing the algorithm with the matrices resulting from the previous step. After that, a feature vector of each image based on the intensity of the two chromogens is determined. Finally, the images are annotated using a systematically initialized k-means clustering algorithm with beta divergences. The method clearly defines the initial boundaries of the categories, although some flexibility is added. Experiments for the semi-quantitative scoring of images in five categories have been carried out by comparing the results with the scores of four expert researchers yielding accuracies that range between 76.60% and 94.58%. These results show that the proposed automatic scoring system, which is definable and reproducible, produces consistent results.

Blind color deconvolution, normalization, and classification of histological images using general super Gaussian priors and Bayesian inference

BACKGROUND AND OBJECTIVE

Color variations in digital histopathology severely impact the performance of computer-aided diagnosis systems. They are due to differences in the staining process and acquisition system, among other reasons. Blind color deconvolution techniques separate multi-stained images into single stained bands which, once normalized, can be used to eliminate these negative color variations and improve the performance of machine learning tasks.

METHODS

In this work, we decompose the observed RGB image in its hematoxylin and eosin components. We apply Bayesian modeling and inference based on the use of Super Gaussian sparse priors for each stain together with prior closeness to a given reference color-vector matrix. The hematoxylin and eosin components are then used for image normalization and classification of histological images. The proposed framework is tested on stain separation, image normalization, and cancer classification problems. The results are measured using the peak signal to noise ratio, normalized median intensity and the area under ROC curve on five different databases.

RESULTS

The obtained results show the superiority of our approach to current state-of-the-art blind color deconvolution techniques. In particular, the fidelity to the tissue improves 1,27 dB in mean PSNR. The normalized median intensity shows a good normalization quality of the proposed approach on the tested datasets. Finally, in cancer classification experiments the area under the ROC curve improves from 0.9491 to 0.9656 and from 0.9279 to 0.9541 on Camelyon-16 and Camelyon-17, respectively, when the original and processed images are used. Furthermore, these figures of merits are better than those obtained by the methods compared with.

CONCLUSIONS

The proposed framework for blind color deconvolution, normalization and classification of images guarantees fidelity to the tissue structure and can be used both for normalization and classification. In addition, color deconvolution enables the use of the optical density space for classification, which improves the classification performance.

The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images

The color variation of hematoxylin and eosin (H&E)-stained tissues has presented a challenge for applications of artificial intelligence (AI) in digital pathology. Many color normalization algorithms have been developed in recent years in order to reduce the color variation between H&E images. However, previous efforts in benchmarking these algorithms have produced conflicting results and none have sufficiently assessed the efficacy of the various color normalization methods for improving diagnostic performance of AI systems. In this study, we systematically investigated eight color normalization algorithms for AI-based classification of H&E-stained histopathology slides, in the context of using images both from one center and from multiple centers. Our results show that color normalization does not consistently improve classification performance when both training and testing data are from a single center. However, using four multi-center datasets of two cancer types (ovarian and pleural) and objective functions, we show that color normalization can significantly improve the classification accuracy of images from external datasets (ovarian cancer: 0.25 AUC increase, p = 1.6 e-05; pleural cancer: 0.21 AUC increase, p = 1.4 e-10). Furthermore, we introduce a novel augmentation strategy by mixing color-normalized images using three easily accessible algorithms that consistently improves the diagnosis of test images from external centers, even when the individual normalization methods had varied results. We anticipate our study to be a starting point for reliable use of color normalization to improve AI-based, digital pathology-empowered diagnosis of cancers sourced from multiple centers. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Rapid pseudo-H&E imaging using a fluorescence-inbuilt optical coherence microscopic imaging system

A technique using Linnik-based optical coherence microscopy (OCM), with built-in fluorescence microscopy (FM), is demonstrated here to describe cellular-level morphology for fresh porcine and biobank tissue specimens. The proposed method utilizes color-coding to generate digital pseudo-H&E (p-H&E) images. Using the same camera, colocalized FM images are merged with corresponding morphological OCM images using a 24-bit RGB composition process to generate position-matched p-H&E images. From receipt of dissected fresh tissue piece to generation of stitched images, the total processing time is <15 min for a 1-cm² specimen, which is on average two times faster than frozen-section H&E process for fatty or water-rich fresh tissue specimens. This technique was successfully used to scan human and animal fresh tissue pieces, demonstrating its applicability for both biobank and veterinary purposes. We provide an in-depth comparison between p-H&E and human frozen-section H&E images acquired from the same metastatic sentinel lymph node slice (∼10 µm thick), and show the differences, like elastic fibers of a tiny blood vessel and cytoplasm of tumor cells. This optical sectioning technique provides histopathologists with a convenient assessment method that outputs large-field H&E-like images of fresh tissue pieces without requiring any physical embedment.

An unsupervised style normalization method for cytopathology images

Diverse styles of cytopathology images have a negative effect on the generalization ability of automated image analysis algorithms. This article proposes an unsupervised method to normalize cytopathology image styles. We design a two-stage style normalization framework with a style removal module to convert the colorful cytopathology image into a gray-scale image with a color-encoding mask and a domain adversarial style reconstruction module to map them back to a colorful image with user-selected style. Our method enforces both hue and structure consistency before and after normalization by using the color-encoding mask and per-pixel regression. Intra-domain and inter-domain adversarial learning are applied to ensure the style of normalized images consistent with the user-selected for input images of different domains. Our method shows superior results against current unsupervised color normalization methods on six cervical cell datasets from different hospitals and scanners. We further demonstrate that our normalization method greatly improves the recognition accuracy of lesion cells on unseen cytopathology images, which is meaningful for model generalization.

Stain Color Adaptive Normalization (SCAN) algorithm: Separation and standardization of histological stains in digital pathology

BACKGROUND AND OBJECTIVE

The diagnosis of histopathological images is based on the visual analysis of tissue slices under a light microscope. However, the histological tissue appearance may assume different color intensities depending on the staining process, operator ability and scanner specifications. This stain variability affects the diagnosis of the pathologist and decreases the accuracy of computer-aided diagnosis systems. In this context, the stain normalization process has proved to be a powerful tool to cope with this issue, allowing to standardize the stain color appearance of a source image respect to a reference image.

METHODS

In this paper, novel fully automated stain separation and normalization approaches for hematoxylin and eosin stained histological slides are presented. The proposed algorithm, named SCAN (Stain Color Adaptive Normalization), is based on segmentation and clustering strategies for cellular structures detection. The SCAN algorithm is able to improve the contrast between histological tissue and background and preserve local structures without changing the color of the lumen and the background.

RESULTS

Both stain separation and normalization techniques were qualitatively and quantitively validated on a multi-tissue and multiscale dataset, with highly satisfactory results, outperforming the state-of-the-art approaches. SCAN was also tested on whole-slide images with high performances and low computational times.

CONCLUSIONS

The potential contribution of the proposed standardization approach is twofold: the improvement of visual diagnosis in digital histopathology and the development of powerful pre-processing strategies to automated classification techniques for cancer detection.

Sustained acoustic medicine; sonophoresis for nonsteroidal anti-inflammatory drug delivery in arthritis

Background: Arthritis pain is primarily managed by nonsteroidal anti-inflammatory drugs (NSAIDs), such as diclofenac. Topical diclofenac gel is limited in efficacy due to its limited penetration through the skin. This study investigates the use of a multihour, wearable, localized, sonophoresis transdermal drug delivery device for the penetration enhancement of diclofenac through the skin. Materials & methods: A commercially available, sustained acoustic medicine (sam®) ultrasound device providing 4 h, 1.3 W, 132 mW/cm2, 3 MHz ultrasound treatment was evaluated for increasing the drug delivery of diclofenac gel through a human skin model and was compared with standard of care topical control diclofenac gel. Results: Sonophoresis of the diclofenac gel for 4 h increases diclofenac delivery by 3.8× (p < 0.01), and penetration by 32% (p < 0.01). Conclusion: Sustained acoustic medicine can be used as a transdermal drug-delivery device for nonsteroidal anti-inflammatory drugs.

Circular Clustering in Fuzzy Approximation Spaces for Color Normalization of Histological Images

One of the foremost and challenging tasks in hematoxylin and eosin stained histological image analysis is to reduce color variation present among images, which may significantly affect the performance of computer-aided histological image analysis. In this regard, the paper introduces a new rough-fuzzy circular clustering algorithm for stain color normalization. It judiciously integrates the merits of both fuzzy and rough sets. While the theory of rough sets deals with uncertainty, vagueness, and incompleteness in stain class definition, fuzzy set handles the overlapping nature of histochemical stains. The proposed circular clustering algorithm works on a weighted hue histogram, which considers both saturation and local neighborhood information of the given image. A new dissimilarity measure is introduced to deal with the circular nature of hue values. Some new quantitative measures are also proposed to evaluate the color constancy after normalization. The performance of the proposed method, along with a comparison with other state-of-the-art methods, is demonstrated on several publicly available standard data sets consisting of hematoxylin and eosin stained histological images.

Variational Bayesian Blind Color Deconvolution of Histopathological Images

Most whole-slide histological images are stained with two or more chemical dyes. Slide stain separation or color deconvolution is a crucial step within the digital pathology workflow. In this paper, the blind color deconvolution problem is formulated within the Bayesian framework. Starting from a multi-stained histological image, our model takes into account both spatial relations among the concentration image pixels and similarity between a given reference color-vector matrix and the estimated one. Using Variational Bayes inference, three efficient new blind color deconvolution methods are proposed which provide automated procedures to estimate all the model parameters in the problem. A comparison with classical and current state-of-the-art color deconvolution algorithms using real images has been carried out demonstrating the superiority of the proposed approach.

Glandular Segmentation of Prostate Cancer: An Illustration of How the Choice of Histopathological Stain Is One Key to Success for Computational Pathology

Digital pathology offers the potential for computer-aided diagnosis, significantly reducing the pathologists' workload and paving the way for accurate prognostication with reduced inter-and intra-observer variations. But successful computer-based analysis requires careful tissue preparation and image acquisition to keep color and intensity variations to a minimum. While the human eye may recognize prostate glands with significant color and intensity variations, a computer algorithm may fail under such conditions. Since malignancy grading of prostate tissue according to Gleason or to the International Society of Urological Pathology (ISUP) grading system is based on architectural growth patterns of prostatic carcinoma, automatic methods must rely on accurate identification of the prostate glands. But due to poor color differentiation between stroma and epithelium from the common stain hematoxylin-eosin, no method is yet able to segment all types of glands, making automatic prognostication hard to attain. We address the effect of tissue preparation on glandular segmentation with an alternative stain, Picrosirius red-hematoxylin, which clearly delineates the stromal boundaries, and couple this stain with a color decomposition that removes intensity variation. In this paper we propose a segmentation algorithm that uses image analysis techniques based on mathematical morphology and that can successfully determine the glandular boundaries. Accurate determination of the stromal and glandular morphology enables the identification of the architectural pattern that determine the malignancy grade and classify each gland into its appropriate Gleason grade or ISUP Grade Group. Segmentation of prostate tissue with the new stain and decomposition method has been successfully tested on more than 11000 objects including well-formed glands (Gleason grade 3), cribriform and fine caliber glands (grade 4), and single cells (grade 5) glands.

Unsupervised Domain Adaptation for Classification of Histopathology Whole-Slide Images

Computational image analysis is one means for evaluating digitized histopathology specimens that can increase the reproducibility and reliability with which cancer diagnoses are rendered while simultaneously providing insight as to the underlying mechanisms of disease onset and progression. A major challenge that is confronted when analyzing samples that have been prepared at disparate laboratories and institutions is that the algorithms used to assess the digitized specimens often exhibit heterogeneous staining characteristics because of slight differences in incubation times and the protocols used to prepare the samples. Unfortunately, such variations can render a prediction model learned from one batch of specimens ineffective for characterizing an ensemble originating from another site. In this work, we propose to adopt unsupervised domain adaptation to effectively transfer the discriminative knowledge obtained from any given source domain to the target domain without requiring any additional labeling or annotation of images at the target site. In this paper, our team investigates the use of two approaches for performing the adaptation: (1) color normalization and (2) adversarial training. The adversarial training strategy is implemented through the use of convolutional neural networks to find an invariant feature space and Siamese architecture within the target domain to add a regularization that is appropriate for the entire set of whole-slide images. The adversarial adaptation results in significant classification improvement compared with the baseline models under a wide range of experimental settings.

A novel computational method for automatic segmentation, quantification and comparative analysis of immunohistochemically labeled tissue sections

Background

In the clinical practice, the objective quantification of histological results is essential not only to define objective and well-established protocols for diagnosis, treatment, and assessment, but also to ameliorate disease comprehension.

Software

The software MIAQuant_Learn presented in this work segments, quantifies and analyzes markers in histochemical and immunohistochemical images obtained by different biological procedures and imaging tools. MIAQuant_Learn employs supervised learning techniques to customize the marker segmentation process with respect to any marker color appearance. Our software expresses the location of the segmented markers with respect to regions of interest by mean-distance histograms, which are numerically compared by measuring their intersection. When contiguous tissue sections stained by different markers are available, MIAQuant_Learn aligns them and overlaps the segmented markers in a unique image enabling a visual comparative analysis of the spatial distribution of each marker (markers’ relative location). Additionally, it computes novel measures of markers’ co-existence in tissue volumes depending on their density.

Conclusions

Applications of MIAQuant_Learn in clinical research studies have proven its effectiveness as a fast and efficient tool for the automatic extraction, quantification and analysis of histological sections. It is robust with respect to several deficits caused by image acquisition systems and produces objective and reproducible results. Thanks to its flexibility, MIAQuant_Learn represents an important tool to be exploited in basic research where needs are constantly changing.

A study about color normalization methods for histopathology images

Histopathology images are used for the diagnosis of the cancerous disease by the examination of tissue with the help of Whole Slide Imaging (WSI) scanner. A decision support system works well by the analysis of the histopathology images but a lot of problems arise in its decision. Color variation in the histopathology images is occurring due to use of the different scanner, use of various equipments, different stain coloring and reactivity from a different manufacturer. In this paper, detailed study and performance evaluation of color normalization methods on histopathology image datasets are presented. Color normalization of the source image by transferring the mean color of the target image in the source image and also to separate stain present in the source image. Stain separation and color normalization of the histopathology images can be helped for both pathology and computerized decision support system. Quality performances of different color normalization methods are evaluated and compared in terms of quaternion structure similarity index matrix (QSSIM), structure similarity index matrix (SSIM) and Pearson correlation coefficient (PCC) on various histopathology image datasets. Our experimental analysis suggests that structure-preserving color normalization (SPCN) provides better qualitatively and qualitatively results in comparison to the all the presented methods for breast and colorectal cancer histopathology image datasets.

Histopathological image classification with bilinear convolutional neural networks

The computer-aided quantitative analysis for histopathological images has attracted considerable attention. The stain decomposition on histopathological images is usually recommended to address the issue of co-localization or aliasing of tissue substances. Although the convolutional neural networks (CNN) is a popular deep learning algorithm for various tasks on histopathological image analysis, it is only directly performed on histopathological images without considering stain decomposition. The bilinear CNN (BCNN) is a new CNN model for fine-grained classification. BCNN consists of two CNNs, whose convolutional-layer outputs are multiplied with outer product at each spatial location. In this work, we propose a novel BCNN-based method for classification of histopathological images, which first decomposes histopathological images into hematoxylin and eosin stain components, and then perform BCNN on the decomposed images to fuse and improve the feature representation performance. The experimental results on the colorectal cancer histopathological image dataset with eight classes indicate that the proposed BCNN-based algorithm is superior to the traditional CNN.

Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer

Tissue Phenomics is the discipline of mining tissue images to identify patterns that are related to clinical outcome providing potential prognostic and predictive value. This involves the discovery process from assay development, image analysis, and data mining to the final interpretation and validation of the findings. Importantly, this process is not linear but allows backward steps and optimization loops over multiple sub-processes. We provide a detailed description of the Tissue Phenomics methodology while exemplifying each step on the application of prostate cancer recurrence prediction. In particular, we automatically identified tissue-based biomarkers having significant prognostic value for low- and intermediate-risk prostate cancer patients (Gleason scores 6–7b) after radical prostatectomy. We found that promising phenes were related to CD8(+) and CD68(+) cells in the microenvironment of cancerous glands in combination with the local micro-vascularization. Recurrence prediction based on the selected phenes yielded accuracies up to 83% thereby clearly outperforming prediction based on the Gleason score. Moreover, we compared different machine learning algorithms to combine the most relevant phenes resulting in increased accuracies of 88% for tumor progression prediction. These findings will be of potential use for future prognostic tests for prostate cancer patients and provide a proof-of-principle of the Tissue Phenomics approach.

Segmentation of digitized histological sections for quantification of the muscularized vasculature in the mouse hind limb

Immunohistochemical tissue staining enhances microvasculature characteristics, including the smooth muscle in the medial layer of the vessel walls that is responsible for regulation of blood flow. The vasculature can be imaged in a comprehensive fashion using whole-slide scanning. However, since each such image potentially contains hundreds of small vessels, manual vessel delineation and quantification is not practically feasible. In this work, we present a fully automatic segmentation and vasculature quantification algorithm for whole-slide images. We evaluated its performance on tissue samples drawn from the hind limbs of wild-type mice, stained for smooth muscle using 3,3'-Diaminobenzidine (DAB) immunostain. The algorithm was designed to be robust to vessel fragmentation due to staining irregularity, and artefactual staining of nonvessel objects. Colour deconvolution was used to isolate the DAB stain for detection of vessel wall fragments. Complete vessels were reconstructed from the fragments by joining endpoints of topological skeletons. Automatic measures of vessel density, perimeter, wall area and local wall thickness were taken. The segmentation algorithm was validated against manual measures, resulting in a Dice similarity coefficient of 89%. The relationships observed between these measures were as expected from a biological standpoint, providing further reinforcement of the accuracy of this system. This system provides a fully automated and accurate means of measuring the arteriolar and venular morphology of vascular smooth muscle.

Stain Deconvolution Using Statistical Analysis of Multi-Resolution Stain Colour Representation

Stain colour estimation is a prominent factor of the analysis pipeline in most of histology image processing algorithms. Providing a reliable and efficient stain colour deconvolution approach is fundamental for robust algorithm. In this paper, we propose a novel method for stain colour deconvolution of histology images. This approach statistically analyses the multi-resolutional representation of the image to separate the independent observations out of the correlated ones. We then estimate the stain mixing matrix using filtered uncorrelated data. We conducted an extensive set of experiments to compare the proposed method to the recent state of the art methods and demonstrate the robustness of this approach using three different datasets of scanned slides, prepared in different labs using different scanners.

Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images

Staining and scanning of tissue samples for microscopic examination is fraught with undesirable color variations arising from differences in raw materials and manufacturing techniques of stain vendors, staining protocols of labs, and color responses of digital scanners. When comparing tissue samples, color normalization and stain separation of the tissue images can be helpful for both pathologists and software. Techniques that are used for natural images fail to utilize structural properties of stained tissue samples and produce undesirable color distortions. The stain concentration cannot be negative. Tissue samples are stained with only a few stains and most tissue regions are characterized by at most one effective stain. We model these physical phenomena that define the tissue structure by first decomposing images in an unsupervised manner into stain density maps that are sparse and non-negative. For a given image, we combine its stain density maps with stain color basis of a pathologist-preferred target image, thus altering only its color while preserving its structure described by the maps. Stain density correlation with ground truth and preference by pathologists were higher for images normalized using our method when compared to other alternatives. We also propose a computationally faster extension of this technique for large whole-slide images that selects an appropriate patch sample instead of using the entire image to compute the stain color basis.

Blind colour separation of H&E stained histological images by linearly transforming the colour space

Blind source separation methods aim to split information into the original sources. In histology, each dye component attempts to specifically characterize different microscopic structures. In the case of the hematoxylin-eosin stain, universally used for routine examination, quantitative analysis may often require the inspection of different morphological signatures related mainly to nuclei patterns, but also to stroma distribution. Stain separation is usually a preprocessing operation that is transversal to different applications. This paper presents a novel colour separation method that finds the hematoxylin and eosin clusters by projecting the whole (r,g,b) space to a folded surface connecting the distributions of a series of [(r-b),g] planes that divide the cloud of H&E tones. The proposed method produces density maps closer to those obtained with the colour mixing matrices set by an expert, when comparing with the density maps obtained using nonnegative matrix factorization (NMF), independent component analysis (ICA) and a state-of-the-art method. The method has outperformed three baseline methods, NMF, Macenko and ICA, in about 8%, 12% and 52% for the eosin component, whereas this was about 4%, 8% and 26% for the hematoxylin component.

A Model based Survey of Colour Deconvolution in Diagnostic Brightfield Microscopy: Error Estimation and Spectral Consideration

Colour deconvolution is a method used in diagnostic brightfield microscopy to transform colour images of multiple stained biological samples into images representing the stain concentrations. It is applied by decomposing the absorbance values of stain mixtures into absorbance values of single stains. The method assumes a linear relation between stain concentration and absorbance, which is only valid under monochromatic conditions. Diagnostic applications, in turn, are often performed under polychromatic conditions, for which an accurate deconvolution result cannot be achieved. To show this, we establish a mathematical model to calculate non-monochromatic absorbance values based on imaging equipment typically used in histology and use this simulated data as the ground truth to evaluate the accuracy of colour deconvolution. We show the non-linear characteristics of the absorbance formation and demonstrate how it leads to significant deconvolution errors. In particular, our calculations reveal that polychromatic illumination causes 10-times higher deconvolution errors than sequential monochromatic LED illumination. In conclusion, our model can be used for a quantitative assessment of system components--and also to assess and compare colour deconvolution methods.

Sparse Non-negative Matrix Factorization (SNMF) based color unmixing for breast histopathological image analysis

Color deconvolution has emerged as a popular method for color unmixing as a pre-processing step for image analysis of digital pathology images. One deficiency of this approach is that the stain matrix is pre-defined which requires specific knowledge of the data. This paper presents an unsupervised Sparse Non-negative Matrix Factorization (SNMF) based approach for color unmixing. We evaluate this approach for color unmixing of breast pathology images. Compared to Non-negative Matrix Factorization (NMF), the sparseness constraint imposed on coefficient matrix aims to use more meaningful representation of color components for separating stained colors. In this work SNMF is leveraged for decomposing pure stained color in both Immunohistochemistry (IHC) and Hematoxylin and Eosin (H&E) images. SNMF is compared with Principle Component Analysis (PCA), Independent Component Analysis (ICA), Color Deconvolution (CD), and Non-negative Matrix Factorization (NMF) based approaches. SNMF demonstrated improved performance in decomposing brown diaminobenzidine (DAB) component from 36 IHC images as well as accurately segmenting about 1400 nuclei and 500 lymphocytes from H & E images.

A Complete Color Normalization Approach to Histopathology Images Using Color Cues Computed From Saturation-Weighted Statistics

GOAL

In digital histopathology, tasks of segmentation and disease diagnosis are achieved by quantitative analysis of image content. However, color variation in image samples makes it challenging to produce reliable results. This paper introduces a complete normalization scheme to address the problem of color variation in histopathology images jointly caused by inconsistent biopsy staining and nonstandard imaging condition. Method : Different from existing normalization methods that either address partial cause of color variation or lump them together, our method identifies causes of color variation based on a microscopic imaging model and addresses inconsistency in biopsy imaging and staining by an illuminant normalization module and a spectral normalization module, respectively. In evaluation, we use two public datasets that are representative of histopathology images commonly received in clinics to examine the proposed method from the aspects of robustness to system settings, performance consistency against achromatic pixels, and normalization effectiveness in terms of histological information preservation.

RESULTS

As the saturation-weighted statistics proposed in this study generates stable and reliable color cues for stain normalization, our scheme is robust to system parameters and insensitive to image content and achromatic colors.

CONCLUSION

Extensive experimentation suggests that our approach outperforms state-of-the-art normalization methods as the proposed method is the only approach that succeeds to preserve histological information after normalization.

SIGNIFICANCE

The proposed color normalization solution would be useful to mitigate effects of color variation in pathology images on subsequent quantitative analysis.

Automated classification of glandular tissue by statistical proximity sampling

Due to the complexity of biological tissue and variations in staining procedures, features that are based on the explicit extraction of properties from subglandular structures in tissue images may have difficulty generalizing well over an unrestricted set of images and staining variations. We circumvent this problem by an implicit representation that is both robust and highly descriptive, especially when combined with a multiple instance learning approach to image classification. The new feature method is able to describe tissue architecture based on glandular structure. It is based on statistically representing the relative distribution of tissue components around lumen regions, while preserving spatial and quantitative information, as a basis for diagnosing and analyzing different areas within an image. We demonstrate the efficacy of the method in extracting discriminative features for obtaining high classification rates for tubular formation in both healthy and cancerous tissue, which is an important component in Gleason and tubule-based Elston grading. The proposed method may be used for glandular classification, also in other tissue types, in addition to general applicability as a region-based feature descriptor in image analysis where the image represents a bag with a certain label (or grade) and the region-based feature vectors represent instances.

Breast cancer histopathology image analysis: a review

This paper presents an overview of methods that have been proposed for the analysis of breast cancer histopathology images. This research area has become particularly relevant with the advent of whole slide imaging (WSI) scanners, which can perform cost-effective and high-throughput histopathology slide digitization, and which aim at replacing the optical microscope as the primary tool used by pathologist. Breast cancer is the most prevalent form of cancers among women, and image analysis methods that target this disease have a huge potential to reduce the workload in a typical pathology lab and to improve the quality of the interpretation. This paper is meant as an introduction for nonexperts. It starts with an overview of the tissue preparation, staining and slide digitization processes followed by a discussion of the different image processing techniques and applications, ranging from analysis of tissue staining to computer-aided diagnosis, and prognosis of breast cancer patients.

Image segmentation and identification of paired antibodies in breast tissue

Comparing staining patterns of paired antibodies designed towards a specific protein but toward different epitopes of the protein provides quality control over the binding and the antibodies' ability to identify the target protein correctly and exclusively. We present a method for automated quantification of immunostaining patterns for antibodies in breast tissue using the Human Protein Atlas database. In such tissue, dark brown dye 3,3′-diaminobenzidine is used as an antibody-specific stain whereas the blue dye hematoxylin is used as a counterstain. The proposed method is based on clustering and relative scaling of features following principal component analysis. Our method is able (1) to accurately segment and identify staining patterns and quantify the amount of staining and (2) to detect paired antibodies by correlating the segmentation results among different cases. Moreover, the method is simple, operating in a low-dimensional feature space, and computationally efficient which makes it suitable for high-throughput processing of tissue microarrays.

Simultaneous sparsity model for histopathological image representation and classification

The multi-channel nature of digital histopathological images presents an opportunity to exploit the correlated color channel information for better image modeling. Inspired by recent work in sparsity for single channel image classification, we propose a new simultaneous sparsity model for multi-channel histopathological image representation and classification (SHIRC). Essentially, we represent a histopathological image as a sparse linear combination of training examples under suitable channel-wise constraints. Classification is performed by solving a newly formulated simultaneous sparsity-based optimization problem. A practical challenge is the correspondence of image objects (cellular and nuclear structures) at different spatial locations in the image. We propose a robust locally adaptive variant of SHIRC (LA-SHIRC) to tackle this issue. Experiments on two challenging real-world image data sets: 1) mammalian tissue images acquired by pathologists of the animal diagnostics lab (ADL) at Pennsylvania State University, and 2) human intraductal breast lesions, reveal the merits of our proposal over state-of-the-art alternatives. Further, we demonstrate that LA-SHIRC exhibits a more graceful decay in classification accuracy against the number of training images which is highly desirable in practice where generous training per class is often not available.

Human Factors and Human-Computer Considerations in Teleradiology and Telepathology

Radiology and pathology are unique among other clinical specialties that incorporate telemedicine technologies into clinical practice, as, for the most part in traditional practice, there are few or no direct patient encounters. The majority of teleradiology and telepathology involves viewing images, which is exactly what occurs without the "tele" component. The images used are generally quite large, require dedicated displays and software for viewing, and present challenges to the clinician who must navigate through the presented data to render a diagnostic decision or interpretation. This digital viewing environment is very different from the more traditional reading environment (i.e., film and microscopy), necessitating a new look at how to optimize reading environments and address human factors issues. This paper will review some of the key components that need to be optimized for effective and efficient practice of teleradiology and telepathology using traditional workstations as well as some of the newer mobile viewing applications.