Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

Digital Pathology as an Innovative Tool for Improving Cancer Diagnosis and Treatment

For more than a century, the "gold" standard for diagnosing malignant neoplasms has been pathohistology. However, the continuous advancement of modern technologies is leading to a radical transformation of this field and the emergence of digital pathology. The main advantages of digital pathology include the convenience of the data storage and transfer, as well as the potential for automating diagnostic processes through the application of artificial intelligence technologies. Integrating digital pathology into clinical practice is expected to accelerate the analysis of histological samples, reduce the costs associated with such procedures, and enable the accumulation of large datasets for future scientific research. At the same time, the development of digital pathology faces certain challenges such as the need for technical upgrades in laboratories, ensuring data cybersecurity, and training qualified personnel.

In vitro models to evaluate multidrug resistance in cancer cells: Biochemical and morphological techniques and pharmacological strategies

The overexpression of ATP-binding cassette (ABC) transporters contributes to the failure of chemotherapies and symbolizes a great challenge in oncology, associated with the adaptation of tumor cells to anticancer drugs such that these transporters become less effective, a mechanism known as multidrug resistance (MDR). The aim of this review is to present the most widely used methodologies for induction and comprehension of in vitro models for detection of multidrug-resistant (MDR) modulators or inhibitors, including biochemical and morphological techniques for chemosensitivity studies. The overexpression of MDR proteins, predominantly, the subfamily glycoprotein-1 (P-gp or ABCB1) multidrug resistance, multidrug resistance-associated protein 1 (MRP1 or ABCCC1), multidrug resistance-associated protein 2 (MRP2 or ABCC2) and cancer resistance protein (ABCG2), in chemotherapy-exposed cancer lines have been established/investigated by several techniques. Amongst these techniques, the most used are (i) colorimetric/fluorescent indirect bioassays, (ii) rhodamine and efflux analysis, (iii) release of 3,30-diethyloxacarbocyanine iodide by fluorescence microscopy and flow cytometry to measure P-gp function and other ABC transporters, (iv) exclusion of calcein-acetoxymethylester, (v) ATPase assays to distinguish types of interaction with ABC transporters, (vi) morphology to detail phenotypic characteristics in transformed cells, (vii) molecular testing of resistance-related proteins (RT-qPCR) and (viii) 2D and 3D models, (ix) organoids, and (x) microfluidic technology. Then, in vitro models for detecting chemotherapy MDR cells to assess innovative therapies to modulate or inhibit tumor cell growth and overcome clinical resistance. It is noteworthy that different therapies including anti-miRNAs, antibody-drug conjugates (to natural products), and epigenetic modifications were also considered as promising alternatives, since currently no anti-MDR therapies are able to improve patient quality of life. Therefore, there is also urgency for new clinical markers of resistance to more reliably reflect in vivo effectiveness of novel antitumor drugs.

An Innovative Attention-based Triplet Deep Hashing Approach to Retrieve Histopathology Images

Content-based histopathology image retrieval (CBHIR) can assist in the diagnosis of different diseases. The retrieval procedure can be complex and time-consuming if high-dimensional features are required. Thus, hashing techniques are employed to address these issues by mapping the feature space into binary values of varying lengths. The performance of deep hashing approaches in image retrieval is often superior to that of traditional hashing methods. Among deep hashing approaches, triplet-based models are typically more effective than pairwise ones. Recent studies have demonstrated that incorporating the attention mechanism into a deep hashing approach can improve its effectiveness in retrieving images. This paper presents an innovative triplet deep hashing strategy based on the attention mechanism for retrieving histopathology images, called histopathology attention triplet deep hashing (HATDH). Three deep attention-based hashing models with identical architectures and weights are employed to produce binary values. The proposed attention module can aid the models in extracting features more efficiently. Moreover, we introduce an improved triplet loss function considering pair inputs separately in addition to triplet inputs for increasing efficiency during the training and retrieval steps. Based on experiments conducted on two public histopathology datasets, BreakHis and Kather, HATDH significantly outperforms state-of-the-art hashing algorithms.

A Dual-Modal, Label-Free Raman Imaging Method for Rapid Virtual Staining of Large-Area Breast Cancer Tissue Sections

As one of the most common cancers, accurate, rapid, and simple histopathological diagnosis is very important for breast cancer. Raman imaging is a powerful technique for label-free analysis of tissue composition and histopathology, but it suffers from slow speed when applied to large-area tissue sections. In this study, we propose a dual-modal Raman imaging method that combines Raman mapping data with microscopy bright-field images to achieve virtual staining of breast cancer tissue sections. We validate our method on various breast tissue sections with different morphologies and biomarker expressions and compare it with the golden standard of histopathological methods. The results demonstrate that our method can effectively distinguish various types and components of tissues, and provide staining images comparable to stained tissue sections. Moreover, our method can improve imaging speed by up to 65 times compared to general spontaneous Raman imaging methods. It is simple, fast, and suitable for clinical applications.

Digital image analysis and machine learning-assisted prediction of neoadjuvant chemotherapy response in triple-negative breast cancer

BACKGROUND

Pathological complete response (pCR) is associated with favorable prognosis in patients with triple-negative breast cancer (TNBC). However, only 30-40% of TNBC patients treated with neoadjuvant chemotherapy (NAC) show pCR, while the remaining 60-70% show residual disease (RD). The role of the tumor microenvironment in NAC response in patients with TNBC remains unclear. In this study, we developed a machine learning-based two-step pipeline to distinguish between various histological components in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) of TNBC tissue biopsies and to identify histological features that can predict NAC response.

METHODS

H&E-stained WSIs of treatment-naïve biopsies from 85 patients (51 with pCR and 34 with RD) of the model development cohort and 79 patients (41 with pCR and 38 with RD) of the validation cohort were separated through a stratified eightfold cross-validation strategy for the first step and leave-one-out cross-validation strategy for the second step. A tile-level histology label prediction pipeline and four machine-learning classifiers were used to analyze 468,043 tiles of WSIs. The best-trained classifier used 55 texture features from each tile to produce a probability profile during testing. The predicted histology classes were used to generate a histology classification map of the spatial distributions of different tissue regions. A patient-level NAC response prediction pipeline was trained with features derived from paired histology classification maps. The top graph-based features capturing the relevant spatial information across the different histological classes were provided to the radial basis function kernel support vector machine (rbfSVM) classifier for NAC treatment response prediction.

RESULTS

The tile-level prediction pipeline achieved 86.72% accuracy for histology class classification, while the patient-level pipeline achieved 83.53% NAC response (pCR vs. RD) prediction accuracy of the model development cohort. The model was validated with an independent cohort with tile histology validation accuracy of 83.59% and NAC prediction accuracy of 81.01%. The histological class pairs with the strongest NAC response predictive ability were tumor and tumor tumor-infiltrating lymphocytes for pCR and microvessel density and polyploid giant cancer cells for RD.

CONCLUSION

Our machine learning pipeline can robustly identify clinically relevant histological classes that predict NAC response in TNBC patients and may help guide patient selection for NAC treatment.

Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology

Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.

Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal

Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987-2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient's medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.

Whole slide imaging (WSI) scanner differences influence optical and computed properties of digitized prostate cancer histology

Purpose

Digital pathology is becoming an increasingly popular area of advancement in both research and clinically. Pathologists are now able to manage and interpret slides digitally, as well as collaborate with external pathologists with digital copies of slides. Differences in slide scanners include variation in resolution, image contrast, and optical properties, which may influence downstream image processing. This study tested the hypothesis that varying slide scanners would result in differences in computed pathomic features on prostate cancer whole mount slides.

Design

This study collected 192 unique tissue slides from 30 patients following prostatectomy. Tissue samples were paraffin-embedded, stained for hematoxylin and eosin (H&E), and digitized using 3 different scanning microscopes at the highest available magnification rate, for a total of 3 digitized slides per tissue slide. These scanners included a (S1) Nikon microscope equipped with an automated sliding stage, an (S2) Olympus VS120 slide scanner, and a (S3) Huron TissueScope LE scanner. A color deconvolution algorithm was then used to optimize contrast by projecting the RGB image into color channels representing optical stain density. The resulting intensity standardized images were then computationally processed to segment tissue and calculate pathomic features including lumen, stroma, epithelium, and epithelial cell density, as well as second-order features including lumen area and roundness; epithelial area, roundness, and wall thickness; and cell fraction. For each tested feature, mean values of that feature per digitized slide were collected and compared across slide scanners using mixed effect models, fit to compare differences in the tested feature associated with all slide scanners for each slide, including a random effect of subject with a nested random effect of slide to account for repeated measures. Similar models were also computed for tissue densities to examine how differences in scanner impact downstream processing.

Results

Each mean color channel intensity (i.e., Red, Green, Blue) differed between slide scanners (all P<.001). Of the color deconvolved images, only the hematoxylin channel was similar in all 3 scanners (all P>.05). Lumen and stroma densities between S3 and S1 slides, and epithelial cell density between S3 and S2 (P>.05) were comparable but all other comparisons were significantly different (P<.05). The second-order features were found to be comparable for all scanner comparisons, except for lumen area and epithelium area.

Conclusion

This study demonstrates that both optical and computed properties of digitized histological samples are impacted by slide scanner differences. Future research is warranted to better understand which scanner properties influence the tissue segmentation process and to develop harmonization techniques for comparing data across multiple slide scanners.

Computational pathology improves risk stratification of a multi-gene assay for early stage ER+ breast cancer

Prognostic markers currently utilized in clinical practice for estrogen receptor-positive (ER+) and lymph node-negative (LN-) invasive breast cancer (IBC) patients include the Nottingham grading system and Oncotype Dx (ODx). However, these biomarkers are not always optimal and remain subject to inter-/intra-observer variability and high cost. In this study, we evaluated the association between computationally derived image features from H&E images and disease-free survival (DFS) in ER+ and LN- IBC. H&E images from a total of n = 321 patients with ER+ and LN- IBC from three cohorts were employed for this study (Training set: D1 (n = 116), Validation sets: D2 (n = 121) and D3 (n = 84)). A total of 343 features relating to nuclear morphology, mitotic activity, and tubule formation were computationally extracted from each slide image. A Cox regression model (IbRiS) was trained to identify significant predictors of DFS and predict a high/low-risk category using D1 and was validated on independent testing sets D2 and D3 as well as within each ODx risk category. IbRiS was significantly prognostic of DFS with a hazard ratio (HR) of 2.33 (95% confidence interval (95% CI) = 1.02-5.32, p = 0.045) on D2 and a HR of 2.94 (95% CI = 1.18-7.35, p = 0.0208) on D3. In addition, IbRiS yielded significant risk stratification within high ODx risk categories (D1 + D2: HR = 10.35, 95% CI = 1.20-89.18, p = 0.0106; D1: p = 0.0238; D2: p = 0.0389), potentially providing more granular risk stratification than offered by ODx alone.

Removing non-nuclei information from histopathological images: A preprocessing step towards improving nuclei segmentation methods

Disease interpretation by computer-aided diagnosis systems in digital pathology depends on reliable detection and segmentation of nuclei in hematoxylin and eosin (HE) images. These 2 tasks are challenging since appearance of both cell nuclei and background structures are very variable. This paper presents a method to improve nuclei detection and segmentation in HE images by removing tiles that only contain background information. The method divides each image into smaller patches and uses their projection to the noiselet space to capture different spatial features from non-nuclei background and nuclei structures. The noiselet features are clustered by a K-means algorithm and the resultant partition, defined by the cluster centroids, is herein named the noiselet code-book. A part of an image, a tile, is divided into patches and represented by the histogram of occurrences of the projected patches in the noiselet code-book. Finally, with these histograms, a classifier learns to differentiate between nuclei and non-nuclei tiles. By applying a conventional watershed-marked method to detect and segment nuclei, evaluation consisted in comparing pure watershed method against denoising-plus-watershed in an open database with 8 different types of tissues. The averaged F-score of nuclei detection improved from 0.830 to 0.86 and the dice score after segmentation increased from 0.701 to 0.723.

Human Hepatocellular Carcinoma Classification from H&E Stained Histopathology Images with 3D Convolutional Neural Networks and Focal Loss Function

This paper proposes a new Hepatocellular Carcinoma (HCC) classification method utilizing a hyperspectral imaging system (HSI) integrated with a light microscope. Using our custom imaging system, we have captured 270 bands of hyperspectral images of healthy and cancer tissue samples with HCC diagnosis from a liver microarray slide. Convolutional Neural Networks with 3D convolutions (3D-CNN) have been used to build an accurate classification model. With the help of 3D convolutions, spectral and spatial features within the hyperspectral cube are incorporated to train a strong classifier. Unlike 2D convolutions, 3D convolutions take the spectral dimension into account while automatically collecting distinctive features during the CNN training stage. As a result, we have avoided manual feature engineering on hyperspectral data and proposed a compact method for HSI medical applications. Moreover, the focal loss function, utilized as a CNN cost function, enables our model to tackle the class imbalance problem residing in the dataset effectively. The focal loss function emphasizes the hard examples to learn and prevents overfitting due to the lack of inter-class balancing. Our empirical results demonstrate the superiority of hyperspectral data over RGB data for liver cancer tissue classification. We have observed that increased spectral dimension results in higher classification accuracy. Both spectral and spatial features are essential in training an accurate learner for cancer tissue classification.

Association of chemokine (CXC motif) receptor 4 expression with lymphovascular invasion and lymph node metastasis of invasive breast cancer

BACKGROUND

The histological tumor grade influences the prognosis of breast cancer. In metastatic breast cancer, stromal cells produce chemokine (CXC motif) ligand 12 or stromal cell-derived factor-1 as a chemoattractant, which binds to chemokine (CXC motif) receptor 4 (CXCR4) expressed by breast cancer cells.

OBJECTIVE

This study aimed to determine the expression of CXCR4 in invasive breast cancer in relation to lymphovascular invasion (LVI) and lymph node metastasis.

METHODS

This observational study retrospectively investigated a paraffin block archived sample diagnosed with invasive breast cancer. The results of immunohistochemical staining with CXCR4 antibody and expression analysis were evaluated using light microscopy. The data were statistically analyzed using the chi-square test and presented in a table using SPSS version 18. P-values of <0.05 were considered statistically significant.

RESULTS

The expression of CXCR4 was significantly associated with the incidence of LVI and lymph node metastasis in invasive breast cancer (both p = 0.001).

CONCLUSIONS

The results show that the expression of CXCR4 varies and support its decisive role in the incidence of LVI and lymph node metastasis in invasive breast cancer.

An Approach toward Automatic Specifics Diagnosis of Breast Cancer Based on an Immunohistochemical Image

The paper explored the problem of automatic diagnosis based on immunohistochemical image analysis. The issue of automated diagnosis is a preliminary and advisory statement for a diagnostician. The authors studied breast cancer histological and immunohistochemical images using the following biomarkers progesterone, estrogen, oncoprotein, and a cell proliferation biomarker. The authors developed a breast cancer diagnosis method based on immunohistochemical image analysis. The proposed method consists of algorithms for image preprocessing, segmentation, and the determination of informative indicators (relative area and intensity of cells) and an algorithm for determining the molecular genetic breast cancer subtype. An adaptive algorithm for image preprocessing was developed to improve the quality of the images. It includes median filtering and image brightness equalization techniques. In addition, the authors developed a software module part of the HIAMS software package based on the Java programming language and the OpenCV computer vision library. Four molecular genetic breast cancer subtypes could be identified using this solution: subtype Luminal A, subtype Luminal B, subtype HER2/neu amplified, and basalt-like subtype. The developed algorithm for the quantitative characteristics of the immunohistochemical images showed sufficient accuracy in determining the cancer subtype "Luminal A". It was experimentally established that the relative area of the nuclei of cells covered with biomarkers of progesterone, estrogen, and oncoprotein was more than 85%. The given approach allows for automating and accelerating the process of diagnosis. Developed algorithms for calculating the quantitative characteristics of cells on immunohistochemical images can increase the accuracy of diagnosis.

Nanoscale biomaterials for terahertz imaging: A non-invasive approach for early cancer detection

Terahertz (THz) technology is developing a non-invasive imaging system for biosensing and clinical diagnosis. THz medical imaging mainly benefits from great sensitivity in detecting changes in water content and structural variations in diseased cells versus normal tissues. Compared to healthy tissues, cancerous tumors contain a higher level of water molecules and show structural changes, resulting in different THz absorption. Here we described the principle of THz imaging and advancement in the field of translational biomedicine and early detection of pathologic tissue, with a particular focus on oncology. In addition, although the main forte of THz imaging relies on detecting differences in water content to distinguish the exact margin of tumor, THz displays limited contrast in living tissue for in-vivo clinical imaging. In the last few years, nanotechnology has attracted attention to aid THz medical imaging and various nanoparticles have been investigated as contrast enhancements to improve the accuracy, sensitivity, and specificity of THz images. Most of these multimodal contrast agents take advantage of the temperature-dependent of THz spectrum to the conformational variation of the water molecule. We discuss advances in developing THz contrast agents to accelerate the advancement of non-invasive THz imaging with improved sensitivity and specificity for translational clinical oncology.

Stain normalization in digital pathology: Clinical multi-center evaluation of image quality

In digital pathology, the final appearance of digitized images is affected by several factors, resulting in stain color and intensity variation. Stain normalization is an innovative solution to overcome stain variability. However, the validation of color normalization tools has been assessed only from a quantitative perspective, through the computation of similarity metrics between the original and normalized images. To the best of our knowledge, no works investigate the impact of normalization on the pathologist's evaluation. The objective of this paper is to propose a multi-tissue (i.e., breast, colon, liver, lung, and prostate) and multi-center qualitative analysis of a stain normalization tool with the involvement of pathologists with different years of experience. Two qualitative studies were carried out for this purpose: (i) a first study focused on the analysis of the perceived image quality and absence of significant image artifacts after the normalization process; (ii) a second study focused on the clinical score of the normalized image with respect to the original one. The results of the first study prove the high quality of the normalized image with a low impact artifact generation, while the second study demonstrates the superiority of the normalized image with respect to the original one in clinical practice. The normalization process can help both to reduce variability due to tissue staining procedures and facilitate the pathologist in the histological examination. The experimental results obtained in this work are encouraging and can justify the use of a stain normalization tool in clinical routine.

Multi-Class Classification of Breast Cancer Using 6B-Net with Deep Feature Fusion and Selection Method

Breast cancer has now overtaken lung cancer as the world's most commonly diagnosed cancer, with thousands of new cases per year. Early detection and classification of breast cancer are necessary to overcome the death rate. Recently, many deep learning-based studies have been proposed for automatic diagnosis and classification of this deadly disease, using histopathology images. This study proposed a novel solution for multi-class breast cancer classification from histopathology images using deep learning. For this purpose, a novel 6B-Net deep CNN model, with feature fusion and selection mechanism, was developed for multi-class breast cancer classification. For the evaluation of the proposed method, two large, publicly available datasets, namely, BreaKHis, with eight classes containing 7909 images, and a breast cancer histopathology dataset, containing 3771 images of four classes, were used. The proposed method achieves a multi-class average accuracy of 94.20%, with a classification training time of 226 s in four classes of breast cancer, and a multi-class average accuracy of 90.10%, with a classification training time of 147 s in eight classes of breast cancer. The experimental outcomes show that the proposed method achieves the highest multi-class average accuracy for breast cancer classification, and hence, the proposed method can effectively be applied for early detection and classification of breast cancer to assist the pathologists in early and accurate diagnosis of breast cancer.

Breast Cancer Molecular Subtype Prediction on Pathological Images with Discriminative Patch Selection and Multi-Instance Learning

Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient’s paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable block sampling error is risky due to the tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&amp;E pathological whole slide images (WSI) using the AI method is useful and critical to assist pathologists to pre-screen proper paraffin block for IHC. It is a challenging task since only WSI-level labels of molecular subtypes from IHC can be obtained without detailed local region information. Gigapixel WSIs are divided into a huge amount of patches to be computationally feasible for deep learning, while with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selection and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&amp;E WSIs. Firstly, co-teaching strategy using two networks was adopted to learn molecular subtype representations and filter out some noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating local patch with global slide constraint information was used to fine-tune MIL framework on obtained discriminative patches and further improve the prediction performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed AI method and our models outperformed even senior pathologists, which has the potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic.

Using parallel pre-trained types of DCNN model to predict breast cancer with color normalization

Objective

Breast cancer is the most common among women, and it causes many deaths every year. Early diagnosis increases the chance of cure through treatment. The traditional manual diagnosis requires effort and time from pathological experts, as it needs a joint experience of a number of pathologists. Diagnostic mistakes can lead to catastrophic results and endanger the lives of patients. The presence of an expert system that is able to specify whether the examined tissue is healthy or not, thus improves the quality of diagnosis and saves the time of experts. In this paper, a model capable of classifying breast cancer anatomy by making use of a pre-trained DCNN has been proposed. To build this model, first of all the image should be color stained by using Vahadane algorithm, then the model which combines three pre-trained DCNN (Xception, NASNet and Inceptoin_Resnet_V2) should be built in parallel, then the three branches should be aggregated to take advantage of each other. The suggested model was tested under different values of threshold ratios and also compared with other models.

Results

The proposed model on the BreaKHis dataset achieved 98% accuracy, which is better than the accuracy of other models used in this field.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13104-021-05902-3.

Assessment of Digital Pathology Imaging Biomarkers Associated with Breast Cancer Histologic Grade

Background: Evaluating histologic grade for breast cancer diagnosis is standard and associated with prognostic outcomes. Current challenges include the time required for manual microscopic evaluation and interobserver variability. This study proposes a computer-aided diagnostic (CAD) pipeline for grading tumors using artificial intelligence. Methods: There were 138 patients included in this retrospective study. Breast core biopsy slides were prepared using standard laboratory techniques, digitized, and pre-processed for analysis. Deep convolutional neural networks (CNNs) were developed to identify the regions of interest containing malignant cells and to segment tumor nuclei. Imaging-based features associated with spatial parameters were extracted from the segmented regions of interest (ROIs). Clinical datasets and pathologic biomarkers (estrogen receptor, progesterone receptor, and human epidermal growth factor 2) were collected from all study subjects. Pathologic, clinical, and imaging-based features were input into machine learning (ML) models to classify histologic grade, and model performances were tested against ground-truth labels at the patient-level. Classification performances were evaluated using receiver-operating characteristic (ROC) analysis. Results: Multiparametric feature sets, containing both clinical and imaging-based features, demonstrated high classification performance. Using imaging-derived markers alone, the classification performance demonstrated an area under the curve (AUC) of 0.745, while modeling these features with other pathologic biomarkers yielded an AUC of 0.836. Conclusion: These results demonstrate an association between tumor nuclear spatial features and tumor grade. If further validated, these systems may be implemented into pathology CADs and can assist pathologists to expeditiously grade tumors at the time of diagnosis and to help guide clinical decisions.

Highly sensitive detection of malignant glioma cells using metamaterial-inspired THz biosensor based on electromagnetically induced transparency

Metamaterial-inspired biosensors have been extensively studied recently years for fast and low-cost THz detection. However, only the variation of the resonance frequency has been closely concerned in such sensors so far, whiles the magnitude variation, which also provide important information of the analyte, has not been sufficiently analyzed. In this paper, by the observation of two degree of variations, we propose a label-free biosensing approach for molecular classification of glioma cells. The metamaterial biosensor consisting of cut wires and split ring resonators are proposed to realize polarization-independent electromagnetic induced transparency (EIT) at THz frequencies. Simulated results show that the EIT-like resonance experiences both resonance frequency and magnitude variations when the properties of analyte change, which is further explained with coupled oscillators model theory. The theoretical sensitivity of the biosensor is evaluated up to 496.01 GHz/RIU. In experiments, two types of glioma cells (mutant and wild-type) are cultured on the biosensor surface. The dependences of frequency shifts and the peak magnitude variations on the cells concentrations for different types give new perspective for molecular classification of glioma cells. The measured results indicate that the mutant and wild-type glioma cells can be distinguished directly by observing both the variations of EIT resonance frequency and magnitude at any cells concentrations without antibody introduction. Our metamaterial-based biosensor shows a great potential in the recognition of molecule types of glioma cells, opening alternative way to sensitive biosensing technology.

Microscope-Based Automated Quantification of Liver Fibrosis in Mice Using a Deep Learning Algorithm

In preclinical studies that involve animal models for hepatic fibrosis, accurate quantification of the fibrosis is of utmost importance. The use of digital image analysis based on deep learning artificial intelligence (AI) algorithms can facilitate accurate evaluation of liver fibrosis in these models. In the present study, we compared the quantitative evaluation of collagen proportionate area in the carbon tetrachloride model of liver fibrosis in the mouse by a newly developed AI algorithm to the semiquantitative assessment of liver fibrosis performed by a board-certified toxicologic pathologist. We found an excellent correlation between the 2 methods of assessment, most evident in the higher magnification (×40) as compared to the lower magnification (×10). These findings strengthen the confidence of using digital tools in the toxicologic pathology field as an adjunct to an expert toxicologic pathologist.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

A New Digital Method for Counting Mast Cells in Cutaneous Specific Lesions of Mastocytosis: A Series of Adult Cases of Mastocytosis With Clinical-Pathological Correlations

ABSTRACT

Cutaneous mastocytosis is characterized by the abnormal accumulation of mast cells in the skin. However, mast cell counting is not always easy and reproducible with classical methods. This work aims to demonstrate the reliability, usability, and virtues of a new software used on digital tablets for counting mast cells in cutaneous specific lesions of mastocytosis, to assess differences in mast cell counts between clinical subtypes of mastocytosis in the skin, and to consider the feasibility of applying a diagnostic mast cell count cutoff to urticaria pigmentosa, which is the most frequent form of cutaneous mastocytosis. Using a new digital tablet software that was accessible by multiple observers through its own wireless network and allowed high resolution of the image without data compression, we counted the number of mast cells on slides of patients and control skins immunostained for CD117. We found that our counting method was highly reproducible and that the new software allowed very quick counting. We evidenced strong differences in the mast cell count between most of the clinical subtypes of mastocytosis in the skin. However, when applied to a subset of patients with urticaria pigmentosa, a diagnostic cutoff in the mast cell count lacked sensitivity. Thus, our digital method for counting CD117-immunostained mast cells was highly accurate and was of a significant value for the diagnosis of mastocytosis in the skin. However, some subtypes with low mast cell counts will still require the application of additional diagnostic criteria.

Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses

BACKGROUND

The mitotic count in breast carcinoma is an important prognostic marker. Unfortunately substantial inter- and intra-laboratory variation exists when pathologists manually count mitotic figures. Artificial intelligence (AI) coupled with whole slide imaging offers a potential solution to this problem. The aim of this study was to accordingly critique an AI tool developed to quantify mitotic figures in whole slide images of invasive breast ductal carcinoma.

METHODS

A representative H&E slide from 320 breast invasive ductal carcinoma cases was scanned at 40x magnification. Ten expert pathologists from two academic medical centers labeled mitotic figures in whole slide images to train and validate an AI algorithm to detect and count mitoses. Thereafter, 24 readers of varying expertise were asked to count mitotic figures with and without AI support in 140 high-power fields derived from a separate dataset. Their accuracy and efficiency of performing these tasks were calculated and statistical comparisons performed.

RESULTS

For each experience level the accuracy, precision and sensitivity of counting mitoses by users improved with AI support. There were 21 readers (87.5%) that identified more mitoses using AI support and 13 reviewers (54.2%) that decreased the quantity of falsely flagged mitoses with AI. More time was spent on this task for most participants when not provided with AI support. AI assistance resulted in an overall time savings of 27.8%.

CONCLUSIONS

This study demonstrates that pathology end-users were more accurate and efficient at quantifying mitotic figures in digital images of invasive breast carcinoma with the aid of AI. Higher inter-pathologist agreement with AI assistance suggests that such algorithms can also help standardize practice. Not surprisingly, there is much enthusiasm in pathology regarding the prospect of using AI in routine practice to perform mundane tasks such as counting mitoses.

PathSTORM: a road to early cancer detection

Disruption of chromatin structure could enable early carcinogenesis by facilitating malignant transformation. Using stochastic optical reconstruction microscopy optimized for pathological tissue (PathSTORM), we uncovered a gradual decompaction of higher-order chromatin folding through progressive stages of carcinogenesis. We demonstrated potential detection of pre-cancerous genomic architecture not easily discernible by conventional pathology.

Digital Microscopy, Image Analysis, and Virtual Slide Repository

Advancements in technology and digitization have ushered in novel ways of enhancing tissue-based research via digital microscopy and image analysis. Whole slide imaging scanners enable digitization of histology slides to be stored in virtual slide repositories and to be viewed via computers instead of microscopes. Easier and faster sharing of histologic images for teaching and consultation, improved storage and preservation of quality of stained slides, and annotation of features of interest in the digital slides are just a few of the advantages of this technology. Combined with the development of software for digital image analysis, digital slides further pave the way for the development of tools that extract quantitative data from tissue-based studies. This review introduces digital microscopy and pathology, and addresses technical and scientific considerations in slide scanning, quantitative image analysis, and slide repositories. It also highlights the current state of the technology and factors that need to be taken into account to insure optimal utility, including preanalytical considerations and the importance of involving a pathologist in all major steps along the digital microscopy and pathology workflow.

New Quantitative Digital Image Analysis Method of Histological Features of Carotid Atherosclerotic Plaques

OBJECTIVE

Atherosclerosis and its thrombotic complications are major causes of morbidity and mortality worldwide. Plaque stability assessment is considered to be important for both clinical and fundamental applications. The current gold standard method to investigate plaque stability is performed by histological assessment of plaque features using semi-quantitative classifications. However, these assessments can be limited by subjectivity and variability. Thus, the aim was to develop a new digital image analysis method to measure quantitatively individual plaque features that is more precise than existing semi-quantitative methods.

METHODS

A quantitative method was developed using Image Pro Primer software. Carotid plaque specimens were obtained from patients who underwent carotid endarterectomy and categorised according to stability (definitely stable, probably stable, probably unstable, definitely unstable) based on the gold standard semi-quantitative method that assesses 10 histological plaque features. Using the new quantitative method, plaque features (n = 15) from each stability grade were then analysed by two independent raters. For the semi-quantitative analysis, quadratic weighted Cohen's kappa was used to test intra- and inter-rater reliability, while for the quantitative analysis, intraclass correlation coefficients (ICCs) were assessed.

RESULTS

Intra-rater reliability demonstrated almost perfect agreement between both methods (Cohen's kappa range 0.831-0.969, ICC range 0.848-1.000). However, inter-rater reliability demonstrated mainly fair to moderate agreement (Cohen's kappa range 0.341-0.778) for the semi-quantitative analysis, while the digital image analysis method performed most optimally regarding reproducibility, yielding high ICCs close to 1 (ICC range 0.816-0.999). Using quantitative measurements, a statistically significant proportion of the individual plaque features (p < .05) were re-classified from one grade to another (shift by one) under the semi-quantitative classification.

CONCLUSION

A new quantitative digital image analysis was developed for the accurate assessment of histological plaque features, which demonstrated higher precision than the gold standard semi-quantitative methods, as measured by between and within rater analysis. Moreover, quantitative image analysis of histological plaque features provided more detailed insight into plaque morphology and composition.

Deep Learning in the Biomedical Applications: Recent and Future Status

Deep neural networks represent, nowadays, the most effective machine learning technology in biomedical domain. In this domain, the different areas of interest concern the Omics (study of the genome—genomics—and proteins—transcriptomics, proteomics, and metabolomics), bioimaging (study of biological cell and tissue), medical imaging (study of the human organs by creating visual representations), BBMI (study of the brain and body machine interface) and public and medical health management (PmHM). This paper reviews the major deep learning concepts pertinent to such biomedical applications. Concise overviews are provided for the Omics and the BBMI. We end our analysis with a critical discussion, interpretation and relevant open challenges.

Introduction to Digital Image Analysis in Whole-slide Imaging: A White Paper from the Digital Pathology Association

The advent of whole-slide imaging in digital pathology has brought about the advancement of computer-aided examination of tissue via digital image analysis. Digitized slides can now be easily annotated and analyzed via a variety of algorithms. This study reviews the fundamentals of tissue image analysis and aims to provide pathologists with basic information regarding the features, applications, and general workflow of these new tools. The review gives an overview of the basic categories of software solutions available, potential analysis strategies, technical considerations, and general algorithm readouts. Advantages and limitations of tissue image analysis are discussed, and emerging concepts, such as artificial intelligence and machine learning, are introduced. Finally, examples of how digital image analysis tools are currently being used in diagnostic laboratories, translational research, and drug development are discussed.

Image quilting and wavelet fusion for creation of synthetic microscopy nuclei images

BACKGROUND AND OBJECTIVE

In this study a texture simulation methodology is proposed for composing synthetic tissue microscopy images that could serve as a quantitative gold standard for the evaluation of the reliability, accuracy and performance of segmentation algorithms in computer-aided diagnosis.

METHODS

A library of background and nuclei regions was generated using pre-segmented Haematoxylin and Eosin images of brain tumours. Background image samples were used as input to an image quilting algorithm that produced the synthetic background image. Randomly selected pre-segmented nuclei were randomly fused on the synthetic background using a wavelet-based fusion approach. To investigate whether the produced synthetic images are meaningful and similar to real world images, two different tests were performed, one qualitative by an experienced histopathologist and one quantitative using the normalized mutual information and the Kullback-Leibler tests. To illustrate the challenges that synthetic images may pose to object recognition algorithms, two segmentation methodologies were utilized for nuclei detection, one based on the Otsu thresholding and another based on the seeded region growing approach.

RESULTS

Results showed a satisfactory to good resemblance of the synthetic with the real world images according to both qualitative and quantitative tests. The segmentation accuracy was slightly higher for the seeded region growing algorithm (87.2%) than the Otsu's algorithm (86.3%).

CONCLUSIONS

Since we know the exact coordinates of the regions of interest within the synthesised images, these images could then serve as a 'gold standard' for evaluation of segmentation algorithms in computer-aided diagnosis in tissue microscopy.

Digital image analysis in breast pathology-from image processing techniques to artificial intelligence

Breast cancer is the most common malignant disease in women worldwide. In recent decades, earlier diagnosis and better adjuvant therapy have substantially improved patient outcome. Diagnosis by histopathology has proven to be instrumental to guide breast cancer treatment, but new challenges have emerged as our increasing understanding of cancer over the years has revealed its complex nature. As patient demand for personalized breast cancer therapy grows, we face an urgent need for more precise biomarker assessment and more accurate histopathologic breast cancer diagnosis to make better therapy decisions. The digitization of pathology data has opened the door to faster, more reproducible, and more precise diagnoses through computerized image analysis. Software to assist diagnostic breast pathology through image processing techniques have been around for years. But recent breakthroughs in artificial intelligence (AI) promise to fundamentally change the way we detect and treat breast cancer in the near future. Machine learning, a subfield of AI that applies statistical methods to learn from data, has seen an explosion of interest in recent years because of its ability to recognize patterns in data with less need for human instruction. One technique in particular, known as deep learning, has produced groundbreaking results in many important problems including image classification and speech recognition. In this review, we will cover the use of AI and deep learning in diagnostic breast pathology, and other recent developments in digital image analysis.