Automated mitosis detection in histopathology using morphological and multi-channel statistics features

Deep radial basis function networks with subcategorization for mitosis detection in breast histopathology images

Due to the intra-class diversity of mitotic cells and the morphological overlap with similarly looking imposters, automatic mitosis detection in histopathology slides is still a challenging task. In this paper, we propose a novel mitosis detection model in a weakly supervised way, which consists of a candidate proposal network and a verification network. The candidate proposal network based on patch learning aims to separate both mitotic cells and their mimics from the background as candidate objects, which substantially reduces missed detections in the screening process of candidates. These obtained candidate results are then fed into the verification network for mitosis refinement. The verification network adopts an RBF-based subcategorization scheme to deal with the problems of high intra-class variability of mitosis and the mimics with similar appearance. We utilize the RBF centers to define subcategories containing mitotic cells with similar properties and capture representative RBF center locations through joint training of classification and clustering. Due to the lower intra-class variation within a subcategory, the localized feature space at subcategory level can better characterize a certain type of mitotic figures and can provide a better similarity measurement for distinguishing mitotic cells from nonmitotic cells. Our experiments manifest that this subcategorization scheme helps improve the performance of mitosis detection and achieves state-of-the-art results on the publicly available mitosis datasets using only weak labels.

Efficient mitosis detection: leveraging pre-trained faster R-CNN and cell-level classification

The assessment of mitotic activity is an integral part of the comprehensive evaluation of breast cancer pathology. Understanding the level of tumor dissemination is essential for assessing the severity of the malignancy and guiding appropriate treatment strategies. A pathologist must manually perform the intricate and time-consuming task of counting mitoses by examining biopsy slices stained with Hematoxylin and Eosin (H&E) under a microscope. Mitotic cells can be challenging to distinguish in H&E-stained sections due to limited available datasets and similarities among mitotic and non-mitotic cells. Computer-assisted mitosis detection approaches have simplified the whole procedure by selecting, detecting, and labeling mitotic cells. Traditional detection strategies rely on image processing techniques that apply custom criteria to distinguish between different aspects of an image. Additionally, the automatic feature extraction from histopathology images that exhibit mitosis using neural networks.Additionally, the possibility of automatically extracting features from histopathological images using deep neural networks was investigated. This study examines mitosis detection as an object detection problem using multiple neural networks. From a medical standpoint, mitosis at the tissue level was also investigated utilising pre-trained Faster R-CNN and raw image data. Experiments were done on the MITOS-ATYPIA- 14 dataset and TUPAC16 dataset, and the results were compared to those of other methods described in the literature.

Toward interpretable and generalized mitosis detection in digital pathology using deep learning

Objective

The mitotic activity index is an important prognostic factor in the diagnosis of cancer. The task of mitosis detection is difficult as the nuclei are microscopic in size and partially labeled, and there are many more non-mitotic nuclei compared to mitotic ones. In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method to tackle these challenges.

Methods

Our proposed methodology is inspired from recent research on deep learning and an extensive analysis on the dataset and training pipeline. We first used the MiDoG'22 dataset for training, validation, and testing. We then tested the methodology without fine-tuning on the TUPAC'16 dataset and on a real-time case from Shaukat Khanum Memorial Cancer Hospital and Research Centre.

Results

Our methodology has shown promising results both quantitatively and qualitatively. Quantitatively, our methodology achieved an F1-score of 0.87 on the MiDoG'22 dataset and an F1-score of 0.83 on the TUPAC dataset. Qualitatively, our methodology is generalizable and interpretable across various datasets and clinical settings.

Conclusion

In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method that can accurately predict mitotic nuclei. We illustrate the accuracy, generalizability, and interpretability of our approach across various datasets and clinical settings. Our methodology can speed up the adoption of computer-aided digital pathology in clinical settings.

CCF-GNN: A Unified Model Aggregating Appearance, Microenvironment, and Topology for Pathology Image Classification

Pathology images contain rich information of cell appearance, microenvironment, and topology features for cancer analysis and diagnosis. Among such features, topology becomes increasingly important in analysis for cancer immunotherapy. By analyzing geometric and hierarchically structured cell distribution topology, oncologists can identify densely-packed and cancer-relevant cell communities (CCs) for making decisions. Compared to commonly-used pixel-level Convolution Neural Network (CNN) features and cell-instance-level Graph Neural Network (GNN) features, CC topology features are at a higher level of granularity and geometry. However, topological features have not been well exploited by recent deep learning (DL) methods for pathology image classification due to lack of effective topological descriptors for cell distribution and gathering patterns. In this paper, inspired by clinical practice, we analyze and classify pathology images by comprehensively learning cell appearance, microenvironment, and topology in a fine-to-coarse manner. To describe and exploit topology, we design Cell Community Forest (CCF), a novel graph that represents the hierarchical formulation process of big-sparse CCs from small-dense CCs. Using CCF as a new geometric topological descriptor of tumor cells in pathology images, we propose CCF-GNN, a GNN model that successively aggregates heterogeneous features (e.g., appearance, microenvironment) from cell-instance-level, cell-community-level, into image-level for pathology image classification. Extensive cross-validation experiments show that our method significantly outperforms alternative methods on H&E-stained and immunofluorescence images for disease grading tasks with multiple cancer types. Our proposed CCF-GNN establishes a new topological data analysis (TDA) based method, which facilitates integrating multi-level heterogeneous features of point clouds (e.g., for cells) into a unified DL framework.

SB. Multi CNN based automatic detection of mitotic nuclei in breast histopathological images

In breast cancer diagnosis, the number of mitotic cells in a specific area is an important measure. It indicates how far the tumour has spread, which has consequences for forecasting the aggressiveness of cancer. Mitosis counting is a time-consuming and challenging technique that a pathologist does manually by examining Hematoxylin and Eosin (H&E) stained biopsy slices under a microscope. Due to limited datasets and the resemblance between mitotic and non-mitotic cells, detecting mitosis in H&E stained slices is difficult. By assisting in the screening, identifying, and labelling of mitotic cells, computer-aided mitosis detection technologies make the entire procedure much easier. For computer-aided detection approaches of smaller datasets, pre-trained convolutional neural networks are extensively employed. The usefulness of a multi CNN framework with three pre-trained CNNs is investigated in this research for mitosis detection. Features were collected from histopathology data and identified using VGG16, ResNet50, and DenseNet201 pre-trained networks. The proposed framework utilises all training folders of the MITOS dataset provided for the MITOS-ATYPIA contest 2014 and all the 73 folders of the TUPAC16 dataset. Each pre-trained Convolutional Neural Network model, such as VGG16, ResNet50 and DenseNet201, provides an accuracy of 83.22%, 73.67%, and 81.75%, respectively. Different combinations of these pre-trained CNNs constitute a multi CNN framework. Performance measures of multi CNN consisting of 3 pre-trained CNNs with Linear SVM give 93.81% precision and 92.41% F1-score compared to multi CNN combinations with other classifiers such as Adaboost and Random Forest.

A Novel Deep Learning-Based Mitosis Recognition Approach and Dataset for Uterine Leiomyosarcoma Histopathology

Uterine leiomyosarcoma (ULMS) is the most common sarcoma of the uterus, with both a high malignant potential and poor prognosis. Its diagnosis is sometimes challenging owing to its resemblance to leiomyosarcoma, often being accompanied by benign smooth muscle neoplasms of the uterus. Pathologists diagnose and grade leiomyosarcoma based on three biomarkers (i.e., mitosis count, necrosis, and nuclear atypia). Among these biomarkers, mitosis count is the most important and challenging biomarker. In general, pathologists use the traditional manual counting method for the detection and counting of mitosis. This procedure is very time-consuming, tedious, and subjective. To overcome these challenges, artificial intelligence (AI) based methods have been developed that automatically detect mitosis. In this paper, we propose a new ULMS dataset and an AI-based approach for mitosis detection. We collected our dataset from a local medical facility in collaboration with highly trained pathologists. Preprocessing and annotations are performed using standard procedures, and a deep learning-based method is applied to provide baseline accuracies. The experimental results showed 0.7462 precision, 0.8981 recall, and 0.8151 F1-score. For research and development, the code and dataset have been made publicly available.

Semantic segmentation of human cell nucleus using deep U-Net and other versions of U-Net models

The deep learning models play an essential role in many areas, including medical image analysis. These models extract important features without human intervention. In this paper, we propose a deep convolution neural network, named as deep U-Net model, for the segmentation of the cell nucleus, a critical functional unit that determines the function and structure of the body. The nucleus contains all kinds of DNA, RNA, chromosomes, and genes governing all life activities, and its disorder may lead to different types of diseases such as cancer, heart disease, diabetes, Alzheimer's, etc. If the nucleus structure is known correctly, diseases due to nucleus disorder may be detected early. It may also reduce the drug discovery time if the shape and size of the nucleus are known. We evaluate the performance of the proposed models on the nucleus segmentation data set used by the Data Science Bowl 2018 competition hosted by Kaggle. We compare its performance with that of the U-Net, Attention U-Net, R2U-Net, Attention R2U-Net, and both versions of the U-Net++ with and without supervision, in terms of loss, dice coefficient, dice loss, intersection over union, and accuracy. Our model performs better than the existing models.

MiNuGAN: Dual Segmentation of Mitoses and Nuclei Using Conditional GANs on Multi-center Breast H&E Images

Breast cancer is the second most commonly diagnosed type of cancer among women as of 2021. Grading of histopathological images is used to guide breast cancer treatment decisions and a critical component of this is a mitotic score, which is related to tumor aggressiveness. Manual mitosis counting is an extremely tedious manual task, but automated approaches can be used to overcome inefficiency and subjectivity. In this paper, we propose an automatic mitosis and nuclear segmentation method for a diverse set of H&E breast cancer pathology images. The method is based on a conditional generative adversarial network to segment both mitoses and nuclei at the same time. Architecture optimizations are investigated, including hyper parameters and the addition of a focal loss. The accuracy of the proposed method is investigated using images from multiple centers and scanners, including TUPAC16, ICPR14 and ICPR12 datasets. In TUPAC16, we use 618 carefully annotated images of size 256×256 scanned at 40×. TUPAC16 is used to train the model, and segmentation performance is measured on the test set for both nuclei and mitoses. Results on 200 held-out testing images from the TUPAC16 dataset were mean DSC = 0.784 and 0.721 for nuclear and mitosis, respectively. On 202 ICPR12 images, mitosis segmentation accuracy had a mean DSC = 0.782, indicating the model generalizes well to unseen datasets. For datasets that had mitosis centroid annotations, which included 200 TUPAC16, 202 ICPR12 and 524 ICPR14, a mean F1-score of 0.854 was found indicating high mitosis detection accuracy.

End-to-End Deep Learning-Based Cells Detection in Microscopic Leucorrhea Images

Vaginitis is a prevalent gynecologic disease that threatens millions of women’s health. Although microscopic examination of vaginal discharge is an effective method to identify vaginal infections, manual analysis of microscopic leucorrhea images is extremely time-consuming and labor-intensive. To automate the detection and identification of visible components in microscopic leucorrhea images for early-stage diagnosis of vaginitis, we propose a novel end-to-end deep learning-based cells detection framework using attention-based detection with transformers (DETR) architecture. The transfer learning was applied to speed up the network convergence while maintaining the lowest annotation cost. To address the issue of detection performance degradation caused by class imbalance, the weighted sampler with on-the-fly data augmentation module was integrated into the detection pipeline. Additionally, the multi-head attention mechanism and the bipartite matching loss system of the DETR model perform well in identifying partially overlapping cells in real-time. With our proposed method, the pipeline achieved a mean average precision (mAP) of 86.00% and the average precision (AP) of epithelium, leukocyte, pyocyte, mildew, and erythrocyte was 96.76, 83.50, 74.20, 89.66, and 88.80%, respectively. The average test time for a microscopic leucorrhea image is approximately 72.3 ms. Currently, this cell detection method represents state-of-the-art performance.

Automated knowledge-assisted mitosis cells detection framework in breast histopathology images

Based on the Nottingham Histopathology Grading (NHG) system, mitosis cells detection is one of the important criteria to determine the grade of breast carcinoma. Mitosis cells detection is a challenging task due to the heterogeneous microenvironment of breast histopathology images. Recognition of complex and inconsistent objects in the medical images could be achieved by incorporating domain knowledge in the field of interest. In this study, the strategies of the histopathologist and domain knowledge approach were used to guide the development of the image processing framework for automated mitosis cells detection in breast histopathology images. The detection framework starts with color normalization and hyperchromatic nucleus segmentation. Then, a knowledge-assisted false positive reduction method is proposed to eliminate the false positive (i.e., non-mitosis cells). This stage aims to minimize the percentage of false positive and thus increase the F1-score. Next, features extraction was performed. The mitosis candidates were classified using a Support Vector Machine (SVM) classifier. For evaluation purposes, the knowledge-assisted detection framework was tested using two datasets: a custom dataset and a publicly available dataset (i.e., MITOS dataset). The proposed knowledge-assisted false positive reduction method was found promising by eliminating at least 87.1% of false positive in both the dataset producing promising results in the F1-score. Experimental results demonstrate that the knowledge-assisted detection framework can achieve promising results in F1-score (custom dataset: 89.1%; MITOS dataset: 88.9%) and outperforms the recent works.

Dynamic Learning Rate in Deep CNN Model for Metastasis Detection and Classification of Histopathology Images

Diagnosis of different breast cancer stages using histopathology whole slide images (WSI) is the gold standard in determining the grade of tissue metastasis. Computer-aided diagnosis (CAD) assists medical experts as a second opinion tool in early detection to prevent further proliferation. The field of pathology has advanced so rapidly that it is possible to obtain high-quality images from glass slides. Patches from the region of interest in histopathology images are extracted and trained using artificial neural network models. The trained model primarily analyzes and predicts the histology images for the benign or malignant class to which it belongs. Classification of medical images focuses on the training of models with layers of abstraction to distinguish between these two classes with less false-positive rates. The learning rate is the crucial hyperparameter used during the training of deep convolutional neural networks (DCNN) to improve model accuracy. This work emphasizes the relevance of the dynamic learning rate than the fixed learning rate during the training of networks. The dynamic learning rate varies with preset conditions between the lower and upper boundaries and repeats at different iterations. The performance of the model thus improves and attains comparatively high accuracy with fewer iterations.

Nuclei Segmentation on Histopathology Images of Breast Carcinoma

With the use of computer-aided diagnostic systems, the automatic detection and segmentation of the cell nuclei have become essential in pathology due to cellular nuclei counting and nuclear pleomorphism analysis are critical for the classification and grading of breast cancer histopathology. This work describes a methodology for automatic detection and segmentation of cellular nuclei in breast cancer histopathology images obtained from the BreakHis database, the Standford tissue microarray database, and the Breast Cancer Cell Segmentation database. The proposed scheme is based on the characterization of Hematoxylin and Eosin (H&E) staining, size, and shape features. In addition, we use the information obtained from morphological transformations and adaptive intensity adjustments to detect and separate each cell nucleus detected in the image. The segmentation was carried out by testing the proposed methodology in a histological breast cancer database that provides the associated groundtruth segmentation. Subsequently, the Sørensen-Dice similarity coefficient was calculated to analyze the suitability of the results.Clinical relevance- In this work, the detection and segmentation of cell nuclei in breast cancer histological images are carried out automatically. The method can identify cell nuclei regardless of variations in the level of staining and image magnification. Moreover, a granulometric analysis of the components allows identifying cell clumps and segment them into individual cell nuclei. Improved identification of cell nuclei under different image conditions was demonstrated to reach a sensitivity average of 0.76 ± 0.12. The results provide a base for further and complex processes such as cell counting, feature analysis, and nuclear pleomorphism, which are relevant tasks in the evaluation and diagnostic performed by the expert pathologist.

A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images

The mitotic activity index is a key prognostic measure in tumour grading. Microscopy based detection of mitotic nuclei is a significant overhead and necessitates automation. This work proposes deep CNN based multi-phase mitosis detection framework "MP-MitDet" for mitotic nuclei identification in breast cancer histopathological images. The workflow constitutes: (1) label-refiner, (2) tissue-level mitotic region selection, (3) blob analysis, and (4) cell-level refinement. We developed an automatic label-refiner to represent weak labels with semi-sematic information for training of deep CNNs. A deep instance-based detection and segmentation model is used to explore probable mitotic regions on tissue patches. More probable regions are screened based on blob area and then analysed at cell-level by developing a custom CNN classifier "MitosRes-CNN" to filter false mitoses. The performance of the proposed "MitosRes-CNN" is compared with the state-of-the-art CNNs that are adapted to cell-level discrimination through cross-domain transfer learning and by adding task-specific layers. The performance of the proposed framework shows good discrimination ability in terms of F-score (0.75), recall (0.76), precision (0.71) and area under the precision-recall curve (0.78) on challenging TUPAC16 dataset. Promising results suggest good generalization of the proposed framework that can learn characteristic features from heterogenous mitotic nuclei.

Representation of Differential Learning Method for Mitosis Detection

The breast cancer microscopy images acquire information about the patient’s ailment, and the automated mitotic cell detection outcomes have generally been utilized to ease the massive amount of pathologist’s work and help the pathologists make clinical decisions quickly. Several previous methods were introduced to solve automated mitotic cell count problems. However, they failed to differentiate between mitotic and nonmitotic cells and come up with an imbalance problem, which affects the performance. This paper proposes a Representation Differential Learning Method (RDLM) for mitosis detection through deep learning to detect the accurate mitotic cell area on pathological images. Our proposed method has been divided into two parts: Global bank Feature Pyramid Network (GLB-FPN) and focal loss (FL). The GLB feature fusion method with FPN essentially makes the encoder-decoder pay attention, to further extract the region of interest (ROIs) for mitotic cells. On this basis, we extend the GLB-FPN with a focal loss to mitigate the data imbalance problem during the training stage. Extensive experiments have shown that RDLM significantly outperforms on visualization view and achieves the best performance in quantitative matrices than other proposed approaches on the MITOS-ATYPIA-14 contest dataset. Our framework reaches a 0.692 F1-score. Additionally, RDLM achieves 5% improvements than GLB with FPN in F1-score on the mitosis detection task.

Attention-Guided Multi-Branch Convolutional Neural Network for Mitosis Detection From Histopathological Images

Mitotic count is an important indicator for assessing the invasiveness of breast cancers. Currently, the number of mitoses is manually counted by pathologists, which is both tedious and time-consuming. To address this situation, we propose a fast and accurate method to automatically detect mitosis from the histopathological images. The proposed method can automatically identify mitotic candidates from histological sections for mitosis screening. Specifically, our method exploits deep convolutional neural networks to extract high-level features of mitosis to detect mitotic candidates. Then, we use spatial attention modules to re-encode mitotic features, which allows the model to learn more efficient features. Finally, we use multi-branch classification subnets to screen the mitosis. Compared to existing related methods in literature, our method obtains the best detection results on the dataset of the International Pattern Recognition Conference (ICPR) 2012 Mitosis Detection Competition. Code has been made available at: https://github.com/liushaomin/MitosisDetection.

Piloting a Deep Learning Model for Predicting Nuclear BAP1 Immunohistochemical Expression of Uveal Melanoma from Hematoxylin-and-Eosin Sections

Background

Uveal melanoma (UM) is the most common primary intraocular malignancy in adults. Monosomy 3 and BAP1 mutation are strong prognostic factors predicting metastatic risk in UM. Nuclear BAP1 (nBAP1) expression is a close immunohistochemical surrogate for both genetic alterations. Not all laboratories perform routine BAP1 immunohistochemistry or genetic testing, and rely mainly on clinical information and anatomic/morphologic analyses for UM prognostication. The purpose of our study was to pilot deep learning (DL) techniques to predict nBAP1 expression on whole slide images (WSIs) of hematoxylin and eosin (H&E) stained UM sections.

Methods

One hundred forty H&E-stained UMs were scanned at 40 × magnification, using commercially available WSI image scanners. The training cohort comprised 66 BAP1⁺ and 74 BAP1⁻ UM, with known chromosome 3 status and clinical outcomes. Nonoverlapping areas of three different dimensions (512 × 512, 1024 × 1024, and 2048 × 2048 pixels) for comparison were extracted from tumor regions in each WSI, and were resized to 256 × 256 pixels. Deep convolutional neural networks (Resnet18 pre-trained on Imagenet) and auto-encoder-decoders (U-Net) were trained to predict nBAP1 expression of these patches. Trained models were tested on the patches cropped from a test cohort of WSIs of 16 BAP1⁺ and 28 BAP1⁻ UM cases.

Results

The trained model with best performance achieved area under the curve values of 0.90 for patches and 0.93 for slides on the test set.

Conclusions

Our results show the effectiveness of DL for predicting nBAP1 expression in UM on the basis of H&E sections only.

Translational Relevance

Our pilot demonstrates a high capacity of artificial intelligence-related techniques for automated prediction on the basis of histomorphology, and may be translatable into routine histology laboratories.

Mitosis detection in breast cancer histopathology images using hybrid feature space

Breast Cancer grading is a challenging task as regards image analysis, which is normally based on mitosis count rate. The mitotic count provides an estimate of aggressiveness of the tumor. The detection of mitosis is a challenging task because in a frame of slides at X40 magnification, there are hundreds of nuclei containing few mitotic nuclei. However, manual counting of mitosis by pathologists is a difficult and time intensive job, moreover conventional method rely mainly on the shape, color, and/or texture features as well as pathologist experience. The objective of this study is to accept the atypaia-2014 mitosis detection challenge, automate the process of mitosis detection and a proposal of a hybrid feature space that provides better discrimination of mitotic and non-mitotic nuclei by combining color features with morphological and texture features. To exploit color channels, they were first selected, and then normalized and cumulative histograms were computed in wavelet domain. A detailed analysis presented on these features in different color channels of respective color spaces using Random Forest (RF) and Support Vector Machine (SVM) classifiers. The proposed hybrid feature space when used with SVM classifier achieved a detection rate of 78.88% and F-measure of 72.07%. Our results, especially high detection rate, indicate that proposed hybrid feature space model contains discriminant information for mitotic nuclei, being therefore a very capable are for exploration to improve the quality of the diagnostic assistance in histopathology.

MaskMitosis: a deep learning framework for fully supervised, weakly supervised, and unsupervised mitosis detection in histopathology images

Counting the mitotic cells in histopathological cancerous tissue areas is the most relevant indicator of tumor grade in aggressive breast cancer diagnosis. In this paper, we propose a robust and accurate technique for the automatic detection of mitoses from histological breast cancer slides using the multi-task deep learning framework for object detection and instance segmentation Mask RCNN. Our mitosis detection and instance segmentation framework is deployed for two main tasks: it is used as a detection network to perform mitosis localization and classification in the fully annotated mitosis datasets (i.e., the pixel-level annotated datasets), and it is used as a segmentation network to estimate the mitosis mask labels for the weakly annotated mitosis datasets (i.e., the datasets with centroid-pixel labels only). We evaluate our approach on the fully annotated 2012 ICPR grand challenge dataset and the weakly annotated 2014 ICPR MITOS-ATYPIA challenge dataset. Our evaluation experiments show that we can obtain the highest F-score of 0.863 on the 2012 ICPR dataset by applying the mitosis detection and instance segmentation model trained on the pixel-level labels provided by this dataset. For the weakly annotated 2014 ICPR dataset, we first employ the mitosis detection and instance segmentation model trained on the fully annotated 2012 ICPR dataset to segment the centroid-pixel annotated mitosis ground truths, and produce the mitosis mask and bounding box labels. These estimated labels are then used to train another mitosis detection and instance segmentation model for mitosis detection on the 2014 ICPR dataset. By adopting this two-stage framework, our method outperforms all state-of-the-art mitosis detection approaches on the 2014 ICPR dataset by achieving an F-score of 0.475. Moreover, we show that the proposed framework can also perform unsupervised mitosis detection through the estimation of pseudo labels for an unlabeled dataset and it can achieve promising detection results. Code has been made available at: https://github.com/MeriemSebai/MaskMitosis. Graphical Abstract Overview of MaskMitosis framework.

Artificial Intelligence-Based Mitosis Detection in Breast Cancer Histopathology Images Using Faster R-CNN and Deep CNNs

Breast cancer is the leading cause of mortality in women. Early diagnosis of breast cancer can reduce the mortality rate. In the diagnosis, the mitotic cell count is an important biomarker for predicting the aggressiveness, prognosis, and grade of breast cancer. In general, pathologists manually examine histopathology images under high-resolution microscopes for the detection of mitotic cells. However, because of the minute differences between the mitotic and normal cells, this process is tiresome, time-consuming, and subjective. To overcome these challenges, artificial-intelligence-based (AI-based) techniques have been developed which automatically detect mitotic cells in the histopathology images. Such AI techniques accelerate the diagnosis and can be used as a second-opinion system for a medical doctor. Previously, conventional image-processing techniques were used for the detection of mitotic cells, which have low accuracy and high computational cost. Therefore, a number of deep-learning techniques that demonstrate outstanding performance and low computational cost were recently developed; however, they still require improvement in terms of accuracy and reliability. Therefore, we present a multistage mitotic-cell-detection method based on Faster region convolutional neural network (Faster R-CNN) and deep CNNs. Two open datasets (international conference on pattern recognition (ICPR) 2012 and ICPR 2014 (MITOS-ATYPIA-14)) of breast cancer histopathology were used in our experiments. The experimental results showed that our method achieves the state-of-the-art results of 0.876 precision, 0.841 recall, and 0.858 F1-measure for the ICPR 2012 dataset, and 0.848 precision, 0.583 recall, and 0.691 F1-measure for the ICPR 2014 dataset, which were higher than those obtained using previous methods. Moreover, we tested the generalization capability of our technique by testing on the tumor proliferation assessment challenge 2016 (TUPAC16) dataset and found that our technique also performs well in a cross-dataset experiment which proved the generalization capability of our proposed technique.

Training Convolutional Neural Networks and Compressed Sensing End-to-End for Microscopy Cell Detection

Automated cell detection and localization from microscopy images are significant tasks in biomedical research and clinical practice. In this paper, we design a new cell detection and localization algorithm that combines deep convolutional neural network (CNN) and compressed sensing (CS) or sparse coding (SC) for end-to-end training. We also derive, for the first time, a backpropagation rule, which is applicable to train any algorithm that implements a sparse code recovery layer. The key innovation behind our algorithm is that the cell detection task is structured as a point object detection task in computer vision, where the cell centers (i.e., point objects) occupy only a tiny fraction of the total number of pixels in an image. Thus, we can apply compressed sensing (or equivalently SC) to compactly represent a variable number of cells in a projected space. Subsequently, CNN regresses this compressed vector from the input microscopy image. The SC/CS recovery algorithm ( L ₁ optimization) can then recover sparse cell locations from the output of CNN. We train this entire processing pipeline end-to-end and demonstrate that end-to-end training improves accuracy over a training paradigm that treats CNN and CS-recovery layers separately. We have validated our algorithm on five benchmark datasets with excellent results.

Computer-Assisted Nuclear Atypia Scoring of Breast Cancer: a Preliminary Study

Inter-pathologist agreement for nuclear atypia scoring of breast cancer is poor. To address this problem, previous studies suggested some criteria for describing the variations appearance of tumor cells relative to normal cells. However, these criteria were still assessed subjectively by pathologists. Previous studies used quantitative computer-extracted features for scoring. However, application of these tools is limited as further improvement in their accuracy is required. This study proposes COMPASS (COMputer-assisted analysis combined with Pathologist's ASSessment) for reproducible nuclear atypia scoring. COMPASS relies on both cytological criteria assessed subjectively by pathologists as well as computer-extracted textural features. Using machine learning, COMPASS combines these two sets of features and output nuclear atypia score. COMPASS's performance was evaluated using 300 images for which expert-consensus derived reference nuclear pleomorphism scores were available, and they were scanned by two scanners from different vendors. A personalized model was built for three pathologists who gave scores to six atypia-related criteria for each image. Leave-one-out cross validation (LOOCV) was used. COMPASS was trained and tested for each pathologist separately. Percentage agreement between COMPASS and the reference nuclear scores was 93.8%, 92.9%, and 93.1% for three pathologists. COMPASS's performance in nuclear grading was almost identical for both scanners, with Cohen's kappa ranging from 0.80 to 0.86 for different pathologists and different scanners. Independently, the images were also assessed by two experienced senior pathologists. Cohen's kappa of COMPASS was comparable to the Cohen's kappa for two senior pathologists (0.79 and 0.68).

Efficient automated detection of mitotic cells from breast histological images using deep convolution neutral network with wavelet decomposed patches

In medical practice, the mitotic cell count from histological images acts as a proliferative marker for cancer diagnosis. Therefore, an accurate method for detecting mitotic cells in histological images is essential for cancer screening. Manual evaluation of clinically relevant image features that might reflect mitotic cells in histological images is time-consuming and error prone, due to the heterogeneous physical characteristics of mitotic cells. Computer-assisted automated detection of mitotic cells could overcome these limitations of manual analysis and act as a useful tool for pathologists to make cancer diagnoses efficiently and accurately. Here, we propose a new approach for mitotic cell detection in breast histological images that uses a deep convolution neural network (CNN) with wavelet decomposed image patches. In this approach, raw image patches of 81 × 81 pixels are decomposed to patches of 21 × 21 pixels using Haar wavelet and subsequently used in developing a deep CNN model for automated detection of mitotic cells. The decomposition step reduces convolution time for mitotic cell detection relative to the use of raw image patches in conventional CNN models. The proposed deep network was tested using the MITOS (ICPR2012) and MITOS-ATYPIA-14 breast cancer histological datasets and shown to outperform existing algorithms for mitotic cell detection. Overall, our method improves the performance and reduces the computational burden of conventional deep CNN approaches for mitotic cell detection.

A novel computational method for automatic segmentation, quantification and comparative analysis of immunohistochemically labeled tissue sections

Background

In the clinical practice, the objective quantification of histological results is essential not only to define objective and well-established protocols for diagnosis, treatment, and assessment, but also to ameliorate disease comprehension.

Software

The software MIAQuant_Learn presented in this work segments, quantifies and analyzes markers in histochemical and immunohistochemical images obtained by different biological procedures and imaging tools. MIAQuant_Learn employs supervised learning techniques to customize the marker segmentation process with respect to any marker color appearance. Our software expresses the location of the segmented markers with respect to regions of interest by mean-distance histograms, which are numerically compared by measuring their intersection. When contiguous tissue sections stained by different markers are available, MIAQuant_Learn aligns them and overlaps the segmented markers in a unique image enabling a visual comparative analysis of the spatial distribution of each marker (markers’ relative location). Additionally, it computes novel measures of markers’ co-existence in tissue volumes depending on their density.

Conclusions

Applications of MIAQuant_Learn in clinical research studies have proven its effectiveness as a fast and efficient tool for the automatic extraction, quantification and analysis of histological sections. It is robust with respect to several deficits caused by image acquisition systems and produces objective and reproducible results. Thanks to its flexibility, MIAQuant_Learn represents an important tool to be exploited in basic research where needs are constantly changing.

Digital image analysis in breast pathology-from image processing techniques to artificial intelligence

Breast cancer is the most common malignant disease in women worldwide. In recent decades, earlier diagnosis and better adjuvant therapy have substantially improved patient outcome. Diagnosis by histopathology has proven to be instrumental to guide breast cancer treatment, but new challenges have emerged as our increasing understanding of cancer over the years has revealed its complex nature. As patient demand for personalized breast cancer therapy grows, we face an urgent need for more precise biomarker assessment and more accurate histopathologic breast cancer diagnosis to make better therapy decisions. The digitization of pathology data has opened the door to faster, more reproducible, and more precise diagnoses through computerized image analysis. Software to assist diagnostic breast pathology through image processing techniques have been around for years. But recent breakthroughs in artificial intelligence (AI) promise to fundamentally change the way we detect and treat breast cancer in the near future. Machine learning, a subfield of AI that applies statistical methods to learn from data, has seen an explosion of interest in recent years because of its ability to recognize patterns in data with less need for human instruction. One technique in particular, known as deep learning, has produced groundbreaking results in many important problems including image classification and speech recognition. In this review, we will cover the use of AI and deep learning in diagnostic breast pathology, and other recent developments in digital image analysis.

Efficient and robust cell detection: A structured regression approach

Efficient and robust cell detection serves as a critical prerequisite for many subsequent biomedical image analysis methods and computer-aided diagnosis (CAD). It remains a challenging task due to touching cells, inhomogeneous background noise, and large variations in cell sizes and shapes. In addition, the ever-increasing amount of available datasets and the high resolution of whole-slice scanned images pose a further demand for efficient processing algorithms. In this paper, we present a novel structured regression model based on a proposed fully residual convolutional neural network for efficient cell detection. For each testing image, our model learns to produce a dense proximity map that exhibits higher responses at locations near cell centers. Our method only requires a few training images with weak annotations (just one dot indicating the cell centroids). We have extensively evaluated our method using four different datasets, covering different microscopy staining methods (e.g., H & E or Ki-67 staining) or image acquisition techniques (e.g., bright-filed image or phase contrast). Experimental results demonstrate the superiority of our method over existing state of the art methods in terms of both detection accuracy and running time.

Efficient deep learning model for mitosis detection using breast histopathology images

Mitosis detection is one of the critical factors of cancer prognosis, carrying significant diagnostic information required for breast cancer grading. It provides vital clues to estimate the aggressiveness and the proliferation rate of the tumour. The manual mitosis quantification from whole slide images is a very labor-intensive and challenging task. The aim of this study is to propose a supervised model to detect mitosis signature from breast histopathology WSI images. The model has been designed using deep learning architecture with handcrafted features. We used handcrafted features issued from previous medical challenges MITOS @ ICPR 2012, AMIDA-13 and projects (MICO ANR TecSan) expertise. The deep learning architecture mainly consists of five convolution layers, four max-pooling layers, four rectified linear units (ReLU), and two fully connected layers. ReLU has been used after each convolution layer as an activation function. Dropout layer has been included after first fully connected layer to avoid overfitting. Handcrafted features mainly consist of morphological, textural and intensity features. The proposed architecture has shown to have an improved 92% precision, 88% recall and 90% F-score. Prospectively, the proposed model will be very beneficial in routine exam, providing pathologists with efficient and - as we will prove - effective second opinion for breast cancer grading from whole slide images. Last but not the least, this model could lead junior and senior pathologists, as medical researchers, to a superior understanding and evaluation of breast cancer stage and genesis.

Two-phase deep convolutional neural network for reducing class skewness in histopathological images based breast cancer detection

Different types of breast cancer are affecting lives of women across the world. Common types include Ductal carcinoma in situ (DCIS), Invasive ductal carcinoma (IDC), Tubular carcinoma, Medullary carcinoma, and Invasive lobular carcinoma (ILC). While detecting cancer, one important factor is mitotic count - showing how rapidly the cells are dividing. But the class imbalance problem, due to the small number of mitotic nuclei in comparison to the overwhelming number of non-mitotic nuclei, affects the performance of classification models. This work presents a two-phase model to mitigate the class biasness issue while classifying mitotic and non-mitotic nuclei in breast cancer histopathology images through a deep convolutional neural network (CNN). First, nuclei are segmented out using blue ratio and global binary thresholding. In Phase-1 a CNN is then trained on the segmented out 80×80 pixel patches based on a standard dataset. Hard non-mitotic examples are identified and augmented; mitotic examples are oversampled by rotation and flipping; whereas non-mitotic examples are undersampled by blue ratio histogram based k-means clustering. Based on this information from Phase-1, the dataset is modified for Phase-2 in order to reduce the effects of class imbalance. The proposed CNN architecture and data balancing technique yielded an F-measure of 0.79, and outperformed all the methods relying on specific handcrafted features, as well as those using a combination of handcrafted and CNN-generated features.

Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent

With the increasing ability to routinely and rapidly digitize whole slide images with slide scanners, there has been interest in developing computerized image analysis algorithms for automated detection of disease extent from digital pathology images. The manual identification of presence and extent of breast cancer by a pathologist is critical for patient management for tumor staging and assessing treatment response. However, this process is tedious and subject to inter- and intra-reader variability. For computerized methods to be useful as decision support tools, they need to be resilient to data acquired from different sources, different staining and cutting protocols and different scanners. The objective of this study was to evaluate the accuracy and robustness of a deep learning-based method to automatically identify the extent of invasive tumor on digitized images. Here, we present a new method that employs a convolutional neural network for detecting presence of invasive tumor on whole slide images. Our approach involves training the classifier on nearly 400 exemplars from multiple different sites, and scanners, and then independently validating on almost 200 cases from The Cancer Genome Atlas. Our approach yielded a Dice coefficient of 75.86%, a positive predictive value of 71.62% and a negative predictive value of 96.77% in terms of pixel-by-pixel evaluation compared to manually annotated regions of invasive ductal carcinoma.

Nuclear spatial and spectral features based evolutionary method for meningioma subtypes classification in histopathology

Meningioma subtypes classification is a real-world multiclass problem from the realm of neuropathology. The major challenge in solving this problem is the inherent complexity due to high intra-class variability and low inter-class variation in tissue samples. The development of computational methods to assist pathologists in characterization of these tissue samples would have great diagnostic and prognostic value. In this article, we proposed an optimized evolutionary framework for the classification of benign meningioma into four subtypes. This framework investigates the imperative role of RGB color channels for discrimination of tumor subtypes and compute structural, statistical and spectral phenotypes. An evolutionary technique, Genetic Algorithm, in combination with Support Vector Machine is applied to tune classifier parameters and to select the best possible combination of extracted phenotypes that improved the classification accuracy (94.88%) on meningioma histology dataset, provided by the Institute of Neuropathology, Bielefeld. These statistics show that computational framework can robustly discriminate four subtypes of benign meningioma and may aid pathologists in the diagnosis and classification of these lesions.

Automatic extraction of cell nuclei from H&E-stained histopathological images

Extraction of cell nuclei from hematoxylin and eosin (H&E)-stained histopathological images is an essential preprocessing step in computerized image analysis for disease detection, diagnosis, and prognosis. We present an automated cell nuclei segmentation approach that works with H&E-stained images. A color deconvolution algorithm was first applied to the image to get the hematoxylin channel. Using a morphological operation and thresholding technique on the hematoxylin channel image, candidate target nuclei and background regions were detected, which were then used as markers for a marker-controlled watershed transform segmentation algorithm. Moreover, postprocessing was conducted to split the touching nuclei. For each segmented region from the previous steps, the regional maximum value positions were identified as potential nuclei centers. These maximum values were further grouped into [Formula: see text]-clusters, and the locations within each cluster were connected with the minimum spanning tree technique. Then, these connected positions were utilized as new markers for a watershed segmentation approach. The final number of nuclei at each region was determined by minimizing an objective function that iterated all of the possible [Formula: see text]-values. The proposed method was applied to the pathological images of the tumor tissues from The Cancer Genome Atlas study. Experimental results show that the proposed method can lead to promising results in terms of segmentation accuracy and separation of touching nuclei.

Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

With the advance of digital pathology, image analysis has begun to show its advantages in information analysis of hematoxylin and eosin histopathology images. Generally, histological features in hematoxylin and eosin images are measured to evaluate tumor grade and prognosis for breast cancer. This review summarized recent works in image analysis of hematoxylin and eosin histopathology images for breast cancer prognosis. First, prognostic factors for breast cancer based on hematoxylin and eosin histopathology images were summarized. Then, usual procedures of image analysis for breast cancer prognosis were systematically reviewed, including image acquisition, image preprocessing, image detection and segmentation, and feature extraction. Finally, the prognostic value of image features and image feature–based prognostic models was evaluated. Moreover, we discussed the issues of current analysis, and some directions for future research.

DCAN: Deep contour-aware networks for object instance segmentation from histology images

In histopathological image analysis, the morphology of histological structures, such as glands and nuclei, has been routinely adopted by pathologists to assess the malignancy degree of adenocarcinomas. Accurate detection and segmentation of these objects of interest from histology images is an essential prerequisite to obtain reliable morphological statistics for quantitative diagnosis. While manual annotation is error-prone, time-consuming and operator-dependant, automated detection and segmentation of objects of interest from histology images can be very challenging due to the large appearance variation, existence of strong mimics, and serious degeneration of histological structures. In order to meet these challenges, we propose a novel deep contour-aware network (DCAN) under a unified multi-task learning framework for more accurate detection and segmentation. In the proposed network, multi-level contextual features are explored based on an end-to-end fully convolutional network (FCN) to deal with the large appearance variation. We further propose to employ an auxiliary supervision mechanism to overcome the problem of vanishing gradients when training such a deep network. More importantly, our network can not only output accurate probability maps of histological objects, but also depict clear contours simultaneously for separating clustered object instances, which further boosts the segmentation performance. Our method ranked the first in two histological object segmentation challenges, including 2015 MICCAI Gland Segmentation Challenge and 2015 MICCAI Nuclei Segmentation Challenge. Extensive experiments on these two challenging datasets demonstrate the superior performance of our method, surpassing all the other methods by a significant margin.

Computer-based image analysis in breast pathology

Whole slide imaging (WSI) has the potential to be utilized in telepathology, teleconsultation, quality assurance, clinical education, and digital image analysis to aid pathologists. In this paper, the potential added benefits of computer-assisted image analysis in breast pathology are reviewed and discussed. One of the major advantages of WSI systems is the possibility of doing computer-based image analysis on the digital slides. The purpose of computer-assisted analysis of breast virtual slides can be (i) segmentation of desired regions or objects such as diagnostically relevant areas, epithelial nuclei, lymphocyte cells, tubules, and mitotic figures, (ii) classification of breast slides based on breast cancer (BCa) grades, the invasive potential of tumors, or cancer subtypes, (iii) prognosis of BCa, or (iv) immunohistochemical quantification. While encouraging results have been achieved in this area, further progress is still required to make computer-based image analysis of breast virtual slides acceptable for clinical practice.

Detection of mitotic nuclei in breast histopathology images using localized ACM and Random Kitchen Sink based classifier

The exact measure of mitotic nuclei is a crucial parameter in breast cancer grading and prognosis. This can be achieved by improving the mitotic detection accuracy by careful design of segmentation and classification techniques. In this paper, segmentation of nuclei from breast histopathology images are carried out by Localized Active Contour Model (LACM) utilizing bio-inspired optimization techniques in the detection stage, in order to handle diffused intensities present along object boundaries. Further, the application of a new optimal machine learning algorithm capable of classifying strong non-linear data such as Random Kitchen Sink (RKS), shows improved classification performance. The proposed method has been tested on Mitosis detection in breast cancer histological images (MITOS) dataset provided for MITOS-ATYPIA CONTEST 2014. The proposed framework achieved 95% recall, 98% precision and 96% F-score.

Clinically-inspired automatic classification of ovarian carcinoma subtypes

CONTEXT

It has been shown that ovarian carcinoma subtypes are distinct pathologic entities with differing prognostic and therapeutic implications. Histotyping by pathologists has good reproducibility, but occasional cases are challenging and require immunohistochemistry and subspecialty consultation. Motivated by the need for more accurate and reproducible diagnoses and to facilitate pathologists' workflow, we propose an automatic framework for ovarian carcinoma classification.

MATERIALS AND METHODS

Our method is inspired by pathologists' workflow. We analyse imaged tissues at two magnification levels and extract clinically-inspired color, texture, and segmentation-based shape descriptors using image-processing methods. We propose a carefully designed machine learning technique composed of four modules: A dissimilarity matrix, dimensionality reduction, feature selection and a support vector machine classifier to separate the five ovarian carcinoma subtypes using the extracted features.

RESULTS

This paper presents the details of our implementation and its validation on a clinically derived dataset of eighty high-resolution histopathology images. The proposed system achieved a multiclass classification accuracy of 95.0% when classifying unseen tissues. Assessment of the classifier's confusion (confusion matrix) between the five different ovarian carcinoma subtypes agrees with clinician's confusion and reflects the difficulty in diagnosing endometrioid and serous carcinomas.

CONCLUSIONS

Our results from this first study highlight the difficulty of ovarian carcinoma diagnosis which originate from the intrinsic class-imbalance observed among subtypes and suggest that the automatic analysis of ovarian carcinoma subtypes could be valuable to clinician's diagnostic procedure by providing a second opinion.

Using Automated Image Analysis Algorithms to Distinguish Normal, Aberrant, and Degenerate Mitotic Figures Induced by Eg5 Inhibition

Modulation of the cell cycle may underlie the toxicologic or pharmacologic responses of a potential therapeutic agent and contributes to decisions on its preclinical and clinical safety and efficacy. The descriptive and quantitative assessment of normal, aberrant, and degenerate mitotic figures in tissue sections is an important end point characterizing the effect of xenobiotics on the cell cycle. Historically, pathologists used manual counting and special staining visualization techniques such as immunohistochemistry for quantification of normal, aberrant, and degenerate mitotic figures. We designed an automated image analysis algorithm for measuring these mitotic figures in hematoxylin and eosin (H&E)-stained sections. Algorithm validation methods used data generated from a subcutaneous human transitional cell carcinoma xenograft model in nude rats treated with the cell cycle inhibitor Eg5. In these studies, we scanned and digitized H&E-stained xenografts and applied a complex ruleset of sequential mathematical filters and shape discriminators for classification of cell populations demonstrating normal, aberrant, or degenerate mitotic figures. The resultant classification system enabled the representations of three identifiable degrees of morphological change associated with tumor differentiation and compound effects. The numbers of mitotic figure variants and mitotic indices data generated corresponded to a manual assessment by a pathologist and supported automated algorithm verification and application for both efficacy and toxicity studies.

A Global Covariance Descriptor for Nuclear Atypia Scoring in Breast Histopathology Images

Nuclear atypia scoring is a diagnostic measure commonly used to assess tumor grade of various cancers, including breast cancer. It provides a quantitative measure of deviation in visual appearance of cell nuclei from those in normal epithelial cells. In this paper, we present a novel image-level descriptor for nuclear atypia scoring in breast cancer histopathology images. The method is based on the region covariance descriptor that has recently become a popular method in various computer vision applications. The descriptor in its original form is not suitable for classification of histopathology images as cancerous histopathology images tend to possess diversely heterogeneous regions in a single field of view. Our proposed image-level descriptor, which we term as the geodesic mean of region covariance descriptors, possesses all the attractive properties of covariance descriptors lending itself to tractable geodesic-distance-based k-nearest neighbor classification using efficient kernels. The experimental results suggest that the proposed image descriptor yields high classification accuracy compared to a variety of widely used image-level descriptors.