Mitotic figure recognition: agreement among pathologists and computerized detector

Deep learning supported mitoses counting on whole slide images: A pilot study for validating breast cancer grading in the clinical workflow

Introduction

Breast cancer (BC) prognosis is largely influenced by histopathological grade, assessed according to the Nottingham modification of Bloom-Richardson (BR). Mitotic count (MC) is a component of histopathological grading but is prone to subjectivity. This study investigated whether mitoses counting in BC using digital whole slide images (WSI) compares better to light microscopy (LM) when assisted by artificial intelligence (AI), and to which extent differences in digital MC (AI assisted or not) result in BR grade variations.

Methods

Fifty BC patients with paired core biopsies and resections were randomly selected. Component scores for BR grade were extracted from pathology reports. MC was assessed using LM, WSI, and AI. Different modalities (LM-MC, WSI-MC, and AI-MC) were analyzed for correlation with scatterplots and linear regression, and for agreement in final BR with Cohen's κ.

Results

MC modalities strongly correlated in both biopsies and resections: LM-MC and WSI-MC (R² 0.85 and 0.83, respectively), LM-MC and AI-MC (R² 0.85 and 0.95), and WSI-MC and AI-MC (R² 0.77 and 0.83). Agreement in BR between modalities was high in both biopsies and resections: LM-MC and WSI-MC (κ 0.93 and 0.83, respectively), LM-MC and AI-MC (κ 0.89 and 0.83), and WSI-MC and AI-MC (κ 0.96 and 0.73).

Conclusion

This first validation study shows that WSI-MC may compare better to LM-MC when using AI. Agreement between BR grade based on the different mitoses counting modalities was high. These results suggest that mitoses counting on WSI can well be done, and validate the presented AI algorithm for pathologist supervised use in daily practice. Further research is required to advance our knowledge of AI-MC, but it appears at least non-inferior to LM-MC.

An Open-Source AI Framework for the Analysis of Single Cells in Whole-Slide Images with a Note on CD276 in Glioblastoma

Routine examination of entire histological slides at cellular resolution poses a significant if not insurmountable challenge to human observers. However, high-resolution data such as the cellular distribution of proteins in tissues, e.g., those obtained following immunochemical staining, are highly desirable. Our present study extends the applicability of the PathoFusion framework to the cellular level. We illustrate our approach using the detection of CD276 immunoreactive cells in glioblastoma as an example. Following automatic identification by means of PathoFusion's bifocal convolutional neural network (BCNN) model, individual cells are automatically profiled and counted. Only discriminable cells selected through data filtering and thresholding were segmented for cell-level analysis. Subsequently, we converted the detection signals into the corresponding heatmaps visualizing the distribution of the detected cells in entire whole-slide images of adjacent H&E-stained sections using the Discrete Wavelet Transform (DWT). Our results demonstrate that PathoFusion is capable of autonomously detecting and counting individual immunochemically labelled cells with a high prediction performance of 0.992 AUC and 97.7% accuracy. The data can be used for whole-slide cross-modality analyses, e.g., relationships between immunochemical signals and anaplastic histological features. PathoFusion has the potential to be applied to additional problems that seek to correlate heterogeneous data streams and to serve as a clinically applicable, weakly supervised system for histological image analyses in (neuro)pathology.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research

Canine mammary carcinoma (CMC) has been used as a model to investigate the pathogenesis of human breast cancer and the same grading scheme is commonly used to assess tumor malignancy in both. One key component of this grading scheme is the density of mitotic figures (MF). Current publicly available datasets on human breast cancer only provide annotations for small subsets of whole slide images (WSIs). We present a novel dataset of 21 WSIs of CMC completely annotated for MF. For this, a pathologist screened all WSIs for potential MF and structures with a similar appearance. A second expert blindly assigned labels, and for non-matching labels, a third expert assigned the final labels. Additionally, we used machine learning to identify previously undetected MF. Finally, we performed representation learning and two-dimensional projection to further increase the consistency of the annotations. Our dataset consists of 13,907 MF and 36,379 hard negatives. We achieved a mean F1-score of 0.791 on the test set and of up to 0.696 on a human breast cancer dataset.

Measurement(s)	Mitotic Figure • Slide Image • non-mitotic structures • anatomical phenotype annotation
Technology Type(s)	Pathology Report • hematoxylin and eosin stain • machine learning
Factor Type(s)	breast cancer tissue
Sample Characteristic - Organism	Canis

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.13182857

Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses

BACKGROUND

The mitotic count in breast carcinoma is an important prognostic marker. Unfortunately substantial inter- and intra-laboratory variation exists when pathologists manually count mitotic figures. Artificial intelligence (AI) coupled with whole slide imaging offers a potential solution to this problem. The aim of this study was to accordingly critique an AI tool developed to quantify mitotic figures in whole slide images of invasive breast ductal carcinoma.

METHODS

A representative H&E slide from 320 breast invasive ductal carcinoma cases was scanned at 40x magnification. Ten expert pathologists from two academic medical centers labeled mitotic figures in whole slide images to train and validate an AI algorithm to detect and count mitoses. Thereafter, 24 readers of varying expertise were asked to count mitotic figures with and without AI support in 140 high-power fields derived from a separate dataset. Their accuracy and efficiency of performing these tasks were calculated and statistical comparisons performed.

RESULTS

For each experience level the accuracy, precision and sensitivity of counting mitoses by users improved with AI support. There were 21 readers (87.5%) that identified more mitoses using AI support and 13 reviewers (54.2%) that decreased the quantity of falsely flagged mitoses with AI. More time was spent on this task for most participants when not provided with AI support. AI assistance resulted in an overall time savings of 27.8%.

CONCLUSIONS

This study demonstrates that pathology end-users were more accurate and efficient at quantifying mitotic figures in digital images of invasive breast carcinoma with the aid of AI. Higher inter-pathologist agreement with AI assistance suggests that such algorithms can also help standardize practice. Not surprisingly, there is much enthusiasm in pathology regarding the prospect of using AI in routine practice to perform mundane tasks such as counting mitoses.

A large-scale dataset for mitotic figure assessment on whole slide images of canine cutaneous mast cell tumor

We introduce a novel, large-scale dataset for microscopy cell annotations. The dataset includes 32 whole slide images (WSI) of canine cutaneous mast cell tumors, selected to include both low grade cases as well as high grade cases. The slides have been completely annotated for mitotic figures and we provide secondary annotations for neoplastic mast cells, inflammatory granulocytes, and mitotic figure look-alikes. Additionally to a blinded two-expert manual annotation with consensus, we provide an algorithm-aided dataset, where potentially missed mitotic figures were detected by a deep neural network and subsequently assessed by two human experts. We included 262,481 annotations in total, out of which 44,880 represent mitotic figures. For algorithmic validation, we used a customized RetinaNet approach, followed by a cell classification network. We find F1-Scores of 0.786 and 0.820 for the manually labelled and the algorithm-aided dataset, respectively. The dataset provides, for the first time, WSIs completely annotated for mitotic figures and thus enables assessment of mitosis detection algorithms on complete WSIs as well as region of interest detection algorithms.

Determining Image Processing Features Describing the Appearance of Challenging Mitotic Figures and Miscounted Nonmitotic Objects

Context:

Previous studies showed that the agreement among pathologists in recognition of mitoses in breast slides is fairly modest.

Aims:

Determining the significantly different quantitative features among easily identifiable mitoses, challenging mitoses, and miscounted nonmitoses within breast slides and identifying which color spaces capture the difference among groups better than others.

Materials and Methods:

The dataset contained 453 mitoses and 265 miscounted objects in breast slides. The mitoses were grouped into three categories based on the confidence degree of three pathologists who annotated them. The mitoses annotated as “probably a mitosis” by the majority of pathologists were considered as the challenging category. The miscounted objects were recognized as a mitosis or probably a mitosis by only one of the pathologists. The mitoses were segmented using k-means clustering, followed by morphological operations. Morphological, intensity-based, and textural features were extracted from the segmented area and also the image patch of 63 × 63 pixels in different channels of eight color spaces. Holistic features describing the mitoses' surrounding cells of each image were also extracted.

Statistical Analysis Used:

The Kruskal–Wallis H-test followed by the Tukey-Kramer test was used to identify significantly different features.

Results:

The results indicated that challenging mitoses were smaller and rounder compared to other mitoses. Among different features, the Gabor textural features differed more than others between challenging mitoses and the easily identifiable ones. Sizes of the non-mitoses were similar to easily identifiable mitoses, but nonmitoses were rounder. The intensity-based features from chromatin channels were the most discriminative features between the easily identifiable mitoses and the miscounted objects.

Conclusions:

Quantitative features can be used to describe the characteristics of challenging mitoses and miscounted nonmitotic objects.

A novel method for morphological pleomorphism and heterogeneity quantitative measurement: Named cell feature level co-occurrence matrix

Background:

Recent developments in molecular pathology and genetic/epigenetic analysis of cancer tissue have resulted in a marked increase in objective and measurable data. In comparison, the traditional morphological analysis approach to pathology diagnosis, which can connect these molecular data and clinical diagnosis, is still mostly subjective. Even though the advent and popularization of digital pathology has provided a boost to computer-aided diagnosis, some important pathological concepts still remain largely non-quantitative and their associated data measurements depend on the pathologist's sense and experience. Such features include pleomorphism and heterogeneity.

Methods and Results:

In this paper, we propose a method for the objective measurement of pleomorphism and heterogeneity, using the cell-level co-occurrence matrix. Our method is based on the widely used Gray-level co-occurrence matrix (GLCM), where relations between neighboring pixel intensity levels are captured into a co-occurrence matrix, followed by the application of analysis functions such as Haralick features. In the pathological tissue image, through image processing techniques, each nucleus can be measured and each nucleus has its own measureable features like nucleus size, roundness, contour length, intra-nucleus texture data (GLCM is one of the methods). In GLCM each nucleus in the tissue image corresponds to one pixel. In this approach the most important point is how to define the neighborhood of each nucleus. We define three types of neighborhoods of a nucleus, then create the co-occurrence matrix and apply Haralick feature functions. In each image pleomorphism and heterogeneity are then determined quantitatively. For our method, one pixel corresponds to one nucleus feature, and we therefore named our method Cell Feature Level Co-occurrence Matrix (CFLCM). We tested this method for several nucleus features.

Conclusion:

CFLCM is showed as a useful quantitative method for pleomorphism and heterogeneity on histopathological image analysis.

Quantitative analysis of nuclear shape in oral squamous cell carcinoma is useful for predicting the chemotherapeutic response

The number of people afflicted with oral carcinoma in Japan has increased in recent years. Although preoperative neoadjuvant therapy with cisplatin and 5-fluorouracil are performed, chemotherapeutic response varies widely among the patients. With the aim of establishing novel indices to predict the therapeutic response to chemotherapy, we investigated the relationship between morphological features of pre-treatment oral carcinoma nuclei and the chemotherapeutic response using quantifying morphology of cell nuclei in pathological specimen images. We measured 4 morphological features of the nucleus of oral squamous cell carcinoma cases classified by the response to chemotherapy: No Change (NC) group, Partial Response (PR) group and Complete Response (CR) group. Furthermore, we performed immunohistochemical staining for p53 and Ki67 and calculated their positive rates in cancer tissues. Compactness and symmetry of the nucleus were significantly higher and nuclear edge response was significantly lower in cancer cells with lower chemotherapeutic responses compared high chemotherapeutic responders. As for positive rates of p53 and Ki67, there were no significant differences between any of the response groups. Morphological features of cancer cell nuclei in pathological specimens are sensitive predictive factors for the chemotherapeutic response to oral squamous cell carcinoma.

Value of counting positive PHH3 cells in the diagnosis of uterine smooth muscle tumors

The diagnosis of uterine smooth muscle tumors including leiomyosarcomas (LMS), smooth muscle tumors of uncertain malignant potential (STUMP), bizarre (atypical) leiomyoma (BLM), mitotically active leiomyoma (MAL) and leiomyoma (LM) depends on a combination of microscopic features, such as mitoses, cytologic atypia, and coagulative tumor cell necrosis. However, a small number of these tumors still pose difficult diagnostic challenges. The assessment of accurate mitotic figures (MF) is one of the major parameters in the proper classification of uterine smooth muscle tumors. This assessment can be hampered by the presence of increased number of apoptotic bodies or pyknotic nuclei, which frequently mimic mitoses. Phospho-histone H3 (PHH3) is a recently described immunomarker specific for cells undergoing mitoses. In our study, we collected 132 cases of uterine smooth muscle tumors, including 26 LMSs, 16 STUMPs, 30 BLMs, 30 MALs and 30 LMs. We used mitosis specific marker PHH3 to count mitotic indexes (MI) of uterine smooth muscle tumors and compared with the mitotic indexes of hematoxylin and eosin (H&E). There is a positive correlation with the number of mitotic figures in H&E-stained sections and PHH3-stained sections (r=0.944, P<0.05). The ratio of PHH3-MI to H&E-MI has no statistically significant difference in each group except for LMs (P>0.05). The counting value of PHH3 in LMSs have significantly higher than STUMPs, BLMs, MALs and LMs (P<0.001) and the counting value of PHH3 is 1.5±0.5 times of the number of mitotic indexes in H&E. To conclude, our results show that counting PHH3 is a useful index in the diagnosis of uterine smooth muscle tumors and it can provide a more accurate index instead of the time-honored mitotic figure counts at a certain ratio.

Mitotic rate in melanoma: prognostic value of immunostaining and computer-assisted image analysis

The prognostic value of mitotic rate in melanoma is increasingly recognized, particularly in thin melanoma in which the presence or absence of a single mitosis/mm can change staging from T1a to T1b. Still, accurate mitotic rate calculation (mitoses/mm) on hematoxylin and eosin (H&E)-stained sections can be challenging. Antimonoclonal mitotic protein-2 (MPM-2) and antiphosphohistone-H3 (PHH3) are 2 antibodies reported to be more mitosis-specific than other markers of proliferation such as Ki-67. We used light microscopy and computer-assisted image analysis software to quantify MPM-2 and PHH3 staining in melanoma. We then compared mitotic rates by each method with conventional H&E-based mitotic rate for correlation with clinical outcomes. Our study included primary tissues from 190 nonconsecutive cutaneous melanoma patients who were prospectively enrolled at New York University Langone Medical Center with information on age, gender, and primary tumor characteristics. The mitotic rate was quantified manually by light microscopy of corresponding H&E-stained, MPM-2-stained, and PHH3-stained sections. Computer-assisted image analysis was then used to quantify immunolabeled mitoses on the previously examined PHH3 and MPM-2 slides. We then analyzed the association between mitotic rate and both progression-free and melanoma-specific survival. Univariate analysis of PHH3 found significant correlation between increased PHH3 mitotic rate and decreased progression-free survival (P=0.04). Computer-assisted image analysis enhanced the correlation of PHH3 mitotic rate with progression-free survival (P=0.02). Regardless of the detection method, neither MPM-2 nor PHH3 offered significant advantage over conventional H&E determination of mitotic rate.

Mitosis detection in breast cancer histological images An ICPR 2012 contest

INTRODUCTION

In the framework of the Cognitive Microscope (MICO) project, we have set up a contest about mitosis detection in images of H and E stained slides of breast cancer for the conference ICPR 2012. Mitotic count is an important parameter for the prognosis of breast cancer. However, mitosis detection in digital histopathology is a challenging problem that needs a deeper study. Indeed, mitosis detection is difficult because mitosis are small objects with a large variety of shapes, and they can thus be easily confused with some other objects or artefacts present in the image. We added a further dimension to the contest by using two different slide scanners having different resolutions and producing red-green-blue (RGB) images, and a multi-spectral microscope producing images in 10 different spectral bands and 17 layers Z-stack. 17 teams participated in the study and the best team achieved a recall rate of 0.7 and precision of 0.89.

CONTEXT

Several studies on automatic tools to process digitized slides have been reported focusing mainly on nuclei or tubule detection. Mitosis detection is a challenging problem that has not yet been addressed well in the literature.

AIMS

Mitotic count is an important parameter in breast cancer grading as it gives an evaluation of the aggressiveness of the tumor. However, consistency, reproducibility and agreement on mitotic count for the same slide can vary largely among pathologists. An automatic tool for this task may help for reaching a better consistency, and at the same time reducing the burden of this demanding task for the pathologists.

SUBJECTS AND METHODS

Professor Frιdιrique Capron team of the pathology department at Pitiι-Salpκtriθre Hospital in Paris, France, has selected a set of five slides of breast cancer. The slides are stained with H and E. They have been scanned by three different equipments: Aperio ScanScope XT slide scanner, Hamamatsu NanoZoomer 2.0-HT slide scanner and 10 bands multispectral microscope. The data set is made up of 50 high power fields (HPF) coming from 5 different slides scanned at ×40 magnification. There are 10 HPFs/slide. The pathologist has annotated all the mitotic cells manually. A HPF has a size of 512 μm × 512 μm (that is an area of 0.262 mm (2) , which is a surface equivalent to that of a microscope field diameter of 0.58 mm. These 50 HPFs contain a total of 326 mitotic cells on images of both scanners, and 322 mitotic cells on the multispectral microscope.

RESULTS

Up to 129 teams have registered to the contest. However, only 17 teams submitted their detection of mitotic cells. The performance of the best team is very promising, with F-measure as high as 0.78. However, the database we provided is by far too small for a good assessment of reliability and robustness of the proposed algorithms.

CONCLUSIONS

Mitotic count is an important criterion in the grading of many types of cancers, however, very little research has been made on automatic mitotic cell detection, mainly because of a lack of available data. A main objective of this contest was to propose a database of mitotic cells on digitized breast cancer histopathology slides to initiate works on automated mitotic cell detection. In the future, we would like to extend this database to have much more images from different patients and also for different types of cancers. In addition, mitotic cells should be annotated by several pathologists to reflect the partial agreement among them.

Classification of mitotic figures with convolutional neural networks and seeded blob features

Background:

The mitotic figure recognition contest at the 2012 International Conference on Pattern Recognition (ICPR) challenges a system to identify all mitotic figures in a region of interest of hematoxylin and eosin stained tissue, using each of three scanners (Aperio, Hamamatsu, and multispectral).

Methods:

Our approach combines manually designed nuclear features with the learned features extracted by convolutional neural networks (CNN). The nuclear features capture color, texture, and shape information of segmented regions around a nucleus. The use of a CNN handles the variety of appearances of mitotic figures and decreases sensitivity to the manually crafted features and thresholds.

Results:

On the test set provided by the contest, the trained system achieves F1 scores up to 0.659 on color scanners and 0.589 on multispectral scanner.

Conclusions:

We demonstrate a powerful technique combining segmentation-based features with CNN, identifying the majority of mitotic figures with a fair precision. Further, we show that the approach accommodates information from the additional focal planes and spectral bands from a multi-spectral scanner without major redesign.