Patch-Based Nonlinear Image Registration for Gigapixel Whole Slide Images

RegWSI: Whole slide image registration using combined deep feature- and intensity-based methods: Winner of the ACROBAT 2023 challenge

BACKGROUND AND OBJECTIVE

The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology.

METHODS

We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The registration time is low, allowing one to perform efficient registration even for large datasets. The method was proposed for the ACROBAT 2023 challenge organized during the MICCAI 2023 conference and scored 1st place. The method is released as open-source software.

RESULTS

The proposed method is evaluated using three open datasets: (i) Automatic Nonrigid Histological Image Registration Dataset (ANHIR), (ii) Automatic Registration of Breast Cancer Tissue Dataset (ACROBAT), and (iii) Hybrid Restained and Consecutive Histological Serial Sections Dataset (HyReCo). The target registration error (TRE) is used as the evaluation metric. We compare the proposed algorithm to other state-of-the-art solutions, showing considerable improvement. Additionally, we perform several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset.

CONCLUSIONS

The article presents an automatic and robust registration method that outperforms other state-of-the-art solutions. The method does not require any fine-tuning to a particular dataset and can be used out-of-the-box for numerous types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level (resolution up to 220k x 220k). We provide free access to the software. The results are fully and easily reproducible. The proposed method is a significant contribution to improving the WSI registration quality, thus advancing the field of digital pathology.

Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer

PURPOSE

This study aims to introduce an innovative multi-step pipeline for automatic tumor-stroma ratio (TSR) quantification as a potential prognostic marker for pancreatic cancer, addressing the limitations of existing staging systems and the lack of commonly used prognostic biomarkers.

METHODS

The proposed approach involves a deep-learning-based method for the automatic segmentation of tumor epithelial cells, tumor bulk, and stroma from whole-slide images (WSIs). Models were trained using five-fold cross-validation and evaluated on an independent external test set. TSR was computed based on the segmented components. Additionally, TSR's predictive value for six-month survival on the independent external dataset was assessed.

RESULTS

Median Dice (inter-quartile range (IQR)) of 0.751(0.15) and 0.726(0.25) for tumor epithelium segmentation on internal and external test sets, respectively. Median Dice of 0.76(0.11) and 0.863(0.17) for tumor bulk segmentation on internal and external test sets, respectively. TSR was evaluated as an independent prognostic marker, demonstrating a cross-validation AUC of 0.61±0.12 for predicting six-month survival on the external dataset.

CONCLUSION

Our pipeline for automatic TSR quantification offers promising potential as a prognostic marker for pancreatic cancer. The results underscore the feasibility of computational biomarker discovery in enhancing patient outcome prediction, thus contributing to personalized patient management.

Whole-Slide Imaging, Mutual Information Registration for Multiplex Immunohistochemistry and Immunofluorescence

Multiplex immunohistochemistry/immunofluorescence (mIHC/mIF) is a developing technology that facilitates the evaluation of multiple, simultaneous protein expressions at single-cell resolution while preserving tissue architecture. These approaches have shown great potential for biomarker discovery, yet many challenges remain. Importantly, streamlined cross-registration of multiplex immunofluorescence images with additional imaging modalities and immunohistochemistry (IHC) can help increase the plex and/or improve the quality of the data generated by potentiating downstream processes such as cell segmentation. To address this problem, a fully automated process was designed to perform a hierarchical, parallelizable, and deformable registration of multiplexed digital whole-slide images (WSIs). We generalized the calculation of mutual information as a registration criterion to an arbitrary number of dimensions, making it well suited for multiplexed imaging. We also used the self-information of a given IF channel as a criterion to select the optimal channels to use for registration. Additionally, as precise labeling of cellular membranes in situ is essential for robust cell segmentation, a pan-membrane immunohistochemical staining method was developed for incorporation into mIF panels or for use as an IHC followed by cross-registration. In this study, we demonstrate this process by registering whole-slide 6-plex/7-color mIF images with whole-slide brightfield mIHC images, including a CD3 and a pan-membrane stain. Our algorithm, WSI, mutual information registration (WSIMIR), performed highly accurate registration allowing the retrospective generation of an 8-plex/9-color, WSI, and outperformed 2 alternative automated methods for cross-registration by Jaccard index and Dice similarity coefficient (WSIMIR vs automated WARPY, P < .01 and P < .01, respectively, vs HALO + transformix, P = .083 and P = .049, respectively). Furthermore, the addition of a pan-membrane IHC stain cross-registered to an mIF panel facilitated improved automated cell segmentation across mIF WSIs, as measured by significantly increased correct detections, Jaccard index (0.78 vs 0.65), and Dice similarity coefficient (0.88 vs 0.79).

Unsupervised Histological Image Registration Using Structural Feature Guided Convolutional Neural Network

Registration of multiple stained images is a fundamental task in histological image analysis. In supervised methods, obtaining ground-truth data with known correspondences is laborious and time-consuming. Thus, unsupervised methods are expected. Unsupervised methods ease the burden of manual annotation but often at the cost of inferior results. In addition, registration of histological images suffers from appearance variance due to multiple staining, repetitive texture, and section missing during making tissue sections. To deal with these challenges, we propose an unsupervised structural feature guided convolutional neural network (SFG). Structural features are robust to multiple staining. The combination of low-resolution rough structural features and high-resolution fine structural features can overcome repetitive texture and section missing, respectively. SFG consists of two components of structural consistency constraints according to the formations of structural features, i.e., dense structural component and sparse structural component. The dense structural component uses structural feature maps of the whole image as structural consistency constraints, which represent local contextual information. The sparse structural component utilizes the distance of automatically obtained matched key points as structural consistency constraints because the matched key points in an image pair emphasize the matching of significant structures, which imply global information. In addition, a multi-scale strategy is used in both dense and sparse structural components to make full use of the structural information at low resolution and high resolution to overcome repetitive texture and section missing. The proposed method was evaluated on a public histological dataset (ANHIR) and ranked first as of Jan 18th, 2022.

Two-Stage Selective Ensemble of CNN via Deep Tree Training for Medical Image Classification

Medical image classification is an important task in computer-aided diagnosis systems. Its performance is critically determined by the descriptiveness and discriminative power of features extracted from images. With rapid development of deep learning, deep convolutional neural networks (CNNs) have been widely used to learn the optimal high-level features from the raw pixels of images for a given classification task. However, due to the limited amount of labeled medical images with certain quality distortions, such techniques crucially suffer from the training difficulties, including overfitting, local optimums, and vanishing gradients. To solve these problems, in this article, we propose a two-stage selective ensemble of CNN branches via a novel training strategy called deep tree training (DTT). In our approach, DTT is adopted to jointly train a series of networks constructed from the hidden layers of CNN in a hierarchical manner, leading to the advantage that vanishing gradients can be mitigated by supplementing gradients for hidden layers of CNN, and intrinsically obtain the base classifiers on the middle-level features with minimum computation burden for an ensemble solution. Moreover, the CNN branches as base learners are combined into the optimal classifier via the proposed two-stage selective ensemble approach based on both accuracy and diversity criteria. Extensive experiments on CIFAR-10 benchmark and two specific medical image datasets illustrate that our approach achieves better performance in terms of accuracy, sensitivity, specificity, and F1 score measurement.

Virtual liver needle biopsy from reconstructed three-dimensional histopathological images: Quantification of sampling error

INTRODUCTION

Prevalently considered as the "gold-standard" for diagnosis of hepatic fibrosis and cirrhosis, the clinical liver needle biopsy is known to be subject to inadequate sampling and a high mis-sampling rate. However, quantifying such sampling bias has been difficult as generating a large number of needle biopsies from the same living patient is practically infeasible. We construct a three-dimension (3D) virtual liver tissue volume by spatially registered high resolution Whole Slide Images (WSIs) of serial liver tissue sections with a novel dynamic registration method. We further develop a Virtual Needle Biopsy Sampling (VNBS) method that mimics the needle biopsy sampling process. We apply the VNBS method to the reconstructed digital liver volume at different tissue locations and angles. Additionally, we quantify Collagen Proportionate Area (CPA) in all resulting virtual needle biopsies in 2D and 3D.

RESULTS

The staging score of the center 2D longitudinal image plane from each 3D biopsy is used as the biopsy staging score, and the highest staging score of all sampled needle biopsies is the diagnostic staging score. The Mean Absolute Difference (MAD) in reference to the Scheuer and Ishak diagnostic staging scores are 0.22 and 1.00, respectively. The absolute Scheuer staging score difference in 22.22% of sampled biopsies is 1. By the Ishak staging method, 55.56% and 22.22% of sampled biopsies present score difference 1 and 2, respectively. There are 4 (Scheuer) and 6 (Ishak) out of 18 3D virtual needle biopsies with intra-needle variations. Additionally, we find a positive correlation between CPA and fibrosis stages by Scheuer but not Ishak method. Overall, CPA measures suffer large intra- and inter- needle variations.

CONCLUSIONS

The developed virtual liver needle biopsy sampling pipeline provides a computational avenue for investigating needle biopsy sampling bias with 3D virtual tissue volumes. This method can be applied to other tissue-based disease diagnoses where the needle biopsy sampling bias substantially affects the diagnostic results.

Fast cross-staining alignment of gigapixel whole slide images with application to prostate cancer and breast cancer analysis

Joint analysis of multiple protein expressions and tissue morphology patterns is important for disease diagnosis, treatment planning, and drug development, requiring cross-staining alignment of multiple immunohistochemical and histopathological slides. However, cross-staining alignment of enormous gigapixel whole slide images (WSIs) at single cell precision is difficult. Apart from gigantic data dimensions of WSIs, there are large variations on the cell appearance and tissue morphology across different staining together with morphological deformations caused by slide preparation. The goal of this study is to build an image registration framework for cross-staining alignment of gigapixel WSIs of histopathological and immunohistochemical microscopic slides and assess its clinical applicability. To the authors' best knowledge, this is the first study to perform real time fully automatic cross staining alignment of WSIs with 40× and 20× objective magnification. The proposed WSI registration framework consists of a rapid global image registration module, a real time interactive field of view (FOV) localization model and a real time propagated multi-level image registration module. In this study, the proposed method is evaluated on two kinds of cancer datasets from two hospitals using different digital scanners, including a dual staining breast cancer data set with 43 hematoxylin and eosin (H&E) WSIs and 43 immunohistochemical (IHC) CK(AE1/AE3) WSIs, and a triple staining prostate cancer data set containing 30 H&E WSIs, 30 IHC CK18 WSIs, and 30 IHC HMCK WSIs. In evaluation, the registration performance is measured by not only registration accuracy but also computational time. The results show that the proposed method achieves high accuracy of 0.833 ± 0.0674 for the triple-staining prostate cancer data set and 0.931 ± 0.0455 for the dual-staining breast cancer data set, respectively, and takes only 4.34 s per WSI registration on average. In addition, for 30.23% data, the proposed method takes less than 1 s for WSI registration. In comparison with the benchmark methods, the proposed method demonstrates superior performance in registration accuracy and computational time, which has great potentials for assisting medical doctors to identify cancerous tissues and determine the cancer stage in clinical practice.

An adaptive registration algorithm for zebrafish larval brain images

BACKGROUND AND OBJECTIVE

Zebrafish (Danio rerio) in their larval stages have grown increasingly popular as excellent vertebrate models for neurobiological research. Researchers can apply various tools in order to decode the neural structure patterns which can aid the understanding of vertebrate brain development. In order to do so, it is essential to map the gene expression patterns to an anatomical reference precisely. However, high accuracy in sample registration is sometimes difficult to achieve due to laboratory- or protocol-dependent variabilities.

METHODS

In this paper, we propose an accurate adaptive registration algorithm for volumetric zebrafish larval image datasets using a synergistic combination of attractive Free-Form-Deformation (FFD) and diffusive Demons algorithms. A coarse registration is achieved first for 3D volumetric data using a 3D affine transformation. A localized registration algorithm in form of a B-splines based FFD is applied next on the coarsely registered volume. Finally, the Demons algorithm is applied on this FFD registered volume for achieving fine registration by making the solution noise resilient.

RESULTS

Results Experimental procedures are carried out on a number of 72 hpf (hours post fertilization) 3D confocal zebrafish larval datasets. Comparisons with state-of-the-art methods including some ablation studies clearly demonstrate the effectiveness of the proposed method.

CONCLUSIONS

Our adaptive registration algorithm significantly aids Zebrafish imaging analysis over current methods for gene expression anatomical mapping, such as Vibe-Z. We believe the proposed solution would be able to overcome the requirement of high quality images which currently limits the applicability of Zebrafish in neuroimaging research.

Feasibility and Efficacy of Fusion Imaging Systems for Immediate Post Ablation Assessment of Liver Neoplasms: Protocol for a Rapid Systematic Review

Introduction:

Percutaneous thermal ablation is widely adopted as a curative treatment approach for unresectable liver neoplasms. Accurate immediate assessment of therapeutic response post-ablation is critical to achieve favourable outcomes. The conventional technique of side-by-side comparison of pre- and post-ablation scans is challenging and hence there is a need for improved methods, which will accurately evaluate the immediate post-therapeutic response.

Objectives and Significance:

This review summarizes the findings of studies investigating the feasibility and efficacy of the fusion imaging systems in the immediate post-operative assessment of the therapeutic response to thermal ablation in liver neoplasms. The findings could potentially empower the clinicians with updated knowledge of the state-of-the-art in the assessment of treatment response for unresectable liver neoplasms.

Methods and Analysis:

A rapid review will be performed on publicly available major electronic databases to identify articles reporting the feasibility and efficacy of the fusion imaging systems in the immediate assessment of the therapeutic response to thermal ablation in liver neoplasms. The risk of bias and quality of articles will be assessed using the Cochrane risk of bias tool 2.0 and Newcastle Ottawa tool.

Ethics and Dissemination:

Being a review, we do not anticipate the need for any approval from the Institutional Review Board. The outcomes of this study will be published in a peer-reviewed journal.

Highlights

Evaluation of the therapeutic response in liver neoplasms immediately post-ablation is critical to achieve favourable patient outcomes. We will examine the feasibility and technical efficacy of different fusion imaging systems in assessing the immediate treatment response post-ablation. The findings are expected to guide the clinicians with updated knowledge on the state-of-the-art when assessing the immediate treatment response for unresectable liver neoplasms.

Improving Algorithm for the Alignment of Consecutive, Whole-Slide, Immunohistochemical Section Images

Background:

Accurate and precise alignment of histopathology tissue sections is a key step for the interpretation of the proteome topology and cell level three-dimensional (3D) reconstruction of diseased tissues. However, the realization of an automated and robust method for aligning nonglobally stained immunohistochemical (IHC) sections is still challenging. In this study, we aim to assess the feasibility of multidimensional graph-based image registration on aligning serial-section and whole-slide IHC section images.

Materials and Methods:

An automated, patch graph-based registration method was established and applied to align serial, whole-slide IHC sections at ×10 magnification (average 32,947 × 27,054 pixels). The alignment began with the initial alignment of high-resolution reference and translated images (object segmentation and rigid registration) and nonlinear registration of low-resolution reference and translated images, followed by the multidimensional graph-based image registration of the segmented patches, and finally, the fusion of deformed patches for inspection. The performance of the proposed method was formulated and evaluated by the Hausdorff distance between continuous image slices.

Results:

Sets of average 315 patches from five serial whole slide, IHC section images were tested using 21 different IHC antibodies across five different tissue types (skin, breast, stomach, prostate, and soft tissue). The proposed method was successfully automated to align most of the images. The average Hausdorff distance was 48.93 μm with a standard deviation of 14.94 μm, showing a significant improvement from the previously published patch-based nonlinear image registration method (average Hausdorff distance of 93.89 μm with 50.85 μm standard deviation).

Conclusions:

Our method was effective in aligning whole-slide tissue sections at the cell-level resolution. Further advancements in the screening of the proteome topology and 3D tissue reconstruction could be expected.

Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics

The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.

Regional registration of whole slide image stacks containing major histological artifacts

Background

High resolution 2D whole slide imaging provides rich information about the tissue structure. This information can be a lot richer if these 2D images can be stacked into a 3D tissue volume. A 3D analysis, however, requires accurate reconstruction of the tissue volume from the 2D image stack. This task is not trivial due to the distortions such as tissue tearing, folding and missing at each slide. Performing registration for the whole tissue slices may be adversely affected by distorted tissue regions. Consequently, regional registration is found to be more effective. In this paper, we propose a new approach to an accurate and robust registration of regions of interest for whole slide images. We introduce the idea of multi-scale attention for registration.

Results

Using mean similarity index as the metric, the proposed algorithm (mean ± SD \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.84 \pm 0.11$$\end{document}0.84±0.11) followed by a fine registration algorithm (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.86 \pm 0.08$$\end{document}0.86±0.08) outperformed the state-of-the-art linear whole tissue registration algorithm (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.74 \pm 0.19$$\end{document}0.74±0.19) and the regional version of this algorithm (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.81 \pm 0.15$$\end{document}0.81±0.15). The proposed algorithm also outperforms the state-of-the-art nonlinear registration algorithm (original: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.82 \pm 0.12$$\end{document}0.82±0.12, regional: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.77 \pm 0.22$$\end{document}0.77±0.22) for whole slide images and a recently proposed patch-based registration algorithm (patch size 256: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.79 \pm 0.16$$\end{document}0.79±0.16 , patch size 512: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.77 \pm 0.16$$\end{document}0.77±0.16) for medical images.

Conclusion

Using multi-scale attention mechanism leads to a more robust and accurate solution to the problem of regional registration of whole slide images corrupted in some parts by major histological artifacts in the imaged tissue.

A machine learning algorithm for simulating immunohistochemistry: development of SOX10 virtual IHC and evaluation on primarily melanocytic neoplasms

Immunohistochemistry (IHC) is a diagnostic technique used throughout pathology. A machine learning algorithm that could predict individual cell immunophenotype based on hematoxylin and eosin (H&E) staining would save money, time, and reduce tissue consumed. Prior approaches have lacked the spatial accuracy needed for cell-specific analytical tasks. Here IHC performed on destained H&E slides is used to create a neural network that is potentially capable of predicting individual cell immunophenotype. Twelve slides were stained with H&E and scanned to create digital whole slide images. The H&E slides were then destained, and stained with SOX10 IHC. The SOX10 IHC slides were scanned, and corresponding H&E and IHC digital images were registered. Color-thresholding and machine learning techniques were applied to the registered H&E and IHC images to segment 3,396,668 SOX10-negative cells and 306,166 SOX10-positive cells. The resulting segmentation was used to annotate the original H&E images, and a convolutional neural network was trained to predict SOX10 nuclear staining. Sixteen thousand three hundred and nine image patches were used to train the virtual IHC (vIHC) neural network, and 1,813 image patches were used to quantitatively evaluate it. The resulting vIHC neural network achieved an area under the curve of 0.9422 in a receiver operator characteristics analysis when sorting individual nuclei. The vIHC network was applied to additional images from clinical practice, and was evaluated qualitatively by a board-certified dermatopathologist. Further work is needed to make the process more efficient and accurate for clinical use. This proof-of-concept demonstrates the feasibility of creating neural network-driven vIHC assays.

Localization of Nuclei in Breast Cancer Using Whole Slide Imaging System Supported by Morphological Features and Shape Formulas

PURPOSE

Cancer rates are exponentially increasing worldwide and over 15 million new cases are expected in the year 2020 according to the World Cancer Report. To support the clinical diagnosis of the disease, recent technical advancements in digital microscopy have been achieved to reduce the cost and increase the efficiency of the process. Food and Drug Administration (FDA or Agency) has issued the guidelines, in particular, the development of digital whole slide image scanning system. It is very helpful to the computer-aided diagnosis of breast cancer.

METHODS

Whole slide imaging supported by fluorescence, immunohistochemistry, and multispectral imaging concepts. Due to the high dimension of WSI images and computation, it is a challenging task to find the region of interest (ROI) on a malignant sample image. The unsupervised machine learning and quantitative analysis of malignant sample images are supported by morphological features and shape formulas to find the correct region of interest. Due to computational limitations, it starts to work on small patches, integrate the results, and automated localize or detect the ROI. It is also compared to the handcrafted and automated region of interest provided in the ICIAR2018 dataset.

RESULTS

A total of 10 hematoxylins and eosin (H&E) stained malignant breast histology microscopy whole slide image samples are labeled and annotated by two medical experts who are team members of the ICIAR 2018 challenge. After applying the proposed methodology, it is successfully able to localize the malignant patches of WSI sample images and getting the ROI with an average accuracy of 85.5%.

CONCLUSION

With the help of the k-means clustering algorithm, morphological features, and shape formula, it is possible to recognize the region of interest using the whole slide imaging concept.

The Society for Immunotherapy of Cancer statement on best practices for multiplex immunohistochemistry (IHC) and immunofluorescence (IF) staining and validation

OBJECTIVES

The interaction between the immune system and tumor cells is an important feature for the prognosis and treatment of cancer. Multiplex immunohistochemistry (mIHC) and multiplex immunofluorescence (mIF) analyses are emerging technologies that can be used to help quantify immune cell subsets, their functional state, and their spatial arrangement within the tumor microenvironment.

METHODS

The Society for Immunotherapy of Cancer (SITC) convened a task force of pathologists and laboratory leaders from academic centers as well as experts from pharmaceutical and diagnostic companies to develop best practice guidelines for the optimization and validation of mIHC/mIF assays across platforms.

RESULTS

Representative outputs and the advantages and disadvantages of mIHC/mIF approaches, such as multiplexed chromogenic IHC, multiplexed immunohistochemical consecutive staining on single slide, mIF (including multispectral approaches), tissue-based mass spectrometry, and digital spatial profiling are discussed.

CONCLUSIONS

mIHC/mIF technologies are becoming standard tools for biomarker studies and are likely to enter routine clinical practice in the near future. Careful assay optimization and validation will help ensure outputs are robust and comparable across laboratories as well as potentially across mIHC/mIF platforms. Quantitative image analysis of mIHC/mIF output and data management considerations will be addressed in a complementary manuscript from this task force.

Robust hierarchical density estimation and regression for re-stained histological whole slide image co-registration

For many disease conditions, tissue samples are colored with multiple dyes and stains to add contrast and location information for specific proteins to accurately identify and diagnose disease. This presents a computational challenge for digital pathology, as whole-slide images (WSIs) need to be properly overlaid (i.e. registered) to identify co-localized features. Traditional image registration methods sometimes fail due to the high variation of cell density and insufficient texture information in WSIs–particularly at high magnifications. In this paper, we proposed a robust image registration strategy to align re-stained WSIs precisely and efficiently. This method is applied to 30 pairs of immunohistochemical (IHC) stains and their hematoxylin and eosin (H&E) counterparts. Our approach advances the existing methods in three key ways. First, we introduce refinements to existing image registration methods. Second, we present an effective weighting strategy using kernel density estimation to mitigate registration errors. Third, we account for the linear relationship across WSI levels to improve accuracy. Our experiments show significant decreases in registration errors when matching IHC and H&E pairs, enabling subcellular-level analysis on stained and re-stained histological images. We also provide a tool to allow users to develop their own registration benchmarking experiments.

Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard

Given the importance of gland morphology in grading prostate cancer (PCa), automatically differentiating between epithelium and other tissues is an important prerequisite for the development of automated methods for detecting PCa. We propose a new deep learning method to segment epithelial tissue in digitised hematoxylin and eosin (H&E) stained prostatectomy slides using immunohistochemistry (IHC) as reference standard. We used IHC to create a precise and objective ground truth compared to manual outlining on H&E slides, especially in areas with high-grade PCa. 102 tissue sections were stained with H&E and subsequently restained with P63 and CK8/18 IHC markers to highlight epithelial structures. Afterwards each pair was co-registered. First, we trained a U-Net to segment epithelial structures in IHC using a subset of the IHC slides that were preprocessed with color deconvolution. Second, this network was applied to the remaining slides to create the reference standard used to train a second U-Net on H&E. Our system accurately segmented both intact glands and individual tumour epithelial cells. The generalisation capacity of our system is shown using an independent external dataset from a different centre. We envision this segmentation as the first part of a fully automated prostate cancer grading pipeline.

Accurate Region-of-Interest Recovery Improves the Measurement of the Cell Migration Rate in the In Vitro Wound Healing Assay

The wound healing assay is widely used for the quantitative analysis of highly regulated cellular events. In this essay, a wound is voluntarily produced on a confluent cell monolayer, and then the rate of wound reduction (WR) is characterized by processing images of the same regions of interest (ROIs) recorded at different time intervals. In this method, sharp-image ROI recovery is indispensable to compensate for displacements of the cell cultures due either to the exploration of multiple sites of the same culture or to transfers from the microscope stage to a cell incubator. ROI recovery is usually done manually and, despite a low-magnification microscope objective is generally used (10x), repositioning imperfections constitute a major source of errors detrimental to the WR measurement accuracy. We address this ROI recovery issue by using pseudoperiodic patterns fixed onto the cell culture dishes, allowing the easy localization of ROIs and the accurate quantification of positioning errors. The method is applied to a tumor-derived cell line, and the WR rates are measured by means of two different image processing software. Sharp ROI recovery based on the proposed method is found to improve significantly the accuracy of the WR measurement and the positioning under the microscope.

DYNAMIC REGISTRATION FOR GIGAPIXEL SERIAL WHOLE SLIDE IMAGES

High-throughput serial histology imaging provides a new avenue for the routine study of micro-anatomical structures in a 3D space. However, the emergence of serial whole slide imaging poses a new registration challenge, as the gigapixel image size precludes the direct application of conventional registration techniques. In this paper, we develop a three-stage registration with multi-resolution mapping and propagation method to dynamically produce registered subvolumes from serial whole slide images. We validate our algorithm with gigapixel images of serial brain tumor sections and synthetic image volumes. The qualitative and quantitative assessment results demonstrate the efficacy of our approach and suggest its promise for 3D histology reconstruction analysis.