Two-phase deep convolutional neural network for reducing class skewness in histopathological images based breast cancer detection

Different types of breast cancer are affecting lives of women across the world. Common types include Ductal carcinoma in situ (DCIS), Invasive ductal carcinoma (IDC), Tubular carcinoma, Medullary carcinoma, and Invasive lobular carcinoma (ILC). While detecting cancer, one important factor is mitotic count - showing how rapidly the cells are dividing. But the class imbalance problem, due to the small number of mitotic nuclei in comparison to the overwhelming number of non-mitotic nuclei, affects the performance of classification models. This work presents a two-phase model to mitigate the class biasness issue while classifying mitotic and non-mitotic nuclei in breast cancer histopathology images through a deep convolutional neural network (CNN). First, nuclei are segmented out using blue ratio and global binary thresholding. In Phase-1 a CNN is then trained on the segmented out 80×80 pixel patches based on a standard dataset. Hard non-mitotic examples are identified and augmented; mitotic examples are oversampled by rotation and flipping; whereas non-mitotic examples are undersampled by blue ratio histogram based k-means clustering. Based on this information from Phase-1, the dataset is modified for Phase-2 in order to reduce the effects of class imbalance. The proposed CNN architecture and data balancing technique yielded an F-measure of 0.79, and outperformed all the methods relying on specific handcrafted features, as well as those using a combination of handcrafted and CNN-generated features.

Automatic extraction of cell nuclei from H&E-stained histopathological images

Extraction of cell nuclei from hematoxylin and eosin (H&E)-stained histopathological images is an essential preprocessing step in computerized image analysis for disease detection, diagnosis, and prognosis. We present an automated cell nuclei segmentation approach that works with H&E-stained images. A color deconvolution algorithm was first applied to the image to get the hematoxylin channel. Using a morphological operation and thresholding technique on the hematoxylin channel image, candidate target nuclei and background regions were detected, which were then used as markers for a marker-controlled watershed transform segmentation algorithm. Moreover, postprocessing was conducted to split the touching nuclei. For each segmented region from the previous steps, the regional maximum value positions were identified as potential nuclei centers. These maximum values were further grouped into [Formula: see text]-clusters, and the locations within each cluster were connected with the minimum spanning tree technique. Then, these connected positions were utilized as new markers for a watershed segmentation approach. The final number of nuclei at each region was determined by minimizing an objective function that iterated all of the possible [Formula: see text]-values. The proposed method was applied to the pathological images of the tumor tissues from The Cancer Genome Atlas study. Experimental results show that the proposed method can lead to promising results in terms of segmentation accuracy and separation of touching nuclei.

Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

With the advance of digital pathology, image analysis has begun to show its advantages in information analysis of hematoxylin and eosin histopathology images. Generally, histological features in hematoxylin and eosin images are measured to evaluate tumor grade and prognosis for breast cancer. This review summarized recent works in image analysis of hematoxylin and eosin histopathology images for breast cancer prognosis. First, prognostic factors for breast cancer based on hematoxylin and eosin histopathology images were summarized. Then, usual procedures of image analysis for breast cancer prognosis were systematically reviewed, including image acquisition, image preprocessing, image detection and segmentation, and feature extraction. Finally, the prognostic value of image features and image feature–based prognostic models was evaluated. Moreover, we discussed the issues of current analysis, and some directions for future research.

MRF-ANN: a machine learning approach for automated ER scoring of breast cancer immunohistochemical images

Molecular pathology, especially immunohistochemistry, plays an important role in evaluating hormone receptor status along with diagnosis of breast cancer. Time-consumption and inter-/intraobserver variability are major hindrances for evaluating the receptor score. In view of this, the paper proposes an automated Allred Scoring methodology for estrogen receptor (ER). White balancing is used to normalize the colour image taking into consideration colour variation during staining in different labs. Markov random field model with expectation-maximization optimization is employed to segment the ER cells. The proposed segmentation methodology is found to have F-measure 0.95. Artificial neural network is subsequently used to obtain intensity-based score for ER cells, from pixel colour intensity features. Simultaneously, proportion score - percentage of ER positive cells is computed via cell counting. The final ER score is computed by adding intensity and proportion scores - a standard Allred scoring system followed by pathologists. The classification accuracy for classification of cells by classifier in terms of F-measure is 0.9626. The problem of subjective interobserver ability is addressed by quantifying ER score from two expert pathologist and proposed methodology. The intraclass correlation achieved is greater than 0.90. The study has potential advantage of assisting pathologist in decision making over manual procedure and could evolve as a part of automated decision support system with other receptor scoring/analysis procedure.

Automated characterization and counting of Ki-67 protein for breast cancer prognosis: A quantitative immunohistochemistry approach

Ki-67 protein expression plays an important role in predicting the proliferative status of tumour cells and deciding the future course of therapy in breast cancer. Immunohistochemical (IHC) determination of Ki-67 score or labelling index, by estimating the fraction of Ki67 positively stained tumour cells, is the most widely practiced method to assess tumour proliferation (Dowsett et al. 2011). Accurate manual counting of these cells (specifically nuclei) due to complex and dense distribution of cells, therefore, becomes critical and presents a major challenge to pathologists. In this paper, we suggest a hybrid clustering algorithm to quantify the proliferative index of breast cancer cells based on automated counting of Ki-67 nuclei. The proposed methodology initially pre-processes the IHC images of Ki-67 stained slides of breast cancer. The RGB images are converted to grey, L*a*b*, HSI, YCbCr, YIQ and XYZ colour space. All the stained cells are then characterized by two stage segmentation process. Fuzzy C-means quantifies all the stained cells as one cluster. The blue channel of the first stage output is given as input to k-means algorithm, which provides separate cluster for Ki-67 positive and negative cells. The count of positive and negative nuclei is used to calculate the F-measure for each colour space. A comparative study of our work with the expert opinion is studied to evaluate the error rate. The positive and negative nuclei detection results for all colour spaces are compared with the ground truth for validation and F-measure is calculated. The F-measure for L*a*b* colour space (0.8847) provides the best statistical result as compared to grey, HSI, YCbCr, YIQ and XYZ colour space. Further, a study is carried out to count nuclei manually and automatically from the proposed algorithm with an average error rate of 6.84% which is significant. The study provides an automated count of positive and negative nuclei using L*a*b*colour space and hybrid segmentation technique. Computerized evaluation of proliferation index can aid pathologist in assessing breast cancer severity. The proposed methodology, further, has the potential advantage of saving time and assisting in decision making over the present manual procedure and could evolve as an assistive pathological decision support system.

DCAN: Deep contour-aware networks for object instance segmentation from histology images

In histopathological image analysis, the morphology of histological structures, such as glands and nuclei, has been routinely adopted by pathologists to assess the malignancy degree of adenocarcinomas. Accurate detection and segmentation of these objects of interest from histology images is an essential prerequisite to obtain reliable morphological statistics for quantitative diagnosis. While manual annotation is error-prone, time-consuming and operator-dependant, automated detection and segmentation of objects of interest from histology images can be very challenging due to the large appearance variation, existence of strong mimics, and serious degeneration of histological structures. In order to meet these challenges, we propose a novel deep contour-aware network (DCAN) under a unified multi-task learning framework for more accurate detection and segmentation. In the proposed network, multi-level contextual features are explored based on an end-to-end fully convolutional network (FCN) to deal with the large appearance variation. We further propose to employ an auxiliary supervision mechanism to overcome the problem of vanishing gradients when training such a deep network. More importantly, our network can not only output accurate probability maps of histological objects, but also depict clear contours simultaneously for separating clustered object instances, which further boosts the segmentation performance. Our method ranked the first in two histological object segmentation challenges, including 2015 MICCAI Gland Segmentation Challenge and 2015 MICCAI Nuclei Segmentation Challenge. Extensive experiments on these two challenging datasets demonstrate the superior performance of our method, surpassing all the other methods by a significant margin.

Detecting and segmenting cell nuclei in two-dimensional microscopy images

Introduction:

Cell nuclei are important indicators of cellular processes and diseases. Segmentation is an essential stage in systems for quantitative analysis of nuclei extracted from microscopy images. Given the wide variety of nuclei appearance in different organs and staining procedures, a plethora of methods have been described in the literature to improve the segmentation accuracy and robustness.

Materials and Methods:

In this paper, we propose an unsupervised method for cell nuclei detection and segmentation in two-dimensional microscopy images. The nuclei in the image are detected automatically using a matching-based method. Next, edge maps are generated at multiple image blurring levels followed by edge selection performed in polar space. The nuclei contours are refined iteratively in the constructed edge pyramid. The validation study was conducted over two cell nuclei datasets with manual labeling, including 25 hematoxylin and eosin-stained liver histopathology images and 35 Papanicolaou-stained thyroid images.

Results:

The nuclei detection accuracy was measured by miss rate, and the segmentation accuracy was evaluated by two types of error metrics. Overall, the nuclei detection efficiency of the proposed method is similar to the supervised template matching method. In comparison to four existing state-of-the-art segmentation methods, the proposed method performed the best with average segmentation error 10.34% and 0.33 measured by area error rate and normalized sum of distances (×10).

Conclusion:

Quantitative analysis showed that the method is automatic and accurate when segmenting cell nuclei from microscopy images with noisy background and has the potential to be used in clinic settings.

Computer-based image analysis in breast pathology

Whole slide imaging (WSI) has the potential to be utilized in telepathology, teleconsultation, quality assurance, clinical education, and digital image analysis to aid pathologists. In this paper, the potential added benefits of computer-assisted image analysis in breast pathology are reviewed and discussed. One of the major advantages of WSI systems is the possibility of doing computer-based image analysis on the digital slides. The purpose of computer-assisted analysis of breast virtual slides can be (i) segmentation of desired regions or objects such as diagnostically relevant areas, epithelial nuclei, lymphocyte cells, tubules, and mitotic figures, (ii) classification of breast slides based on breast cancer (BCa) grades, the invasive potential of tumors, or cancer subtypes, (iii) prognosis of BCa, or (iv) immunohistochemical quantification. While encouraging results have been achieved in this area, further progress is still required to make computer-based image analysis of breast virtual slides acceptable for clinical practice.

Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images

Nuclei detection is often a critical initial step in the development of computer aided diagnosis and prognosis schemes in the context of digital pathology images. While over the last few years, a number of nuclei detection methods have been proposed, most of these approaches make idealistic assumptions about the staining quality of the tissue. In this paper, we present a new Multi-Pass Adaptive Voting (MPAV) for nuclei detection which is specifically geared towards images with poor quality staining and noise on account of tissue preparation artifacts. The MPAV utilizes the symmetric property of nuclear boundary and adaptively selects gradient from edge fragments to perform voting for a potential nucleus location. The MPAV was evaluated in three cohorts with different staining methods: Hematoxylin &Eosin, CD31 &Hematoxylin, and Ki-67 and where most of the nuclei were unevenly and imprecisely stained. Across a total of 47 images and nearly 17,700 manually labeled nuclei serving as the ground truth, MPAV was able to achieve a superior performance, with an area under the precision-recall curve (AUC) of 0.73. Additionally, MPAV also outperformed three state-of-the-art nuclei detection methods, a single pass voting method, a multi-pass voting method, and a deep learning based method.

Size-Invariant Detection of Cell Nuclei in Microscopy Images

Accurate detection of individual cell nuclei in microscopy images is an essential and fundamental task for many biological studies. In particular, multivariate fluorescence microscopy is used to observe different aspects of cells in cultures. Manual detection of individual cell nuclei by visual inspection is time consuming, and prone to induce subjective bias. This makes automatic detection of cell nuclei essential for large-scale, objective studies of cell cultures. Blur, clutter, bleed-through, imaging noise and touching and partially overlapping nuclei with varying sizes and shapes make automated detection of individual cell nuclei a challenging task using image analysis. In this paper we propose a new automated method for fast and robust detection of individual cell nuclei based on their radial symmetric nature in fluorescence in-situ hybridization (FISH) images obtained via confocal microscopy. The main contributions are two-fold. 1) This work presents a more accurate cell nucleus detection system using the fast radial symmetry transform (FRST). 2) The proposed cell nucleus detection system is robust against most occlusions and variations in size and moderate shape deformations. We evaluate the performance of the proposed algorithm using precision/recall rates, Fβ-score and root-mean-squared distance (RMSD) and show that our algorithm provides improved detection accuracy compared to existing algorithms.

Robust Cell Detection of Histopathological Brain Tumor Images Using Sparse Reconstruction and Adaptive Dictionary Selection

Successful diagnostic and prognostic stratification, treatment outcome prediction, and therapy planning depend on reproducible and accurate pathology analysis. Computer aided diagnosis (CAD) is a useful tool to help doctors make better decisions in cancer diagnosis and treatment. Accurate cell detection is often an essential prerequisite for subsequent cellular analysis. The major challenge of robust brain tumor nuclei/cell detection is to handle significant variations in cell appearance and to split touching cells. In this paper, we present an automatic cell detection framework using sparse reconstruction and adaptive dictionary learning. The main contributions of our method are: 1) A sparse reconstruction based approach to split touching cells; 2) An adaptive dictionary learning method used to handle cell appearance variations. The proposed method has been extensively tested on a data set with more than 2000 cells extracted from 32 whole slide scanned images. The automatic cell detection results are compared with the manually annotated ground truth and other state-of-the-art cell detection algorithms. The proposed method achieves the best cell detection accuracy with a F1 score = 0.96.

Effects of Noninhibitory Serpin Maspin on the Actin Cytoskeleton: A Quantitative Image Modeling Approach

Recent developments in quantitative image analysis allow us to interrogate confocal microscopy images to answer biological questions. Clumped and layered cell nuclei and cytoplasm in confocal images challenges the ability to identify subcellular compartments. To date, there is no perfect image analysis method to identify cytoskeletal changes in confocal images. Here, we present a multidisciplinary study where an image analysis model was developed to allow quantitative measurements of changes in the cytoskeleton of cells with different maspin exposure. Maspin, a noninhibitory serpin influences cell migration, adhesion, invasion, proliferation, and apoptosis in ways that are consistent with its identification as a tumor metastasis suppressor. Using different cell types, we tested the hypothesis that reduction in cell migration by maspin would be reflected in the architecture of the actin cytoskeleton. A hybrid marker-controlled watershed segmentation technique was used to segment the nuclei, cytoplasm, and ruffling regions before measuring cytoskeletal changes. This was informed by immunohistochemical staining of cells transfected stably or transiently with maspin proteins, or with added bioactive peptides or protein. Image analysis results showed that the effects of maspin were mirrored by effects on cell architecture, in a way that could be described quantitatively.

Automatic Nuclei Detection Based on Generalized Laplacian of Gaussian Filters

Efficient and accurate detection of cell nuclei is an important step toward automatic analysis in histopathology. In this work, we present an automatic technique based on generalized Laplacian of Gaussian (gLoG) filter for nuclei detection in digitized histological images. The proposed technique first generates a bank of gLoG kernels with different scales and orientations and then performs convolution between directional gLoG kernels and the candidate image to obtain a set of response maps. The local maxima of response maps are detected and clustered into different groups by mean-shift algorithm based on their geometrical closeness. The point which has the maximum response in each group is finally selected as the nucleus seed. Experimental results on two datasets show that the proposed technique provides a superior performance in nuclei detection compared to existing techniques.

Iterative h-minima-based marker-controlled watershed for cell nucleus segmentation

Automated microscopy imaging systems facilitate high-throughput screening in molecular cellular biology research. The first step of these systems is cell nucleus segmentation, which has a great impact on the success of the overall system. The marker-controlled watershed is a technique commonly used by the previous studies for nucleus segmentation. These studies define their markers finding regional minima on the intensity/gradient and/or distance transform maps. They typically use the h-minima transform beforehand to suppress noise on these maps. The selection of the h value is critical; unnecessarily small values do not sufficiently suppress the noise, resulting in false and oversegmented markers, and unnecessarily large ones suppress too many pixels, causing missing and undersegmented markers. Because cell nuclei show different characteristics within an image, the same h value may not work to define correct markers for all the nuclei. To address this issue, in this work, we propose a new watershed algorithm that iteratively identifies its markers, considering a set of different h values. In each iteration, the proposed algorithm defines a set of candidates using a particular h value and selects the markers from those candidates provided that they fulfill the size requirement. Working with widefield fluorescence microscopy images, our experiments reveal that the use of multiple h values in our iterative algorithm leads to better segmentation results, compared to its counterparts. © 2016 International Society for Advancement of Cytometry.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images

Automated nuclear detection is a critical step for a number of computer assisted pathology related image analysis algorithms such as for automated grading of breast cancer tissue specimens. The Nottingham Histologic Score system is highly correlated with the shape and appearance of breast cancer nuclei in histopathological images. However, automated nucleus detection is complicated by 1) the large number of nuclei and the size of high resolution digitized pathology images, and 2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently there has been interest in the application of "Deep Learning" strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this paper, a Stacked Sparse Autoencoder (SSAE), an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer. The SSAE learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. A sliding window operation is applied to each image in order to represent image patches via high-level features obtained via the auto-encoder, which are then subsequently fed to a classifier which categorizes each image patch as nuclear or non-nuclear. Across a cohort of 500 histopathological images (2200 × 2200) and approximately 3500 manually segmented individual nuclei serving as the groundtruth, SSAE was shown to have an improved F-measure 84.49% and an average area under Precision-Recall curve (AveP) 78.83%. The SSAE approach also out-performed nine other state of the art nuclear detection strategies.

High-throughput histopathological image analysis via robust cell segmentation and hashing

Computer-aided diagnosis of histopathological images usually requires to examine all cells for accurate diagnosis. Traditional computational methods may have efficiency issues when performing cell-level analysis. In this paper, we propose a robust and scalable solution to enable such analysis in a real-time fashion. Specifically, a robust segmentation method is developed to delineate cells accurately using Gaussian-based hierarchical voting and repulsive balloon model. A large-scale image retrieval approach is also designed to examine and classify each cell of a testing image by comparing it with a massive database, e.g., half-million cells extracted from the training dataset. We evaluate this proposed framework on a challenging and important clinical use case, i.e., differentiation of two types of lung cancers (the adenocarcinoma and squamous carcinoma), using thousands of lung microscopic tissue images extracted from hundreds of patients. Our method has achieved promising accuracy and running time by searching among half-million cells .

Segmentation of Overlapping Elliptical Objects in Silhouette Images

Segmentation of partially overlapping objects with a known shape is needed in an increasing amount of various machine vision applications. This paper presents a method for segmentation of clustered partially overlapping objects with a shape that can be approximated using an ellipse. The method utilizes silhouette images, which means that it requires only that the foreground (objects) and background can be distinguished from each other. The method starts with seedpoint extraction using bounded erosion and fast radial symmetry transform. Extracted seedpoints are then utilized to associate edge points to objects in order to create contour evidence. Finally, contours of the objects are estimated by fitting ellipses to the contour evidence. The experiments on one synthetic and two different real data sets showed that the proposed method outperforms two current state-of-art approaches in overlapping objects segmentation.

Segmentation of densely populated cell nuclei from confocal image stacks using 3D non-parametric shape priors

An approach to jointly estimate 3D shapes and poses of stained nuclei from confocal microscopy images, using statistical prior information, is presented. Extracting nuclei boundaries from our experimental images of cell migration is challenging due to clustered nuclei and variations in their shapes. This issue is formulated as a maximum a posteriori estimation problem. By incorporating statistical prior models of 3D nuclei shapes into level set functions, the active contour evolutions applied on the images is constrained. A 3D alignment algorithm is developed to build the training databases and to match contours obtained from the images to them. To address the issue of aligning the model over multiple clustered nuclei, a watershed-like technique is used to detect and separate clustered regions prior to active contour evolution. Our method is tested on confocal images of endothelial cells in microfluidic devices, compared with existing approaches.

Segmenting overlapping cell nuclei in digital histopathology images

Automatic quantification of cell nuclei in immunostained images is highly desired by pathologists in diagnosis. In this paper, we present a new approach for the segmentation of severely clustered overlapping nuclei. The proposed approach first involves applying a combined global and local threshold method to extract foreground regions. In order to segment clustered overlapping nuclei in the foreground regions, seed markers are obtained by utilizing morphological filtering and intensity based region growing. Seeded watershed is then applied and clustered nuclei are separated. As pixels corresponding to stained cellular cytoplasm can be falsely identified as belonging to nuclei, a post processing step identifying positive nuclei pixels is added to eliminate these false pixels. This new approach has been tested on a set of manually labeled Tissue Microarray (TMA) and Whole Slide Images (WSI) colorectal cancers stained for the biomarker P53. Experimental results show that it outperformed currently available state of the art methods in nuclei segmentation.

Clustered nuclei splitting via curvature information and gray-scale distance transform

Clusters or clumps of cells or nuclei are frequently observed in two dimensional images of thick tissue sections. Correct and accurate segmentation of overlapping cells and nuclei is important for many biological and biomedical applications. Many existing algorithms split clumps through the binarization of the input images; therefore, the intensity information of the original image is lost during this process. In this paper, we present a curvature information, gray scale distance transform, and shortest path splitting line-based algorithm which can make full use of the concavity and image intensity information to find out markers, each of which represents an individual object, and detect accurate splitting lines between objects using shortest path and junction adjustment. The proposed algorithm is tested on both synthetic and real nuclei images. Experiment results show that the performance of the proposed method is better than that of marker-controlled watershed method and ellipse fitting method.

An efficient technique for nuclei segmentation based on ellipse descriptor analysis and improved seed detection algorithm

In this paper, we propose an efficient method for segmenting cell nuclei in the skin histopathological images. The proposed technique consists of four modules. First, it separates the nuclei regions from the background with an adaptive threshold technique. Next, an elliptical descriptor is used to detect the isolated nuclei with elliptical shapes. This descriptor classifies the nuclei regions based on two ellipticity parameters. Nuclei clumps and nuclei with irregular shapes are then localized by an improved seed detection technique based on voting in the eroded nuclei regions. Finally, undivided nuclei regions are segmented by a marked watershed algorithm. Experimental results on 114 different image patches indicate that the proposed technique provides a superior performance in nuclei detection and segmentation.

An improved joint optimization of multiple level set functions for the segmentation of overlapping cervical cells

In this paper, we present an improved algorithm for the segmentation of cytoplasm and nuclei from clumps of overlapping cervical cells. This problem is notoriously difficult because of the degree of overlap among cells, the poor contrast of cell cytoplasm and the presence of mucus, blood, and inflammatory cells. Our methodology addresses these issues by utilizing a joint optimization of multiple level set functions, where each function represents a cell within a clump, that have both unary (intracell) and pairwise (intercell) constraints. The unary constraints are based on contour length, edge strength, and cell shape, while the pairwise constraint is computed based on the area of the overlapping regions. In this way, our methodology enables the analysis of nuclei and cytoplasm from both free-lying and overlapping cells. We provide a systematic evaluation of our methodology using a database of over 900 images generated by synthetically overlapping images of free-lying cervical cells, where the number of cells within a clump is varied from 2 to 10 and the overlap coefficient between pairs of cells from 0.1 to 0.5. This quantitative assessment demonstrates that our methodology can successfully segment clumps of up to 10 cells, provided the overlap between pairs of cells is <;0.2. Moreover, if the clump consists of three or fewer cells, then our methodology can successfully segment individual cells even when the overlap is ~0.5. We also evaluate our approach quantitatively and qualitatively on a set of 16 extended depth of field images, where we are able to segment a total of 645 cells, of which only ~10% are free-lying. Finally, we demonstrate that our method of cell nuclei segmentation is competitive when compared with the current state of the art.

A robust algorithm for segmenting and tracking clustered cells in time-lapse fluorescent microscopy

We present herein a robust algorithm for cell tracking in a sequence of time-lapse 2-D fluorescent microscopy images. Tracking is performed automatically via a multiphase active contours algorithm adapted to the segmentation of clustered nuclei with obscure boundaries. An ellipse fitting method is applied to avoid problems typically associated with clustered, overlapping, or dying cells, and to obtain more accurate segmentation and tracking results. We provide quantitative validation of results obtained with this new algorithm by comparing them to the results obtained from the established CellProfiler, MTrack2 (plugin for Fiji), and LSetCellTracker software.

Methods for nuclei detection, segmentation, and classification in digital histopathology: a review-current status and future potential

Digital pathology represents one of the major evolutions in modern medicine. Pathological examinations constitute the gold standard in many medical protocols, and also play a critical and legal role in the diagnosis process. In the conventional cancer diagnosis, pathologists analyze biopsies to make diagnostic and prognostic assessments, mainly based on the cell morphology and architecture distribution. Recently, computerized methods have been rapidly evolving in the area of digital pathology, with growing applications related to nuclei detection, segmentation, and classification. In cancer research, these approaches have played, and will continue to play a key (often bottleneck) role in minimizing human intervention, consolidating pertinent second opinions, and providing traceable clinical information. Pathological studies have been conducted for numerous cancer detection and grading applications, including brain, breast, cervix, lung, and prostate cancer grading. Our study presents, discusses, and extracts the major trends from an exhaustive overview of various nuclei detection, segmentation, feature computation, and classification techniques used in histopathology imagery, specifically in hematoxylin-eosin and immunohistochemical staining protocols. This study also enables us to measure the challenges that remain, in order to reach robust analysis of whole slide images, essential high content imaging with diagnostic biomarkers and prognosis support in digital pathology.

Automatic Myonuclear Detection in Isolated Single Muscle Fibers Using Robust Ellipse Fitting and Sparse Representation

Accurate and robust detection of myonuclei in isolated single muscle fibers is required to calculate myonuclear domain size. However, this task is challenging because: 1) shape and size variations of the nuclei, 2) overlapping nuclear clumps, and 3) multiple z-stack images with out-of-focus regions. In this paper, we have proposed a novel automatic detection algorithm to robustly quantify myonuclei in isolated single skeletal muscle fibers. The original z-stack images are first converted into one all-in-focus image using multi-focus image fusion. A sufficient number of ellipse fitting hypotheses are then generated from the myonuclei contour segments using heteroscedastic errors-in-variables (HEIV) regression. A set of representative training samples and a set of discriminative features are selected by a two-stage sparse model. The selected samples with representative features are utilized to train a classifier to select the best candidates. A modified inner geodesic distance based mean-shift clustering algorithm is used to produce the final nuclei detection results. The proposed method was extensively tested using 42 sets of z-stack images containing over 1,500 myonuclei. The method demonstrates excellent results that are better than current state-of-the-art approaches.

Antiproliferative efficacies but minor drug transporter inducing effects of paclitaxel, cisplatin, or 5-fluorouracil in a murine xenograft model for head and neck squamous cell carcinoma

Drug-induced multidrug resistance (MDR) has been linked to overexpression of drug transporting proteins in head and neck squamous cell carcinoma (HNSCC) in vitro. The aim of this work was to reassess these findings in a murine xenograft model. NOD-SCID mice xenotransplanted with 10 (6) HNO97 cells were treated for four consecutive weeks with weekly paclitaxel, biweekly cisplatin (both intraperitoneal), or 5-fluorouracil (5-FU, administered by osmotic pump). Tumor volume and body weight were weekly documented. Expression of drug transporters and Ki-67 marker were examined using quantitative real-time polymerase chain reaction and/or immunohistochemistry. Both paclitaxel and cisplatin significantly reduced tumor volumes after 2-3 weeks. 5-FU-treated animals had significantly lower body weights after 2 or 4 weeks of chemotherapy. None of the drugs affected expression of drug transporters at the mRNA level. However, P-glycoprotein (Pgp) protein expression was increased by paclitaxel (P<0.01). Ki-67 expression did not change during treatment irrespective of the drug applied. Paclitaxel and cisplatin are effectively tumor volume reducing drugs in a murine xenograft model of HNSCC. Paclitaxel enhanced Pgp expression at the protein level, but not at the mRNA level suggesting transcriptional induction to be of minor relevance. In contrast, posttranscriptional mechanisms or Darwinian selection of intrinsically drug transporter overexpressing MDR cells might lead to iatrogenic chemotherapy resistance in HNSCC.

A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching

We describe a new supervised learning-based template matching approach for segmenting cell nuclei from microscopy images. The method uses examples selected by a user for building a statistical model that captures the texture and shape variations of the nuclear structures from a given dataset to be segmented. Segmentation of subsequent, unlabeled, images is then performed by finding the model instance that best matches (in the normalized cross correlation sense) local neighborhood in the input image. We demonstrate the application of our method to segmenting nuclei from a variety of imaging modalities, and quantitatively compare our results to several other methods. Quantitative results using both simulated and real image data show that, while certain methods may work well for certain imaging modalities, our software is able to obtain high accuracy across several imaging modalities studied. Results also demonstrate that, relative to several existing methods, the template-based method we propose presents increased robustness in the sense of better handling variations in illumination, variations in texture from different imaging modalities, providing more smooth and accurate segmentation borders, as well as handling better cluttered nuclei.

Separation of overlapped particles in digital holographic microscopy

In this paper, we present a procedure to separate aggregates of overlapped particles in digital holograms, based on a focus plane analysis applied to each particle. The method can be applied either on phase or on amplitude objects, according that each object has a border in one focus plane. Numerical simulations are performed to quantify the robustness of the process by increasing the overlapping areas between the particles. The separation algorithm is successfully demonstrated experimentally on different types of aggregates.

Smart markers for watershed-based cell segmentation

Automated cell imaging systems facilitate fast and reliable analysis of biological events at the cellular level. In these systems, the first step is usually cell segmentation that greatly affects the success of the subsequent system steps. On the other hand, similar to other image segmentation problems, cell segmentation is an ill-posed problem that typically necessitates the use of domain-specific knowledge to obtain successful segmentations even by human subjects. The approaches that can incorporate this knowledge into their segmentation algorithms have potential to greatly improve segmentation results. In this work, we propose a new approach for the effective segmentation of live cells from phase contrast microscopy. This approach introduces a new set of "smart markers" for a marker-controlled watershed algorithm, for which the identification of its markers is critical. The proposed approach relies on using domain-specific knowledge, in the form of visual characteristics of the cells, to define the markers. We evaluate our approach on a total of 1,954 cells. The experimental results demonstrate that this approach, which uses the proposed definition of smart markers, is quite effective in identifying better markers compared to its counterparts. This will, in turn, be effective in improving the segmentation performance of a marker-controlled watershed algorithm.

Overlapping cell nuclei segmentation using a spatially adaptive active physical model

A method for the segmentation of overlapping nuclei is presented, which combines local characteristics of the nuclei boundary and a priori knowledge about the expected shape of the nuclei. A deformable model whose behavior is driven by physical principles is trained on images containing a single nuclei, and attributes of the shapes of the nuclei are expressed in terms of modal analysis. Based on the estimated modal distribution and driven by the image characteristics, we develop a framework to detect and describe the unknown nuclei boundaries in images containing two overlapping nuclei. The problem of the estimation of an accurate nucleus boundary in the overlapping areas is successfully addressed with the use of appropriate weight parameters that control the contribution of the image force in the total energy of the deformable model. The proposed method was evaluated using 152 images of conventional Pap smears, each containing two overlapping nuclei. Comparisons with other segmentation methods indicate that our method produces more accurate nuclei boundaries which are closer to the ground truth.

Efficient globally optimal segmentation of cells in fluorescence microscopy images using level sets and convex energy functionals

In high-throughput applications, accurate and efficient segmentation of cells in fluorescence microscopy images is of central importance for the quantification of protein expression and the understanding of cell function. We propose an approach for segmenting cell nuclei which is based on active contours using level sets and convex energy functionals. Compared to previous work, our approach determines the global solution. Thus, the approach does not suffer from local minima and the segmentation result does not depend on the initialization. We consider three different well-known energy functionals for active contour-based segmentation and introduce convex formulations of these functionals. We also suggest a numeric approach for efficiently computing the solution. The performance of our approach has been evaluated using fluorescence microscopy images from different experiments comprising different cell types. We have also performed a quantitative comparison with previous segmentation approaches.

Automatic calculation of tree diameter from stereoscopic image pairs using digital image processing

Automatic operations play an important role in societies by saving time and improving efficiency. In this paper, we apply the digital image processing method to the field of lumbering to automatically calculate tree diameters in order to reduce culler work and enable a third party to verify tree diameters. To calculate the cross-sectional diameter of a tree, the image was first segmented by the marker-controlled watershed transform algorithm based on the hue saturation intensity (HSI) color model. Then, the tree diameter was obtained by measuring the area of every isolated region in the segmented image. Finally, the true diameter was calculated by multiplying the diameter computed in the image and the scale, which was derived from the baseline and disparity of correspondence points from stereoscopic image pairs captured by rectified configuration cameras.

Learning histopathological patterns

Aims:

The aim was to demonstrate a method for automated image analysis of immunohistochemically stained tissue samples for extracting features that correlate with patient disease. We address the problem of quantifying tumor tissue and segmenting and counting cell nuclei.

Materials and Methods:

Our method utilizes a flexible segmentation method based on sparse coding trained from representative image samples. Nuclei counting is based on a nucleus model that takes size, shape, and nucleus probability into account. Nuclei clustering and overlays are resolved using a gray-weighted distance transform. We obtain a probability measure for pixels belonging to a nucleus from our segmentation procedure. Experiments are carried out on two sets of immunohistochemically stained images – one set based on the estrogen receptor (ER) and the other on antigen KI-67. For the nuclei separation we have selected 207 ER image samples from 58 tissue micro array-cores corresponding to 58 patients and 136 KI-67 image samples also from 58 cores. The images are hand-annotated by marking the center position of each nucleus. For the ER data we have a total of 1006 nuclei and for the KI-67 we have 796 nuclei. Segmentation performance was evaluated in terms of missing nuclei, falsely detected nuclei, and multiple detections. The proposed method is compared to state-of-the-art Bayesian classification.

Statistical analysis used:

The performance of the proposed method and a state-of-the-art algorithm including variations thereof is compared using the Wilcoxon rank sum test.

Results:

For both the ER experiment and the KI-67 experiment the proposed method exhibits lower error rates than the state-of-the-art method. Total error rates were 4.8 % and 7.7 % in the two experiments, corresponding to an average of 0.23 and 0.45 errors per image, respectively. The Wilcoxon rank sum tests show statistically significant improvements over the state-of-the-art method.

Conclusions:

We have demonstrated a method and obtained good performance compared to state-of-the-art nuclei separation. The segmentation procedure is simple, highly flexible, and we demonstrate how it, in addition to the nuclei separation, can perform precise segmentation of cancerous tissue. The complexity of the segmentation procedure is linear in the image size and the nuclei separation is linear in the number of nuclei. Additionally the method can be parallelized to obtain high-speed computations.