Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set

Deep learning framework for automated goblet cell density analysis in in-vivo rabbit conjunctiva

Goblet cells (GCs) in the conjunctiva are specialized epithelial cells secreting mucins for the mucus layer of protective tear film and playing immune tolerance functions for ocular surface health. Because GC loss is observed in various ocular surface diseases, GC examination is important for precision diagnosis. Moxifloxacin-based fluorescence microscopy (MBFM) was recently developed for non-invasive high-contrast GC visualization. MBFM showed promise for GC examination by high-speed large-area imaging and a robust analysis method is needed to provide GC information. In this study, we developed a deep learning framework for GC image analysis, named dual-channel attention U-Net (DCAU-Net). Dual-channel convolution was used both to extract the overall image texture and to acquire the GC morphological characteristics. A global channel attention module was adopted by combining attention algorithms and channel-wise pooling. DCAU-Net showed 93.1% GC segmentation accuracy and 94.3% GC density estimation accuracy. Further application to both normal and ocular surface damage rabbit models revealed the spatial variations of both GC density and size in normal rabbits and the decreases of both GC density and size in damage rabbit models during recovery after acute damage. The GC analysis results were consistent with histology. Together with the non-invasive high-contrast imaging method, DCAU-Net would provide GC information for the diagnosis of ocular surface diseases.

Learning with limited target data to detect cells in cross-modality images

Deep neural networks have achieved excellent cell or nucleus quantification performance in microscopy images, but they often suffer from performance degradation when applied to cross-modality imaging data. Unsupervised domain adaptation (UDA) based on generative adversarial networks (GANs) has recently improved the performance of cross-modality medical image quantification. However, current GAN-based UDA methods typically require abundant target data for model training, which is often very expensive or even impossible to obtain for real applications. In this paper, we study a more realistic yet challenging UDA situation, where (unlabeled) target training data is limited and previous work seldom delves into cell identification. We first enhance a dual GAN with task-specific modeling, which provides additional supervision signals to assist with generator learning. We explore both single-directional and bidirectional task-augmented GANs for domain adaptation. Then, we further improve the GAN by introducing a differentiable, stochastic data augmentation module to explicitly reduce discriminator overfitting. We examine source-, target-, and dual-domain data augmentation for GAN enhancement, as well as joint task and data augmentation in a unified GAN-based UDA framework. We evaluate the framework for cell detection on multiple public and in-house microscopy image datasets, which are acquired with different imaging modalities, staining protocols and/or tissue preparations. The experiments demonstrate that our method significantly boosts performance when compared with the reference baseline, and it is superior to or on par with fully supervised models that are trained with real target annotations. In addition, our method outperforms recent state-of-the-art UDA approaches by a large margin on different datasets.

A Comprehensive Landscape of Imaging Feature-Associated RNA Expression Profiles in Human Breast Tissue

The expression abundance of transcripts in nondiseased breast tissue varies among individuals. The association study of genotypes and imaging phenotypes may help us to understand this individual variation. Since existing reports mainly focus on tumors or lesion areas, the heterogeneity of pathological image features and their correlations with RNA expression profiles for nondiseased tissue are not clear. The aim of this study is to discover the association between the nucleus features and the transcriptome-wide RNAs. We analyzed both microscopic histology images and RNA-sequencing data of 456 breast tissues from the Genotype-Tissue Expression (GTEx) project and constructed an automatic computational framework. We classified all samples into four clusters based on their nucleus morphological features and discovered feature-specific gene sets. The biological pathway analysis was performed on each gene set. The proposed framework evaluates the morphological characteristics of the cell nucleus quantitatively and identifies the associated genes. We found image features that capture population variation in breast tissue associated with RNA expressions, suggesting that the variation in expression pattern affects population variation in the morphological traits of breast tissue. This study provides a comprehensive transcriptome-wide view of imaging-feature-specific RNA expression for healthy breast tissue. Such a framework could also be used for understanding the connection between RNA expression and morphology in other tissues and organs. Pathway analysis indicated that the gene sets we identified were involved in specific biological processes, such as immune processes.

Geometric deep learning reveals the spatiotemporal features of microscopic motion

The characterization of dynamical processes in living systems provides important clues for their mechanistic interpretation and link to biological functions. Owing to recent advances in microscopy techniques, it is now possible to routinely record the motion of cells, organelles and individual molecules at multiple spatiotemporal scales in physiological conditions. However, the automated analysis of dynamics occurring in crowded and complex environments still lags behind the acquisition of microscopic image sequences. Here we present a framework based on geometric deep learning that achieves the accurate estimation of dynamical properties in various biologically relevant scenarios. This deep-learning approach relies on a graph neural network enhanced by attention-based components. By processing object features with geometric priors, the network is capable of performing multiple tasks, from linking coordinates into trajectories to inferring local and global dynamic properties. We demonstrate the flexibility and reliability of this approach by applying it to real and simulated data corresponding to a broad range of biological experiments.

Recent Advances of Deep Learning for Computational Histopathology: Principles and Applications

With the remarkable success of digital histopathology, we have witnessed a rapid expansion of the use of computational methods for the analysis of digital pathology and biopsy image patches. However, the unprecedented scale and heterogeneous patterns of histopathological images have presented critical computational bottlenecks requiring new computational histopathology tools. Recently, deep learning technology has been extremely successful in the field of computer vision, which has also boosted considerable interest in digital pathology applications. Deep learning and its extensions have opened several avenues to tackle many challenging histopathological image analysis problems including color normalization, image segmentation, and the diagnosis/prognosis of human cancers. In this paper, we provide a comprehensive up-to-date review of the deep learning methods for digital H&E-stained pathology image analysis. Specifically, we first describe recent literature that uses deep learning for color normalization, which is one essential research direction for H&E-stained histopathological image analysis. Followed by the discussion of color normalization, we review applications of the deep learning method for various H&E-stained image analysis tasks such as nuclei and tissue segmentation. We also summarize several key clinical studies that use deep learning for the diagnosis and prognosis of human cancers from H&E-stained histopathological images. Finally, online resources and open research problems on pathological image analysis are also provided in this review for the convenience of researchers who are interested in this exciting field.

Whale Optimizer-Based Clustering for Breast Histopathology Image Segmentation

Breast histopathology image segmentation is a complex task due to indiscernibly correlated and noisy regions of interest. Breast histopathological images are composed of different types of cells. Some of these cells can be harmful for humans due to the presence of cancer. Under such circumstances, many segmentation techniques for automatic detection of cancer cells have been proposed considering clustering schemes. However, such clustering methodologies are sensitive to initial cluster centers, which promote false-positive solutions. This paper presents the use of the Whale Optimization Algorithm (WOA) for proper clustering segmentation of breast histopathological images to overcome clustering issues. Also, a rigorous comparative study is conducted among the proposed approach and several state-of-art Nature-Inspired Optimization Algorithms (NIOAs) and traditional clustering techniques. The numerical results indicate that the proposed approach outperforms the other utilized clustering methods in terms of precision, robustness, and quality of the segmented outputs.

Evaluating Cancer-Related Biomarkers Based on Pathological Images: A Systematic Review

Many diseases are accompanied by changes in certain biochemical indicators called biomarkers in cells or tissues. A variety of biomarkers, including proteins, nucleic acids, antibodies, and peptides, have been identified. Tumor biomarkers have been widely used in cancer risk assessment, early screening, diagnosis, prognosis, treatment, and progression monitoring. For example, the number of circulating tumor cell (CTC) is a prognostic indicator of breast cancer overall survival, and tumor mutation burden (TMB) can be used to predict the efficacy of immune checkpoint inhibitors. Currently, clinical methods such as polymerase chain reaction (PCR) and next generation sequencing (NGS) are mainly adopted to evaluate these biomarkers, which are time-consuming and expansive. Pathological image analysis is an essential tool in medical research, disease diagnosis and treatment, functioning by extracting important physiological and pathological information or knowledge from medical images. Recently, deep learning-based analysis on pathological images and morphology to predict tumor biomarkers has attracted great attention from both medical image and machine learning communities, as this combination not only reduces the burden on pathologists but also saves high costs and time. Therefore, it is necessary to summarize the current process of processing pathological images and key steps and methods used in each process, including: (1) pre-processing of pathological images, (2) image segmentation, (3) feature extraction, and (4) feature model construction. This will help people choose better and more appropriate medical image processing methods when predicting tumor biomarkers.

Nuclei Segmentation on Histopathology Images of Breast Carcinoma

With the use of computer-aided diagnostic systems, the automatic detection and segmentation of the cell nuclei have become essential in pathology due to cellular nuclei counting and nuclear pleomorphism analysis are critical for the classification and grading of breast cancer histopathology. This work describes a methodology for automatic detection and segmentation of cellular nuclei in breast cancer histopathology images obtained from the BreakHis database, the Standford tissue microarray database, and the Breast Cancer Cell Segmentation database. The proposed scheme is based on the characterization of Hematoxylin and Eosin (H&E) staining, size, and shape features. In addition, we use the information obtained from morphological transformations and adaptive intensity adjustments to detect and separate each cell nucleus detected in the image. The segmentation was carried out by testing the proposed methodology in a histological breast cancer database that provides the associated groundtruth segmentation. Subsequently, the Sørensen-Dice similarity coefficient was calculated to analyze the suitability of the results.Clinical relevance- In this work, the detection and segmentation of cell nuclei in breast cancer histological images are carried out automatically. The method can identify cell nuclei regardless of variations in the level of staining and image magnification. Moreover, a granulometric analysis of the components allows identifying cell clumps and segment them into individual cell nuclei. Improved identification of cell nuclei under different image conditions was demonstrated to reach a sensitivity average of 0.76 ± 0.12. The results provide a base for further and complex processes such as cell counting, feature analysis, and nuclear pleomorphism, which are relevant tasks in the evaluation and diagnostic performed by the expert pathologist.

Automatic Whole Slide Pathology Image Diagnosis Framework via Unit Stochastic Selection and Attention Fusion

Pathology tissue slides are taken as the gold standard for the diagnosis of most cancer diseases. Automatic pathology slide diagnosis is still a challenging task for researchers because of the high-resolution, significant morphological variation, and ambiguity between malignant and benign regions in whole slide images (WSIs). In this study, we introduce a general framework to automatically diagnose different types of WSIs via unit stochastic selection and attention fusion. For example, a unit can denote a patch in a histopathology slide or a cell in a cytopathology slide. To be specific, we first train a unit-level convolutional neural network (CNN) to perform two tasks: constructing feature extractors for the units and for estimating a unit's non-benign probability. Then we use our novel stochastic selection algorithm to choose a small subset of units that are most likely to be non-benign, referred to as the Units Of Interest (UOI), as determined by the CNN. Next, we use the attention mechanism to fuse the representations of the UOI to form a fixed-length descriptor for the WSI's diagnosis. We evaluate the proposed framework on three datasets: histological thyroid frozen sections, histological colonoscopy tissue slides, and cytological cervical pap smear slides. The framework achieves diagnosis accuracies higher than 0.8 and AUC values higher than 0.85 in all three applications. Experiments demonstrate the generality and effectiveness of the proposed framework and its potentiality for clinical applications.

A whole-slide image grading benchmark and tissue classification for cervical cancer precursor lesions with inter-observer variability

The cervical cancer developing from the precancerous lesions caused by the human papillomavirus (HPV) has been one of the preventable cancers with the help of periodic screening. Cervical intraepithelial neoplasia (CIN) and squamous intraepithelial lesion (SIL) are two types of grading conventions widely accepted by pathologists. On the other hand, inter-observer variability is an important issue for final diagnosis. In this paper, a whole-slide image grading benchmark for cervical cancer precursor lesions is created and the "Uterine Cervical Cancer Database" introduced in this article is the first publicly available cervical tissue microscopy image dataset. In addition, a morphological feature representing the angle between the basal membrane (BM) and the major axis of each nucleus in the tissue is proposed. The presence of papillae of the cervical epithelium and overlapping cell problems are also discussed. Besides that, the inter-observer variability is also evaluated by thorough comparisons among decisions of pathologists, as well as the final diagnosis.

Resolving Colliding Larvae by Fitting ASM to Random Walker-Based Pre-Segmentations

Drosophila melanogaster is an important model organism for research in neuro- and behavioral biology. Automated studies of their locomotion are crucial to link sensory input and neural processing to motor output which has led to numerous vision-based tracking systems. However, most of these approaches share the inability to segment the contours of colliding animals causing identity losses, appearing and disappearing animals, and the absence of posture and motion related measurements during the time of the collision. We present a novel collision resolution algorithm enabling an accurate contour segmentation of multiple touching Drosophila larvae. Our algorithm utilizes an adapted active shape model (ASM) to learn a low dimensional posture space which is fitted to random-walker generated pre-segmentations. We evaluate our collision resolution algorithm using three publicly available datasets and compare it with the current state-of-the-art methods. In addition, we introduce a refined dataset enabling a segmentation evaluation on the level of pixel accuracy. The results demonstrate that our approach outperforms the state-of-the-art approaches in both accuracy and computational time. We will incorporate this algorithm into our widely used tracking program to improve the statistical strength of the behavioral quantification and allow marker-free studies of interacting Drosophila larvae.

Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics

Purpose of Review

Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images.

Recent Findings

Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond.

Summary

Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.

piNET-An Automated Proliferation Index Calculator Framework for Ki67 Breast Cancer Images

In this work, a novel proliferation index (PI) calculator for Ki67 images called piNET is proposed. It is successfully tested on four datasets, from three scanners comprised of patches, tissue microarrays (TMAs) and whole slide images (WSI), representing a diverse multi-centre dataset for evaluating Ki67 quantification. Compared to state-of-the-art methods, piNET consistently performs the best over all datasets with an average PI difference of 5.603%, PI accuracy rate of 86% and correlation coefficient R = 0.927. The success of the system can be attributed to several innovations. Firstly, this tool is built based on deep learning, which can adapt to wide variability of medical images-and it was posed as a detection problem to mimic pathologists' workflow which improves accuracy and efficiency. Secondly, the system is trained purely on tumor cells, which reduces false positives from non-tumor cells without needing the usual pre-requisite tumor segmentation step for Ki67 quantification. Thirdly, the concept of learning background regions through weak supervision is introduced, by providing the system with ideal and non-ideal (artifact) patches that further reduces false positives. Lastly, a novel hotspot analysis is proposed to allow automated methods to score patches from WSI that contain "significant" activity.

Fusion of whole and part features for the classification of histopathological image of breast tissue

Purpose

Nowadays Computer-Aided Diagnosis (CAD) models, particularly those based on deep learning, have been widely used to analyze histopathological images in breast cancer diagnosis. However, due to the limited availability of such images, it is always tedious to train deep learning models that require a huge amount of training data. In this paper, we propose a new deep learning-based CAD framework that can work with less amount of training data.

Methods

We use pre-trained models to extract image features that can then be used with any classifier. Our proposed features are extracted by the fusion of two different types of features (foreground and background) at two levels (whole-level and part-level). Foreground and background feature to capture information about different structures and their layout in microscopic images of breast tissues. Similarly, part-level and whole-level features capture are useful in detecting interesting regions scattered in high-resolution histopathological images at local and whole image levels. At each level, we use VGG16 models pre-trained on ImageNet and Places datasets to extract foreground and background features, respectively. All features are extracted from mid-level pooling layers of such models.

Results

We show that proposed fused features with a Support Vector Classifier (SVM) produce better classification accuracy than recent methods on BACH dataset and our approach is orders of magnitude faster than the best performing recent method (EMS-Net).

Conclusion

We believe that our method would be another alternative in the diagnosis of breast cancer because of performance and prediction time.

Interactive thyroid whole slide image diagnostic system using deep representation

BACKGROUND AND OBJECTIVES

The vast size of the histopathology whole slide image poses formidable challenges to its automatic diagnosis. With the goal of computer-aided diagnosis and the insights that suspicious regions are generally easy to identify in thyroid whole slide images (WSIs), we develop an interactive whole slide diagnostic system for thyroid frozen sections based on the suspicious regions preselected by pathologists.

METHODS

We propose to generate feature representations for the suspicious regions via extracting and fusing patch features using deep neural networks. We then evaluate region classification and retrieval on four classifiers and three supervised hashing methods based on the feature representations. The code is released at https://github.com/PingjunChen/ThyroidInteractive.

RESULTS

We evaluate the proposed system on 345 thyroid frozen sections and achieve 96.1% cross-validated classification accuracy, and retrieval mean average precision (MAP) of 0.972.

CONCLUSIONS

With the participation of pathologists, the system possesses the following four notable advantages compared to directly handling whole slide images: 1) Reduced interference of irrelevant regions; 2) Alleviated computation and memory cost. 3) Fine-grained and precise suspicious region retrieval. 4) Cooperative relationship between pathologists and the diagnostic system. Additionally, experimental results demonstrate the potential of the proposed system on the practical thyroid frozen section diagnosis.

Automated Cell Tracking Using Motion Prediction-Based Matching and Event Handling

Automated cell segmentation and tracking enables the quantification of static and dynamic cell characteristics and is significant for disease diagnosis, treatment, drug development, and other biomedical applications. This paper introduces a method for fully automated cell tracking, lineage construction, and quantification. Cell detection is performed in the joint spatio-temporal domain by a motion diffusion-based Partial Differential Equation (PDE) combined with energy minimizing active contours. In the tracking stage, we adopt a variational joint local-global optical flow technique to determine the motion vector field. We utilize the predicted cell motion jointly with spatial cell features to define a maximum likelihood criterion to find inter-frame cell correspondences assuming Markov dependency. We formulate cell tracking and cell event detection as a graph partitioning problem. We propose a solution obtained by minimization of a global cost function defined over the set of all cell tracks. We construct a cell lineage tree that represents the cell tracks and cell events. Finally, we compute morphological, motility, and diffusivity measures and validate cell tracking against manually generated reference standards. The automated tracking method applied to reference segmentation maps produces an average tracking accuracy score ( TRA) of 99 percent, and the fully automated segmentation and tracking system produces an average TRA of 89 percent.

A deep learning-based framework for lung cancer survival analysis with biomarker interpretation

BACKGROUND

Lung cancer is the leading cause of cancer-related deaths in both men and women in the United States, and it has a much lower five-year survival rate than many other cancers. Accurate survival analysis is urgently needed for better disease diagnosis and treatment management.

RESULTS

In this work, we propose a survival analysis system that takes advantage of recently emerging deep learning techniques. The proposed system consists of three major components. 1) The first component is an end-to-end cellular feature learning module using a deep neural network with global average pooling. The learned cellular representations encode high-level biologically relevant information without requiring individual cell segmentation, which is aggregated into patient-level feature vectors by using a locality-constrained linear coding (LLC)-based bag of words (BoW) encoding algorithm. 2) The second component is a Cox proportional hazards model with an elastic net penalty for robust feature selection and survival analysis. 3) The third commponent is a biomarker interpretation module that can help localize the image regions that contribute to the survival model's decision. Extensive experiments show that the proposed survival model has excellent predictive power for a public (i.e., The Cancer Genome Atlas) lung cancer dataset in terms of two commonly used metrics: log-rank test (p-value) of the Kaplan-Meier estimate and concordance index (c-index).

CONCLUSIONS

In this work, we have proposed a segmentation-free survival analysis system that takes advantage of the recently emerging deep learning framework and well-studied survival analysis methods such as the Cox proportional hazards model. In addition, we provide an approach to visualize the discovered biomarkers, which can serve as concrete evidence supporting the survival model's decision.

Histo-genomics: digital pathology at the forefront of precision medicine

The toughest challenge OMICs face is that they provide extremely high molecular resolution but poor spatial information. Understanding the cellular/histological context of the overwhelming genetic data is critical for a full understanding of the clinical behavior of a malignant tumor. Digital pathology can add an extra layer of information to help visualize in a spatial and microenvironmental context the molecular information of cancer. Thus, histo-genomics provide a unique chance for data integration. In the era of a precision medicine, a four-dimensional (4D) (temporal/spatial) analysis of cancer aided by digital pathology can be a critical step to understand the evolution/progression of different cancers and consequently tailor individual treatment plans. For instance, the integration of molecular biomarkers expression into a three-dimensional (3D) image of a digitally scanned tumor can offer a better understanding of its subtype, behavior, host immune response and prognosis. Using advanced digital image analysis, a larger spectrum of parameters can be analyzed as potential predictors of clinical behavior. Correlation between morphological features and host immune response can be also performed with therapeutic implications. Radio-histomics, or the interface of radiological images and histology is another emerging exciting field which encompasses the integration of radiological imaging with digital pathological images, genomics, and clinical data to portray a more holistic approach to understating and treating disease. These advances in digital slide scanning are not without technical challenges, which will be addressed carefully in this review with quick peek at its future.

Refinement of Convolutional Neural Network Based Cell Nuclei Detection Using Bayesian Inference

Cytological samples provide useful data for cancer diagnostics but their visual analysis under a microscope is tedious and time-consuming. Moreover, some scientific tests indicate that various pathologists can classify the same sample differently or the same pathologist can classify the sample differently if there is a long interval between subsequent examinations. We can help pathologists by providing tools for automatic analysis of cellular structures. Unfortunately, cytological samples usually consist of clumped structures, so it is difficult to extract single cells to measure their morphometric parameters. To deal with this problem, we are proposing a nuclei detection approach, which combines convolutional neural network and Bayesian inference. The input image is preprocessed by the stain separation procedure to extract a blue dye (hematoxylin) which is mainly absorbed by nuclei. Next, a convolutional neural network is trained to provide a semantic segmentation of the image. Finally, the segmentation results are post processed in order to detect nuclei. To do that, we model the nuclei distribution on a plane using marked point process and apply the Besag's iterated conditional modes to find the configuration of ellipses that fit the nuclei distribution. Thanks to this we can represent clusters of occluded cell nuclei as a set of an overlapping ellipses. The accuracy of the proposed method was tested on 50 cytological images of breast cancer. Reference data was generated by the manual labeling of cell nuclei in images. The effectiveness of the proposed method was compared with the marker-controlled watershed. We applied our method and marker controlled watershed to detect nuclei in the semantic segmentation maps generated by the convolutional neural network. The accuracy of nuclei detection is measured as the number of true positive (TP) detections and false positive (FP) detections. It was recorded that the method can detect correctly 93.5% of nuclei (TP) and at the same time it generates only 6.1% of FP. The proposed approach has led to better results than the marker-controlled watershed both in the number of correctly detected nuclei and in the number of false detections.

IHC Color Histograms for Unsupervised Ki67 Proliferation Index Calculation

Automated image analysis tools for Ki67 breast cancer digital pathology images would have significant value if integrated into diagnostic pathology workflows. Such tools would reduce the workload of pathologists, while improving efficiency, and accuracy. Developing tools that are robust and reliable to multicentre data is challenging, however, differences in staining protocols, digitization equipment, staining compounds, and slide preparation can create variabilities in image quality and color across digital pathology datasets. In this work, a novel unsupervised color separation framework based on the IHC color histogram (IHCCH) is proposed for the robust analysis of Ki67 and hematoxylin stained images in multicentre datasets. An "overstaining" threshold is implemented to adjust for background overstaining, and an automated nuclei radius estimator is designed to improve nuclei detection. Proliferation index and F1 scores were compared between the proposed method and manually labeled ground truth data for 30 TMA cores that have ground truths for Ki67+ and Ki67- nuclei. The method accurately quantified the PI over the dataset, with an average proliferation index difference of 3.25%. To ensure the method generalizes to new, diverse datasets, 50 Ki67 TMAs from the Protein Atlas were used to test the validated approach. As the ground truth for this dataset is PI ranges, the automated result was compared to the PI range. The proposed method correctly classified 74 out of 80 TMA images, resulting in a 92.5% accuracy. In addition to these validations experiments, performance was compared to two color-deconvolution based methods, and to six machine learning classifiers. In all cases, the proposed work maintained more consistent (reproducible) results, and higher PI quantification accuracy.

A segmentation method combining probability map and boundary based on multiple fully convolutional networks and repetitive training

Cell nuclei image segmentation technology can help researchers observe each cell's stress response to drug treatment. However, it is still a challenge to accurately segment the adherent cell nuclei. At present, image segmentation based on a fully convolutional network (FCN) is attracting researchers' attention. We propose a multiple FCN architecture and repetitive training (M-FCN-RT) method to learn features of cell nucleus images. In M-FCN-RT, the multiple FCN (M-FCN) architecture is composed of several single FCNs (S-FCNs) with the same structure, and each FCN is used to learn the specific features of image datasets. In this paper, the M-FCN contains three FCNs; FCN_1-2, FCN₃ and FCN_B. FCN_1-2 and FCN₃ are respectively used to learn the spatial features of cell nuclei for generating probability maps to indicate nucleus regions of an image; FCN_B (boundary FCN) is used to learn the edge features of cell nuclei for generating the nucleus boundary. For the training of each FCN, we propose a repetitive training (RT) method to improve the classification accuracy of the model. To segment cell nuclei, we finally propose an algorithm combining the probability map and boundary (PMB) to segment the adherent nuclei. This paper uses a public opening nucleus image dataset to train, verify and evaluate the proposed M-FCN-RT and PMB methods. Our M-FCN-RT method achieves a high Dice similarity coefficient (DSC) of 92.11%, 95.64% and 87.99% on the three types of sub-datasets respectively for probability maps. In addition, segmentation experimental results show the PMB method is more effective and efficient compared with other methods.

Fast ScanNet: Fast and Dense Analysis of Multi-Gigapixel Whole-Slide Images for Cancer Metastasis Detection

Lymph node metastasis is one of the most important indicators in breast cancer diagnosis, that is traditionally observed under the microscope by pathologists. In recent years, with the dramatic advance of high-throughput scanning and deep learning technology, automatic analysis of histology from whole-slide images has received a wealth of interest in the field of medical image computing, which aims to alleviate pathologists' workload and simultaneously reduce misdiagnosis rate. However, the automatic detection of lymph node metastases from whole-slide images remains a key challenge because such images are typically very large, where they can often be multiple gigabytes in size. Also, the presence of hard mimics may result in a large number of false positives. In this paper, we propose a novel method with anchor layers for model conversion, which not only leverages the efficiency of fully convolutional architectures to meet the speed requirement in clinical practice but also densely scans the whole-slide image to achieve accurate predictions on both micro- and macro-metastases. Incorporating the strategies of asynchronous sample prefetching and hard negative mining, the network can be effectively trained. The efficacy of our method is corroborated on the benchmark dataset of 2016 Camelyon Grand Challenge. Our method achieved significant improvements in comparison with the state-of-the-art methods on tumor localization accuracy with a much faster speed and even surpassed human performance on both challenge tasks.

Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison

BACKGROUND

Because of its non-destructive nature, label-free imaging is an important strategy for studying biological processes. However, routine microscopic techniques like phase contrast or DIC suffer from shadow-cast artifacts making automatic segmentation challenging. The aim of this study was to compare the segmentation efficacy of published steps of segmentation work-flow (image reconstruction, foreground segmentation, cell detection (seed-point extraction) and cell (instance) segmentation) on a dataset of the same cells from multiple contrast microscopic modalities.

RESULTS

We built a collection of routines aimed at image segmentation of viable adherent cells grown on the culture dish acquired by phase contrast, differential interference contrast, Hoffman modulation contrast and quantitative phase imaging, and we performed a comprehensive comparison of available segmentation methods applicable for label-free data. We demonstrated that it is crucial to perform the image reconstruction step, enabling the use of segmentation methods originally not applicable on label-free images. Further we compared foreground segmentation methods (thresholding, feature-extraction, level-set, graph-cut, learning-based), seed-point extraction methods (Laplacian of Gaussians, radial symmetry and distance transform, iterative radial voting, maximally stable extremal region and learning-based) and single cell segmentation methods. We validated suitable set of methods for each microscopy modality and published them online.

CONCLUSIONS

We demonstrate that image reconstruction step allows the use of segmentation methods not originally intended for label-free imaging. In addition to the comprehensive comparison of methods, raw and reconstructed annotated data and Matlab codes are provided.

A Review of Automated Methods for the Detection of Sickle Cell Disease

Detection of sickle cell disease is a crucial job in medical image analysis. It emphasizes elaborate analysis of proper disease diagnosis after accurate detection followed by a classification of irregularities, which plays a vital role in the sickle cell disease diagnosis, treatment planning, and treatment outcome evaluation. Proper segmentation of complex cell clusters makes sickle cell detection more accurate and robust. Cell morphology has a key role in the detection of the sickle cell because the shapes of the normal blood cell and sickle cell differ significantly. This review emphasizes state-of-the-art methods and recent advances in detection, segmentation, and classification of sickle cell disease. We discuss key challenges encountered during the segmentation of overlapping blood cells. Moreover, standard validation measures that have been employed to yield performance analysis of various methods are also discussed. The methodologies and experiments in this review will be useful to further research and work in this area.

Practical Considerations in Particle and Object Tracking and Analysis

The rapid advancement of live-cell imaging technologies has enabled biologists to generate high-dimensional data to follow biological movement at the microscopic level. Yet, the "perceived" ease of use of modern microscopes has led to challenges whereby sub-optimal data are commonly generated that cannot support quantitative tracking and analysis as a result of various ill-advised decisions made during image acquisition. Even optimally acquired images often require further optimization through digital processing before they can be analyzed. In writing this article, we presume our target audience to be biologists with a foundational understanding of digital image acquisition and processing, who are seeking to understand the essential steps for particle/object tracking experiments. It is with this targeted readership in mind that we review the basic principles of image-processing techniques as well as analysis strategies commonly used for tracking experiments. We conclude this technical survey with a discussion of how movement behavior can be mathematically modeled and described. © 2019 by John Wiley & Sons, Inc.

Separating Touching Cells Using Pixel Replicated Elliptical Shape Models

One of the most important and error-prone tasks in biological image analysis is the segmentation of touching or overlapping cells. Particularly for optical microscopy, including transmitted light and confocal fluorescence microscopy, there is often no consistent discriminative information to separate cells that touch or overlap. It is desired to partition touching foreground pixels into cells using the binary threshold image information only, and optionally incorporating gradient information. The most common approaches for segmenting touching and overlapping cells in these scenarios are based on the watershed transform. We describe a new approach called pixel replication for the task of segmenting elliptical objects that touch or overlap. Pixel replication uses the image Euclidean distance transform in combination with Gaussian mixture models to better exploit practically effective optimization for delineating objects with elliptical decision boundaries. Pixel replication improves significantly on commonly used methods based on watershed transforms, or based on fitting Gaussian mixtures directly to the thresholded image data. Pixel replication works equivalently on both 2-D and 3-D image data, and naturally combines information from multi-channel images. The accuracy of the proposed technique is measured using both the segmentation accuracy on simulated ellipse data and the tracking accuracy on validated stem cell tracking results extracted from hundreds of live-cell microscopy image sequences. Pixel replication is shown to be significantly more accurate compared with other approaches. Variance relationships are derived, allowing a more practically effective Gaussian mixture model to extract cell boundaries for data generated from the threshold image using the uniform elliptical distribution and from the distance transform image using the triangular elliptical distribution.

Effective nuclei segmentation with sparse shape prior and dynamic occlusion constraint for glioblastoma pathology images

We propose a segmentation method for nuclei in glioblastoma histopathologic images based on a sparse shape prior guided variational level set framework. By spectral clustering and sparse coding, a set of shape priors is exploited to accommodate complicated shape variations. We automate the object contour initialization by a seed detection algorithm and deform contours by minimizing an energy functional that incorporates a shape term in a sparse shape prior representation, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to deal with mutual occlusions and detect contours of multiple intersected nuclei simultaneously. Our method is applied to several whole-slide histopathologic image datasets for nuclei segmentation. The proposed method is compared with other state-of-the-art methods and demonstrates good accuracy for nuclei detection and segmentation, suggesting its promise to support biomedical image-based investigations.

Introduction to Digital Image Analysis in Whole-slide Imaging: A White Paper from the Digital Pathology Association

The advent of whole-slide imaging in digital pathology has brought about the advancement of computer-aided examination of tissue via digital image analysis. Digitized slides can now be easily annotated and analyzed via a variety of algorithms. This study reviews the fundamentals of tissue image analysis and aims to provide pathologists with basic information regarding the features, applications, and general workflow of these new tools. The review gives an overview of the basic categories of software solutions available, potential analysis strategies, technical considerations, and general algorithm readouts. Advantages and limitations of tissue image analysis are discussed, and emerging concepts, such as artificial intelligence and machine learning, are introduced. Finally, examples of how digital image analysis tools are currently being used in diagnostic laboratories, translational research, and drug development are discussed.

Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images

Automated detection and segmentation of nuclei from high-resolution histopathological images is a challenging problem owing to the size and complexity of digitized histopathologic images. In the context of breast cancer, the modified Bloom-Richardson Grading system is highly correlated with the morphological and topological nuclear features are highly correlated with Modified Bloom-Richardson grading. Therefore, to develop a computer-aided prognosis system, automated detection and segmentation of nuclei are critical prerequisite steps. We present a method for automated detection and segmentation of breast cancer nuclei named a convolutional neural network initialized active contour model with adaptive ellipse fitting (CoNNACaeF). The CoNNACaeF model is able to detect and segment nuclei simultaneously, which consist of three different modules: convolutional neural network (CNN) for accurate nuclei detection, (2) region-based active contour (RAC) model for subsequent nuclear segmentation based on the initial CNN-based detection of nuclear patches, and (3) adaptive ellipse fitting for overlapping solution of clumped nuclear regions. The performance of the CoNNACaeF model is evaluated on three different breast histological data sets, comprising a total of 257 H&E-stained images. The model is shown to have improved detection accuracy of F-measure 80.18%, 85.71%, and 80.36% and average area under precision-recall curves (AveP) 77%, 82%, and 74% on a total of 3 million nuclei from 204 whole slide images from three different datasets. Additionally, CoNNACaeF yielded an F-measure at 74.01% and 85.36%, respectively, for two different breast cancer datasets. The CoNNACaeF model also outperformed the three other state-of-the-art nuclear detection and segmentation approaches, which are blue ratio initialized local region active contour, iterative radial voting initialized local region active contour, and maximally stable extremal region initialized local region active contour models.

Fast unsupervised nuclear segmentation and classification scheme for automatic allred cancer scoring in immunohistochemical breast tissue images

BACKGROUND AND OBJECTIVE

This paper presents an improved scheme able to perform accurate segmentation and classification of cancer nuclei in immunohistochemical (IHC) breast tissue images in order to provide quantitative evaluation of estrogen or progesterone (ER/PR) receptor status that will assist pathologists in cancer diagnostic process.

METHODS

The proposed segmentation method is based on adaptive local thresholding and an enhanced morphological procedure, which are applied to extract all stained nuclei regions and to split overlapping nuclei. In fact, a new segmentation approach is presented here for cell nuclei detection from the IHC image using a modified Laplacian filter and an improved watershed algorithm. Stromal cells are then removed from the segmented image using an adaptive criterion in order to get fast tumor nuclei recognition. Finally, unsupervised classification of cancer nuclei is obtained by the combination of four common color separation techniques for a subsequent Allred cancer scoring.

RESULTS

Experimental results on various IHC tissue images of different cancer affected patients, demonstrate the effectiveness of the proposed scheme when compared to the manual scoring of pathological experts. A statistical analysis is performed on the whole image database between immuno-score of manual and automatic method, and compared with the scores that have reached using other state-of-art segmentation and classification strategies. According to the performance evaluation, we recorded more than 98% for both accuracy of detected nuclei and image cancer scoring over the truths provided by experienced pathologists which shows the best correlation with the expert's score (Pearson's correlation coefficient = 0.993, p-value < 0.005) and the lowest computational total time of 72.3 s/image (±1.9) compared to recent studied methods.

CONCLUSIONS

The proposed scheme can be easily applied for any histopathological diagnostic process that needs stained nuclear quantification and cancer grading. Moreover, the reduced processing time and manual interactions of our procedure can facilitate its implementation in a real-time device to construct a fully online evaluation system of IHC tissue images.

Measuring multiple parameters of CD8+ tumor-infiltrating lymphocytes in human cancers by image analysis

BACKGROUND

Immuno-oncology and cancer immunotherapies are areas of intense research. The numbers and locations of CD8+ tumor-infiltrating lymphocytes (TILs) are important measures of the immune response to cancer with prognostic, pharmacodynamic, and predictive potential. We describe the development, validation, and application of advanced image analysis methods to characterize multiple immunohistochemistry-derived CD8 parameters in clinical and nonclinical tumor tissues.

METHODS

Commercial resection tumors from nine cancer types, and paired screening/on-drug biopsies of non-small-cell lung carcinoma (NSCLC) patients enrolled in a phase 1/2 clinical trial investigating the PD-L1 antibody therapy durvalumab (NCT01693562), were immunostained for CD8. Additional NCT01693562 samples were immunostained with a CD8/PD-L1 dual immunohistochemistry assay. Whole-slide scanning was performed, tumor regions were annotated by a pathologist, and images were analyzed with customized algorithms using Definiens Developer XD software. Validation of image analysis data used cell-by-cell comparison to pathologist scoring across a range of CD8+ TIL densities of all nine cancers, relying primarily on 95% confidence in having at least moderate agreement regarding Lin concordance correlation coefficient (CCC = 0.88-0.99, CCC_lower = 0.65-0.96).

RESULTS

We found substantial variability in CD8+ TILs between individual patients and across the nine types of human cancer. Diffuse large B-cell lymphoma had several-fold more CD8+ TILs than some other cancers. TIL densities were significantly higher in the invasive margin versus tumor center for carcinomas of head and neck, kidney and pancreas, and NSCLC; the reverse was true only for prostate cancer. In paired patient biopsies, there were significantly increased CD8+ TILs 6 weeks after onset of durvalumab therapy (mean of 365 cells/mm2 over baseline; P = 0.009), consistent with immune activation. Image analysis accurately enumerated CD8+ TILs in PD-L1+ regions of lung tumors using the dual assay and also measured elongate CD8+ lymphocytes which constituted a fraction of overall TILs.

CONCLUSIONS

Validated image analysis accurately enumerates CD8+ TILs, permitting comparisons of CD8 parameters among tumor regions, individual patients, and cancer types. It also enables the more complex digital solutions needed to better understand cancer immunity, like analysis of multiplex immunohistochemistry and spatial evaluation of the various components comprising the tumor microenvironment.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT01693562 . Study code: CD-ON-MEDI4736-1108. Interventional study (ongoing but not currently recruiting). Actual study start date: August 29, 2012. Primary completion date: June 23, 2017 (final data collection date for primary outcome measure).

I-AbACUS: a Reliable Software Tool for the Semi-Automatic Analysis of Invasion and Migration Transwell Assays

The quantification of invasion and migration is an important aspect of cancer research, used both in the study of the molecular processes involved in this collection of diseases and the evaluation of the efficacy of new potential treatments. The transwell assay, while being one of the most widely used techniques for the evaluation of these characteristics, shows a high dependence on the operator's ability to correctly identify the cells and a low protocol standardization. Here we present I-AbACUS, a software tool specifically designed to aid the analysis of transwell assays that automatically and specifically recognizes cells in images of stained membranes and provides the user with a suggested cell count. A complete description of this instrument, together with its validation against the standard analysis technique for this assay is presented. Furthermore, we show that I-AbACUS is versatile and able to elaborate images containing cells with different morphologies and that the obtained results are less dependent on the operator and their experience. We anticipate that this instrument, freely available (Gnu Public Licence GPL v2) at www.marilisacortesi.com as a standalone application, could significantly improve the quantification of invasion and migration of cancer cells.

Efficient and robust cell detection: A structured regression approach

Efficient and robust cell detection serves as a critical prerequisite for many subsequent biomedical image analysis methods and computer-aided diagnosis (CAD). It remains a challenging task due to touching cells, inhomogeneous background noise, and large variations in cell sizes and shapes. In addition, the ever-increasing amount of available datasets and the high resolution of whole-slice scanned images pose a further demand for efficient processing algorithms. In this paper, we present a novel structured regression model based on a proposed fully residual convolutional neural network for efficient cell detection. For each testing image, our model learns to produce a dense proximity map that exhibits higher responses at locations near cell centers. Our method only requires a few training images with weak annotations (just one dot indicating the cell centroids). We have extensively evaluated our method using four different datasets, covering different microscopy staining methods (e.g., H & E or Ki-67 staining) or image acquisition techniques (e.g., bright-filed image or phase contrast). Experimental results demonstrate the superiority of our method over existing state of the art methods in terms of both detection accuracy and running time.

Tumor Cell Load and Heterogeneity Estimation From Diffusion-Weighted MRI Calibrated With Histological Data: an Example From Lung Cancer

Diffusion-weighted magnetic resonance imaging (DWI) is a key non-invasive imaging technique for cancer diagnosis and tumor treatment assessment, reflecting Brownian movement of water molecules in tissues. Since densely packed cells restrict molecule mobility, tumor tissues produce usually higher signal (a.k.a. less attenuated signal) on isotropic maps compared with normal tissues. However, no general quantitative relation between DWI data and the cell density has been established. In order to link low-resolution clinical cross-sectional data with high-resolution histological information, we developed an image processing and analysis chain, which was used to study the correlation between the diffusion coefficient (D value) estimated from DWI and tumor cellularity from serial histological slides of a resected non-small cell lung cancer tumor. Color deconvolution followed by cell nuclei segmentation was performed on digitized histological images to determine local and cell-type specific 2d (two-dimensional) densities. From these, the 3d cell density was inferred by a model-based sampling technique, which is necessary for the calculation of local and global 3d tumor cell count. Next, DWI sequence information was overlaid with high-resolution CT data and the resected histology using prominent anatomical hallmarks for co-registration of histology tissue blocks and non-invasive imaging modalities' data. The integration of cell numbers information and DWI data derived from different tumor areas revealed a clear negative correlation between cell density and D value. Importantly, spatial tumor cell density can be calculated based on DWI data. In summary, our results demonstrate that tumor cell count and heterogeneity can be predicted from DWI data, which may open new opportunities for personalized diagnosis and therapy optimization.

A novel overlapped nuclei splitting algorithm for histopathological images

BACKGROUND AND OBJECTIVE

Nuclei segmentation is a common process for quantitative analysis of histopathological images. However, this process generally results in overlapping of nuclei due to the nature of images, the sample preparation and staining, and image acquisition processes as well as insufficiency of 2D histopathological images to represent 3D characteristics of tissues. We present a novel algorithm to split overlapped nuclei.

METHODS

The histopathological images are initially segmented by K-Means segmentation algorithm. Then, nuclei cluster are converted to binary image. The overlapping is detected by applying threshold area value to nuclei in the binary image. The splitting algorithm is applied to the overlapped nuclei. In first stage of splitting, circles are drawn on overlapped nuclei. The radius of the circles is calculated by using circle area formula, and each pixel's coordinates of overlapped nuclei are selected as center coordinates for each circle. The pixels in the circle that contains maximum number of intersected pixels in both the circle and the overlapped nuclei are removed from the overlapped nuclei, and the filled circle labeled as a nucleus.

RESULTS

The algorithm has been tested on histopathological images of healthy and damaged kidney tissues and compared with the results provided by an expert and three related studies. The results demonstrated that the proposed splitting algorithm can segment the overlapping nuclei with accuracy of 84%.

CONCLUSIONS

The study presents a novel algorithm splitting the overlapped nuclei in histopathological images and provides more accurate cell counting in histopathological analysis. Furthermore, the proposed splitting algorithm has the potential to be used in different fields to split any overlapped circular patterns.

A platform for stereological quantitative analysis of the brain-wide distribution of type-specific neurons

Quantifying the distribution of specific neurons throughout the whole brain is crucial for understanding physiological actions, pathological alterations and pharmacological treatments. However, the precise cell number and density of specific neurons in the entire brain remain unknown because of a lack of suitable research tools. Here, we propose a pipeline to automatically acquire and analyse the brain-wide distribution of type-specific neurons in a mouse brain. We employed a Brain-wide Positioning System to collect high-throughput anatomical information with the co-localized cytoarchitecture of the whole brain at subcellular resolution and utilized the NeuroGPS algorithm to locate and count cells in the whole brain. We evaluated the data continuity of the 3D dataset and the accuracy of stereological cell counting in 3D. To apply this pipeline, we acquired and quantified the brain-wide distributions and somatic morphology of somatostatin-expressing neurons in transgenic mouse brains. The results indicated that this whole-brain cell counting pipeline has the potential to become a routine tool for cell type neuroscience studies.

Segmentation and classification of two-channel C. elegans nucleus-labeled fluorescence images

BACKGROUND

Aging is characterized by a gradual breakdown of cellular structures. Nuclear abnormality is a hallmark of progeria in human. Analysis of age-dependent nuclear morphological changes in Caenorhabditis elegans is of great value to aging research, and this calls for an automatic image processing method that is suitable for both normal and abnormal structures.

RESULTS

Our image processing method consists of nuclear segmentation, feature extraction and classification. First, taking up the challenges of defining individual nuclei with fuzzy boundaries or in a clump, we developed an accurate nuclear segmentation method using fused two-channel images with seed-based cluster splitting and k-means algorithm, and achieved a high precision against the manual segmentation results. Next, we extracted three groups of nuclear features, among which five features were selected by minimum Redundancy Maximum Relevance (mRMR) for classifiers. After comparing the classification performances of several popular techniques, we identified that Random Forest, which achieved a mean class accuracy (MCA) of 98.69%, was the best classifier for our data set. Lastly, we demonstrated the method with two quantitative analyses of C. elegans nuclei, which led to the discovery of two possible longevity indicators.

CONCLUSIONS

We produced an automatic image processing method for two-channel C. elegans nucleus-labeled fluorescence images. It frees biologists from segmenting and classifying the nuclei manually.

Quantifying Mesoscale Neuroanatomy Using X-Ray Microtomography

Methods for resolving the three-dimensional (3D) microstructure of the brain typically start by thinly slicing and staining the brain, followed by imaging numerous individual sections with visible light photons or electrons. In contrast, X-rays can be used to image thick samples, providing a rapid approach for producing large 3D brain maps without sectioning. Here we demonstrate the use of synchrotron X-ray microtomography (µCT) for producing mesoscale (∼1 µm 3 resolution) brain maps from millimeter-scale volumes of mouse brain. We introduce a pipeline for µCT-based brain mapping that develops and integrates methods for sample preparation, imaging, and automated segmentation of cells, blood vessels, and myelinated axons, in addition to statistical analyses of these brain structures. Our results demonstrate that X-ray tomography achieves rapid quantification of large brain volumes, complementing other brain mapping and connectomics efforts.

AUTOMATED LEVEL SET SEGMENTATION OF HISTOPATHOLOGIC CELLS WITH SPARSE SHAPE PRIOR SUPPORT AND DYNAMIC OCCLUSION CONSTRAINT

In this paper, we propose a novel segmentation method for cells in histopathologic images based on a sparse shape prior guided variational level set framework. We automate the cell contour initialization by detecting seeds and deform contours by minimizing a new energy functional that incorporates a shape term involving sparse shape priors, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to accommodate mutual occlusions and detect contours of multiple intersected cells. We apply our algorithm to a set of whole-slide histopathologic images of brain tumor sections. The proposed method is compared with other popular methods, and demonstrates good accuracy for cell segmentation by quantitative measures, suggesting its promise to support biomedical image-based investigations.

AIIMDs: An Integrated Framework of Automatic Idiopathic Inflammatory Myopathy Diagnosis for Muscle

Idiopathic inflammatory myopathy (IIM) is a common skeletal muscle disease that relates to weakness and inflammation of muscle. Early diagnosis and prognosis of different types of IIMs will guide the effective treatment. Interpretation of digitized images of the cross-section muscle biopsy, which is currently done manually, provides the most reliable diagnostic information. With the increasing volume of images, the management and manual interpretation of the digitized muscle images suffer from low efficiency and high interobserver variabilities. In order to address these problems, we propose the first complete framework of automatic IIM diagnosis system for the management and interpretation of digitized skeletal muscle histopathology images. The proposed framework consists of several key components: (1) Automatic cell segmentation, perimysium annotation, and nuclei detection; (2) histogram-based feature extraction and quantification; (3) content-based image retrieval to search and retrieve similar cases in the database for comparative study; and (4) majority voting-based classification to provide decision support for computer-aided clinical diagnosis. Experiments show that the proposed diagnosis system provides efficient and robust interpretation of the digitized muscle image and computer-aided diagnosis of IIM.

Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

With the advance of digital pathology, image analysis has begun to show its advantages in information analysis of hematoxylin and eosin histopathology images. Generally, histological features in hematoxylin and eosin images are measured to evaluate tumor grade and prognosis for breast cancer. This review summarized recent works in image analysis of hematoxylin and eosin histopathology images for breast cancer prognosis. First, prognostic factors for breast cancer based on hematoxylin and eosin histopathology images were summarized. Then, usual procedures of image analysis for breast cancer prognosis were systematically reviewed, including image acquisition, image preprocessing, image detection and segmentation, and feature extraction. Finally, the prognostic value of image features and image feature–based prognostic models was evaluated. Moreover, we discussed the issues of current analysis, and some directions for future research.

INTEGRATIVE ANALYSIS FOR LUNG ADENOCARCINOMA PREDICTS MORPHOLOGICAL FEATURES ASSOCIATED WITH GENETIC VARIATIONS

Lung cancer is one of the most deadly cancers and lung adenocarcinoma (LUAD) is the most common histological type of lung cancer. However, LUAD is highly heterogeneous due to genetic difference as well as phenotypic differences such as cellular and tissue morphology. In this paper, we systematically examine the relationships between histological features and gene transcription. Specifically, we calculated 283 morphological features from histology images for 201 LUAD patients from TCGA project and identified the morphological feature with strong correlation with patient outcome. We then modeled the morphology feature using multiple co-expressed gene clusters using Lasso-regression. Many of the gene clusters are highly associated with genetic variations, specifically DNA copy number variations, implying that genetic variations play important roles in the development cancer morphology. As far as we know, our finding is the first to directly link the genetic variations and functional genomics to LUAD histology. These observations will lead to new insight on lung cancer development and potential new integrative biomarkers for prediction patient prognosis and response to treatments.