A boosted Bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies

Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology

The emergent field of digital pathology employing automated image analysis techniques is to revolutionize traditional pathology at the center of clinical diagnostics. Histological images provide important tumor features unavailable in molecular profiling or omics data- the spatial context of tumor and stromal cells at single-cell resolution. Methods to map the spatial and morphological patterns of cancer and normal cells can contribute to a more comprehensive understanding of the highly heterogeneous tumor microenvironment. This review focuses on methods that help expand our knowledge of intra-tumoral spatial heterogeneity of the tumor microenvironment and their potential synergies with molecular profiling technologies.

Cell type classifiers for breast cancer microscopic images based on fractal dimension texture analysis of image color layers

Having a classifier of cell types in a breast cancer microscopic image (BCMI), obtained with immunohistochemical staining, is required as part of a computer-aided system that counts the cancer cells in such BCMI. Such quantitation by cell counting is very useful in supporting decisions and planning of the medical treatment of breast cancer. This study proposes and evaluates features based on texture analysis by fractal dimension (FD), for the classification of histological structures in a BCMI into either cancer cells or non-cancer cells. The cancer cells include positive cells (PC) and negative cells (NC), while the normal cells comprise stromal cells (SC) and lymphocyte cells (LC). The FD feature values were calculated with the box-counting method from binarized images, obtained by automatic thresholding with Otsu's method of the grayscale images for various color channels. A total of 12 color channels from four color spaces (RGB, CIE-L*a*b*, HSV, and YCbCr) were investigated, and the FD feature values from them were used with decision tree classifiers. The BCMI data consisted of 1,400, 1,200, and 800 images with pixel resolutions 128 × 128, 192 × 192, and 256 × 256, respectively. The best cross-validated classification accuracy was 93.87%, for distinguishing between cancer and non-cancer cells, obtained using the Cr color channel with window size 256. The results indicate that the proposed algorithm, based on fractal dimension features extracted from a color channel, performs well in the automatic classification of the histology in a BCMI. This might support accurate automatic cell counting in a computer-assisted system for breast cancer diagnosis.

Towards large-scale histopathological image analysis: hashing-based image retrieval

Automatic analysis of histopathological images has been widely utilized leveraging computational image-processing methods and modern machine learning techniques. Both computer-aided diagnosis (CAD) and content-based image-retrieval (CBIR) systems have been successfully developed for diagnosis, disease detection, and decision support in this area. Recently, with the ever-increasing amount of annotated medical data, large-scale and data-driven methods have emerged to offer a promise of bridging the semantic gap between images and diagnostic information. In this paper, we focus on developing scalable image-retrieval techniques to cope intelligently with massive histopathological images. Specifically, we present a supervised kernel hashing technique which leverages a small amount of supervised information in learning to compress a 10 000-dimensional image feature vector into only tens of binary bits with the informative signatures preserved. These binary codes are then indexed into a hash table that enables real-time retrieval of images in a large database. Critically, the supervised information is employed to bridge the semantic gap between low-level image features and high-level diagnostic information. We build a scalable image-retrieval framework based on the supervised hashing technique and validate its performance on several thousand histopathological images acquired from breast microscopic tissues. Extensive evaluations are carried out in terms of image classification (i.e., benign versus actionable categorization) and retrieval tests. Our framework achieves about 88.1% classification accuracy as well as promising time efficiency. For example, the framework can execute around 800 queries in only 0.01 s, comparing favorably with other commonly used dimensionality reduction and feature selection methods.

Optical quality assessment of whole slide imaging systems for digital pathology

Whole Slide Imaging (WSI) systems are high-throughput automated microscopes for digital pathology applications. We present a method for testing and monitoring the optical quality of WSI-systems using a measurement of the through-focus Optical Transfer Function (OTF) obtained from the edge response of a custom made resolution target, composed of sagittal and tangential edges. This enables quantitative analysis of a number of primary aberrations. The curvature of the best focus as a function of spatial frequency is indicative for spherical aberration, the argument of the OTF quantifies for coma, and the best focus as a function of field position for sagittal and tangential edges allows assessment of astigmatism and field curvature. The statistical error in the determined aberrations is typically below 20 mλ. We use the method to compare different tube lens designs and to study the effect of objective lens aging. The results are in good agreement with direct measurement of aberrations based on Shack-Hartmann wavefront sensing with a typical error ranging from 10 mλ to 40 mλ.

Exploring automatic prostate histopathology image Gleason grading via local structure modeling

Gleason-grading of prostate cancer pathology specimens reveal the malignancy of the cancer tissues, thus provides critical guidance for prostate cancer diagnoses and treatment. Computer-aided automatic grading methods have been providing efficient and result-consistent alternative to traditional manually slide reading approach, through statistical and structural feature analysis of the digitized pathology slides. In this paper, we propose a novel automatic Gleason grading algorithm through local structure model learning and classification. We use attributed graph to represent the tissue glandular structures in histopathology images; representative sub-graphs features were learned as bags-of-words features from labeled samples of each grades. Then structural similarity between sub-graphs in the unlabeled images and the representative sub-graphs were obtained using the learned codebook. Gleason grade was given based on an overall similarity score. We validated the proposed algorithm on 300 prostate histopathology images from the TCGA dataset, and the algorithm achieved average grading accuracy of 91.25%, 76.36% and 64.75% on images with Gleason grade 3, 4 and 5 respectively.

Two-tier tissue decomposition for histopathological image representation and classification

In digital pathology, devising effective image representations is crucial to design robust automated diagnosis systems. To this end, many studies have proposed to develop object-based representations, instead of directly using image pixels, since a histopathological image may contain a considerable amount of noise typically at the pixel-level. These previous studies mostly employ color information to define their objects, which approximately represent histological tissue components in an image, and then use the spatial distribution of these objects for image representation and classification. Thus, object definition has a direct effect on the way of representing the image, which in turn affects classification accuracies. In this paper, our aim is to design a classification system for histopathological images. Towards this end, we present a new model for effective representation of these images that will be used by the classification system. The contributions of this model are twofold. First, it introduces a new two-tier tissue decomposition method for defining a set of multityped objects in an image. Different than the previous studies, these objects are defined combining texture, shape, and size information and they may correspond to individual histological tissue components as well as local tissue subregions of different characteristics. As its second contribution, it defines a new metric, which we call dominant blob scale, to characterize the shape and size of an object with a single scalar value. Our experiments on colon tissue images reveal that this new object definition and characterization provides distinguishing representation of normal and cancerous histopathological images, which is effective to obtain more accurate classification results compared to its counterparts.

Computer-aided diagnostics in digital pathology: automated evaluation of early-phase pancreatic cancer in mice

PURPOSE

Digital pathology diagnostics are often based on subjective qualitative measures. A murine model of early-phase pancreatic ductal adenocarcinoma provides a controlled environment with a priori knowledge of the genetic mutation and stage of the disease. Use of this model enables the application of supervised learning methods to digital pathology. A computerized diagnostics system for histological detection of pancreatic adenocarcinoma was developed and tested. METHODS : Pathological H&E-stained specimens with early pancreatic lesions were identified and evaluated with a system that models cancer detection using a top-down object learning paradigm, mimicking the way a pathologist learns. First, the dominant primitives were identified and segmented in the images, i.e., the ducts, nuclei and tumor stroma. A boost-based machine learning technique was used for duct segmentation, classification and outlier pruning. Second, a set of morphological features traditionally used for cancer diagnosis which provides quantitative image features was employed to quantify subtle findings such as duct deformation and nuclei malformations. Finally, a visually interpretable predictive model was trained to distinguish between normal tissue and premalignant cancer lesions, given ground truth samples. RESULTS : A predictive success rate of 92% was achieved using tenfold cross-validation and 93% on an independent test set. Comparison was made with state-of-the-art classification algorithms that are not interpretable as visible features yielded the contribution of individual primitive features to the prediction outcome.

CONCLUSIONS

Quantitative image analysis and classification were successful in preclinical histology diagnosis for early-stage pancreatic adenocarcinoma. The usage of annotated contours coupled with interpretable supervised learning methods and outlier pruning can be adapted to other medical imaging tasks. The usage of interpretable supervised learning techniques may improve the success of CAD in histopathological diagnosis.

Detection of leukocoria using a soft fusion of expert classifiers under non-clinical settings

BACKGROUND

Leukocoria is defined as a white reflection and its manifestation is symptomatic of several ocular pathologies, including retinoblastoma (Rb). Early detection of recurrent leukocoria is critical for improved patient outcomes and can be accomplished via the examination of recreational photography. To date, there exists a paucity of methods to automate leukocoria detection within such a dataset.

METHODS

This research explores a novel classification scheme that uses fuzzy logic theory to combine a number of classifiers that are experts in performing multichannel detection of leukocoria from recreational photography. The proposed scheme extracts features aided by the discrete cosine transform and the Karhunen-Loeve transformation.

RESULTS

The soft fusion of classifiers is significantly better than other methods of combining classifiers with p = 1.12 × 10-5. The proposed methodology performs at a 92% accuracy rate, with an 89% true positive rate, and an 11% false positive rate. Furthermore, the results produced by our methodology exhibit the lowest average variance.

CONCLUSIONS

The proposed methodology overcomes non-ideal conditions of image acquisition, presenting a competent approach for the detection of leukocoria. Results suggest that recreational photography can be used in combination with the fusion of individual experts in multichannel classification and preprocessing tools such as the discrete cosine transform and the Karhunen-Loeve transformation.

Computer-Aided Prostate Cancer Diagnosis From Digitized Histopathology: A Review on Texture-Based Systems

Prostate cancer (PCa) is currently diagnosed by microscopic evaluation of biopsy samples. Since tissue assessment heavily relies on the pathologists level of expertise and interpretation criteria, it is still a subjective process with high intra- and interobserver variabilities. Computer-aided diagnosis (CAD) may have a major impact on detection and grading of PCa by reducing the pathologists reading time, and increasing the accuracy and reproducibility of diagnosis outcomes. However, the complexity of the prostatic tissue and the large volumes of data generated by biopsy procedures make the development of CAD systems for PCa a challenging task. The problem of automated diagnosis of prostatic carcinoma from histopathology has received a lot of attention. As a result, a number of CAD systems, have been proposed for quantitative image analysis and classification. This review aims at providing a detailed description of selected literature in the field of CAD of PCa, emphasizing the role of texture analysis methods in tissue description. It includes a review of image analysis tools for image preprocessing, feature extraction, classification, and validation techniques used in PCa detection and grading, as well as future directions in pursuit of better texture-based CAD systems.

Histostitcher™: An informatics software platform for reconstructing whole-mount prostate histology using the extensible imaging platform framework

CONTEXT

Co-registration of ex-vivo histologic images with pre-operative imaging (e.g., magnetic resonance imaging [MRI]) can be used to align and map disease extent, and to identify quantitative imaging signatures. However, ex-vivo histology images are frequently sectioned into quarters prior to imaging.

AIMS

This work presents Histostitcher™, a software system designed to create a pseudo whole mount histology section (WMHS) from a stitching of four individual histology quadrant images.

MATERIALS AND METHODS

Histostitcher™ uses user-identified fiducials on the boundary of two quadrants to stitch such quadrants. An original prototype of Histostitcher™ was designed using the Matlab programming languages. However, clinical use was limited due to slow performance, computer memory constraints and an inefficient workflow. The latest version was created using the extensible imaging platform (XIP™) architecture in the C++ programming language. A fast, graphics processor unit renderer was designed to intelligently cache the visible parts of the histology quadrants and the workflow was significantly improved to allow modifying existing fiducials, fast transformations of the quadrants and saving/loading sessions.

RESULTS

The new stitching platform yielded significantly more efficient workflow and reconstruction than the previous prototype. It was tested on a traditional desktop computer, a Windows 8 Surface Pro table device and a 27 inch multi-touch display, with little performance difference between the different devices.

CONCLUSIONS

Histostitcher™ is a fast, efficient framework for reconstructing pseudo WMHS from individually imaged quadrants. The highly modular XIP™ framework was used to develop an intuitive interface and future work will entail mapping the disease extent from the pseudo WMHS onto pre-operative MRI.

Dual optimization based prostate zonal segmentation in 3D MR images

Efficient and accurate segmentation of the prostate and two of its clinically meaningful sub-regions: the central gland (CG) and peripheral zone (PZ), from 3D MR images, is of great interest in image-guided prostate interventions and diagnosis of prostate cancer. In this work, a novel multi-region segmentation approach is proposed to simultaneously segment the prostate and its two major sub-regions from only a single 3D T2-weighted (T2w) MR image, which makes use of the prior spatial region consistency and incorporates a customized prostate appearance model into the segmentation task. The formulated challenging combinatorial optimization problem is solved by means of convex relaxation, for which a novel spatially continuous max-flow model is introduced as the dual optimization formulation to the studied convex relaxed optimization problem with region consistency constraints. The proposed continuous max-flow model derives an efficient duality-based algorithm that enjoys numerical advantages and can be easily implemented on GPUs. The proposed approach was validated using 18 3D prostate T2w MR images with a body-coil and 25 images with an endo-rectal coil. Experimental results demonstrate that the proposed method is capable of efficiently and accurately extracting both the prostate zones: CG and PZ, and the whole prostate gland from the input 3D prostate MR images, with a mean Dice similarity coefficient (DSC) of 89.3±3.2% for the whole gland (WG), 82.2±3.0% for the CG, and 69.1±6.9% for the PZ in 3D body-coil MR images; 89.2±3.3% for the WG, 83.0±2.4% for the CG, and 70.0±6.5% for the PZ in 3D endo-rectal coil MR images. In addition, the experiments of intra- and inter-observer variability introduced by user initialization indicate a good reproducibility of the proposed approach in terms of volume difference (VD) and coefficient-of-variation (CV) of DSC.

Weakly supervised histopathology cancer image segmentation and classification

Labeling a histopathology image as having cancerous regions or not is a critical task in cancer diagnosis; it is also clinically important to segment the cancer tissues and cluster them into various classes. Existing supervised approaches for image classification and segmentation require detailed manual annotations for the cancer pixels, which are time-consuming to obtain. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL) (along the line of weakly supervised learning) for histopathology image segmentation. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), medical image segmentation (cancer vs. non-cancer tissue), and patch-level clustering (different classes). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to performing the above three tasks in an integrated framework. In addition, we introduce contextual constraints as a prior for MCIL, which further reduces the ambiguity in MIL. Experimental results on histopathology colon cancer images and cytology images demonstrate the great advantage of MCIL over the competing methods.

Explicit shape descriptors: novel morphologic features for histopathology classification

Object morphology, defined as shape and size characteristics, observed on medical imagery is often an important marker for disease presence and/or aggressiveness. In the context of prostate cancer histopathology, gland morphology is an integral component of the Gleason grading system which enables discrimination between low and high grade disease. However, clinicians are often unable to distinguish between subtle differences in object morphology, as evidenced by high inter-observer variability in Gleason grading. Boundary-based morphologic descriptors, such as the variance in the distance from points on the boundary of an object to its center, may not have the requisite discriminability to separate objects with subtle shape differences. In this paper, we present a set of novel explicit shape descriptors (ESDs) which are capable of distinguishing subtle shape differences between prostate glands of intermediate Gleason grades (grades 3 and 4) on prostate cancer histopathology. Calculation of ESDs involves: (1) representing object morphology using an explicit shape model (e.g. medial axis); (2) aligning the shape models via a non-rigid registration scheme with a diffeomorphic constraint and quantifying shape model dissimilarity; and (3) applying a non-linear dimensionality reduction scheme (e.g. Graph Embedding) to learn a low dimensional projection encoding the shape differences between objects. ESDs are hence the principal eigenvectors in the reduced embedding space. In this work we demonstrate that ESDs in conjunction with a Support Vector Machine classifier are able to correctly distinguish between 888 prostate glands corresponding to different Gleason grades (benign, grade 3, or grade 4) of prostate cancer from 58 needle biopsy specimens with a maximum accuracy of 0.89 and corresponding area under the receiver operating characteristic curve of 0.78.

Nuclear morphometry, epigenetic changes, and clinical relevance in prostate cancer

Nuclear structure alterations in cancer involve global genetic (mutations, amplifications, copy number variations, translocations, etc.) and epigenetic (DNA methylation and histone modifications) events that dramatically and dynamically spatially change chromatin, nuclear body, and chromosome organization. In prostate cancer (CaP) there appears to be early (<50 years) versus late (>60 years) onset clinically significant cancers, and we have yet to clearly understand the hereditary and somatic-based molecular pathways involved. We do know that once cancer is initiated, dedifferentiation of the prostate gland occurs with significant changes in nuclear structure driven by numerous genetic and epigenetic processes. This review focuses upon the nuclear architecture and epigenetic dynamics with potential translational clinically relevant applications to CaP. Further, the review correlates changes in the cancer-driven epigenetic process at the molecular level and correlates these alterations to nuclear morphological quantitative measurements. Finally, we address how we can best utilize this knowledge to improve the efficacy of personalized treatment of cancer.

High-speed spectral nanocytology for early cancer screening

High-throughput partial wave spectroscopy (HTPWS) is introduced as a high-speed spectral nanocytology technique that utilizes the field effect of carcinogenesis to perform minimally invasive cancer screening on at-risk populations. HTPWS uses fully automated hardware and an acousto-optic tunable filter to scan slides at low magnification, to select cells, and to rapidly acquire spectra at each spatial pixel in a cell between 450 and 700 nm, completing measurements of 30 cells in 40 min. Statistical quantitative analysis on the size and density of intracellular nanostructures extracted from the spectra at each pixel in a cell yields the diagnostic biomarker, disorder strength (Ld). Linear correlation between Ld and the length scale of nanostructures was measured in phantoms with R2=0.93. Diagnostic sensitivity was demonstrated by measuring significantly higher Ld from a human colon cancer cell line (HT29 control vector) than a less aggressive variant (epidermal growth factor receptor knockdown). Clinical diagnostic performance for lung cancer screening was tested on 23 patients, yielding a significant difference in Ld between smokers and cancer patients, p=0.02 and effect size=1.00. The high-throughput performance, nanoscale sensitivity, and diagnostic sensitivity make HTPWS a potentially clinically relevant modality for risk stratification of the large populations at risk of developing cancer.

Prostate histopathology: learning tissue component histograms for cancer detection and classification

Radical prostatectomy is performed on approximately 40% of men with organ-confined prostate cancer. Pathologic information obtained from the prostatectomy specimen provides important prognostic information and guides recommendations for adjuvant treatment. The current pathology protocol in most centers involves primarily qualitative assessment. In this paper, we describe and evaluate our system for automatic prostate cancer detection and grading on hematoxylin & eosin-stained tissue images. Our approach is intended to address the dual challenges of large data size and the need for high-level tissue information about the locations and grades of tumors. Our system uses two stages of AdaBoost-based classification. The first provides high-level tissue component labeling of a superpixel image partitioning. The second uses the tissue component labeling to provide a classification of cancer versus noncancer, and low-grade versus high-grade cancer. We evaluated our system using 991 sub-images extracted from digital pathology images of 50 whole-mount tissue sections from 15 prostatectomy patients. We measured accuracies of 90% and 85% for the cancer versus noncancer and high-grade versus low-grade classification tasks, respectively. This system represents a first step toward automated cancer quantification on prostate digital histopathology imaging, which could pave the way for more accurately informed postprostatectomy patient care.

Pathology imaging informatics for quantitative analysis of whole-slide images

Objectives

With the objective of bringing clinical decision support systems to reality, this article reviews histopathological whole-slide imaging informatics methods, associated challenges, and future research opportunities.

Target audience

This review targets pathologists and informaticians who have a limited understanding of the key aspects of whole-slide image (WSI) analysis and/or a limited knowledge of state-of-the-art technologies and analysis methods.

Scope

First, we discuss the importance of imaging informatics in pathology and highlight the challenges posed by histopathological WSI. Next, we provide a thorough review of current methods for: quality control of histopathological images; feature extraction that captures image properties at the pixel, object, and semantic levels; predictive modeling that utilizes image features for diagnostic or prognostic applications; and data and information visualization that explores WSI for de novo discovery. In addition, we highlight future research directions and discuss the impact of large public repositories of histopathological data, such as the Cancer Genome Atlas, on the field of pathology informatics. Following the review, we present a case study to illustrate a clinical decision support system that begins with quality control and ends with predictive modeling for several cancer endpoints. Currently, state-of-the-art software tools only provide limited image processing capabilities instead of complete data analysis for clinical decision-making. We aim to inspire researchers to conduct more research in pathology imaging informatics so that clinical decision support can become a reality.

Unsupervised content classification based nonrigid registration of differently stained histology images

Registration of histopathology images of consecutive tissue sections stained with different histochemical or immunohistochemical stains is an important step in a number of application areas, such as the investigation of the pathology of a disease, validation of MRI sequences against tissue images, multiscale physical modeling, etc. In each case, information from each stain needs to be spatially aligned and combined to ascertain physical or functional properties of the tissue. However, in addition to the gigabyte-size images and nonrigid distortions present in the tissue, a major challenge for registering differently stained histology image pairs is the dissimilar structural appearance due to different stains highlighting different substances in tissues. In this paper, we address this challenge by developing an unsupervised content classification method that generates multichannel probability images from a roughly aligned image pair. Each channel corresponds to one automatically identified content class. The probability images enhance the structural similarity between image pairs. By integrating the classification method into a multiresolution-block-matching-based nonrigid registration scheme (N. Roberts, D. Magee, Y. Song, K. Brabazon, M. Shires, D. Crellin, N. Orsi, P. Quirke, and D. Treanor, "Toward routine use of 3D histopathology as a research tool," Amer. J. Pathology, vol. 180, no. 5, 2012.), we improve the performance of registering multistained histology images. Evaluation was conducted on 77 histological image pairs taken from three liver specimens and one intervertebral disc specimen. In total, six types of histochemical stains were tested. We evaluated our method against the same registration method implemented without applying the classification algorithm (intensity-based registration) and the state-of-the-art mutual information based registration. Superior results are obtained with the proposed method.

Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides

Modified Bloom-Richardson (mBR) grading is known to have prognostic value in breast cancer (BCa), yet its use in clinical practice has been limited by intra- and interobserver variability. The development of a computerized system to distinguish mBR grade from entire estrogen receptor-positive (ER+) BCa histopathology slides will help clinicians identify grading discrepancies and improve overall confidence in the diagnostic result. In this paper, we isolate salient image features characterizing tumor morphology and texture to differentiate entire hematoxylin and eosin (H and E) stained histopathology slides based on mBR grade. The features are used in conjunction with a novel multi-field-of-view (multi-FOV) classifier--a whole-slide classifier that extracts features from a multitude of FOVs of varying sizes--to identify important image features at different FOV sizes. Image features utilized include those related to the spatial arrangement of cancer nuclei (i.e., nuclear architecture) and the textural patterns within nuclei (i.e., nuclear texture). Using slides from 126 ER+ patients (46 low, 60 intermediate, and 20 high mBR grade), our grading system was able to distinguish low versus high, low versus intermediate, and intermediate versus high grade patients with area under curve values of 0.93, 0.72, and 0.74, respectively. Our results suggest that the multi-FOV classifier is able to 1) successfully discriminate low, medium, and high mBR grade and 2) identify specific image features at different FOV sizes that are important for distinguishing mBR grade in H and E stained ER+ BCa histology slides.

Automatic differential diagnosis of pancreatic serous and mucinous cystadenomas based on morphological features

Generally, pathological diagnosis using an electron microscope is time-consuming and likely to result in a subjective judgment, because pathologists perform manual screening of tissue slides at high magnifications. Recently, the advent of digital pathology technology has provided the basis for convenient screening and quantitative analysis by digitizing tissue slides through a computer system. However, a screening process with high magnification still takes quite a long time. To solve these problems, recently the use of computer-aided design techniques for performing pathologic diagnosis has been increasing in digital pathology. For pathological diagnosis, we need different diagnostic methods for different regions with different characteristics. Therefore, in order to effectively diagnose different lesions and types of diseases, a quantitative method for extracting specific features is required in computerized pathologic diagnosis. This study is about an automated differential diagnosis system to differentiate between benign serous cystadenoma and possibly-malignant mucinous cystadenoma. In order to diagnose cystic tumors, the first step is identifying a cystic region and inspecting its epithelial cells. First, we identify the lumen boundary of a cyst using the Direction Cumulative Map considering 8-ways. Then, the Epithelial Nuclei Identification algorithm is used to discern epithelial nuclei. After that, three morphological features for the differential diagnosis of mucinous and serous cystadenomas are extracted. To demonstrate the superiority of the proposed features, the experiments compared performance of the classifiers learned by using the proposed morphological features and the classical morphological features based on nuclei. The classifiers in the simulations are as follows; Bayesian Classifier, k-Nearest Neighbors, Support Vector Machine, and Artificial Neural Network. The results show that all classifiers using the proposed features have the best classification performance.

Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer

BACKGROUND

Automated classification of histopathology involves identification of multiple classes, including benign, cancerous, and confounder categories. The confounder tissue classes can often mimic and share attributes with both the diseased and normal tissue classes, and can be particularly difficult to identify, both manually and by automated classifiers. In the case of prostate cancer, they may be several confounding tissue types present in a biopsy sample, posing as major sources of diagnostic error for pathologists. Two common multi-class approaches are one-shot classification (OSC), where all classes are identified simultaneously, and one-versus-all (OVA), where a "target" class is distinguished from all "non-target" classes. OSC is typically unable to handle discrimination of classes of varying similarity (e.g. with images of prostate atrophy and high grade cancer), while OVA forces several heterogeneous classes into a single "non-target" class. In this work, we present a cascaded (CAS) approach to classifying prostate biopsy tissue samples, where images from different classes are grouped to maximize intra-group homogeneity while maximizing inter-group heterogeneity.

RESULTS

We apply the CAS approach to categorize 2000 tissue samples taken from 214 patient studies into seven classes: epithelium, stroma, atrophy, prostatic intraepithelial neoplasia (PIN), and prostate cancer Gleason grades 3, 4, and 5. A series of increasingly granular binary classifiers are used to split the different tissue classes until the images have been categorized into a single unique class. Our automatically-extracted image feature set includes architectural features based on location of the nuclei within the tissue sample as well as texture features extracted on a per-pixel level. The CAS strategy yields a positive predictive value (PPV) of 0.86 in classifying the 2000 tissue images into one of 7 classes, compared with the OVA (0.77 PPV) and OSC approaches (0.76 PPV).

CONCLUSIONS

Use of the CAS strategy increases the PPV for a multi-category classification system over two common alternative strategies. In classification problems such as histopathology, where multiple class groups exist with varying degrees of heterogeneity, the CAS system can intelligently assign class labels to objects by performing multiple binary classifications according to domain knowledge.

Automatic detection of melanoma progression by histological analysis of secondary sites

We present results from machine classification of melanoma biopsies sectioned and stained with hematoxylin/eosin (H&E) on tissue microarrays (TMA). The four stages of melanoma progression were represented by seven tissue types, including benign nevus, primary tumors with radial and vertical growth patterns (stage I) and four secondary metastatic tumors: subcutaneous (stage II), lymph node (stage III), gastrointestinal and soft tissue (stage IV). Our experiment setup comprised 14,208 image samples based on 164 TMA cores. In our experiments, we constructed an HE color space by digitally deconvolving the RGB images into separate H (hematoxylin) and E (eosin) channels. We also compared three different classifiers: Weighted Neighbor Distance (WND), Radial Basis Functions (RBF), and k-Nearest Neighbors (kNN). We found that the HE color space consistently outperformed other color spaces with all three classifiers, while the different classifiers did not have as large of an effect on accuracy. This showed that a more physiologically relevant representation of color can have a larger effect on correct image interpretation than downstream processing steps. We were able to correctly classify individual fields of view with an average of 96% accuracy when randomly splitting the dataset into training and test fields. We also obtained a classification accuracy of 100% when testing entire cores that were not previously used in training (four random trials with one test core for each of 7 classes, 28 tests total). Because each core corresponded to a different patient, this test more closely mimics a clinically relevant setting where new patients are evaluated based on training with previous cases. The analysis method used in this study contains no parameters or adjustments that are specific to melanoma morphology, suggesting it can be used for analyzing other tissues and phenotypes, as well as potentially different image modalities and contrast techniques.

Prostate cancer detection: Fusion of cytological and textural features

A computer-assisted system for histological prostate cancer diagnosis can assist pathologists in two stages: (i) to locate cancer regions in a large digitized tissue biopsy, and (ii) to assign Gleason grades to the regions detected in stage 1. Most previous studies on this topic have primarily addressed the second stage by classifying the preselected tissue regions. In this paper, we address the first stage by presenting a cancer detection approach for the whole slide tissue image. We propose a novel method to extract a cytological feature, namely the presence of cancer nuclei (nuclei with prominent nucleoli) in the tissue, and apply this feature to detect the cancer regions. Additionally, conventional image texture features which have been widely used in the literature are also considered. The performance comparison among the proposed cytological textural feature combination method, the texture-based method and the cytological feature-based method demonstrates the robustness of the extracted cytological feature. At a false positive rate of 6%, the proposed method is able to achieve a sensitivity of 78% on a dataset including six training images (each of which has approximately 4,000×7,000 pixels) and 1 1 whole-slide test images (each of which has approximately 5,000×23,000 pixels). All images are at 20X magnification.

A fully automated approach to prostate biopsy segmentation based on level-set and mean filtering

With modern automated microscopes and digital cameras, pathologists no longer have to examine samples looking through microscope binoculars. Instead, the slide is digitized to an image, which can then be examined on a screen. This creates the possibility for computers to analyze the image. In this work, a fully automated approach to region of interest (ROI) segmentation in prostate biopsy images is proposed. This will allow the pathologists to focus on the most important areas of the image. The method proposed is based on level-set and mean filtering techniques for lumen centered expansion and cell density localization respectively. The novelty of the technique lies in the ability to detect complete ROIs, where a ROI is composed by the conjunction of three different structures, that is, lumen, cytoplasm, and cells, as well as regions with a high density of cells. The method is capable of dealing with full biopsies digitized at different magnifications. In this paper, results are shown with a set of 100 H and E slides, digitized at 5×, and ranging from 12 MB to 500 MB. The tests carried out show an average specificity above 99% across the board and average sensitivities of 95% and 80%, respectively, for the lumen centered expansion and cell density localization. The algorithms were also tested with images at 10× magnification (up to 1228 MB) obtaining similar results.

Structure and context in prostatic gland segmentation and classification

A novel gland segmentation and classification scheme applied to an H&E histology image of the prostate tissue is proposed. For gland segmentation, we associate appropriate nuclei objects with each lumen object to create a gland segment. We further extract 22 features to describe the structural information and contextual information for each segment. These features are used to classify a gland segment into one of the three classes: artifact, normal gland and cancer gland. On a dataset of 48 images at 5x magnification (which includes 525 artifacts, 931 normal glands and 1,375 cancer glands), we achieved the following classification accuracies: 93% for artifacts v. true glands; 79% for normal v. cancer glands, and 77% for discriminating all three classes. The proposed method outperforms state of the art methods in terms of segmentation and classification accuracies and computational efficiency.

Image microarrays (IMA): Digital pathology's missing tool

Introduction:

The increasing availability of whole slide imaging (WSI) data sets (digital slides) from glass slides offers new opportunities for the development of computer-aided diagnostic (CAD) algorithms. With the all-digital pathology workflow that these data sets will enable in the near future, literally millions of digital slides will be generated and stored. Consequently, the field in general and pathologists, specifically, will need tools to help extract actionable information from this new and vast collective repository.

Methods:

To address this limitation, we designed and implemented a tool (dCORE) to enable the systematic capture of image tiles with constrained size and resolution that contain desired histopathologic features.

Results:

In this communication, we describe a user-friendly tool that will enable pathologists to mine digital slides archives to create image microarrays (IMAs). IMAs are to digital slides as tissue microarrays (TMAs) are to cell blocks. Thus, a single digital slide could be transformed into an array of hundreds to thousands of high quality digital images, with each containing key diagnostic morphologies and appropriate controls. Current manual digital image cut-and-paste methods that allow for the creation of a grid of images (such as an IMA) of matching resolutions are tedious.

Conclusion:

The ability to create IMAs representing hundreds to thousands of vetted morphologic features has numerous applications in education, proficiency testing, consensus case review, and research. Lastly, in a manner analogous to the way conventional TMA technology has significantly accelerated in situ studies of tissue specimens use of IMAs has similar potential to significantly accelerate CAD algorithm development.

An active learning based classification strategy for the minority class problem: application to histopathology annotation

Background

Supervised classifiers for digital pathology can improve the ability of physicians to detect and diagnose diseases such as cancer. Generating training data for classifiers is problematic, since only domain experts (e.g. pathologists) can correctly label ground truth data. Additionally, digital pathology datasets suffer from the "minority class problem", an issue where the number of exemplars from the non-target class outnumber target class exemplars which can bias the classifier and reduce accuracy. In this paper, we develop a training strategy combining active learning (AL) with class-balancing. AL identifies unlabeled samples that are "informative" (i.e. likely to increase classifier performance) for annotation, avoiding non-informative samples. This yields high accuracy with a smaller training set size compared with random learning (RL). Previous AL methods have not explicitly accounted for the minority class problem in biomedical images. Pre-specifying a target class ratio mitigates the problem of training bias. Finally, we develop a mathematical model to predict the number of annotations (cost) required to achieve balanced training classes. In addition to predicting training cost, the model reveals the theoretical properties of AL in the context of the minority class problem.

Results

Using this class-balanced AL training strategy (CBAL), we build a classifier to distinguish cancer from non-cancer regions on digitized prostate histopathology. Our dataset consists of 12,000 image regions sampled from 100 biopsies (58 prostate cancer patients). We compare CBAL against: (1) unbalanced AL (UBAL), which uses AL but ignores class ratio; (2) class-balanced RL (CBRL), which uses RL with a specific class ratio; and (3) unbalanced RL (UBRL). The CBAL-trained classifier yields 2% greater accuracy and 3% higher area under the receiver operating characteristic curve (AUC) than alternatively-trained classifiers. Our cost model accurately predicts the number of annotations necessary to obtain balanced classes. The accuracy of our prediction is verified by empirically-observed costs. Finally, we find that over-sampling the minority class yields a marginal improvement in classifier accuracy but the improved performance comes at the expense of greater annotation cost.

Conclusions

We have combined AL with class balancing to yield a general training strategy applicable to most supervised classification problems where the dataset is expensive to obtain and which suffers from the minority class problem. An intelligent training strategy is a critical component of supervised classification, but the integration of AL and intelligent choice of class ratios, as well as the application of a general cost model, will help researchers to plan the training process more quickly and effectively.

Quantitative metric profiles capture three-dimensional temporospatial architecture to discriminate cellular functional states

BACKGROUND

Computational analysis of tissue structure reveals sub-visual differences in tissue functional states by extracting quantitative signature features that establish a diagnostic profile. Incomplete and/or inaccurate profiles contribute to misdiagnosis.

METHODS

In order to create more complete tissue structure profiles, we adapted our cell-graph method for extracting quantitative features from histopathology images to now capture temporospatial traits of three-dimensional collagen hydrogel cell cultures. Cell-graphs were proposed to characterize the spatial organization between the cells in tissues by exploiting graph theory wherein the nuclei of the cells constitute the nodes and the approximate adjacency of cells are represented with edges. We chose 11 different cell types representing non-tumorigenic, pre-cancerous, and malignant states from multiple tissue origins.

RESULTS

We built cell-graphs from the cellular hydrogel images and computed a large set of features describing the structural characteristics captured by the graphs over time. Using three-mode tensor analysis, we identified the five most significant features (metrics) that capture the compactness, clustering, and spatial uniformity of the 3D architectural changes for each cell type throughout the time course. Importantly, four of these metrics are also the discriminative features for our histopathology data from our previous studies.

CONCLUSIONS

Together, these descriptive metrics provide rigorous quantitative representations of image information that other image analysis methods do not. Examining the changes in these five metrics allowed us to easily discriminate between all 11 cell types, whereas differences from visual examination of the images are not as apparent. These results demonstrate that application of the cell-graph technique to 3D image data yields discriminative metrics that have the potential to improve the accuracy of image-based tissue profiles, and thus improve the detection and diagnosis of disease.

A high-throughput active contour scheme for segmentation of histopathological imagery

In this paper a minimally interactive high-throughput system which employs a color gradient based active contour model for rapid and accurate segmentation of multiple target objects on very large images is presented. While geodesic active contours (GAC) have become very popular tools for image segmentation, they tend to be sensitive to model initialization. A second limitation of GAC models is that the edge detector function typically involves use of gray scale gradients; color images usually being converted to gray scale, prior to gradient computation. For color images, however, the gray scale gradient image results in broken edges and weak boundaries, since the other channels are not exploited in the gradient computation. To cope with these limitations, we present a new GAC model that is driven by an accurate and rapid object initialization scheme; hierarchical normalized cuts (HNCut). HNCut draws its strength from the integration of two powerful segmentation strategies-mean shift clustering and normalized cuts. HNCut involves first defining a color swatch (typically a few pixels) from the object of interest. A multi-scale, mean shift coupled normalized cuts algorithm then rapidly yields an initial accurate detection of all objects in the scene corresponding to the colors in the swatch. This detection result provides the initial contour for a GAC model. The edge-detector function of the GAC model employs a local structure tensor based color gradient, obtained by calculating the local min/max variations contributed from each color channel. We show that the color gradient based edge-detector function results in more prominent boundaries compared to the classical gray scale gradient based function. By integrating the HNCut initialization scheme with color gradient based GAC (CGAC), HNCut-CGAC embodies five unique and novel attributes: (1) efficiency in segmenting multiple target structures; (2) the ability to segment multiple objects from very large images; (3) minimal human interaction; (4) accuracy; and (5) reproducibility. A quantitative and qualitative comparison of the HNCut-CGAC model against other state of the art active contour schemes (including a Hybrid Active Contour model (Paragios-Deriche) and a region-based AC model (Rousson-Deriche)), across 196 digitized prostate histopathology images, suggests that HNCut-CGAC is able to outperform state of the art hybrid and region based AC techniques. Our results show that HNCut-CGAC is computationally efficient and may be easily applied to a variety of different problems and applications.

Spatially Invariant Vector Quantization: A pattern matching algorithm for multiple classes of image subject matter including pathology

Introduction:

Historically, effective clinical utilization of image analysis and pattern recognition algorithms in pathology has been hampered by two critical limitations: 1) the availability of digital whole slide imagery data sets and 2) a relative domain knowledge deficit in terms of application of such algorithms, on the part of practicing pathologists. With the advent of the recent and rapid adoption of whole slide imaging solutions, the former limitation has been largely resolved. However, with the expectation that it is unlikely for the general cohort of contemporary pathologists to gain advanced image analysis skills in the short term, the latter problem remains, thus underscoring the need for a class of algorithm that has the concurrent properties of image domain (or organ system) independence and extreme ease of use, without the need for specialized training or expertise.

Results:

In this report, we present a novel, general case pattern recognition algorithm, Spatially Invariant Vector Quantization (SIVQ), that overcomes the aforementioned knowledge deficit. Fundamentally based on conventional Vector Quantization (VQ) pattern recognition approaches, SIVQ gains its superior performance and essentially zero-training workflow model from its use of ring vectors, which exhibit continuous symmetry, as opposed to square or rectangular vectors, which do not. By use of the stochastic matching properties inherent in continuous symmetry, a single ring vector can exhibit as much as a millionfold improvement in matching possibilities, as opposed to conventional VQ vectors. SIVQ was utilized to demonstrate rapid and highly precise pattern recognition capability in a broad range of gross and microscopic use-case settings.

Conclusion:

With the performance of SIVQ observed thus far, we find evidence that indeed there exist classes of image analysis/pattern recognition algorithms suitable for deployment in settings where pathologists alone can effectively incorporate their use into clinical workflow, as a turnkey solution. We anticipate that SIVQ, and other related class-independent pattern recognition algorithms, will become part of the overall armamentarium of digital image analysis approaches that are immediately available to practicing pathologists, without the need for the immediate availability of an image analysis expert.

Computer-aided prognosis: predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing and applying computerized image analysis and multi-modal data fusion algorithms to digitized patient data (e.g. imaging, tissue, genomic) for helping physicians predict disease outcome and patient survival. While a number of data channels, ranging from the macro (e.g. MRI) to the nano-scales (proteins, genes) are now being routinely acquired for disease characterization, one of the challenges in predicting patient outcome and treatment response has been in our inability to quantitatively fuse these disparate, heterogeneous data sources. At the Laboratory for Computational Imaging and Bioinformatics (LCIB)(1) at Rutgers University, our team has been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities including MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on non-linear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate information from multiple data sources and modalities with the overarching goal of optimizing meta-classifiers for making prognostic predictions. In this paper, we briefly describe 4 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of Estrogen receptor positive breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in human epidermal growth factor amplified breast cancers) from digitized histopathology, (3) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitized needle biopsy specimens, and (4) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence following radical prostatectomy.