Computer-assisted discrimination among malignant lymphomas and leukemia using immunophenotyping, intelligent image repositories, and telemicroscopy

An Expandable Informatics Framework for Enhancing Central Cancer Registries with Digital Pathology Specimens, Computational Imaging Tools, and Advanced Mining Capabilities

BACKGROUND

Population-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI's Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features).

MATERIALS AND METHODS

As part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated.

RESULTS

Our team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics.

CONCLUSION

To meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

Automated Decision Support System for Detection of Leukemia from Peripheral Blood Smear Images

Peripheral blood smear analysis plays a vital role in diagnosing many diseases including cancer. Leukemia is a type of cancer which begins in bone marrow and results in increased number of white blood cells in peripheral blood. Unusual variations in appearance of white blood cells indicate leukemia. In this paper, an automated method for detection of leukemia using image processing approach is proposed. In the present study, 1159 images of different brightness levels and color shades were acquired from Leishman stained peripheral blood smears. SVM classifier was used for classification of white blood cells into normal and abnormal, and also for detection of leukemic WBCs from the abnormal class. Classification of the normal white blood cells into five sub-types was performed using NN classifier. Overall classification accuracy of 98.8% was obtained using the combination of NN and SVM.

Image processing and machine learning in the morphological analysis of blood cells

INTRODUCTION

This review focuses on how image processing and machine learning can be useful for the morphological characterization and automatic recognition of cell images captured from peripheral blood smears.

METHODS

The basics of the 3 core elements (segmentation, quantitative features, and classification) are outlined, and recent literature is discussed. Although red blood cells are a significant part of this context, this study focuses on malignant lymphoid cells and blast cells.

RESULTS

There is no doubt that these technologies may help the cytologist to perform efficient, objective, and fast morphological analysis of blood cells. They may also help in the interpretation of some morphological features and may serve as learning and survey tools.

CONCLUSION

Although research is still needed, it is important to define screening strategies to exploit the potential of image-based automatic recognition systems integrated in the daily routine of laboratories along with other analysis methodologies.

Characterization and automatic screening of reactive and abnormal neoplastic B lymphoid cells from peripheral blood

INTRODUCTION

The objective was to advance in the automatic, image-based, characterization and recognition of a heterogeneous set of lymphoid cells from peripheral blood, including normal, reactive, and five groups of abnormal lymphocytes: hairy cells, mantle cells, follicular lymphoma, chronic lymphocytic leukemia, and prolymphocytes.

METHODS

A number of 4389 images from 105 patients were selected by pathologists, based on morphologic visual appearance, from patients whose diagnosis was confirmed by all the remaining complementary tests. Besides geometry, new color and texture features were extracted using six alternative color spaces to obtain rich information to characterize the cell groups. The recognition system was designed using support vector machines trained with the whole image set.

RESULTS

In the experimental tests, individual sets of images from 21 new patients were analyzed by the trained recognition system and compared with the true diagnosis. An overall recognition accuracy of 97.67% was achieved when the cell screening was performed into three groups: normal lymphocytes, abnormal lymphoid cells, and reactive lymphocytes. The accuracy of the whole experimental study was 91.23% when considering the further discrimination of the abnormal lymphoid cells into the specific five groups.

CONCLUSION

The excellent automatic screening of the three groups of normal, reactive, and abnormal lymphocytes is useful as it discriminates between malignancy and not malignancy. The discrimination of the five groups of abnormal lymphoid cells is encouraging toward the idea that the system could be an automated image-based screening method to identify blood involvement by a variety of B lymphomas.

The history of pathology informatics: A global perspective

Pathology informatics has evolved to varying levels around the world. The history of pathology informatics in different countries is a tale with many dimensions. At first glance, it is the familiar story of individuals solving problems that arise in their clinical practice to enhance efficiency, better manage (e.g., digitize) laboratory information, as well as exploit emerging information technologies. Under the surface, however, lie powerful resource, regulatory, and societal forces that helped shape our discipline into what it is today. In this monograph, for the first time in the history of our discipline, we collectively perform a global review of the field of pathology informatics. In doing so, we illustrate how general far-reaching trends such as the advent of computers, the Internet and digital imaging have affected pathology informatics in the world at large. Major drivers in the field included the need for pathologists to comply with national standards for health information technology and telepathology applications to meet the scarcity of pathology services and trained people in certain countries. Following trials by a multitude of investigators, not all of them successful, it is apparent that innovation alone did not assure the success of many informatics tools and solutions. Common, ongoing barriers to the widespread adoption of informatics devices include poor information technology infrastructure in undeveloped areas, the cost of technology, and regulatory issues. This review offers a deeper understanding of how pathology informatics historically developed and provides insights into what the promising future might hold.

A new algorithm for the simulation of SDS 2D-PAGE datasets

This chapter describes a new software for the generation of simulated realistic sodium dodecyl sulfate two-dimensional polyacrylamide gel electrophoresis (SDS 2D-PAGE) images. In order to choose the simulation strategy to provide realistic 2D-PAGE maps the statistical characteristics of such images were taken into account, such as the distributions of sizes, intensities, and volumes of the spots. Also, the low reproducibility typical of replicated SDS 2D-PAGE maps of the same sample was simulated. This approach can be used to generate simulated datasets useful in the development and performance evaluation of new classification and/or image analysis algorithms applied to two-dimensional electrophoresis datasets, given the usually small number of experimental replications available.

A generic and extensible automatic classification framework applied to brain tumour diagnosis in HealthAgents

New biomedical technologies enable the diagnosis of brain tumours by using non-invasive methods. HealthAgents is a European Union-funded research project that aims to build an agent-based distributed decision support system (dDSS) for the diagnosis of brain tumours. This is achieved using the latest biomedical knowledge, information and communication technologies and pattern recognition (PR) techniques. As part of the PR development of HealthAgents, an independent and automatic classification framework (CF) has been developed. This framework has been integrated with the HealthAgents dDSS using the HealthAgents agent platform. The system offers (1) the functionality to search for distributed classifiers to solve specific questions; (2) automatic classification of new cases; (3) instant deployment of new validated classifiers; and (4) the ability to rank a set of classifiers according to their performance and suitability for the case in hand. The CF enables both the deployment of new classifiers using the provided Extensible Markup Language1 classifier specification, and the inclusion of new PR techniques that make the system extensible. These features may enable the rapid integration of PR laboratory results into industrial or research applications, such as the HealthAgents dDSS. Two classification nodes have been deployed and they currently offer classification services by means of dedicated servers connected to the HealthAgents agent platform: one node being located at the Katholieke Universiteit Leuven, Belgium and the other at the Universidad Politécnica de Valencia, Spain. These classification nodes share the current set of brain tumour classifiers that have been trained from in vivo magnetic resonance spectroscopy data. The combination of the CF with a distributed agent system constitutes the basis of the brain tumour dDSS developed in HealthAgents.

Automatic classification of lymphoma images with transform-based global features

We propose a report on automatic classification of three common types of malignant lymphoma: chronic lymphocytic leukemia, follicular lymphoma, and mantle cell lymphoma. The goal was to find patterns indicative of lymphoma malignancies and allowing classifying these malignancies by type. We used a computer vision approach for quantitative characterization of image content. A unique two-stage approach was employed in this study. At the outer level, raw pixels were transformed with a set of transforms into spectral planes. Simple (Fourier, Chebyshev, and wavelets) and compound transforms (Chebyshev of Fourier and wavelets of Fourier) were computed. Raw pixels and spectral planes were then routed to the second stage (the inner level). At the inner level, the set of multipurpose global features was computed on each spectral plane by the same feature bank. All computed features were fused into a single feature vector. The specimens were stained with hematoxylin (H) and eosin (E) stains. Several color spaces were used: RGB, gray, CIE-L*a*b*, and also the specific stain-attributed H&E space, and experiments on image classification were carried out for these sets. The best signal (98%-99% on earlier unseen images) was found for the HE, H, and E channels of the H&E data set.

Diagnostic pathology and laboratory medicine in the age of "omics": a paper from the 2006 William Beaumont Hospital Symposium on Molecular Pathology

Functional genomics and proteomics involve the simultaneous analysis of hundreds or thousands of expressed genes or proteins and have spawned the modern discipline of computational biology. Novel informatic applications, including sophisticated dimensionality reduction strategies and cancer outlier profile analysis, can distill clinically exploitable biomarkers from enormous experimental datasets. Diagnostic pathologists are now charged with translating the knowledge generated by the "omics" revolution into clinical practice. Food and Drug Administration-approved proprietary testing platforms based on microarray technologies already exist and will expand greatly in the coming years. However, for diagnostic pathology, the greatest promise of the "omics" age resides in the explosion in information technology (IT). IT applications allow for the digitization of histological slides, transforming them into minable data and enabling content-based searching and archiving of histological materials. IT will also allow for the optimization of existing (and often underused) clinical laboratory technologies such as flow cytometry and high-throughput core laboratory functions. The state of pathology practice does not always keep up with the pace of technological advancement. However, to use fully the potential of these emerging technologies for the benefit of patients, pathologists and clinical scientists must embrace the changes and transformational advances that will characterize this new era.

A method for linking computed image features to histological semantics in neuropathology

In medical image analysis the image content is often represented by features computed from the pixel matrix in order to support the development of improved clinical diagnosis systems. These features need to be interpreted and understood at a clinical level of understanding Many features are of abstract nature, as for instance features derived from a wavelet transform. The interpretation and analysis of such features are difficult. This lack of coincidence between computed features and their meaning for a user in a given situation is commonly referred to as the semantic gap. In this work, we propose a method for feature analysis and interpretation based on the simultaneous visualization of feature and image domain. Histopathological images of meningiomas WHO (World Health Organization) grade I are represented by features derived from color transforms and the Discrete Wavelet Transform. The wavelet-based feature space is then visualized and explored using unsupervised machine learning methods. We show how to analyze and select features according to their relevance for the description of clinically relevant characteristics.

Classification of hematologic malignancies using texton signatures

We describe a decision support system to distinguish among hematology cases directly from microscopic specimens. The system uses an image database containing digitized specimens from normal and four different hematologic malignancies. Initially, the nuclei and cytoplasmic components of the specimens are segmented using a robust color gradient vector flow active contour model. Using a few cell images from each class, the basic texture elements (textons) for the nuclei and cytoplasm are learned, and the cells are represented through texton histograms. We propose to use support vector machines on the texton histogram based cell representation and achieve major improvement over the commonly used classification methods in texture research. Experiments with 3,691 cell images from 105 patients which originated from four different hospitals indicate more than 84% classification performance for individual cells and 89% for case based classification for the five class problem.

Visual positioning of previously defined ROIs on microscopic slides

In microscopy, regions of interest are usually much smaller than the whole slide area. Various microscopy related medical applications, such as telepathology and computer aided diagnosis, are liable to benefit greatly from microscope auto positioning on previously defined regions of interest. In this paper, we present a method for image-based auto positioning on a microscope slide. The method is based on localization of a microscopic query image using a previously acquired slide map. It uses geometric hashing, a highly efficient technique drawn from the object recognition field. The algorithm exhibits high tolerance to possible variations in visual appearance due to slide rotations, scaling and illumination changes. Experimental results indicate high reliability of the algorithm.

Decentralized Data Sharing of Tissue Microarrays for Investigative Research in Oncology

Tissue microarray technology (TMA) is a relatively new approach for efficiently and economically assessing protein and gene expression across large ensembles of tissue specimens. Tissue microarray technology holds great potential for reducing the time and cost associated with conducting research in tissue banking, proteomics, and outcome studies. However, the sheer volume of images and other data generated from even limited studies involving tissue microarrays quickly approach the processing capacity and resources of a division or department. This challenge is compounded by the fact that large-scale projects in several areas of modern research rely upon multi-institutional efforts in which investigators and resources are spread out over multiple campuses, cities, and states. To address some of the data management issues several leading institutions have begun to develop their own “in-house” systems, independently, but such data will be only minimally useful if it isn't accessible to others in the scientific community. Investigators at different institutions studying the same or related disorders might benefit from the synergy of sharing results. To facilitate sharing of TMA data across different database implementations, the Technical Standards Committee of the Association for Pathology Informatics organized workshops in efforts to establish a standardized TMA data exchange specification. The focus of our research does not relate to the establishment of standards for exchange, but rather builds on these efforts and concentrates on the design, development and deployment of a decentralized collaboratory for the unsupervised characterization, and seamless and secure discovery and sharing of TMA data. Specifically, we present a self-organizing, peer-to-peer indexing and discovery infrastructure for quantitatively assessing digitized TMA's. The system utilizes a novel, optimized decentralized search engine that supports flexible querying, while guaranteeing that once information has been stored in the system, it will be found with bounded costs.

Unsupervised segmentation based on robust estimation and color active contour models

One of the most commonly used clinical tests performed today is the routine evaluation of peripheral blood smears. In this paper, we investigate the design, development, and implementation of a robust color gradient vector flow (GVF) active contour model for performing segmentation, using a database of 1791 imaged cells. The algorithms developed for this research operate in Luv color space, and introduce a color gradient and L2E robust estimation into the traditional GVF snake. The accuracy of the new model was compared with the segmentation results using a mean-shift approach, the traditional color GVF snake, and several other commonly used segmentation strategies. The unsupervised robust color snake with L2E robust estimation was shown to provide results which were superior to the other unsupervised approaches, and was comparable with supervised segmentation, as judged by a panel of human experts.

Image mining for investigative pathology using optimized feature extraction and data fusion

In many subspecialties of pathology, the intrinsic complexity of rendering accurate diagnostic decisions is compounded by a lack of definitive criteria for detecting and characterizing diseases and their corresponding histological features. In some cases, there exists a striking disparity between the diagnoses rendered by recognized authorities and those provided by non-experts. We previously reported the development of an Image Guided Decision Support (IGDS) system, which was shown to reliably discriminate among malignant lymphomas and leukemia that are sometimes confused with one another during routine microscopic evaluation. As an extension of those efforts, we report here a web-based intelligent archiving subsystem that can automatically detect, image, and index new cells into distributed ground-truth databases. Systematic experiments showed that through the use of robust texture descriptors and density estimation based fusion the reliability and performance of the governing classifications of the system were improved significantly while simultaneously reducing the dimensionality of the feature space.

A prototype for unsupervised analysis of tissue microarrays for cancer research and diagnostics

The tissue microarray (TMA) technique enables researchers to extract small cylinders of tissue from histological sections and arrange them in a matrix configuration on a recipient paraffin block such that hundreds can be analyzed simultaneously. TMA offers several advantages over traditional specimen preparation by maximizing limited tissue resources and providing a highly efficient means for visualizing molecular targets. By enabling researchers to reliably determine the protein expression profile for specific types of cancer, it may be possible to elucidate the mechanism by which healthy tissues are transformed into malignancies. Currently, the primary methods used to evaluate arrays involve the interactive review of TMA samples while they are viewed under a microscope, subjectively evaluated, and scored by a technician. This process is extremely slow, tedious, and prone to error. In order to facilitate large-scale, multi-institutional studies, a more automated and reliable means for analyzing TMAs is needed. We report here a web-based prototype which features automated imaging, registration, and distributed archiving of TMAs in multiuser network environments. The system utilizes a principal color decomposition approach to identify and characterize the predominant staining signatures of specimens in color space. This strategy was shown to be reliable for detecting and quantifying the immunohistochemical expression levels for TMAs.

The virtual microscope

We present the design and implementation of the Virtual Microscope, a software system employing a client/server architecture to provide a realistic emulation of a high power light microscope. The system provides a form of completely digital telepathology, allowing simultaneous access to archived digital slide images by multiple clients. The main problem the system targets is storing and processing the extremely large quantities of data required to represent a collection of slides. The Virtual Microscope client software runs on the end user's PC or workstation, while database software for storing, retrieving and processing the microscope image data runs on a parallel computer or on a set of workstations at one or more potentially remote sites. We have designed and implemented two versions of the data server software. One implementation is a customization of a database system framework that is optimized for a tightly coupled parallel machine with attached local disks. The second implementation is component-based, and has been designed to accommodate access to and processing of data in a distributed, heterogeneous environment. We also have developed caching client software, implemented in Java, to achieve good response time and portability across different computer platforms. The performance results presented show that the Virtual Microscope systems scales well, so that many clients can be adequately serviced by an appropriately configured data server.

The virtual blood film

The computer and the digital camera offer unprecedented possibilities for improving hematology education, research, and patient service. Peripheral blood smear images of exceptional quality can be acquired rapidly and conveniently from the peripheral blood smear with a modern, high-resolution digital camera and a quality microscope. Digital cameras use CCD or CMOS image sensors to measure light energy and additional circuitry to convert the measured information into a digital signal. Because digital cameras do not use photographic film, images are immediately available for incorporation into web sites or digital publications, printing, transfer to other individuals by e-mail, or other applications. Several excellent consumer digital still cameras are now available for less than $1000 that capture high-quality images comprised of more than three megapixels. These images are essentially indistinguishable from conventional film images when viewed on a quality color monitor or printed on a quality color or black and white printer at sizes up to 8 x 10 in. Several recent dedicated digital photomicroscopy cameras provide an ultrahigh quality image output of more than 12 megapixels and have low noise circuit designs permitting the direct capture of darkfield and fluorescence images. There are many applications of digital images of peripheral blood smears. Because hematology is a visual science, the inclusion of quality digital images into lectures, teaching handouts, and electronic documents is essential. A few institutions have gone beyond the basic application of digital images to develop large electronic hematology atlases; animated, audio-enhanced learning experiences; multidisciplinary Internet conferences; and other innovative applications. Digital images of single microscopic fields (single-frame images) are the most widely used in hematology education at this time, but single images of many adjacent microscopic fields can be stitched together to prepare zoomable panoramas that encompass a large part of a microscope slide and closely stimulate observation through a real microscope. With further advances in computer speed and Internet streaming technology, the virtual microscope could easily replace the real microscope in pathology education. Interactive, immersive computer experiences may completely revolutionize hematology education and make the conventional lecture and laboratory format obsolete later in this decade. Patient care is enhanced by the transmission of digital images to other individuals for consultation and education, and by the inclusion of these images in patient care documents. In research laboratories, digital cameras are widely used to document experimental results and obtain experimental data.