Optimal scheduling of tracing computations for real-time vascular landmark extraction from retinal fundus images

Automated identification of retinal vessels using a multiscale directional contrast quantification (MDCQ) strategy

PURPOSE

A novel algorithm is presented to automatically identify the retinal vessels depicted in color fundus photographs.

METHODS

The proposed algorithm quantifies the contrast of each pixel in retinal images at multiple scales and fuses the resulting consequent contrast images in a progressive manner by leveraging their spatial difference and continuity. The multiscale strategy is to deal with the variety of retinal vessels in width, intensity, resolution, and orientation; and the progressive fusion is to combine consequent images and meanwhile avoid a sudden fusion of image noise and/or artifacts in space. To quantitatively assess the performance of the algorithm, we tested it on three publicly available databases, namely, DRIVE, STARE, and HRF. The agreement between the computer results and the manual delineation in these databases were quantified by computing their overlapping in both area and length (centerline). The measures include sensitivity, specificity, and accuracy.

RESULTS

For the DRIVE database, the sensitivities in identifying vessels in area and length were around 90% and 70%, respectively, the accuracy in pixel classification was around 99%, and the precisions in terms of both area and length were around 94%. For the STARE database, the sensitivities in identifying vessels were around 90% in area and 70% in length, and the accuracy in pixel classification was around 97%. For the HRF database, the sensitivities in identifying vessels were around 92% in area and 83% in length for the healthy subgroup, around 92% in area and 75% in length for the glaucomatous subgroup, around 91% in area and 73% in length for the diabetic retinopathy subgroup. For all three subgroups, the accuracy was around 98%.

CONCLUSIONS

The experimental results demonstrate that the developed algorithm is capable of identifying retinal vessels depicted in color fundus photographs in a relatively reliable manner.

Retinal vessel width measurement at branchings using an improved electric field theory-based graph approach

The retinal vessel width relationship at vessel branch points in fundus images is an important biomarker of retinal and systemic disease. We propose a fully automatic method to measure the vessel widths at branch points in fundus images. The method is a graph-based method, in which a graph construction method based on electric field theory is applied which specifically deals with complex branching patterns. The vessel centerline image is used as the initial segmentation of the graph. Branching points are detected on the vessel centerline image using a set of detection kernels. Crossing points are distinguished from branch points and excluded. The electric field based graph method is applied to construct the graph. This method is inspired by the non-intersecting force lines in an electric field. At last, the method is further improved to give a consistent vessel width measurement for the whole vessel tree. The algorithm was validated on 100 artery branchings and 100 vein branchings selected from 50 fundus images by comparing with vessel width measurements from two human experts.

Blood vessel segmentation methodologies in retinal images--a survey

Retinal vessel segmentation algorithms are a fundamental component of automatic retinal disease screening systems. This work examines the blood vessel segmentation methodologies in two dimensional retinal images acquired from a fundus camera and a survey of techniques is presented. The aim of this paper is to review, analyze and categorize the retinal vessel extraction algorithms, techniques and methodologies, giving a brief description, highlighting the key points and the performance measures. We intend to give the reader a framework for the existing research; to introduce the range of retinal vessel segmentation algorithms; to discuss the current trends and future directions and summarize the open problems. The performance of algorithms is compared and analyzed on two publicly available databases (DRIVE and STARE) of retinal images using a number of measures which include accuracy, true positive rate, false positive rate, sensitivity, specificity and area under receiver operating characteristic (ROC) curve.

A function for quality evaluation of retinal vessel segmentations

Retinal blood vessel assessment plays an important role in the diagnosis of ophthalmic pathologies. The use of digital images for this purpose enables the application of a computerized approach and has fostered the development of multiple methods for automated vascular tree segmentation. Metrics based on contingency tables for binary classification have been widely used for evaluating the performance of these algorithms. Metrics from this family are based on the measurement of a success or failure rate in the detected pixels, obtained by means of pixel-to-pixel comparison between the automated segmentation and a manually-labeled reference image. Therefore, vessel pixels are not considered as a part of a vascular structure with specific features. This paper contributes a function for the evaluation of global quality in retinal vessel segmentations. This function is based on the characterization of vascular structures as connected segments with measurable area and length. Thus, its design is meant to be sensitive to anatomical vascularity features. Comparison of results between the proposed function and other general quality evaluation functions shows that this proposal renders a high matching degree with human quality perception. Therefore, it can be used to enhance quality evaluation in retinal vessel segmentations, supplementing the existing functions. On the other hand, from a general point of view, the applied concept of measuring descriptive properties may be used to design specialized functions aimed at segmentation quality evaluation in other complex structures.

Anisotropic path searching for automatic neuron reconstruction

Full reconstruction of neuron morphology is of fundamental interest for the analysis and understanding of their functioning. We have developed a novel method capable of automatically tracing neurons in three-dimensional microscopy data. In contrast to template-based methods, the proposed approach makes no assumptions about the shape or appearance of neurite structure. Instead, an efficient seeding approach is applied to capture complex neuronal structures and the tracing problem is solved by computing the optimal reconstruction with a weighted graph. The optimality is determined by the cost function designed for the path between each pair of seeds and by topological constraints defining the component interrelations and completeness. In addition, an automated neuron comparison method is introduced for performance evaluation and structure analysis. The proposed algorithm is computationally efficient and has been validated using different types of microscopy data sets including Drosophila's projection neurons and fly neurons with presynaptic sites. In all cases, the approach yielded promising results.

A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features

This paper presents a new supervised method for blood vessel detection in digital retinal images. This method uses a neural network (NN) scheme for pixel classification and computes a 7-D vector composed of gray-level and moment invariants-based features for pixel representation. The method was evaluated on the publicly available DRIVE and STARE databases, widely used for this purpose, since they contain retinal images where the vascular structure has been precisely marked by experts. Method performance on both sets of test images is better than other existing solutions in literature. The method proves especially accurate for vessel detection in STARE images. Its application to this database (even when the NN was trained on the DRIVE database) outperforms all analyzed segmentation approaches. Its effectiveness and robustness with different image conditions, together with its simplicity and fast implementation, make this blood vessel segmentation proposal suitable for retinal image computer analyses such as automated screening for early diabetic retinopathy detection.

Automatic neuron tracing in volumetric microscopy images with anisotropic path searching

Full reconstruction of neuron morphology is of fundamental interest for the analysis and understanding of neuron function. We have developed a novel method capable of tracing neurons in three-dimensional microscopy data automatically. In contrast to template-based methods, the proposed approach makes no assumptions on the shape or appearance of neuron's body. Instead, an efficient seeding approach is applied to find significant pixels almost certainly within complex neuronal structures and the tracing problem is solved by computing an graph tree structure connecting these seeds. In addition, an automated neuron comparison method is introduced for performance evaluation and structure analysis. The proposed algorithm is computationally efficient. Experiments on different types of data show promising results.

FABC: retinal vessel segmentation using AdaBoost

This paper presents a method for automated vessel segmentation in retinal images. For each pixel in the field of view of the image, a 41-D feature vector is constructed, encoding information on the local intensity structure, spatial properties, and geometry at multiple scales. An AdaBoost classifier is trained on 789 914 gold standard examples of vessel and nonvessel pixels, then used for classifying previously unseen images. The algorithm was tested on the public digital retinal images for vessel extraction (DRIVE) set, frequently used in the literature and consisting of 40 manually labeled images with gold standard. Results were compared experimentally with those of eight algorithms as well as the additional manual segmentation provided by DRIVE. Training was conducted confined to the dedicated training set from the DRIVE database, and feature-based AdaBoost classifier (FABC) was tested on the 20 images from the test set. FABC achieved an area under the receiver operating characteristic (ROC) curve of 0.9561, in line with state-of-the-art approaches, but outperforming their accuracy ( 0.9597 versus 0.9473 for the nearest performer).

3D Axon structure extraction and analysis in confocal fluorescence microscopy images

The morphological properties of axons, such as their branching patterns and oriented structures, are of great interest for biologists in the study of the synaptic connectivity of neurons. In these studies, researchers use triple immunofluorescent confocal microscopy to record morphological changes of neuronal processes. Three-dimensional (3D) microscopy image analysis is then required to extract morphological features of the neuronal structures. In this article, we propose a highly automated 3D centerline extraction tool to assist in this task. For this project, the most difficult part is that some axons are overlapping such that the boundaries distinguishing them are barely visible. Our approach combines a 3D dynamic programming (DP) technique and marker-controlled watershed algorithm to solve this problem. The approach consists of tracking and updating along the navigation directions of multiple axons simultaneously. The experimental results show that the proposed method can rapidly and accurately extract multiple axon centerlines and can handle complicated axon structures such as cross-over sections and overlapping objects.

Automatic Identification of Retinal Arteries and Veins From Dual-Wavelength Images Using Structural and Functional Features

This paper presents an automated method to identify arteries and veins in dual-wavelength retinal fundus images recorded at 570 and 600 nm. Dual-wavelength imaging provides both structural and functional features that can be exploited for identification. The processing begins with automated tracing of the vessels from the 570-nm image. The 600-nm image is registered to this image, and structural and functional features are computed for each vessel segment. We use the relative strength of the vessel central reflex as the structural feature. The central reflex phenomenon, caused by light reflection from vessel surfaces that are parallel to the incident light, is especially pronounced at longer wavelengths for arteries compared to veins. We use a dual-Gaussian to model the cross-sectional intensity profile of vessels. The model parameters are estimated using a robust M-estimator, and the relative strength of the central reflex is computed from these parameters. The functional feature exploits the fact that arterial blood is more oxygenated relative to that in veins. This motivates use of the ratio of the vessel optical densities (ODs) from images at oxygen-sensitive and oxygen-insensitive wavelengths (ODR = OD600/OD570) as a functional indicator. Finally, the structural and functional features are combined in a classifier to identify the type of the vessel. We experimented with four different classifiers and the best result was given by a support vector machine (SVM) classifier. With the SVM classifier, the proposed algorithm achieved true positive rates of 97% for the arteries and 90% for the veins, when applied to a set of 251 vessel segments obtained from 25 dual wavelength images. The ability to identify the vessel type is useful in applications such as automated retinal vessel oximetry and automated analysis of vascular changes without manual intervention.

NeuronMetrics: software for semi-automated processing of cultured neuron images

Using primary cell culture to screen for changes in neuronal morphology requires specialized analysis software. We developed NeuronMetrics for semi-automated, quantitative analysis of two-dimensional (2D) images of fluorescently labeled cultured neurons. It skeletonizes the neuron image using two complementary image-processing techniques, capturing fine terminal neurites with high fidelity. An algorithm was devised to span wide gaps in the skeleton. NeuronMetrics uses a novel strategy based on geometric features called faces to extract a branch number estimate from complex arbors with numerous neurite-to-neurite contacts, without creating a precise, contact-free representation of the neurite arbor. It estimates total neurite length, branch number, primary neurite number, territory (the area of the convex polygon bounding the skeleton and cell body), and Polarity Index (a measure of neuronal polarity). These parameters provide fundamental information about the size and shape of neurite arbors, which are critical factors for neuronal function. NeuronMetrics streamlines optional manual tasks such as removing noise, isolating the largest primary neurite, and correcting length for self-fasciculating neurites. Numeric data are output in a single text file, readily imported into other applications for further analysis. Written as modules for ImageJ, NeuronMetrics provides practical analysis tools that are easy to use and support batch processing. Depending on the need for manual intervention, processing time for a batch of approximately 60 2D images is 1.0-2.5 h, from a folder of images to a table of numeric data. NeuronMetrics' output accelerates the quantitative detection of mutations and chemical compounds that alter neurite morphology in vitro, and will contribute to the use of cultured neurons for drug discovery.

Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction

This paper presents an automated method for the segmentation of the vascular network in retinal images. The algorithm starts with the extraction of vessel centerlines, which are used as guidelines for the subsequent vessel filling phase. For this purpose, the outputs of four directional differential operators are processed in order to select connected sets of candidate points to be further classified as centerline pixels using vessel derived features. The final segmentation is obtained using an iterative region growing method that integrates the contents of several binary images resulting from vessel width dependent morphological filters. Our approach was tested on two publicly available databases and its results are compared with recently published methods. The results demonstrate that our algorithm outperforms other solutions and approximates the average accuracy of a human observer without a significant degradation of sensitivity and specificity.

Robust detection and classification of longitudinal changes in color retinal fundus images for monitoring diabetic retinopathy

A fully automated approach is presented for robust detection and classification of changes in longitudinal time-series of color retinal fundus images of diabetic retinopathy. The method is robust to: 1) spatial variations in illumination resulting from instrument limitations and changes both within, and between patient visits; 2) imaging artifacts such as dust particles; 3) outliers in the training data; 4) segmentation and alignment errors. Robustness to illumination variation is achieved by a novel iterative algorithm to estimate the reflectance of the retina exploiting automatically extracted segmentations of the retinal vasculature, optic disk, fovea, and pathologies. Robustness to dust artifacts is achieved by exploiting their spectral characteristics, enabling application to film-based, as well as digital imaging systems. False changes from alignment errors are minimized by subpixel accuracy registration using a 12-parameter transformation that accounts for unknown retinal curvature and camera parameters. Bayesian detection and classification algorithms are used to generate a color-coded output that is readily inspected. A multiobserver validation on 43 image pairs from 22 eyes involving nonproliferative and proliferative diabetic retinopathies, showed a 97% change detection rate, a 3% miss rate, and a 10% false alarm rate. The performance in correctly classifying the changes was 99.3%. A self-consistency metric, and an error factor were developed to measure performance over more than two periods. The average self consistency was 94% and the error factor was 0.06%. Although this study focuses on diabetic changes, the proposed techniques have broader applicability in ophthalmology.

Algorithms for automated oximetry along the retinal vascular tree from dual-wavelength fundus images

We present an automated method to perform accurate, rapid, and objective measurement of the blood oxygen saturation over each segment of the retinal vascular hierarchy from dual-wavelength fundus images. Its speed and automation (2 s per entire image versus 20 s per segment for manual methods) enables detailed level-by-level measurements over wider areas. An automated tracing algorithm is used to estimate vessel centerlines, thickness, directions, and locations of landmarks such as bifurcations and crossover points. The hierarchical structure of the vascular network is recovered from the trace fragments and landmarks by a novel algorithm. Optical densities (OD) are measured from vascular segments using the minimum reflected intensities inside and outside the vessel. The OD ratio (ODR=OD600/OD570) bears an inverse relationship to systemic HbO2 saturation (SO2). The sensitivity for detecting saturation change when breathing air versus pure oxygen was calculated from the measurements made on six subjects and was found to be 0.0226 ODR units, which is in good agreement with previous manual measurements by the dual-wavelength technique, indicating the validity of the automation. A fully automated system for retinal vessel oximetry would prove useful to achieve early assessments of risk for progression of disease conditions associated with oxygen utilization.

Robust Model-Based Vasculature Detection in Noisy Biomedical Images

This paper presents a set of algorithms for robust detection of vasculature in noisy retinal video images. Three methods are studied for effective handling of outliers. The first method is based on Huber's censored likelihood ratio test. The second is based on the use of a alpha-trimmed test statistic. The third is based on robust model selection algorithms. All of these algorithms rely on a mathematical model for the vasculature that accounts for the expected variations in intensity/texture profile, width, orientation, scale, and imaging noise. These unknown parameters are estimated implicitly within a robust detection and estimation framework. The proposed algorithms are also useful as nonlinear vessel enhancement filters. The proposed algorithms were evaluated over carefully constructed phantom images, where the ground truth is known a priori, as well as clinically recorded images for which the ground truth was manually compiled. A comparative evaluation of the proposed approaches is presented. Collectively, these methods outperformed prior approaches based on Chaudhuri et al. (1989) matched filtering, as well as the verification methods used by prior exploratory tracing algorithms, such as the work of Can et aL (1999). The Huber censored likelihood test yielded the best overall improvement, with a 145.7% improvement over the exploratory tracing algorithm, and a 43.7% improvement in detection rates over the matched filter.

Median-based robust algorithms for tracing neurons from noisy confocal microscope images

This paper presents a method to exploit rank statistics to improve fully automatic tracing of neurons from noisy digital confocal microscope images. Previously proposed exploratory tracing (vectorization) algorithms work by recursively following the neuronal topology, guided by responses of multiple directional correlation kernels. These algorithms were found to fail when the data was of lower quality (noisier, less contrast, weak signal, or more discontinuous structures). This type of data is commonly encountered in the study of neuronal growth on microfabricated surfaces. We show that by partitioning the correlation kernels in the tracing algorithm into multiple subkernels, and using the median of their responses as the guiding criterion improves the tracing precision from 41% to 89% for low-quality data, with a 5% improvement in recall. Improved handling was observed for artifacts such as discontinuities and/or hollowness of structures. The new algorithms require slightly higher amounts of computation, but are still acceptably fast, typically consuming less than 2 seconds on a personal computer (Pentium III, 500 MHz, 128 MB). They produce labeling for all somas present in the field, and a graph-theoretic representation of all dendritic/axonal structures that can be edited. Topological and size measurements such as area, length, and tortuosity are derived readily. The efficiency, accuracy, and fully-automated nature of the proposed method makes it attractive for large-scale applications such as high-throughput assays in the pharmaceutical industry, and study of neuron growth on nano/micro-fabricated structures. A careful quantitative validation of the proposed algorithms is provided against manually derived tracing, using a performance measure that combines the precision and recall metrics.

Topographically modified surfaces affect orientation and growth of hippocampal neurons

Extracellular matrix molecules provide biochemical and topographical cues that influence cell growth in vivo and in vitro. Effects of topographical cues on hippocampal neuron growth were examined after 14 days in vitro. Neurons from hippocampi of rat embryos were grown on poly-L-lysine-coated silicon surfaces containing fields of pillars with varying geometries. Photolithography was used to fabricate 1 microm high pillar arrays with different widths and spacings. Beta(III)-tubulin and MAP-2 immunocytochemistry and scanning electron microscopy were used to describe neuronal processes. Automated two-dimensional tracing software quantified process orientation and length. Process growth on smooth surfaces was random, while growth on pillared surfaces exhibited the most faithful alignment to pillar geometries with smallest gap sizes. Neurite lengths were significantly longer on pillars with the smallest inter-pillar spacings (gaps) and 2 microm pillar widths. These data indicate that physical cues affect neuron growth, suggesting that extracellular matrix topography may contribute to cell growth and differentiation. These results demonstrate new strategies for directing and promoting neuronal growth that will facilitate studies of synapse formation and function and provide methods to establish defined neural networks.

Model-Based Method for Improving the Accuracy and Repeatability of Estimating Vascular Bifurcations and Crossovers From Retinal Fundus Images

A model-based algorithm, termed exclusion region and position refinement (ERPR), is presented for improving the accuracy and repeatability of estimating the locations where vascular structures branch and cross over, in the context of human retinal images. The goal is two fold. First, accurate morphometry of branching and crossover points (landmarks) in neuronal/vascular structure is important to several areas of biology and medicine. Second, these points are valuable as landmarks for image registration, so improved accuracy and repeatability in estimating their locations and signatures leads to more reliable image registration for applications such as change detection and mosaicing. The ERPR algorithm is shown to reduce the median location error from 2.04 pixels down to 1.1 pixels, while improving the median spread (a measure of repeatability) from 2.09 pixels down to 1.05 pixels. Errors in estimating vessel orientations were similarly reduced from 7.2 degrees down to 3.8 degrees.

Efficient Migration of Complex Off-Line Computer Vision Software to Real-Time System Implementation on Generic Computer Hardware

This paper addresses the problem of migrating large and complex computer vision code bases that have been developed off-line, into efficient real-time implementations avoiding the need for rewriting the software, and the associated costs. Creative linking strategies based on Linux loadable kernel modules are presented to create a simultaneous realization of real-time and off-line frame rate computer vision systems from a single code base. In this approach, systemic predictability is achieved by inserting time-critical components of a user-level executable directly into the kernel as a virtual device driver. This effectively emulates a single process space model that is nonpreemptable, nonpageable, and that has direct access to a powerful set of system-level services. This overall approach is shown to provide the basis for building a predictable frame-rate vision system using commercial off-the-shelf hardware and a standard uniprocessor Linux operating system. Experiments on a frame-rate vision system designed for computer-assisted laser retinal surgery show that this method reduces the variance of observed per-frame central processing unit cycle counts by two orders of magnitude. The conclusion is that when predictable application algorithms are used, it is possible to efficiently migrate to a predictable frame-rate computer vision system.

Predictive Scheduling Algorithms for Real-Time Feature Extraction and Spatial Referencing: Application to Retinal Image Sequences

Real-time spatial referencing is an important alternative to tracking for designing spatially aware ophthalmic instrumentation for procedures such as laser photocoagulation and perimetry. It requires independent, fast registration of each image frame from a digital video stream (1024 x 1024 pixels) to a spatial map of the retina. Recently, we have introduced a spatial referencing algorithm that works in three primary steps: 1) tracing the retinal vasculature to extract image feature (landmarks); 2) invariant indexing to generate hypothesized landmark correspondences and initial transformations; and 3) alignment and verification steps to robustly estimate a 12-parameter quadratic spatial transformation between the image frame and the map. The goal of this paper is to introduce techniques to minimize the amount of computation for successful spatial referencing. The fundamental driving idea is to make feature extraction subservient to registration and, therefore, only produce the information needed for verified, accurate transformations. To this end, the image is analyzed along one-dimensional, vertical and horizontal grid lines to produce a regular sampling of the vasculature, needed for step 3) and to initiate step 1). Tracing of the vascular is then prioritized hierarchically to quickly extract landmarks and groups (constellations) of landmarks for indexing. Finally, the tracing and spatial referencing computations are integrated so that landmark constellations found by tracing are tested immediately. The resulting implementation is an order-of-magnitude faster with the same success rate. The average total computation time is 31.2 ms per image on a 2.2-GHz Pentium Xeon processor.

Automated tracing and change analysis of angiogenic vasculature from in vivo multiphoton confocal image time series

Automated methods are described for tracing and analysis of changes in angiogenic vasculature imaged by a multiphoton laser-scanning confocal microscope. Utilizing chronic animal window models, time series of in vivo 3-D images were acquired on approximately the same target volume of the same specimen while undergoing angiogenic change (typically every 24 h for 7 days). Objective, precise, 3-D, rapid, and fully automated vessel morphometry was performed using an adaptive tracing algorithm that is based on a generalized irregular cylinder model of the vasculature. This algorithm was found to be not only adaptive enough for tracing angiogenic vasculature, but also very efficient in its use of computer memory, and fast, taking less than 1 min to trace a 768 x 512 x 32, 8-bit/pixel 3-D image stack on a Dell Pentium III 1-GHz computer. The automatically traced centerlines were manually validated on six image stacks and the average spatial error was measured to be 2 pixels, with an average concordance of 81% between manual and automated traces on a voxel basis. The tracing output includes geometrical statistics of traced vasculature and serves as the basis of statistical change analysis. The computer methods described here are designed to be scalable to much larger hypothesis testing studies involving quantitative measurements of tumor angiogenesis, gene expression relative to known vascular structures, and impact of drug delivery.

Rapid automated three-dimensional tracing of neurons from confocal image stacks

Algorithms are presented for fully automatic three-dimensional (3-D) tracing of neurons that are imaged by fluorescence confocal microscopy. Unlike previous voxel-based skeletonization methods, the present approach works by recursively following the neuronal topology, using a set of 4 x N2 directional kernels (e.g., N = 32), guided by a generalized 3-D cylinder model. This method extends our prior work on exploratory tracing of retinal vasculature to 3-D space. Since the centerlines are of primary interest, the 3-D extension can be accomplished by four rather than six sets of kernels. Additional modifications, such as dynamic adaptation of the correlation kernels, and adaptive step size estimation, were introduced for achieving robustness to photon noise, varying contrast, and apparent discontinuity and/or hollowness of structures. The end product is a labeling of all somas present, graph-theoretic representations of all dendritic/axonal structures, and image statistics such as soma volume and centroid, soma interconnectivity, the longest branch, and lengths of all graph branches originating from a soma. This method is able to work directly with unprocessed confocal images, without expensive deconvolution or other preprocessing. It is much faster that skeletonization, typically consuming less than a minute to trace a 70-MB image on a 500-MHz computer. These properties make it attractive for large-scale automated tissue studies that require rapid on-line image analysis, such as high-throughput neurobiology/angiogenesis assays, and initiatives such as the Human Brain Project.