Rapid automated three-dimensional tracing of neurons from confocal image stacks

NetMets: software for quantifying and visualizing errors in biological network segmentation

One of the major goals in biomedical image processing is accurate segmentation of networks embedded in volumetric data sets. Biological networks are composed of a meshwork of thin filaments that span large volumes of tissue. Examples of these structures include neurons and microvasculature, which can take the form of both hierarchical trees and fully connected networks, depending on the imaging modality and resolution. Network function depends on both the geometric structure and connectivity. Therefore, there is considerable demand for algorithms that segment biological networks embedded in three-dimensional data. While a large number of tracking and segmentation algorithms have been published, most of these do not generalize well across data sets. One of the major reasons for the lack of general-purpose algorithms is the limited availability of metrics that can be used to quantitatively compare their effectiveness against a pre-constructed ground-truth. In this paper, we propose a robust metric for measuring and visualizing the differences between network models. Our algorithm takes into account both geometry and connectivity to measure network similarity. These metrics are then mapped back onto an explicit model for visualization.

Automated tracing of neurites from light microscopy stacks of images

Automating the process of neural circuit reconstruction on a large-scale is one of the foremost challenges in the field of neuroscience. In this study we examine the methodology for circuit reconstruction from three-dimensional light microscopy (LM) stacks of images. We show how the minimal error-rate of an ideal reconstruction procedure depends on the density of labeled neurites, giving rise to the fundamental limitation of an LM based approach for neural circuit research. Circuit reconstruction procedures typically involve steps related to neuron labeling and imaging, and subsequent image pre-processing and tracing of neurites. In this study, we focus on the last step--detection of traces of neurites from already pre-processed stacks of images. Our automated tracing algorithm, implemented as part of the Neural Circuit Tracer software package, consists of the following main steps. First, image stack is filtered to enhance labeled neurites. Second, centerline of the neurites is detected and optimized. Finally, individual branches of the optimal trace are merged into trees based on a cost minimization approach. The cost function accounts for branch orientations, distances between their end-points, curvature of the merged structure, and its intensity. The algorithm is capable of connecting branches which appear broken due to imperfect labeling and can resolve situations where branches appear to be fused due the limited resolution of light microscopy. The Neural Circuit Tracer software is designed to automatically incorporate ImageJ plug-ins and functions written in MatLab and provides roughly a 10-fold increases in speed in comparison to manual tracing.

Neuronal tracing for connectomic studies

Reconstruction of the complete wiring diagram, or connectome, of a neural circuit provides an alternative approach to conventional circuit analysis. One major obstacle of connectomics lies in segmenting and tracing neuronal processes from the vast number of images obtained with optical or electron microscopy. Here I review recent progress in automated tracing algorithms for connectomic reconstruction with fluorescence and electron microscopy, and discuss the challenges to image analysis posed by novel optical imaging techniques.

Automated reconstruction of neuronal morphology based on local geometrical and global structural models

Digital reconstruction of neurons from microscope images is an important and challenging problem in neuroscience. In this paper, we propose a model-based method to tackle this problem. We first formulate a model structure, then develop an algorithm for computing it by carefully taking into account morphological characteristics of neurons, as well as the image properties under typical imaging protocols. The method has been tested on the data sets used in the DIADEM competition and produced promising results for four out of the five data sets.

Automated reconstruction of dendritic and axonal trees by global optimization with geometric priors

We present a novel probabilistic approach to fully automated delineation of tree structures in noisy 2D images and 3D image stacks. Unlike earlier methods that rely mostly on local evidence, ours builds a set of candidate trees over many different subsets of points likely to belong to the optimal tree and then chooses the best one according to a global objective function that combines image evidence with geometric priors. Since the best tree does not necessarily span all the points, the algorithm is able to eliminate false detections while retaining the correct tree topology. Manually annotated brightfield micrographs, retinal scans and the DIADEM challenge datasets are used to evaluate the performance of our method. We used the DIADEM metric to quantitatively evaluate the topological accuracy of the reconstructions and showed that the use of the geometric regularization yields a substantial improvement.

A broadly applicable 3-D neuron tracing method based on open-curve snake

This paper presents a broadly applicable algorithm and a comprehensive open-source software implementation for automated tracing of neuronal structures in 3-D microscopy images. The core 3-D neuron tracing algorithm is based on three-dimensional (3-D) open-curve active Contour (Snake). It is initiated from a set of automatically detected seed points. Its evolution is driven by a combination of deforming forces based on the Gradient Vector Flow (GVF), stretching forces based on estimation of the fiber orientations, and a set of control rules. In this tracing model, bifurcation points are detected implicitly as points where multiple snakes collide. A boundariness measure is employed to allow local radius estimation. A suite of pre-processing algorithms enable the system to accommodate diverse neuronal image datasets by reducing them to a common image format. The above algorithms form the basis for a comprehensive, scalable, and efficient software system developed for confocal or brightfield images. It provides multiple automated tracing modes. The user can optionally interact with the tracing system using multiple view visualization, and exercise full control to ensure a high quality reconstruction. We illustrate the utility of this tracing system by presenting results from a synthetic dataset, a brightfield dataset and two confocal datasets from the DIADEM challenge.

Three-dimensional image quantification as a new morphometry method for tissue engineering

Morphological analysis is an essential step in verifying the success of a tissue engineering strategy where the presence of a desired cellular phenotype must be determined. While morphometry has transitioned from observational grading to computational quantification, established quantitative methods eliminate information by relying on two-dimensional (2D) analysis to describe three-dimensional (3D) niches. In this study, we demonstrate the validity and utility of 3D morphological quantification using two common angiogenesis assays in our fibrin-based in vitro model: (1) the microcarrier bead assay with human mesenchymal stem cells and (2) the rat aortic ring outgrowth assay. The quantification method is based on collecting and segmenting fluorescent confocal z-stacks into 3D models with 3D Slicer, an open-source magnetic resonance imaging/computed tomography analysis program. Data from 3D models are then processed into biologically relevant metrics in MATLAB for statistical analysis. Metrics include descriptive parameters such as vascular network length, volume, number of network segments, and degree of network branching. Our results indicate that 2D measures are significantly different than their 3D counterparts unless the vascular network exhibits anisotropic growth along the plane of imaging. Additionally, the statistical outcomes of 3D morphological quantification agreed with our initial qualitative observations among different test groups. This novel quantification approach generates more spatially accurate and objective measures, representing an important step toward improving the reliability of morphological comparisons.

Multiscale exploration of mouse brain microstructures using the knife-edge scanning microscope brain atlas

Connectomics is the study of the full connection matrix of the brain. Recent advances in high-throughput, high-resolution 3D microscopy methods have enabled the imaging of whole small animal brains at a sub-micrometer resolution, potentially opening the road to full-blown connectomics research. One of the first such instruments to achieve whole-brain-scale imaging at sub-micrometer resolution is the Knife-Edge Scanning Microscope (KESM). KESM whole-brain data sets now include Golgi (neuronal circuits), Nissl (soma distribution), and India ink (vascular networks). KESM data can contribute greatly to connectomics research, since they fill the gap between lower resolution, large volume imaging methods (such as diffusion MRI) and higher resolution, small volume methods (e.g., serial sectioning electron microscopy). Furthermore, KESM data are by their nature multiscale, ranging from the subcellular to the whole organ scale. Due to this, visualization alone is a huge challenge, before we even start worrying about quantitative connectivity analysis. To solve this issue, we developed a web-based neuroinformatics framework for efficient visualization and analysis of the multiscale KESM data sets. In this paper, we will first provide an overview of KESM, then discuss in detail the KESM data sets and the web-based neuroinformatics framework, which is called the KESM brain atlas (KESMBA). Finally, we will discuss the relevance of the KESMBA to connectomics research, and identify challenges and future directions.

Automatic 3D neuron tracing using all-path pruning

Motivation: Digital reconstruction, or tracing, of 3D neuron structures is critical toward reverse engineering the wiring and functions of a brain. However, despite a number of existing studies, this task is still challenging, especially when a 3D microscopic image has low signal-to-noise ratio (SNR) and fragmented neuron segments. Published work can handle these hard situations only by introducing global prior information, such as where a neurite segment starts and terminates. However, manual incorporation of such global information can be very time consuming. Thus, a completely automatic approach for these hard situations is highly desirable.

Results: We have developed an automatic graph algorithm, called the all-path pruning (APP), to trace the 3D structure of a neuron. To avoid potential mis-tracing of some parts of a neuron, an APP first produces an initial over-reconstruction, by tracing the optimal geodesic shortest path from the seed location to every possible destination voxel/pixel location in the image. Since the initial reconstruction contains all the possible paths and thus could contain redundant structural components (SC), we simplify the entire reconstruction without compromising its connectedness by pruning the redundant structural elements, using a new maximal-covering minimal-redundant (MCMR) subgraph algorithm. We show that MCMR has a linear computational complexity and will converge. We examined the performance of our method using challenging 3D neuronal image datasets of model organisms (e.g. fruit fly).

Availability: The software is available upon request. We plan to eventually release the software as a plugin of the V3D-Neuron package at http://penglab.janelia.org/proj/v3d.

Contact:pengh@janelia.hhmi.org

Automated segmentation of electron tomograms for a quantitative description of actin filament networks

Cryo-electron tomography allows to visualize individual actin filaments and to describe the three-dimensional organization of actin networks in the context of unperturbed cellular environments. For a quantitative characterization of actin filament networks, the tomograms must be segmented in a reproducible manner. Here, we describe an automated procedure for the segmentation of actin filaments, which combines template matching with a new tracing algorithm. The result is a set of lines, each one representing the central line of a filament. As demonstrated with cryo-tomograms of cellular actin networks, these line sets can be used to characterize filament networks in terms of filament length, orientation, density, stiffness (persistence length), or the occurrence of branching points.

Anisotropic path searching for automatic neuron reconstruction

Full reconstruction of neuron morphology is of fundamental interest for the analysis and understanding of their functioning. We have developed a novel method capable of automatically tracing neurons in three-dimensional microscopy data. In contrast to template-based methods, the proposed approach makes no assumptions about the shape or appearance of neurite structure. Instead, an efficient seeding approach is applied to capture complex neuronal structures and the tracing problem is solved by computing the optimal reconstruction with a weighted graph. The optimality is determined by the cost function designed for the path between each pair of seeds and by topological constraints defining the component interrelations and completeness. In addition, an automated neuron comparison method is introduced for performance evaluation and structure analysis. The proposed algorithm is computationally efficient and has been validated using different types of microscopy data sets including Drosophila's projection neurons and fly neurons with presynaptic sites. In all cases, the approach yielded promising results.

Fast extraction of neuron morphologies from large-scale SBFSEM image stacks

Neuron morphology is frequently used to classify cell-types in the mammalian cortex. Apart from the shape of the soma and the axonal projections, morphological classification is largely defined by the dendrites of a neuron and their subcellular compartments, referred to as dendritic spines. The dimensions of a neuron’s dendritic compartment, including its spines, is also a major determinant of the passive and active electrical excitability of dendrites. Furthermore, the dimensions of dendritic branches and spines change during postnatal development and, possibly, following some types of neuronal activity patterns, changes depending on the activity of a neuron. Due to their small size, accurate quantitation of spine number and structure is difficult to achieve (Larkman, J Comp Neurol 306:332, 1991). Here we follow an analysis approach using high-resolution EM techniques. Serial block-face scanning electron microscopy (SBFSEM) enables automated imaging of large specimen volumes at high resolution. The large data sets generated by this technique make manual reconstruction of neuronal structure laborious. Here we present NeuroStruct, a reconstruction environment developed for fast and automated analysis of large SBFSEM data sets containing individual stained neurons using optimized algorithms for CPU and GPU hardware. NeuroStruct is based on 3D operators and integrates image information from image stacks of individual neurons filled with biocytin and stained with osmium tetroxide. The focus of the presented work is the reconstruction of dendritic branches with detailed representation of spines. NeuroStruct delivers both a 3D surface model of the reconstructed structures and a 1D geometrical model corresponding to the skeleton of the reconstructed structures. Both representations are a prerequisite for analysis of morphological characteristics and simulation signalling within a neuron that capture the influence of spines.

Automated reconstruction of neuronal morphology: an overview

Digital reconstruction of neuronal morphology is a powerful technique for investigating the nervous system. This process consists of tracing the axonal and dendritic arbors of neurons imaged by optical microscopy into a geometrical format suitable for quantitative analysis and computational modeling. Algorithmic automation of neuronal tracing promises to increase the speed, accuracy, and reproducibility of morphological reconstructions. Together with recent breakthroughs in cellular imaging and accelerating progress in optical microscopy, automated reconstruction of neuronal morphology will play a central role in the development of high throughput screening and the acquisition of connectomic data. Yet, despite continuous advances in image processing algorithms, to date manual tracing remains the overwhelming choice for digitizing neuronal morphology. We summarize the issues involved in automated reconstruction, overview the available techniques, and provide a realistic assessment of future perspectives.

Automatic neuron tracing in volumetric microscopy images with anisotropic path searching

Full reconstruction of neuron morphology is of fundamental interest for the analysis and understanding of neuron function. We have developed a novel method capable of tracing neurons in three-dimensional microscopy data automatically. In contrast to template-based methods, the proposed approach makes no assumptions on the shape or appearance of neuron's body. Instead, an efficient seeding approach is applied to find significant pixels almost certainly within complex neuronal structures and the tracing problem is solved by computing an graph tree structure connecting these seeds. In addition, an automated neuron comparison method is introduced for performance evaluation and structure analysis. The proposed algorithm is computationally efficient. Experiments on different types of data show promising results.

Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model

Motivation: Digital reconstruction of 3D neuron structures is an important step toward reverse engineering the wiring and functions of a brain. However, despite a number of existing studies, this task is still challenging, especially when a 3D microscopic image has low single-to-noise ratio and discontinued segments of neurite patterns.

Results: We developed a graph-augmented deformable model (GD) to reconstruct (trace) the 3D structure of a neuron when it has a broken structure and/or fuzzy boundary. We formulated a variational problem using the geodesic shortest path, which is defined as a combination of Euclidean distance, exponent of inverse intensity of pixels along the path and closeness to local centers of image intensity distribution. We solved it in two steps. We first used a shortest path graph algorithm to guarantee that we find the global optimal solution of this step. Then we optimized a discrete deformable curve model to achieve visually more satisfactory reconstructions. Within our framework, it is also easy to define an optional prior curve that reflects the domain knowledge of a user. We investigated the performance of our method using a number of challenging 3D neuronal image datasets of different model organisms including fruit fly, Caenorhabditis elegans, and mouse. In our experiments, the GD method outperformed several comparison methods in reconstruction accuracy, consistency, robustness and speed. We further used GD in two real applications, namely cataloging neurite morphology of fruit fly to build a 3D ‘standard’ digital neurite atlas, and estimating the synaptic bouton density along the axons for a mouse brain.

Availability: The software is provided as part of the V3D-Neuron 1.0 package freely available at http://penglab.janelia.org/proj/v3d

Contact:pengh@janelia.hhmi.org

Reconstruction of the neuromuscular junction connectome

MOTIVATION

Unraveling the structure and behavior of the brain and central nervous system (CNS) has always been a major goal of neuroscience. Understanding the wiring diagrams of the neuromuscular junction connectomes (full connectivity of nervous system neuronal components) is a starting point for this, as it helps in the study of the organizational and developmental properties of the mammalian CNS. The phenomenon of synapse elimination during developmental stages of the neuronal circuitry is such an example. Due to the organizational specificity of the axons in the connectomes, it becomes important to label and extract individual axons for morphological analysis. Features such as axonal trajectories, their branching patterns, geometric information, the spatial relations of groups of axons, etc. are of great interests for neurobiologists in the study of wiring diagrams. However, due to the complexity of spatial structure of the axons, automatically tracking and reconstructing them from microscopy images in 3D is an unresolved problem. In this article, AxonTracker-3D, an interactive 3D axon tracking and labeling tool is built to obtain quantitative information by reconstruction of the axonal structures in the entire innervation field. The ease of use along with accuracy of results makes AxonTracker-3D an attractive tool to obtain valuable quantitative information from axon datasets.

AVAILABILITY

The software is freely available for download at http://www.cbi-tmhs.org/AxonTracker/.

Neuron tracing in perspective

The study of the structure and function of neuronal cells and networks is of crucial importance in the endeavor to understand how the brain works. A key component in this process is the extraction of neuronal morphology from microscopic imaging data. In the past four decades, many computational methods and tools have been developed for digital reconstruction of neurons from images, with limited success. As witnessed by the growing body of literature on the subject, as well as the organization of challenging competitions in the field, the quest for a robust and fully automated system of more general applicability still continues. The aim of this work, is to contribute by surveying recent developments in the field for anyone interested in taking up the challenge. Relevant aspects discussed in the article include proposed image segmentation methods, quantitative measures of neuronal morphology, currently available software tools for various related purposes, and morphology databases. (c) 2010 International Society for Advancement of Cytometry.

Automatic robust neurite detection and morphological analysis of neuronal cell cultures in high-content screening

Cell-based high content screening (HCS) is becoming an important and increasingly favored approach in therapeutic drug discovery and functional genomics. In HCS, changes in cellular morphology and biomarker distributions provide an information-rich profile of cellular responses to experimental treatments such as small molecules or gene knockdown probes. One obstacle that currently exists with such cell-based assays is the availability of image processing algorithms that are capable of reliably and automatically analyzing large HCS image sets. HCS images of primary neuronal cell cultures are particularly challenging to analyze due to complex cellular morphology. Here we present a robust method for quantifying and statistically analyzing the morphology of neuronal cells in HCS images. The major advantages of our method over existing software lie in its capability to correct non-uniform illumination using the contrast-limited adaptive histogram equalization method; segment neuromeres using Gabor-wavelet texture analysis; and detect faint neurites by a novel phase-based neurite extraction algorithm that is invariant to changes in illumination and contrast and can accurately localize neurites. Our method was successfully applied to analyze a large HCS image set generated in a morphology screen for polyglutamine-mediated neuronal toxicity using primary neuronal cell cultures derived from embryos of a Drosophila Huntington's Disease (HD) model.

Advanced microscopy techniques for quantitative analysis in neuromorphology and neuropathology research: current status and requirements for the future

Visualizing neuromorphology and in particular neuropathology has been the focus of many researchers in the quest to solve the numerous questions that are still remaining related to several neurological and neuropsychiatric diseases. Over the last years, intense research into microscopy techniques has resulted in the development of various new types of microscopes, software imaging systems, and analysis programs. This review briefly discusses some key techniques, such as confocal stereology and automated neuron tracing and reconstruction, and their applications in neuroscience research. Special emphasis is placed on needs for further developments, such as the demand for higher-throughput analyses in quantitative neuromorphology. These developments will advance basic neuroscience research as well as pharmaceutical and biotechnology research and development.

Steerable features for statistical 3D dendrite detection

Most state-of-the-art algorithms for filament detection in 3-D image-stacks rely on computing the Hessian matrix around individual pixels and labeling these pixels according to its eigenvalues. This approach, while very effective for clean data in which linear structures are nearly cylindrical, loses its effectiveness in the presence of noisy data and irregular structures. In this paper, we show that using steerable filters to create rotationally invariant features that include higher-order derivatives and training a classifier based on these features lets us handle such irregular structures. This can be done reliably and at acceptable computational cost and yields better results than state-of-the-art methods.

Neurite outgrowth at the biomimetic interface

Understanding the cues that guide axons and how we can optimize these cues to achieve directed neuronal growth is imperative for neural tissue engineering. Cells in the local environment influence neurons with a rich combination of cues. This study deconstructs the complex mixture of guidance cues by working at the biomimetic interface--isolating the topographical information presented by cells and determining its capacity to guide neurons. We generated replica materials presenting topographies of oriented astrocytes (ACs), endothelial cells (ECs), and Schwann cells (SCs) as well as computer-aided design materials inspired by the contours of these cells (bioinspired-CAD). These materials presented distinct topographies and anisotropies and in all cases were sufficient to guide neurons. Dorsal root ganglia (DRG) cells and neurites demonstrated the most directed response on bioinspired-CAD materials which presented anisotropic features with 90 degrees edges. DRG alignment was strongest on SC bioinspired-CAD materials followed by AC bioinspired-CAD materials, with more uniform orientation to EC bioinspired-CAD materials. Alignment on replicas was strongest on SC replica materials followed by AC and EC replicas. These results suggest that the topographies of anisotropic tissue structures are sufficient for neuronal guidance. This work is discussed in the context of feature dimensions, morphology, and guidepost hypotheses.

A computational framework for studying neuron morphology from in vitro high content neuron-based screening

High content neuron image processing is considered as an important method for quantitative neurobiological studies. The main goal of analysis in this paper is to provide automatic image processing approaches to process neuron images for studying neuron mechanism in high content screening. In the nuclei channel, all nuclei are segmented and detected by applying the gradient vector field based watershed. Then the neuronal nuclei are selected based on the soma region detected in neurite channel. In neurite images, we propose a novel neurite centerline extraction approach using the improved line-pixel detection technique. The proposed neurite tracing method can detect the curvilinear structure more accurately compared with the current existing methods. An interface called NeuriteIQ based on the proposed algorithms is developed finally for better application in high content screening.

Automated image analysis for high-content screening and analysis

The field of high-content screening and analysis consists of a set of methodologies for automated discovery in cell biology and drug development using large amounts of image data. In most cases, imaging is carried out by automated microscopes, often assisted by automated liquid handling and cell culture. Image processing, computer vision, and machine learning are used to automatically process high-dimensional image data into meaningful cell biological results. The key is creating automated analysis pipelines typically consisting of 4 basic steps: (1) image processing (normalization, segmentation, tracing, tracking), (2) spatial transformation to bring images to a common reference frame (registration), (3) computation of image features, and (4) machine learning for modeling and interpretation of data. An overview of these image analysis tools is presented here, along with brief descriptions of a few applications.

V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets

The V3D system provides three-dimensional (3D) visualization of gigabyte-sized microscopy image stacks in real time on current laptops and desktops. Combined with highly ergonomic features for selecting an X, Y, Z location of an image directly in 3D space and for visualizing overlays of a variety of surface objects, V3D streamlines the on-line analysis, measurement, and proofreading of complicated image patterns. V3D is cross-platform and can be enhanced by plug-ins. We built V3D-Neuron on top of V3D to reconstruct complex 3D neuronal structures from large brain images. V3D-Neuron enables us to precisely digitize the morphology of a single neuron in a fruit fly brain in minutes, with about 17-fold improvement in reliability and 10-fold savings in time compared to other neuron reconstruction tools. Using V3D-Neuron, we demonstrated the feasibility of building a 3D digital atlas of neurite tracts in the fruit fly brain.

Convolutional networks can learn to generate affinity graphs for image segmentation

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

Robust adaptive 3-D segmentation of vessel laminae from fluorescence confocal microscope images and parallel GPU implementation

This paper presents robust 3-D algorithms to segment vasculature that is imaged by labeling laminae, rather than the lumenal volume. The signal is weak, sparse, noisy, nonuniform, low-contrast, and exhibits gaps and spectral artifacts, so adaptive thresholding and Hessian filtering based methods are not effective. The structure deviates from a tubular geometry, so tracing algorithms are not effective. We propose a four step approach. The first step detects candidate voxels using a robust hypothesis test based on a model that assumes Poisson noise and locally planar geometry. The second step performs an adaptive region growth to extract weakly labeled and fine vessels while rejecting spectral artifacts. To enable interactive visualization and estimation of features such as statistical confidence, local curvature, local thickness, and local normal, we perform the third step. In the third step, we construct an accurate mesh representation using marching tetrahedra, volume-preserving smoothing, and adaptive decimation algorithms. To enable topological analysis and efficient validation, we describe a method to estimate vessel centerlines using a ray casting and vote accumulation algorithm which forms the final step of our algorithm. Our algorithm lends itself to parallel processing, and yielded an 8 x speedup on a graphics processor (GPU). On synthetic data, our meshes had average error per face (EPF) values of (0.1-1.6) voxels per mesh face for peak signal-to-noise ratios from (110-28 dB). Separately, the error from decimating the mesh to less than 1% of its original size, the EPF was less than 1 voxel/face. When validated on real datasets, the average recall and precision values were found to be 94.66% and 94.84%, respectively.

Dendritic vulnerability in neurodegenerative disease: insights from analyses of cortical pyramidal neurons in transgenic mouse models

In neurodegenerative disorders, such as Alzheimer's disease, neuronal dendrites and dendritic spines undergo significant pathological changes. Because of the determinant role of these highly dynamic structures in signaling by individual neurons and ultimately in the functionality of neuronal networks that mediate cognitive functions, a detailed understanding of these changes is of paramount importance. Mutant murine models, such as the Tg2576 APP mutant mouse and the rTg4510 tau mutant mouse have been developed to provide insight into pathogenesis involving the abnormal production and aggregation of amyloid and tau proteins, because of the key role that these proteins play in neurodegenerative disease. This review showcases the multidimensional approach taken by our collaborative group to increase understanding of pathological mechanisms in neurodegenerative disease using these mouse models. This approach includes analyses of empirical 3D morphological and electrophysiological data acquired from frontal cortical pyramidal neurons using confocal laser scanning microscopy and whole-cell patch-clamp recording techniques, combined with computational modeling methodologies. These collaborative studies are designed to shed insight on the repercussions of dystrophic changes in neocortical neurons, define the cellular phenotype of differential neuronal vulnerability in relevant models of neurodegenerative disease, and provide a basis upon which to develop meaningful therapeutic strategies aimed at preventing, reversing, or compensating for neurodegenerative changes in dementia.

Multi scale and slice-based approach for automatic spine detection

Dendritic spines play an essential role in the central nervous system. Recent experiments have revealed that neuron functional properties are highly correlated with the statistical and morphological changes of the dendritic spines. In this paper, we propose a new multi scale approach for detecting dendritic spines in a 2D Maximum Intensity Projection (MIP) image of the 3D neuron data stacks collected from a 2-photon laser scanning confocal microscope. The proposed method utilizes the curvilinear structure detector in conjunction with the multi scale spine detection algorithm which automatically and accurately extracts and segments the spines with variational sizes along the dendrite. In addition, a slice-based spine detection algorithm is also proposed to detect spines which are hidden from the MIP image within the dendrite area. Experimental results show that our proposed method is effective for automatic spine detection and is able to accurately segment dendrite.

MDL constrained 3-D grayscale skeletonization algorithm for automated extraction of dendrites and spines from fluorescence confocal images

This paper presents a method for improved automatic delineation of dendrites and spines from three-dimensional (3-D) images of neurons acquired by confocal or multi-photon fluorescence microscopy. The core advance presented here is a direct grayscale skeletonization algorithm that is constrained by a structural complexity penalty using the minimum description length (MDL) principle, and additional neuroanatomy-specific constraints. The 3-D skeleton is extracted directly from the grayscale image data, avoiding errors introduced by image binarization. The MDL method achieves a practical tradeoff between the complexity of the skeleton and its coverage of the fluorescence signal. Additional advances include the use of 3-D spline smoothing of dendrites to improve spine detection, and graph-theoretic algorithms to explore and extract the dendritic structure from the grayscale skeleton using an intensity-weighted minimum spanning tree (IW-MST) algorithm. This algorithm was evaluated on 30 datasets organized in 8 groups from multiple laboratories. Spines were detected with false negative rates less than 10% on most datasets (the average is 7.1%), and the average false positive rate was 11.8%. The software is available in open source form.

Automatic summarization of changes in biological image sequences using algorithmic information theory

An algorithmic information-theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG). Time courses of object states are compared using an adaptive information distance measure, aided by a closed-form multidimensional quantization. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. The summary is the clustering result and feature subset that maximize the gap statistic. This approach was validated on four bioimaging applications: 1) It was applied to a synthetic data set containing two populations of cells differing in the rate of growth, for which it correctly identified the two populations and the single feature out of 23 that separated them; 2) it was applied to 59 movies of three types of neuroprosthetic devices being inserted in the brain tissue at three speeds each, for which it correctly identified insertion speed as the primary factor affecting tissue strain; 3) when applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain; and 4) when analyzing intracellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification.

Three-dimensional neuron tracing by voxel scooping

Tracing the centerline of the dendritic arbor of neurons is a powerful technique for analyzing neuronal morphology. In the various neuron tracing algorithms in use nowadays, the competing goals of computational efficiency and robustness are generally traded off against each other. We present a novel method for tracing the centerline of a neuron from confocal image stacks, which provides an optimal balance between these objectives. Using only local information, thin cross-sectional layers of voxels ('scoops') are iteratively carved out of the structure, and clustered based on connectivity. Each cluster contributes a node along the centerline, which is created by connecting successive nodes until all object voxels are exhausted. While data segmentation is independent of this algorithm, we illustrate the use of the ISODATA method to achieve dynamic (local) segmentation. Diameter estimation at each node is calculated using the Rayburst Sampling algorithm, and spurious end nodes caused by surface irregularities are then removed. On standard computing hardware the algorithm can process hundreds of thousands of voxels per second, easily handling the multi-gigabyte datasets resulting from high-resolution confocal microscopy imaging of neurons. This method provides an accurate and efficient means for centerline extraction that is suitable for interactive neuron tracing applications.

Hardware accelerated segmentation of complex volumetric filament networks

We present a framework for segmenting and storing filament networks from scalar volume data. Filament networks are encountered more and more commonly in biomedical imaging due to advances in high-throughput microscopy. These data sets are characterized by a complex volumetric network of thin filaments embedded in a scalar volume field. High-throughput microscopy volumes are also difficult to manage since they can require several terabytes of storage, even though the total volume of the embedded structure is much smaller. Filaments in microscopy data sets are difficult to segment because their diameter is often near the sampling resolution of the microscope, yet these networks can span large regions of the data set. We describe a novel method to trace filaments through scalar volume data sets that is robust to both noisy and undersampled data. We use graphics hardware to accelerate the tracing algorithm, making it more useful for large data sets. After the initial network is traced, we use an efficient encoding scheme to store volumetric data pertaining to the network.

Quantitative neurite outgrowth measurement based on image segmentation with topological dependence

The study of neuronal morphology and neurite outgrowth has been enhanced by the combination of imaging informatics and high content screening, in which thousands of images are acquired using robotic fluorescent microscopy. To understand the process of neurite outgrowth in the context of neuroregeneration, we used mouse neuroblastoma N1E115 as our model neuronal cell. Six-thousand cellular images of four different culture conditions were acquired with two-channel widefield fluorescent microscopy. We developed a software package called NeuronCyto. It is a fully automatic solution for neurite length measurement and complexity analysis. A novel approach based on topological analysis is presented to segment cells. The detected nuclei were used as references to initialize the level set function. Merging and splitting of cells segments were prevented using dynamic watershed lines based on the constraint of topological dependence. A tracing algorithm was developed to automatically trace neurites and measure their lengths quantitatively on a cell-by-cell basis. NeuronCyto analyzes three important biologically relevant features, which are the length, branching complexity, and number of neurites. The application of NeuronCyto on the experiments of Toca-1 and serum starvation show that the transfection of Toca-1 cDNA induces longer neurites with more complexities than serum starvation.

Semi-automated reconstruction of neural processes from large numbers of fluorescence images

We introduce a method for large scale reconstruction of complex bundles of neural processes from fluorescent image stacks. We imaged yellow fluorescent protein labeled axons that innervated a whole muscle, as well as dendrites in cerebral cortex, in transgenic mice, at the diffraction limit with a confocal microscope. Each image stack was digitally re-sampled along an orientation such that the majority of axons appeared in cross-section. A region growing algorithm was implemented in the open-source Reconstruct software and applied to the semi-automatic tracing of individual axons in three dimensions. The progression of region growing is constrained by user-specified criteria based on pixel values and object sizes, and the user has full control over the segmentation process. A full montage of reconstructed axons was assembled from the approximately 200 individually reconstructed stacks. Average reconstruction speed is approximately 0.5 mm per hour. We found an error rate in the automatic tracing mode of approximately 1 error per 250 um of axonal length. We demonstrated the capacity of the program by reconstructing the connectome of motor axons in a small mouse muscle.

An automated pipeline for dendrite spine detection and tracking of 3D optical microscopy neuron images of in vivo mouse models

The variations in dendritic branch morphology and spine density provide insightful information about the brain function and possible treatment to neurodegenerative disease, for example investigating structural plasticity during the course of Alzheimer's disease. Most automated image processing methods aiming at analyzing these problems are developed for in vitro data. However, in vivo neuron images provide real time information and direct observation of the dynamics of a disease process in a live animal model. This paper presents an automated approach for detecting spines and tracking spine evolution over time with in vivo image data in an animal model of Alzheimer's disease. We propose an automated pipeline starting with curvilinear structure detection to determine the medial axis of the dendritic backbone and spines connected to the backbone. We, then, propose the adaptive local binary fitting (aLBF) energy level set model to accurately locate the boundary of dendritic structures using the central line of curvilinear structure as initialization. To track the growth or loss of spines, we present a maximum likelihood based technique to find the graph homomorphism between two image graph structures at different time points. We employ dynamic programming to search for the optimum solution. The pipeline enables us to extract dynamically changing information from real time in vivo data. We validate our proposed approach by comparing with manual results generated by neurologists. In addition, we discuss the performance of 3D based segmentation and conclude that our method is more accurate in identifying weak spines. Experiments show that our approach can quickly and accurately detect and quantify spines of in vivo neuron images and is able to identify spine elimination and formation.

Sharpening of neurite morphology using complex coherence enhanced diffusion

The study of the molecular mechanisms involved in neurite outgrowth and differentiation, requires essential accurate and reproducible segmentation and quantification of neuronal processes. The common method used in this study is to detect and trace individual neurites, i.e. neurite tracing. The challenge comes mainly from the morphological problem in which these images contains ambiguities such as neurites discontinuities and intensity differences. In our work, we encounter a bigger challenge as the neurites in our images have a higher density of neurites. In this paper, we present a hybrid complex coherence-enhanced method for sharpening the morphology of neurons from such images. Coherence-enhanced diffusion (CED) is used to enhance the flowlike structures of the neurites, while the imaginary part of the complex nonlinear diffusion of the image cancels the appearance of 'clouds'. We also describe an elementary method for estimating the density of neuritis based on the obtained images. Our preliminary results show that the proposed methodology is a step ahead toward an effective neuronal morphology algorithm.

Neuron branch detection and description using random walk

The morphological studies of neuron structures are of great interests for biologists. However, manually detecting dendrites structures is very labor intensive, therefore unfeasible in studies that involve a large number of images. In this paper, we propose an automated neuron detection and description method. The proposed method uses ratios of probability maps from random walk sessions to detect initial seed-points and minimal cost path integrals with Delaunay triangulations.

Detection of the optimal neuron traces in confocal microscopy images

Quantitative methods of analysis of neural circuits rely on large datasets of neurons reconstructed accurately in three dimensions (3D). Due to the complexity of neuronal arbors, large datasets of reconstructed neurons must be generated with automated algorithms. Here, we attempted to automate the process of neuron tracing from sparsely labeled 3D stacks of confocal microscopy images. Our algorithm involves two steps. In the first step, the segmented image of neurites in the stack is voxel-coded. Centers of intensity of consecutively coded wave fronts are connected into a branched structure, which represents a coarse trace of the neurites. In the second step, this trace is optimized with the modified active contour method, which tends to maximize the intensity along the trace while keeping it under tension. To assess the performance of the algorithm we used manual reconstructions of neurons and converted them into artificial stacks of intensity images. These images were traced using the developed algorithm and quantitatively compared to the corresponding manual traces. The optimal traces were on average 6.0% shorter than the manual traces. This reduction in length resulted from the smoothness of the optimal traces, which, in comparison to the manual ones, were built out of shorter segments, and, as a result, were 3.3% less tortuous. The average distance between the optimal and the manual traces was 0.14 microm, and the average distance between their corresponding branch-points was 2.2 microm, illustrating good agreement between the traces.

Visualization of cellular and microvascular relationships

Understanding the structure of microvasculature structures and their relationship to cells in biological tissue is an important and complex problem. Brain microvasculature in particular is known to play an important role in chronic diseases. However, these networks are only visible at the microscopic level and can span large volumes of tissue. Due to recent advances in microscopy, large volumes of data can be imaged at the resolution necessary to reconstruct these structures. Due to the dense and complex nature of microscopy data sets, it is important to limit the amount of information displayed. In this paper, we describe methods for encoding the unique structure of microvascular data, allowing researchers to selectively explore microvascular anatomy. We also identify the queries most useful to researchers studying microvascular and cellular relationships. By associating cellular structures with our microvascular framework, we allow researchers to explore interesting anatomical relationships in dense and complex data sets.

3D Axon structure extraction and analysis in confocal fluorescence microscopy images

The morphological properties of axons, such as their branching patterns and oriented structures, are of great interest for biologists in the study of the synaptic connectivity of neurons. In these studies, researchers use triple immunofluorescent confocal microscopy to record morphological changes of neuronal processes. Three-dimensional (3D) microscopy image analysis is then required to extract morphological features of the neuronal structures. In this article, we propose a highly automated 3D centerline extraction tool to assist in this task. For this project, the most difficult part is that some axons are overlapping such that the boundaries distinguishing them are barely visible. Our approach combines a 3D dynamic programming (DP) technique and marker-controlled watershed algorithm to solve this problem. The approach consists of tracking and updating along the navigation directions of multiple axons simultaneously. The experimental results show that the proposed method can rapidly and accurately extract multiple axon centerlines and can handle complicated axon structures such as cross-over sections and overlapping objects.

Robust 3D reconstruction and identification of dendritic spines from optical microscopy imaging

In neurobiology, the 3D reconstruction of neurons followed by the identification of dendritic spines is essential for studying neuronal morphology, function and biophysical properties. Most existing methods suffer from problems of low reliability, poor accuracy and require much user interaction. In this paper, we present a method to reconstruct dendrites using a surface representation of the neuron. The skeleton of the dendrite is extracted by a procedure based on the medial geodesic function that is robust and topology preserving, and it is used to accurately identify spines. The sensitivity of the algorithm on the various parameters is explored in detail and the method is shown to be robust.

Bioimage informatics: a new area of engineering biology

In recent years, the deluge of complicated molecular and cellular microscopic images creates compelling challenges for the image computing community. There has been an increasing focus on developing novel image processing, data mining, database and visualization techniques to extract, compare, search and manage the biological knowledge in these data-intensive problems. This emerging new area of bioinformatics can be called ‘bioimage informatics’. This article reviews the advances of this field from several aspects, including applications, key techniques, available tools and resources. Application examples such as high-throughput/high-content phenotyping and atlas building for model organisms demonstrate the importance of bioimage informatics. The essential techniques to the success of these applications, such as bioimage feature identification, segmentation and tracking, registration, annotation, mining, image data management and visualization, are further summarized, along with a brief overview of the available bioimage databases, analysis tools and other resources.

Contact:pengh@janelia.hhmi.org

Supplementary information:Supplementary data are available at Bioinformatics online.

Automated three-dimensional detection and shape classification of dendritic spines from fluorescence microscopy images

A fundamental challenge in understanding how dendritic spine morphology controls learning and memory has been quantifying three-dimensional (3D) spine shapes with sufficient precision to distinguish morphologic types, and sufficient throughput for robust statistical analysis. The necessity to analyze large volumetric data sets accurately, efficiently, and in true 3D has been a major bottleneck in deriving reliable relationships between altered neuronal function and changes in spine morphology. We introduce a novel system for automated detection, shape analysis and classification of dendritic spines from laser scanning microscopy (LSM) images that directly addresses these limitations. The system is more accurate, and at least an order of magnitude faster, than existing technologies. By operating fully in 3D the algorithm resolves spines that are undetectable with standard two-dimensional (2D) tools. Adaptive local thresholding, voxel clustering and Rayburst Sampling generate a profile of diameter estimates used to classify spines into morphologic types, while minimizing optical smear and quantization artifacts. The technique opens new horizons on the objective evaluation of spine changes with synaptic plasticity, normal development and aging, and with neurodegenerative disorders that impair cognitive function.

Semi-automated quantification of filopodial dynamics

Cellular motility underlies critical physiological processes including embryogenesis, metastasis and wound healing. Nerve cells undergo cellular migration during development and also extend neuronal processes for long distances through a complex microenvironment to appropriately wire the nervous system. The growth cone is a highly dynamic structure that responds to extracellular cues by extending and retracting filopodia and lamellipodia to explore the microenvironment and to dictate the path and speed of process extension. Neuronal responses to a myriad of guidance cues have been studied biochemically, however, these approaches fail to capture critical spatio-temporal elements of growth cone dynamics. Live imaging of growth cones in culture has emerged as a powerful tool to study growth cone responses to guidance cues but the dynamic nature of the growth cone requires careful quantitative analysis. Space time kymographs have been developed as a tool to quantify lamellipodia dynamics in a semi-automated fashion but no such tools exist to analyze filopodial dynamics. In this work we present an algorithm to quantify filopodial dynamics from cultured neurons imaged by time-lapse fluorescence microscopy. The method is based on locating the end tips of filopodia and tracking their locations as if they were free-moving particles. The algorithm is a useful tool and should be broadly applicable to filopodial tracking from multiple cell types.

Automated axon tracking of 3D confocal laser scanning microscopy images using guided probabilistic region merging

This paper presents a new algorithm for extracting the centerlines of the axons from a 3D data stack collected by a confocal laser scanning microscope. Recovery of neuronal structures from such datasets is critical for quantitatively addressing a range of neurobiological questions such as the manner in which the branching pattern of motor neurons change during synapse elimination. Unfortunately, the data acquired using fluorescence microscopy contains many imaging artifacts, such as blurry boundaries and non-uniform intensities of fluorescent radiation. This makes the centerline extraction difficult. We propose a robust segmentation method based on probabilistic region merging to extract the centerlines of individual axons with minimal user interaction. The 3D model of the extracted axon centerlines in three datasets is presented in this paper. The results are validated with the manual tracking results while the robustness of the algorithm is compared with the published repulsive snake algorithm.