Shape-driven three-dimensional watersnake segmentation of biological membranes in electron tomography

Computational Methods Toward Unbiased Pattern Mining and Structure Determination in Cryo-Electron Tomography Data

Cryo-electron tomography can uniquely probe the native cellular environment for macromolecular structures. Tomograms feature complex data with densities of diverse, densely crowded macromolecular complexes, low signal-to-noise, and artifacts such as the missing wedge effect. Post-processing of this data generally involves isolating regions or particles of interest from tomograms, organizing them into related groups, and rendering final structures through subtomogram averaging. Template-matching and reference-based structure determination are popular analysis methods but are vulnerable to biases and can often require significant user input. Most importantly, these approaches cannot identify novel complexes that reside within the imaged cellular environment. To reliably extract and resolve structures of interest, efficient and unbiased approaches are therefore of great value. This review highlights notable computational software and discusses how they contribute to making automated structural pattern discovery a possibility. Perspectives emphasizing the importance of features for user-friendliness and accessibility are also presented.

Volume Segmentation and Analysis of Biological Materials Using SuRVoS (Super-region Volume Segmentation) Workbench

Segmentation is the process of isolating specific regions or objects within an imaged volume, so that further study can be undertaken on these areas of interest. When considering the analysis of complex biological systems, the segmentation of three-dimensional image data is a time consuming and labor intensive step. With the increased availability of many imaging modalities and with automated data collection schemes, this poses an increased challenge for the modern experimental biologist to move from data to knowledge. This publication describes the use of SuRVoS Workbench, a program designed to address these issues by providing methods to semi-automatically segment complex biological volumetric data. Three datasets of differing magnification and imaging modalities are presented here, each highlighting different strategies of segmenting with SuRVoS. Phase contrast X-ray tomography (microCT) of the fruiting body of a plant is used to demonstrate segmentation using model training, cryo electron tomography (cryoET) of human platelets is used to demonstrate segmentation using super- and megavoxels, and cryo soft X-ray tomography (cryoSXT) of a mammalian cell line is used to demonstrate the label splitting tools. Strategies and parameters for each datatype are also presented. By blending a selection of semi-automatic processes into a single interactive tool, SuRVoS provides several benefits. Overall time to segment volumetric data is reduced by a factor of five when compared to manual segmentation, a mainstay in many image processing fields. This is a significant savings when full manual segmentation can take weeks of effort. Additionally, subjectivity is addressed through the use of computationally identified boundaries, and splitting complex collections of objects by their calculated properties rather than on a case-by-case basis.

Hierarchical detection and analysis of macromolecular complexes in cryo-electron tomograms using Pyto software

Molecular complexes, arguably the basic units carrying cellular function, can be visualized directly in their native environment by cryo-electron tomography. Here we describe a procedure for the detection of small, pleomorphic membrane-bound molecular complexes in cryo-tomograms by a hierarchical connectivity segmentation. Validation on phantom and real data showed above 90% true positive rates. This segmentation procedure is implemented in the Pyto software package, together with methods for quantitative characterization and classification of complexes detected by our segmentation procedure and for statistical analysis between experimental conditions. Therefore, the methods presented provide a means for the detection and quantitative interpretation of structures captured in cryo-electron tomograms, as well as for the elucidation of their cellular function.

Quantifying Variability of Manual Annotation in Cryo-Electron Tomograms

Although acknowledged to be variable and subjective, manual annotation of cryo-electron tomography data is commonly used to answer structural questions and to create a "ground truth" for evaluation of automated segmentation algorithms. Validation of such annotation is lacking, but is critical for understanding the reproducibility of manual annotations. Here, we used voxel-based similarity scores for a variety of specimens, ranging in complexity and segmented by several annotators, to quantify the variation among their annotations. In addition, we have identified procedures for merging annotations to reduce variability, thereby increasing the reliability of manual annotation. Based on our analyses, we find that it is necessary to combine multiple manual annotations to increase the confidence level for answering structural questions. We also make recommendations to guide algorithm development for automated annotation of features of interest.

Accurate membrane tracing in three-dimensional reconstructions from electron cryotomography data

The connection between the extracellular matrix and the cell is of major importance for mechanotransduction and mechanobiology. Electron cryo-tomography, in principle, enables better than nanometer-resolution analysis of these connections, but restrictions of data collection geometry hamper the accurate extraction of the ventral membrane location from these tomograms, an essential prerequisite for the analysis. Here, we introduce a novel membrane tracing strategy that enables ventral membrane extraction at high fidelity and extraordinary accuracy. The approach is based on detecting the boundary between the inside and the outside of the cell rather than trying to explicitly trace the membrane. Simulation studies show that over 99% of the membrane can be correctly modeled using this principle and the excellent match of visually identifiable membrane stretches with the extracted boundary of experimental data indicates that the accuracy is comparable for actual data.

Automatic segmentation of mitochondria in EM data using pairwise affinity factorization and graph-based contour searching

In this paper, we investigate the segmentation of closed contours in subcellular data using a framework that primarily combines the pairwise affinity grouping principles with a graph partitioning contour searching approach. One salient problem that precluded the application of these methods to large scale segmentation problems is the onerous computational complexity required to generate comprehensive representations that include all pairwise relationships between all pixels in the input data. To compensate for this problem, a practical solution is to reduce the complexity of the input data by applying an over-segmentation technique prior to the application of the computationally demanding strands of the segmentation process. This approach opens the opportunity to build specific shape and intensity models that can be successfully employed to extract the salient structures in the input image which are further processed to identify the cycles in an undirected graph. The proposed framework has been applied to the segmentation of mitochondria membranes in electron microscopy data which are characterized by low contrast and low signal-to-noise ratio. The algorithm has been quantitatively evaluated using two datasets where the segmentation results have been compared with the corresponding manual annotations. The performance of the proposed algorithm has been measured using standard metrics, such as precision and recall, and the experimental results indicate a high level of segmentation accuracy.

Local regularization of tilt projections reduces artifacts in electron tomography

Electron tomography produces very high resolution 3D image volumes useful for investigating the structure and function of cellular components. Unfortunately, unavoidable discontinuities and physical constraints in the acquisition geometry lead to a range of artifacts that can affect the reconstructed image. In particular, highly electron dense regions, such as gold nanoparticles, can hide proximal biological structures and degrade the overall quality of the reconstructed tomograms. In this work we introduce a pre-reconstruction non-conservative non-linear isotropic diffusion (NID) filter that automatically identifies and reduces local irregularities in the tilt projections. We illustrate the improvement in quality obtained using this approach for reconstructed tomograms generated from samples of malaria parasite-infected red blood cells. A quantitative and qualitative evaluation for our approach on both simulated and real data is provided.

A novel framework for segmentation of secretory granules in electron micrographs

It is still a standard practice for biologists to manually analyze transmission electron microscopy images. This is not only time consuming but also not reproducible and prone to induce subjective bias. For large-scale studies of insulin granules inside beta cells of the islet of Langerhans, an automated method for analysis is essential. Due to the complex structure of the images, standard microscopy segmentation techniques cannot be applied. We present a new approach to segment and measure transmission electron microscopy images of insulin granule cores and membranes from beta cells of rat islets of Langerhans. The algorithm is separated into two broad components, core segmentation and membrane segmentation. Core segmentation proceeds through three steps: pre-segmentation using a novel level-set active contour, morphological cleaning and a refining segmentation on each granule using a novel dual level-set active contour. Membrane segmentation is achieved in four steps: morphological cleaning, membrane sampling and scaling, vector field convolution for gap filling and membrane verification using a novel convergence filter. We show results from our algorithm alongside popular microscopy segmentation methods; the advantages of our method are demonstrated. Our algorithm is validated by comparing automated results to a manually defined ground truth. When the number of granules detected is compared to the number of granules in the ground truth a precision of 91% and recall of 87% is observed. The average granule areas differ by 13.35% and 6.08% for core and membranes respectively, when compared to the average areas of the ground truth. These results compare favorably to previously published data.

Learning context cues for synapse segmentation

We present a new approach for the automated segmentation of synapses in image stacks acquired by electron microscopy (EM) that relies on image features specifically designed to take spatial context into account. These features are used to train a classifier that can effectively learn cues such as the presence of a nearby post-synaptic region. As a result, our algorithm successfully distinguishes synapses from the numerous other organelles that appear within an EM volume, including those whose local textural properties are relatively similar. Furthermore, as a by-product of the segmentation, our method flawlessly determines synaptic orientation, a crucial element in the interpretation of brain circuits. We evaluate our approach on three different datasets, compare it against the state-of-the-art in synapse segmentation and demonstrate our ability to reliably collect shape, density, and orientation statistics over hundreds of synapses.

Active contour based segmentation for insulin granule cores in electron micrographs of beta islet cells

Transmission electron microscopy images of beta islet cells contain many complex structures, making it difficult to accurately segment insulin granule cores. Quantification of sub cellular structures will allow biologists to better understand cellular mechanics. Two novel, level set active contour models are presented in this paper. The first utilizes a shape regularizer to reduce oversegmentation. The second contribution is a dual active contour, which achieves accurate core segmentations. The segmentation algorithm proceeds through three stages: an initial rough segmentation using the first contribution, cleaning using morphological techniques and a refining step using the proposed dual active contour. Our method is validated on a set of manually defined ground truths.

Computerized detection and segmentation of mitochondria on electron microscope images

Mitochondrial function plays an important role in the regulation of cellular life and death, including disease states. Disturbance in mitochondrial function and distribution can be accompanied by significant morphological alterations. Electron microscopy tomography (EMT) is a powerful technique to study the 3D structure of mitochondria, but the automatic detection and segmentation of mitochondria in EMT volumes has been challenging due to the presence of subcellular structures and imaging artifacts. Therefore, the interpretation, measurement and analysis of mitochondrial distribution and features have been time consuming, and development of specialized software tools is very important for high-throughput analyses needed to expedite the myriad studies on cellular events. Typically, mitochondrial EMT volumes are segmented manually using special software tools. Automatic contour extraction on large images with multiple mitochondria and many other subcellular structures is still an unaddressed problem. The purpose of this work is to develop computer algorithms to detect and segment both fully and partially seen mitochondria on electron microscopy images. The detection method relies on mitochondria's approximately elliptical shape and double membrane boundary. Initial detection results are first refined using active contours. Then, our seed point selection method automatically selects reliable seed points along the contour, and segmentation is finalized by automatically incorporating a live-wire graph search algorithm between these seed points. In our evaluations on four images containing multiple mitochondria, 52 ellipses are detected among which 42 are true and 10 are false detections. After false ellipses are eliminated manually, 14 out of 15 fully seen mitochondria and 4 out of 7 partially seen mitochondria are successfully detected. When compared with the segmentation of a trained reader, 91% Dice similarity coefficient was achieved with an average 4.9 nm boundary error.

Supervoxel-based segmentation of mitochondria in em image stacks with learned shape features

It is becoming increasingly clear that mitochondria play an important role in neural function. Recent studies show mitochondrial morphology to be crucial to cellular physiology and synaptic function and a link between mitochondrial defects and neuro-degenerative diseases is strongly suspected. Electron microscopy (EM), with its very high resolution in all three directions, is one of the key tools to look more closely into these issues but the huge amounts of data it produces make automated analysis necessary. State-of-the-art computer vision algorithms designed to operate on natural 2-D images tend to perform poorly when applied to EM data for a number of reasons. First, the sheer size of a typical EM volume renders most modern segmentation schemes intractable. Furthermore, most approaches ignore important shape cues, relying only on local statistics that easily become confused when confronted with noise and textures inherent in the data. Finally, the conventional assumption that strong image gradients always correspond to object boundaries is violated by the clutter of distracting membranes. In this work, we propose an automated graph partitioning scheme that addresses these issues. It reduces the computational complexity by operating on supervoxels instead of voxels, incorporates shape features capable of describing the 3-D shape of the target objects, and learns to recognize the distinctive appearance of true boundaries. Our experiments demonstrate that our approach is able to segment mitochondria at a performance level close to that of a human annotator, and outperforms a state-of-the-art 3-D segmentation technique.

Methods for segmentation and interpretation of electron tomographic reconstructions

Electron tomography has become a powerful tool for revealing the molecular architecture of biological cells and tissues. In principle, electron tomography can provide high-resolution mapping of entire proteomes. The achievable resolution (3-8 nm) is capable of bridging the gap between live-cell imaging and atomic resolution structures. However, the relevant information is not readily accessible from the data and needs to be identified, extracted, and processed before it can be used. Because electron tomography imaging and image acquisition technologies have enjoyed major advances in the last few years and continue to increase data throughput, the need for approaches that allow automatic and objective interpretation of electron tomograms becomes more and more urgent. This chapter provides an overview of the state of the art in this field and attempts to identify the major bottlenecks that prevent approaches for interpreting electron tomography data to develop their full potential.

Chapter 2 Correlated light and electron microscopy/electron tomography of mitochondria in situ

Three-dimensional light microscopy and three-dimensional electron microscopy (electron tomography) separately provide very powerful tools to study cellular structure and physiology, including the structure and physiology of mitochondria. Fluorescence microscopy allows one to study processes in live cells with specific labels and stains that follow the movement of labeled proteins and changes within cellular compartments but does not have sufficient resolution to define the ultrastructure of intracellular organelles such as mitochondria. Electron microscopy and electron tomography provide the highest resolution currently available to study mitochondrial ultrastructure but cannot follow processes in living cells. We describe the combination of these two techniques in which fluorescence confocal microscopy is used to study structural and physiologic changes in mitochondria within apoptotic HeLa cells to define the apoptotic timeframe. Cells can then be selected at various stages of the apoptotic timeframe for examination at higher resolution by electron microscopy and electron tomography. This is a form of "virtual" 4-dimensional electron microscopy that has revealed interesting structural changes in the mitochondria of HeLa cells during apoptosis. The same techniques can be applied, with modification, to study other dynamic processes within cells in other experimental contexts.