Fast segmentation of stained nuclei in terabyte-scale, time resolved 3D microscopy image stacks

HFM-Tracker: a cell tracking algorithm based on hybrid feature matching

Cell migration is known to be a fundamental biological process, playing an essential role in development, homeostasis, and diseases. This paper introduces a cell tracking algorithm named HFM-Tracker (Hybrid Feature Matching Tracker) that automatically identifies cell migration behaviours in consecutive images. It combines Contour Attention (CA) and Adaptive Confusion Matrix (ACM) modules to accurately capture cell contours in each image and track the dynamic behaviors of migrating cells in the field of view. Cells are firstly located and identified via the CA module-based cell detection network, and then associated and tracked via a cell tracking algorithm employing a hybrid feature-matching strategy. This proposed HFM-Tracker exhibits superiorities in cell detection and tracking, achieving 75% in MOTA (Multiple Object Tracking Accuracy) and 65% in IDF1 (ID F1 score). It provides quantitative analysis of the cell morphology and migration features, which could further help in understanding the complicated and diverse cell migration processes.

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models.

Topology-based fluorescence image analysis for automated cell identification and segmentation

Cell segmentation refers to the body of techniques used to identify cells in images and extract biologically relevant information from them; however, manual segmentation is laborious and subjective. We present Topological Boundary Line Estimation using Recurrence Of Neighbouring Emissions (TOBLERONE), a topological image analysis tool which identifies persistent homological image features as opposed to the geometric analysis commonly employed. We demonstrate that topological data analysis can provide accurate segmentation of arbitrarily-shaped cells, offering a means for automatic and objective data extraction. One cellular feature of particular interest in biology is the plasma membrane, which has been shown to present varying degrees of lipid packing, or membrane order, depending on the function and morphology of the cell type. With the use of environmentally-sensitive dyes, images derived from confocal microscopy can be used to quantify the degree of membrane order. We demonstrate that TOBLERONE is capable of automating this task.

LiveCellMiner: A new tool to analyze mitotic progression

Live-cell imaging has become state of the art to accurately identify the nature of mitotic and cell cycle defects. Low- and high-throughput microscopy setups have yield huge data amounts of cells recorded in different experimental and pathological conditions. Tailored semi-automated and automated image analysis approaches allow the analysis of high-content screening data sets, saving time and avoiding bias. However, they were mostly designed for very specific experimental setups, which restricts their flexibility and usability. The general need for dedicated experiment-specific user-annotated training sets and experiment-specific user-defined segmentation parameters remains a major bottleneck for fully automating the analysis process. In this work we present LiveCellMiner, a highly flexible open-source software tool to automatically extract, analyze and visualize both aggregated and time-resolved image features with potential biological relevance. The software tool allows analysis across high-content data sets obtained in different platforms, in a quantitative and unbiased manner. As proof of principle application, we analyze here the dynamic chromatin and tubulin cytoskeleton features in human cells passing through mitosis highlighting the versatile and flexible potential of this tool set.

Using positional information to provide context for biological image analysis with MorphoGraphX 2.0

Positional information is a central concept in developmental biology. In developing organs, positional information can be idealized as a local coordinate system that arises from morphogen gradients controlled by organizers at key locations. This offers a plausible mechanism for the integration of the molecular networks operating in individual cells into the spatially coordinated multicellular responses necessary for the organization of emergent forms. Understanding how positional cues guide morphogenesis requires the quantification of gene expression and growth dynamics in the context of their underlying coordinate systems. Here, we present recent advances in the MorphoGraphX software (Barbier de Reuille et al., 2015⁠) that implement a generalized framework to annotate developing organs with local coordinate systems. These coordinate systems introduce an organ-centric spatial context to microscopy data, allowing gene expression and growth to be quantified and compared in the context of the positional information thought to control them.

3D fluorescence microscopy data synthesis for segmentation and benchmarking

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.

Ventral striatal islands of Calleja neurons control grooming in mice

The striatum comprises multiple subdivisions and neural circuits that differentially control motor output. The islands of Calleja (IC) contain clusters of densely packed granule cells situated in the ventral striatum, predominantly in the olfactory tubercle (OT). Characterized by expression of the D3 dopamine receptor, the IC are evolutionally conserved, but have undefined functions. Here, we show that optogenetic activation of OT D3 neurons robustly initiates self-grooming in mice while suppressing other ongoing behaviors. Conversely, optogenetic inhibition of these neurons halts ongoing grooming, and genetic ablation reduces spontaneous grooming. Furthermore, OT D3 neurons show increased activity before and during grooming and influence local striatal output via synaptic connections with neighboring OT neurons (primarily spiny projection neurons), whose firing rates display grooming-related modulation. Our study uncovers a new role of the ventral striatum's IC in regulating motor output and has important implications for the neural control of grooming.

hiPSC-Derived Schwann Cells Influence Myogenic Differentiation in Neuromuscular Cocultures

Motoneurons, skeletal muscle fibers, and Schwann cells form synapses, termed neuromuscular junctions (NMJs). These control voluntary body movement and are affected in numerous neuromuscular diseases. Therefore, a variety of NMJ in vitro models have been explored to enable mechanistic and pharmacological studies. So far, selective integration of Schwann cells in these models has been hampered, due to technical limitations. Here we present robust protocols for derivation of Schwann cells from human induced pluripotent stem cells (hiPSC) and their coculture with hiPSC-derived motoneurons and C2C12 muscle cells. Upon differentiation with tuned BMP signaling, Schwann cells expressed marker proteins, S100b, Gap43, vimentin, and myelin protein zero. Furthermore, they displayed typical spindle-shaped morphologies with long processes, which often aligned with motoneuron axons. Inclusion of Schwann cells in coculture experiments with hiPSC-derived motoneurons and C2C12 myoblasts enhanced myotube growth and affected size and number of acetylcholine receptor plaques on myotubes. Altogether, these data argue for the availability of a consistent differentiation protocol for Schwann cells and their amenability for functional integration into neuromuscular in vitro models, fostering future studies of neuromuscular mechanisms and disease.

Application of 3D MAPs pipeline identifies the morphological sequence chondrocytes undergo and the regulatory role of GDF5 in this process

The activity of epiphyseal growth plates, which drives long bone elongation, depends on extensive changes in chondrocyte size and shape during differentiation. Here, we develop a pipeline called 3D Morphometric Analysis for Phenotypic significance (3D MAPs), which combines light-sheet microscopy, segmentation algorithms and 3D morphometric analysis to characterize morphogenetic cellular behaviors while maintaining the spatial context of the growth plate. Using 3D MAPs, we create a 3D image database of hundreds of thousands of chondrocytes. Analysis reveals broad repertoire of morphological changes, growth strategies and cell organizations during differentiation. Moreover, identifying a reduction in Smad 1/5/9 activity together with multiple abnormalities in cell growth, shape and organization provides an explanation for the shortening of Gdf5 KO tibias. Overall, our findings provide insight into the morphological sequence that chondrocytes undergo during differentiation and highlight the ability of 3D MAPs to uncover cellular mechanisms that may regulate this process.

Automatic wavelet-based 3D nuclei segmentation and analysis for multicellular embryo quantification

Identification of individual cells in tissues, organs, and in various developing systems is a well-studied problem because it is an essential part of objectively analyzing quantitative images in numerous biological contexts. We developed a size-dependent wavelet-based segmentation method that provides robust segmentation without any preprocessing, filtering or fine-tuning steps, and is robust to the signal-to-noise ratio. The wavelet-based method achieves robust segmentation results with respect to True Positive rate, Precision, and segmentation accuracy compared with other commonly used methods. We applied the segmentation program to zebrafish embryonic development IN TOTO for nuclei segmentation, image registration, and nuclei shape analysis. These new approaches to segmentation provide a means to carry out quantitative patterning analysis with single-cell precision throughout three dimensional tissues and embryos and they have a high tolerance for non-uniform and noisy image data sets.

Digital postprocessing and image segmentation for objective analysis of colorimetric reactions

Recently, there has been an explosion of scientific literature describing the use of colorimetry for monitoring the progression or the endpoint result of colorimetric reactions. The availability of inexpensive imaging technology (e.g., scanners, Raspberry Pi, smartphones and other sub-$50 digital cameras) has lowered the barrier to accessing cost-efficient, objective detection methodologies. However, to exploit these imaging devices as low-cost colorimetric detectors, it is paramount that they interface with flexible software that is capable of image segmentation and probing a variety of color spaces (RGB, HSB, Y'UV, L*a*b*, etc.). Development of tailor-made software (e.g., smartphone applications) for advanced image analysis requires complex, custom-written processing algorithms, advanced computer programming knowledge and/or expertise in physics, mathematics, pattern recognition and computer vision and learning. Freeware programs, such as ImageJ, offer an alternative, affordable path to robust image analysis. Here we describe a protocol that uses the ImageJ program to process images of colorimetric experiments. In practice, this protocol consists of three distinct workflow options. This protocol is accessible to uninitiated users with little experience in image processing or color science and does not require fluorescence signals, expensive imaging equipment or custom-written algorithms. We anticipate that total analysis time per region of interest is ~6 min for new users and <3 min for experienced users, although initial color threshold determination might take longer.

Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy

The accurate segmentation and tracking of cells in microscopy image sequences is an important task in biomedical research, e.g., for studying the development of tissues, organs or entire organisms. However, the segmentation of touching cells in images with a low signal-to-noise-ratio is still a challenging problem. In this paper, we present a method for the segmentation of touching cells in microscopy images. By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process. Furthermore, this representation is notably robust to annotation errors and shows promising results for the segmentation of microscopy images containing in the training data underrepresented or not included cell types. For the prediction of the proposed neighbor distances, an adapted U-Net convolutional neural network (CNN) with two decoder paths is used. In addition, we adapt a graph-based cell tracking algorithm to evaluate our proposed method on the task of cell tracking. The adapted tracking algorithm includes a movement estimation in the cost function to re-link tracks with missing segmentation masks over a short sequence of frames. Our combined tracking by detection method has proven its potential in the IEEE ISBI 2020 Cell Tracking Challenge (http://celltrackingchallenge.net/) where we achieved as team KIT-Sch-GE multiple top three rankings including two top performances using a single segmentation model for the diverse data sets.

BeadNet: deep learning-based bead detection and counting in low-resolution microscopy images

MOTIVATION

An automated counting of beads is required for many high-throughput experiments such as studying mimicked bacterial invasion processes. However, state-of-the-art algorithms under- or overestimate the number of beads in low-resolution images. In addition, expert knowledge is needed to adjust parameters.

RESULTS

In combination with our image labeling tool, BeadNet enables biologists to easily annotate and process their data reducing the expertise required in many existing image analysis pipelines. BeadNet outperforms state-of-the-art-algorithms in terms of missing, added and total amount of beads.

AVAILABILITY AND IMPLEMENTATION

BeadNet (software, code and dataset) is available at https://bitbucket.org/t_scherr/beadnet. The image labeling tool is available at https://bitbucket.org/abartschat/imagelabelingtool.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

MondoA regulates gene expression in cholesterol biosynthesis-associated pathways required for zebrafish epiboly

The glucose-sensing Mondo pathway regulates expression of metabolic genes in mammals. Here, we characterized its function in the zebrafish and revealed an unexpected role of this pathway in vertebrate embryonic development. We showed that knockdown of mondoa impaired the early morphogenetic movement of epiboly in zebrafish embryos and caused microtubule defects. Expression of genes in the terpenoid backbone and sterol biosynthesis pathways upstream of pregnenolone synthesis was coordinately downregulated in these embryos, including the most downregulated gene nsdhl. Loss of Nsdhl function likewise impaired epiboly, similar to MondoA loss of function. Both epiboly and microtubule defects were partially restored by pregnenolone treatment. Maternal-zygotic mutants of mondoa showed perturbed epiboly with low penetrance and compensatory changes in the expression of terpenoid/sterol/steroid metabolism genes. Collectively, our results show a novel role for MondoA in the regulation of early vertebrate development, connecting glucose, cholesterol and steroid hormone metabolism with early embryonic cell movements.

Software tools for 3D nuclei segmentation and quantitative analysis in multicellular aggregates

Today, we are fully immersed into the era of 3D biology. It has been extensively demonstrated that 3D models: (a) better mimic the physiology of human tissues; (b) can effectively replace animal models; (c) often provide more reliable results than 2D ones. Accordingly, anti-cancer drug screenings and toxicology studies based on multicellular 3D biological models, the so-called "-oids" (e.g. spheroids, tumoroids, organoids), are blooming in the literature. However, the complex nature of these systems limit the manual quantitative analyses of single cells' behaviour in the culture. Accordingly, the demand for advanced software tools that are able to perform phenotypic analysis is fundamental. In this work, we describe the freely accessible tools that are currently available for biologists and researchers interested in analysing the effects of drugs/treatments on 3D multicellular -oids at a single-cell resolution level. In addition, using publicly available nuclear stained datasets we quantitatively compare the segmentation performance of 9 specific tools.

CytoCensus, mapping cell identity and division in tissues and organs using machine learning

A major challenge in cell and developmental biology is the automated identification and quantitation of cells in complex multilayered tissues. We developed CytoCensus: an easily deployed implementation of supervised machine learning that extends convenient 2D 'point-and-click' user training to 3D detection of cells in challenging datasets with ill-defined cell boundaries. In tests on such datasets, CytoCensus outperforms other freely available image analysis software in accuracy and speed of cell detection. We used CytoCensus to count stem cells and their progeny, and to quantify individual cell divisions from time-lapse movies of explanted Drosophila larval brains, comparing wild-type and mutant phenotypes. We further illustrate the general utility and future potential of CytoCensus by analysing the 3D organisation of multiple cell classes in Zebrafish retinal organoids and cell distributions in mouse embryos. CytoCensus opens the possibility of straightforward and robust automated analysis of developmental phenotypes in complex tissues.

DeepSynth: Three-dimensional nuclear segmentation of biological images using neural networks trained with synthetic data

The scale of biological microscopy has increased dramatically over the past ten years, with the development of new modalities supporting collection of high-resolution fluorescence image volumes spanning hundreds of microns if not millimeters. The size and complexity of these volumes is such that quantitative analysis requires automated methods of image processing to identify and characterize individual cells. For many workflows, this process starts with segmentation of nuclei that, due to their ubiquity, ease-of-labeling and relatively simple structure, make them appealing targets for automated detection of individual cells. However, in the context of large, three-dimensional image volumes, nuclei present many challenges to automated segmentation, such that conventional approaches are seldom effective and/or robust. Techniques based upon deep-learning have shown great promise, but enthusiasm for applying these techniques is tempered by the need to generate training data, an arduous task, particularly in three dimensions. Here we present results of a new technique of nuclear segmentation using neural networks trained on synthetic data. Comparisons with results obtained using commonly-used image processing packages demonstrate that DeepSynth provides the superior results associated with deep-learning techniques without the need for manual annotation.

Ultra-thin fluorocarbon foils optimise multiscale imaging of three-dimensional native and optically cleared specimens

In three-dimensional light microscopy, the heterogeneity of the optical density in a specimen ultimately limits the achievable penetration depth and hence the three-dimensional resolution. The most direct approach to reduce aberrations, improve the contrast and achieve an optimal resolution is to minimise the impact of changes of the refractive index along an optical path. Many implementations of light sheet fluorescence microscopy operate with a large chamber filled with an aqueous immersion medium and a further inner container with the specimen embedded in a possibly entirely different non-aqueous medium. In order to minimise the impact of the latter on the optical quality of the images, we use multi-facetted cuvettes fabricated from vacuum-formed ultra-thin fluorocarbon (FEP) foils. The ultra-thin FEP-foil cuvettes have a wall thickness of about 10–12 µm. They are impermeable to liquids, but not to gases, inert, durable, mechanically stable and flexible. Importantly, the usually fragile specimen can remain in the same cuvette from seeding to fixation, clearing and observation, without the need to remove or remount it during any of these steps. We confirm the improved imaging performance of ultra-thin FEP-foil cuvettes with excellent quality images of whole organs such us mouse oocytes, of thick tissue sections from mouse brain and kidney as well as of dense pancreas and liver organoid clusters. Our ultra-thin FEP-foil cuvettes outperform many other sample-mounting techniques in terms of a full separation of the specimen from the immersion medium, compatibility with aqueous and organic clearing media, quick specimen mounting without hydrogel embedding and their applicability for multiple-view imaging and automated image segmentation. Additionally, we show that ultra-thin FEP foil cuvettes are suitable for seeding and growing organoids over a time period of at least ten days. The new cuvettes allow the fixation and staining of specimens inside the holder, preserving the delicate morphology of e.g. fragile, mono-layered three-dimensional organoids.

Quantitative Spatial Analysis of Metabolic Heterogeneity Across in vivo and in vitro Tumor Models

Metabolic preferences of tumor cells vary within a single tumor, contributing to tumor heterogeneity, drug resistance, and patient relapse. However, the relationship between tumor treatment response and metabolically distinct tumor cell populations is not well-understood. Here, a quantitative approach was developed to characterize spatial patterns of metabolic heterogeneity in tumor cell populations within in vivo xenografts and 3D in vitro cultures (i.e., organoids) of head and neck cancer. Label-free images of cell metabolism were acquired using two-photon fluorescence lifetime microscopy of the metabolic co-enzymes NAD(P)H and FAD. Previous studies have shown that NAD(P)H mean fluorescence lifetimes can identify metabolically distinct cells with varying drug response. Thus, density-based clustering of the NAD(P)H mean fluorescence lifetime was used to identify metabolic sub-populations of cells, then assessed in control, cetuximab-, cisplatin-, and combination-treated xenografts 13 days post-treatment and organoids 24 h post-treatment. Proximity analysis of these metabolically distinct cells was designed to quantify differences in spatial patterns between treatment groups and between xenografts and organoids. Multivariate spatial autocorrelation and principal components analyses of all autofluorescence intensity and lifetime variables were developed to further improve separation between cell sub-populations. Spatial principal components analysis and Z-score calculations of autofluorescence and spatial distribution variables also visualized differences between models. This analysis captures spatial distributions of tumor cell sub-populations influenced by treatment conditions and model-specific environments. Overall, this novel spatial analysis could provide new insights into tumor growth, treatment resistance, and more effective drug treatments across a range of microscopic imaging modalities (e.g., immunofluorescence, imaging mass spectrometry).

CytoPacq: a web-interface for simulating multi-dimensional cell imaging

MOTIVATION

Objective assessment of bioimage analysis methods is an essential step towards understanding their robustness and parameter sensitivity, calling for the availability of heterogeneous bioimage datasets accompanied by their reference annotations. Because manual annotations are known to be arduous, highly subjective and barely reproducible, numerous simulators have emerged over past decades, generating synthetic bioimage datasets complemented with inherent reference annotations. However, the installation and configuration of these tools generally constitutes a barrier to their widespread use.

RESULTS

We present a modern, modular web-interface, CytoPacq, to facilitate the generation of synthetic benchmark datasets relevant for multi-dimensional cell imaging. CytoPacq poses a user-friendly graphical interface with contextual tooltips and currently allows a comfortable access to various cell simulation systems of fluorescence microscopy, which have already been recognized and used by the scientific community, in a straightforward and self-contained form.

AVAILABILITY AND IMPLEMENTATION

CytoPacq is a publicly available online service running at https://cbia.fi.muni.cz/simulator. More information about it as well as examples of generated bioimage datasets are available directly through the web-interface.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Light-Sheet Microscopy and Its Potential for Understanding Developmental Processes

The ability to visualize and quantitatively measure dynamic biological processes in vivo and at high spatiotemporal resolution is of fundamental importance to experimental investigations in developmental biology. Light-sheet microscopy is particularly well suited to providing such data, since it offers exceptionally high imaging speed and good spatial resolution while minimizing light-induced damage to the specimen. We review core principles and recent advances in light-sheet microscopy, with a focus on concepts and implementations relevant for applications in developmental biology. We discuss how light-sheet microcopy has helped advance our understanding of developmental processes from single-molecule to whole-organism studies, assess the potential for synergies with other state-of-the-art technologies, and introduce methods for computational image and data analysis. Finally, we explore the future trajectory of light-sheet microscopy, discuss key efforts to disseminate new light-sheet technology, and identify exciting opportunities for further advances.

Three-Dimensional Segmentation and Reconstruction of Neuronal Nuclei in Confocal Microscopic Images

The detailed architectural examination of the neuronal nuclei in any brain region, using confocal microscopy, requires quantification of fluorescent signals in three-dimensional stacks of confocal images. An essential prerequisite to any quantification is the segmentation of the nuclei which are typically tightly packed in the tissue, the extreme being the hippocampal dentate gyrus (DG), in which nuclei frequently appear to overlap due to limitations in microscope resolution. Segmentation in DG is a challenging task due to the presence of a significant amount of image artifacts and densely packed nuclei. Accordingly, we established an algorithm based on continuous boundary tracing criterion aiming to reconstruct the nucleus surface and to separate the adjacent nuclei. The presented algorithm neither uses a pre-built nucleus model, nor performs image thresholding, which makes it robust against variations in image intensity and poor contrast. Further, the reconstructed surface is used to study morphology and spatial arrangement of the nuclear interior. The presented method is generally dedicated to segmentation of crowded, overlapping objects in 3D space. In particular, it allows us to study quantitatively the architecture of the neuronal nucleus using confocal-microscopic approach.

Embryo development stage prediction algorithm for automated time lapse incubators

BACKGROUND AND OBJECTIVE

Time-lapse microscopy has become an important tool for studying the embryo development process. Embryologists can monitor the entire embryo growth process and thus select the best embryos for fertilization. This time and the resource consuming process are among the key factors for success of pregnancies. Tools for automated evaluation of the embryo quality and development stage prediction are developed for improving embryo selection.

METHODS

We present two-classifier vote-based method for embryo image classification. Our classification algorithms have been trained with features extracted using a Convolutional Neural Network (CNN). Prediction of embryo development stage is then completed by comparing confidence of two classifiers. Images are labeled depending on which one receives a larger confidence rating.

RESULTS

The evaluation has been done with imagery of real embryos, taken in the ESCO Time Lapse incubator from four different developing embryos. The results illustrate the most effective combination of two classifiers leading to an increase of prediction accuracy and achievement of overall 97.62% accuracy for a test set classification.

CONCLUSIONS

We have presented an approach for automated prediction of the embryo development stage for microscopy time-lapse incubator image. Our algorithm has extracted high-complexity image feature using CNN. Classification is done by comparing prediction of two classifiers and selecting the label of that classifier, which has a higher confidence value. This combination of two classifiers has allowed us to increase the overall accuracy of CNN from 96.58% by 1.04% up to 97.62%. The best results are achieved when combining the CNN and Discriminant classifiers. Practical implications include improvement of embryo selection process for in vitro fertilization.

A high-throughput imaging and nuclear segmentation analysis protocol for cleared 3D culture models

Imaging and subsequent segmentation analysis in three-dimensional (3D) culture models are complicated by the light scattering that occurs when collecting fluorescent signal through multiple cell and extracellular matrix layers. For 3D cell culture models to be usable for drug discovery, effective and efficient imaging and analysis protocols need to be developed that enable high-throughput data acquisition and quantitative analysis of fluorescent signal. Here we report the first high-throughput protocol for optical clearing of spheroids, fluorescent high-content confocal imaging, 3D nuclear segmentation, and post-segmentation analysis. We demonstrate nuclear segmentation in multiple cell types, with accurate identification of fluorescently-labeled subpopulations, and develop a metric to assess the ability of clearing to improve nuclear segmentation deep within the tissue. Ultimately this analysis pipeline allows for previously unattainable segmentation throughput of 3D culture models due to increased sample clarity and optimized batch-processing analysis.

EmbryoMiner: A new framework for interactive knowledge discovery in large-scale cell tracking data of developing embryos

State-of-the-art light-sheet and confocal microscopes allow recording of entire embryos in 3D and over time (3D+t) for many hours. Fluorescently labeled structures can be segmented and tracked automatically in these terabyte-scale 3D+t images, resulting in thousands of cell migration trajectories that provide detailed insights to large-scale tissue reorganization at the cellular level. Here we present EmbryoMiner, a new interactive open-source framework suitable for in-depth analyses and comparisons of entire embryos, including an extensive set of trajectory features. Starting at the whole-embryo level, the framework can be used to iteratively focus on a region of interest within the embryo, to investigate and test specific trajectory-based hypotheses and to extract quantitative features from the isolated trajectories. Thus, the new framework provides a valuable new way to quantitatively compare corresponding anatomical regions in different embryos that were manually selected based on biological prior knowledge. As a proof of concept, we analyzed 3D+t light-sheet microscopy images of zebrafish embryos, showcasing potential user applications that can be performed using the new framework.

A New Unsupervised Approach for Segmenting and Counting Cells in High-Throughput Microscopy Image Sets

New technological advances in automated microscopy have given rise to large volumes of data, which have made human-based analysis infeasible, heightening the need for automatic systems for high-throughput microscopy applications. In particular, in the field of fluorescence microscopy, automatic tools for image analysis are making an essential contribution in order to increase the statistical power of the cell analysis process. The development of these automatic systems is a difficult task due to both the diversification of the staining patterns and the local variability of the images. In this paper, we present an unsupervised approach for automatic cell segmentation and counting, namely CSC, in high-throughput microscopy images. The segmentation is performed by dividing the whole image into square patches that undergo a gray level clustering followed by an adaptive thresholding. Subsequently, the cell labeling is obtained by detecting the centers of the cells, using both distance transform and curvature analysis, and by applying a region growing process. The advantages of CSC are manifold. The foreground detection process works on gray levels rather than on individual pixels, so it proves to be very efficient. Moreover, the combination of distance transform and curvature analysis makes the counting process very robust to clustered cells. A further strength of the CSC method is the limited number of parameters that must be tuned. Indeed, two different versions of the method have been considered, CSC-7 and CSC-3, depending on the number of parameters to be tuned. The CSC method has been tested on several publicly available image datasets of real and synthetic images. Results in terms of standard metrics and spatially aware measures show that CSC outperforms the current state-of-the-art techniques.

Deciphering tissue morphodynamics using bioimage informatics

In recent years developmental biology has greatly benefited from the latest advances in fluorescence microscopy techniques. Consequently, quantitative and automated analysis of this data is becoming a vital first step in the quest for novel insights into the various aspects of development. Here we present an introductory overview of the various image analysis methods proposed for developmental biology images, with particular attention to openly available software packages. These tools, as well as others to come, are rapidly paving the way towards standardized and reproducible bioimaging studies at the whole-tissue level. Reflecting on these achievements, we discuss the remaining challenges and the future endeavours lying ahead in the post-image analysis era.This article is part of the themed issue 'Systems morphodynamics: understanding the development of tissue hardware'.

NeuroSeg: automated cell detection and segmentation for in vivo two-photon Ca2+ imaging data

Two-photon Ca²⁺ imaging has become a popular approach for monitoring neuronal population activity with cellular or subcellular resolution in vivo. This approach allows for the recording of hundreds to thousands of neurons per animal and thus leads to a large amount of data to be processed. In particular, manually drawing regions of interest is the most time-consuming aspect of data analysis. However, the development of automated image analysis pipelines, which will be essential for dealing with the likely future deluge of imaging data, remains a major challenge. To address this issue, we developed NeuroSeg, an open-source MATLAB program that can facilitate the accurate and efficient segmentation of neurons in two-photon Ca²⁺ imaging data. We proposed an approach using a generalized Laplacian of Gaussian filter to detect cells and weighting-based segmentation to separate individual cells from the background. We tested this approach on an in vivo two-photon Ca²⁺ imaging dataset obtained from mouse cortical neurons with differently sized view fields. We show that this approach exhibits superior performance for cell detection and segmentation compared with the existing published tools. In addition, we integrated the previously reported, activity-based segmentation into our approach and found that this combined method was even more promising. The NeuroSeg software, including source code and graphical user interface, is freely available and will be a useful tool for in vivo brain activity mapping.

Fuzzy-based propagation of prior knowledge to improve large-scale image analysis pipelines

Many automatically analyzable scientific questions are well-posed and a variety of information about expected outcomes is available a priori. Although often neglected, this prior knowledge can be systematically exploited to make automated analysis operations sensitive to a desired phenomenon or to evaluate extracted content with respect to this prior knowledge. For instance, the performance of processing operators can be greatly enhanced by a more focused detection strategy and by direct information about the ambiguity inherent in the extracted data. We present a new concept that increases the result quality awareness of image analysis operators by estimating and distributing the degree of uncertainty involved in their output based on prior knowledge. This allows the use of simple processing operators that are suitable for analyzing large-scale spatiotemporal (3D+t) microscopy images without compromising result quality. On the foundation of fuzzy set theory, we transform available prior knowledge into a mathematical representation and extensively use it to enhance the result quality of various processing operators. These concepts are illustrated on a typical bioimage analysis pipeline comprised of seed point detection, segmentation, multiview fusion and tracking. The functionality of the proposed approach is further validated on a comprehensive simulated 3D+t benchmark data set that mimics embryonic development and on large-scale light-sheet microscopy data of a zebrafish embryo. The general concept introduced in this contribution represents a new approach to efficiently exploit prior knowledge to improve the result quality of image analysis pipelines. The generality of the concept makes it applicable to practically any field with processing strategies that are arranged as linear pipelines. The automated analysis of terabyte-scale microscopy data will especially benefit from sophisticated and efficient algorithms that enable a quantitative and fast readout.

Determining the impact of cell mixing on signaling during development

Cell movement and intercellular signaling occur simultaneously to organize morphogenesis during embryonic development. Cell movement can cause relative positional changes between neighboring cells. When intercellular signals are local such cell mixing may affect signaling, changing the flow of information in developing tissues. Little is known about the effect of cell mixing on intercellular signaling in collective cellular behaviors and methods to quantify its impact are lacking. Here we discuss how to determine the impact of cell mixing on cell signaling drawing an example from vertebrate embryogenesis: the segmentation clock, a collective rhythm of interacting genetic oscillators. We argue that comparing cell mixing and signaling timescales is key to determining the influence of mixing. A signaling timescale can be estimated by combining theoretical models with cell signaling perturbation experiments. A mixing timescale can be obtained by analysis of cell trajectories from live imaging. After comparing cell movement analyses in different experimental settings, we highlight challenges in quantifying cell mixing from embryonic timelapse experiments, especially a reference frame problem due to embryonic motions and shape changes. We propose statistical observables characterizing cell mixing that do not depend on the choice of reference frames. Finally, we consider situations in which both cell mixing and signaling involve multiple timescales, precluding a direct comparison between single characteristic timescales. In such situations, physical models based on observables of cell mixing and signaling can simulate the flow of information in tissues and reveal the impact of observed cell mixing on signaling.

Multiscale image analysis reveals structural heterogeneity of the cell microenvironment in homotypic spheroids

Three-dimensional multicellular aggregates such as spheroids provide reliable in vitro substitutes for tissues. Quantitative characterization of spheroids at the cellular level is fundamental. We present the first pipeline that provides three-dimensional, high-quality images of intact spheroids at cellular resolution and a comprehensive image analysis that completes traditional image segmentation by algorithms from other fields. The pipeline combines light sheet-based fluorescence microscopy of optically cleared spheroids with automated nuclei segmentation (F score: 0.88) and concepts from graph analysis and computational topology. Incorporating cell graphs and alpha shapes provided more than 30 features of individual nuclei, the cellular neighborhood and the spheroid morphology. The application of our pipeline to a set of breast carcinoma spheroids revealed two concentric layers of different cell density for more than 30,000 cells. The thickness of the outer cell layer depends on a spheroid’s size and varies between 50% and 75% of its radius. In differently-sized spheroids, we detected patches of different cell densities ranging from 5 × 10⁵ to 1 × 10⁶cells/mm³. Since cell density affects cell behavior in tissues, structural heterogeneities need to be incorporated into existing models. Our image analysis pipeline provides a multiscale approach to obtain the relevant data for a system-level understanding of tissue architecture.

Virtual cell imaging: A review on simulation methods employed in image cytometry

The simulations of cells and microscope images thereof have been used to facilitate the development, selection, and validation of image analysis algorithms employed in cytometry as well as for modeling and understanding cell structure and dynamics beyond what is visible in the eyepiece. The simulation approaches vary from simple parametric models of specific cell components-especially shapes of cells and cell nuclei-to learning-based synthesis and multi-stage simulation models for complex scenes that simultaneously visualize multiple object types and incorporate various properties of the imaged objects and laws of image formation. This review covers advances in artificial digital cell generation at scales ranging from particles up to tissue synthesis and microscope image simulation methods, provides examples of the use of simulated images for various purposes ranging from subcellular object detection to cell tracking, and discusses how such simulators have been validated. Finally, the future possibilities and limitations of simulation-based validation are considered. © 2016 International Society for Advancement of Cytometry.

Emerging Imaging and Genomic Tools for Developmental Systems Biology

Animal development is a complex and dynamic process orchestrated by exquisitely timed cell lineage commitment, divisions, migration, and morphological changes at the single-cell level. In the past decade, extensive genetic, stem cell, and genomic studies provided crucial insights into molecular underpinnings and the functional importance of genetic pathways governing various cellular differentiation processes. However, it is still largely unknown how the precise coordination of these pathways is achieved at the whole-organism level and how the highly regulated spatiotemporal choreography of development is established in turn. Here, we discuss the latest technological advances in imaging and single-cell genomics that hold great promise for advancing our understanding of this intricate process. We propose an integrated approach that combines such methods to quantitatively decipher in vivo cellular dynamic behaviors and their underlying molecular mechanisms at the systems level with single-cell, single-molecule resolution.

Object Segmentation and Ground Truth in 3D Embryonic Imaging

Many questions in developmental biology depend on measuring the position and movement of individual cells within developing embryos. Yet, tools that provide this data are often challenged by high cell density and their accuracy is difficult to measure. Here, we present a three-step procedure to address this problem. Step one is a novel segmentation algorithm based on image derivatives that, in combination with selective post-processing, reliably and automatically segments cell nuclei from images of densely packed tissue. Step two is a quantitative validation using synthetic images to ascertain the efficiency of the algorithm with respect to signal-to-noise ratio and object density. Finally, we propose an original method to generate reliable and experimentally faithful ground truth datasets: Sparse-dense dual-labeled embryo chimeras are used to unambiguously measure segmentation errors within experimental data. Together, the three steps outlined here establish a robust, iterative procedure to fine-tune image analysis algorithms and microscopy settings associated with embryonic 3D image data sets.

Real-Time Three-Dimensional Cell Segmentation in Large-Scale Microscopy Data of Developing Embryos

We present the Real-time Accurate Cell-shape Extractor (RACE), a high-throughput image analysis framework for automated three-dimensional cell segmentation in large-scale images. RACE is 55-330 times faster and 2-5 times more accurate than state-of-the-art methods. We demonstrate the generality of RACE by extracting cell-shape information from entire Drosophila, zebrafish, and mouse embryos imaged with confocal and light-sheet microscopes. Using RACE, we automatically reconstructed cellular-resolution tissue anisotropy maps across developing Drosophila embryos and quantified differences in cell-shape dynamics in wild-type and mutant embryos. We furthermore integrated RACE with our framework for automated cell lineaging and performed joint segmentation and cell tracking in entire Drosophila embryos. RACE processed these terabyte-sized datasets on a single computer within 1.4 days. RACE is easy to use, as it requires adjustment of only three parameters, takes full advantage of state-of-the-art multi-core processors and graphics cards, and is available as open-source software for Windows, Linux, and Mac OS.

A Parallel Distributed-Memory Particle Method Enables Acquisition-Rate Segmentation of Large Fluorescence Microscopy Images

Modern fluorescence microscopy modalities, such as light-sheet microscopy, are capable of acquiring large three-dimensional images at high data rate. This creates a bottleneck in computational processing and analysis of the acquired images, as the rate of acquisition outpaces the speed of processing. Moreover, images can be so large that they do not fit the main memory of a single computer. We address both issues by developing a distributed parallel algorithm for segmentation of large fluorescence microscopy images. The method is based on the versatile Discrete Region Competition algorithm, which has previously proven useful in microscopy image segmentation. The present distributed implementation decomposes the input image into smaller sub-images that are distributed across multiple computers. Using network communication, the computers orchestrate the collectively solving of the global segmentation problem. This not only enables segmentation of large images (we test images of up to 10(10) pixels), but also accelerates segmentation to match the time scale of image acquisition. Such acquisition-rate image segmentation is a prerequisite for the smart microscopes of the future and enables online data compression and interactive experiments.

Automatic Nuclei Detection Based on Generalized Laplacian of Gaussian Filters

Efficient and accurate detection of cell nuclei is an important step toward automatic analysis in histopathology. In this work, we present an automatic technique based on generalized Laplacian of Gaussian (gLoG) filter for nuclei detection in digitized histological images. The proposed technique first generates a bank of gLoG kernels with different scales and orientations and then performs convolution between directional gLoG kernels and the candidate image to obtain a set of response maps. The local maxima of response maps are detected and clustered into different groups by mean-shift algorithm based on their geometrical closeness. The point which has the maximum response in each group is finally selected as the nucleus seed. Experimental results on two datasets show that the proposed technique provides a superior performance in nuclei detection compared to existing techniques.

A versatile pipeline for the multi-scale digital reconstruction and quantitative analysis of 3D tissue architecture

A prerequisite for the systems biology analysis of tissues is an accurate digital three-dimensional reconstruction of tissue structure based on images of markers covering multiple scales. Here, we designed a flexible pipeline for the multi-scale reconstruction and quantitative morphological analysis of tissue architecture from microscopy images. Our pipeline includes newly developed algorithms that address specific challenges of thick dense tissue reconstruction. Our implementation allows for a flexible workflow, scalable to high-throughput analysis and applicable to various mammalian tissues. We applied it to the analysis of liver tissue and extracted quantitative parameters of sinusoids, bile canaliculi and cell shapes, recognizing different liver cell types with high accuracy. Using our platform, we uncovered an unexpected zonation pattern of hepatocytes with different size, nuclei and DNA content, thus revealing new features of liver tissue organization. The pipeline also proved effective to analyse lung and kidney tissue, demonstrating its generality and robustness.

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Tracking motile cells in time-lapse series is challenging and is required in many biomedical applications. Cell tracks can be mathematically represented as acyclic oriented graphs. Their vertices describe the spatio-temporal locations of individual cells, whereas the edges represent temporal relationships between them. Such a representation maintains the knowledge of all important cellular events within a captured field of view, such as migration, division, death, and transit through the field of view. The increasing number of cell tracking algorithms calls for comparison of their performance. However, the lack of a standardized cell tracking accuracy measure makes the comparison impracticable. This paper defines and evaluates an accuracy measure for objective and systematic benchmarking of cell tracking algorithms. The measure assumes the existence of a ground-truth reference, and assesses how difficult it is to transform a computed graph into the reference one. The difficulty is measured as a weighted sum of the lowest number of graph operations, such as split, delete, and add a vertex and delete, add, and alter the semantics of an edge, needed to make the graphs identical. The measure behavior is extensively analyzed based on the tracking results provided by the participants of the first Cell Tracking Challenge hosted by the 2013 IEEE International Symposium on Biomedical Imaging. We demonstrate the robustness and stability of the measure against small changes in the choice of weights for diverse cell tracking algorithms and fluorescence microscopy datasets. As the measure penalizes all possible errors in the tracking results and is easy to compute, it may especially help developers and analysts to tune their algorithms according to their needs.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Efficient processing and analysis of large-scale light-sheet microscopy data

Light-sheet microscopy is a powerful method for imaging the development and function of complex biological systems at high spatiotemporal resolution and over long time scales. Such experiments typically generate terabytes of multidimensional image data, and thus they demand efficient computational solutions for data management, processing and analysis. We present protocols and software to tackle these steps, focusing on the imaging-based study of animal development. Our protocols facilitate (i) high-speed lossless data compression and content-based multiview image fusion optimized for multicore CPU architectures, reducing image data size 30-500-fold; (ii) automated large-scale cell tracking and segmentation; and (iii) visualization, editing and annotation of multiterabyte image data and cell-lineage reconstructions with tens of millions of data points. These software modules are open source. They provide high data throughput using a single computer workstation and are readily applicable to a wide spectrum of biological model systems.

XPIWIT--an XML pipeline wrapper for the Insight Toolkit

The Insight Toolkit offers plenty of features for multidimensional image analysis. Current implementations, however, often suffer either from a lack of flexibility due to hard-coded C++ pipelines for a certain task or by slow execution times, e.g. caused by inefficient implementations or multiple read/write operations for separate filter execution. We present an XML-based wrapper application for the Insight Toolkit that combines the performance of a pure C++ implementation with an easy-to-use graphical setup of dynamic image analysis pipelines. Created XML pipelines can be interpreted and executed by XPIWIT in console mode either locally or on large clusters. We successfully applied the software tool for the automated analysis of terabyte-scale, time-resolved 3D image data of zebrafish embryos.

AVAILABILITY AND IMPLEMENTATION

XPIWIT is implemented in C++ using the Insight Toolkit and the Qt SDK. It has been successfully compiled and tested under Windows and Unix-based systems. Software and documentation are distributed under Apache 2.0 license and are publicly available for download at https://bitbucket.org/jstegmaier/xpiwit/downloads/.

CONTACT

johannes.stegmaier@kit.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A Benchmark Data Set to Evaluate the Illumination Robustness of Image Processing Algorithms for Object Segmentation and Classification

Developers of image processing routines rely on benchmark data sets to give qualitative comparisons of new image analysis algorithms and pipelines. Such data sets need to include artifacts in order to occlude and distort the required information to be extracted from an image. Robustness, the quality of an algorithm related to the amount of distortion is often important. However, using available benchmark data sets an evaluation of illumination robustness is difficult or even not possible due to missing ground truth data about object margins and classes and missing information about the distortion. We present a new framework for robustness evaluation. The key aspect is an image benchmark containing 9 object classes and the required ground truth for segmentation and classification. Varying levels of shading and background noise are integrated to distort the data set. To quantify the illumination robustness, we provide measures for image quality, segmentation and classification success and robustness. We set a high value on giving users easy access to the new benchmark, therefore, all routines are provided within a software package, but can as well easily be replaced to emphasize other aspects.

New breast cancer prognostic factors identified by computer-aided image analysis of HE stained histopathology images

Computer-aided image analysis (CAI) can help objectively quantify morphologic features of hematoxylin-eosin (HE) histopathology images and provide potentially useful prognostic information on breast cancer. We performed a CAI workflow on 1,150 HE images from 230 patients with invasive ductal carcinoma (IDC) of the breast. We used a pixel-wise support vector machine classifier for tumor nests (TNs)-stroma segmentation, and a marker-controlled watershed algorithm for nuclei segmentation. 730 morphologic parameters were extracted after segmentation, and 12 parameters identified by Kaplan-Meier analysis were significantly associated with 8-year disease free survival (P < 0.05 for all). Moreover, four image features including TNs feature (HR 1.327, 95%CI [1.001 - 1.759], P = 0.049), TNs cell nuclei feature (HR 0.729, 95%CI [0.537 - 0.989], P = 0.042), TNs cell density (HR 1.625, 95%CI [1.177 - 2.244], P = 0.003), and stromal cell structure feature (HR 1.596, 95%CI [1.142 - 2.229], P = 0.006) were identified by multivariate Cox proportional hazards model to be new independent prognostic factors. The results indicated that CAI can assist the pathologist in extracting prognostic information from HE histopathology images for IDC. The TNs feature, TNs cell nuclei feature, TNs cell density, and stromal cell structure feature could be new prognostic factors.

An ensemble-averaged, cell density-based digital model of zebrafish embryo development derived from light-sheet microscopy data with single-cell resolution

A new era in developmental biology has been ushered in by recent advances in the quantitative imaging of all-cell morphogenesis in living organisms. Here we have developed a light-sheet fluorescence microscopy-based framework with single-cell resolution for identification and characterization of subtle phenotypical changes of millimeter-sized organisms. Such a comparative study requires analyses of entire ensembles to be able to distinguish sample-to-sample variations from definitive phenotypical changes. We present a kinetic digital model of zebrafish embryos up to 16 h of development. The model is based on the precise overlay and averaging of data taken on multiple individuals and describes the cell density and its migration direction at every point in time. Quantitative metrics for multi-sample comparative studies have been introduced to analyze developmental variations within the ensemble. The digital model may serve as a canvas on which the behavior of cellular subpopulations can be studied. As an example, we have investigated cellular rearrangements during germ layer formation at the onset of gastrulation. A comparison of the one-eyed pinhead (oep) mutant with the digital model of the wild-type embryo reveals its abnormal development at the onset of gastrulation, many hours before changes are obvious to the eye.

Graph-regularized 3D shape reconstruction from highly anisotropic and noisy images

Analysis of microscopy images can provide insight into many biological processes. One particularly challenging problem is cellular nuclear segmentation in highly anisotropic and noisy 3D image data. Manually localizing and segmenting each and every cellular nucleus is very time-consuming, which remains a bottleneck in large-scale biological experiments. In this work, we present a tool for automated segmentation of cellular nuclei from 3D fluorescent microscopic data. Our tool is based on state-of-the-art image processing and machine learning techniques and provides a user-friendly graphical user interface. We show that our tool is as accurate as manual annotation and greatly reduces the time for the registration.