Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations

Early coordination of cell migration and cardiac fate determination during mammalian gastrulation

During gastrulation, mesodermal cells derived from distinct regions are destined to acquire specific cardiac fates after undergoing complex migratory movements. Here, we used light-sheet imaging of live mouse embryos between gastrulation and heart tube formation to track mesodermal cells and to reconstruct lineage trees and 3D migration paths for up to five cell divisions. We found independent progenitors emerging at specific times, contributing exclusively to left ventricle/atrioventricular canal (LV/AVC) or atrial myocytes. LV/AVC progenitors differentiated early to form the cardiac crescent, while atrial progenitors later generated the heart tube's Nr2f2+ inflow tract during morphogenesis. We also identified short-lived multipotent progenitors with broad potential, illustrating early developmental plasticity. Descendants of multipotent progenitors displayed greater dispersion and more diverse migratory trajectories within the anterior mesoderm than the progeny of uni-fated progenitors. Progenitors contributing to extraembryonic mesoderm (ExEm) exhibited the fastest and most dispersed migrations. In contrast, those giving rise to endocardial, LV/AVC, and pericardial cells showed a more gradual divergence, with late-stage behavioural shifts: endocardial cells increased in speed, while pericardial cells slowed down in comparison to LV/AVC cells. Together, these data reveal patterns of individual cell directionality and cardiac fate allocation within the seemingly unorganised migratory pattern of mesoderm cells.

4D light sheet imaging, computational reconstruction, and cell tracking in mouse embryos

As light sheet fluorescence microscopy (LSFM) becomes widely available, reconstruction of time-lapse imaging will further our understanding of complex biological processes at cellular resolution. Here, we present a comprehensive workflow for in toto capture, processing, and analysis of multi-view LSFM experiments using the ex vivo mouse embryo as a model system of development. Our protocol describes imaging on a commercial LSFM instrument followed by computational analysis in discrete segments, using open-source software. Quantification of migration and morphodynamics is included. For complete details on the use and execution of this protocol, please refer to Dominguez et al.1.

A model for boundary-driven tissue morphogenesis

Tissue deformations during morphogenesis can be active, driven by internal processes, or passive, resulting from stresses applied at their boundaries. Here, we introduce the Drosophila hindgut primordium as a model for studying boundary-driven tissue morphogenesis. We characterize its deformations and show that its complex shape changes can be a passive consequence of the deformations of the active regions of the embryo that surround it. First, we find an intermediate characteristic triangular shape in the 3D deformations of the hindgut. We construct a minimal model of the hindgut primordium as an elastic ring deformed by active midgut invagination and germ band extension on an ellipsoidal surface, which robustly captures the symmetry-breaking into this triangular shape. We then quantify the 3D kinematics of the tissue by a set of contours and discover that the hindgut deforms in two stages: an initial translation on the curved embryo surface followed by a rapid breaking of shape symmetry. We extend our model to show that the contour kinematics in both stages are consistent with our passive picture. Our results suggest that the role of in-plane deformations during hindgut morphogenesis is to translate the tissue to a region with anisotropic embryonic curvature and show that uniform boundary conditions are sufficient to generate the observed nonuniform shape change. Our work thus provides a possible explanation for the various characteristic shapes of blastopore-equivalents in different organisms and a framework for the mechanical emergence of global morphologies in complex developmental systems.

Segment Anything for Microscopy

Accurate segmentation of objects in microscopy images remains a bottleneck for many researchers despite the number of tools developed for this purpose. Here, we present Segment Anything for Microscopy (μSAM), a tool for segmentation and tracking in multidimensional microscopy data. It is based on Segment Anything, a vision foundation model for image segmentation. We extend it by fine-tuning generalist models for light and electron microscopy that clearly improve segmentation quality for a wide range of imaging conditions. We also implement interactive and automatic segmentation in a napari plugin that can speed up diverse segmentation tasks and provides a unified solution for microscopy annotation across different microscopy modalities. Our work constitutes the application of vision foundation models in microscopy, laying the groundwork for solving image analysis tasks in this domain with a small set of powerful deep learning models.

A multimodal zebrafish developmental atlas reveals the state-transition dynamics of late-vertebrate pluripotent axial progenitors

Elucidating organismal developmental processes requires a comprehensive understanding of cellular lineages in the spatial, temporal, and molecular domains. In this study, we introduce Zebrahub, a dynamic atlas of zebrafish embryonic development that integrates single-cell sequencing time course data with lineage reconstructions facilitated by light-sheet microscopy. This atlas offers high-resolution and in-depth molecular insights into zebrafish development, achieved through the sequencing of individual embryos across ten developmental stages, complemented by reconstructions of cellular trajectories. Zebrahub also incorporates an interactive tool to navigate the complex cellular flows and lineages derived from light-sheet microscopy data, enabling in silico fate-mapping experiments. To demonstrate the versatility of our multimodal resource, we utilize Zebrahub to provide fresh insights into the pluripotency of neuro-mesodermal progenitors (NMPs) and the origins of a joint kidney-hemangioblast progenitor population.

Automated cell lineage reconstruction using label-free 4D microscopy

Patterns of lineal descent play a critical role in the development of metazoan embryos. In eutelic organisms that generate a fixed number of somatic cells, invariance in the topology of their cell lineage provides a powerful opportunity to interrogate developmental events with empirical repeatability across individuals. Studies of embryonic development using the nematode Caenorhabditis elegans have been drivers of discovery. These studies have depended heavily on high-throughput lineage tracing enabled by 4D fluorescence microscopy and robust computer vision pipelines. For a range of applications, computer-aided yet manual lineage tracing using 4D label-free microscopy remains an essential tool. Deep learning approaches to cell detection and tracking in fluorescence microscopy have advanced significantly in recent years, yet solutions for automating cell detection and tracking in 3D label-free imaging of dense tissues and embryos remain inaccessible. Here, we describe embGAN, a deep learning pipeline that addresses the challenge of automated cell detection and tracking in label-free 3D time-lapse imaging. embGAN requires no manual data annotation for training, learns robust detections that exhibits a high degree of scale invariance, and generalizes well to images acquired in multiple labs on multiple instruments. We characterize embGAN's performance using lineage tracing in the C. elegans embryo as a benchmark. embGAN achieves near-state-of-the-art performance in cell detection and tracking, enabling high-throughput studies of cell lineage without the need for fluorescent reporters or transgenics.

Ultrack: pushing the limits of cell tracking across biological scales

Tracking live cells across 2D, 3D, and multi-channel time-lapse recordings is crucial for understanding tissue-scale biological processes. Despite advancements in imaging technology, achieving accurate cell tracking remains challenging, particularly in complex and crowded tissues where cell segmentation is often ambiguous. We present Ultrack, a versatile and scalable cell-tracking method that tackles this challenge by considering candidate segmentations derived from multiple algorithms and parameter sets. Ultrack employs temporal consistency to select optimal segments, ensuring robust performance even under segmentation uncertainty. We validate our method on diverse datasets, including terabyte-scale developmental time-lapses of zebrafish, fruit fly, and nematode embryos, as well as multi-color and label-free cellular imaging. We show that Ultrack achieves state-of-the-art performance on the Cell Tracking Challenge and demonstrates superior accuracy in tracking densely packed embryonic cells over extended periods. Moreover, we propose an approach to tracking validation via dual-channel sparse labeling that enables high-fidelity ground truth generation, pushing the boundaries of long-term cell tracking assessment. Our method is freely available as a Python package with Fiji and napari plugins and can be deployed in a high-performance computing environment, facilitating widespread adoption by the research community.

Optical microscopic imaging, manipulation, and analysis methods for morphogenesis research

Morphogenesis is a developmental process of organisms being shaped through complex and cooperative cellular movements. To understand the interplay between genetic programs and the resulting multicellular morphogenesis, it is essential to characterize the morphologies and dynamics at the single-cell level and to understand how physical forces serve as both signaling components and driving forces of tissue deformations. In recent years, advances in microscopy techniques have led to improvements in imaging speed, resolution and depth. Concurrently, the development of various software packages has supported large-scale, analyses of challenging images at the single-cell resolution. While these tools have enhanced our ability to examine dynamics of cells and mechanical processes during morphogenesis, their effective integration requires specialized expertise. With this background, this review provides a practical overview of those techniques. First, we introduce microscopic techniques for multicellular imaging and image analysis software tools with a focus on cell segmentation and tracking. Second, we provide an overview of cutting-edge techniques for mechanical manipulation of cells and tissues. Finally, we introduce recent findings on morphogenetic mechanisms and mechanosensations that have been achieved by effectively combining microscopy, image analysis tools and mechanical manipulation techniques.

Automated Cell Lineage Reconstruction using Label-Free 4D Microscopy

Here we describe embGAN, a deep learning pipeline that addresses the challenge of automated cell detection and tracking in label-free 3D time lapse imaging. embGAN requires no manual data annotation for training, learns robust detections that exhibits a high degree of scale invariance and generalizes well to images acquired in multiple labs on multiple instruments.

Tissue Morphogenesis Through Dynamic Cell and Matrix Interactions

Multicellular organisms generate tissues of diverse shapes and functions from cells and extracellular matrices. Their adhesion molecules mediate cell-cell and cell-matrix interactions, which not only play crucial roles in maintaining tissue integrity but also serve as key regulators of tissue morphogenesis. Cells constantly probe their environment to make decisions: They integrate chemical and mechanical information from the environment via diffusible ligand- or adhesion-based signaling to decide whether to release specific signaling molecules or enzymes, to divide or differentiate, to move away or stay, or even whether to live or die. These decisions in turn modify their environment, including the chemical nature and mechanical properties of the extracellular matrix. Tissue morphology is the physical manifestation of the remodeling of cells and matrices by their historical biochemical and biophysical landscapes. We review our understanding of matrix and adhesion molecules in tissue morphogenesis, with an emphasis on key physical interactions that drive morphogenesis.

Differentiable optimization layers enhance GNN-based mitosis detection

Automatic mitosis detection from video is an essential step in analyzing proliferative behaviour of cells. In existing studies, a conventional object detector such as Unet is combined with a link prediction algorithm to find correspondences between parent and daughter cells. However, they do not take into account the biological constraint that a cell in a frame can correspond to up to two cells in the next frame. Our model called GNN-DOL enables mitosis detection by complementing a graph neural network (GNN) with a differentiable optimization layer (DOL) that implements the constraint. In time-lapse microscopy sequences cultured under four different conditions, we observed that the layer substantially improved detection performance in comparison with GNN-based link prediction. Our results illustrate the importance of incorporating biological knowledge explicitly into deep learning models.

Machine learning enhanced cell tracking

Quantifying cell biology in space and time requires computational methods to detect cells, measure their properties, and assemble these into meaningful trajectories. In this aspect, machine learning (ML) is having a transformational effect on bioimage analysis, now enabling robust cell detection in multidimensional image data. However, the task of cell tracking, or constructing accurate multi-generational lineages from imaging data, remains an open challenge. Most cell tracking algorithms are largely based on our prior knowledge of cell behaviors, and as such, are difficult to generalize to new and unseen cell types or datasets. Here, we propose that ML provides the framework to learn aspects of cell behavior using cell tracking as the task to be learned. We suggest that advances in representation learning, cell tracking datasets, metrics, and methods for constructing and evaluating tracking solutions can all form part of an end-to-end ML-enhanced pipeline. These developments will lead the way to new computational methods that can be used to understand complex, time-evolving biological systems.

The Cell Tracking Challenge: 10 years of objective benchmarking

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.