Deep learning for bioimage analysis in developmental biology

Dev-ResNet: automated developmental event detection using deep learning

Delineating developmental events is central to experimental research using early life stages, permitting widespread identification of changes in event timing between species and environments. Yet, identifying developmental events is incredibly challenging, limiting the scale, reproducibility and throughput of using early life stages in experimental biology. We introduce Dev-ResNet, a small and efficient 3D convolutional neural network capable of detecting developmental events characterised by both spatial and temporal features, such as the onset of cardiac function and radula activity. We demonstrate the efficacy of Dev-ResNet using 10 diverse functional events throughout the embryonic development of the great pond snail, Lymnaea stagnalis. Dev-ResNet was highly effective in detecting the onset of all events, including the identification of thermally induced decoupling of event timings. Dev-ResNet has broad applicability given the ubiquity of bioimaging in developmental biology, and the transferability of deep learning, and so we provide comprehensive scripts and documentation for applying Dev-ResNet to different biological systems.

OoCount: A Machine-Learning Based Approach to Mouse Ovarian Follicle Counting and Classification

The number and distribution of ovarian follicles in each growth stage provides a reliable readout of ovarian health and function. Leveraging techniques for three-dimensional (3D) imaging of ovaries in toto has the potential to uncover total, accurate ovarian follicle counts. However, because of the size and holistic nature of these images, counting oocytes is time consuming and difficult. The advent of deep-learning algorithms has allowed for the rapid development of ultra-fast, automated methods to analyze microscopy images. In recent years, these pipelines have become more user-friendly and accessible to non-specialists. We used these tools to create OoCount, a high-throughput, open-source method for automatic oocyte segmentation and classification from fluorescent 3D microscopy images of whole mouse ovaries using a deep-learning convolutional neural network (CNN) based approach. We developed a fast tissue-clearing and spinning disk confocal-based imaging protocol to obtain 3D images of whole mount perinatal and adult mouse ovaries. Fluorescently labeled oocytes from 3D images of ovaries were manually annotated in Napari to develop a machine learning training dataset. This dataset was used to retrain StarDist using a CNN within DL4MicEverywhere to automatically label all oocytes in the ovary. In a second phase, we utilize Accelerated Pixel and Object Classification, a Napari plugin, to classify labeled oocytes and sort them into growth stages. Here, we provide an end-to-end protocol for producing high-quality 3D images of the perinatal and adult mouse ovary, obtaining follicle counts and staging. We also demonstrate how to customize OoCount to fit images produced in any lab. Using OoCount, we can obtain accurate counts of oocytes in each growth stage in the perinatal and adult ovary, improving our ability to study ovarian function and fertility.

Summary sentence

This protocol introduces OoCount, a high-throughput, open-source method for automatic oocyte segmentation and classification from fluorescent 3D microscopy images of whole mouse ovaries using a machine learning-based approach.

Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets

Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and spatial analysis is necessary. Here we describe a practical and annotation-efficient pipeline for organelle-specific segmentation, spatial analysis and visualization of large volume electron microscopy datasets using freely available, user-friendly software tools that can be run on a single standard workstation. The procedures are aimed at researchers in the life sciences with modest computational expertise, who use volume electron microscopy and need to generate three-dimensional (3D) segmentation labels for different types of cell organelles while minimizing manual annotation efforts, to analyze the spatial interactions between organelle instances and to visualize the 3D segmentation results. We provide detailed guidelines for choosing well-suited segmentation tools for specific cell organelles, and to bridge compatibility issues between freely available open-source tools, we distribute the critical steps as easily installable Album solutions for deep learning segmentation, spatial analysis and 3D rendering. Our detailed description can serve as a reference for similar projects requiring particular strategies for single- or multiple-organelle analysis, which can be achieved with computational resources commonly available to single-user setups.

Swin-cryoEM: Multi-class cryo-electron micrographs single particle mixed detection method

Cryo-electron micrograph images have various characteristics such as varying sizes, shapes, and distribution densities of individual particles, severe background noise, high levels of impurities, irregular shapes, blurred edges, and similar color to the background. How to demonstrate good adaptability in the field of image vision by picking up single particles from multiple types of cryo-electron micrographs is currently a challenge in the field of cryo-electron micrographs. This paper combines the characteristics of the MixUp hybrid enhancement algorithm, enhances the image feature information in the pre-processing stage, builds a feature perception network based on the channel self-attention mechanism in the forward network of the Swin Transformer model network, achieving adaptive adjustment of self-attention mechanism between different single particles, increasing the network's tolerance to noise, Incorporating PReLU activation function to enhance information exchange between pixel blocks of different single particles, and combining the Cross-Entropy function with the softmax function to construct a classification network based on Swin Transformer suitable for cryo-electron micrograph single particle detection model (Swin-cryoEM), achieving mixed detection of multiple types of single particles. Swin-cryoEM algorithm can better solve the problem of good adaptability in picking single particles of many types of cryo-electron micrographs, improve the accuracy and generalization ability of the single particle picking method, and provide high-quality data support for the three-dimensional reconstruction of a single particle. In this paper, ablation experiments and comparison experiments were designed to evaluate and compare Swin-cryoEM algorithms in detail and comprehensively on multiple datasets. The Average Precision is an important evaluation index of the evaluation model, and the optimal Average Precision reached 95.5% in the training stage Swin-cryoEM, and the single particle picking performance was also superior in the prediction stage. This model inherits the advantages of the Swin Transformer detection model and is superior to mainstream models such as Faster R-CNN and YOLOv5 in terms of the single particle detection capability of cryo-electron micrographs.

Image restoration of degraded time-lapse microscopy data mediated by near-infrared imaging

Time-lapse fluorescence microscopy is key to unraveling biological development and function; however, living systems, by their nature, permit only limited interrogation and contain untapped information that can only be captured by more invasive methods. Deep-tissue live imaging presents a particular challenge owing to the spectral range of live-cell imaging probes/fluorescent proteins, which offer only modest optical penetration into scattering tissues. Herein, we employ convolutional neural networks to augment live-imaging data with deep-tissue images taken on fixed samples. We demonstrate that convolutional neural networks may be used to restore deep-tissue contrast in GFP-based time-lapse imaging using paired final-state datasets acquired using near-infrared dyes, an approach termed InfraRed-mediated Image Restoration (IR2). Notably, the networks are remarkably robust over a wide range of developmental times. We employ IR2 to enhance the information content of green fluorescent protein time-lapse images of zebrafish and Drosophila embryo/larval development and demonstrate its quantitative potential in increasing the fidelity of cell tracking/lineaging in developing pescoids. Thus, IR2 is poised to extend live imaging to depths otherwise inaccessible.

Tissue clearing and imaging approaches for in toto analysis of the reproductive system

New microscopy techniques in combination with tissue clearing protocols and emerging analytical approaches have presented researchers with the tools to understand dynamic biological processes in a three-dimensional context. This paves the road for the exploration of new research questions in reproductive biology, for which previous techniques have provided only approximate resolution. These new methodologies now allow for contextualized analysis of far larger volumes than was previously possible. Tissue optical clearing and three-dimensional imaging techniques posit the bridging of molecular mechanisms, macroscopic morphogenic development, and maintenance of reproductive function into one cohesive and comprehensive understanding of the biology of the reproductive system. In this review, we present a survey of the various tissue clearing techniques and imaging systems, as they have been applied to the developing and adult reproductive system. We provide an overview of tools available for analysis of experimental data, giving particular attention to the emergence of AI-assisted methods and their applicability to image analysis. We conclude with an evaluation of how novel image analysis approaches which have been applied to other organ systems could be incorporated into future experimental evaluation of reproductive biology.

Moving beyond the desktop: prospects for practical bioimage analysis via the web

As biological imaging continues to rapidly advance, it results in increasingly complex image data, necessitating a reevaluation of conventional bioimage analysis methods and their accessibility. This perspective underscores our belief that a transition from desktop-based tools to web-based bioimage analysis could unlock immense opportunities for improved accessibility, enhanced collaboration, and streamlined workflows. We outline the potential benefits, such as reduced local computational demands and solutions to common challenges, including software installation issues and limited reproducibility. Furthermore, we explore the present state of web-based tools, hurdles in implementation, and the significance of collective involvement from the scientific community in driving this transition. In acknowledging the potential roadblocks and complexity of data management, we suggest a combined approach of selective prototyping and large-scale workflow application for optimal usage. Embracing web-based bioimage analysis could pave the way for the life sciences community to accelerate biological research, offering a robust platform for a more collaborative, efficient, and democratized science.

Spatial topology of organelle is a new breast cancer cell classifier

Genomics and proteomics have been central to identify tumor cell populations, but more accurate approaches to classify cell subtypes are still lacking. We propose a new methodology to accurately classify cancer cells based on their organelle spatial topology. Herein, we developed an organelle topology-based cell classification pipeline (OTCCP), which integrates artificial intelligence (AI) and imaging quantification to analyze organelle spatial distribution and inter-organelle topology. OTCCP was used to classify a panel of human breast cancer cells, grown as 2D monolayer or 3D tumor spheroids using early endosomes, mitochondria, and their inter-organelle contacts. Organelle topology allows for a highly precise differentiation between cell lines of different subtypes and aggressiveness. These findings lay the groundwork for using organelle topological profiling as a fast and efficient method for phenotyping breast cancer function as well as a discovery tool to advance our understanding of cancer cell biology at the subcellular level.

Quantifying the relationship between cell proliferation and morphology during development of the face

Morphogenesis requires highly coordinated, complex interactions between cellular processes: proliferation, migration, and apoptosis, along with physical tissue interactions. How these cellular and tissue dynamics drive morphogenesis remains elusive. Three dimensional (3D) microscopic imaging poses great promise, and generates elegant images. However, generating even moderate through-put quantified images is challenging for many reasons. As a result, the association between morphogenesis and cellular processes in 3D developing tissues has not been fully explored. To address this critical gap, we have developed an imaging and image analysis pipeline to enable 3D quantification of cellular dynamics along with 3D morphology for the same individual embryo. Specifically, we focus on how 3D distribution of proliferation relates to morphogenesis during mouse facial development. Our method involves imaging with light-sheet microscopy, automated segmentation of cells and tissues using machine learning-based tools, and quantification of external morphology via geometric morphometrics. Applying this framework, we show that changes in proliferation are tightly correlated to changes in morphology over the course of facial morphogenesis. These analyses illustrate the potential of this pipeline to investigate mechanistic relationships between cellular dynamics and morphogenesis during embryonic development.

EmbryoNet: using deep learning to link embryonic phenotypes to signaling pathways

Evolutionarily conserved signaling pathways are essential for early embryogenesis, and reducing or abolishing their activity leads to characteristic developmental defects. Classification of phenotypic defects can identify the underlying signaling mechanisms, but this requires expert knowledge and the classification schemes have not been standardized. Here we use a machine learning approach for automated phenotyping to train a deep convolutional neural network, EmbryoNet, to accurately identify zebrafish signaling mutants in an unbiased manner. Combined with a model of time-dependent developmental trajectories, this approach identifies and classifies with high precision phenotypic defects caused by loss of function of the seven major signaling pathways relevant for vertebrate development. Our classification algorithms have wide applications in developmental biology and robustly identify signaling defects in evolutionarily distant species. Furthermore, using automated phenotyping in high-throughput drug screens, we show that EmbryoNet can resolve the mechanism of action of pharmaceutical substances. As part of this work, we freely provide more than 2 million images that were used to train and test EmbryoNet.

Automated staging of zebrafish embryos using machine learning

The zebrafish ( Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

Automated staging of zebrafish embryos using machine learning

The zebrafish ( Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

A study of transfer of information in animal collectives using deep learning tools

We studied how the interactions among animals in a collective allow for the transfer of information. We performed laboratory experiments to study how zebrafish in a collective follow a subset of trained animals that move towards a light when it turns on because they expect food at that location. We built some deep learning tools to distinguish from video which are the trained and the naïve animals and to detect when each animal reacts to the light turning on. These tools gave us the data to build a model of interactions that we designed to have a balance between transparency and accuracy. The model finds a low-dimensional function that describes how a naïve animal weights neighbours depending on focal and neighbour variables. According to this low-dimensional function, neighbour speed plays an important role in the interactions. Specifically, a naïve animal weights more a neighbour in front than to the sides or behind, and more so the faster the neighbour is moving; and if the neighbour moves fast enough, the differences coming from the neighbour's relative position largely disappear. From the lens of decision-making, neighbour speed acts as confidence measure about where to go. This article is part of a discussion meeting issue 'Collective behaviour through time'.

Automated staging of zebrafish embryos using machine learning

The zebrafish (Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

Liver inter-organelle membrane contact sites revealed by serial section electron tomography

Inter-organelle membrane contact sites (MCSs) are defined as areas of close proximity between the membranes of two organelles (10-80nm). They have been implicated in many physiological processes such as Ca++, lipids or small molecules transfer, organelles biogenesis or dynamic and have an important role in many cellular processes such as apoptosis, autophagy, and signaling. Since the distance and the extent of these contacts are in the nanometer range, high resolution techniques are ideal for imaging these structures. It is for this reason that transmission electron microscopy (TEM) has been considered the gold standard for MCSs visualization and the first technique that described them. However, often TEM analysis is limited to 2D lacking information on the 3D association between the organelles involved in MCSs. To fully describe the complex architecture of MSCs and to unveil their role in cellular physiology a 3D analysis is required. This chapter provides a method for the analysis of MCSs using serial section electron tomography (ssET), a technique able to reconstruct in 3D at nanometer resolution cellular and subcellular structures. By applying this procedure, it was possible to elucidate the role of the contacts between Endoplasmic Reticulum (ER) and other organelles in liver lipid metabolism.

Analyzing angiogenesis on a chip using deep learning-based image processing

Angiogenesis, the formation of new blood vessels from existing vessels, has been associated with more than 70 diseases. Although numerous studies have established angiogenesis models, only a few indicators can be used to analyze angiogenic structures. In the present study, we developed an image-processing pipeline based on deep learning to analyze and quantify angiogenesis. We utilized several image-processing algorithms to quantify angiogenesis, including a deep learning-based cell nuclear segmentation algorithm and image skeletonization. This method could quantify and measure changes in blood vessels in response to biochemical gradients using 16 indicators, including length, width, number, and nuclear distribution. Moreover, this procedure is highly efficient for the three-dimensional quantitative analysis of angiogenesis and can be applied to diverse angiogenesis investigations.

BASIN: A Semi-automatic Workflow, with Machine Learning Segmentation, for Objective Statistical Analysis of Biomedical and Biofilm Image Datasets

Micrograph comparison remains useful in bioscience. This technology provides researchers with a quick snapshot of experimental conditions. But sometimes a two- condition comparison relies on researchers' eyes to draw conclusions. Our Bioimage Analysis, Statistic, and Comparison (BASIN) software provides an objective and reproducible comparison leveraging inferential statistics to bridge image data with other modalities. Users have access to machine learning-based object segmentation. BASIN provides several data points such as images' object counts, intensities, and areas. Hypothesis testing may also be performed. To improve BASIN's accessibility, we implemented it using R Shiny and provided both an online and offline version. We used BASIN to process 498 image pairs involving five bioscience topics. Our framework supported either direct claims or extrapolations 57% of the time. Analysis results were manually curated to determine BASIN's accuracy which was shown to be 78%. Additionally, each BASIN version's initial release shows an average 82% FAIR compliance score.

“Tonga”: A Novel Toolbox for Straightforward Bioimage Analysis

Techniques to acquire and analyze biological images are central to life science. However, the workflow downstream of imaging can be complex and involve several tools, leading to creation of very specialized scripts and pipelines that are difficult to reproduce by other users. Although many commercial and open-source software are available, non-expert users are often challenged by a knowledge gap in setting up analysis pipelines and selecting correct tools for extracting data from images. Moreover, a significant share of everyday image analysis requires simple tools, such as precise segmentation, cell counting, and recording of fluorescent intensities. Hence, there is a need for user-friendly platforms for everyday image analysis that do not require extensive prior knowledge on bioimage analysis or coding. We set out to create a bioimage analysis software that has a straightforward interface and covers common analysis tasks such as object segmentation and analysis, in a practical, reproducible, and modular fashion. We envision our software being useful for analysis of cultured cells, histological sections, and high-content data.

Deep Learning-Enabled Technologies for Bioimage Analysis

Deep learning (DL) is a subfield of machine learning (ML), which has recently demonstrated its potency to significantly improve the quantification and classification workflows in biomedical and clinical applications. Among the end applications profoundly benefitting from DL, cellular morphology quantification is one of the pioneers. Here, we first briefly explain fundamental concepts in DL and then we review some of the emerging DL-enabled applications in cell morphology quantification in the fields of embryology, point-of-care ovulation testing, as a predictive tool for fetal heart pregnancy, cancer diagnostics via classification of cancer histology images, autosomal polycystic kidney disease, and chronic kidney diseases.

Towards Tabula Gallus

The Tabula Gallus is a proposed project that aims to create a map of every cell type in the chicken body and chick embryos. Chickens (Gallus gallus) are one of the most recognized model animals that recapitulate the development and physiology of mammals. The Tabula Gallus will generate a compendium of single-cell transcriptome data from Gallus gallus, characterize each cell type, and provide tools for the study of the biology of this species, similar to other ongoing cell atlas projects (Tabula Muris and Tabula Sapiens/Human Cell Atlas for mice and humans, respectively). The Tabula Gallus will potentially become an international collaboration between many researchers. This project will be useful for the basic scientific study of Gallus gallus and other birds (e.g., cell biology, molecular biology, developmental biology, neuroscience, physiology, oncology, virology, behavior, ecology, and evolution). It will eventually be beneficial for a better understanding of human health and diseases.

A Systematic, Open-Science Framework for Quantification of Cell-Types in Mouse Brain Sections Using Fluorescence Microscopy

The ever-expanding availability and evolution of microscopy tools has enabled ground-breaking discoveries in neurobiology, particularly with respect to the analysis of cell-type density and distribution. Widespread implementation of many of the elegant image processing tools available continues to be impeded by the lack of complete workflows that span from experimental design, labeling techniques, and analysis workflows, to statistical methods and data presentation. Additionally, it is important to consider open science principles (e.g., open-source software and tools, user-friendliness, simplicity, and accessibility). In the present methodological article, we provide a compendium of resources and a FIJI-ImageJ-based workflow aimed at improving the quantification of cell density in mouse brain samples using semi-automated open-science-based methods. Our proposed framework spans from principles and best practices of experimental design, histological and immunofluorescence staining, and microscopy imaging to recommendations for statistical analysis and data presentation. To validate our approach, we quantified neuronal density in the mouse barrel cortex using antibodies against pan-neuronal and interneuron markers. This framework is intended to be simple and yet flexible, such that it can be adapted to suit distinct project needs. The guidelines, tips, and proposed methodology outlined here, will support researchers of wide-ranging experience levels and areas of focus in neuroscience research.

Advances in live imaging early mouse development: exploring the researcher's interdisciplinary toolkit

Live imaging is an important part of the developmental biologist's armoury of methods. In the case of the mouse embryo, recent advances in several disciplines including embryo culture, microscopy hardware and computational analysis have all contributed to our ability to probe dynamic events during early development. Together, these advances have provided us with a versatile and powerful 'toolkit', enabling us not only to image events during mouse embryogenesis, but also to intervene with them. In this short Spotlight article, we summarise advances and challenges in using live imaging specifically for understanding early mouse embryogenesis.

Tissue clearing and 3D imaging in developmental biology

Tissue clearing increases the transparency of late developmental stages and enables deep imaging in fixed organisms. Successful implementation of these methodologies requires a good grasp of sample processing, imaging and the possibilities offered by image analysis. In this Primer, we highlight how tissue clearing can revolutionize the histological analysis of developmental processes and we advise on how to implement effective clearing protocols, imaging strategies and analysis methods for developmental biology.