3DeeCellTracker, a deep learning-based pipeline for segmenting and tracking cells in 3D time lapse images

Mechanism of sensory perception unveiled by simultaneous measurement of membrane voltage and intracellular calcium

Measuring neuronal activity is important for understanding neuronal function. Ca2+ imaging by genetically encoded calcium indicators (GECIs) is a powerful way to measure neuronal activity. Although it revealed important aspects of neuronal function, measuring the neuronal membrane voltage is important to understand neuronal function as it triggers neuronal activation. Recent progress of genetically encoded voltage indicators (GEVIs) enabled us fast and precise measurements of neuronal membrane voltage. To clarify the relation of the membrane voltage and intracellular Ca2+, we analyzed neuronal activities of olfactory neuron AWA in Caenorhabditis elegans by GCaMP6f (GECI) and paQuasAr3 (GEVI) responding to odorants. We found that the membrane voltage encodes the stimuli change by the timing and the duration by the weak semi-stable depolarization. However, the change of the intracellular Ca2+ encodes the strength of the stimuli. Furthermore, ODR-3, a G-protein alpha subunit, was shown to be important for stabilizing the membrane voltage. These results suggest that the combination of calcium and voltage imaging provides a deeper understanding of the information in neural circuits.

Computational Methods for Image Analysis in Craniofacial Development and Disease

Observation is at the center of all biological sciences. Advances in imaging technologies are therefore essential to derive novel biological insights to better understand the complex workings of living systems. Recent high-throughput sequencing and imaging techniques are allowing researchers to simultaneously address complex molecular variations spatially and temporarily in tissues and organs. The availability of increasingly large dataset sizes has allowed for the evolution of robust deep learning models, designed to interrogate biomedical imaging data. These models are emerging as transformative tools in diagnostic medicine. Combined, these advances allow for dynamic, quantitative, and predictive observations of entire organisms and tissues. Here, we address 3 main tasks of bioimage analysis, image restoration, segmentation, and tracking and discuss new computational tools allowing for 3-dimensional spatial genomics maps. Finally, we demonstrate how these advances have been applied in studies of craniofacial development and oral disease pathogenesis.

AI-Powered Microfluidics: Shaping the Future of Phenotypic Drug Discovery

Phenotypic drug discovery (PDD), which involves harnessing biological systems directly to uncover effective drugs, has undergone a resurgence in recent years. The rapid advancement of artificial intelligence (AI) over the past few years presents numerous opportunities for augmenting phenotypic drug screening on microfluidic platforms, leveraging its predictive capabilities, data analysis, efficient data processing, etc. Microfluidics coupled with AI is poised to revolutionize the landscape of phenotypic drug discovery. By integrating advanced microfluidic platforms with AI algorithms, researchers can rapidly screen large libraries of compounds, identify novel drug candidates, and elucidate complex biological pathways with unprecedented speed and efficiency. This review provides an overview of recent advances and challenges in AI-based microfluidics and their applications in drug discovery. We discuss the synergistic combination of microfluidic systems for high-throughput screening and AI-driven analysis for phenotype characterization, drug-target interactions, and predictive modeling. In addition, we highlight the potential of AI-powered microfluidics to achieve an automated drug screening system. Overall, AI-powered microfluidics represents a promising approach to shaping the future of phenotypic drug discovery by enabling rapid, cost-effective, and accurate identification of therapeutically relevant compounds.

Unifying community-wide whole-brain imaging datasets enables robust automated neuron identification and reveals determinants of neuron positioning in C. elegans

We develop a data harmonization approach for C. elegans volumetric microscopy data, still or video, consisting of a standardized format, data pre-processing techniques, and a set of human-in-the-loop machine learning based analysis software tools. We unify a diverse collection of 118 whole-brain neural activity imaging datasets from 5 labs, storing these and accompanying tools in an online repository called WormID (wormid.org). We use this repository to train three existing automated cell identification algorithms to, for the first time, enable accuracy in neural identification that generalizes across labs, approaching human performance in some cases. We mine this repository to identify factors that influence the developmental positioning of neurons. To facilitate communal use of this repository, we created open-source software, code, web-based tools, and tutorials to explore and curate datasets for contribution to the scientific community. This repository provides a growing resource for experimentalists, theorists, and toolmakers to (a) study neuroanatomical organization and neural activity across diverse experimental paradigms, (b) develop and benchmark algorithms for automated neuron detection, segmentation, cell identification, tracking, and activity extraction, and (c) inform models of neurobiological development and function.

Protocol for measuring cell cycle Zn2+ dynamics using a FRET-based biosensor

The exchangeable Zn2+ pool in cells is not static but responds to perturbations as well as fluctuates naturally through the cell cycle. Here, we present a protocol to carry out long-term live-cell imaging of cells expressing a cytosolic Zn2+ sensor. We then describe how to track cells using the published pipeline EllipTrack and how to analyze the single-cell traces to determine changes in labile Zn2+ in response to perturbation. For complete details on the use and execution of this protocol, please refer to Rakshit and Holtzen et al.1.

Antibody design using deep learning: from sequence and structure design to affinity maturation

Deep learning has achieved impressive results in various fields such as computer vision and natural language processing, making it a powerful tool in biology. Its applications now encompass cellular image classification, genomic studies and drug discovery. While drug development traditionally focused deep learning applications on small molecules, recent innovations have incorporated it in the discovery and development of biological molecules, particularly antibodies. Researchers have devised novel techniques to streamline antibody development, combining in vitro and in silico methods. In particular, computational power expedites lead candidate generation, scaling and potential antibody development against complex antigens. This survey highlights significant advancements in protein design and optimization, specifically focusing on antibodies. This includes various aspects such as design, folding, antibody-antigen interactions docking and affinity maturation.

A general algorithm for consensus 3D cell segmentation from 2D segmented stacks

Cell segmentation is the fundamental task. Only by segmenting, can we define the quantitative spatial unit for collecting measurements to draw biological conclusions. Deep learning has revolutionized 2D cell segmentation, enabling generalized solutions across cell types and imaging modalities. This has been driven by the ease of scaling up image acquisition, annotation and computation. However 3D cell segmentation, which requires dense annotation of 2D slices still poses significant challenges. Labelling every cell in every 2D slice is prohibitive. Moreover it is ambiguous, necessitating cross-referencing with other orthoviews. Lastly, there is limited ability to unambiguously record and visualize 1000's of annotated cells. Here we develop a theory and toolbox, u-Segment3D for 2D-to-3D segmentation, compatible with any 2D segmentation method. Given optimal 2D segmentations, u-Segment3D generates the optimal 3D segmentation without data training, as demonstrated on 11 real life datasets, >70,000 cells, spanning single cells, cell aggregates and tissue.

Automated segmentation and recognition of C. elegans whole-body cells

MOTIVATION

Accurate segmentation and recognition of C.elegans cells are critical for various biological studies, including gene expression, cell lineages, and cell fates analysis at single-cell level. However, the highly dense distribution, similar shapes, and inhomogeneous intensity profiles of whole-body cells in 3D fluorescence microscopy images make automatic cell segmentation and recognition a challenging task. Existing methods either rely on additional fiducial markers or only handle a subset of cells. Given the difficulty or expense associated with generating fiducial features in many experimental settings, a marker-free approach capable of reliably segmenting and recognizing C.elegans whole-body cells is highly desirable.

RESULTS

We report a new pipeline, called automated segmentation and recognition (ASR) of cells, and applied it to 3D fluorescent microscopy images of L1-stage C.elegans with 558 whole-body cells. A novel displacement vector field based deep learning model is proposed to address the problem of reliable segmentation of highly crowded cells with blurred boundary. We then realize the cell recognition by encoding and exploiting statistical priors on cell positions and structural similarities of neighboring cells. To the best of our knowledge, this is the first method successfully applied to the segmentation and recognition of C.elegans whole-body cells. The ASR-segmentation module achieves an F1-score of 0.8956 on a dataset of 116 C.elegans image stacks with 64 728 cells (accuracy 0.9880, AJI 0.7813). Based on the segmentation results, the ASR recognition module achieved an average accuracy of 0.8879. We also show ASR's applicability to other cell types, e.g. platynereis and rat kidney cells.

AVAILABILITY AND IMPLEMENTATION

The code is available at https://github.com/reaneyli/ASR.

Versatile multiple object tracking in sparse 2D/3D videos via deformable image registration

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.

See Elegans: Simple-to-use, accurate, and automatic 3D detection of neural activity from densely packed neurons

In the emerging field of whole-brain imaging at single-cell resolution, which represents one of the new frontiers to investigate the link between brain activity and behavior, the nematode Caenorhabditis elegans offers one of the most characterized models for systems neuroscience. Whole-brain recordings consist of 3D time series of volumes that need to be processed to obtain neuronal traces. Current solutions for this task are either computationally demanding or limited to specific acquisition setups. Here, we propose See Elegans, a direct programming algorithm that combines different techniques for automatic neuron segmentation and tracking without the need for the RFP channel, and we compare it with other available algorithms. While outperforming them in most cases, our solution offers a novel method to guide the identification of a subset of head neurons based on position and activity. The built-in interface allows the user to follow and manually curate each of the processing steps. See Elegans is thus a simple-to-use interface aimed at speeding up the post-processing of volumetric calcium imaging recordings while maintaining a high level of accuracy and low computational demands. (Contact: enrico.lanza@iit.it).

3DCellComposer - A Versatile Pipeline Utilizing 2D Cell Segmentation Methods for 3D Cell Segmentation

Background

Cell segmentation is crucial in bioimage informatics, as its accuracy directly impacts conclusions drawn from cellular analyses. While many approaches to 2D cell segmentation have been described, 3D cell segmentation has received much less attention. 3D segmentation faces significant challenges, including limited training data availability due to the difficulty of the task for human annotators, and inherent three-dimensional complexity. As a result, existing 3D cell segmentation methods often lack broad applicability across different imaging modalities.

Results

To address this, we developed a generalizable approach for using 2D cell segmentation methods to produce accurate 3D cell segmentations. We implemented this approach in 3DCellComposer, a versatile, open-source package that allows users to choose any existing 2D segmentation model appropriate for their tissue or cell type(s) without requiring any additional training. Importantly, we have enhanced our open source CellSegmentationEvaluator quality evaluation tool to support 3D images. It provides metrics that allow selection of the best approach for a given imaging source and modality, without the need for human annotations to assess performance. Using these metrics, we demonstrated that our approach produced high-quality 3D segmentations of tissue images, and that it could outperform an existing 3D segmentation method on the cell culture images with which it was trained.

Conclusions

3DCellComposer, when paired with well-trained 2D segmentation models, provides an important alternative to acquiring human-annotated 3D images for new sample types or imaging modalities and then training 3D segmentation models using them. It is expected to be of significant value for large scale projects such as the Human BioMolecular Atlas Program.

Deep Learning-Based 3D Single-Cell Imaging Analysis Pipeline Enables Quantification of Cell-Cell Interaction Dynamics in the Tumor Microenvironment

The three-dimensional (3D) tumor microenvironment (TME) comprises multiple interacting cell types that critically impact tumor pathology and therapeutic response. Efficient 3D imaging assays and analysis tools could facilitate profiling and quantifying distinctive cell-cell interaction dynamics in the TMEs of a wide spectrum of human cancers. Here, we developed a 3D live-cell imaging assay using confocal microscopy of patient-derived tumor organoids and a software tool, SiQ-3D (single-cell image quantifier for 3D), that optimizes deep learning (DL)-based 3D image segmentation, single-cell phenotype classification, and tracking to automatically acquire multidimensional dynamic data for different interacting cell types in the TME. An organoid model of tumor cells interacting with natural killer cells was used to demonstrate the effectiveness of the 3D imaging assay to reveal immuno-oncology dynamics as well as the accuracy and efficiency of SiQ-3D to extract quantitative data from large 3D image datasets. SiQ-3D is Python-based, publicly available, and customizable to analyze data from both in vitro and in vivo 3D imaging. The DL-based 3D imaging analysis pipeline can be employed to study not only tumor interaction dynamics with diverse cell types in the TME but also various cell-cell interactions involved in other tissue/organ physiology and pathology.

SIGNIFICANCE

A 3D single-cell imaging pipeline that quantifies cancer cell interaction dynamics with other TME cell types using primary patient-derived samples can elucidate how cell-cell interactions impact tumor behavior and treatment responses.

From pixels to insights: Machine learning and deep learning for bioimage analysis

Bioimage analysis plays a critical role in extracting information from biological images, enabling deeper insights into cellular structures and processes. The integration of machine learning and deep learning techniques has revolutionized the field, enabling the automated, reproducible, and accurate analysis of biological images. Here, we provide an overview of the history and principles of machine learning and deep learning in the context of bioimage analysis. We discuss the essential steps of the bioimage analysis workflow, emphasizing how machine learning and deep learning have improved preprocessing, segmentation, feature extraction, object tracking, and classification. We provide examples that showcase the application of machine learning and deep learning in bioimage analysis. We examine user-friendly software and tools that enable biologists to leverage these techniques without extensive computational expertise. This review is a resource for researchers seeking to incorporate machine learning and deep learning in their bioimage analysis workflows and enhance their research in this rapidly evolving field.

Unlocking cardiac motion: assessing software and machine learning for single-cell and cardioid kinematic insights

The heart coordinates its functional parameters for optimal beat-to-beat mechanical activity. Reliable detection and quantification of these parameters still represent a hot topic in cardiovascular research. Nowadays, computer vision allows the development of open-source algorithms to measure cellular kinematics. However, the analysis software can vary based on analyzed specimens. In this study, we compared different software performances in in-silico model, in-vitro mouse adult ventricular cardiomyocytes and cardioids. We acquired in-vitro high-resolution videos during suprathreshold stimulation at 0.5-1-2 Hz, adapting the protocol for the cardioids. Moreover, we exposed the samples to inotropic and depolarizing substances. We analyzed in-silico and in-vitro videos by (i) MUSCLEMOTION, the gold standard among open-source software; (ii) CONTRACTIONWAVE, a recently developed tracking software; and (iii) ViKiE, an in-house customized video kinematic evaluation software. We enriched the study with three machine-learning algorithms to test the robustness of the motion-tracking approaches. Our results revealed that all software produced comparable estimations of cardiac mechanical parameters. For instance, in cardioids, beat duration measurements at 0.5 Hz were 1053.58 ms (MUSCLEMOTION), 1043.59 ms (CONTRACTIONWAVE), and 937.11 ms (ViKiE). ViKiE exhibited higher sensitivity in exposed samples due to its localized kinematic analysis, while MUSCLEMOTION and CONTRACTIONWAVE offered temporal correlation, combining global assessment with time-efficient analysis. Finally, machine learning reveals greater accuracy when trained with MUSCLEMOTION dataset in comparison with the other software (accuracy > 83%). In conclusion, our findings provide valuable insights for the accurate selection and integration of software tools into the kinematic analysis pipeline, tailored to the experimental protocol.

4D Light-sheet Imaging of Zebrafish Cardiac Contraction

Zebrafish is an intriguing model organism known for its remarkable cardiac regeneration capacity. Studying the contracting heart in vivo is essential for gaining insights into structural and functional changes in response to injuries. However, obtaining high-resolution and high-speed 4-dimensional (4D, 3D spatial + 1D temporal) images of the zebrafish heart to assess cardiac architecture and contractility remains challenging. In this context, an in-house light-sheet microscope (LSM) and customized computational analysis are used to overcome these technical limitations. This strategy, involving LSM system construction, retrospective synchronization, single cell tracking, and user-directed analysis, enables one to investigate the micro-structure and contractile function across the entire heart at the single-cell resolution in the transgenic Tg(myl7:nucGFP) zebrafish larvae. Additionally, we are able to further incorporate microinjection of small molecule compounds to induce cardiac injury in a precise and controlled manner. Overall, this framework allows one to track physiological and pathophysiological changes, as well as the regional mechanics at the single-cell level during cardiac morphogenesis and regeneration.

Deep Learning-Based Cell Tracking in Deforming Organs and Moving Animals

Cell tracking is an essential step in extracting cellular signals from moving cells, which is vital for understanding the mechanisms underlying various biological functions and processes, particularly in organs such as the brain and heart. However, cells in living organisms often exhibit extensive and complex movements caused by organ deformation and whole-body motion. These movements pose a challenge in obtaining high-quality time-lapse cell images and tracking the intricate cell movements in the captured images. Recent advances in deep learning techniques provide powerful tools for detecting cells in low-quality images with densely packed cell populations, as well as estimating cell positions for cells undergoing large nonrigid movements. This chapter introduces the challenges of cell tracking in deforming organs and moving animals, outlines the solutions to these challenges, and presents a detailed protocol for data preparation, as well as for performing cell segmentation and tracking using the latest version of 3DeeCellTracker. This protocol is expected to enable researchers to gain deeper insights into organ dynamics and biological processes.

Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation

Reading out neuronal activity from three-dimensional (3D) functional imaging requires segmenting and tracking individual neurons. This is challenging in behaving animals if the brain moves and deforms. The traditional approach is to train a convolutional neural network with ground-truth (GT) annotations of images representing different brain postures. For 3D images, this is very labor intensive. We introduce 'targeted augmentation', a method to automatically synthesize artificial annotations from a few manual annotations. Our method ('Targettrack') learns the internal deformations of the brain to synthesize annotations for new postures by deforming GT annotations. This reduces the need for manual annotation and proofreading. A graphical user interface allows the application of the method end-to-end. We demonstrate Targettrack on recordings where neurons are labeled as key points or 3D volumes. Analyzing freely moving animals exposed to odor pulses, we uncover rich patterns in interneuron dynamics, including switching neuronal entrainment on and off.

Engineered organoids for biomedical applications

As miniaturized and simplified stem cell-derived 3D organ-like structures, organoids are rapidly emerging as powerful tools for biomedical applications. With their potential for personalized therapeutic interventions and high-throughput drug screening, organoids have gained significant attention recently. In this review, we discuss the latest developments in engineering organoids and using materials engineering, biochemical modifications, and advanced manufacturing technologies to improve organoid culture and replicate vital anatomical structures and functions of human tissues. We then explore the diverse biomedical applications of organoids, including drug development and disease modeling, and highlight the tools and analytical techniques used to investigate organoids and their microenvironments. We also examine the latest clinical trials and patents related to organoids that show promise for future clinical translation. Finally, we discuss the challenges and future perspectives of using organoids to advance biomedical research and potentially transform personalized medicine.

A deep learning-based pipeline for analyzing the influences of interfacial mechanochemical microenvironments on spheroid invasion using differential interference contrast microscopic images

Metastasis is the leading cause of cancer-related deaths. During this process, cancer cells are likely to navigate discrete tissue-tissue interfaces, enabling them to infiltrate and spread throughout the body. Three-dimensional (3D) spheroid modeling is receiving more attention due to its strengths in studying the invasive behavior of metastatic cancer cells. While microscopy is a conventional approach for investigating 3D invasion, post-invasion image analysis, which is a time-consuming process, remains a significant challenge for researchers. In this study, we presented an image processing pipeline that utilized a deep learning (DL) solution, with an encoder-decoder architecture, to assess and characterize the invasion dynamics of tumor spheroids. The developed models, equipped with feature extraction and measurement capabilities, could be successfully utilized for the automated segmentation of the invasive protrusions as well as the core region of spheroids situated within interfacial microenvironments with distinct mechanochemical factors. Our findings suggest that a combination of the spheroid culture and DL-based image analysis enable identification of time-lapse migratory patterns for tumor spheroids above matrix-substrate interfaces, thus paving the foundation for delineating the mechanism of local invasion during cancer metastasis.

Bridging live-cell imaging and next-generation cancer treatment

By providing spatial, molecular and morphological data over time, live-cell imaging can provide a deeper understanding of the cellular and signalling events that determine cancer response to treatment. Understanding this dynamic response has the potential to enhance clinical outcome by identifying biomarkers or actionable targets to improve therapeutic efficacy. Here, we review recent applications of live-cell imaging for uncovering both tumour heterogeneity in treatment response and the mode of action of cancer-targeting drugs. Given the increasing uses of T cell therapies, we discuss the unique opportunity of time-lapse imaging for capturing the interactivity and motility of immunotherapies. Although traditionally limited in the number of molecular features captured, novel developments in multidimensional imaging and multi-omics data integration offer strategies to connect single-cell dynamics to molecular phenotypes. We review the effect of these recent technological advances on our understanding of the cellular dynamics of tumour targeting and discuss their implication for next-generation precision medicine.

Dual-mode LiDAR SoC with an on-chip interframe filter and common optical platform design for keystone correction and auto-focus

This paper presents an innovative methodology that incorporates direct time-of-flight technology into intelligent sensing for projectors, along with a lightweight, dual-mode optically integrated LiDAR system. The proposed LiDAR system-on-chip, which utilizes a single-photon avalanche diode and time to digital converter with 0.13 µm bipolar CMOS DMOS technology, integrates an on-chip interframe filter, a common optical platform design, and a lightweight keystone correction assist algorithm. This comprehensive integration enables the system to achieve a measurement range of 11 m with 1% relative precision (simulations indicate the potential to achieve 30 m) in auto-focus mode. Additionally, it facilitates high frame-per-second keystone correction within a range of ±30∘ with an error of ±2∘ under illumination conditions of 20 klux.

Electric shock causes a fleeing-like persistent behavioral response in the nematode Caenorhabditis elegans

Behavioral persistency reflects internal brain states, which are the foundations of multiple brain functions. However, experimental paradigms enabling genetic analyses of behavioral persistency and its associated brain functions have been limited. Here, we report novel persistent behavioral responses caused by electric stimuli in the nematode Caenorhabditis elegans. When the animals on bacterial food are stimulated by alternating current, their movement speed suddenly increases 2- to 3-fold, persisting for more than 1 minute even after a 5-second stimulation. Genetic analyses reveal that voltage-gated channels in the neurons are required for the response, possibly as the sensors, and neuropeptide signaling regulates the duration of the persistent response. Additional behavioral analyses implicate that the animal's response to electric shock is scalable and has a negative valence. These properties, along with persistence, have been recently regarded as essential features of emotion, suggesting that C. elegans response to electric shock may reflect a form of emotion, akin to fear.

The molecular basis of tumor metastasis and current approaches to decode targeted migration-promoting events in pediatric neuroblastoma

Cell motility is a crucial biological process that plays a critical role in the development of multicellular organisms and is essential for tissue formation and regeneration. However, uncontrolled cell motility can lead to the development of various diseases, including neoplasms. In this review, we discuss recent advances in the discovery of regulatory mechanisms underlying the metastatic spread of neuroblastoma, a solid pediatric tumor that originates in the embryonic migratory cells of the neural crest. The highly motile phenotype of metastatic neuroblastoma cells requires targeting of intracellular and extracellular processes, that, if affected, would be helpful for the treatment of high-risk patients with neuroblastoma, for whom current therapies remain inadequate. Development of new potentially migration-inhibiting compounds and standardized preclinical approaches for the selection of anti-metastatic drugs in neuroblastoma will also be discussed.

MIA is an open-source standalone deep learning application for microscopic image analysis

In recent years, the amount of data generated by imaging techniques has grown rapidly, along with increasing computational power and the development of deep learning algorithms. To address the need for powerful automated image analysis tools for a broad range of applications in the biomedical sciences, the Microscopic Image Analyzer (MIA) was developed. MIA combines a graphical user interface that obviates the need for programming skills with state-of-the-art deep-learning algorithms for segmentation, object detection, and classification. It runs as a standalone, platform-independent application and uses open data formats, which are compatible with commonly used open-source software packages. The software provides a unified interface for easy image labeling, model training, and inference. Furthermore, the software was evaluated in a public competition and performed among the top three for all tested datasets.

4D Light-sheet imaging and interactive analysis of cardiac contractility in zebrafish larvae

Despite ongoing efforts in cardiovascular research, the acquisition of high-resolution and high-speed images for the purpose of assessing cardiac contraction remains challenging. Light-sheet fluorescence microscopy (LSFM) offers superior spatiotemporal resolution and minimal photodamage, providing an indispensable opportunity for the in vivo study of cardiac micro-structure and contractile function in zebrafish larvae. To track the myocardial architecture and contractility, we have developed an imaging strategy ranging from LSFM system construction, retrospective synchronization, single cell tracking, to user-directed virtual reality (VR) analysis. Our system enables the four-dimensional (4D) investigation of individual cardiomyocytes across the entire atrium and ventricle during multiple cardiac cycles in a zebrafish larva at the cellular resolution. To enhance the throughput of our model reconstruction and assessment, we have developed a parallel computing-assisted algorithm for 4D synchronization, resulting in a nearly tenfold enhancement of reconstruction efficiency. The machine learning-based nuclei segmentation and VR-based interaction further allow us to quantify cellular dynamics in the myocardium from end-systole to end-diastole. Collectively, our strategy facilitates noninvasive cardiac imaging and user-directed data interpretation with improved efficiency and accuracy, holding great promise to characterize functional changes and regional mechanics at the single cell level during cardiac development and regeneration.

Integration of immune cells in organs-on-chips: a tutorial

Viral and bacterial infections continue to pose significant challenges for numerous individuals globally. To develop novel therapies to combat infections, more insight into the actions of the human innate and adaptive immune system during infection is necessary. Human in vitro models, such as organs-on-chip (OOC) models, have proven to be a valuable addition to the tissue modeling toolbox. The incorporation of an immune component is needed to bring OOC models to the next level and enable them to mimic complex biological responses. The immune system affects many (patho)physiological processes in the human body, such as those taking place during an infection. This tutorial review introduces the reader to the building blocks of an OOC model of acute infection to investigate recruitment of circulating immune cells into the infected tissue. The multi-step extravasation cascade in vivo is described, followed by an in-depth guide on how to model this process on a chip. Next to chip design, creation of a chemotactic gradient and incorporation of endothelial, epithelial, and immune cells, the review focuses on the hydrogel extracellular matrix (ECM) to accurately model the interstitial space through which extravasated immune cells migrate towards the site of infection. Overall, this tutorial review is a practical guide for developing an OOC model of immune cell migration from the blood into the interstitial space during infection.

A review of open-source image analysis tools for mammalian cell culture: algorithms, features and implementations

Cell culture is undeniably important for multiple scientific applications, including pharmaceuticals, transplants, and cosmetics. However, cell culture involves multiple manual steps, such as regularly analyzing cell images for their health and morphology. Computer scientists have developed algorithms to automate cell imaging analysis, but they are not widely adopted by biologists, especially those lacking an interactive platform. To address the issue, we compile and review existing open-source cell image processing tools that provide interactive interfaces for management and prediction tasks. We highlight the prediction tools that can detect, segment, and track different mammalian cell morphologies across various image modalities and present a comparison of algorithms and unique features of these tools, whether they work locally or in the cloud. This would guide non-experts to determine which is best suited for their purposes and, developers to acknowledge what is worth further expansion. In addition, we provide a general discussion on potential implementations of the tools for a more extensive scope, which guides the reader to not restrict them to prediction tasks only. Finally, we conclude the article by stating new considerations for the development of interactive cell imaging tools and suggesting new directions for future research.

Automated assessment of human engineered heart tissues using deep learning and template matching for segmentation and tracking

The high rate of drug withdrawal from the market due to cardiovascular toxicity or lack of efficacy, the economic burden, and extremely long time before a compound reaches the market, have increased the relevance of human in vitro models like human (patient-derived) pluripotent stem cell (hPSC)-derived engineered heart tissues (EHTs) for the evaluation of the efficacy and toxicity of compounds at the early phase in the drug development pipeline. Consequently, the EHT contractile properties are highly relevant parameters for the analysis of cardiotoxicity, disease phenotype, and longitudinal measurements of cardiac function over time. In this study, we developed and validated the software HAARTA (Highly Accurate, Automatic and Robust Tracking Algorithm), which automatically analyzes contractile properties of EHTs by segmenting and tracking brightfield videos, using deep learning and template matching with sub-pixel precision. We demonstrate the robustness, accuracy, and computational efficiency of the software by comparing it to the state-of-the-art method (MUSCLEMOTION), and by testing it with a data set of EHTs from three different hPSC lines. HAARTA will facilitate standardized analysis of contractile properties of EHTs, which will be beneficial for in vitro drug screening and longitudinal measurements of cardiac function.

Microsystem Advances through Integration with Artificial Intelligence

Microfluidics is a rapidly growing discipline that involves studying and manipulating fluids at reduced length scale and volume, typically on the scale of micro- or nanoliters. Under the reduced length scale and larger surface-to-volume ratio, advantages of low reagent consumption, faster reaction kinetics, and more compact systems are evident in microfluidics. However, miniaturization of microfluidic chips and systems introduces challenges of stricter tolerances in designing and controlling them for interdisciplinary applications. Recent advances in artificial intelligence (AI) have brought innovation to microfluidics from design, simulation, automation, and optimization to bioanalysis and data analytics. In microfluidics, the Navier-Stokes equations, which are partial differential equations describing viscous fluid motion that in complete form are known to not have a general analytical solution, can be simplified and have fair performance through numerical approximation due to low inertia and laminar flow. Approximation using neural networks trained by rules of physical knowledge introduces a new possibility to predict the physicochemical nature. The combination of microfluidics and automation can produce large amounts of data, where features and patterns that are difficult to discern by a human can be extracted by machine learning. Therefore, integration with AI introduces the potential to revolutionize the microfluidic workflow by enabling the precision control and automation of data analysis. Deployment of smart microfluidics may be tremendously beneficial in various applications in the future, including high-throughput drug discovery, rapid point-of-care-testing (POCT), and personalized medicine. In this review, we summarize key microfluidic advances integrated with AI and discuss the outlook and possibilities of combining AI and microfluidics.

Segmentation-based tracking of macrophages in 2D＋time microscopy movies inside a living animal

The automated segmentation and tracking of macrophages during their migration are challenging tasks due to their dynamically changing shapes and motions. This paper proposes a new algorithm to achieve automatic cell tracking in time-lapse microscopy macrophage data. First, we design a segmentation method employing space-time filtering, local Otsu's thresholding, and the SUBSURF (subjective surface segmentation) method. Next, the partial trajectories for cells overlapping in the temporal direction are extracted in the segmented images. Finally, the extracted trajectories are linked by considering their direction of movement. The segmented images and the obtained trajectories from the proposed method are compared with those of the semi-automatic segmentation and manual tracking. The proposed tracking achieved 97.4% of accuracy for macrophage data under challenging situations, feeble fluorescent intensity, irregular shapes, and motion of macrophages. We expect that the automatically extracted trajectories of macrophages can provide pieces of evidence of how macrophages migrate depending on their polarization modes in the situation, such as during wound healing.

Tracking unlabeled cancer cells imaged with low resolution in wide migration chambers via U-NET class-1 probability (pseudofluorescence)

Cell migration is a pivotal biological process, whose dysregulation is found in many diseases including inflammation and cancer. Advances in microscopy technologies allow now to study cell migration in vitro, within engineered microenvironments that resemble in vivo conditions. However, to capture an entire 3D migration chamber for extended periods of time and with high temporal resolution, images are generally acquired with low resolution, which poses a challenge for data analysis. Indeed, cell detection and tracking are hampered due to the large pixel size (i.e., cell diameter down to 2 pixels), the possible low signal-to-noise ratio, and distortions in the cell shape due to changes in the z-axis position. Although fluorescent staining can be used to facilitate cell detection, it may alter cell behavior and it may suffer from fluorescence loss over time (photobleaching).Here we describe a protocol that employs an established deep learning method (U-NET), to specifically convert transmitted light (TL) signal from unlabeled cells imaged with low resolution to a fluorescent-like signal (class 1 probability). We demonstrate its application to study cancer cell migration, obtaining a significant improvement in tracking accuracy, while not suffering from photobleaching. This is reflected in the possibility of tracking cells for three-fold longer periods of time. To facilitate the application of the protocol we provide WID-U, an open-source plugin for FIJI and Imaris imaging software, the training dataset used in this paper, and the code to train the network for custom experimental settings.

BCM3D 2.0: accurate segmentation of single bacterial cells in dense biofilms using computationally generated intermediate image representations

Accurate detection and segmentation of single cells in three-dimensional (3D) fluorescence time-lapse images is essential for observing individual cell behaviors in large bacterial communities called biofilms. Recent progress in machine-learning-based image analysis is providing this capability with ever-increasing accuracy. Leveraging the capabilities of deep convolutional neural networks (CNNs), we recently developed bacterial cell morphometry in 3D (BCM3D), an integrated image analysis pipeline that combines deep learning with conventional image analysis to detect and segment single biofilm-dwelling cells in 3D fluorescence images. While the first release of BCM3D (BCM3D 1.0) achieved state-of-the-art 3D bacterial cell segmentation accuracies, low signal-to-background ratios (SBRs) and images of very dense biofilms remained challenging. Here, we present BCM3D 2.0 to address this challenge. BCM3D 2.0 is entirely complementary to the approach utilized in BCM3D 1.0. Instead of training CNNs to perform voxel classification, we trained CNNs to translate 3D fluorescence images into intermediate 3D image representations that are, when combined appropriately, more amenable to conventional mathematical image processing than a single experimental image. Using this approach, improved segmentation results are obtained even for very low SBRs and/or high cell density biofilm images. The improved cell segmentation accuracies in turn enable improved accuracies of tracking individual cells through 3D space and time. This capability opens the door to investigating time-dependent phenomena in bacterial biofilms at the cellular level.

Computational Analysis of Cardiac Contractile Function

PURPOSE OF REVIEW

Heart failure results in the high incidence and mortality all over the world. Mechanical properties of myocardium are critical determinants of cardiac function, with regional variations in myocardial contractility demonstrated within infarcted ventricles. Quantitative assessment of cardiac contractile function is therefore critical to identify myocardial infarction for the early diagnosis and therapeutic intervention.

RECENT FINDINGS

Current advancement of cardiac functional assessments is in pace with the development of imaging techniques. The methods tailored to advanced imaging have been widely used in cardiac magnetic resonance, echocardiography, and optical microscopy. In this review, we introduce fundamental concepts and applications of representative methods for each imaging modality used in both fundamental research and clinical investigations. All these methods have been designed or developed to quantify time-dependent 2-dimensional (2D) or 3D cardiac mechanics, holding great potential to unravel global or regional myocardial deformation and contractile function from end-systole to end-diastole. Computational methods to assess cardiac contractile function provide a quantitative insight into the analysis of myocardial mechanics during cardiac development, injury, and remodeling.

Rapid detection and recognition of whole brain activity in a freely behaving Caenorhabditis elegans

Advanced volumetric imaging methods and genetically encoded activity indicators have permitted a comprehensive characterization of whole brain activity at single neuron resolution in Caenorhabditis elegans. The constant motion and deformation of the nematode nervous system, however, impose a great challenge for consistent identification of densely packed neurons in a behaving animal. Here, we propose a cascade solution for long-term and rapid recognition of head ganglion neurons in a freely moving C. elegans. First, potential neuronal regions from a stack of fluorescence images are detected by a deep learning algorithm. Second, 2-dimensional neuronal regions are fused into 3-dimensional neuron entities. Third, by exploiting the neuronal density distribution surrounding a neuron and relative positional information between neurons, a multi-class artificial neural network transforms engineered neuronal feature vectors into digital neuronal identities. With a small number of training samples, our bottom-up approach is able to process each volume—1024 × 1024 × 18 in voxels—in less than 1 second and achieves an accuracy of 91% in neuronal detection and above 80% in neuronal tracking over a long video recording. Our work represents a step towards rapid and fully automated algorithms for decoding whole brain activity underlying naturalistic behaviors.

An important question in neuroscience is to understand the relationship between brain dynamics and naturalistic behaviors when an animal is freely exploring its environment. In the last decade, it has become possible to genetically engineer animals whose neurons produce fluorescence reporters that change their brightness in response to brain activity. In small animals such as the nematode C. elegans, we can now record the fluorescence changes in and thereby infer neural activity from most neurons in the head of a worm, when the animal is freely moving. These neurons are densely packed in a small volume. Since the brain and body are moving and its shape undergoes significant deformation, a human expert, even after long hours of inspection, may still have difficulty to tell where the neurons are and who they are.

We sought to develop an automatic method for rapidly detecting and tracking most of these neurons in a moving animal. To do this, we asked a human expert to annotate all head neurons—their locations and digital identities—across a small number of volumes. Then, we trained a computer to learn the locations and digital identities of these neurons across different imaging volumes. Our machine inference method is fast and accurate. While it takes a human expert several hours to complete a sequence of volumes, a machine can finish the task in a few minutes. We hope our method provides a better and more efficient engine for extracting knowledge from whole brain imaging datasets and animal behaviors.

Imaging approaches for monitoring three-dimensional cell and tissue culture systems

The past decade has seen an increasing demand for more complex, reproducible and physiologically relevant tissue cultures that can mimic the structural and biological features of living tissues. Monitoring the viability, development and responses of such tissues in real-time are challenging due to the complexities of cell culture physical characteristics and the environments in which these cultures need to be maintained in. Significant developments in optics, such as optical manipulation, improved detection and data analysis, have made optical imaging a preferred choice for many three-dimensional (3D) cell culture monitoring applications. The aim of this review is to discuss the challenges associated with imaging and monitoring 3D tissues and cell culture, and highlight topical label-free imaging tools that enable bioengineers and biophysicists to non-invasively characterise engineered living tissues.

Toward a more accurate 3D atlas of C. elegans neurons

Background

Determining cell identity in volumetric images of tagged neuronal nuclei is an ongoing challenge in contemporary neuroscience. Frequently, cell identity is determined by aligning and matching tags to an “atlas” of labeled neuronal positions and other identifying characteristics. Previous analyses of such C. elegans datasets have been hampered by the limited accuracy of such atlases, especially for neurons present in the ventral nerve cord, and also by time-consuming manual elements of the alignment process.

Results

We present a novel automated alignment method for sparse and incomplete point clouds of the sort resulting from typical C. elegans fluorescence microscopy datasets. This method involves a tunable learning parameter and a kernel that enforces biologically realistic deformation. We also present a pipeline for creating alignment atlases from datasets of the recently developed NeuroPAL transgene. In combination, these advances allow us to label neurons in volumetric images with confidence much higher than previous methods.

Conclusions

We release, to the best of our knowledge, the most complete full-body C. elegans 3D positional neuron atlas, incorporating positional variability derived from at least 7 animals per neuron, for the purposes of cell-type identity prediction for myriad applications (e.g., imaging neuronal activity, gene expression, and cell-fate).

Bioimaging approaches for quantification of individual cell behavior during cell fate decisions

Tracking individual cells has allowed a new understanding of cellular behavior in human health and disease by adding a dynamic component to the already complex heterogeneity of single cells. Technically, despite countless advances, numerous experimental variables can affect data collection and interpretation and need to be considered. In this review, we discuss the main technical aspects and biological findings in the analysis of the behavior of individual cells. We discuss the most relevant contributions provided by these approaches in clinically relevant human conditions like embryo development, stem cells biology, inflammation, cancer and microbiology, along with the cellular mechanisms and molecular pathways underlying these conditions. We also discuss the key technical aspects to be considered when planning and performing experiments involving the analysis of individual cells over long periods. Despite the challenges in automatic detection, features extraction and long-term tracking that need to be tackled, the potential impact of single-cell bioimaging is enormous in understanding the pathogenesis and development of new therapies in human pathophysiology.

Live Plant Cell Tracking: Fiji plugin to analyze cell proliferation dynamics and understand morphogenesis

Arabidopsis (Arabidopsis thaliana) primary and lateral roots (LRs) are well suited for 3D and 4D microscopy, and their development provides an ideal system for studying morphogenesis and cell proliferation dynamics. With fast-advancing microscopy techniques used for live-imaging, whole tissue data are increasingly available, yet present the great challenge of analyzing complex interactions within cell populations. We developed a plugin "Live Plant Cell Tracking" (LiPlaCeT) coupled to the publicly available ImageJ image analysis program and generated a pipeline that allows, with the aid of LiPlaCeT, 4D cell tracking and lineage analysis of populations of dividing and growing cells. The LiPlaCeT plugin contains ad hoc ergonomic curating tools, making it very simple to use for manual cell tracking, especially when the signal-to-noise ratio of images is low or variable in time or 3D space and when automated methods may fail. Performing time-lapse experiments and using cell-tracking data extracted with the assistance of LiPlaCeT, we accomplished deep analyses of cell proliferation and clonal relations in the whole developing LR primordia and constructed genealogical trees. We also used cell-tracking data for endodermis cells of the root apical meristem (RAM) and performed automated analyses of cell population dynamics using ParaView software (also publicly available). Using the RAM as an example, we also showed how LiPlaCeT can be used to generate information at the whole-tissue level regarding cell length, cell position, cell growth rate, cell displacement rate, and proliferation activity. The pipeline will be useful in live-imaging studies of roots and other plant organs to understand complex interactions within proliferating and growing cell populations. The plugin includes a step-by-step user manual and a dataset example that are available at https://www.ibt.unam.mx/documentos/diversos/LiPlaCeT.zip.

Tracking cell lineages in 3D by incremental deep learning

Deep learning is emerging as a powerful approach for bioimage analysis. Its use in cell tracking is limited by the scarcity of annotated data for the training of deep-learning models. Moreover, annotation, training, prediction, and proofreading currently lack a unified user interface. We present ELEPHANT, an interactive platform for 3D cell tracking that addresses these challenges by taking an incremental approach to deep learning. ELEPHANT provides an interface that seamlessly integrates cell track annotation, deep learning, prediction, and proofreading. This enables users to implement cycles of incremental learning starting from a few annotated nuclei. Successive prediction-validation cycles enrich the training data, leading to rapid improvements in tracking performance. We test the software's performance against state-of-the-art methods and track lineages spanning the entire course of leg regeneration in a crustacean over 1 week (504 timepoints). ELEPHANT yields accurate, fully-validated cell lineages with a modest investment in time and effort.

Neurophotonic tools for microscopic measurements and manipulation: status report

Neurophotonics was launched in 2014 coinciding with the launch of the BRAIN Initiative focused on development of technologies for advancement of neuroscience. For the last seven years, Neurophotonics' agenda has been well aligned with this focus on neurotechnologies featuring new optical methods and tools applicable to brain studies. While the BRAIN Initiative 2.0 is pivoting towards applications of these novel tools in the quest to understand the brain, this status report reviews an extensive and diverse toolkit of novel methods to explore brain function that have emerged from the BRAIN Initiative and related large-scale efforts for measurement and manipulation of brain structure and function. Here, we focus on neurophotonic tools mostly applicable to animal studies. A companion report, scheduled to appear later this year, will cover diffuse optical imaging methods applicable to noninvasive human studies. For each domain, we outline the current state-of-the-art of the respective technologies, identify the areas where innovation is needed, and provide an outlook for the future directions.

Deep learning is widely applicable to phenotyping embryonic development and disease

Genome editing simplifies the generation of new animal models for congenital disorders. However, the detailed and unbiased phenotypic assessment of altered embryonic development remains a challenge. Here, we explore how deep learning (U-Net) can automate segmentation tasks in various imaging modalities, and we quantify phenotypes of altered renal, neural and craniofacial development in Xenopus embryos in comparison with normal variability. We demonstrate the utility of this approach in embryos with polycystic kidneys (pkd1 and pkd2) and craniofacial dysmorphia (six1). We highlight how in toto light-sheet microscopy facilitates accurate reconstruction of brain and craniofacial structures within X. tropicalis embryos upon dyrk1a and six1 loss of function or treatment with retinoic acid inhibitors. These tools increase the sensitivity and throughput of evaluating developmental malformations caused by chemical or genetic disruption. Furthermore, we provide a library of pre-trained networks and detailed instructions for applying deep learning to the reader's own datasets. We demonstrate the versatility, precision and scalability of deep neural network phenotyping on embryonic disease models. By combining light-sheet microscopy and deep learning, we provide a framework for higher-throughput characterization of embryonic model organisms. This article has an associated 'The people behind the papers' interview.

Avoiding a replication crisis in deep-learning-based bioimage analysis

Deep learning algorithms are powerful tools to analyse, restore and transform bioimaging data, increasingly used in life sciences research. These approaches now outperform most other algorithms for a broad range of image analysis tasks. In particular, one of the promises of deep learning is the possibility to provide parameter-free, one-click data analysis achieving expert-level performances in a fraction of the time previously required. However, as with most new and upcoming technologies, the potential for inappropriate use is raising concerns among the biomedical research community. This perspective aims to provide a short overview of key concepts that we believe are important for researchers to consider when using deep learning for their microscopy studies. These comments are based on our own experience gained while optimising various deep learning tools for bioimage analysis and discussions with colleagues from both the developer and user community. In particular, we focus on describing how results obtained using deep learning can be validated and discuss what should, in our views, be considered when choosing a suitable tool. We also suggest what aspects of a deep learning analysis would need to be reported in publications to describe the use of such tools to guarantee that the work can be reproduced. We hope this perspective will foster further discussion between developers, image analysis specialists, users and journal editors to define adequate guidelines and ensure that this transformative technology is used appropriately.

Deep learning for bioimage analysis in developmental biology

Deep learning has transformed the way large and complex image datasets can be processed, reshaping what is possible in bioimage analysis. As the complexity and size of bioimage data continues to grow, this new analysis paradigm is becoming increasingly ubiquitous. In this Review, we begin by introducing the concepts needed for beginners to understand deep learning. We then review how deep learning has impacted bioimage analysis and explore the open-source resources available to integrate it into a research project. Finally, we discuss the future of deep learning applied to cell and developmental biology. We analyze how state-of-the-art methodologies have the potential to transform our understanding of biological systems through new image-based analysis and modelling that integrate multimodal inputs in space and time.