Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training

Unifying community whole-brain imaging datasets enables robust neuron identification and reveals determinants of neuron position in C. elegans

We develop a data harmonization approach for C. elegans volumetric microscopy data, consisting of a standardized format, pre-processing techniques, and human-in-the-loop machine-learning-based analysis tools. Using this approach, we unify a diverse collection of 118 whole-brain neural activity imaging datasets from five labs, storing these and accompanying tools in an online repository WormID (wormid.org). With this repository, we train three existing automated cell-identification algorithms, CPD, StatAtlas, and CRF_ID, to enable accuracy that generalizes across labs, recovering all human-labeled neurons in some cases. We mine this repository to identify factors that influence the developmental positioning of neurons. This growing resource of data, code, apps, and tutorials enables users to (1) study neuroanatomical organization and neural activity across diverse experimental paradigms, (2) develop and benchmark algorithms for automated neuron detection, segmentation, cell identification, tracking, and activity extraction, and (3) share data with the community and comply with data-sharing policies.

Automated cell annotation in multi-cell images using an improved CRF_ID algorithm

Cell identification is an important yet difficult process in data analysis of biological images. Previously, we developed an automated cell identification method called CRF_ID and demonstrated its high performance in Caenorhabditis elegans whole-brain images (Chaudhary et al., 2021). However, because the method was optimized for whole-brain imaging, comparable performance could not be guaranteed for application in commonly used C. elegans multi-cell images that display a subpopulation of cells. Here, we present an advancement, CRF_ID 2.0, that expands the generalizability of the method to multi-cell imaging beyond whole-brain imaging. To illustrate the application of the advance, we show the characterization of CRF_ID 2.0 in multi-cell imaging and cell-specific gene expression analysis in C. elegans. This work demonstrates that high-accuracy automated cell annotation in multi-cell imaging can expedite cell identification and reduce its subjectivity in C. elegans and potentially other biological images of various origins.

Olfactory combinatorial coding supports risk-reward decision making in C. elegans

Olfactory-driven behaviors are essential for animal survival, but mechanisms for decoding olfactory inputs remain poorly understood. We have used whole-network Ca ++ imaging to study olfactory coding in Caenorhabditis elegans. We show that the odorant 1-octanol is encoded combinatorially in the periphery as both an attractant and a repellant. These inputs are integrated centrally, and their relative strengths determine the sensitivity and valence of the behavioral response through modulation of locomotory reversals and speed. The balance of these pathways also dictates the activity of the locomotory command interneurons, which control locomotory reversals. This balance serves as a regulatory node for response modulation, allowing C. elegans to weigh opportunities and hazards in its environment when formulating behavioral responses. Thus, an odorant can be encoded simultaneously as inputs of opposite valence, focusing attention on the integration of these inputs in determining perception, response, and plasticity.

TWISP: a transgenic worm for interrogating signal propagation in Caenorhabditis elegans

Genetically encoded optical indicators and actuators of neural activity allow for all-optical investigations of signaling in the nervous system. But commonly used indicators, actuators, and expression strategies are poorly suited for systematic measurements of signal propagation at brain scale and cellular resolution. Large-scale measurements of the brain require indicators and actuators with compatible excitation spectra to avoid optical crosstalk. They must be highly expressed in every neuron but at the same time avoid lethality and permit the animal to reach adulthood. Their expression must also be compatible with additional fluorescent labels to locate and identify neurons, such as those in the NeuroPAL cell identification system. We present TWISP, a transgenic worm for interrogating signal propagation, that addresses these needs and enables optical measurements of evoked calcium activity at brain scale and cellular resolution in the nervous system of the nematode Caenorhabditis elegans. In every neuron we express a nonconventional optical actuator, the gustatory receptor homolog GUR-3 + PRDX-2, under the control of a drug-inducible system QF + hGR, and a calcium indicator GCAMP6s, in a background with additional fluorophores from the NeuroPAL cell ID system. We show that this combination, but not others tested, avoids optical crosstalk, creates strong expression in the adult, and generates stable transgenic lines for systematic measurements of signal propagation in the worm brain.

Unifying community-wide whole-brain imaging datasets enables robust automated neuron identification and reveals determinants of neuron positioning in C. elegans

We develop a data harmonization approach for C. elegans volumetric microscopy data, still or video, consisting of a standardized format, data pre-processing techniques, and a set of human-in-the-loop machine learning based analysis software tools. We unify a diverse collection of 118 whole-brain neural activity imaging datasets from 5 labs, storing these and accompanying tools in an online repository called WormID (wormid.org). We use this repository to train three existing automated cell identification algorithms to, for the first time, enable accuracy in neural identification that generalizes across labs, approaching human performance in some cases. We mine this repository to identify factors that influence the developmental positioning of neurons. To facilitate communal use of this repository, we created open-source software, code, web-based tools, and tutorials to explore and curate datasets for contribution to the scientific community. This repository provides a growing resource for experimentalists, theorists, and toolmakers to (a) study neuroanatomical organization and neural activity across diverse experimental paradigms, (b) develop and benchmark algorithms for automated neuron detection, segmentation, cell identification, tracking, and activity extraction, and (c) inform models of neurobiological development and function.

Automated segmentation and recognition of C. elegans whole-body cells

MOTIVATION

Accurate segmentation and recognition of C.elegans cells are critical for various biological studies, including gene expression, cell lineages, and cell fates analysis at single-cell level. However, the highly dense distribution, similar shapes, and inhomogeneous intensity profiles of whole-body cells in 3D fluorescence microscopy images make automatic cell segmentation and recognition a challenging task. Existing methods either rely on additional fiducial markers or only handle a subset of cells. Given the difficulty or expense associated with generating fiducial features in many experimental settings, a marker-free approach capable of reliably segmenting and recognizing C.elegans whole-body cells is highly desirable.

RESULTS

We report a new pipeline, called automated segmentation and recognition (ASR) of cells, and applied it to 3D fluorescent microscopy images of L1-stage C.elegans with 558 whole-body cells. A novel displacement vector field based deep learning model is proposed to address the problem of reliable segmentation of highly crowded cells with blurred boundary. We then realize the cell recognition by encoding and exploiting statistical priors on cell positions and structural similarities of neighboring cells. To the best of our knowledge, this is the first method successfully applied to the segmentation and recognition of C.elegans whole-body cells. The ASR-segmentation module achieves an F1-score of 0.8956 on a dataset of 116 C.elegans image stacks with 64 728 cells (accuracy 0.9880, AJI 0.7813). Based on the segmentation results, the ASR recognition module achieved an average accuracy of 0.8879. We also show ASR's applicability to other cell types, e.g. platynereis and rat kidney cells.

AVAILABILITY AND IMPLEMENTATION

The code is available at https://github.com/reaneyli/ASR.

See Elegans: Simple-to-use, accurate, and automatic 3D detection of neural activity from densely packed neurons

In the emerging field of whole-brain imaging at single-cell resolution, which represents one of the new frontiers to investigate the link between brain activity and behavior, the nematode Caenorhabditis elegans offers one of the most characterized models for systems neuroscience. Whole-brain recordings consist of 3D time series of volumes that need to be processed to obtain neuronal traces. Current solutions for this task are either computationally demanding or limited to specific acquisition setups. Here, we propose See Elegans, a direct programming algorithm that combines different techniques for automatic neuron segmentation and tracking without the need for the RFP channel, and we compare it with other available algorithms. While outperforming them in most cases, our solution offers a novel method to guide the identification of a subset of head neurons based on position and activity. The built-in interface allows the user to follow and manually curate each of the processing steps. See Elegans is thus a simple-to-use interface aimed at speeding up the post-processing of volumetric calcium imaging recordings while maintaining a high level of accuracy and low computational demands. (Contact: enrico.lanza@iit.it).

Automated cell annotation in multi-cell images using an improved CRF_ID algorithm

Cell identification is an important yet difficult process in data analysis of biological images. Previously, we developed an automated cell identification method called CRF_ID and demonstrated its high performance in C. elegans whole-brain images (Chaudhary et al, 2021). However, because the method was optimized for whole-brain imaging, comparable performance could not be guaranteed for application in commonly used C. elegans multi-cell images that display a subpopulation of cells. Here, we present an advance CRF_ID 2.0 that expands the generalizability of the method to multi-cell imaging beyond whole-brain imaging. To illustrate the application of the advance, we show the characterization of CRF_ID 2.0 in multi-cell imaging and cell-specific gene expression analysis in C. elegans. This work demonstrates that high accuracy automated cell annotation in multi-cell imaging can expedite cell identification and reduce its subjectivity in C. elegans and potentially other biological images of various origins.

Unraveling the hierarchical structure of posture and muscle activity changes during mating of Caenorhabditis elegans

One goal of neurobiology is to explain how decision-making in neuromuscular circuits produces behaviors. However, two obstacles complicate such efforts: individual behavioral variability and the challenge of simultaneously assessing multiple neuronal activities during behavior. Here, we circumvent these obstacles by analyzing whole animal behavior from a library of Caenorhabditis elegans male mating recordings. The copulating males express the GCaMP calcium sensor in the muscles, allowing simultaneous recording of posture and muscle activities. Our library contains wild type and males with selective neuronal desensitization in serotonergic neurons, which include male-specific posterior cord motor/interneurons and sensory ray neurons that modulate mating behavior. Incorporating deep learning-enabled computer vision, we developed a software to automatically quantify posture and muscle activities. By modeling, the posture and muscle activity data are classified into stereotyped modules, with the behaviors represented by serial executions and transitions among the modules. Detailed analysis of the modules reveals previously unidentified subtypes of the male's copulatory spicule prodding behavior. We find that wild-type and serotonergic neurons-suppressed males had different usage preferences for those module subtypes, highlighting the requirement of serotonergic neurons in the coordinated function of some muscles. In the structure of the behavior, bi-module repeats coincide with most of the previously described copulation steps, suggesting a recursive "repeat until success/give up" program is used for each step during mating. On the other hand, the transition orders of the bi-module repeats reveal the sub-behavioral hierarchy males employ to locate and inseminate hermaphrodites.

Deep Learning-Based Cell Tracking in Deforming Organs and Moving Animals

Cell tracking is an essential step in extracting cellular signals from moving cells, which is vital for understanding the mechanisms underlying various biological functions and processes, particularly in organs such as the brain and heart. However, cells in living organisms often exhibit extensive and complex movements caused by organ deformation and whole-body motion. These movements pose a challenge in obtaining high-quality time-lapse cell images and tracking the intricate cell movements in the captured images. Recent advances in deep learning techniques provide powerful tools for detecting cells in low-quality images with densely packed cell populations, as well as estimating cell positions for cells undergoing large nonrigid movements. This chapter introduces the challenges of cell tracking in deforming organs and moving animals, outlines the solutions to these challenges, and presents a detailed protocol for data preparation, as well as for performing cell segmentation and tracking using the latest version of 3DeeCellTracker. This protocol is expected to enable researchers to gain deeper insights into organ dynamics and biological processes.

Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation

Reading out neuronal activity from three-dimensional (3D) functional imaging requires segmenting and tracking individual neurons. This is challenging in behaving animals if the brain moves and deforms. The traditional approach is to train a convolutional neural network with ground-truth (GT) annotations of images representing different brain postures. For 3D images, this is very labor intensive. We introduce 'targeted augmentation', a method to automatically synthesize artificial annotations from a few manual annotations. Our method ('Targettrack') learns the internal deformations of the brain to synthesize annotations for new postures by deforming GT annotations. This reduces the need for manual annotation and proofreading. A graphical user interface allows the application of the method end-to-end. We demonstrate Targettrack on recordings where neurons are labeled as key points or 3D volumes. Analyzing freely moving animals exposed to odor pulses, we uncover rich patterns in interneuron dynamics, including switching neuronal entrainment on and off.

Neural signal propagation atlas of Caenorhabditis elegans

Establishing how neural function emerges from network properties is a fundamental problem in neuroscience1. Here, to better understand the relationship between the structure and the function of a nervous system, we systematically measure signal propagation in 23,433 pairs of neurons across the head of the nematode Caenorhabditis elegans by direct optogenetic activation and simultaneous whole-brain calcium imaging. We measure the sign (excitatory or inhibitory), strength, temporal properties and causal direction of signal propagation between these neurons to create a functional atlas. We find that signal propagation differs from model predictions that are based on anatomy. Using mutants, we show that extrasynaptic signalling not visible from anatomy contributes to this difference. We identify many instances of dense-core-vesicle-dependent signalling, including on timescales of less than a second, that evoke acute calcium transients-often where no direct wired connection exists but where relevant neuropeptides and receptors are expressed. We propose that, in such cases, extrasynaptically released neuropeptides serve a similar function to that of classical neurotransmitters. Finally, our measured signal propagation atlas better predicts the neural dynamics of spontaneous activity than do models based on anatomy. We conclude that both synaptic and extrasynaptic signalling drive neural dynamics on short timescales, and that measurements of evoked signal propagation are crucial for interpreting neural function.

TWISP: A Transgenic Worm for Interrogating Signal Propagation in C. elegans

Genetically encoded optical indicators and actuators of neural activity allow for all-optical investigations of signaling in the nervous system. But commonly used indicators, actuators and expression strategies are poorly suited for systematic measurements of signal propagation at brain scale and cellular resolution. Large scale measurements of the brain require indicators and actuators with compatible excitation spectra to avoid optical crosstalk. They must be highly expressed in every neuron but at the same time avoid lethality and permit the animal to reach adulthood. And finally, their expression must be compatible with additional fluorescent labels to locate and identify neurons, such as those in the NeuroPAL cell identification system. We present TWISP, a Transgenic Worm for Interrogating Signal Propagation, that address these needs and enables optical measurements of evoked calcium activity at brain scale and cellular resolution in the nervous system of the nematode Caenorhabditis elegans. We express in every neuron a non-conventional optical actuator, the gustatory receptor homolog GUR-3+PRDX-2 under the control of a drug-inducible system QF+hGR, and calcium indicator GCAMP6s, in a background with additional fluorophores of the NeuroPAL cell ID system. We show that this combination, but not others tested, avoids optical-crosstalk, creates strong expression in the adult, and generates stable transgenic lines for systematic measurements of signal propagation in the worm brain.

Rapid detection and recognition of whole brain activity in a freely behaving Caenorhabditis elegans

Advanced volumetric imaging methods and genetically encoded activity indicators have permitted a comprehensive characterization of whole brain activity at single neuron resolution in Caenorhabditis elegans. The constant motion and deformation of the nematode nervous system, however, impose a great challenge for consistent identification of densely packed neurons in a behaving animal. Here, we propose a cascade solution for long-term and rapid recognition of head ganglion neurons in a freely moving C. elegans. First, potential neuronal regions from a stack of fluorescence images are detected by a deep learning algorithm. Second, 2-dimensional neuronal regions are fused into 3-dimensional neuron entities. Third, by exploiting the neuronal density distribution surrounding a neuron and relative positional information between neurons, a multi-class artificial neural network transforms engineered neuronal feature vectors into digital neuronal identities. With a small number of training samples, our bottom-up approach is able to process each volume—1024 × 1024 × 18 in voxels—in less than 1 second and achieves an accuracy of 91% in neuronal detection and above 80% in neuronal tracking over a long video recording. Our work represents a step towards rapid and fully automated algorithms for decoding whole brain activity underlying naturalistic behaviors.

An important question in neuroscience is to understand the relationship between brain dynamics and naturalistic behaviors when an animal is freely exploring its environment. In the last decade, it has become possible to genetically engineer animals whose neurons produce fluorescence reporters that change their brightness in response to brain activity. In small animals such as the nematode C. elegans, we can now record the fluorescence changes in and thereby infer neural activity from most neurons in the head of a worm, when the animal is freely moving. These neurons are densely packed in a small volume. Since the brain and body are moving and its shape undergoes significant deformation, a human expert, even after long hours of inspection, may still have difficulty to tell where the neurons are and who they are.

We sought to develop an automatic method for rapidly detecting and tracking most of these neurons in a moving animal. To do this, we asked a human expert to annotate all head neurons—their locations and digital identities—across a small number of volumes. Then, we trained a computer to learn the locations and digital identities of these neurons across different imaging volumes. Our machine inference method is fast and accurate. While it takes a human expert several hours to complete a sequence of volumes, a machine can finish the task in a few minutes. We hope our method provides a better and more efficient engine for extracting knowledge from whole brain imaging datasets and animal behaviors.

Cross-modality synthesis of EM time series and live fluorescence imaging

Analyses across imaging modalities allow the integration of complementary spatiotemporal information about brain development, structure, and function. However, systematic atlasing across modalities is limited by challenges to effective image alignment. We combine highly spatially resolved electron microscopy (EM) and highly temporally resolved time-lapse fluorescence microscopy (FM) to examine the emergence of a complex nervous system in Caenorhabditis elegans embryogenesis. We generate an EM time series at four classic developmental stages and create a landmark-based co-optimization algorithm for cross-modality image alignment, which handles developmental heterochrony among datasets to achieve accurate single-cell level alignment. Synthesis based on the EM series and time-lapse FM series carrying different cell-specific markers reveals critical dynamic behaviors across scales of identifiable individual cells in the emergence of the primary neuropil, the nerve ring, as well as a major sensory organ, the amphid. Our study paves the way for systematic cross-modality data synthesis in C. elegans and demonstrates a powerful approach that may be applied broadly.

Imaging whole-brain activity to understand behavior

The brain evolved to produce behaviors that help an animal inhabit the natural world. During natural behaviors, the brain is engaged in many levels of activity from the detection of sensory inputs to decision-making to motor planning and execution. To date, most brain studies have focused on small numbers of neurons that interact in limited circuits. This allows analyzing individual computations or steps of neural processing. During behavior, however, brain activity must integrate multiple circuits in different brain regions. The activities of different brain regions are not isolated, but may be contingent on one another. Coordinated and concurrent activity within and across brain areas is organized by (1) sensory information from the environment, (2) the animal's internal behavioral state, and (3) recurrent networks of synaptic and non-synaptic connectivity. Whole-brain recording with cellular resolution provides a new opportunity to dissect the neural basis of behavior, but whole-brain activity is also mutually contingent on behavior itself. This is especially true for natural behaviors like navigation, mating, or hunting, which require dynamic interaction between the animal, its environment, and other animals. In such behaviors, the sensory experience of an unrestrained animal is actively shaped by its movements and decisions. Many of the signaling and feedback pathways that an animal uses to guide behavior only occur in freely moving animals. Recent technological advances have enabled whole-brain recording in small behaving animals including nematodes, flies, and zebrafish. These whole-brain experiments capture neural activity with cellular resolution spanning sensory, decision-making, and motor circuits, and thereby demand new theoretical approaches that integrate brain dynamics with behavioral dynamics. Here, we review the experimental and theoretical methods that are being employed to understand animal behavior and whole-brain activity, and the opportunities for physics to contribute to this emerging field of systems neuroscience.