Synaptic Cleft Segmentation in Non-isotropic Volume Electron Microscopy of the Complete Drosophila Brain

Compensatory enhancement of input maintains aversive dopaminergic reinforcement in hungry Drosophila

Hungry animals need compensatory mechanisms to maintain flexible brain function, while modulation reconfigures circuits to prioritize resource seeking. In Drosophila, hunger inhibits aversively reinforcing dopaminergic neurons (DANs) to permit the expression of food-seeking memories. Multitasking the reinforcement system for motivation potentially undermines aversive learning. We find that chronic hunger mildly enhances aversive learning and that satiated-baseline and hunger-enhanced learning require endocrine adipokinetic hormone (AKH) signaling. Circulating AKH influences aversive learning via its receptor in four neurons in the ventral brain, two of which are octopaminergic. Connectomics revealed AKH receptor-expressing neurons to be upstream of several classes of ascending neurons, many of which are presynaptic to aversively reinforcing DANs. Octopaminergic modulation of and output from at least one of these ascending pathways is required for shock- and bitter-taste-reinforced aversive learning. We propose that coordinated enhancement of input compensates for hunger-directed inhibition of aversive DANs to preserve reinforcement when required.

Divergent neural circuits for proprioceptive and exteroceptive sensing of the Drosophila leg

Somatosensory neurons provide the nervous system with information about mechanical forces originating inside and outside the body. Here, we use connectomics to reconstruct and analyze neural circuits downstream of the largest somatosensory organ in the Drosophila leg, the femoral chordotonal organ (FeCO). The FeCO has been proposed to support both proprioceptive sensing of the fly's femur-tibia joint and exteroceptive sensing of substrate vibrations, but it remains unknown which sensory neurons and central circuits contribute to each of these functions. We found that different subtypes of FeCO sensory neurons feed into distinct proprioceptive and exteroceptive pathways. Position- and movement-encoding FeCO neurons connect to local leg motor control circuits in the ventral nerve cord (VNC), indicating a proprioceptive function. In contrast, signals from the vibration-encoding FeCO neurons are integrated across legs and transmitted to auditory regions in the brain, indicating an exteroceptive function. Overall, our analyses reveal the structure of specialized circuits for processing proprioceptive and exteroceptive signals from the fly leg. They also demonstrate how analyzing patterns of synaptic connectivity can distill organizing principles from complex sensorimotor circuits.

Somatotopic organization among parallel sensory pathways that promote a grooming sequence in Drosophila

Mechanosensory neurons located across the body surface respond to tactile stimuli and elicit diverse behavioral responses, from relatively simple stimulus location-aimed movements to complex movement sequences. How mechanosensory neurons and their postsynaptic circuits influence such diverse behaviors remains unclear. We previously discovered that Drosophila perform a body location-prioritized grooming sequence when mechanosensory neurons at different locations on the head and body are simultaneously stimulated by dust (Hampel et al., 2017; Seeds et al., 2014). Here, we identify nearly all mechanosensory neurons on the Drosophila head that individually elicit aimed grooming of specific head locations, while collectively eliciting a whole head grooming sequence. Different tracing methods were used to reconstruct the projections of these neurons from different locations on the head to their distinct arborizations in the brain. This provides the first synaptic resolution somatotopic map of a head, and defines the parallel-projecting mechanosensory pathways that elicit head grooming.

Neurons underlying aggressive actions that are shared by both males and females in Drosophila

Aggression involves both sexually monomorphic and dimorphic actions. How the brain implements these two types of actions is poorly understood. We found that a set of neurons, which we call CL062, previously shown to mediate male aggression also mediate female aggression. These neurons elicit aggression acutely and without the presence of a target. Although the same set of actions is elicited in males and females, the overall behavior is sexually dimorphic. The CL062 neurons do not express fruitless , a gene required for sexual dimorphism in flies, and expressed by most other neurons important for controlling fly aggression. Connectomic analysis suggests that these neurons have limited connections with fruitless expressing neurons that have been shown to be important for aggression, and signal to different descending neurons. Thus, CL062 is part of a monomorphic circuit for aggression that functions parallel to the known dimorphic circuits.

Somatotopic organization among parallel sensory pathways that promote a grooming sequence in Drosophila

Mechanosensory neurons located across the body surface respond to tactile stimuli and elicit diverse behavioral responses, from relatively simple stimulus location-aimed movements to complex movement sequences. How mechanosensory neurons and their postsynaptic circuits influence such diverse behaviors remains unclear. We previously discovered that Drosophila perform a body location-prioritized grooming sequence when mechanosensory neurons at different locations on the head and body are simultaneously stimulated by dust (Hampel et al., 2017; Seeds et al., 2014). Here, we identify nearly all mechanosensory neurons on the Drosophila head that individually elicit aimed grooming of specific head locations, while collectively eliciting a whole head grooming sequence. Different tracing methods were used to reconstruct the projections of these neurons from different locations on the head to their distinct arborizations in the brain. This provides the first synaptic resolution somatotopic map of a head, and defines the parallel-projecting mechanosensory pathways that elicit head grooming.

Isochronic development of cortical synapses in primates and mice

The neotenous, or delayed, development of primate neurons, particularly human ones, is thought to underlie primate-specific abilities like cognition. We tested whether synaptic development follows suit-would synapses, in absolute time, develop slower in longer-lived, highly cognitive species like non-human primates than in shorter-lived species with less human-like cognitive abilities, e.g., the mouse? Instead, we find that excitatory and inhibitory synapses in the male Mus musculus (mouse) and Rhesus macaque (primate) cortex form at similar rates, at similar times after birth. Primate excitatory and inhibitory synapses and mouse excitatory synapses also prune in such an isochronic fashion. Mouse inhibitory synapses are the lone exception, which are not pruned and instead continuously added throughout life. The monotony of synaptic development clocks across species with disparate lifespans, experiences, and cognitive abilities argues that such programs are likely orchestrated by genetic events rather than experience.

Fine-grained descending control of steering in walking Drosophila

Locomotion involves rhythmic limb movement patterns that originate in circuits outside the brain. Purposeful locomotion requires descending commands from the brain, but we do not understand how these commands are structured. Here we investigate this issue, focusing on the control of steering in walking Drosophila. First, we describe different limb "gestures" associated with different steering maneuvers. Next, we identify a set of descending neurons whose activity predicts steering. Focusing on two descending cell types downstream from distinct brain networks, we show that they evoke specific limb gestures: one lengthens strides on the outside of a turn, while the other attenuates strides on the inside of a turn. Notably, a single descending neuron can have opposite effects during different locomotor rhythm phases, and we identify networks positioned to implement this phase-specific gating. Together, our results show how purposeful locomotion emerges from brain cells that drive specific, coordinated modulations of low-level patterns.

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer that represents olfactory stimuli and links them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their tuning and function are poorly understood. Here, we use the FlyWire adult whole-brain connectome to identify inputs to visual Kenyon cells. The types of visual neurons we identify are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual projection neurons presynaptic to Kenyon cells receive input from large swathes of visual space, while local visual interneurons, providing smaller fractions of input, receive more spatially restricted signals that may be tuned to specific features of the visual scene. Like olfactory Kenyon cells, visual Kenyon cells receive sparse inputs from different combinations of visual channels, including inputs from multiple optic lobe neuropils. The sets of inputs to individual visual Kenyon cells are consistent with random sampling of available inputs. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the expansion coding properties appear different, with a specific repertoire of visual inputs projecting onto a relatively small number of visual Kenyon cells.

Descending control and regulation of spontaneous flight turns in Drosophila

The clumped distribution of resources in the world has influenced the pattern of foraging behavior since the origins of life, selecting for a common locomotor search motif in which straight movements through resource-poor regions alternate with zig -zag exploration in resource-rich domains. For example, flies execute rapid changes in flight heading called body saccades during local search, but suppress these turns during long-distance dispersal or when surging upwind after encountering an attractive odor plume. Here, we describe the key cellular components of a neural network in flies that generates spontaneous turns as well as a specialized neuron that inhibits the network to promote straight flight. Using 2-photon imaging, optogenetic activation, and genetic ablation, we show that only four descending neurons appear sufficient to generate the descending commands to execute flight saccades. The network is organized into two functional couplets-one for right turns and one for left-with each couplet consisting of an excitatory (DNae014) and inhibitory (DNb01) neuron that project to the flight motor neuropil within the ventral nerve cord. Using resources from recently published connectomes of the fly brain, we identified a large, unique interneuron (VES041) that forms inhibitory connections to all four saccade command neurons and created specific genetic driver lines for this cell. As suggested by its connectivity, activation of VES041 strongly suppresses saccades, suggesting that it regulates the transition between local search and long-distance dispersal. These results thus identify the critical elements of a network that not only structures the locomotor behavior of flies, but may also play a crucial role in their natural foraging ecology.

Neuronal wiring diagram of an adult brain

Connections between neurons can be mapped by acquiring and analyzing electron microscopic (EM) brain images. In recent years, this approach has been applied to chunks of brains to reconstruct local connectivity maps that are highly informative, yet inadequate for understanding brain function more globally. Here, we present the first neuronal wiring diagram of a whole adult brain, containing 5×107 chemical synapses between ~130,000 neurons reconstructed from a female Drosophila melanogaster. The resource also incorporates annotations of cell classes and types, nerves, hemilineages, and predictions of neurotransmitter identities. Data products are available by download, programmatic access, and interactive browsing and made interoperable with other fly data resources. We show how to derive a projectome, a map of projections between regions, from the connectome. We demonstrate the tracing of synaptic pathways and the analysis of information flow from inputs (sensory and ascending neurons) to outputs (motor, endocrine, and descending neurons), across both hemispheres, and between the central brain and the optic lobes. Tracing from a subset of photoreceptors all the way to descending motor pathways illustrates how structure can uncover putative circuit mechanisms underlying sensorimotor behaviors. The technologies and open ecosystem of the FlyWire Consortium set the stage for future large-scale connectome projects in other species.

Hunger- and thirst-sensing neurons modulate a neuroendocrine network to coordinate sugar and water ingestion

Consumption of food and water is tightly regulated by the nervous system to maintain internal nutrient homeostasis. Although generally considered independently, interactions between hunger and thirst drives are important to coordinate competing needs. In Drosophila , four neurons called the Interoceptive Subesophageal zone Neurons (ISNs) respond to intrinsic hunger and thirst signals to oppositely regulate sucrose and water ingestion. Here, we investigate the neural circuit downstream of the ISNs to examine how ingestion is regulated based on internal needs. Utilizing the recently available fly brain connectome, we find that the ISNs synapse with a novel cell type Bilateral T-shaped neuron (BiT) that projects to neuroendocrine centers. In vivo neural manipulations revealed that BiT oppositely regulates sugar and water ingestion. Neuroendocrine cells downstream of ISNs include several peptide-releasing and peptide-sensing neurons, including insulin producing cells (IPC), crustacean cardioactive peptide (CCAP) neurons, and CCHamide-2 receptor isoform RA (CCHa2R-RA) neurons. These neurons contribute differentially to ingestion of sugar and water, with IPCs and CCAP neurons oppositely regulating sugar and water ingestion, and CCHa2R-RA neurons modulating only water ingestion. Thus, the decision to consume sugar or water occurs via regulation of a broad peptidergic network that integrates internal signals of nutritional state to generate nutrient-specific ingestion.

A leaky integrate-and-fire computational model based on the connectome of the entire adult Drosophila brain reveals insights into sensorimotor processing

The forthcoming assembly of the adult Drosophila melanogaster central brain connectome, containing over 125,000 neurons and 50 million synaptic connections, provides a template for examining sensory processing throughout the brain. Here, we create a leaky integrate-and-fire computational model of the entire Drosophila brain, based on neural connectivity and neurotransmitter identity, to study circuit properties of feeding and grooming behaviors. We show that activation of sugar-sensing or water-sensing gustatory neurons in the computational model accurately predicts neurons that respond to tastes and are required for feeding initiation. Computational activation of neurons in the feeding region of the Drosophila brain predicts those that elicit motor neuron firing, a testable hypothesis that we validate by optogenetic activation and behavioral studies. Moreover, computational activation of different classes of gustatory neurons makes accurate predictions of how multiple taste modalities interact, providing circuit-level insight into aversive and appetitive taste processing. Our computational model predicts that the sugar and water pathways form a partially shared appetitive feeding initiation pathway, which our calcium imaging and behavioral experiments confirm. Additionally, we applied this model to mechanosensory circuits and found that computational activation of mechanosensory neurons predicts activation of a small set of neurons comprising the antennal grooming circuit that do not overlap with gustatory circuits, and accurately describes the circuit response upon activation of different mechanosensory subtypes. Our results demonstrate that modeling brain circuits purely from connectivity and predicted neurotransmitter identity generates experimentally testable hypotheses and can accurately describe complete sensorimotor transformations.

The Development of Synapses in Mouse and Macaque Primary Sensory Cortices

We report that the rate of synapse development in primary sensory cortices of mice and macaques is unrelated to lifespan, as was previously thought. We analyzed 28,084 synapses over multiple developmental time points in both species and find, instead, that net excitatory synapse development of mouse and macaque neurons primarily increased at similar rates in the first few postnatal months, and then decreased over a span of 1-1.5 years of age. The development of inhibitory synapses differed qualitatively across species. In macaques, net inhibitory synapses first increase and then decrease on excitatory soma at similar ages as excitatory synapses. In mice, however, such synapses are added throughout life. These findings contradict the long-held belief that the cycle of synapse formation and pruning occurs earlier in shorter-lived animals. Instead, our results suggest more nuanced rules, with the development of different types of synapses following different timing rules or different trajectories across species.

Local shape descriptors for neuron segmentation

We present an auxiliary learning task for the problem of neuron segmentation in electron microscopy volumes. The auxiliary task consists of the prediction of local shape descriptors (LSDs), which we combine with conventional voxel-wise direct neighbor affinities for neuron boundary detection. The shape descriptors capture local statistics about the neuron to be segmented, such as diameter, elongation, and direction. On a study comparing several existing methods across various specimen, imaging techniques, and resolutions, auxiliary learning of LSDs consistently increases segmentation accuracy of affinity-based methods over a range of metrics. Furthermore, the addition of LSDs promotes affinity-based segmentation methods to be on par with the current state of the art for neuron segmentation (flood-filling networks), while being two orders of magnitudes more efficient-a critical requirement for the processing of future petabyte-sized datasets.

Joint reconstruction of neuron and ultrastructure via connectivity consensus in electron microscope volumes

Background

Nanoscale connectomics, which aims to map the fine connections between neurons with synaptic-level detail, has attracted increasing attention in recent years. Currently, the automated reconstruction algorithms in electron microscope volumes are in great demand. Most existing reconstruction methodologies for cellular and subcellular structures are independent, and exploring the inter-relationships between structures will contribute to image analysis. The primary goal of this research is to construct a joint optimization framework to improve the accuracy and efficiency of neural structure reconstruction algorithms.

Results

In this investigation, we introduce the concept of connectivity consensus between cellular and subcellular structures based on biological domain knowledge for neural structure agglomeration problems. We propose a joint graph partitioning model for solving ultrastructural and neuronal connections to overcome the limitations of connectivity cues at different levels. The advantage of the optimization model is the simultaneous reconstruction of multiple structures in one optimization step. The experimental results on several public datasets demonstrate that the joint optimization model outperforms existing hierarchical agglomeration algorithms.

Conclusions

We present a joint optimization model by connectivity consensus to solve the neural structure agglomeration problem and demonstrate its superiority to existing methods. The intention of introducing connectivity consensus between different structures is to build a suitable optimization model that makes the reconstruction goals more consistent with biological plausible and domain knowledge. This idea can inspire other researchers to optimize existing reconstruction algorithms and other areas of biological data analysis.

How innovations in methodology offer new prospects for volume electron microscopy

Detailed knowledge of biological structure has been key in understanding biology at several levels of organisation, from organs to cells and proteins. Volume electron microscopy (volume EM) provides high resolution 3D structural information about tissues on the nanometre scale. However, the throughput rate of conventional electron microscopes has limited the volume size and number of samples that can be imaged. Recent improvements in methodology are currently driving a revolution in volume EM, making possible the structural imaging of whole organs and small organisms. In turn, these recent developments in image acquisition have created or stressed bottlenecks in other parts of the pipeline, like sample preparation, image analysis and data management. While the progress in image analysis is stunning due to the advent of automatic segmentation and server‐based annotation tools, several challenges remain. Here we discuss recent trends in volume EM, emerging methods for increasing throughput and implications for sample preparation, image analysis and data management.

Segmentation of Drug-Treated Cell Image and Mitochondrial-Oxidative Stress Using Deep Convolutional Neural Network

Most multicellular organisms require apoptosis, or programmed cell death, to function properly and survive. On the other hand, morphological and biochemical characteristics of apoptosis have remained remarkably consistent throughout evolution. Apoptosis is thought to have at least three functionally distinct phases: induction, effector, and execution. Recent studies have revealed that reactive oxygen species (ROS) and the oxidative stress could play an essential role in apoptosis. Advanced microscopic imaging techniques allow biologists to acquire an extensive amount of cell images within a matter of minutes which rule out the manual analysis of image data acquisition. The segmentation of cell images is often considered the cornerstone and central problem for image analysis. Currently, the issue of segmentation of mitochondrial cell images via deep learning receives increasing attention. The manual labeling of cell images is time-consuming and challenging to train a pro. As a courtesy method, mitochondrial cell imaging (MCI) is proposed to identify the normal, drug-treated, and diseased cells. Furthermore, cell movement (fission and fusion) is measured to evaluate disease risk. The newly proposed drug-treated, normal, and diseased image segmentation (DNDIS) algorithm can quickly segment mitochondrial cell images without supervision and further segment the highly drug-treated cells in the picture, i.e., normal, diseased, and drug-treated cells. The proposed method is based on the ResNet-50 deep learning algorithm. The dataset consists of 414 images mainly categorised into different sets (drug, diseased, and normal) used microscopically. The proposed automated segmentation method has outperformed and secured high precision (90%, 92%, and 94%); moreover, it also achieves proper training. This study will benefit medicines and diseased cell measurements in medical tests and clinical practices.

Automated Synapse Detection Method for Cerebellar Connectomics

The connectomic analyses of large-scale volumetric electron microscope (EM) images enable the discovery of hidden neural connectivity. While the technologies for neuronal reconstruction of EM images are under rapid progress, the technologies for synapse detection are lagging behind. Here, we propose a method that automatically detects the synapses in the 3D EM images, specifically for the mouse cerebellar molecular layer (CML). The method aims to accurately detect the synapses between the reconstructed neuronal fragments whose types can be identified. It extracts the contacts between the reconstructed neuronal fragments and classifies them as synaptic or non-synaptic with the help of type information and two deep learning artificial intelligences (AIs). The method can also assign the pre- and postsynaptic sides of a synapse and determine excitatory and inhibitory synapse types. The accuracy of this method is estimated to be 0.955 in F1-score for a test volume of CML containing 508 synapses. To demonstrate the usability, we measured the size and number of the synapses in the volume and investigated the subcellular connectivity between the CML neuronal fragments. The basic idea of the method to exploit tissue-specific properties can be extended to other brain regions.

FlyWire: online community for whole-brain connectomics

Due to advances in automated image acquisition and analysis, whole-brain connectomes with 100,000 or more neurons are on the horizon. Proofreading of whole-brain automated reconstructions will require many person-years of effort, due to the huge volumes of data involved. Here we present FlyWire, an online community for proofreading neural circuits in a Drosophila melanogaster brain and explain how its computational and social structures are organized to scale up to whole-brain connectomics. Browser-based three-dimensional interactive segmentation by collaborative editing of a spatially chunked supervoxel graph makes it possible to distribute proofreading to individuals located virtually anywhere in the world. Information in the edit history is programmatically accessible for a variety of uses such as estimating proofreading accuracy or building incentive systems. An open community accelerates proofreading by recruiting more participants and accelerates scientific discovery by requiring information sharing. We demonstrate how FlyWire enables circuit analysis by reconstructing and analyzing the connectome of mechanosensory neurons.

CleftNet: Augmented Deep Learning for Synaptic Cleft Detection From Brain Electron Microscopy

Detecting synaptic clefts is a crucial step to investigate the biological function of synapses. The volume electron microscopy (EM) allows the identification of synaptic clefts by photoing EM images with high resolution and fine details. Machine learning approaches have been employed to automatically predict synaptic clefts from EM images. In this work, we propose a novel and augmented deep learning model, known as CleftNet, for improving synaptic cleft detection from brain EM images. We first propose two novel network components, known as the feature augmentor and the label augmentor, for augmenting features and labels to improve cleft representations. The feature augmentor can fuse global information from inputs and learn common morphological patterns in clefts, leading to augmented cleft features. In addition, it can generate outputs with varying dimensions, making it flexible to be integrated in any deep network. The proposed label augmentor augments the label of each voxel from a value to a vector, which contains both the segmentation label and boundary label. This allows the network to learn important shape information and to produce more informative cleft representations. Based on the proposed feature augmentor and label augmentor, We build the CleftNet as a U-Net like network. The effectiveness of our methods is evaluated on both external and internal tasks. Our CleftNet currently ranks #1 on the external task of the CREMI open challenge. In addition, both quantitative and qualitative results in the internal tasks show that our method outperforms the baseline approaches significantly.

Whole-cell organelle segmentation in volume electron microscopy

Cells contain hundreds of organelles and macromolecular assemblies. Obtaining a complete understanding of their intricate organization requires the nanometre-level, three-dimensional reconstruction of whole cells, which is only feasible with robust and scalable automatic methods. Here, to support the development of such methods, we annotated up to 35 different cellular organelle classes-ranging from endoplasmic reticulum to microtubules to ribosomes-in diverse sample volumes from multiple cell types imaged at a near-isotropic resolution of 4 nm per voxel with focused ion beam scanning electron microscopy (FIB-SEM)¹. We trained deep learning architectures to segment these structures in 4 nm and 8 nm per voxel FIB-SEM volumes, validated their performance and showed that automatic reconstructions can be used to directly quantify previously inaccessible metrics including spatial interactions between cellular components. We also show that such reconstructions can be used to automatically register light and electron microscopy images for correlative studies. We have created an open data and open-source web repository, 'OpenOrganelle', to share the data, computer code and trained models, which will enable scientists everywhere to query and further improve automatic reconstruction of these datasets.

Automatic detection of synaptic partners in a whole-brain Drosophila electron microscopy data set

We develop an automatic method for synaptic partner identification in insect brains and use it to predict synaptic partners in a whole-brain electron microscopy dataset of the fruit fly. The predictions can be used to infer a connectivity graph with high accuracy, thus allowing fast identification of neural pathways. To facilitate circuit reconstruction using our results, we develop CIRCUITMAP, a user interface add-on for the circuit annotation tool CATMAID.

An unbiased template of the Drosophila brain and ventral nerve cord

The fruit fly Drosophila melanogaster is an important model organism for neuroscience with a wide array of genetic tools that enable the mapping of individual neurons and neural subtypes. Brain templates are essential for comparative biological studies because they enable analyzing many individuals in a common reference space. Several central brain templates exist for Drosophila, but every one is either biased, uses sub-optimal tissue preparation, is imaged at low resolution, or does not account for artifacts. No publicly available Drosophila ventral nerve cord template currently exists. In this work, we created high-resolution templates of the Drosophila brain and ventral nerve cord using the best-available technologies for imaging, artifact correction, stitching, and template construction using groupwise registration. We evaluated our central brain template against the four most competitive, publicly available brain templates and demonstrate that ours enables more accurate registration with fewer local deformations in shorter time.

nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation

Biomedical imaging is a driver of scientific discovery and a core component of medical care and is being stimulated by the field of deep learning. While semantic segmentation algorithms enable image analysis and quantification in many applications, the design of respective specialized solutions is non-trivial and highly dependent on dataset properties and hardware conditions. We developed nnU-Net, a deep learning-based segmentation method that automatically configures itself, including preprocessing, network architecture, training and post-processing for any new task. The key design choices in this process are modeled as a set of fixed parameters, interdependent rules and empirical decisions. Without manual intervention, nnU-Net surpasses most existing approaches, including highly specialized solutions on 23 public datasets used in international biomedical segmentation competitions. We make nnU-Net publicly available as an out-of-the-box tool, rendering state-of-the-art segmentation accessible to a broad audience by requiring neither expert knowledge nor computing resources beyond standard network training.

Two Stream Active Query Suggestion for Active Learning in Connectomics

For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.

Learning cellular morphology with neural networks

Reconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite the diversity of possible applications. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training, we infer morphology embeddings (Neuron2vec) of neuron reconstructions and train CMNs to identify glia cells in a supervised classification paradigm, which are then used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.

Fully-Automatic Synapse Prediction and Validation on a Large Data Set

Extracting a connectome from an electron microscopy (EM) data set requires identification of neurons and determination of connections (synapses) between neurons. As manual extraction of this information is very time-consuming, there has been extensive research efforts to automatically segment the neurons to help guide and eventually replace manual tracing. Until recently, there has been comparatively little research on automatic detection of the actual synapses between neurons. This discrepancy can, in part, be attributed to several factors: obtaining neuronal shapes is a prerequisite for the first step in extracting a connectome, manual tracing is much more time-consuming than annotating synapses, and neuronal contact area can be used as a proxy for synapses in determining connections. However, recent research has demonstrated that contact area alone is not a sufficient predictor of a synaptic connection. Moreover, as segmentation improved, we observed that synapse annotation consumes a more significant fraction of overall reconstruction time (upwards of 50% of total effort). This ratio will only get worse as segmentation improves, gating the overall possible speed-up. Therefore, we address this problem by developing algorithms that automatically detect presynaptic neurons and their postsynaptic partners. In particular, presynaptic structures are detected using a U-Net convolutional neural network (CNN), and postsynaptic partners are detected using a multilayer perceptron (MLP) with features conditioned on the local segmentation. This work is novel because it requires minimal amount of training, leverages advances in image segmentation directly, and provides a complete solution for polyadic synapse detection. We further introduce novel metrics to evaluate our algorithm on connectomes of meaningful size. When applied to the output of our method on EM data from Drosphila, these metrics demonstrate that a completely automatic prediction can be used to effectively characterize most of the connectivity correctly.