Mapping model units to visual neurons reveals population code for social behaviour

The rich variety of behaviours observed in animals arises through the interplay between sensory processing and motor control. To understand these sensorimotor transformations, it is useful to build models that predict not only neural responses to sensory input1-5 but also how each neuron causally contributes to behaviour6,7. Here we demonstrate a novel modelling approach to identify a one-to-one mapping between internal units in a deep neural network and real neurons by predicting the behavioural changes that arise from systematic perturbations of more than a dozen neuronal cell types. A key ingredient that we introduce is 'knockout training', which involves perturbing the network during training to match the perturbations of the real neurons during behavioural experiments. We apply this approach to model the sensorimotor transformations of Drosophila melanogaster males during a complex, visually guided social behaviour8-11. The visual projection neurons at the interface between the optic lobe and central brain form a set of discrete channels12, and prior work indicates that each channel encodes a specific visual feature to drive a particular behaviour13,14. Our model reaches a different conclusion: combinations of visual projection neurons, including those involved in non-social behaviours, drive male interactions with the female, forming a rich population code for behaviour. Overall, our framework consolidates behavioural effects elicited from various neural perturbations into a single, unified model, providing a map from stimulus to neuronal cell type to behaviour, and enabling future incorporation of wiring diagrams of the brain15 into the model.

BIFROST: a method for registering diverse imaging datasets of the Drosophila brain

The heterogeneity of brain imaging methods in neuroscience provides rich data that cannot be captured by a single technique, and our interpretations benefit from approaches that enable easy comparison both within and across different data types. For example, comparing brain-wide neural dynamics across experiments and aligning such data to anatomical resources, such as gene expression patterns or connectomes, requires precise alignment to a common set of anatomical coordinates. However, this is challenging because registering in vivo functional imaging data to ex vivo reference atlases requires accommodating differences in imaging modality, microscope specification, and sample preparation. We overcome these challenges in Drosophila by building an in vivo reference atlas from multiphoton-imaged brains, called the Functional Drosophila Atlas (FDA). We then develop a two-step pipeline, BrIdge For Registering Over Statistical Templates (BIFROST), for transforming neural imaging data into this common space and for importing ex vivo resources such as connectomes. Using genetically labeled cell types as ground truth, we demonstrate registration with a precision of less than 10 microns. Overall, BIFROST provides a pipeline for registering functional imaging datasets in the fly, both within and across experiments.

Significance

Large-scale functional imaging experiments in Drosophila have given us new insights into neural activity in various sensory and behavioral contexts. However, precisely registering volumetric images from different studies has proven challenging, limiting quantitative comparisons of data across experiments. Here, we address this limitation by developing BIFROST, a registration pipeline robust to differences across experimental setups and datasets. We benchmark this pipeline by genetically labeling cell types in the fly brain and demonstrate sub-10 micron registration precision, both across specimens and across laboratories. We further demonstrate accurate registration between in-vivo brain volumes and ultrastructural connectomes, enabling direct structure-function comparisons in future experiments.

Neurotransmitter classification from electron microscopy images at synaptic sites in Drosophila melanogaster

High-resolution electron microscopy of nervous systems has enabled the reconstruction of synaptic connectomes. However, we do not know the synaptic sign for each connection (i.e., whether a connection is excitatory or inhibitory), which is implied by the released transmitter. We demonstrate that artificial neural networks can predict transmitter types for presynapses from electron micrographs: a network trained to predict six transmitters (acetylcholine, glutamate, GABA, serotonin, dopamine, octopamine) achieves an accuracy of 87% for individual synapses, 94% for neurons, and 91% for known cell types across a D. melanogaster whole brain. We visualize the ultrastructural features used for prediction, discovering subtle but significant differences between transmitter phenotypes. We also analyze transmitter distributions across the brain and find that neurons that develop together largely express only one fast-acting transmitter (acetylcholine, glutamate, or GABA). We hope that our publicly available predictions act as an accelerant for neuroscientific hypothesis generation for the fly.

Neuronal "parts list" and wiring diagram for a visual system

A catalog of neuronal cell types has often been called a "parts list" of the brain, and regarded as a prerequisite for understanding brain function. In the optic lobe of Drosophila, rules of connectivity between cell types have already proven essential for understanding fly vision. Here we analyze the fly connectome to complete the list of cell types intrinsic to the optic lobe, as well as the rules governing their connectivity. We more than double the list of known types. Most new cell types contain between 10 and 100 cells, and integrate information over medium distances in the visual field. Some existing type families (Tm, Li, and LPi) at least double in number of types. We introduce a new Sm interneuron family, which contains more types than any other, and three new families of cross-neuropil types. Self-consistency of cell types is demonstrated through automatic assignment of cells to types by distance in high-dimensional feature space, and further validation is provided by algorithms that select small subsets of discriminative features. Cell types with similar connectivity patterns divide into clusters that are interpretable in terms of motion, object, and color vision. Our work showcases the advantages of connectomic cell typing: complete and unbiased sampling, a rich array of features based on connectivity, and reduction of the connectome to a drastically simpler wiring diagram of cell types, with immediate relevance for brain function and development.

The role of fruitless in specifying courtship behaviors across divergent Drosophila species

Sex-specific behaviors are critical for reproduction and species survival. The sex-specifically spliced transcription factor fruitless (fru) helps establish male courtship behaviors in invertebrates. Forcing male-specific fru (fruM) splicing in Drosophila melanogaster females produces male-typical behaviors while disrupting female-specific behaviors. However, whether fru's joint role in specifying male and inhibiting female behaviors is conserved across species is unknown. We used CRISPR-Cas9 to force FruM expression in female Drosophila virilis, a species in which males and females produce sex-specific songs. In contrast to D. melanogaster, in which one fruM allele is sufficient to generate male behaviors in females, two alleles are needed in D. virilis females. D. virilis females expressing FruM maintain the ability to sing female-typical song as well as lay eggs, whereas D. melanogaster FruM females cannot lay eggs. These results reveal potential differences in fru function between divergent species and underscore the importance of studying diverse behaviors and species for understanding the genetic basis of sex differences.

Network Statistics of the Whole-Brain Connectome of Drosophila

Brains comprise complex networks of neurons and connections. Network analysis applied to the wiring diagrams of brains can offer insights into how brains support computations and regulate information flow. The completion of the first whole-brain connectome of an adult Drosophila, the largest connectome to date, containing 130,000 neurons and millions of connections, offers an unprecedented opportunity to analyze its network properties and topological features. To gain insights into local connectivity, we computed the prevalence of two- and three-node network motifs, examined their strengths and neurotransmitter compositions, and compared these topological metrics with wiring diagrams of other animals. We discovered that the network of the fly brain displays rich club organization, with a large population (30% percent of the connectome) of highly connected neurons. We identified subsets of rich club neurons that may serve as integrators or broadcasters of signals. Finally, we examined subnetworks based on 78 anatomically defined brain regions or neuropils. These data products are shared within the FlyWire Codex and will serve as a foundation for models and experiments exploring the relationship between neural activity and anatomical structure.

From connectome to effectome: learning the causal interaction map of the fly brain

A long-standing goal of neuroscience is to obtain a causal model of the nervous system. This would allow neuroscientists to explain animal behavior in terms of the dynamic interactions between neurons. The recently reported whole-brain fly connectome [1-7] specifies the synaptic paths by which neurons can affect each other but not whether, or how, they do affect each other in vivo. To overcome this limitation, we introduce a novel combined experimental and statistical strategy for efficiently learning a causal model of the fly brain, which we refer to as the "effectome". Specifically, we propose an estimator for a dynamical systems model of the fly brain that uses stochastic optogenetic perturbation data to accurately estimate causal effects and the connectome as a prior to drastically improve estimation efficiency. We then analyze the connectome to propose circuits that have the greatest total effect on the dynamics of the fly nervous system. We discover that, fortunately, the dominant circuits significantly involve only relatively small populations of neurons-thus imaging, stimulation, and neuronal identification are feasible. Intriguingly, we find that this approach also re-discovers known circuits and generates testable hypotheses about their dynamics. Overall, our analyses of the connectome provide evidence that global dynamics of the fly brain are generated by a large collection of small and often anatomically localized circuits operating, largely, independently of each other. This in turn implies that a causal model of a brain, a principal goal of systems neuroscience, can be feasibly obtained in the fly.

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

Transcriptional profiling of Drosophila male-specific P1 (pC1) neurons

In Drosophila melanogaster, the P1 (pC1) cluster of male-specific neurons both integrates sensory cues and drives or modulates behavioral programs such as courtship, in addition to contributing to a social arousal state. The behavioral function of these neurons is linked to the genes they express, which underpin their capacity for synaptic signaling, neuromodulation, and physiology. Yet, P1 (pC1) neurons have not been fully characterized at the transcriptome level. Moreover, it is unknown how the molecular landscape of P1 (pC1) neurons acutely changes after flies engage in social behaviors, where baseline P1 (pC1) neural activity is expected to increase. To address these two gaps, we use single cell-type RNA sequencing to profile and compare the transcriptomes of P1 (pC1) neurons harvested from socially paired versus solitary male flies. Compared to control transcriptome datasets, we find that P1 (pC1) neurons are enriched in 2,665 genes, including those encoding receptors, neuropeptides, and cell-adhesion molecules (dprs/DIPs). Furthermore, courtship is characterized by changes in ~300 genes, including those previously implicated in regulating behavior (e.g. DopEcR, Octβ3R, Fife, kairos, rad). Finally, we identify a suite of genes that link conspecific courtship with the innate immune system. Together, these data serve as a molecular map for future studies of an important set of higher-order and sexually-dimorphic neurons.

Flexible circuit mechanisms for context-dependent song sequencing

Sequenced behaviours, including locomotion, reaching and vocalization, are patterned differently in different contexts, enabling animals to adjust to their environments. How contextual information shapes neural activity to flexibly alter the patterning of actions is not fully understood. Previous work has indicated that this could be achieved via parallel motor circuits, with differing sensitivities to context1,2. Here we demonstrate that a single pathway operates in two regimes dependent on recent sensory history. We leverage the Drosophila song production system3 to investigate the role of several neuron types4-7 in song patterning near versus far from the female fly. Male flies sing 'simple' trains of only one mode far from the female fly but complex song sequences comprising alternations between modes when near her. We find that ventral nerve cord (VNC) circuits are shaped by mutual inhibition and rebound excitability8 between nodes driving the two song modes. Brief sensory input to a direct brain-to-VNC excitatory pathway drives simple song far from the female, whereas prolonged input enables complex song production via simultaneous recruitment of functional disinhibition of VNC circuitry. Thus, female proximity unlocks motor circuit dynamics in the correct context. We construct a compact circuit model to demonstrate that the identified mechanisms suffice to replicate natural song dynamics. These results highlight how canonical circuit motifs8,9 can be combined to enable circuit flexibility required for dynamic communication.

Neuronal wiring diagram of an adult brain

Connections between neurons can be mapped by acquiring and analyzing electron microscopic (EM) brain images. In recent years, this approach has been applied to chunks of brains to reconstruct local connectivity maps that are highly informative, yet inadequate for understanding brain function more globally. Here, we present the first neuronal wiring diagram of a whole adult brain, containing 5×107 chemical synapses between ~130,000 neurons reconstructed from a female Drosophila melanogaster. The resource also incorporates annotations of cell classes and types, nerves, hemilineages, and predictions of neurotransmitter identities. Data products are available by download, programmatic access, and interactive browsing and made interoperable with other fly data resources. We show how to derive a projectome, a map of projections between regions, from the connectome. We demonstrate the tracing of synaptic pathways and the analysis of information flow from inputs (sensory and ascending neurons) to outputs (motor, endocrine, and descending neurons), across both hemispheres, and between the central brain and the optic lobes. Tracing from a subset of photoreceptors all the way to descending motor pathways illustrates how structure can uncover putative circuit mechanisms underlying sensorimotor behaviors. The technologies and open ecosystem of the FlyWire Consortium set the stage for future large-scale connectome projects in other species.

A consensus cell type atlas from multiple connectomes reveals principles of circuit stereotypy and variation

The fruit fly Drosophila melanogaster combines surprisingly sophisticated behaviour with a highly tractable nervous system. A large part of the fly's success as a model organism in modern neuroscience stems from the concentration of collaboratively generated molecular genetic and digital resources. As presented in our FlyWire companion paper ¹ , this now includes the first full brain connectome of an adult animal. Here we report the systematic and hierarchical annotation of this ∼130,000-neuron connectome including neuronal classes, cell types and developmental units (hemilineages). This enables any researcher to navigate this huge dataset and find systems and neurons of interest, linked to the literature through the Virtual Fly Brain database ² . Crucially, this resource includes 4,179 cell types of which 3,166 consensus cell types are robustly defined by comparison with a second dataset, the "hemibrain" connectome ³ . Comparative analysis showed that cell type counts and strong connections were largely stable, but connection weights were surprisingly variable within and across animals. Further analysis defined simple heuristics for connectome interpretation: connections stronger than 10 unitary synapses or providing >1% of the input to a target cell are highly conserved. Some cell types showed increased variability across connectomes: the most common cell type in the mushroom body, required for learning and memory, is almost twice as numerous in FlyWire than in the hemibrain. We find evidence for functional homeostasis through adjustments of the absolute amount of excitatory input while maintaining the excitation-inhibition ratio. Finally, and surprisingly, about one third of the cell types recorded in the hemibrain connectome could not be robustly identified in the FlyWire connectome, cautioning against defining cell types based on single connectomes. We propose that a cell type should be robust to inter-individual variation, and therefore defined as a group of cells that are more similar to cells in a different brain than to any other cell in the same brain. We show that this new definition can be consistently applied to whole connectome datasets. Our work defines a consensus cell type atlas for the fly brain and provides both an intellectual framework and open source toolchain for brain-scale comparative connectomics.

A leaky integrate-and-fire computational model based on the connectome of the entire adult Drosophila brain reveals insights into sensorimotor processing

The forthcoming assembly of the adult Drosophila melanogaster central brain connectome, containing over 125,000 neurons and 50 million synaptic connections, provides a template for examining sensory processing throughout the brain. Here, we create a leaky integrate-and-fire computational model of the entire Drosophila brain, based on neural connectivity and neurotransmitter identity, to study circuit properties of feeding and grooming behaviors. We show that activation of sugar-sensing or water-sensing gustatory neurons in the computational model accurately predicts neurons that respond to tastes and are required for feeding initiation. Computational activation of neurons in the feeding region of the Drosophila brain predicts those that elicit motor neuron firing, a testable hypothesis that we validate by optogenetic activation and behavioral studies. Moreover, computational activation of different classes of gustatory neurons makes accurate predictions of how multiple taste modalities interact, providing circuit-level insight into aversive and appetitive taste processing. Our computational model predicts that the sugar and water pathways form a partially shared appetitive feeding initiation pathway, which our calcium imaging and behavioral experiments confirm. Additionally, we applied this model to mechanosensory circuits and found that computational activation of mechanosensory neurons predicts activation of a small set of neurons comprising the antennal grooming circuit that do not overlap with gustatory circuits, and accurately describes the circuit response upon activation of different mechanosensory subtypes. Our results demonstrate that modeling brain circuits purely from connectivity and predicted neurotransmitter identity generates experimentally testable hypotheses and can accurately describe complete sensorimotor transformations.

Neural network organization for courtship-song feature detection in Drosophila

Animals communicate using sounds in a wide range of contexts, and auditory systems must encode behaviorally relevant acoustic features to drive appropriate reactions. How feature detection emerges along auditory pathways has been difficult to solve due to challenges in mapping the underlying circuits and characterizing responses to behaviorally relevant features. Here, we study auditory activity in the Drosophila melanogaster brain and investigate feature selectivity for the two main modes of fly courtship song, sinusoids and pulse trains. We identify 24 new cell types of the intermediate layers of the auditory pathway, and using a new connectomic resource, FlyWire, we map all synaptic connections between these cell types, in addition to connections to known early and higher-order auditory neurons-this represents the first circuit-level map of the auditory pathway. We additionally determine the sign (excitatory or inhibitory) of most synapses in this auditory connectome. We find that auditory neurons display a continuum of preferences for courtship song modes and that neurons with different song-mode preferences and response timescales are highly interconnected in a network that lacks hierarchical structure. Nonetheless, we find that the response properties of individual cell types within the connectome are predictable from their inputs. Our study thus provides new insights into the organization of auditory coding within the Drosophila brain.

SLEAP: A deep learning system for multi-animal pose tracking

The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.

FlyWire: online community for whole-brain connectomics

Due to advances in automated image acquisition and analysis, whole-brain connectomes with 100,000 or more neurons are on the horizon. Proofreading of whole-brain automated reconstructions will require many person-years of effort, due to the huge volumes of data involved. Here we present FlyWire, an online community for proofreading neural circuits in a Drosophila melanogaster brain and explain how its computational and social structures are organized to scale up to whole-brain connectomics. Browser-based three-dimensional interactive segmentation by collaborative editing of a spatially chunked supervoxel graph makes it possible to distribute proofreading to individuals located virtually anywhere in the world. Information in the edit history is programmatically accessible for a variety of uses such as estimating proofreading accuracy or building incentive systems. An open community accelerates proofreading by recruiting more participants and accelerates scientific discovery by requiring information sharing. We demonstrate how FlyWire enables circuit analysis by reconstructing and analyzing the connectome of mechanosensory neurons.

Auditory activity is diverse and widespread throughout the central brain of Drosophila

Sensory pathways are typically studied by starting at receptor neurons and following postsynaptic neurons into the brain. However, this leads to a bias in analyses of activity toward the earliest layers of processing. Here, we present new methods for volumetric neural imaging with precise across-brain registration to characterize auditory activity throughout the entire central brain of Drosophila and make comparisons across trials, individuals and sexes. We discover that auditory activity is present in most central brain regions and in neurons responsive to other modalities. Auditory responses are temporally diverse, but the majority of activity is tuned to courtship song features. Auditory responses are stereotyped across trials and animals in early mechanosensory regions, becoming more variable at higher layers of the putative pathway, and this variability is largely independent of ongoing movements. This study highlights the power of using an unbiased, brain-wide approach for mapping the functional organization of sensory activity.

Quantifying behavior to understand the brain

Over the past years, numerous methods have emerged to automate the quantification of animal behavior at a resolution not previously imaginable. This has opened up a new field of computational ethology and will, in the near future, make it possible to quantify in near completeness what an animal is doing as it navigates its environment. The importance of improving the techniques with which we characterize behavior is reflected in the emerging recognition that understanding behavior is an essential (or even prerequisite) step to pursuing neuroscience questions. The use of these methods, however, is not limited to studying behavior in the wild or in strictly ethological settings. Modern tools for behavioral quantification can be applied to the full gamut of approaches that have historically been used to link brain to behavior, from psychophysics to cognitive tasks, augmenting those measurements with rich descriptions of how animals navigate those tasks. Here we review recent technical advances in quantifying behavior, particularly in methods for tracking animal motion and characterizing the structure of those dynamics. We discuss open challenges that remain for behavioral quantification and highlight promising future directions, with a strong emphasis on emerging approaches in deep learning, the core technology that has enabled the markedly rapid pace of progress of this field. We then discuss how quantitative descriptions of behavior can be leveraged to connect brain activity with animal movements, with the ultimate goal of resolving the relationship between neural circuits, cognitive processes and behavior.

The neural basis for a persistent internal state in Drosophila females

Sustained changes in mood or action require persistent changes in neural activity, but it has been difficult to identify the neural circuit mechanisms that underlie persistent activity and contribute to long-lasting changes in behavior. Here, we show that a subset of Doublesex+ pC1 neurons in the Drosophila female brain, called pC1d/e, can drive minutes-long changes in female behavior in the presence of males. Using automated reconstruction of a volume electron microscopic (EM) image of the female brain, we map all inputs and outputs to both pC1d and pC1e. This reveals strong recurrent connectivity between, in particular, pC1d/e neurons and a specific subset of Fruitless+ neurons called aIPg. We additionally find that pC1d/e activation drives long-lasting persistent neural activity in brain areas and cells overlapping with the pC1d/e neural network, including both Doublesex+ and Fruitless+ neurons. Our work thus links minutes-long persistent changes in behavior with persistent neural activity and recurrent circuit architecture in the female brain.

Neuroscience: Tantalized Flies Are Primed for Satiety

How animals maintain and switch between distinct motivational states is an important question in neuroscience. New work in Drosophila identifies an excitatory neuronal circuit that builds up mating drive while priming itself for satiety.

Author Correction: Unsupervised identification of the internal states that shape natural behavior

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Unsupervised identification of the internal states that shape natural behavior

Internal states shape stimulus responses and decision-making, but we lack methods to identify them. To address this gap, we developed an unsupervised method to identify internal states from behavioral data and applied it to a dynamic social interaction. During courtship, Drosophila melanogaster males pattern their songs using feedback cues from their partner. Our model uncovers three latent states underlying this behavior and is able to predict moment-to-moment variation in song-patterning decisions. These states correspond to different sensorimotor strategies, each of which is characterized by different mappings from feedback cues to song modes. We show that a pair of neurons previously thought to be command neurons for song production are sufficient to drive switching between states. Our results reveal how animals compose behavior from previously unidentified internal states, which is a necessary step for quantitative descriptions of animal behavior that link environmental cues, internal needs, neuronal activity and motor outputs.

Shared Song Detector Neurons in Drosophila Male and Female Brains Drive Sex-Specific Behaviors

Males and females often produce distinct responses to the same sensory stimuli. How such differences arise-at the level of sensory processing or in the circuits that generate behavior-remains largely unresolved across sensory modalities. We address this issue in the acoustic communication system of Drosophila. During courtship, males generate time-varying songs, and each sex responds with specific behaviors. We characterize male and female behavioral tuning for all aspects of song and show that feature tuning is similar between sexes, suggesting sex-shared song detectors drive divergent behaviors. We then identify higher-order neurons in the Drosophila brain, called pC2, that are tuned for multiple temporal aspects of one mode of the male's song and drive sex-specific behaviors. We thus uncover neurons that are specifically tuned to an acoustic communication signal and that reside at the sensory-motor interface, flexibly linking auditory perception with sex-specific behavioral responses.

Acoustic Pattern Recognition and Courtship Songs: Insights from Insects

Across the animal kingdom, social interactions rely on sound production and perception. From simple cricket chirps to more elaborate bird songs, animals go to great lengths to communicate information critical for reproduction and survival via acoustic signals. Insects produce a wide array of songs to attract a mate, and the intended receivers must differentiate these calls from competing sounds, analyze the quality of the sender from spectrotemporal signal properties, and then determine how to react. Insects use numerically simple nervous systems to analyze and respond to courtship songs, making them ideal model systems for uncovering the neural mechanisms underlying acoustic pattern recognition. We highlight here how the combination of behavioral studies and neural recordings in three groups of insects-crickets, grasshoppers, and fruit flies-reveals common strategies for extracting ethologically relevant information from acoustic patterns and how these findings might translate to other systems.

Fast animal pose estimation using deep neural networks

The need for automated and efficient systems for tracking full animal pose has increased with the complexity of behavioral data and analyses. Here we introduce LEAP (LEAP estimates animal pose), a deep-learning-based method for predicting the positions of animal body parts. This framework consists of a graphical interface for labeling of body parts and training the network. LEAP offers fast prediction on new data, and training with as few as 100 frames results in 95% of peak performance. We validated LEAP using videos of freely behaving fruit flies and tracked 32 distinct points to describe the pose of the head, body, wings and legs, with an error rate of <3% of body length. We recapitulated reported findings on insect gait dynamics and demonstrated LEAP's applicability for unsupervised behavioral classification. Finally, we extended the method to more challenging imaging situations and videos of freely moving mice.

Discovery of a New Song Mode in Drosophila Reveals Hidden Structure in the Sensory and Neural Drivers of Behavior

Deciphering how brains generate behavior depends critically on an accurate description of behavior. If distinct behaviors are lumped together, separate modes of brain activity can be wrongly attributed to the same behavior. Alternatively, if a single behavior is split into two, the same neural activity can appear to produce different behaviors. Here, we address this issue in the context of acoustic communication in Drosophila. During courtship, males vibrate their wings to generate time-varying songs, and females evaluate songs to inform mating decisions. For 50 years, Drosophila melanogaster song was thought to consist of only two modes, sine and pulse, but using unsupervised classification methods on large datasets of song recordings, we now establish the existence of at least three song modes: two distinct pulse types, along with a single sine mode. We show how this seemingly subtle distinction affects our interpretation of the mechanisms underlying song production and perception. Specifically, we show that visual feedback influences the probability of producing each song mode and that male song mode choice affects female responses and contributes to modulating his song amplitude with distance. At the neural level, we demonstrate how the activity of four separate neuron types within the fly’s song pathway differentially affects the probability of producing each song mode. Our results highlight the importance of carefully segmenting behavior to map the underlying sensory, neural, and genetic mechanisms.

For 50 years, Drosophila melanogaster song was thought to consist of only two modes: sine and pulse. Using unsupervised classification methods, Clemens et al. establish the existence of three distinct song modes. They show how this distinction affects the interpretation of the mechanisms underlying song production and perception.

To Fight or Not to Fight

In this issue of Neuron, Watanabe et al. (2017) uncover how octopamine, an invertebrate norepinephrine analog, modulates the neural pathways that bias Drosophila males toward aggression.

Quantifying behavior to solve sensorimotor transformations: advances from worms and flies

The development of new computational tools has recently opened up the study of natural behaviors at a precision that was previously unachievable. These tools permit a highly quantitative analysis of behavioral dynamics at timescales that are well matched to the timescales of neural activity. Here we examine how combining these methods with established techniques for estimating an animal's sensory experience presents exciting new opportunities for dissecting the sensorimotor transformations performed by the nervous system. We focus this review primarily on examples from Caenorhabditis elegans and Drosophila melanogaster-for these model systems, computational approaches to characterize behavior, in combination with unparalleled genetic tools for neural activation, silencing, and recording, have already proven instrumental for illuminating underlying neural mechanisms.

Sensorimotor Transformations Underlying Variability in Song Intensity during Drosophila Courtship

Diverse animal species, from insects to humans, utilize acoustic signals for communication. Studies of the neural basis for song or speech production have focused almost exclusively on the generation of spectral and temporal patterns, but animals can also adjust acoustic signal intensity when communicating. For example, humans naturally regulate the loudness of speech in accord with a visual estimate of receiver distance. The underlying mechanisms for this ability remain uncharacterized in any system. Here, we show that Drosophila males modulate courtship song amplitude with female distance, and we investigate each stage of the sensorimotor transformation underlying this behavior, from the detection of particular visual stimulus features and the timescales of sensory processing to the modulation of neural and muscle activity that generates song. Our results demonstrate an unanticipated level of control in insect acoustic communication and uncover novel computations and mechanisms underlying the regulation of acoustic signal intensity.

Cell-Type-Specific Transcriptome Analysis in the Drosophila Mushroom Body Reveals Memory-Related Changes in Gene Expression

Learning and memory formation in Drosophila rely on a network of neurons in the mushroom bodies (MBs). Whereas numerous studies have delineated roles for individual cell types within this network in aspects of learning or memory, whether or not these cells can also be distinguished by the genes they express remains unresolved. In addition, the changes in gene expression that accompany long-term memory formation within the MBs have not yet been studied by neuron type. Here, we address both issues by performing RNA sequencing on single cell types (harvested via patch pipets) within the MB. We discover that the expression of genes that encode cell surface receptors is sufficient to identify cell types and that a subset of these genes, required for sensory transduction in peripheral sensory neurons, is not only expressed within individual neurons of the MB in the central brain, but is also critical for memory formation.

Singing on the fly: sensorimotor integration and acoustic communication in Drosophila

The capacity to communicate via acoustic signals is prevalent across the animal kingdom, from insects to humans. What are the neural circuit mechanisms that underlie this ability? New methods for behavioral analysis along with an unparalleled genetic toolkit have recently opened up studies of acoustic communication in the fruit fly, Drosophila melanogaster. Its nervous system comprises roughly 100,000 neurons, yet flies are able to both produce and process time-varying sounds during courtship. Just as with more complex animals, sensory feedback plays an important role in shaping communication between the sexes. Here, we review recent work in Drosophila that has laid the foundation for solving the mechanisms by which sensory information dynamically modulates behavior.

Connecting Neural Codes with Behavior in the Auditory System of Drosophila

Brains are optimized for processing ethologically relevant sensory signals. However, few studies have characterized the neural coding mechanisms that underlie the transformation from natural sensory information to behavior. Here, we focus on acoustic communication in Drosophila melanogaster and use computational modeling to link natural courtship song, neuronal codes, and female behavioral responses to song. We show that melanogaster females are sensitive to long timescale song structure (on the order of tens of seconds). From intracellular recordings, we generate models that recapitulate neural responses to acoustic stimuli. We link these neural codes with female behavior by generating model neural responses to natural courtship song. Using a simple decoder, we predict female behavioral responses to the same song stimuli with high accuracy. Our modeling approach reveals how long timescale song features are represented by the Drosophila brain and how neural representations can be decoded to generate behavioral selectivity for acoustic communication signals.

Acoustic duetting in Drosophila virilis relies on the integration of auditory and tactile signals

Many animal species, including insects, are capable of acoustic duetting, a complex social behavior in which males and females tightly control the rate and timing of their courtship song syllables relative to each other. The mechanisms underlying duetting remain largely unknown across model systems. Most studies of duetting focus exclusively on acoustic interactions, but the use of multisensory cues should aid in coordinating behavior between individuals. To test this hypothesis, we develop Drosophila virilis as a new model for studies of duetting. By combining sensory manipulations, quantitative behavioral assays, and statistical modeling, we show that virilis females combine precisely timed auditory and tactile cues to drive song production and duetting. Tactile cues delivered to the abdomen and genitalia play the larger role in females, as even headless females continue to coordinate song production with courting males. These data, therefore, reveal a novel, non-acoustic, mechanism for acoustic duetting. Finally, our results indicate that female-duetting circuits are not sexually differentiated, as males can also produce 'female-like' duets in a context-dependent manner.

Dynamic sensory cues shape song structure in Drosophila

The generation of acoustic communication signals is widespread across the animal kingdom, and males of many species, including Drosophilidae, produce patterned courtship songs to increase their chance of success with a female. For some animals, song structure can vary considerably from one rendition to the next; neural noise within pattern generating circuits is widely assumed to be the primary source of such variability, and statistical models that incorporate neural noise are successful at reproducing the full variation present in natural songs. In direct contrast, here we demonstrate that much of the pattern variability in Drosophila courtship song can be explained by taking into account the dynamic sensory experience of the male. In particular, using a quantitative behavioural assay combined with computational modelling, we find that males use fast modulations in visual and self-motion signals to pattern their songs, a relationship that we show is evolutionarily conserved. Using neural circuit manipulations, we also identify the pathways involved in song patterning choices and show that females are sensitive to song features. Our data not only demonstrate that Drosophila song production is not a fixed action pattern, but establish Drosophila as a valuable new model for studies of rapid decision-making under both social and naturalistic conditions.