Predictability and hierarchy in Drosophila behavior

Inferring neural population codes for Drosophila acoustic communication

Social communication between animals is often mediated by sequences of acoustic signals, sometimes spanning long timescales. How auditory neural circuits respond to extended input sequences to guide behavior is not understood. We address this problem using Drosophila acoustic communication, a behavior involving the male's production of and female's response to long, highly variable courtship songs. Here we ask whether female neural and behavioral responses to song are better described by a linear-nonlinear feature detection model vs. a nonlinear accumulation model. Comparing both models against head-fixed neural recordings and pure-behavioral recordings of unrestrained courtship, we found that while both models could explain the neural data, the accumulation model better predicted female locomotion during courtship, outperforming several alternative predictors. To understand how the accumulation model encoded song to predict locomotion, we analyzed the relationship between neural activity simulated by the model and female locomotion during courtship-this revealed the model's reliance on heterogeneous nonlinear adaptation and slow integration. Finally, we asked how adaptation and integration processes could cooperate across the model neural population to encode temporal patterns in song. Simulations revealed how adaptation can transform song inputs prior to integration, allowing fine-scale song information to be retained in the population code for long periods. Thus, modeling fly auditory responses as a nonlinearly adaptive, accumulating population code accounts for female locomotor responses to song during courtship and suggests a biologically plausible mechanism for the online encoding of extended communication sequences.

Behavioral sequences across multiple animal species in the wild share common structural features

Animal behavior can be decomposed into a sequence of discrete activity bouts over time. Analyzing the statistical structure of such behavioral sequences can provide insights into the drivers of behavioral decisions. Laboratory studies, predominantly in invertebrates, have suggested that behavioral sequences exhibit multiple timescales and long-range memory, but whether these results can be generalized to other taxa and to animals in natural settings remains unclear. By analyzing accelerometer-inferred predictions of behavioral states in three species of social mammals (meerkats, white-nosed coatis, and spotted hyenas) in the wild, we found surprisingly consistent structuring of behavioral sequences across all behavioral states, all individuals, and all study species. Behavioral bouts were characterized by decreasing hazard functions, wherein the longer a behavioral bout had progressed, the less likely it was to end within the next instant. The predictability of an animal's future behavioral state as a function of its present state always decreased as a truncated power-law for predictions made farther into the future, with very similar estimates for the power law exponent across all species. Finally, the distributions of bout durations were also heavy-tailed. Why such shared structural principles emerge remains unknown, and we explore multiple plausible explanations, including environmental nonstationarity, behavioral self-reinforcement, and the hierarchical nature of behavior. The existence of highly consistent patterns in behavioral sequences across our study species suggests that these phenomena could be widespread in nature, and points to the existence of fundamental properties of behavioral dynamics that could drive such convergent patterns.

Sensorimotor delays constrain robust locomotion in a 3D kinematic model of fly walking

Walking animals must maintain stability in the presence of external perturbations, despite significant temporal delays in neural signaling and muscle actuation. Here, we develop a 3D kinematic model with a layered control architecture to investigate how sensorimotor delays constrain the robustness of walking behavior in the fruit fly, Drosophila. Motivated by the anatomical architecture of insect locomotor control circuits, our model consists of three component layers: a neural network that generates realistic 3D joint kinematics for each leg, an optimal controller that executes the joint kinematics while accounting for delays, and an inter-leg coordinator. The model generates realistic simulated walking that resembles real fly walking kinematics and sustains walking even when subjected to unexpected perturbations, generalizing beyond its training data. However, we found that the model's robustness to perturbations deteriorates when sensorimotor delay parameters exceed the physiological range. These results suggest that fly sensorimotor control circuits operate close to the temporal limit at which they can detect and respond to external perturbations. More broadly, we show how a modular, layered model architecture can be used to investigate physiological constraints on animal behavior.

Inhibitory control explains locomotor statistics in walking Drosophila

In order to forage for food, many animals regulate not only specific limb movements but the statistics of locomotor behavior, switching between long-range dispersal and local search depending on resource availability. How premotor circuits regulate locomotor statistics is not clear. Here, we analyze and model locomotor statistics and their modulation by attractive food odor in walking Drosophila. Food odor evokes three motor regimes in flies: baseline walking, upwind running during odor, and search behavior following odor loss. During search, we find that flies adopt higher angular velocities and slower ground speeds and turn for longer periods in the same direction. We further find that flies adopt periods of different mean ground speed and that these state changes influence the length of odor-evoked runs. We next developed a simple model of neural locomotor control that suggests that contralateral inhibition plays a key role in regulating the statistical features of locomotion. As the fly connectome predicts decussating inhibitory neurons in the premotor lateral accessory lobe (LAL), we gained genetic access to a subset of these neurons and tested their effects on behavior. We identified one population whose activation induces all three signature of local search and that regulates angular velocity at odor offset. We identified a second population, including a single LAL neuron pair, that bidirectionally regulates ground speed. Together, our work develops a biologically plausible computational architecture that captures the statistical features of fly locomotion across behavioral states and identifies neural substrates of these computations.

Using AI to prevent the insect apocalypse: toward new environmental risk assessment procedures

Insect populations are declining globally, with multiple potential drivers identified. However, experimental data are needed to understand their relative contributions. We highlight the sublethal effects of pesticides at field-relevant concentrations, often overlooked in standard environmental risk assessments (ERA), as significant contributors to these declines. Behavior, as an easily monitored high-level phenotype, reflects alterations at various phenotypic levels. We propose incorporating behavioral assays with AI-based analytical methods into ERA protocols to better assess the safety of molecules intended for large-scale field use. This approach aims to safeguard food supplies and protect vital ecosystems in the future.

Capturing continuous, long timescale behavioral changes in Drosophila melanogaster postural data

Animal behavior spans many timescales, from short, seconds-scale actions to daily rhythms over many hours to life-long changes during aging. To access longer timescales of behavior, we continuously recorded individual Drosophila melanogaster at 100 frames per second for up to 7 days at a time in featureless arenas on sucrose-agarose media. We use the deep learning framework SLEAP to produce a full-body postural dataset for 47 individuals resulting in nearly 2 billion pose instances. We identify stereotyped behaviors such as grooming, proboscis extension, and locomotion and use the resulting ethograms to explore how the flies' behavior varies across time of day and days in the experiment. We find distinct daily patterns in all stereotyped behaviors, adding specific information about trends in different grooming modalities, proboscis extension duration, and locomotion speed to what is known about the D. melanogaster circadian cycle. Using our holistic measurements of behavior, we find that the hour after dawn is a unique time point in the flies' daily pattern of behavior, and that the behavioral composition of this hour tracks well with other indicators of health such as locomotion speed and the fraction of time spend moving vs. resting. The method, data, and analysis presented here give us a new and clearer picture of D. melanogaster behavior across timescales, revealing novel features that hint at unexplored underlying biological mechanisms.

Inhibitory control of locomotor statistics in walking Drosophila

In order to forage for food, many animals regulate not only specific limb movements but the statistics of locomotor behavior over time, switching between long-range dispersal and localized search depending on resource availability. How pre-motor circuits regulate such locomotor statistics is not clear. Here we analyze and model locomotor statistics in walking Drosophila, and their modulation by attractive food odor. Odor evokes three motor regimes in flies: baseline walking, upwind running during odor, and search behavior following odor loss. During search behavior, we find that flies adopt higher angular velocities and slower ground speeds, and tend to turn for longer periods of time in one direction. We further find that flies spontaneously adopt periods of different mean ground speed, and that these changes in state influence the length of odor-evoked runs. We next developed a simple model of neural locomotor control that suggests that contralateral inhibition plays a key role in regulating the statistical features of locomotion. As the fly connectome predicts decussating inhibitory neurons in the lateral accessory lobe (LAL), a pre-motor structure, we gained genetic access to a subset of these neurons and tested their effects on behavior. We identified one population of neurons whose activation induces all three signature of search and that bi-directionally regulates angular velocity at odor offset. We identified a second group of neurons, including a single LAL neuron pair, that bi-directionally regulate ground speed. Together, our work develops a biologically plausible computational architecture that captures the statistical features of fly locomotion across behavioral states and identifies potential neural substrates of these computations.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, exhibiting variability across scales that reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build Markov models of movement sequences that bridge across timescales and enable a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish responding to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising versus wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive stimuli, or in search for appetitive prey. As our method encodes the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies, we use it to uncover a hierarchical structure in the phenotypic variability that reflects exploration-exploitation trade-offs. Across a wide range of sensory cues, a major source of variation among fish is driven by prior and/or immediate exposure to prey that induces exploitation phenotypes. A large degree of variability that is not explained by environmental cues unravels hidden states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, by extracting the timescales of motor strategies deployed during navigation, our approach exposes structure among individuals and reveals internal states tuned by prior experience.

Hierarchical behavior control by a single class of interneurons

Animal behavior is organized into nested temporal patterns that span multiple timescales. This behavior hierarchy is believed to arise from a hierarchical neural architecture: Neurons near the top of the hierarchy are involved in planning, selecting, initiating, and maintaining motor programs, whereas those near the bottom of the hierarchy act in concert to produce fine spatiotemporal motor activity. In Caenorhabditis elegans, behavior on a long timescale emerges from ordered and flexible transitions between different behavioral states, such as forward, reversal, and turn. On a short timescale, different parts of the animal body coordinate fast rhythmic bending sequences to produce directional movements. Here, we show that Sublateral Anterior A (SAA), a class of interneurons that enable cross-communication between dorsal and ventral head motor neurons, play a dual role in shaping behavioral dynamics on different timescales. On a short timescale, SAA regulate and stabilize rhythmic bending activity during forward movements. On a long timescale, the same neurons suppress spontaneous reversals and facilitate reversal termination by inhibiting Ring Interneuron M (RIM), an integrating neuron that helps maintain a behavioral state. These results suggest that feedback from a lower-level cell assembly to a higher-level command center is essential for bridging behavioral dynamics at different levels.

Characterizing Behavioral Dynamics in Bipolar Disorder with Computational Ethology

New technologies for the quantification of behavior have revolutionized animal studies in social, cognitive, and pharmacological neurosciences. However, comparable studies in understanding human behavior, especially in psychiatry, are lacking. In this study, we utilized data-driven machine learning to analyze natural, spontaneous open-field human behaviors from people with euthymic bipolar disorder (BD) and non-BD participants. Our computational paradigm identified representations of distinct sets of actions (motifs) that capture the physical activities of both groups of participants. We propose novel measures for quantifying dynamics, variability, and stereotypy in BD behaviors. These fine-grained behavioral features reflect patterns of cognitive functions of BD and better predict BD compared with traditional ethological and psychiatric measures and action recognition approaches. This research represents a significant computational advancement in human ethology, enabling the quantification of complex behaviors in real-world conditions and opening new avenues for characterizing neuropsychiatric conditions from behavior.

Characterizing the structure of mouse behavior using Motion Sequencing

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

Hand-Jaw Coordination as Mice Handle Food Is Organized around Intrinsic Structure-Function Relationships

Rodent jaws evolved structurally to support dual functionality, for either biting or chewing food. Rodent hands also function dually during food handling, for actively manipulating or statically holding food. How are these oral and manual functions coordinated? We combined electrophysiological recording of muscle activity and kilohertz kinematic tracking to analyze masseter and hand actions as mice of both sexes handled food. Masseter activity was organized into two modes synchronized to hand movement modes. In holding/chewing mode, mastication occurred as rhythmic (∼5 Hz) masseter activity while the hands held food below the mouth. In oromanual/ingestion mode, bites occurred as lower-amplitude aperiodic masseter events that were precisely timed to follow regrips (by ∼200 ms). Thus, jaw and hand movements are flexibly coordinated during food handling: uncoupled in holding/chewing mode and tightly coordinated in oromanual/ingestion mode as regrip-bite sequences. Key features of this coordination were captured in a simple model of hierarchically orchestrated mode-switching and intramode action sequencing. We serendipitously detected an additional masseter-related action, tooth sharpening, identified as bouts of higher-frequency (∼13 Hz) rhythmic masseter activity, which was accompanied by eye displacement, including rhythmic proptosis, attributable to masseter contractions. Collectively, the findings demonstrate how a natural, complex, and goal-oriented activity is organized as an assemblage of distinct modes and complex actions, adapted for the divisions of function arising from anatomical structure. These results reveal intricate, high-speed coordination of disparate effectors and show how natural forms of dexterity can serve as a model for understanding the behavioral neurobiology of multi-body-part coordination.

Ethnokinesiology: towards a neuromechanical understanding of cultural differences in movement

Each individual's movements are sculpted by constant interactions between sensorimotor and sociocultural factors. A theoretical framework grounded in motor control mechanisms articulating how sociocultural and biological signals converge to shape movement is currently missing. Here, we propose a framework for the emerging field of ethnokinesiology aiming to provide a conceptual space and vocabulary to help bring together researchers at this intersection. We offer a first-level schema for generating and testing hypotheses about cultural differences in movement to bridge gaps between the rich observations of cross-cultural movement variations and neurophysiological and biomechanical accounts of movement. We explicitly dissociate two interacting feedback loops that determine culturally relevant movement: one governing sensorimotor tasks regulated by neural signals internal to the body, the other governing ecological tasks generated through actions in the environment producing ecological consequences. A key idea is the emergence of individual-specific and culturally influenced motor concepts in the nervous system, low-dimensional functional mappings between sensorimotor and ecological task spaces. Motor accents arise from perceived differences in motor concept topologies across cultural contexts. We apply the framework to three examples: speech, gait and grasp. Finally, we discuss how ethnokinesiological studies may inform personalized motor skill training and rehabilitation, and challenges moving forward.This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Glial expression of Drosophila UBE3A causes spontaneous seizures that can be modulated by 5-HT signaling

Misexpression of the E3 ubiquitin ligase gene UBE3A is thought to contribute to a range of neurological disorders. In the context of Dup15q syndrome, additional genomic copies of UBE3A give rise to the autism, muscle hypotonia and spontaneous seizures characteristics of the disorder. In a Drosophila model of Dup 15q syndrome, it was recently shown that glial-driven expression of the UBE3A ortholog dube3a led to a "bang-sensitive" phenotype, where mechanical shock triggers convulsions, suggesting glial dube3a expression contributes to hyperexcitability in flies. Here we directly compare the consequences of glial- and neuronal-driven dube3a expression on motor coordination and seizure susceptibility in Drosophila. To quantify seizure-related behavioral events, we developed and trained a hidden Markov model that identified these events based on automated video tracking of fly locomotion. Both glial and neuronal driven dube3a expression led to clear motor phenotypes. However, only glial-driven dube3a expression displayed spontaneous seizure-associated immobilization events, that were clearly observed at high-temperature (38 °C). Using a tethered fly preparation amenable to electrophysiological monitoring of seizure activity, we found glial-driven dube3a flies display aberrant spontaneous spike discharges which are bilaterally synchronized. Neither neuronal-dube3a overexpressing flies, nor control flies displayed these firing patterns. We previously performed a drug screen for FDA approved compounds that can suppress bang-sensitivity in glial-driven dube3a expressing flies and identified certain 5-HT modulators as strong seizure suppressors. Here we found glial-driven dube3a flies fed the serotonin reuptake inhibitor vortioxetine and the 5-HT2A antagonist ketanserin displayed reduced immobilization and spike bursting, consistent with the previous study. Together these findings highlight the potential for glial pathophysiology to drive Dup15q syndrome-related seizure activity.

Exploring motion using geometric morphometrics in microscopic aquatic invertebrates: 'modes' and movement patterns during feeding in a bdelloid rotifer model species

BACKGROUND

Movement is a defining aspect of animals, but it is rarely studied using quantitative methods in microscopic invertebrates. Bdelloid rotifers are a cosmopolitan class of aquatic invertebrates of great scientific interest because of their ability to survive in very harsh environment and also because they represent a rare example of an ancient lineage that only includes asexually reproducing species. In this class, Adineta ricciae has become a model species as it is unusually easy to culture. Yet, relatively little is known of its ethology and almost nothing on how it behaves during feeding.

METHODS

To explore feeding behaviour in A. ricciae, as well as to provide an example of application of computational ethology in a microscopic invertebrate, we apply Procrustes motion analysis in combination with ordination and clustering methods to a laboratory bred sample of individuals recorded during feeding.

RESULTS

We demonstrate that movement during feeding can be accurately described in a simple two-dimensional shape space with three main 'modes' of motion. Foot telescoping, with the body kept straight, is the most frequent 'mode', but it is accompanied by periodic rotations of the foot together with bending while the foot is mostly retracted.

CONCLUSIONS

Procrustes motion analysis is a relatively simple but effective tool for describing motion during feeding in A. ricciae. The application of this method generates quantitative data that could be analysed in relation to genetic and ecological differences in a variety of experimental settings. The study provides an example that is easy to replicate in other invertebrates, including other microscopic animals whose behavioural ecology is often poorly known.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ('syllables') from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, exhibiting variability across scales that reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build Markov models of movement sequences that bridges across time scales and enables a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish responding to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising versus wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive stimuli, or in search for appetitive prey. As our method encodes the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies, we use it to uncover a hierarchical structure in the phenotypic variability that reflects exploration-exploitation trade-offs. Across a wide range of sensory cues, a major source of variation among fish is driven by prior and/or immediate exposure to prey that induces exploitation phenotypes. A large degree of variability that is not explained by environmental cues unravels motivational states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, by extracting the timescales of motor strategies deployed during navigation, our approach exposes structure among individuals and reveals internal states tuned by prior experience.

STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation

Animal behavior analysis plays a crucial role in contemporary neuroscience research. However, the performance of the frame-by-frame approach may degrade in scenarios with occlusions or motion blur. In this study, we propose a spatiotemporal network model based on YOLOv8 to enhance the accuracy of key-point detection in mouse behavioral experimental videos. This model integrates a time-domain tracking strategy comprising two components: the first part utilizes key-point detection results from the previous frame to detect potential target locations in the subsequent frame; the second part employs Kalman filtering to analyze key-point changes prior to detection, allowing for the estimation of missing key-points. In the comparison of pose estimation results between our approach, YOLOv8, DeepLabCut and SLEAP on videos of three mouse behavioral experiments, our approach demonstrated significantly superior performance. This suggests that our method offers a new and effective means of accurately tracking and estimating pose in mice through spatiotemporal processing.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, resulting in variability that ultimately reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build models that bridges across time scales that enable a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish exposed to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising and wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive (dark) stimuli or in search for prey. Our method enables us to encode the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies. By doing so, we uncover a hierarchical structure to the phenotypic variability that corresponds to exploration-exploitation trade-offs. Within a wide range of sensory cues, a major source of variation among fish is driven by prior and immediate exposure to prey that induces exploitation phenotypes. However, a large degree of variability is unexplained by environmental cues, pointing to hidden states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, our approach extracts the timescales of motor strategies deployed during navigation, exposing undiscovered structure among individuals and pointing to internal states tuned by prior experience.

Glial expression of Drosophila UBE3A causes spontaneous seizures modulated by 5-HT signaling

Misexpression of the E3 ubiquitin ligase UBE3A is thought to contribute to a range of neurological disorders. In the context of Dup15q syndrome, excess genomic copies of UBE3A is thought to contribute to the autism, muscle tone and spontaneous seizures characteristic of the disorder. In a Drosophila model of Dup 15q syndrome, it was recently shown glial-driven expression of the UBE3A ortholog dube3a led to a "bang-sensitive" phenotype, where mechanical shock triggers convulsions, suggesting glial dube3a expression contributes to hyperexcitability in flies. Here we directly compare the consequences of glial- and neuronal-driven dube3a expression on motor coordination and neuronal excitability in Drosophila. We utilized IowaFLI tracker and developed a hidden Markov Model to classify seizure-related immobilization. Both glial and neuronal driven dube3a expression led to clear motor phenotypes. However, only glial-driven dube3a expression displayed spontaneous immobilization events, that were exacerbated at high-temperature (38 °C). Using a tethered fly preparation we monitored flight muscle activity, we found glial-driven dube3a flies display spontaneous spike discharges which were bilaterally synchronized indicative of seizure activity. Neither control flies, nor neuronal- dube3a overexpressing flies display such firing patterns. Prior drug screen indicated bang-sensitivity in glial-driven dube3a expressing flies could be suppressed by certain 5-HT modulators. Consistent with this report, we found glial-driven dube3a flies fed the serotonin reuptake inhibitor vortioxetine and the 5HT 2A antagonist ketanserin displayed reduced immobilization and spike bursting. Together these findings highlight the potential for glial pathophysiology to drive Dup15q syndrome-related seizure activity.

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms have revolutionized the analysis of animal behavior, enabling investigators to flexibly quantify behavioral dynamics from conventional video recordings obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into the modules out of which behavior is organized. This challenge is particularly acute because keypoint data is susceptible to high frequency jitter that clustering algorithms can mistake for transitions between behavioral modules. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ("syllables") from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to effectively identify syllables whose boundaries correspond to natural sub-second discontinuities inherent to mouse behavior. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior, and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq therefore renders behavioral syllables and grammar accessible to the many researchers who use standard video to capture animal behavior.

NAPS: Integrating pose estimation and tag-based tracking

1. Significant advances in computational ethology have allowed the quantification of behaviour in unprecedented detail. Tracking animals in social groups, however, remains challenging as most existing methods can either capture pose or robustly retain individual identity over time but not both. 2. To capture finely resolved behaviours while maintaining individual identity, we built NAPS (NAPS is ArUco Plus SLEAP), a hybrid tracking framework that combines state-of-the-art, deep learning-based methods for pose estimation (SLEAP) with unique markers for identity persistence (ArUco). We show that this framework allows the exploration of the social dynamics of the common eastern bumblebee (Bombus impatiens). 3. We provide a stand-alone Python package for implementing this framework along with detailed documentation to allow for easy utilization and expansion. We show that NAPS can scale to long timescale experiments at a high frame rate and that it enables the investigation of detailed behavioural variation within individuals in a group. 4. Expanding the toolkit for capturing the constituent behaviours of social groups is essential for understanding the structure and dynamics of social networks. NAPS provides a key tool for capturing these behaviours and can provide critical data for understanding how individual variation influences collective dynamics.

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Deep learning-based markerless tracking has revolutionized studies of animal behavior. Yet the generalizability of trained models tends to be limited, as new training data typically needs to be generated manually for each setup or visual environment. With each model trained from scratch, researchers track distinct landmarks and analyze the resulting kinematic data in idiosyncratic ways. Moreover, due to inherent limitations in manual annotation, only a sparse set of landmarks are typically labeled. To address these issues, we developed an approach, which we term GlowTrack, for generating orders of magnitude more training data, enabling models that generalize across experimental contexts. We describe: a) a high-throughput approach for producing hidden labels using fluorescent markers; b) a multi-camera, multi-light setup for simulating diverse visual conditions; and c) a technique for labeling many landmarks in parallel, enabling dense tracking. These advances lay a foundation for standardized behavioral pipelines and more complete scrutiny of movement.

Capturing continuous, long timescale behavioral changes in Drosophila melanogaster postural data

Animal behavior spans many timescales, from short, seconds-scale actions to circadian rhythms over many hours to life-long changes during aging. Most quantitative behavior studies have focused on short-timescale behaviors such as locomotion and grooming. Analysis of these data suggests there exists a hierarchy of timescales; however, the limited duration of these experiments prevents the investigation of the full temporal structure. To access longer timescales of behavior, we continuously recorded individual Drosophila melanogaster at 100 frames per second for up to 7 days at a time in featureless arenas on sucrose-agarose media. We use the deep learning framework SLEAP to produce a full-body postural data set for 47 individuals resulting in nearly 2 billion pose instances. We identify stereotyped behaviors such as grooming, proboscis extension, and locomotion and use the resulting ethograms to explore how the flies' behavior varies across time of day and days in the experiment. We find distinct circadian patterns in all of our stereotyped behavior and also see changes in behavior over the course of the experiment as the flies weaken and die.

Emergent programmable behavior and chaos in dynamically driven active filaments

How the behavior of cells emerges from their constituent subcellular biochemical and physical parts is an outstanding challenge at the intersection of biology and physics. A remarkable example of single-cell behavior occurs in the ciliate Lacrymaria olor, which hunts for its prey via rapid movements and protrusions of a slender neck, many times the size of the original cell body. The dynamics of this cell neck is powered by a coat of cilia across its length and tip. How a cell can program this active filamentous structure to produce desirable behaviors like search and homing to a target remains unknown. Here, we present an active filament model that allows us to uncover how a "program" (time sequence of active forcing) leads to "behavior" (filament shape dynamics). Our model captures two key features of this system-time-varying activity patterns (extension and compression cycles) and active stresses that are uniquely aligned with the filament geometry-a "follower force" constraint. We show that active filaments under deterministic, time-varying follower forces display rich behaviors including periodic and aperiodic dynamics over long times. We further show that aperiodicity occurs due to a transition to chaos in regions of a biologically accessible parameter space. We also identify a simple nonlinear iterated map of filament shape that approximately predicts long-term behavior suggesting simple, artificial "programs" for filament functions such as homing and searching space. Last, we directly measure the statistical properties of biological programs in L. olor, enabling comparisons between model predictions and experiments.

SAMPL is a high-throughput solution to study unconstrained vertical behavior in small animals

Balance and movement are impaired in many neurological disorders. Recent advances in behavioral monitoring provide unprecedented access to posture and locomotor kinematics but without the throughput and scalability necessary to screen candidate genes/potential therapeutics. Here, we present a scalable apparatus to measure posture and locomotion (SAMPL). SAMPL includes extensible hardware and open-source software with real-time processing and can acquire data from D. melanogaster, C. elegans, and D. rerio as they move vertically. Using SAMPL, we define how zebrafish balance as they navigate vertically and discover small but systematic variations among kinematic parameters between genetic backgrounds. We demonstrate SAMPL's ability to resolve differences in posture and navigation as a function of effect size and data gathered, providing key data for screens. SAMPL is therefore both a tool to model balance and locomotor disorders and an exemplar of how to scale apparatus to support screens.

Hierarchical organization of rhesus macaque behavior

Primatologists, psychologists and neuroscientists have long hypothesized that primate behavior is highly structured. However, delineating that structure has been impossible due to the difficulties of precision behavioral tracking. Here we analyzed a dataset consisting of continuous measures of the 3D position of two male rhesus macaques (Macaca mulatta) performing three different tasks in a large unrestrained environment over several hours. Using an unsupervised embedding approach on the tracked joints, we identified commonly repeated pose patterns, which we call postures. We found that macaques' behavior is characterized by 49 distinct postures, lasting an average of 0.6 seconds. We found evidence that behavior is hierarchically organized, in that transitions between poses tend to occur within larger modules, which correspond to identifiable actions; these actions are further organized hierarchically. Our behavioral decomposition allows us to identify universal (cross-individual and cross-task) and unique (specific to each individual and task) principles of behavior. These results demonstrate the hierarchical nature of primate behavior, provide a method for the automated ethogramming of primate behavior, and provide important constraints on neural models of pose generation.

Arc regulates a second-guessing cognitive bias during naturalistic foraging through effects on discrete behavior modules

Foraging in animals relies on innate decision-making heuristics that can result in suboptimal cognitive biases in some contexts. The mechanisms underlying these biases are not well understood, but likely involve strong genetic effects. To explore this, we studied fasted mice using a naturalistic foraging paradigm and discovered an innate cognitive bias called "second-guessing." This involves repeatedly investigating an empty former food patch instead of consuming available food, which hinders the mice from maximizing feeding benefits. The synaptic plasticity gene Arc is revealed to play a role in this bias, as Arc-deficient mice did not exhibit second-guessing and consumed more food. In addition, unsupervised machine learning decompositions of foraging identified specific behavior sequences, or "modules", that are affected by Arc. These findings highlight the genetic basis of cognitive biases in decision making, show links between behavior modules and cognitive bias, and provide insight into the ethological roles of Arc in naturalistic foraging.

Global change in brain state during spontaneous and forced walk in Drosophila is composed of combined activity patterns of different neuron classes

Movement-correlated brain activity has been found across species and brain regions. Here, we used fast whole brain lightfield imaging in adult Drosophila to investigate the relationship between walk and brain-wide neuronal activity. We observed a global change in activity that tightly correlated with spontaneous bouts of walk. While imaging specific sets of excitatory, inhibitory, and neuromodulatory neurons highlighted their joint contribution, spatial heterogeneity in walk- and turning-induced activity allowed parsing unique responses from subregions and sometimes individual candidate neurons. For example, previously uncharacterized serotonergic neurons were inhibited during walk. While activity onset in some areas preceded walk onset exclusively in spontaneously walking animals, spontaneous and forced walk elicited similar activity in most brain regions. These data suggest a major contribution of walk and walk-related sensory or proprioceptive information to global activity of all major neuronal classes.

Intrinsic timescales in the visual cortex change with selective attention and reflect spatial connectivity

Intrinsic timescales characterize dynamics of endogenous fluctuations in neural activity. Variation of intrinsic timescales across the neocortex reflects functional specialization of cortical areas, but less is known about how intrinsic timescales change during cognitive tasks. We measured intrinsic timescales of local spiking activity within columns of area V4 in male monkeys performing spatial attention tasks. The ongoing spiking activity unfolded across at least two distinct timescales, fast and slow. The slow timescale increased when monkeys attended to the receptive fields location and correlated with reaction times. By evaluating predictions of several network models, we found that spatiotemporal correlations in V4 activity were best explained by the model in which multiple timescales arise from recurrent interactions shaped by spatially arranged connectivity, and attentional modulation of timescales results from an increase in the efficacy of recurrent interactions. Our results suggest that multiple timescales may arise from the spatial connectivity in the visual cortex and flexibly change with the cognitive state due to dynamic effective interactions between neurons.

Emerging technologies for behavioral research in changing environments

The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.

Scalable Apparatus to Measure Posture and Locomotion (SAMPL): a high-throughput solution to study unconstrained vertical behavior in small animals

Balance and movement are impaired in a wide variety of neurological disorders. Recent advances in behavioral monitoring provide unprecedented access to posture and locomotor kinematics, but without the throughput and scalability necessary to screen candidate genes / potential therapeutics. We present a powerful solution: a Scalable Apparatus to Measure Posture and Locomotion (SAMPL). SAMPL includes extensible imaging hardware and low-cost open-source acquisition software with real-time processing. We first demonstrate that SAMPL's hardware and acquisition software can acquire data from from D. melanogaster, C. elegans, and D. rerio as they move vertically. Next, we leverage SAMPL's throughput to rapidly (two weeks) gather a new zebrafish dataset. We use SAMPL's analysis and visualization tools to replicate and extend our current understanding of how zebrafish balance as they navigate through a vertical environment. Next, we discover (1) that key kinematic parameters vary systematically with genetic background, and (2) that such background variation is small relative to the changes that accompany early development. Finally, we simulate SAMPL's ability to resolve differences in posture or vertical navigation as a function of affect size and data gathered -- key data for screens. Taken together, our apparatus, data, and analysis provide a powerful solution for labs using small animals to investigate balance and locomotor disorders at scale. More broadly, SAMPL is both an adaptable resource for labs looking process videographic measures of behavior in real-time, and an exemplar of how to scale hardware to enable the throughput necessary for screening.

Leveraging big data to uncover the eco-evolutionary factors shaping behavioural development

Mapping the eco-evolutionary factors shaping the development of animals' behavioural phenotypes remains a great challenge. Recent advances in 'big behavioural data' research-the high-resolution tracking of individuals and the harnessing of that data with powerful analytical tools-have vastly improved our ability to measure and model developing behavioural phenotypes. Applied to the study of behavioural ontogeny, the unfolding of whole behavioural repertoires can be mapped in unprecedented detail with relative ease. This overcomes long-standing experimental bottlenecks and heralds a surge of studies that more finely define and explore behavioural-experiential trajectories across development. In this review, we first provide a brief guide to state-of-the-art approaches that allow the collection and analysis of high-resolution behavioural data across development. We then outline how such approaches can be used to address key issues regarding the ecological and evolutionary factors shaping behavioural development: developmental feedbacks between behaviour and underlying states, early life effects and behavioural transitions, and information integration across development.

Social behavioral profiling by unsupervised deep learning reveals a stimulative effect of dopamine D3 agonists on zebrafish sociality

It has been a major challenge to systematically evaluate and compare how pharmacological perturbations influence social behavioral outcomes. Although some pharmacological agents are known to alter social behavior, precise description and quantification of such effects have proven difficult. We developed a scalable social behavioral assay for zebrafish named ZeChat based on unsupervised deep learning to characterize sociality at high resolution. High-dimensional and dynamic social behavioral phenotypes are automatically classified using this method. By screening a neuroactive compound library, we found that different classes of chemicals evoke distinct patterns of social behavioral fingerprints. By examining these patterns, we discovered that dopamine D3 agonists possess a social stimulative effect on zebrafish. The D3 agonists pramipexole, piribedil, and 7-hydroxy-DPAT-HBr rescued social deficits in a valproic-acid-induced zebrafish autism model. The ZeChat platform provides a promising approach for dissecting the pharmacology of social behavior and discovering novel social-modulatory compounds.

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes

Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents' emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.

Structural analyses in the study of behavior: From rodents to non-human primates

The term "structure" indicates a set of components that, in relation to each other, shape an organic complex. Such a complex takes on essential connotations of functionally unitary entity resulting from the mutual relationships of its constituent elements. In a broader sense, we can use the word "structure" to define the set of relationships among the elements of an emergent system that is not determined by the mere algebraic sum of these elements, but by the interdependence relationships of these components from which the function of the entire structure itself derives. The behavior of an integrated living being can be described in structural terms via an ethogram, defined as an itemized list of behavioral units. Akin to an architectural structure, a behavioral structure arises from the reciprocal relationships that the individual units of behavior establish. Like an architectural structure, the function of the resulting behaving complex emerges from the relationships of the parts. Hence, studying behavior in its wholeness necessitates not only the identification of its constitutive units in their autarchic individuality, but also, and importantly, some understanding of their relationships. This paper aimed to critically review different methods to study behavior in structural terms. First, we emphasized the utilization of T-pattern analysis, i.e., one of the most effective and reliable tools to provide structural information on behavior. Second, we discussed the application of other methodological approaches that are based on the analysis of transition matrices, such as hierarchical clustering, stochastic analyses, and adjusted residuals. Unlike T-pattern analysis, these methods allow researchers to explore behavioral structure beyond its temporal characteristics and through other relational constraints. After an overview of how these methods are used in the study of animal behavior, from rodents to non-human primates, we discussed the specificities, advantages and challenges of each approach. This paper could represent a useful background for all scientists who intend to study behavior both quantitatively and structurally, that is in terms of the reciprocal relationships that the various units of a given behavioral repertoire normally weave together.

Inference, Prediction, & Entropy-Rate Estimation of Continuous-Time, Discrete-Event Processes

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

A unicellular walker controlled by a microtubule-based finite-state machine

Cells are complex biochemical systems whose behaviors emerge from interactions among myriad molecular components. Computation is often invoked as a general framework for navigating this cellular complexity. However, it is unclear how cells might embody computational processes such that the theories of computation, including finite-state machine models, could be productively applied. Here, we demonstrate finite-state-machine-like processing embodied in cells using the walking behavior of Euplotes eurystomus, a ciliate that walks across surfaces using fourteen motile appendages (cirri). We found that cellular walking entails regulated transitions among a discrete set of gait states. The set of observed transitions decomposes into a small group of high-probability, temporally irreversible transitions and a large group of low-probability, time-symmetric transitions, thus revealing stereotypy in the sequential patterns of state transitions. Simulations and experiments suggest that the sequential logic of the gait is functionally important. Taken together, these findings implicate a finite-state-machine-like process. Cirri are connected by microtubule bundles (fibers), and we found that the dynamics of cirri involved in different state transitions are associated with the structure of the fiber system. Perturbative experiments revealed that the fibers mediate gait coordination, suggesting a mechanical basis of gait control.

The emergence and influence of internal states

Animal behavior is shaped by a variety of "internal states"-partially hidden variables that profoundly shape perception, cognition, and action. The neural basis of internal states, such as fear, arousal, hunger, motivation, aggression, and many others, is a prominent focus of research efforts across animal phyla. Internal states can be inferred from changes in behavior, physiology, and neural dynamics and are characterized by properties such as pleiotropy, persistence, scalability, generalizability, and valence. To date, it remains unclear how internal states and their properties are generated by nervous systems. Here, we review recent progress, which has been driven by advances in behavioral quantification, cellular manipulations, and neural population recordings. We synthesize research implicating defined subsets of state-inducing cell types, widespread changes in neural activity, and neuromodulation in the formation and updating of internal states. In addition to highlighting the significance of these findings, our review advocates for new approaches to clarify the underpinnings of internal brain states across the animal kingdom.

The vertical light-gradient and its potential impact on animal distribution and behavior

The visual environment provides vital cues allowing animals to assess habitat quality, weather conditions or measure time of day. Together with other sensory cues and physiological conditions, the visual environment sets behavioral states that make the animal more prone to engage in some behaviors, and less in others. This master-control of behavior serves a fundamental and essential role in determining the distribution and behavior of all animals. Although it is obvious that visual information contains vital input for setting behavioral states, the precise nature of these visual cues remains unknown. Here we use a recently described method to quantify the distribution of light reaching animals’ eyes in different environments. The method records the vertical gradient (as a function of elevation angle) of intensity, spatial structure and spectral balance. Comparison of measurements from different types of environments, weather conditions, times of day, and seasons reveal that these aspects can be readily discriminated from one another. The vertical gradients of radiance, spatial structure (contrast) and color are thus reliable indicators that are likely to have a strong impact on animal behavior and spatial distribution.

Neural mechanisms underlying the temporal organization of naturalistic animal behavior

Naturalistic animal behavior exhibits a strikingly complex organization in the temporal domain, with variability arising from at least three sources: hierarchical, contextual, and stochastic. What neural mechanisms and computational principles underlie such intricate temporal features? In this review, we provide a critical assessment of the existing behavioral and neurophysiological evidence for these sources of temporal variability in naturalistic behavior. Recent research converges on an emergent mechanistic theory of temporal variability based on attractor neural networks and metastable dynamics, arising via coordinated interactions between mesoscopic neural circuits. We highlight the crucial role played by structural heterogeneities as well as noise from mesoscopic feedback loops in regulating flexible behavior. We assess the shortcomings and missing links in the current theoretical and experimental literature and propose new directions of investigation to fill these gaps.

T cell morphodynamics reveal periodic shape oscillations in three-dimensional migration

T cells use sophisticated shape dynamics (morphodynamics) to migrate towards and neutralize infected and cancerous cells. However, there is limited quantitative understanding of the migration process in three-dimensional extracellular matrices (ECMs) and across timescales. Here, we leveraged recent advances in lattice light-sheet microscopy to quantitatively explore the three-dimensional morphodynamics of migrating T cells at high spatio-temporal resolution. We first developed a new shape descriptor based on spherical harmonics, incorporating key polarization information of the uropod. We found that the shape space of T cells is low-dimensional. At the behavioural level, run-and-stop migration modes emerge at approximately 150 s, and we mapped the morphodynamic composition of each mode using multiscale wavelet analysis, finding 'stereotyped' motifs. Focusing on the run mode, we found morphodynamics oscillating periodically (every approx. 100 s) that can be broken down into a biphasic process: front-widening with retraction of the uropod, followed by a rearward surface motion and forward extension, where intercalation with the ECM in both of these steps likely facilitates forward motion. Further application of these methods may enable the comparison of T cell migration across different conditions (e.g. differentiation, activation, tissues and drug treatments) and improve the precision of immunotherapeutic development.

Deep phenotyping reveals movement phenotypes in mouse neurodevelopmental models

BACKGROUND

Repetitive action, resistance to environmental change and fine motor disruptions are hallmarks of autism spectrum disorder (ASD) and other neurodevelopmental disorders, and vary considerably from individual to individual. In animal models, conventional behavioral phenotyping captures such fine-scale variations incompletely. Here we observed male and female C57BL/6J mice to methodically catalog adaptive movement over multiple days and examined two rodent models of developmental disorders against this dynamic baseline. We then investigated the behavioral consequences of a cerebellum-specific deletion in Tsc1 protein and a whole-brain knockout in Cntnap2 protein in mice. Both of these mutations are found in clinical conditions and have been associated with ASD.

METHODS

We used advances in computer vision and deep learning, namely a generalized form of high-dimensional statistical analysis, to develop a framework for characterizing mouse movement on multiple timescales using a single popular behavioral assay, the open-field test. The pipeline takes virtual markers from pose estimation to find behavior clusters and generate wavelet signatures of behavior classes. We measured spatial and temporal habituation to a new environment across minutes and days, different types of self-grooming, locomotion and gait.

RESULTS

Both Cntnap2 knockouts and L7-Tsc1 mutants showed forelimb lag during gait. L7-Tsc1 mutants and Cntnap2 knockouts showed complex defects in multi-day adaptation, lacking the tendency of wild-type mice to spend progressively more time in corners of the arena. In L7-Tsc1 mutant mice, failure to adapt took the form of maintained ambling, turning and locomotion, and an overall decrease in grooming. However, adaptation in these traits was similar between wild-type mice and Cntnap2 knockouts. L7-Tsc1 mutant and Cntnap2 knockout mouse models showed different patterns of behavioral state occupancy.

LIMITATIONS

Genetic risk factors for autism are numerous, and we tested only two. Our pipeline was only done under conditions of free behavior. Testing under task or social conditions would reveal more information about behavioral dynamics and variability.

CONCLUSIONS

Our automated pipeline for deep phenotyping successfully captures model-specific deviations in adaptation and movement as well as differences in the detailed structure of behavioral dynamics. The reported deficits indicate that deep phenotyping constitutes a robust set of ASD symptoms that may be considered for implementation in clinical settings as quantitative diagnosis criteria.

Long-range sequential dependencies precede complex syntactic production in language acquisition

To convey meaning, human language relies on hierarchically organized, long-range relationships spanning words, phrases, sentences and discourse. As the distances between elements (e.g. phonemes, characters, words) in human language sequences increase, the strength of the long-range relationships between those elements decays following a power law. This power-law relationship has been attributed variously to long-range sequential organization present in human language syntax, semantics and discourse structure. However, non-linguistic behaviours in numerous phylogenetically distant species, ranging from humpback whale song to fruit fly motility, also demonstrate similar long-range statistical dependencies. Therefore, we hypothesized that long-range statistical dependencies in human speech may occur independently of linguistic structure. To test this hypothesis, we measured long-range dependencies in several speech corpora from children (aged 6 months-12 years). We find that adult-like power-law statistical dependencies are present in human vocalizations at the earliest detectable ages, prior to the production of complex linguistic structure. These linguistic structures cannot, therefore, be the sole cause of long-range statistical dependencies in language.

Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group

The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an "input" signal X while maximizing its mutual information with some stochastic "relevance" variable Y. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be "discovered" by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of "large scale" structure, such as biological function, on an RG procedure.

Measuring the repertoire of age-related behavioral changes in Drosophila melanogaster

Aging affects almost all aspects of an organism—its morphology, its physiology, its behavior. Isolating which biological mechanisms are regulating these changes, however, has proven difficult, potentially due to our inability to characterize the full repertoire of an animal’s behavior across the lifespan. Using data from fruit flies (D. melanogaster) we measure the full repertoire of behaviors as a function of age. We observe a sexually dimorphic pattern of changes in the behavioral repertoire during aging. Although the stereotypy of the behaviors and the complexity of the repertoire overall remains relatively unchanged, we find evidence that the observed alterations in behavior can be explained by changing the fly’s overall energy budget, suggesting potential connections between metabolism, aging, and behavior.

Aging is a ubiquitous biological phenomenon that affects many aspects of an animal’s appearance, physiology, and behavior. Our understanding of how changes in physiology lead to behavioral changes, however, has been partially limited by our ability to robustly quantify how behavior alters over timescales of days and weeks. In this study, we measure a large repertoire of behaviors of fruit flies at various ages, finding how the actions the animals perform shift with age. We observe a difference between the aging dynamics of male and female flies, and we show that many of these changes can be explained with a model of energy consumption, leading us to make predictions as to the role of metabolism in changes in aging behavior.

The evolution of brain architectures for predictive coding and active inference

This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

The Evolution of Visual Roles – Ancient Vision Versus Object Vision

Just like other complex biological features, image vision (multi-pixel light sensing) did not evolve suddenly. Animal visual systems have a long prehistory of non-imaging light sensitivity. The first spatial vision was likely very crude with only few pixels, and evolved to improve orientation behaviors previously supported by single-channel directional photoreception. The origin of image vision was simply a switch from single to multiple spatial channels, which improved the behaviors for finding a suitable habitat and position itself within it. Orientation based on spatial vision obviously involves active guidance of behaviors but, by necessity, also assessment of habitat suitability and environmental conditions. These conditions are crucial for deciding when to forage, reproduce, seek shelter, rest, etc. When spatial resolution became good enough to see other animals and interact with them, a whole range of new visual roles emerged: pursuit, escape, communication and other interactions. All these new visual roles require entirely new types of visual processing. Objects needed to be separated from the background, identified and classified to make the correct choice of interaction. Object detection and identification can be used actively to guide behaviors but of course also to assess the over-all situation. Visual roles can thus be classified as either ancient non-object-based tasks, or object vision. Each of these two categories can also be further divided into active visual tasks and visual assessment tasks. This generates four major categories of vision into which I propose that all visual roles can be categorized.