Deep learning tools for the measurement of animal behavior in neuroscience

Next generation behavioral sequencing for advancing pain quantification

With symptoms such as spontaneous pain and pathologically heightened sensitivity to stimuli, chronic pain accounts for about 20% of physician visits and up to 2/3 of patients are dissatisfied with current treatments. Much of our knowledge on pain processing and analgesics has emerged from behavioral studies performed on animals presenting the same symptoms under pathological conditions. While humans can verbally describe their pain, studies on rodents have relied on behavioral assays providing non-exhaustive characterization or altering animals' original sensitivity through repetitive stimulations. The emergence of what we term "next-generation behavioral sequencing" is now permitting us to quantitatively describe behavioral features on millisecond to minutes long timescales that lie beyond easy detection with the unaided eye. Here, we summarize emerging videography and computational based behavioral approaches that have the potential to significantly improve pain research.

Machine learning and deep learning frameworks for the automated analysis of pain and opioid withdrawal behaviors

The automation of behavioral tracking and analysis in preclinical research can serve to advance the rate of research outcomes, increase experimental scalability, and challenge the scientific reproducibility crisis. Recent advances in the efficiency, accuracy, and accessibility of deep learning (DL) and machine learning (ML) frameworks are enabling this automation. As the ongoing opioid epidemic continues to worsen alongside increasing rates of chronic pain, there are ever-growing needs to understand opioid use disorders (OUDs) and identify non-opioid therapeutic options for pain. In this review, we examine how these related needs can be advanced by the development and validation of DL and ML resources for automated pain and withdrawal behavioral tracking. We aim to emphasize the utility of these tools for automated behavioral analysis, and we argue that currently developed models should be deployed to address novel questions in the fields of pain and OUD research.

Functional Gait Assessment Using Manual, Semi-Automated and Deep Learning Approaches Following Standardized Models of Peripheral Nerve Injury in Mice

Objective: To develop a standardized model of stretch−crush sciatic nerve injury in mice, and to compare outcomes of crush and novel stretch−crush injuries using standard manual gait and sensory assays, and compare them to both semi-automated as well as deep-learning gait analysis methods. Methods: Initial studies in C57/Bl6 mice were used to develop crush and stretch−crush injury models followed by histologic analysis. In total, 12 eight-week-old 129S6/SvEvTac mice were used in a six-week behavioural study. Behavioral assessments using the von Frey monofilament test and gait analysis recorded on a DigiGait platform and analyzed through both Visual Gait Lab (VGL) deep learning and standardized sciatic functional index (SFI) measurements were evaluated weekly. At the termination of the study, neurophysiological nerve conduction velocities were recorded, calf muscle weight ratios measured and histological analyses performed. Results: Histological evidence confirmed more severe histomorphological injury in the stretch−crush injured group compared to the crush-only injured group at one week post-injury. Von Frey monofilament paw withdrawal was significant for both groups at week one compared to baseline (p < 0.05), but not between groups with return to baseline at week five. SFI showed hindered gait at week one and two for both groups, compared to baseline (p < 0.0001), with return to baseline at week five. Hind stance width (HSW) showed similar trends as von Frey monofilament test as well as SFI measurements, yet hind paw angle (HPA) peaked at week two. Nerve conduction velocity (NCV), measured six weeks post-injury, at the termination of the study, did not show any significant difference between the two groups; yet, calf muscle weight measurements were significantly different between the two, with the stretch−crush group demonstrating a lower (poorer) weight ratio relative to uninjured contralateral legs (p < 0.05). Conclusion: Stretch−crush injury achieved a more reproducible and constant injury compared to crush-only injuries, with at least a Sunderland grade 3 injury (perineurial interruption) in histological samples one week post-injury in the former. However, serial behavioral outcomes were comparable between the two crush groups, with similar kinetics of recovery by von Frey testing, SFI and certain VGL parameters, the latter reported for the first time in rodent peripheral nerve injury. Semi-automated and deep learning-based approaches for gait analysis are promising, but require further validation for evaluation in murine hind-limb nerve injuries.

Using DeepLabCut as a Real-Time and Markerless Tool for Cardiac Physiology Assessment in Zebrafish

DeepLabCut (DLC) is a deep learning-based tool initially invented for markerless pose estimation in mammals. In this study, we explored the possibility of adopting this tool for conducting markerless cardiac physiology assessment in an important aquatic toxicology model of zebrafish (Danio rerio). Initially, high-definition videography was applied to capture heartbeat information at a frame rate of 30 frames per second (fps). Next, 20 videos from different individuals were used to perform convolutional neural network training by labeling the heart chamber (ventricle) with eight landmarks. Using Residual Network (ResNet) 152, a neural network with 152 convolutional neural network layers with 500,000 iterations, we successfully obtained a trained model that can track the heart chamber in a real-time manner. Later, we validated DLC performance with the previously published ImageJ Time Series Analysis (TSA) and Kymograph (KYM) methods. We also evaluated DLC performance by challenging experimental animals with ethanol and ponatinib to induce cardiac abnormality and heartbeat irregularity. The results showed that DLC is more accurate than the TSA method in several parameters tested. The DLC-trained model also detected the ventricle of zebrafish embryos even in the occurrence of heart abnormalities, such as pericardial edema. We believe that this tool is beneficial for research studies, especially for cardiac physiology assessment in zebrafish embryos.

The emergence and influence of internal states

Animal behavior is shaped by a variety of "internal states"-partially hidden variables that profoundly shape perception, cognition, and action. The neural basis of internal states, such as fear, arousal, hunger, motivation, aggression, and many others, is a prominent focus of research efforts across animal phyla. Internal states can be inferred from changes in behavior, physiology, and neural dynamics and are characterized by properties such as pleiotropy, persistence, scalability, generalizability, and valence. To date, it remains unclear how internal states and their properties are generated by nervous systems. Here, we review recent progress, which has been driven by advances in behavioral quantification, cellular manipulations, and neural population recordings. We synthesize research implicating defined subsets of state-inducing cell types, widespread changes in neural activity, and neuromodulation in the formation and updating of internal states. In addition to highlighting the significance of these findings, our review advocates for new approaches to clarify the underpinnings of internal brain states across the animal kingdom.

An Attention-Refined Light-Weight High-Resolution Network for Macaque Monkey Pose Estimation

Macaque monkey is a rare substitute which plays an important role for human beings in relation to psychological and spiritual science research. It is essential for these studies to accurately estimate the pose information of macaque monkeys. Many large-scale models have achieved state-of-the-art results in pose macaque estimation. However, it is difficult to deploy when computing resources are limited. Combining the structure of high-resolution network and the design principle of light-weight network, we propose the attention-refined light-weight high-resolution network for macaque monkey pose estimation (HR-MPE). The multi-branch parallel structure is adopted to maintain high-resolution representation throughout the process. Moreover, a novel basic block is designed by a powerful transformer structure and polarized self-attention, where there is a simple structure and fewer parameters. Two attention refined blocks are added at the end of the parallel structure, which are composed of light-weight asymmetric convolutions and a triplet attention with almost no parameter, obtaining richer representation information. An unbiased data processing method is also utilized to obtain an accurate flipping result. The experiment is conducted on a macaque dataset containing more than 13,000 pictures. Our network has reached a 77.0 AP score, surpassing HRFormer with fewer parameters by 1.8 AP.

Blueprints for measuring natural behavior

Until recently laboratory tasks for studying behavior were highly artificial, simplified, and designed without consideration for the environmental or social context. Although such an approach offers good control over behavior, it does not allow for researching either voluntary responses or individual differences. Importantly for neuroscience studies, the activity of the neural circuits involved in producing unnatural, artificial behavior is variable and hard to predict. In addition, different ensembles may be activated depending on the strategy the animal adopts to deal with the spurious problem. Thus, artificial and simplified tasks based on responses, which do not occur spontaneously entail problems with modeling behavioral impairments and underlying brain deficits. To develop valid models of human disorders we need to test spontaneous behaviors consistently engaging well-defined, evolutionarily conserved neuronal circuits. Such research focuses on behavioral patterns relevant for surviving and thriving under varying environmental conditions, which also enable high reproducibility across different testing settings.

Motor Control: A Conceptual Framework for Rehabilitation

There is a lack of conceptual and theoretical clarity among clinicians and researchers regarding the control of motor actions based on the use of the term "motor control." It is important to differentiate control processes from observations of motor output to improve communication and to make progress in understanding motor disorders and their remediation. This article clarifies terminology related to theoretical concepts underlying the control of motor actions, emphasizing how the term "motor control" is applied in neurorehabilitation. Two major opposing theoretical frameworks are described (i.e., direct and indirect), and their strengths and pitfalls are discussed. Then, based on the proposition that sensorimotor rehabilitation should be predicated on one comprehensive theory instead of an eclectic mix of theories and models, several solutions are offered about how to address controversies in motor learning, optimality, and adaptability of movement.

The Promise of Behavioral Tracking Systems for Advancing Primate Animal Welfare

Recent years have witnessed major advances in the ability of computerized systems to track the positions of animals as they move through large and unconstrained environments. These systems have so far been a great boon in the fields of primatology, psychology, neuroscience, and biomedicine. Here, we discuss the promise of these technologies for animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare within the context of the three Rs of animal research (reduction, refinement, and replacement), and applying our understanding of animal behavior to increase the "natural" behaviors in captive and wild populations facing human impact challenges. We note that these benefits are often incidental to the designed purpose of these tracking systems, a reflection of the fact that animal welfare is not inimical to research progress, but instead, that the aligned interests between basic research and welfare hold great promise for improvements to animal well-being.

Advancements in Genomic and Behavioral Neuroscience Analysis for the Study of Normal and Pathological Brain Function

Psychiatric and neurological disorders are influenced by an undetermined number of genes and molecular pathways that may differ among afflicted individuals. Functionally testing and characterizing biological systems is essential to discovering the interrelationship among candidate genes and understanding the neurobiology of behavior. Recent advancements in genetic, genomic, and behavioral approaches are revolutionizing modern neuroscience. Although these tools are often used separately for independent experiments, combining these areas of research will provide a viable avenue for multidimensional studies on the brain. Herein we will briefly review some of the available tools that have been developed for characterizing novel cellular and animal models of human disease. A major challenge will be openly sharing resources and datasets to effectively integrate seemingly disparate types of information and how these systems impact human disorders. However, as these emerging technologies continue to be developed and adopted by the scientific community, they will bring about unprecedented opportunities in our understanding of molecular neuroscience and behavior.

The what, how, and why of naturalistic behavior

In the past few years, advances in machine learning have fueled an explosive growth of descriptive and generative models of animal behavior. These new approaches offer higher levels of detail and granularity than has previously been possible, allowing for fine-grained segmentation of animals' actions and precise quantitative mappings between an animal's sensory environment and its behavior. How can these new methods help us understand the governing principles shaping complex and naturalistic behavior? In this review, we will recap ways in which our ability to detect and model behavior have improved in recent years, and consider how these techniques might be used to revisit classical normative theories of behavioral control.

Studies of the Behavioral Sequences: The Neuroethological Morphology Concept Crossing Ethology and Functional Morphology

Simple Summary

Behavioral sequences analysis is a relevant method for quantifying the behavioral repertoire of animals to respond to the classical Tinbergen’s four questions. Research in ethology and functional morphology intercepts at the level of analysis of behaviors through the recording and interpretation of data from of movement sequence studies with various types of imaging and sensor systems. We propose the concept of Neuroethological morphology to build a holistic framework for understanding animal behavior. This concept integrates ethology (including behavioral ecology and neuroethology) with functional morphology (including biomechanics and physics) to provide a heuristic approach in behavioral biology.

Abstract

Postures and movements have been one of the major modes of human expression for understanding and depicting organisms in their environment. In ethology, behavioral sequence analysis is a relevant method to describe animal behavior and to answer Tinbergen’s four questions testing the causes of development, mechanism, adaptation, and evolution of behaviors. In functional morphology (and in biomechanics), the analysis of behavioral sequences establishes the motor pattern and opens the discussion on the links between “form” and “function”. We propose here the concept of neuroethological morphology in order to build a holistic framework for understanding animal behavior. This concept integrates ethology with functional morphology, and physics. Over the past hundred years, parallel developments in both disciplines have been rooted in the study of the sequential organization of animal behavior. This concept allows for testing genetic, epigenetic, and evo-devo predictions of phenotypic traits between structures, performances, behavior, and fitness in response to environmental constraints. Based on a review of the literature, we illustrate this concept with two behavioral cases: (i) capture behavior in squamates, and (ii) the ritualistic throat display in lizards.

On the dimensionality of behavior

How do we characterize animal behavior? Psychophysics started with human behavior in the laboratory, and focused on simple contexts, such as the decision among just a few alternative actions in response to sensory inputs. In contrast, ethology focused on animal behavior in the natural environment, emphasizing that evolution selects potentially complex behaviors that are useful in specific contexts. New experimental methods now make it possible to monitor animal and human behaviors in vastly greater detail. This “physics of behavior” holds the promise of combining the psychophysicist’s quantitative approach with the ethologist’s appreciation of natural context. One question surrounding this growing body of data concerns the dimensionality of behavior. Here I try to give this concept a precise definition.

There is a growing effort in the “physics of behavior” that aims at complete quantitative characterization of animal movements under more complex, naturalistic conditions. One reaction to the resulting explosion of high-dimensional data is the search for low-dimensional structure. Here I try to define more clearly what we mean by the dimensionality of behavior, where observable behavior may consist of either continuous trajectories or sequences of discrete states. This discussion also serves to isolate situations in which the dimensionality of behavior is effectively infinite.

Inflammation-driven brain and gut barrier dysfunction in stress and mood disorders

Regulation of emotions is generally associated exclusively with the brain. However, there is evidence that peripheral systems are also involved in mood, stress vulnerability vs. resilience, and emotion-related memory encoding. Prevalence of stress and mood disorders such as major depression, bipolar disorder, and post-traumatic stress disorder is increasing in our modern societies. Unfortunately, 30%-50% of individuals respond poorly to currently available treatments highlighting the need to further investigate emotion-related biology to gain mechanistic insights that could lead to innovative therapies. Here, we provide an overview of inflammation-related mechanisms involved in mood regulation and stress responses discovered using animal models. If clinical studies are available, we discuss translational value of these findings including limitations. Neuroimmune mechanisms of depression and maladaptive stress responses have been receiving increasing attention, and thus, the first part is centered on inflammation and dysregulation of brain and circulating cytokines in stress and mood disorders. Next, recent studies supporting a role for inflammation-driven leakiness of the blood-brain and gut barriers in emotion regulation and mood are highlighted. Stress-induced exacerbated inflammation fragilizes these barriers which become hyperpermeable through loss of integrity and altered biology. At the gut level, this could be associated with dysbiosis, an imbalance in microbial communities, and alteration of the gut-brain axis which is central to production of mood-related neurotransmitter serotonin. Novel therapeutic approaches such as anti-inflammatory drugs, the fast-acting antidepressant ketamine, and probiotics could directly act on the mechanisms described here improving mood disorder-associated symptomatology. Discovery of biomarkers has been a challenging quest in psychiatry, and we end by listing promising targets worth further investigation.

Challenges of a small world analysis for the continuous monitoring of behavior in mice

Documenting a mouse's "real world" behavior in the "small world" of a laboratory cage with continuous video recordings offers insights into phenotypical expression of mouse genotypes, development and aging, and neurological disease. Nevertheless, there are challenges in the design of a small world, the behavior selected for analysis, and the form of the analysis used. Here we offer insights into small world analyses by describing how acute behavioral procedures can guide continuous behavioral methodology. We show how algorithms can identify behavioral acts including walking and rearing, circadian patterns of action including sleep duration and waking activity, and the organization of patterns of movement into home base activity and excursions, and how they are altered with aging. We additionally describe how specific tests can be incorporated within a mouse's living arrangement. We emphasize how machine learning can condense and organize continuous activity that extends over extended periods of time.

A Large-Scale Mouse Pose Dataset for Mouse Pose Estimation

Mouse pose estimations have important applications in the fields of animal behavior research, biomedicine, and animal conservation studies. Accurate and efficient mouse pose estimations using computer vision are necessary. Although methods for mouse pose estimations have developed, bottlenecks still exist. One of the most prominent problems is the lack of uniform and standardized training datasets. Here, we resolve this difficulty by introducing the mouse pose dataset. Our mouse pose dataset contains 40,000 frames of RGB images and large-scale 2D ground-truth motion images. All the images were captured from interacting lab mice through a stable single viewpoint, including 5 distinct species and 20 mice in total. Moreover, to improve the annotation efficiency, five keypoints of mice are creatively proposed, in which one keypoint is at the center and the other two pairs of keypoints are symmetric. Then, we created simple, yet effective software that works for annotating images. It is another important link to establish a benchmark model for 2D mouse pose estimations. We employed modified object detections and pose estimation algorithms to achieve precise, effective, and robust performances. As the first large and standardized mouse pose dataset, our proposed mouse pose dataset will help advance research on animal pose estimations and assist in application areas related to animal experiments.

Optogenetic and thermogenetic manipulation of defined neural circuits and behaviors in Drosophila

Neural network dynamics underlying flexible animal behaviors remain elusive. The fruit fly Drosophila melanogaster is considered an excellent model in behavioral neuroscience because of its simple neuroanatomical architecture and the availability of various genetic methods. Moreover, Drosophila larvae's transparent body allows investigators to use optical methods on freely moving animals, broadening research directions. Activating or inhibiting well-defined events in excitable cells with a fine temporal resolution using optogenetics and thermogenetics led to the association of functions of defined neural populations with specific behavioral outputs such as the induction of associative memory. Furthermore, combining optogenetics and thermogenetics with state-of-the-art approaches, including connectome mapping and machine learning-based behavioral quantification, might provide a complete view of the experience- and time-dependent variations of behavioral responses. These methodologies allow further understanding of the functional connections between neural circuits and behaviors such as chemosensory, motivational, courtship, and feeding behaviors and sleep, learning, and memory.

Multi-animal pose estimation, identification and tracking with DeepLabCut

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking-features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal's identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development.

Automated and Dynamic Control of Chemical Content in Droplets for Scalable Screens of Small Animals

Screening functional phenotypes in small animals is important for genetics and drug discovery. Multiphase microfluidics has great potential for enhancing throughput but has been hampered by inefficient animal encapsulation and limited control over the animal's environment in droplets. Here, a highly efficient single-animal encapsulation unit, a liquid exchanger system for controlling the droplet chemical environment dynamically, and an automation scheme for the programming and robust execution of complex protocols are demonstrated. By careful use of interfacial forces, the liquid exchanger unit allows for adding and removing chemicals from a droplet and, therefore, generating chemical gradients inaccessible in previous multiphase systems. Using Caenorhabditis elegans as an example, it is demonstrated that these advances can serve to analyze dynamic phenotyping, such as behavior and neuronal activity, perform forward genetic screen, and are scalable to manipulate animals of different sizes. This platform paves the way for large-scale screens of complex dynamic phenotypes in small animals.

Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking

Accurate tracking is the basis of behavioral analysis, an important research method in neuroscience and many other fields. However, the currently available tracking methods have limitations. Traditional computer vision methods have problems in complex environments, and deep learning methods are hard to be applied universally due to the requirement of laborious annotations. To address the trade-off between accuracy and universality, we developed an easy-to-use tracking tool, Siamese Network-based All-Purpose Tracker (SNAP-Tracker), a model-free tracking software built on the Siamese network. The pretrained Siamese network offers SNAP-Tracker a remarkable feature extraction ability to keep tracking accuracy, and the model-free design makes it usable directly before laborious annotations and network refinement. SNAP-Tracker provides a “tracking with detection” mode to track longer videos with an additional detection module. We demonstrate the stability of SNAP-Tracker through different experimental conditions and different tracking tasks. In short, SNAP-Tracker provides a general solution to behavioral tracking without compromising accuracy. For the user’s convenience, we have integrated the tool into a tidy graphic user interface and opened the source code for downloading and using (https://github.com/slh0302/SNAP).

MouseVenue3D: A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice

Understanding the connection between brain and behavior in animals requires precise monitoring of their behaviors in three-dimensional (3-D) space. However, there is no available three-dimensional behavior capture system that focuses on rodents. Here, we present MouseVenue3D, an automated and low-cost system for the efficient capture of 3-D skeleton trajectories in markerless rodents. We improved the most time-consuming step in 3-D behavior capturing by developing an automatic calibration module. Then, we validated this process in behavior recognition tasks, and showed that 3-D behavioral data achieved higher accuracy than 2-D data. Subsequently, MouseVenue3D was combined with fast high-resolution miniature two-photon microscopy for synchronous neural recording and behavioral tracking in the freely-moving mouse. Finally, we successfully decoded spontaneous neuronal activity from the 3-D behavior of mice. Our findings reveal that subtle, spontaneous behavior modules are strongly correlated with spontaneous neuronal activity patterns.

Post-Stroke Hemiplegic Rodent Evaluation: A Framework for Assessing Forelimb Movement Quality Using Kinematics

Kinematics is the gold-standard method for measuring detailed joint motions. Recent research demonstrates that post-stroke kinematic analysis in rats reveals reaching abnormalities similar to those seen in humans after stroke. Nonetheless, behavioral neuroscientists have failed to incorporate kinematic methods for assessing movement quality in stroke models. The availability of a user-friendly method to assess multi-segment forelimb kinematics models should greatly increase uptake of this approach. Here, we present a framework for multi-segment forelimb analysis in rodents after stroke. This method greatly enhances the understanding of post-stroke forelimb motor recovery by including several movement quality metrics often used in human clinical work, such as upper-limb linear and angular kinematics, movement smoothness and kinetics, abnormal synergies, and compensations. These metrics may constitute a preclinical surrogate for the Fugl-Meyer assessment of hemiplegic patients. The data obtained using this method are 83 outputs of linear and angular kinematics and kinetics. The outputs also include 24 time series of continuous data, which afford a graphical representation of the kinematics and kinetics of the reaching cycle. We show that post-stroke rodents displayed many features resembling those seen in humans after stroke that are evident only when multi-segment kinematics models are considered. This method expands the knowledge derived from methods constrained to paw movements to a multi-segment forelimb movement quality framework. Moreover, it highlights the need for preclinical work to consider more sensitive measures of sensorimotor impairment and recovery as a means to enhance the interpretation of true recovery and compensation. © 2022 Wiley Periodicals LLC. Basic Protocol: Recording and data analysis of rodents performing the Montoya staircase task.

Emerging Role of Translational Digital Biomarkers Within Home Cage Monitoring Technologies in Preclinical Drug Discovery and Development

In drug discovery and development, traditional assessment of human patients and preclinical subjects occurs at limited time points in potentially stressful surroundings (i.e., the clinic or a test arena), which can impact data quality and welfare. However, recent advances in remote digital monitoring technologies enable the assessment of human patients and preclinical subjects across multiple time points in familiar surroundings. The ability to monitor a patient throughout disease progression provides an opportunity for more relevant and efficient diagnosis as well as improved assessment of drug efficacy and safety. In preclinical in vivo animal models, these digital technologies allow for continuous, longitudinal, and non-invasive monitoring in the home environment. This manuscript provides an overview of digital monitoring technologies for use in preclinical studies including their history and evolution, current engagement through use cases, and impact of digital biomarkers (DBs) on drug discovery and the 3Rs. We also discuss barriers to implementation and strategies to overcome them. Finally, we address data consistency and technology standards from the perspective of technology providers, end-users, and subject matter experts. Overall, this review establishes an improved understanding of the value and implementation of digital biomarker (DB) technologies in preclinical research.

Perspectives in machine learning for wildlife conservation

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into relevant information. We argue that animal ecologists can capitalize on large datasets generated by modern sensors by combining machine learning approaches with domain knowledge. Incorporating machine learning into ecological workflows could improve inputs for ecological models and lead to integrated hybrid modeling tools. This approach will require close interdisciplinary collaboration to ensure the quality of novel approaches and train a new generation of data scientists in ecology and conservation.

Beyond the Edge: Markerless Pose Estimation of Speech Articulators from Ultrasound and Camera Images Using DeepLabCut

Automatic feature extraction from images of speech articulators is currently achieved by detecting edges. Here, we investigate the use of pose estimation deep neural nets with transfer learning to perform markerless estimation of speech articulator keypoints using only a few hundred hand-labelled images as training input. Midsagittal ultrasound images of the tongue, jaw, and hyoid and camera images of the lips were hand-labelled with keypoints, trained using DeepLabCut and evaluated on unseen speakers and systems. Tongue surface contours interpolated from estimated and hand-labelled keypoints produced an average mean sum of distances (MSD) of 0.93, s.d. 0.46 mm, compared with 0.96, s.d. 0.39 mm, for two human labellers, and 2.3, s.d. 1.5 mm, for the best performing edge detection algorithm. A pilot set of simultaneous electromagnetic articulography (EMA) and ultrasound recordings demonstrated partial correlation among three physical sensor positions and the corresponding estimated keypoints and requires further investigation. The accuracy of the estimating lip aperture from a camera video was high, with a mean MSD of 0.70, s.d. 0.56 mm compared with 0.57, s.d. 0.48 mm for two human labellers. DeepLabCut was found to be a fast, accurate and fully automatic method of providing unique kinematic data for tongue, hyoid, jaw, and lips.

The role of food odor in invertebrate foraging

Foraging for food is an integral part of animal survival. In small insects and invertebrates, multisensory information and optimized locomotion strategies are used to effectively forage in patchy and complex environments. Here, the importance of olfactory cues for effective invertebrate foraging is discussed in detail. We review how odors are used by foragers to move toward a likely food source and the recent models that describe this sensory-driven behavior. We argue that smell serves a second function by priming an organism for the efficient exploitation of food. By appraising food odors, invertebrates can establish preferences and better adapt to their ecological niches, thereby promoting survival. The smell of food pre-prepares the gastrointestinal system and primes feeding motor programs for more effective ingestion as well. Optimizing resource utilization affects longevity and reproduction as a result, leading to drastic changes in survival. We propose that models of foraging behavior should include odor priming, and illustrate this with a simple toy model based on the marginal value theorem. Lastly, we discuss the novel techniques and assays in invertebrate research that could investigate the interactions between odor sensing and food intake. Overall, the sense of smell is indispensable for efficient foraging and influences not only locomotion, but also organismal physiology, which should be reflected in behavioral modeling.

A SWOT Analysis of Portable and Low-Cost Markerless Motion Capture Systems to Assess Lower-Limb Musculoskeletal Kinematics in Sport

Markerless motion capture systems are promising for the assessment of movement in more real world research and clinical settings. While the technology has come a long way in the last 20 years, it is important for researchers and clinicians to understand the capacities and considerations for implementing these types of systems. The current review provides a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis related to the successful adoption of markerless motion capture technology for the assessment of lower-limb musculoskeletal kinematics in sport medicine and performance settings. 31 articles met the a priori inclusion criteria of this analysis. Findings from the analysis indicate that the improving accuracy of these systems via the refinement of machine learning algorithms, combined with their cost efficacy and the enhanced ecological validity outweighs the current weaknesses and threats. Further, the analysis makes clear that there is a need for multidisciplinary collaboration between sport scientists and computer vision scientists to develop accurate clinical and research applications that are specific to sport. While work remains to be done for broad application, markerless motion capture technology is currently on a positive trajectory and the data from this analysis provide an efficient roadmap toward widespread adoption.

Artificial intelligence-driven phenotyping of zebrafish psychoactive drug responses

Zebrafish (Danio rerio) are rapidly emerging in biomedicine as promising tools for disease modelling and drug discovery. The use of zebrafish for neuroscience research is also growing rapidly, necessitating novel reliable and unbiased methods of neurophenotypic data collection and analyses. Here, we applied the artificial intelligence (AI) neural network-based algorithms to a large dataset of adult zebrafish locomotor tracks collected previously in a series of in vivo experiments with multiple established psychotropic drugs. We first trained AI to recognize various drugs from a wide range of psychotropic agents tested, and then confirmed prediction accuracy of trained AI by comparing several agents with known similar behavioral and pharmacological profiles. Presenting a framework for innovative neurophenotyping, this proof-of-concept study aims to improve AI-driven movement pattern classification in zebrafish, thereby fostering drug discovery and development utilizing this key model organism.

A Novel, Automated, and Real-Time Method for the Analysis of Non-Human Primate Behavioral Patterns Using a Depth Image Sensor

By virtue of their upright locomotion, similar to that of humans, motion analysis of non-human primates has been widely used in order to better understand musculoskeletal biomechanics and neuroscience problems. Given the difficulty of conducting a marker-based infrared optical tracking system for the behavior analysis of primates, a 2-dimensional (D) video analysis has been applied. Distinct from a conventional marker-based optical tracking system, a depth image sensor system provides 3-D information on movement without any skin markers. The specific aim of this study was to develop a novel algorithm to analyze the behavioral patterns of non-human primates in a home cage using a depth image sensor. The behavioral patterns of nine monkeys in their home cage, including sitting, standing, and pacing, were captured using a depth image sensor. Thereafter, these were analyzed by observers’ manual assessment and the newly written automated program. We confirmed that the measurement results from the observers’ manual assessments and the automated program with depth image analysis were statistically identical.

Large-scale neural recordings call for new insights to link brain and behavior

Neuroscientists today can measure activity from more neurons than ever before, and are facing the challenge of connecting these brain-wide neural recordings to computation and behavior. In the present review, we first describe emerging tools and technologies being used to probe large-scale brain activity and new approaches to characterize behavior in the context of such measurements. We next highlight insights obtained from large-scale neural recordings in diverse model systems, and argue that some of these pose a challenge to traditional theoretical frameworks. Finally, we elaborate on existing modeling frameworks to interpret these data, and argue that the interpretation of brain-wide neural recordings calls for new theoretical approaches that may depend on the desired level of understanding. These advances in both neural recordings and theory development will pave the way for critical advances in our understanding of the brain.

Dissecting the molecular basis of human interneuron migration in forebrain assembloids from Timothy syndrome

Defects in interneuron migration can disrupt the assembly of cortical circuits and lead to neuropsychiatric disease. Using forebrain assembloids derived by integration of cortical and ventral forebrain organoids, we have previously discovered a cortical interneuron migration defect in Timothy syndrome (TS), a severe neurodevelopmental disease caused by a mutation in the L-type calcium channel (LTCC) Ca_v1.2. Here, we find that acute pharmacological modulation of Ca_v1.2 can regulate the saltation length, but not the frequency, of interneuron migration in TS. Interestingly, the defect in saltation length is related to aberrant actomyosin and myosin light chain (MLC) phosphorylation, while the defect in saltation frequency is driven by enhanced γ-aminobutyric acid (GABA) sensitivity and can be restored by GABA-A receptor antagonism. Finally, we describe hypersynchronous hCS network activity in TS that is exacerbated by interneuron migration. Taken together, these studies reveal a complex role of LTCC function in human cortical interneuron migration and strategies to restore deficits in the context of disease.

The Stop Signal Task for Measuring Behavioral Inhibition in Mice With Increased Sensitivity and High-Throughput Operation

Ceasing an ongoing motor response requires action cancelation. This is impaired in many pathologies such as attention deficit disorder and schizophrenia. Action cancelation is measured by the stop signal task that estimates how quickly a motor response can be stopped when it is already being executed. Apart from human studies, the stop signal task has been used to investigate neurobiological mechanisms of action cancelation overwhelmingly in rats and only rarely in mice, despite the need for a genetic model approach. Contributing factors to the limited number of mice studies may be the long and laborious training that is necessary and the requirement for a very loud (100 dB) stop signal. We overcame these limitations by employing a fully automated home-cage-based setup. We connected a home-cage to the operant box via a gating mechanism, that allowed individual ID chipped mice to start sessions voluntarily. Furthermore, we added a negative reinforcement consisting of a mild air puff with escape option to the protocol. This specifically improved baseline inhibition to 94% (from 84% with the conventional approach). To measure baseline inhibition the stop is signaled immediately with trial onset thus measuring action restraint rather than action cancelation ability. A high baseline allowed us to measure action cancelation ability with higher sensitivity. Furthermore, our setup allowed us to reduce the intensity of the acoustic stop signal from 100 to 70 dB. We constructed inhibition curves from stop trials with daily adjusted delays to estimate stop signal reaction times (SSRTs). SSRTs (median 88 ms) were lower than reported previously, which we attribute to the observed high baseline inhibition. Our automated training protocol reduced training time by 17% while also promoting minimal experimenter involvement. This sensitive and labor efficient stop signal task procedure should therefore facilitate the investigation of action cancelation pathologies in genetic mouse models.

Functional ultrasound imaging: A useful tool for functional connectomics?

Functional ultrasound (fUS) is a hemodynamic-based functional neuroimaging technique, primarily used in animal models, that combines a high spatiotemporal resolution, a large field of view, and compatibility with behavior. These assets make fUS especially suited to interrogating brain activity at the systems level. In this review, we describe the technical capabilities offered by fUS and discuss how this technique can contribute to the field of functional connectomics. First, fUS can be used to study intrinsic functional connectivity, namely patterns of correlated activity between brain regions. In this area, fUS has made the most impact by following connectivity changes in disease models, across behavioral states, or dynamically. Second, fUS can also be used to map brain-wide pathways associated with an external event. For example, fUS has helped obtain finer descriptions of several sensory systems, and uncover new pathways implicated in specific behaviors. Additionally, combining fUS with direct circuit manipulations such as optogenetics is an attractive way to map the brain-wide connections of defined neuronal populations. Finally, technological improvements and the application of new analytical tools promise to boost fUS capabilities. As brain coverage and the range of behavioral contexts that can be addressed with fUS keep on increasing, we believe that fUS-guided connectomics will only expand in the future. In this regard, we consider the incorporation of fUS into multimodal studies combining diverse techniques and behavioral tasks to be the most promising research avenue.

The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice

The study of naturalistic social behavior requires quantification of animals' interactions. This is generally done through manual annotation-a highly time-consuming and tedious process. Recent advances in computer vision enable tracking the pose (posture) of freely behaving animals. However, automatically and accurately classifying complex social behaviors remains technically challenging. We introduce the Mouse Action Recognition System (MARS), an automated pipeline for pose estimation and behavior quantification in pairs of freely interacting mice. We compare MARS's annotations to human annotations and find that MARS's pose estimation and behavior classification achieve human-level performance. We also release the pose and annotation datasets used to train MARS to serve as community benchmarks and resources. Finally, we introduce the Behavior Ensemble and Neural Trajectory Observatory (BENTO), a graphical user interface for analysis of multimodal neuroscience datasets. Together, MARS and BENTO provide an end-to-end pipeline for behavior data extraction and analysis in a package that is user-friendly and easily modifiable.

A New Look at Infant Problem-Solving: Using DeepLabCut to Investigate Exploratory Problem-Solving Approaches

When confronted with novel problems, problem-solvers must decide whether to copy a modeled solution or to explore their own unique solutions. While past work has established that infants can learn to solve problems both through their own exploration and through imitation, little work has explored the factors that influence which of these approaches infants select to solve a given problem. Moreover, past work has treated imitation and exploration as qualitatively distinct, although these two possibilities may exist along a continuum. Here, we apply a program novel to developmental psychology (DeepLabCut) to archival data (Lucca et al., 2020) to investigate the influence of the effort and success of an adult's modeled solution, and infants' firsthand experience with failure, on infants' imitative versus exploratory problem-solving approaches. Our results reveal that tendencies toward exploration are relatively immune to the information from the adult model, but that exploration generally increased in response to firsthand experience with failure. In addition, we found that increases in maximum force and decreases in trying time were associated with greater exploration, and that exploration subsequently predicted problem-solving success on a new iteration of the task. Thus, our results demonstrate that infants increase exploration in response to failure and that exploration may operate in a larger motivational framework with force, trying time, and expectations of task success.

Functional coupling between target selection and acquisition in the superior colliculus

Survival in unpredictable environments requires that animals continuously evaluate their surroundings for behavioral targets, direct their movements toward those targets, and terminate movements once a target is reached. The ability to select, move toward, and acquire spatial targets depends on a network of brain regions, but it remains unknown how these goal-directed processes are linked by neural circuits. Within this network, common circuits in the midbrain superior colliculus (SC) mediate the selection and initiation of movements to spatial targets. However, SC activity often persists throughout movement, suggesting that the same SC circuits underlying target selection and movement initiation may also contribute to "target acquisition": stopping the movement at the selected target. Here, we examine the hypothesis that SC functional circuitry couples target selection and acquisition using a "default motor plan" generated by selection-related neuronal activity. Recordings from intermediate and deep layer SC neurons in mice performing a spatial choice task demonstrate that choice-predictive neurons, including optogenetically identified GABAergic neurons whose activity mediates target selection, exhibit increased activity during movement to the target. By recording from rostral and caudal SC in separate groups of mice, we also revealed higher activity in rostral than caudal neurons during target acquisition. Finally, we used an attractor model to examine how-invoking only SC circuitry-caudal SC activity related to selecting an eccentric target could generate higher rostral than caudal acquisition-related activity. Overall, our results suggest a functional coupling between SC circuits for target selection and acquisition, elucidating a key mechanism for goal-directed behavior.NEW & NOTEWORTHY How do neural circuits ensure that selected targets are successfully acquired? Here, we examine whether choice-related activity in the superior colliculus (SC) promotes a motor plan for target acquisition. By demonstrating that choice-predictive SC neurons-including GABAergic neurons-remain active throughout movement, while the activity of rostral SC neurons increases during acquisition, and by recapitulating these dynamics with an attractor model, our results support a role for SC circuits in coupling target selection and acquisition.

Modelling behaviors relevant to brain disorders in the nonhuman primate: Are we there yet?

Recent years have seen a profound resurgence of activity with nonhuman primates (NHPs) to model human brain disorders. From marmosets to macaques, the study of NHP species offers a unique window into the function of primate-specific neural circuits that are impossible to examine in other models. Examining how these circuits manifest into the complex behaviors of primates, such as advanced cognitive and social functions, has provided enormous insights to date into the mechanisms underlying symptoms of numerous neurological and neuropsychiatric illnesses. With the recent optimization of modern techniques to manipulate and measure neural activity in vivo, such as optogenetics and calcium imaging, NHP research is more well-equipped than ever to probe the neural mechanisms underlying pathological behavior. However, methods for behavioral experimentation and analysis in NHPs have noticeably failed to keep pace with these advances. As behavior ultimately lies at the junction between preclinical findings and its translation to clinical outcomes for brain disorders, approaches to improve the integrity, reproducibility, and translatability of behavioral experiments in NHPs requires critical evaluation. In this review, we provide a unifying account of existing brain disorder models using NHPs, and provide insights into the present and emerging contributions of behavioral studies to the field.

Pain assessment in horses using automatic facial expression recognition through deep learning-based modeling

The aim of this study was to develop and evaluate a machine vision algorithm to assess the pain level in horses, using an automatic computational classifier based on the Horse Grimace Scale (HGS) and trained by machine learning method. The use of the Horse Grimace Scale is dependent on a human observer, who most of the time does not have availability to evaluate the animal for long periods and must also be well trained in order to apply the evaluation system correctly. In addition, even with adequate training, the presence of an unknown person near an animal in pain can result in behavioral changes, making the evaluation more complex. As a possible solution, the automatic video-imaging system will be able to monitor pain responses in horses more accurately and in real-time, and thus allow an earlier diagnosis and more efficient treatment for the affected animals. This study is based on assessment of facial expressions of 7 horses that underwent castration, collected through a video system positioned on the top of the feeder station, capturing images at 4 distinct timepoints daily for two days before and four days after surgical castration. A labeling process was applied to build a pain facial image database and machine learning methods were used to train the computational pain classifier. The machine vision algorithm was developed through the training of a Convolutional Neural Network (CNN) that resulted in an overall accuracy of 75.8% while classifying pain on three levels: not present, moderately present, and obviously present. While classifying between two categories (pain not present and pain present) the overall accuracy reached 88.3%. Although there are some improvements to be made in order to use the system in a daily routine, the model appears promising and capable of measuring pain on images of horses automatically through facial expressions, collected from video images.

Beyond Single Discrete Responses: An Integrative and Multidimensional Analysis of Behavioral Dynamics Assisted by Machine Learning

Understanding behavioral systems as emergent systems comprising the environment and organism subsystems, include spatial dynamics as a primary dimension in natural settings. Nevertheless, under the standard approaches, the experimental analysis of behavior is based on the single response paradigm and the temporal distribution of discrete responses. Thus, the continuous analysis of spatial behavioral dynamics is a scarcely studied field. The technological advancements in computer vision have opened new methodological perspectives for the continuous sensing of spatial behavior. With the application of such advancements, recent studies suggest that there are multiple features embedded in the spatial dynamics of behavior, such as entropy, and that they are affected by programmed stimuli (e.g., schedules of reinforcement) at least as much as features related to discrete responses. Despite the progress, the characterization of behavioral systems is still segmented, and integrated data analysis and representations between discrete responses and continuous spatial behavior are exiguous in the experimental analysis of behavior. Machine learning advancements, such as t-distributed stochastic neighbor embedding and variable ranking, provide invaluable tools to crystallize an integrated approach for analyzing and representing multidimensional behavioral data. Under this rationale, the present work (1) proposes a multidisciplinary approach for the integrative and multilevel analysis of behavioral systems, (2) provides sensitive behavioral measures based on spatial dynamics and helpful data representations to study behavioral systems, and (3) reveals behavioral aspects usually ignored under the standard approaches in the experimental analysis of behavior. To exemplify and evaluate our approach, the spatial dynamics embedded in phenomena relevant to behavioral science, namely, water-seeking behavior and motivational operations, are examined, showing aspects of behavioral systems hidden until now.

Anipose: A toolkit for robust markerless 3D pose estimation

Quantifying movement is critical for understanding animal behavior. Advances in computer vision now enable markerless tracking from 2D video, but most animals move in 3D. Here, we introduce Anipose, an open-source toolkit for robust markerless 3D pose estimation. Anipose is built on the 2D tracking method DeepLabCut, so users can expand their existing experimental setups to obtain accurate 3D tracking. It consists of four components: (1) a 3D calibration module, (2) filters to resolve 2D tracking errors, (3) a triangulation module that integrates temporal and spatial regularization, and (4) a pipeline to structure processing of large numbers of videos. We evaluate Anipose on a calibration board as well as mice, flies, and humans. By analyzing 3D leg kinematics tracked with Anipose, we identify a key role for joint rotation in motor control of fly walking. To help users get started with 3D tracking, we provide tutorials and documentation at http://anipose.org/.

From human wellbeing to animal welfare

What does it mean to be "well" and how might such a state be cultivated? When we speak of wellbeing, it is of ourselves and fellow humans. When it comes to nonhuman animals, consideration turns to welfare. My aim herein is to suggest that theoretical approaches to human wellbeing might be beneficially applied to consideration of animal welfare, and in so doing, introduce new lines of inquiry and practice. I will review current approaches to human wellbeing, adopting a triarchic structure that delineates hedonic wellbeing, eudaimonic wellbeing, and social wellbeing. For each, I present a conceptual definition and a review of how researchers have endeavored to measure the construct. Drawing these three domains of research together, I highlight how these traditionally anthropocentric lines of inquiry might be extended to the question of animal welfare - namely by considering hedonic welfare, eudaimonic welfare, and social welfare as potentially distinguishable and complementary components of the broader construct of animal welfare.

A Validation of Supervised Deep Learning for Gait Analysis in the Cat

Gait analysis in cats and other animals is generally performed with custom-made or commercially developed software to track reflective markers placed on bony landmarks. This often involves costly motion tracking systems. However, deep learning, and in particular DeepLabCut^TM (DLC), allows motion tracking without requiring placing reflective markers or an expensive system. The purpose of this study was to validate the accuracy of DLC for gait analysis in the adult cat by comparing results obtained with DLC and a custom-made software (Expresso) that has been used in several cat studies. Four intact adult cats performed tied-belt (both belts at same speed) and split-belt (belts operating at different speeds) locomotion at different speeds and left-right speed differences on a split-belt treadmill. We calculated several kinematic variables, such as step/stride lengths and joint angles from the estimates made by the two software and assessed the agreement between the two measurements using intraclass correlation coefficient or Lin's concordance correlation coefficient as well as Pearson's correlation coefficients. The results showed that DLC is at least as precise as Expresso with good to excellent agreement for all variables. Indeed, all 12 variables showed an agreement above 0.75, considered good, while nine showed an agreement above 0.9, considered excellent. Therefore, deep learning, specifically DLC, is valid for measuring kinematic variables during locomotion in cats, without requiring reflective markers and using a relatively low-cost system.

Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain

Neural computation has evolved to optimize the behaviors that enable our survival. Although much previous work in neuroscience has focused on constrained task behaviors, recent advances in computer vision are fueling a trend toward the study of naturalistic behaviors. Automated tracking of fine-scale behaviors is generating rich datasets for animal models including rodents, fruit flies, zebrafish, and worms. However, extracting meaning from these large and complex data often requires sophisticated computational techniques. Here we review the latest methods and modeling approaches providing new insights into the brain from behavior. We focus on unsupervised methods for identifying stereotyped behaviors and for resolving details of the structure and dynamics of behavioral sequences.

A framework for studying behavioral evolution by reconstructing ancestral repertoires

Although different animal species often exhibit extensive variation in many behaviors, typically scientists examine one or a small number of behaviors in any single study. Here, we propose a new framework to simultaneously study the evolution of many behaviors. We measured the behavioral repertoire of individuals from six species of fruit flies using unsupervised techniques and identified all stereotyped movements exhibited by each species. We then fit a Generalized Linear Mixed Model to estimate the intra- and inter-species behavioral covariances, and, by using the known phylogenetic relationships among species, we estimated the (unobserved) behaviors exhibited by ancestral species. We found that much of intra-specific behavioral variation has a similar covariance structure to previously described long-time scale variation in an individual's behavior, suggesting that much of the measured variation between individuals of a single species in our assay reflects differences in the status of neural networks, rather than genetic or developmental differences between individuals. We then propose a method to identify groups of behaviors that appear to have evolved in a correlated manner, illustrating how sets of behaviors, rather than individual behaviors, likely evolved. Our approach provides a new framework for identifying co-evolving behaviors and may provide new opportunities to study the mechanistic basis of behavioral evolution.

Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders

Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone.

Moving outside the lab: Markerless motion capture accurately quantifies sagittal plane kinematics during the vertical jump

Markerless motion capture using deep learning approaches have potential to revolutionize the field of biomechanics by allowing researchers to collect data outside of the laboratory environment, yet there remain questions regarding the accuracy and ease of use of these approaches. The purpose of this study was to apply a markerless motion capture approach to extract lower limb angles in the sagittal plane during the vertical jump and to evaluate agreement between the custom trained model and gold standard motion capture. We performed this study using a large open source data set (N = 84) that included synchronized commercial video and gold standard motion capture. We split these data into a training set for model development (n = 69) and test set to evaluate capture performance relative to gold standard motion capture using coefficient of multiple correlations (CMC) (n = 15). We found very strong agreement between the custom trained markerless approach and marker-based motion capture within the test set across the entire movement (CMC > 0.991, RMSE < 3.22°), with at least strong CMC values across all trials for the hip (0.853 ± 0.23), knee (0.963 ± 0.471), and ankle (0.970 ± 0.055). The strong agreement between markerless and marker-based motion capture provides evidence that markerless motion capture is a viable tool to extend data collection to outside of the laboratory. As biomechanical research struggles with representative sampling practices, markerless motion capture has potential to transform biomechanical research away from traditional laboratory settings into venues convenient to populations that are under sampled without sacrificing measurement fidelity.

An Algorithmic Approach to Natural Behavior

Uncovering the mechanisms and implications of natural behavior is a goal that unites many fields of biology. Yet, the diversity, flexibility, and multi-scale nature of these behaviors often make understanding elusive. Here, we review studies of animal pursuit and evasion - two special classes of behavior where theory-driven experiments and new modeling techniques are beginning to uncover the general control principles underlying natural behavior. A key finding of these studies is that intricate sequences of pursuit and evasion behavior can often be constructed through simple, repeatable rules that link sensory input to motor output: we refer to these rules as behavioral algorithms. Identifying and mathematically characterizing these algorithms has led to important insights, including the discovery of guidance rules that attacking predators use to intercept mobile prey, and coordinated neural and biomechanical mechanisms that animals use to avoid impending collisions. Here, we argue that algorithms provide a good starting point for studies of natural behavior more generally. Rather than beginning at the neural or ecological levels of organization, we advocate starting in the middle, where the algorithms that link sensory input to behavioral output can provide a solid foundation from which to explore both the implementation and the ecological outcomes of behavior. We review insights that have been gained through such an algorithmic approach to pursuit and evasion behaviors. From these, we synthesize theoretical principles and lay out key modeling tools needed to apply an algorithmic approach to the study of other complex natural behaviors.