Automated image-based tracking and its application in ecology

Metabolic plasticity drives mismatches in physiological traits between prey and predator

Metabolic rate, the rate of energy use, underpins key ecological traits of organisms, from development and locomotion to interaction rates between individuals. In a warming world, the temperature-dependence of metabolic rate is anticipated to shift predator-prey dynamics. Yet, there is little real-world evidence on the effects of warming on trophic interactions. We measured the respiration rates of aquatic larvae of three insect species from populations experiencing a natural temperature gradient in a large-scale mesocosm experiment. Using a mechanistic model we predicted the effects of warming on these taxa's predator-prey interaction rates. We found that species-specific differences in metabolic plasticity lead to mismatches in the temperature-dependence of their relative velocities, resulting in altered predator-prey interaction rates. This study underscores the role of metabolic plasticity at the species level in modifying trophic interactions and proposes a mechanistic modelling approach that allows an efficient, high-throughput estimation of climate change threats across species pairs.

Herbivorous fish feeding dynamics and energy expenditure on a coral reef: Insights from stereo-video and AI-driven 3D tracking

Unveiling the intricate relationships between animal movement ecology, feeding behavior, and internal energy budgeting is crucial for a comprehensive understanding of ecosystem functioning, especially on coral reefs under significant anthropogenic stress. Here, herbivorous fishes play a vital role as mediators between algae growth and coral recruitment. Our research examines the feeding preferences, bite rates, inter-bite distances, and foraging energy expenditure of the Brown surgeonfish (Acanthurus nigrofuscus) and the Yellowtail tang (Zebrasoma xanthurum) within the fish community on a Red Sea coral reef. To this end, we used advanced methods such as remote underwater stereo-video, AI-driven object recognition, species classification, and 3D tracking. Despite their comparatively low biomass, the two surgeonfish species significantly influence grazing pressure on the studied coral reef. A. nigrofuscus exhibits specialized feeding preferences and Z. xanthurum a more generalist approach, highlighting niche differentiation and their importance in maintaining reef ecosystem balance. Despite these differences in their foraging strategies, on a population level, both species achieve a similar level of energy efficiency. This study highlights the transformative potential of cutting-edge technologies in revealing the functional feeding traits and energy utilization of keystone species. It facilitates the detailed mapping of energy seascapes, guiding targeted conservation efforts to enhance ecosystem health and biodiversity.

Predicting thresholds for population replacement gene drives

BACKGROUND

Threshold-dependent gene drives (TDGDs) could be used to spread desirable traits through a population, and are likely to be less invasive and easier to control than threshold-independent gene drives. Engineered Genetic Incompatibility (EGI) is an extreme underdominance system previously demonstrated in Drosophila melanogaster that can function as a TDGD when EGI agents of both sexes are released into a wild-type population.

RESULTS

Here we use a single generation fitness assay to compare the fecundity, mating preferences, and temperature-dependent relative fitness to wild-type of two distinct genotypes of EGI agents. We find significant differences in the behavior/performance of these EGI agents that would not be predicted a priori based on their genetic design. We report a surprising temperature-dependent change in the predicted threshold for population replacement in an EGI agent that drives ectopic expression of the developmental morphogen pyramus.

CONCLUSIONS

The single-generation fitness assay presented here could reduce the amount of time required to estimate the threshold for TDGD strategies for which hybrid genotypes are inviable. Additionally, this work underscores the importance of empirical characterization of multiple engineered lines, as behavioral differences can arise in unique genotypes for unknown reasons.

Markerless Mouse Tracking for Social Experiments

Automated behavior quantification in socially interacting animals requires accurate tracking. While many methods have been very successful and highly generalizable to different settings, issues of mistaken identities and lost information on key anatomical features are common, although they can be alleviated by increased human effort in training or post-processing. We propose a markerless video-based tool to simultaneously track two interacting mice of the same appearance in controlled settings for quantifying behaviors such as different types of sniffing, touching, and locomotion to improve tracking accuracy under these settings without increased human effort. It incorporates conventional handcrafted tracking and deep-learning-based techniques. The tool is trained on a small number of manually annotated images from a basic experimental setup and outputs body masks and coordinates of the snout and tail-base for each mouse. The method was tested on several commonly used experimental conditions including bedding in the cage and fiberoptic or headstage implants on the mice. Results obtained without any human corrections after the automated analysis showed a near elimination of identities switches and a ∼15% improvement in tracking accuracy over pure deep-learning-based pose estimation tracking approaches. Our approach can be optionally ensembled with such techniques for further improvement. Finally, we demonstrated an application of this approach in studies of social behavior of mice by quantifying and comparing interactions between pairs of mice in which some lack olfaction. Together, these results suggest that our approach could be valuable for studying group behaviors in rodents, such as social interactions.

A latest progress in the study of fish behavior: cross-generational effects of behavior under pollution pressure and new technologies for behavior monitoring

With the development of agriculture and industry, an increasing number of pollutants are being discharged into the aquatic environment. These pollutants can harm aquatic life. The behavioral characteristics of animals are an external manifestation of their internal mechanisms. Changes in behavior reflect damage and changes in the internal mechanisms. Environmental pollution may lead to behavioral changes not only in the parental generation but also in the offspring that has not been exposed to the pollutants. That is, the intrinsic mechanism that leads to behavioral changes is inheritable. Fish are representative species of aquatic organisms and are commonly used in various research studies. The behavior of fish has also received extensive attention, and the monitoring technology for fish behavior has developed rapidly. This article summarizes the development process of behavior monitoring technology and introduces some of the latest technologies for studying fish behavior. This article also summarizes the intergenerational effects of pollutants on fish behavior, as well as the potential intrinsic and genetic mechanisms that may lead to behavioral changes. This article provides a reference for future relevant neurobehavioral studies.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for behavioral studies in various disciplines, including neuroscience. Recently, machine learning and computer vision techniques have become integral to the neuroscientist's toolkit, enabling high-throughput behavioral studies. Despite advancements in localizing individual animals in simple environments, the task remains challenging in complex conditions due to intra-class visual variability and environmental diversity. These limitations hinder studies in ethologically-relevant conditions, such as when animals are concealed within nests or in obscured environments. Moreover, current tools are laborious and time-consuming to employ, requiring extensive, setup-specific annotation and model training/validation procedures. To address these challenges, we introduce the 'Detect Any Mouse Model' (DAMM), a pretrained object detector for localizing mice in complex environments, capable of robust performance with zero to minimal additional training on new experimental setups. Our approach involves collecting and annotating a diverse dataset that encompasses single and multi-housed mice in various lighting conditions, experimental setups, and occlusion levels. We utilize the Mask R-CNN architecture for instance segmentation and validate DAMM's performance with no additional training data (zero-shot inference) and with few examples for fine-tuning (few-shot inference). DAMM excels in zero-shot inference, detecting mice, and even rats, in entirely unseen scenarios and further improves with minimal additional training. By integrating DAMM with the SORT algorithm, we demonstrate robust tracking, competitively performing with keypoint-estimation-based methods. Finally, to advance and simplify behavioral studies, we made DAMM accessible to the scientific community with a user-friendly Python API, shared model weights, and a Google Colab implementation.

Insect communities under skyglow: diffuse night-time illuminance induces spatio-temporal shifts in movement and predation

Artificial light at night (ALAN) is predicted to have far-reaching consequences for natural ecosystems given its influence on organismal physiology and behaviour, species interactions and community composition. Movement and predation are fundamental ecological processes that are of critical importance to ecosystem functioning. The natural movements and foraging behaviours of nocturnal invertebrates may be particularly sensitive to the presence of ALAN. However, we still lack evidence of how these processes respond to ALAN within a community context. We assembled insect communities to quantify their movement activity and predation rates during simulated Moon cycles across a gradient of diffuse night-time illuminance including the full range of observed skyglow intensities. Using radio frequency identification, we tracked the movements of insects within a fragmented grassland Ecotron experiment. We additionally quantified predation rates using prey dummies. Our results reveal that even low-intensity skyglow causes a temporal shift in movement activity from day to night, and a spatial shift towards open habitats at night. Changes in movement activity are associated with indirect shifts in predation rates. Spatio-temporal shifts in movement and predation have important implications for ecological networks and ecosystem functioning, highlighting the disruptive potential of ALAN for global biodiversity and the provision of ecosystem services. This article is part of the theme issue 'Light pollution in complex ecological systems'.

Characterizing Pairwise U-Turn Behavior in Fish: A Data-Driven Analysis

We applied the time-series clustering method to analyze the trajectory data of rummy-nose tetra (Hemigrammus rhodostomus), with a particular focus on their spontaneous paired turning behavior. Firstly, an automated U-turn maneuver identification method was proposed to extract turning behaviors from the open trajectory data of two fish swimming in an annular tank. We revealed two distinct ways of pairwise U-turn swimming, named dominated turn and non-dominated turn. Upon comparison, the dominated turn is smoother and more efficient, with a fixed leader-follower relationship, i.e., the leader dominates the turning process. Because these two distinct ways corresponded to different patterns of turning feature parameters over time, we incorporated the Toeplitz inverse covariance-based clustering (TICC) method to gain deeper insights into this process. Pairwise turning behavior was decomposed into some elemental state compositions. Specifically, we found that the main influencing factor for a spontaneous U-turn is collision avoidance with the wall. In dominated turn, when inter-individual distances were appropriate, fish adjusted their positions and movement directions to achieve turning. Conversely, in closely spaced non-dominated turn, various factors such as changes in distance, velocity, and movement direction resulted in more complex behaviors. The purpose of our study is to integrate common location-based analysis methods with time-series clustering methods to analyze biological behavioral data. The study provides valuable insights into the U-turn behavior, motion characteristics, and decision factors of rummy-nose tetra during pairwise swimming. Additionally, the study extends the analysis of fish interaction features through the application of time-series clustering methods, offering a fresh perspective for the analysis of biological collective data.

Toward Real-Time Animal Tracking with Integrated Stimulus Control for Automated Conditioning in Aquatic Eco-Neurotoxicology

Aquatic eco-neurotoxicology is an emerging field that requires new analytical systems to study the effects of pollutants on animal behaviors. This is especially true if we are to gain insights into one of the least studied aspects: the potential perturbations that neurotoxicants can have on cognitive behaviors. The paucity of experimental data is partly caused by a lack of low-cost technologies for the analysis of higher-level neurological functions (e.g., associative learning) in small aquatic organisms. Here, we present a proof-of-concept prototype that utilizes a new real-time animal tracking software for on-the-fly video analysis and closed-loop, external hardware communications to deliver stimuli based on specific behaviors in aquatic organisms, spanning three animal phyla: chordates (fish, frog), platyhelminthes (flatworm), and arthropods (crustacean). The system's open-source software features an intuitive graphical user interface and advanced adaptive threshold-based image segmentation for precise animal detection. We demonstrate the precision of animal tracking across multiple aquatic species with varying modes of locomotion. The presented technology interfaces easily with low-cost and open-source hardware such as the Arduino microcontroller family for closed-loop stimuli control. The new system has potential future applications in eco-neurotoxicology, where it could enable new opportunities for cognitive research in diverse small aquatic model organisms.

replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine

Deep learning-based computer vision methods are transforming animal behavioural research. Transfer learning has enabled work in non-model species, but still requires hand-annotation of example footage, and is only performant in well-defined conditions. To help overcome these limitations, we developed replicAnt, a configurable pipeline implemented in Unreal Engine 5 and Python, designed to generate large and variable training datasets on consumer-grade hardware. replicAnt places 3D animal models into complex, procedurally generated environments, from which automatically annotated images can be exported. We demonstrate that synthetic data generated with replicAnt can significantly reduce the hand-annotation required to achieve benchmark performance in common applications such as animal detection, tracking, pose-estimation, and semantic segmentation. We also show that it increases the subject-specificity and domain-invariance of the trained networks, thereby conferring robustness. In some applications, replicAnt may even remove the need for hand-annotation altogether. It thus represents a significant step towards porting deep learning-based computer vision tools to the field.

Information Ecology: an integrative framework for studying animal behavior

Information is simultaneously a valuable resource for animals and a tractable variable for researchers. We propose the name Information Ecology to describe research focused on how individual animals use information to enhance fitness. An explicit focus on information in animal behavior is far from novel - we simply build on these ideas and promote a unified approach to how and why animals use information. The value of information to animals favors the theoretically rich adaptive approach of field-based research. Simultaneously, our ability to manipulate information lends itself to the strong methods of laboratory-based research. Information Ecology asks three questions: What information is available? How is it used (or not)? And, why is it used (or not)?

Elephants and algorithms: a review of the current and future role of AI in elephant monitoring

Artificial intelligence (AI) and machine learning (ML) present revolutionary opportunities to enhance our understanding of animal behaviour and conservation strategies. Using elephants, a crucial species in Africa and Asia's protected areas, as our focal point, we delve into the role of AI and ML in their conservation. Given the increasing amounts of data gathered from a variety of sensors like cameras, microphones, geophones, drones and satellites, the challenge lies in managing and interpreting this vast data. New AI and ML techniques offer solutions to streamline this process, helping us extract vital information that might otherwise be overlooked. This paper focuses on the different AI-driven monitoring methods and their potential for improving elephant conservation. Collaborative efforts between AI experts and ecological researchers are essential in leveraging these innovative technologies for enhanced wildlife conservation, setting a precedent for numerous other species.

A lightweight cow mounting behavior recognition system based on improved YOLOv5s

To improve the detection speed of cow mounting behavior and the lightness of the model in dense scenes, this study proposes a lightweight rapid detection system for cow mounting behavior. Using the concept of EfficientNetV2, a lightweight backbone network is designed using an attention mechanism, inverted residual structure, and depth-wise separable convolution. Next, a feature enhancement module is designed using residual structure, efficient attention mechanism, and Ghost convolution. Finally, YOLOv5s, the lightweight backbone network, and the feature enhancement module are combined to construct a lightweight rapid recognition model for cow mounting behavior. Multiple cameras were installed in a barn with 200 cows to obtain 3343 images that formed the cow mounting behavior dataset. Based on the experimental results, the inference speed of the model put forward in this study is as high as 333.3 fps, the inference time per image is 4.1 ms, and the model mAP value is 87.7%. The mAP value of the proposed model is shown to be 2.1% higher than that of YOLOv5s, the inference speed is 0.47 times greater than that of YOLOv5s, and the model weight is 2.34 times less than that of YOLOv5s. According to the obtained results, the model proposed in the current work shows high accuracy and inference speed and acquires the automatic detection of cow mounting behavior in dense scenes, which would be beneficial for the all-weather real-time monitoring of multi-channel cameras in large cattle farms.

Mechanisms of group-hunting in vertebrates

Group-hunting is ubiquitous across animal taxa and has received considerable attention in the context of its functions. By contrast much less is known about the mechanisms by which grouping predators hunt their prey. This is primarily due to a lack of experimental manipulation alongside logistical difficulties quantifying the behaviour of multiple predators at high spatiotemporal resolution as they search, select, and capture wild prey. However, the use of new remote-sensing technologies and a broadening of the focal taxa beyond apex predators provides researchers with a great opportunity to discern accurately how multiple predators hunt together and not just whether doing so provides hunters with a per capita benefit. We incorporate many ideas from collective behaviour and locomotion throughout this review to make testable predictions for future researchers and pay particular attention to the role that computer simulation can play in a feedback loop with empirical data collection. Our review of the literature showed that the breadth of predator:prey size ratios among the taxa that can be considered to hunt as a group is very large (<100 to >102 ). We therefore synthesised the literature with respect to these predator:prey ratios and found that they promoted different hunting mechanisms. Additionally, these different hunting mechanisms are also related to particular stages of the hunt (search, selection, capture) and thus we structure our review in accordance with these two factors (stage of the hunt and predator:prey size ratio). We identify several novel group-hunting mechanisms which are largely untested, particularly under field conditions, and we also highlight a range of potential study organisms that are amenable to experimental testing of these mechanisms in connection with tracking technology. We believe that a combination of new hypotheses, study systems and methodological approaches should help push the field of group-hunting in new directions.

Postural analysis reveals persistent changes in paper wasp foundress behavioral state after conspecific challenge

Vigilant animals detect and respond to threats in the environment, often changing posture and movement patterns. Vigilance is modulated not only by predators but also by conspecific threats. In social animals, precisely how conspecific threats alter vigilance behavior over time is relevant to long-standing hypotheses about social plasticity. We report persistent effects of a simulated conspecific challenge on behavior of wild northern paper wasp foundresses, Polistes fuscatus. During the founding phase of the colony cycle, conspecific wasps can usurp nests from the resident foundress, representing a severe threat. We used automated tracking to monitor the movement and posture of P. fuscatus foundresses in response to simulated intrusions. Wasps displayed increased movement, greater bilateral wing extension, and reduced antennal separation after the threat was removed. These changes were not observed after presentation with a wooden dowel. By rapidly adjusting individual behavior after fending off an intruder, paper wasp foundresses might invest in surveillance of potential threats, even when such threats are no longer immediately present. The prolonged vigilance-like behavioral state observed here is relevant to plasticity of social recognition processes in paper wasps.

Microhabitat conditions remedy heat stress effects on insect activity

Anthropogenic global warming has major implications for mobile terrestrial insects, including long-term effects from constant warming, for example, on species distribution patterns, and short-term effects from heat extremes that induce immediate physiological responses. To cope with heat extremes, they either have to reduce their activity or move to preferable microhabitats. The availability of favorable microhabitat conditions is strongly promoted by the spatial heterogeneity of habitats, which is often reduced by anthropogenic land transformation. Thus, it is decisive to understand the combined effects of these global change drivers on insect activity. Here, we assessed the movement activity of six insect species (from three orders) in response to heat stress using a unique tracking approach via radio frequency identification. We tracked 465 individuals at the iDiv Ecotron across a temperature gradient up to 38.7°C. In addition, we varied microhabitat conditions by adding leaf litter from four different tree species to the experimental units, either spatially separated or well mixed. Our results show opposing effects of heat extremes on insect activity depending on the microhabitat conditions. The insect community significantly decreased its activity in the mixed litter scenario, while we found a strong positive effect on activity in the separated litter scenario. We hypothesize that the simultaneous availability of thermal refugia as well as resources provided by the mixed litter scenario allows animals to reduce their activity and save energy in response to heat stress. Contrary, the spatial separation of beneficial microclimatic conditions and resources forces animals to increase their activity to fulfill their energetic needs. Thus, our study highlights the importance of habitat heterogeneity on smaller scales, because it may buffer the consequences of extreme temperatures of insect performance and survival under global change.

Multi-Object Tracking in Heterogeneous environments (MOTHe) for animal video recordings

Aerial imagery and video recordings of animals are used for many areas of research such as animal behaviour, behavioural neuroscience and field biology. Many automated methods are being developed to extract data from such high-resolution videos. Most of the available tools are developed for videos taken under idealised laboratory conditions. Therefore, the task of animal detection and tracking for videos taken in natural settings remains challenging due to heterogeneous environments. Methods that are useful for field conditions are often difficult to implement and thus remain inaccessible to empirical researchers. To address this gap, we present an open-source package called Multi-Object Tracking in Heterogeneous environments (MOTHe), a Python-based application that uses a basic convolutional neural network for object detection. MOTHe offers a graphical interface to automate the various steps related to animal tracking such as training data generation, animal detection in complex backgrounds and visually tracking animals in the videos. Users can also generate training data and train a new model which can be used for object detection tasks for a completely new dataset. MOTHe doesn't require any sophisticated infrastructure and can be run on basic desktop computing units. We demonstrate MOTHe on six video clips in varying background conditions. These videos are from two species in their natural habitat-wasp colonies on their nests (up to 12 individuals per colony) and antelope herds in four different habitats (up to 156 individuals in a herd). Using MOTHe, we are able to detect and track individuals in all these videos. MOTHe is available as an open-source GitHub repository with a detailed user guide and demonstrations at: https://github.com/tee-lab/MOTHe-GUI.

Dynamic pathogen detection and social feedback shape collective hygiene in ants

Cooperative disease defense emerges as group-level collective behavior, yet how group members make the underlying individual decisions is poorly understood. Using garden ants and fungal pathogens as an experimental model, we derive the rules governing individual ant grooming choices and show how they produce colony-level hygiene. Time-resolved behavioral analysis, pathogen quantification, and probabilistic modeling reveal that ants increase grooming and preferentially target highly-infectious individuals when perceiving high pathogen load, but transiently suppress grooming after having been groomed by nestmates. Ants thus react to both, the infectivity of others and the social feedback they receive on their own contagiousness. While inferred solely from momentary ant decisions, these behavioral rules quantitatively predict hour-long experimental dynamics, and synergistically combine into efficient colony-wide pathogen removal. Our analyses show that noisy individual decisions based on only local, incomplete, yet dynamically-updated information on pathogen threat and social feedback can lead to potent collective disease defense.

Supervised machine learning aided behavior classification in pigeons

Manual behavioral observations have been applied in both environment and laboratory experiments in order to analyze and quantify animal movement and behavior. Although these observations contributed tremendously to ecological and neuroscientific disciplines, there have been challenges and disadvantages following in their footsteps. They are not only time-consuming, labor-intensive, and error-prone but they can also be subjective, which induces further difficulties in reproducing the results. Therefore, there is an ongoing endeavor towards automated behavioral analysis, which has also paved the way for open-source software approaches. Even though these approaches theoretically can be applied to different animal groups, the current applications are mostly focused on mammals, especially rodents. However, extending those applications to other vertebrates, such as birds, is advisable not only for extending species-specific knowledge but also for contributing to the larger evolutionary picture and the role of behavior within. Here we present an open-source software package as a possible initiation of bird behavior classification. It can analyze pose-estimation data generated by established deep-learning-based pose-estimation tools such as DeepLabCut for building supervised machine learning predictive classifiers for pigeon behaviors, which can be broadened to support other bird species as well. We show that by training different machine learning and deep learning architectures using multivariate time series data as input, an F1 score of 0.874 can be achieved for a set of seven distinct behaviors. In addition, an algorithm for further tuning the bias of the predictions towards either precision or recall is introduced, which allows tailoring the classifier to specific needs.

Shortwave infrared fluorescence imaging of peripheral organs in awake and freely moving mice

Extracting biological information from awake and unrestrained mice is imperative to in vivo basic and pre-clinical research. Accordingly, imaging methods which preclude invasiveness, anesthesia, and/or physical restraint enable more physiologically relevant biological data extraction by eliminating these extrinsic confounders. In this article, we discuss the recent development of shortwave infrared (SWIR) fluorescent imaging to visualize peripheral organs in freely-behaving mice, as well as propose potential applications of this imaging modality in the neurosciences.

Predicting movement speed of beetles from body size and temperature

Movement facilitates and alters species interactions, the resulting food web structures, species distribution patterns, community structures and survival of populations and communities. In the light of global change, it is crucial to gain a general understanding of how movement depends on traits and environmental conditions. Although insects and notably Coleoptera represent the largest and a functionally important taxonomic group, we still know little about their general movement capacities and how they respond to warming. Here, we measured the exploratory speed of 125 individuals of eight carabid beetle species across different temperatures and body masses using automated image-based tracking. The resulting data revealed a power-law scaling relationship of average movement speed with body mass. By additionally fitting a thermal performance curve to the data, we accounted for the unimodal temperature response of movement speed. Thereby, we yielded a general allometric and thermodynamic equation to predict exploratory speed from temperature and body mass. This equation predicting temperature-dependent movement speed can be incorporated into modeling approaches to predict trophic interactions or spatial movement patterns. Overall, these findings will help improve our understanding of how temperature effects on movement cascade from small to large spatial scales as well as from individual to population fitness and survival across communities.

Effects of environmental concentrations of caffeine on adult zebrafish behaviour: a short-term exposure scenario

Caffeine (CAF) has been considered an emerging environmental contaminant and its presence indicator of anthropogenic contamination. This study evaluated the effects of environmental concentrations of CAF (0, 0.5, 1.5, and 300 μg. L-1) on the behaviour of adult zebrafish (Danio rerio) after 7 days of exposure. The components of feeding, locomotion, boldness (new tank test), sociability (schooling test), and aggression (mirror test) were analysed. Growth rate and weight were investigated as complementary measures. CAF (0.5, 1.5, and 300 μg. L-1) reduced exploratory behaviour in zebrafish, increased feeding latency time (1.5, and 300 μg. L-1), and decreased growth rate and fish weight (300 μg. L-1). CAF also induced aggressive behaviour (0.5, 1.5, and 300 μg. L-1) and decreased appetence to the shoal (sociability) (0.5, and 1.5 μg. L-1). This study showed that low doses of CAF can induce behavioural effects in zebrafish that may have significant long-term impacts on vital ecological functions.

Shortwave infrared (SWIR) fluorescence imaging of peripheral organs in awake and freely moving mice

Extracting biological information from awake and unrestrained mice is imperative to in vivo basic and pre-clinical research. Accordingly, imaging methods which preclude invasiveness, anesthesia, and/or physical restraint enable more physiologically relevant biological data extraction by eliminating these extrinsic confounders. In this article we discuss the recent development of shortwave infrared (SWIR) fluorescent imaging to visualize peripheral organs in freely-behaving mice, as well as propose potential applications of this imaging modality in the neurosciences.

A study of transfer of information in animal collectives using deep learning tools

We studied how the interactions among animals in a collective allow for the transfer of information. We performed laboratory experiments to study how zebrafish in a collective follow a subset of trained animals that move towards a light when it turns on because they expect food at that location. We built some deep learning tools to distinguish from video which are the trained and the naïve animals and to detect when each animal reacts to the light turning on. These tools gave us the data to build a model of interactions that we designed to have a balance between transparency and accuracy. The model finds a low-dimensional function that describes how a naïve animal weights neighbours depending on focal and neighbour variables. According to this low-dimensional function, neighbour speed plays an important role in the interactions. Specifically, a naïve animal weights more a neighbour in front than to the sides or behind, and more so the faster the neighbour is moving; and if the neighbour moves fast enough, the differences coming from the neighbour's relative position largely disappear. From the lens of decision-making, neighbour speed acts as confidence measure about where to go. This article is part of a discussion meeting issue 'Collective behaviour through time'.

Dynamics of collective motion across time and species

Most studies of collective animal behaviour rely on short-term observations, and comparisons of collective behaviour across different species and contexts are rare. We therefore have a limited understanding of intra- and interspecific variation in collective behaviour over time, which is crucial if we are to understand the ecological and evolutionary processes that shape collective behaviour. Here, we study the collective motion of four species: shoals of stickleback fish (Gasterosteus aculeatus), flocks of homing pigeons (Columba livia), a herd of goats (Capra aegagrus hircus) and a troop of chacma baboons (Papio ursinus). First, we describe how local patterns (inter-neighbour distances and positions), and group patterns (group shape, speed and polarization) during collective motion differ across each system. Based on these, we place data from each species within a 'swarm space', affording comparisons and generating predictions about the collective motion across species and contexts. We encourage researchers to add their own data to update the 'swarm space' for future comparative work. Second, we investigate intraspecific variation in collective motion over time and provide guidance for researchers on when observations made over different time scales can result in confident inferences regarding species collective motion. This article is part of a discussion meeting issue 'Collective behaviour through time'.

A double-spiral maze and hi-resolution tracking pipeline to study dispersal by groups of minute insects

Minute insects such as parasitic micro-wasps have high basic and applied importance for their widespread use as biocontrol agents. Their dispersal is a phenotype of particular interest. Classically, it is evaluated using field releases, but those are time consuming, costly, and their results highly variable, preventing high-throughput and repeatability. Alternatively, dispersal can be studied using small-scale assays, but those neglect important higher-scale processes. Consequently, proper evaluation of dispersal is often complicated or lacking in academic studies and biocontrol breeding programs. Here we introduce a new method, the double-spiral maze, that allows the study of spatial propagation of groups of micro-wasps at relevant scales (several hours and meters), retaining high throughput and experimental power. The method records the location of every individual at every time, enabling accurate estimates of diffusion coefficients or other dispersal metrics. We describe this affordable, scalable, and easy-to-implement method, and illustrate its application with a species of agricultural interest.

Analysis of Feeding Behavior Characteristics in the Cu/Zn Superoxide Dismutase 1 (SOD1) SOD1G93A Mice Model for Amyotrophic Lateral Sclerosis (ALS)

Amyotrophic lateral sclerosis (ALS) is a progressive disease affecting upper and lower motor neurons. Feeding disorders are observed in patients with ALS. The mastication movements and their systemic effects in patients with ALS with feeding disorders remain unclear. Currently, there is no effective treatment for ALS. However, it has been suggested that treating feeding disorders and improving nutritional status may prolong the lives of patients with ALS. Therefore, this study elucidates feeding disorders observed in patients with ALS and future therapeutic agents. We conducted a temporal observation of feeding behavior and mastication movements using an open-closed mouth evaluation artificial intelligence (AI) model in an ALS mouse model. Furthermore, to determine the cause of masticatory rhythm modulation, we conducted electrophysiological analyses of mesencephalic trigeminal neurons (MesV). Here, we observed the modulation of masticatory rhythm with a prolonged open phase in the ALS mouse model from the age of 12 weeks. A decreased body weight was observed simultaneously, indicating a correlation between the prolongation of the open phase and the decrease observed. We found that the percentage of firing MesV was markedly decreased. This study partially clarifies the role of feeding disorders in ALS.

Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision

Methods for collecting animal behaviour data in natural environments, such as direct observation and biologging, are typically limited in spatiotemporal resolution, the number of animals that can be observed and information about animals' social and physical environments. Video imagery can capture rich information about animals and their environments, but image-based approaches are often impractical due to the challenges of processing large and complex multi-image datasets and transforming resulting data, such as animals' locations, into geographical coordinates. We demonstrate a new system for studying behaviour in the wild that uses drone-recorded videos and computer vision approaches to automatically track the location and body posture of free-roaming animals in georeferenced coordinates with high spatiotemporal resolution embedded in contemporaneous 3D landscape models of the surrounding area. We provide two worked examples in which we apply this approach to videos of gelada monkeys and multiple species of group-living African ungulates. We demonstrate how to track multiple animals simultaneously, classify individuals by species and age-sex class, estimate individuals' body postures (poses) and extract environmental features, including topography of the landscape and animal trails. By quantifying animal movement and posture while reconstructing a detailed 3D model of the landscape, our approach opens the door to studying the sensory ecology and decision-making of animals within their natural physical and social environments.

Density of predating Asian hornets at hives disturbs the 3D flight performance of honey bees and decreases predation success

Automated 3D image-based tracking systems are new and promising devices to investigate the foraging behavior of flying animals with great accuracy and precision. 3D analyses can provide accurate assessments of flight performance in regard to speed, curvature, and hovering. However, there have been few applications of this technology in ecology, particularly for insects. We used this technology to analyze the behavioral interactions between the Western honey bee Apis mellifera and its invasive predator the Asian hornet, Vespa velutina nigrithorax. We investigated whether predation success could be affected by flight speed, flight curvature, and hovering of the Asian hornet and honey bees in front of one beehive. We recorded a total of 603,259 flight trajectories and 5175 predator-prey flight interactions leading to 126 successful predation events, representing 2.4% predation success. Flight speeds of hornets in front of hive entrances were much lower than that of their bee prey; in contrast to hovering capacity, while curvature range overlapped between the two species. There were large differences in speed, curvature, and hovering between the exit and entrance flights of honey bees. Interestingly, we found hornet density affected flight performance of both honey bees and hornets. Higher hornet density led to a decrease in the speed of honey bees leaving the hive, and an increase in the speed of honey bees entering the hive, together with more curved flight trajectories. These effects suggest some predator avoidance behavior by the bees. Higher honey bee flight curvature resulted in lower hornet predation success. Results showed an increase in predation success when hornet number increased up to 8 individuals, above which predation success decreased, likely due to competition among predators. Although based on a single colony, this study reveals interesting outcomes derived from the use of automated 3D tracking to derive accurate measures of individual behavior and behavioral interactions among flying species.

Predicting effects of multiple interacting global change drivers across trophic levels

Global change encompasses many co-occurring anthropogenic drivers, which can act synergistically or antagonistically on ecological systems. Predicting how different global change drivers simultaneously contribute to observed biodiversity change is a key challenge for ecology and conservation. However, we lack the mechanistic understanding of how multiple global change drivers influence the vital rates of multiple interacting species. We propose that reaction norms, the relationships between a driver and vital rates like growth, mortality, and consumption, provide insights to the underlying mechanisms of community responses to multiple drivers. Understanding how multiple drivers interact to affect demographic rates using a reaction-norm perspective can improve our ability to make predictions of interactions at higher levels of organization-that is, community and food web. Building on the framework of consumer-resource interactions and widely studied thermal performance curves, we illustrate how joint driver impacts can be scaled up from the population to the community level. A simple proof-of-concept model demonstrates how reaction norms of vital rates predict the prevalence of driver interactions at the community level. A literature search suggests that our proposed approach is not yet used in multiple driver research. We outline how realistic response surfaces (i.e., multidimensional reaction norms) can be inferred by parametric and nonparametric approaches. Response surfaces have the potential to strengthen our understanding of how multiple drivers affect communities as well as improve our ability to predict when interactive effects emerge, two of the major challenges of ecology today.

When complex movement yields simple dispersal: behavioural heterogeneity, spatial spread and parasitism in groups of micro-wasps

BACKGROUND

Understanding how behavioural dynamics, inter-individual variability and individual interactions scale-up to shape the spatial spread and dispersal of animal populations is a major challenge in ecology. For biocontrol agents, such as the microscopic Trichogramma parasitic wasps, an understanding of movement strategies is also critical to predict pest-suppression performance in the field.

METHODS

We experimentally studied the spatial propagation of groups of parasitoids and their patterns of parasitism. We investigated whether population spread is density-dependent, how it is affected by the presence of hosts, and whether the spatial distribution of parasitism (dispersal kernel) can be predicted from the observed spread of individuals. Using a novel experimental device and high-throughput imaging techniques, we continuously tracked the spatial spread of groups of parasitoids over large temporal and spatial scales (8 h; and 6 m, ca. 12,000 body lengths). We could thus study how population density, the presence of hosts and their spatial distribution impacted the rate of population spread, the spatial distribution of individuals during population expansion, the overall rate of parasitism and the dispersal kernel (position of parasitism events).

RESULTS

Higher population density accelerated population spread, but only transiently: the rate of spread reverted to low values after 4 h, in a "tortoise-hare" effect. Interestingly, the presence of hosts suppressed this transiency and permitted a sustained high rate of population spread. Importantly, we found that population spread did not obey classical diffusion, but involved dynamical switches between resident and explorer movement modes. Population distribution was therefore not Gaussian, though surprisingly the distribution of parasitism (dispersal kernel) was.

CONCLUSIONS

Even homogenous asexual groups of insects develop behavioural heterogeneities over a few hours, and the latter control patterns of population spread. Behavioural switching between resident and explorer states determined population distribution, density-dependence and dispersal. A simple Gaussian dispersal kernel did not reflect classical diffusion, but rather the interplay of several non-linearities at individual level. These results highlight the need to take into account behaviour and inter-individual heterogeneity to understand population spread in animals.

Predicting ecosystem metaphenome from community metagenome: A grand challenge for environmental biology

Elucidating how an organism's characteristics emerge from its DNA sequence has been one of the great triumphs of biology. This triumph has cumulated in sophisticated computational models that successfully predict how an organism's detailed phenotype emerges from its specific genotype. Inspired by that effort's vision and empowered by its methodologies, a grand challenge is described here that aims to predict the biotic characteristics of an ecosystem, its metaphenome, from nucleic acid sequences of all the species in its community, its metagenome. Meeting this challenge would integrate rapidly advancing abilities of environmental nucleic acids (eDNA and eRNA) to identify organisms, their ecological interactions, and their evolutionary relationships with advances in mechanistic models of complex ecosystems. Addressing the challenge would help integrate ecology and evolutionary biology into a more unified and successfully predictive science that can better help describe and manage ecosystems and the services they provide to humanity.

Motor deficit and lack of overt dystonia in Dlx conditional Dyt1 knockout mice

DYT1 or DYT-TOR1A dystonia is early-onset generalized dystonia caused by a trinucleotide deletion of GAG in the TOR1A or DYT1 gene leads to the loss of a glutamic acid residue in the resulting torsinA protein. A mouse model with overt dystonia is of unique importance to better understand the DYT1 pathophysiology and evaluate preclinical drug efficacy. DYT1 dystonia is likely a network disorder involving multiple brain regions, particularly the basal ganglia. Tor1a conditional knockout in the striatum or cerebral cortex leads to motor deficits, suggesting the importance of corticostriatal connection in the pathogenesis of dystonia. Indeed, corticostriatal long-term depression impairment has been demonstrated in multiple targeted DYT1 mouse models. Pappas and colleagues developed a conditional knockout line (Dlx-CKO) that inactivated Tor1a in the forebrain and surprisingly displayed overt dystonia. We set out to validate whether conditional knockout affecting both cortex and striatum would lead to overt dystonia and whether machine learning-based video behavioral analysis could be used to facilitate high throughput preclinical drug screening. We generated Dlx-CKO mice and found no overt dystonia or motor deficits at 4 months. At 8 months, retesting revealed motor deficits in rotarod, beam walking, grip strength, and hyperactivity in the open field; however, no overt dystonia was visually discernible or through the machine learning-based video analysis. Consistent with other targeted DYT1 mouse models, we observed age-dependent deficits in the beam walking test, which is likely a better motor behavioral test for preclinical drug testing but more labor-intensive when overt dystonia is absent.

Leveraging big data to uncover the eco-evolutionary factors shaping behavioural development

Mapping the eco-evolutionary factors shaping the development of animals' behavioural phenotypes remains a great challenge. Recent advances in 'big behavioural data' research-the high-resolution tracking of individuals and the harnessing of that data with powerful analytical tools-have vastly improved our ability to measure and model developing behavioural phenotypes. Applied to the study of behavioural ontogeny, the unfolding of whole behavioural repertoires can be mapped in unprecedented detail with relative ease. This overcomes long-standing experimental bottlenecks and heralds a surge of studies that more finely define and explore behavioural-experiential trajectories across development. In this review, we first provide a brief guide to state-of-the-art approaches that allow the collection and analysis of high-resolution behavioural data across development. We then outline how such approaches can be used to address key issues regarding the ecological and evolutionary factors shaping behavioural development: developmental feedbacks between behaviour and underlying states, early life effects and behavioural transitions, and information integration across development.

A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior

Accurate and efficient quantification of animal behavior facilitates the understanding of the brain. An emerging approach within machine learning (ML) field is to combine multiple ML-based algorithms to quantify animal behavior. These so-called hybrid models have emerged because of limitations associated with supervised [e.g., random forest (RF)] and unsupervised [e.g., hidden Markov model (HMM)] ML models. For example, RF models lack temporal information across video frames, and HMM latent states are often difficult to interpret. We sought to develop a hybrid model, and did so in the context of a study of mouse risk assessment behavior. We used DeepLabCut to estimate the positions of mouse body parts. Positional features were calculated using DeepLabCut outputs and were used to train RF and HMM models with equal number of states, separately. The per-frame predictions from RF and HMM models were then passed to a second HMM model layer ("reHMM"). The outputs of the reHMM layer showed improved interpretability over the initial HMM output. Finally, we combined predictions from RF and HMM models with selected positional features to train a third HMM model ("reHMM+"). This reHMM+ layered hybrid model unveiled distinctive temporal and human-interpretable behavioral patterns. We applied this workflow to investigate risk assessment to trimethylthiazoline and snake feces odor, finding unique behavioral patterns to each that were separable from attractive and neutral stimuli. We conclude that this layered, hybrid ML workflow represents a balanced approach for improving the depth and reliability of ML classifiers in chemosensory and other behavioral contexts.

Exploiting animal personality to reduce chronic stress in captive fish populations

Chronic stress is a major source of welfare problems in many captive populations, including fishes. While we have long known that chronic stress effects arise from maladaptive expression of acute stress response pathways, predicting where and when problems will arise is difficult. Here we highlight how insights from animal personality research could be useful in this regard. Since behavior is the first line of organismal defense when challenged by a stressor, assays of shy-bold type personality variation can provide information about individual stress response that is expected to predict susceptibility to chronic stress. Moreover, recent demonstrations that among-individual differences in stress-related physiology and behaviors are underpinned by genetic factors means that selection on behavioral biomarkers could offer a route to genetic improvement of welfare outcomes in captive fish stocks. Here we review the evidence in support of this proposition, identify remaining empirical gaps in our understanding, and set out appropriate criteria to guide development of biomarkers. The article is largely prospective: fundamental research into fish personality shows how behavioral biomarkers could be used to achieve welfare gains in captive fish populations. However, translating potential to actual gains will require an interdisciplinary approach that integrates the expertise and viewpoints of researchers working across animal behavior, genetics, and welfare science.

Experimental identification of individual insect visual tracking delays in free flight and their effects on visual swarm patterns

Insects are model systems for swarming robotic agents, yet engineered descriptions do not fully explain the mechanisms by which they provide onboard sensing and feedback to support such motions; in particular, the exact value and population distribution of visuomotor processing delays are not yet quantified, nor the effect of such delays on a visually-interconnected swarm. This study measures untethered insects performing a solo in-flight visual tracking task and applies system identification techniques to build an experimentally-consistent model of the visual tracking behaviors, and then integrates the measured experimental delay and its variation into a visually interconnected swarm model to develop theoretical and simulated solutions and stability limits. The experimental techniques include the development of a moving visual stimulus and real-time multi camera based tracking system called VISIONS (Visual Input System Identification from Outputs of Naturalistic Swarms) providing the capability to recognize and simultaneously track both a visual stimulus (input) and an insect at a frame rate of 60-120 Hz. A frequency domain analysis of honeybee tracking trajectories is conducted via fast Fourier and Chirp Z transforms, identifying a coherent linear region and its model structure. The model output is compared in time and frequency domain simulations. The experimentally measured delays are then related to probability density functions, and both the measured delays and their distribution are incorporated as inter-agent interaction delays in a second order swarming dynamics model. Linear stability and bifurcation analysis on the long range asymptotic behavior is used to identify delay distributions leading to a family of solutions with stable and unstable swarm center of mass (barycenter) locations. Numerical simulations are used to verify these results with both continuous and measured distributions. The results of this experiment quantify a model structure and temporal lag (transport delay) in the closed loop dynamics, and show that this delay varies across 50 individuals from 5-110ms, with an average delay of 22ms and a standard deviation of 40ms. When analyzed within the swarm model, the measured delays support a diversity of solutions and indicate an unstable barycenter.

A Data-Driven Simulation of the Trophallactic Network and Intranidal Food Flow Dissemination in Ants

Food sharing can occur in both social and non-social species, but it is crucial in eusocial species, in which only some group members collect food. This food collection and the intranidal (i.e., inside the nest) food distribution through trophallactic (i.e., mouth-to-mouth) exchanges are fundamental in eusocial insects. However, the behavioural rules underlying the regulation and the dynamics of food intake and the resulting networks of exchange are poorly understood. In this study, we provide new insights into the behavioural rules underlying the structure of trophallactic networks and food dissemination dynamics within the colony. We build a simple data-driven model that implements interindividual variability and the division of labour to investigate the processes of food accumulation/dissemination inside the nest, both at the individual and collective levels. We also test the alternative hypotheses (no variability and no division of labour). The division of labour, combined with inter-individual variability, leads to predictions of the food dynamics and exchange networks that run, contrary to the other models. Our results suggest a link between the interindividual heterogeneity of the trophallactic behaviours, the food flow dynamics and the network of trophallactic events. Our results show that a slight level of heterogeneity in the number of trophallactic events is enough to generate the properties of the experimental networks and seems to be crucial for the creation of efficient trophallactic networks. Despite the relative simplicity of the model rules, efficient trophallactic networks may emerge as the networks observed in ants, leading to a better understanding of the evolution of self-organisation in such societies.

Estimation of skeletal kinematics in freely moving rodents

Forming a complete picture of the relationship between neural activity and skeletal kinematics requires quantification of skeletal joint biomechanics during free behavior; however, without detailed knowledge of the underlying skeletal motion, inferring limb kinematics using surface-tracking approaches is difficult, especially for animals where the relationship between the surface and underlying skeleton changes during motion. Here we developed a videography-based method enabling detailed three-dimensional kinematic quantification of an anatomically defined skeleton in untethered freely behaving rats and mice. This skeleton-based model was constrained using anatomical principles and joint motion limits and provided skeletal pose estimates for a range of body sizes, even when limbs were occluded. Model-inferred limb positions and joint kinematics during gait and gap-crossing behaviors were verified by direct measurement of either limb placement or limb kinematics using inertial measurement units. Together we show that complex decision-making behaviors can be accurately reconstructed at the level of skeletal kinematics using our anatomically constrained model.

Deep learning-based behavioral profiling of rodent stroke recovery

Background

Stroke research heavily relies on rodent behavior when assessing underlying disease mechanisms and treatment efficacy. Although functional motor recovery is considered the primary targeted outcome, tests in rodents are still poorly reproducible and often unsuitable for unraveling the complex behavior after injury.

Results

Here, we provide a comprehensive 3D gait analysis of mice after focal cerebral ischemia based on the new deep learning-based software (DeepLabCut, DLC) that only requires basic behavioral equipment. We demonstrate a high precision 3D tracking of 10 body parts (including all relevant joints and reference landmarks) in several mouse strains. Building on this rigor motion tracking, a comprehensive post-analysis (with >100 parameters) unveils biologically relevant differences in locomotor profiles after a stroke over a time course of 3 weeks. We further refine the widely used ladder rung test using deep learning and compare its performance to human annotators. The generated DLC-assisted tests were then benchmarked to five widely used conventional behavioral set-ups (neurological scoring, rotarod, ladder rung walk, cylinder test, and single-pellet grasping) regarding sensitivity, accuracy, time use, and costs.

Conclusions

We conclude that deep learning-based motion tracking with comprehensive post-analysis provides accurate and sensitive data to describe the complex recovery of rodents following a stroke. The experimental set-up and analysis can also benefit a range of other neurological injuries that affect locomotion.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01434-9.

Color in motion: Generating 3-dimensional multispectral models to study dynamic visual signals in animals

Analyzing color and pattern in the context of motion is a central and ongoing challenge in the quantification of animal coloration. Many animal signals are spatially and temporally variable, but traditional methods fail to capture this dynamism because they use stationary animals in fixed positions. To investigate dynamic visual displays and to understand the evolutionary forces that shape dynamic colorful signals, we require cross-disciplinary methods that combine measurements of color, pattern, 3-dimensional (3D) shape, and motion. Here, we outline a workflow for producing digital 3D models with objective color information from museum specimens with diffuse colors. The workflow combines multispectral imaging with photogrammetry to produce digital 3D models that contain calibrated ultraviolet (UV) and human-visible (VIS) color information and incorporate pattern and 3D shape. These “3D multispectral models” can subsequently be animated to incorporate both signaler and receiver movement and analyzed in silico using a variety of receiver-specific visual models. This approach—which can be flexibly integrated with other tools and methods—represents a key first step toward analyzing visual signals in motion. We describe several timely applications of this workflow and next steps for multispectral 3D photogrammetry and animation techniques.

Advances in non-invasive tracking of wave-type electric fish in natural and laboratory settings

Recent technological advances greatly improved the possibility to study freely behaving animals in natural conditions. However, many systems still rely on animal-mounted devices, which can already bias behavioral observations. Alternatively, animal behaviors can be detected and tracked in recordings of stationary sensors, e.g., video cameras. While these approaches circumvent the influence of animal-mounted devices, identification of individuals is much more challenging. We take advantage of the individual-specific electric fields electric fish generate by discharging their electric organ (EOD) to record and track their movement and communication behaviors without interfering with the animals themselves. EODs of complete groups of fish can be recorded with electrode arrays submerged in the water and then be tracked for individual fish. Here, we present an improved algorithm for tracking electric signals of wave-type electric fish. Our algorithm benefits from combining and refining previous approaches of tracking individual specific EOD frequencies and spatial electric field properties. In this process, the similarity of signal pairs in extended data windows determines their tracking order, making the algorithm more robust against detection losses and intersections. We quantify the performance of the algorithm and show its application for a data set recorded with an array of 64 electrodes distributed over a 12 m² section of a stream in the Llanos, Colombia, where we managed, for the first time, to track Apteronotus leptorhynchus over many days. These technological advances make electric fish a unique model system for a detailed analysis of social and communication behaviors, with strong implications for our research on sensory coding.

Statistical properties of large data sets with linear latent features

Analytical understanding of how low-dimensional latent features reveal themselves in large-dimensional data is still lacking. We study this by defining a probabilistic linear latent features model with additive noise and by analytically and numerically computing the statistical distributions of pairwise correlations and eigenvalues of the data correlation matrix. This allows us to resolve the latent feature structure across a wide range of data regimes set by the number of recorded variables, observations, latent features, and the signal-to-noise ratio. We find a characteristic imprint of latent features in the distribution of correlations and eigenvalues and provide an analytic estimate for the boundary between signal and noise, even in the absence of a spectral gap.

Measuring inter-individual differences in behavioural types of gilthead seabreams in the laboratory using deep learning

Deep learning allows us to automatize the acquisition of large amounts of behavioural animal data with applications for fisheries and aquaculture. In this work, we have trained an image-based deep learning algorithm, the Faster R-CNN (Faster region-based convolutional neural network), to automatically detect and track the gilthead seabream, Sparus aurata, to search for individual differences in behaviour. We collected videos using a novel Raspberry Pi high throughput recording system attached to individual experimental behavioural arenas. From the continuous recording during behavioural assays, we acquired and labelled a total of 14,000 images and used them, along with data augmentation techniques, to train the network. Then, we evaluated the performance of our network at different training levels, increasing the number of images and applying data augmentation. For every validation step, we processed more than 52,000 images, with and without the presence of the gilthead seabream, in normal and altered (i.e., after the introduction of a non-familiar object to test for explorative behaviour) behavioural arenas. The final and best version of the neural network, trained with all the images and with data augmentation, reached an accuracy of 92,79% ± 6.78% [89.24-96.34] of correct classification and 10.25 ± 61.59 pixels [6.59-13.91] of fish positioning error. Our recording system based on a Raspberry Pi and a trained convolutional neural network provides a valuable non-invasive tool to automatically track fish movements in experimental arenas and, using the trajectories obtained during behavioural tests, to assay behavioural types.

Measuring Locomotor Activity and Behavioral Aspects of Rodents Living in the Home-Cage

Automatization and technological advances have led to a larger number of methods and systems to monitor and measure locomotor activity and more specific behavior of a wide variety of animal species in various environmental conditions in laboratory settings. In rodents, the majority of these systems require the animals to be temporarily taken away from their home-cage into separate observation cage environments which requires manual handling and consequently evokes distress for the animal and may alter behavioral responses. An automated high-throughput approach can overcome this problem. Therefore, this review describes existing automated methods and technologies which enable the measurement of locomotor activity and behavioral aspects of rodents in their most meaningful and stress-free laboratory environment: the home-cage. In line with the Directive 2010/63/EU and the 3R principles (replacement, reduction, refinement), this review furthermore assesses their suitability and potential for group-housed conditions as a refinement strategy, highlighting their current technological and practical limitations. It covers electrical capacitance technology and radio-frequency identification (RFID), which focus mainly on voluntary locomotor activity in both single and multiple rodents, respectively. Infrared beams and force plates expand the detection beyond locomotor activity toward basic behavioral traits but discover their full potential in individually housed rodents only. Despite the great premises of these approaches in terms of behavioral pattern recognition, more sophisticated methods, such as (RFID-assisted) video tracking technology need to be applied to enable the automated analysis of advanced behavioral aspects of individual animals in social housing conditions.

BOVIDS: A deep learning-based software package for pose estimation to evaluate nightly behavior and its application to common elands (Tragelaphus oryx) in zoos

Only a few studies on the nocturnal behavior of African ungulates exist so far, with mostly small sample sizes. For a comprehensive understanding of nocturnal behavior, the data basis needs to be expanded. Results obtained by observing zoo animals can provide clues for the study of wild animals and furthermore contribute to a better understanding of animal welfare and better husbandry conditions in zoos. The current contribution reduces the lack of data in two ways. First, we present a stand-alone open-source software package based on deep learning techniques, named Behavioral Observations by Videos and Images using Deep-Learning Software (BOVIDS). It can be used to identify ungulates in their enclosure and to determine the three behavioral poses "Standing," "Lying-head up," and "Lying-head down" on 11,411 h of video material with an accuracy of 99.4%. Second, BOVIDS is used to conduct a case study on 25 common elands (Tragelaphus oryx) out of 5 EAZA zoos with a total of 822 nights, yielding the first detailed description of the nightly behavior of common elands. Our results indicate that age and sex are influencing factors on the nocturnal activity budget, the length of behavioral phases as well as the number of phases per behavioral state during the night while the keeping zoo has no significant influence. It is found that males spend more time in REM sleep posture than females while young animals spend more time in this position than adult ones. Finally, the results suggest a rhythm between the Standing and Lying phases among common elands that opens future research directions.

Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking

Accurate tracking is the basis of behavioral analysis, an important research method in neuroscience and many other fields. However, the currently available tracking methods have limitations. Traditional computer vision methods have problems in complex environments, and deep learning methods are hard to be applied universally due to the requirement of laborious annotations. To address the trade-off between accuracy and universality, we developed an easy-to-use tracking tool, Siamese Network-based All-Purpose Tracker (SNAP-Tracker), a model-free tracking software built on the Siamese network. The pretrained Siamese network offers SNAP-Tracker a remarkable feature extraction ability to keep tracking accuracy, and the model-free design makes it usable directly before laborious annotations and network refinement. SNAP-Tracker provides a “tracking with detection” mode to track longer videos with an additional detection module. We demonstrate the stability of SNAP-Tracker through different experimental conditions and different tracking tasks. In short, SNAP-Tracker provides a general solution to behavioral tracking without compromising accuracy. For the user’s convenience, we have integrated the tool into a tidy graphic user interface and opened the source code for downloading and using (https://github.com/slh0302/SNAP).

The impact of individual perceptual and cognitive factors on collective states in a data-driven fish school model

In moving animal groups, social interactions play a key role in the ability of individuals to achieve coordinated motion. However, a large number of environmental and cognitive factors are able to modulate the expression of these interactions and the characteristics of the collective movements that result from these interactions. Here, we use a data-driven fish school model to quantitatively investigate the impact of perceptual and cognitive factors on coordination and collective swimming patterns. The model describes the interactions involved in the coordination of burst-and-coast swimming in groups of Hemigrammus rhodostomus. We perform a comprehensive investigation of the respective impacts of two interactions strategies between fish based on the selection of the most or the two most influential neighbors, of the range and intensity of social interactions, of the intensity of individual random behavioral fluctuations, and of the group size, on the ability of groups of fish to coordinate their movements. We find that fish are able to coordinate their movements when they interact with their most or two most influential neighbors, provided that a minimal level of attraction between fish exist to maintain group cohesion. A minimal level of alignment is also required to allow the formation of schooling and milling. However, increasing the strength of social interactions does not necessarily enhance group cohesion and coordination. When attraction and alignment strengths are too high, or when the heading random fluctuations are too large, schooling and milling can no longer be maintained and the school switches to a swarming phase. Increasing the interaction range between fish has a similar impact on collective dynamics as increasing the strengths of attraction and alignment. Finally, we find that coordination and schooling occurs for a wider range of attraction and alignment strength in small group sizes.