DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Neural representation of self-initiated locomotion in the secondary motor cortex of mice across different environmental contexts

The secondary motor cortex (M2) plays an important role in the adaptive control of locomotor behaviors. However, it is unclear how M2 neurons encode the same type of locomotor control variables in different environmental contexts. Here we image the neuronal activity in M2 with a miniscope while mice are moving freely in each of three environments: a Y-maze, a running-wheel, and an open-field. These animals show distinct locomotor patterns in different environmental contexts. Surprisingly, a large population of M2 neurons are active before starting and after ceasing locomotion, while maintaining decreased neural activity during locomotion. Furthermore, the majority of these neurons are consistently engaged across various contexts, suggesting egocentric voluntary control functions. In contrast, the smaller populations of locomotion-activated M2 neurons are mostly context-specific, suggesting exocentric navigation functions. Thus, our results demonstrate that M2 neurons encode motor control variables for self-initiated locomotor behaviors in both context-dependent and context-independent manners.

Real-time fear emotion recognition in mice based on multimodal data fusion

A multimodal emotion recognition method that utilizes facial expressions, body postures, and movement trajectories to detect fear in mice is proposed in this study. By integrating and analyzing these distinct data sources through feature encoders and attention classifiers, we developed a robust emotion classification model. The performance of the model was evaluated by comparing it with single-modal methods, and the results showed significant accuracy improvements. Our findings indicate that the multimodal fusion emotion recognition model enhanced the precision of emotion detection, achieving a fear recognition accuracy of 86.7%. Additionally, the impacts of different monitoring durations and frame sampling rates on the achieved recognition accuracy were investigated in this study. The proposed method provides an efficient and simple solution for conducting real-time, comprehensive emotion monitoring in animal research, with potential applications in neuroscience and psychiatric studies.

Ledged Beam Walking Test Automatic Tracker: Artificial intelligence-based functional evaluation in a stroke model

The quantitative evaluation of motor function in experimental stroke models is essential for the preclinical assessment of new therapeutic strategies that can be transferred to clinical research; however, conventional assessment tests are hampered by the evaluator's subjectivity. We present an artificial intelligence-based system for the automatic, accurate, and objective analysis of target parameters evaluated by the ledged beam walking test, which offers higher sensitivity than the current methodology based on manual and visual counting. This system employs a residual deep network model, trained with DeepLabCut (DLC) to extract target paretic hindlimb coordinates, which are categorized to provide a ratio measurement of the animal's neurological deficit. The results correlate with the measurements performed by a professional observer and have greater reproducibility, easing the analysis of motor deficits and providing a reliable and useful tool applicable to other diseases causing motor deficits.

A real-time, multi-subject three-dimensional pose tracking system for the behavioral analysis of non-human primates

The ability to track the positions and poses of multiple animals in three-dimensional (3D) space in real time is highly desired by non-human primate (NHP) researchers in behavioral and systems neuroscience. This capability enables the analysis of social behaviors involving multiple NHPs and supports closed-loop experiments. Although several animal 3D pose tracking systems have been developed, most are difficult to deploy in new environments and lack real-time analysis capabilities. To address these limitations, we developed MarmoPose, a deep-learning-based, real-time 3D pose tracking system for multiple common marmosets, an increasingly critical NHP model in neuroscience research. This system can accurately track the 3D poses of multiple marmosets freely moving in their home cage with minimal hardware requirements. By employing a marmoset skeleton model, MarmoPose can further optimize 3D poses and estimate invisible body locations. Additionally, MarmoPose achieves high inference speeds and enables real-time closed-loop experimental control based on events detected from 3D poses.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for studying brain and behavior. Recently, computer vision techniques have become essential for high-throughput behavioral studies; however, localizing animals in complex conditions remains challenging due to intra-class visual variability and environmental diversity. These challenges hinder studies in naturalistic settings, such as when animals are partially concealed within nests. Moreover, current tools are laborious and time-consuming, requiring extensive, setup-specific annotation and training procedures. To address these challenges, we introduce the 'Detect-Any-Mouse-Model' (DAMM), an object detector for localizing mice in complex environments with minimal training. Our approach involved collecting and annotating a diverse dataset of single- and multi-housed mice in complex setups. We trained a Mask R-CNN, a popular object detector in animal studies, to perform instance segmentation and validated DAMM's performance on a collection of downstream datasets using zero-shot and few-shot inference. DAMM excels in zero-shot inference, detecting mice and even rats, in entirely unseen scenarios and further improves with minimal training. Using the SORT algorithm, we demonstrate robust tracking, competitive with keypoint-estimation-based methods. Notably, to advance and simplify behavioral studies, we release our code, model weights, and data, along with a user-friendly Python API and a Google Colab implementation.

Computer vision analysis of mother-infant interaction identified efficient pup retrieval in V1b receptor knockout mice

Close contact between lactating rodent mothers and their infants is essential for effective nursing. Whether the mother's effort to retrieve the infants to their nest requires the vasopressin-signaling via V1b receptor has not been fully defined. To address this question, V1b receptor knockout (V1bKO) and control mice were analyzed in pup retrieval test. Because an exploring mother in a new test cage randomly accessed to multiple infants in changing backgrounds over time, a computer vision-based deep learning analysis was applied to continuously calculate the distances between the mother and the infants as a parameter of their relationship. In an open-field, a virgin female V1bKO mice entered fewer times into the center area and moved shorter distances than wild-type (WT). While this behavioral pattern persisted in V1bKO mother, the pup retrieval test demonstrated that total distances between a V1bKO mother and infants came closer in a shorter time than with a WT mother. Moreover, in the medial preoptic area, parts of the V1b receptor transcripts were detected in galanin- and c-fos-positive neurons following maternal stimulation by infants. This research highlights the effectiveness of deep learning analysis in evaluating the mother-infant relationship and the critical role of V1b receptor in pup retrieval during the early lactation phase.

Development of a 3D tracking system for multiple marmosets under free-moving conditions

Assessment of social interactions and behavioral changes in nonhuman primates is useful for understanding brain function changes during life events and pathogenesis of neurological diseases. The common marmoset (Callithrix jacchus), which lives in a nuclear family like humans, is a useful model, but longitudinal automated behavioral observation of multiple animals has not been achieved. Here, we developed a Full Monitoring and Animal Identification (FulMAI) system for longitudinal detection of three-dimensional (3D) trajectories of each individual in multiple marmosets under free-moving conditions by combining video tracking, Light Detection and Ranging, and deep learning. Using this system, identification of each animal was more than 97% accurate. Location preferences and inter-individual distance could be calculated, and deep learning could detect grooming behavior. The FulMAI system allows us to analyze the natural behavior of individuals in a family over their lifetime and understand how behavior changes due to life events together with other data.

Functional trajectories during innate spinal cord repair

Adult zebrafish are capable of anatomical and functional recovery following severe spinal cord injury. Axon growth, glial bridging and adult neurogenesis are hallmarks of cellular regeneration during spinal cord repair. However, the correlation between these cellular regenerative processes and functional recovery remains to be elucidated. Whereas the majority of established functional regeneration metrics measure swim capacity, we hypothesize that gait quality is more directly related to neurological health. Here, we performed a longitudinal swim tracking study for 60 individual zebrafish spanning 8 weeks of spinal cord regeneration. Multiple swim parameters as well as axonal and glial bridging were integrated. We established rostral compensation as a new gait quality metric that highly correlates with functional recovery. Tensor component analysis of longitudinal data supports a correspondence between functional recovery trajectories and neurological outcomes. Moreover, our studies predicted and validated that a subset of functional regeneration parameters measured 1 to 2 weeks post-injury is sufficient to predict the regenerative outcomes of individual animals at 8 weeks post-injury. Our findings established new functional regeneration parameters and generated a comprehensive correlative database between various functional and cellular regeneration outputs.

Functional Trajectories during innate spinal cord repair

Adult zebrafish are capable of anatomical and functional recovery following severe spinal cord injury. Axon growth, glial bridging and adult neurogenesis are hallmarks of cellular regeneration during spinal cord repair. However, the correlation between these cellular regenerative processes and functional recovery remains to be elucidated. Whereas the majority of established functional regeneration metrics measure swim capacity, we hypothesize that gait quality is more directly related to neurological health. Here, we performed a longitudinal swim tracking study for sixty individual zebrafish spanning eight weeks of spinal cord regeneration. Multiple swim parameters as well as axonal and glial bridging were integrated. We established rostral compensation as a new gait quality metric that highly correlates with functional recovery. Tensor component analysis of longitudinal data supports a correspondence between functional recovery trajectories and neurological outcomes. Moreover, our studies predicted and validated that a subset of functional regeneration parameters measured 1 to 2 weeks post-injury is sufficient to predict the regenerative outcomes of individual animals at 8 weeks post-injury. Our findings established new functional regeneration parameters and generated a comprehensive correlative database between various functional and cellular regeneration outputs.

Using deep learning to study emotional behavior in rodent models

Quantifying emotional aspects of animal behavior (e.g., anxiety, social interactions, reward, and stress responses) is a major focus of neuroscience research. Because manual scoring of emotion-related behaviors is time-consuming and subjective, classical methods rely on easily quantified measures such as lever pressing or time spent in different zones of an apparatus (e.g., open vs. closed arms of an elevated plus maze). Recent advancements have made it easier to extract pose information from videos, and multiple approaches for extracting nuanced information about behavioral states from pose estimation data have been proposed. These include supervised, unsupervised, and self-supervised approaches, employing a variety of different model types. Representations of behavioral states derived from these methods can be correlated with recordings of neural activity to increase the scope of connections that can be drawn between the brain and behavior. In this mini review, we will discuss how deep learning techniques can be used in behavioral experiments and how different model architectures and training paradigms influence the type of representation that can be obtained.

Deconstruction of Vermal Cerebellum in Ramp Locomotion in Mice

The cerebellum is involved in encoding balance, posture, speed, and gravity during locomotion. However, most studies are carried out on flat surfaces, and little is known about cerebellar activity during free ambulation on slopes. Here, it has been imaged the neuronal activity of cerebellar molecular interneurons (MLIs) and Purkinje cells (PCs) using a miniaturized microscope while a mouse is walking on a slope. It has been found that the neuronal activity of vermal MLIs specifically enhanced during uphill and downhill locomotion. In addition, a subset of MLIs is activated during entire uphill or downhill positions on the slope and is modulated by the slope inclines. In contrast, PCs showed counter-balanced neuronal activity to MLIs, which reduced activity at the ramp peak. So, PCs may represent the ramp environment at the population level. In addition, chemogenetic inactivation of lobule V of the vermis impaired uphill locomotion. These results revealed a novel micro-circuit in the vermal cerebellum that regulates ambulatory behavior in 3D terrains.

A virtual library for behavioral performance in standard conditions-rodent spontaneous activity in an open field during repeated testing and after treatment with drugs or brain lesions

BACKGROUND

Beyond their specific experiment, video records of behavior have future value-for example, as inputs for new experiments or for yet unknown types of analysis of behavior-similar to tissue or blood sample banks in life sciences where clinically derived or otherwise well-described experimental samples are stored to be available for some unknown potential future purpose.

FINDINGS

Research using an animal model of obsessive-compulsive disorder employed a standardized paradigm where the behavior of rats in a large open field was video recorded for 55 minutes on each test. From 43 experiments, there are 19,976 such trials that amount to over 2 years of continuous recording. In addition to videos, there are 2 video-derived raw data objects: XY locomotion coordinates and plots of animal trajectory. To motivate future use, the 3 raw data objects are annotated with a general schema-one that abstracts the data records from their particular experiment while providing, at the same time, a detailed list of independent variables bearing on behavioral performance. The raw data objects are deposited as 43 datasets but constitute, functionally, a library containing 1 large dataset.

CONCLUSIONS

Size and annotation schema give the library high reuse potential: in applications using machine learning techniques, statistical evaluation of subtle factors, simulation of new experiments, or as educational resource. Ultimately, the library can serve both as the seed and as the test bed to create a machine-searchable virtual library of linked open datasets for behavioral performance in defined conditions.