Classifying sex and strain from mouse ultrasonic vocalizations using deep learning

Characterization of Ultrasonic Vocalization-Modulated Neurons in Rat Motor Cortex Based on Their Activity Modulation and Axonal Projection to the Periaqueductal Gray

Vocalization, a means of social communication, is prevalent among many species, including humans. Both rats and mice use ultrasonic vocalizations (USVs) in various social contexts and affective states. The motor cortex is hypothesized to be involved in precisely controlling USVs through connections with critical regions of the brain for vocalization, such as the periaqueductal gray matter (PAG). However, it is unclear how neurons in the motor cortex are modulated during USVs. Moreover, the relationship between USV modulation of neurons and anatomical connections from the motor cortex to PAG is also not clearly understood. In this study, we first characterized the activity patterns of neurons in the primary and secondary motor cortices during emission of USVs in rats using large-scale electrophysiological recordings. We also examined the axonal projection of the motor cortex to PAG using retrograde labeling and identified two clusters of PAG-projecting neurons in the anterior and posterior parts of the motor cortex. The neural activity patterns around the emission of USVs differed between the anterior and posterior regions, which were divided based on the distribution of PAG-projecting neurons in the motor cortex. Furthermore, using optogenetic tagging, we recorded the USV modulation of PAG-projecting neurons in the posterior part of the motor cortex and found that they showed predominantly sustained excitatory responses during USVs. These results contribute to our understanding of the involvement of the motor cortex in the generation of USV at the neuronal and circuit levels.

Attraction of female house mice to male ultrasonic courtship vocalizations depends on their social experience and estrous stage

Male house mice (Mus musculus) produce complex ultrasonic vocalizations (USVs), especially during courtship and mating. Playback experiments suggest that female attraction towards recordings of male USVs depends on their social experience, paternal exposure, and estrous stage. We conducted a playback experiment with wild-derived female house mice (M. musculus musculus) and compared their attraction to male USVs versus the same recording without USVs (background noise). We tested whether female attraction to USVs is influenced by the following factors: (1) social housing (two versus one female per cage); (2) neonatal paternal exposure (rearing females with versus without father); and (3) estrous stage. We found that females showed a significant attraction to male USVs but only when they were housed socially with another female. Individually housed females showed the opposite response. We found no evidence that pre-weaning exposure to a father influenced females' preferences, whereas estrous stage influenced females' attraction to male USVs: females not in estrus showed preferences towards male USVs, whereas estrous females did not. Finally, we found that individually housed females were more likely to be in sexually receptive estrous stages than those housed socially, and that attraction to male USVs was most pronounced amongst non-receptive females that were socially housed. Our findings indicate that the attraction of female mice to male USVs depends upon their social experience and estrous stage, though not paternal exposure. They contribute to the growing number of studies showing that social housing and estrous stage can influence the behavior of house mice and we show how such unreported variables can contribute to the replication crisis.

Who is calling? Optimizing source identification from marmoset vocalizations with hierarchical machine learning classifiers

With their highly social nature and complex vocal communication system, marmosets are important models for comparative studies of vocal communication and, eventually, language evolution. However, our knowledge about marmoset vocalizations predominantly originates from playback studies or vocal interactions between dyads, and there is a need to move towards studying group-level communication dynamics. Efficient source identification from marmoset vocalizations is essential for this challenge, and machine learning algorithms (MLAs) can aid it. Here we built a pipeline capable of plentiful feature extraction, meaningful feature selection, and supervised classification of vocalizations of up to 18 marmosets. We optimized the classifier by building a hierarchical MLA that first learned to determine the sex of the source, narrowed down the possible source individuals based on their sex and then determined the source identity. We were able to correctly identify the source individual with high precisions (87.21%-94.42%, depending on call type, and up to 97.79% after the removal of twins from the dataset). We also examine the robustness of identification across varying sample sizes. Our pipeline is a promising tool not only for source identification from marmoset vocalizations but also for analysing vocalizations of other species.

Cortical circuits modulate mouse social vocalizations

Vocalizations provide a means of communication with high fidelity and information rate for many species. Diencephalon and brainstem neural circuits have been shown to control mouse vocal production; however, the role of cortical circuits in this process is debatable. Using electrical and optogenetic stimulation, we identified a cortical region in the anterior cingulate cortex in which stimulation elicits ultrasonic vocalizations. Moreover, fiber photometry showed an increase in Ca2+ dynamics preceding vocal initiation, whereas optogenetic suppression in this cortical area caused mice to emit fewer vocalizations. Last, electrophysiological recordings indicated a differential increase in neural activity in response to female social exposure dependent on vocal output. Together, these results indicate that the cortex is a key node in the neuronal circuits controlling vocal behavior in mice.

Predictive Mouse Ultrasonic Vocalization Sequences: Uncovering Behavioral Significance, Auditory Cortex Neuronal Preferences, and Social-Experience-Driven Plasticity

Mouse ultrasonic vocalizations (USVs) contain predictable sequential structures like bird songs and speech. Neural representation of USVs in the mouse primary auditory cortex (Au1) and its plasticity with experience has been largely studied with single-syllables or dyads, without using the predictability in USV sequences. Studies using playback of USV sequences have used randomly selected sequences from numerous possibilities. The current study uses mutual information to obtain context-specific natural sequences (NSeqs) of USV syllables capturing the observed predictability in male USVs in different contexts of social interaction with females. Behavioral and physiological significance of NSeqs over random sequences (RSeqs) lacking predictability were examined. Female mice, never having the social experience of being exposed to males, showed higher selectivity for NSeqs behaviorally and at cellular levels probed by expression of immediate early gene c-fos in Au1. The Au1 supragranular single units also showed higher selectivity to NSeqs over RSeqs. Social-experience-driven plasticity in encoding NSeqs and RSeqs in adult females was probed by examining neural selectivities to the same sequences before and after the above social experience. Single units showed enhanced selectivity for NSeqs over RSeqs after the social experience. Further, using two-photon Ca2+ imaging, we observed social experience-dependent changes in the selectivity of sequences of excitatory and somatostatin-positive inhibitory neurons but not parvalbumin-positive inhibitory neurons of Au1. Using optogenetics, somatostatin-positive neurons were identified as a possible mediator of the observed social-experience-driven plasticity. Our study uncovers the importance of predictive sequences and introduces mouse USVs as a promising model to study context-dependent speech like communications.SIGNIFICANCE STATEMENT Humans need to detect patterns in the sensory world. For instance, speech is meaningful sequences of acoustic tokens easily differentiated from random ordered tokens. The structure derives from the predictability of the tokens. Similarly, mouse vocalization sequences have predictability and undergo context-dependent modulation. Our work investigated whether mice differentiate such informative predictable sequences (NSeqs) of communicative significance from RSeqs at the behavioral, molecular, and neuronal levels. Following a social experience in which NSeqs occur as a crucial component, mouse auditory cortical neurons become more sensitive to differences between NSeqs and RSeqs, although preference for individual tokens is unchanged. Thus, speech-like communication and its dysfunction may be studied in circuit, cellular, and molecular levels in mice.

Rodent ultrasonic vocal interaction resolved with millimeter precision using hybrid beamforming

Ultrasonic vocalizations (USVs) fulfill an important role in communication and navigation in many species. Because of their social and affective significance, rodent USVs are increasingly used as a behavioral measure in neurodevelopmental and neurolinguistic research. Reliably attributing USVs to their emitter during close interactions has emerged as a difficult, key challenge. If addressed, all subsequent analyses gain substantial confidence. We present a hybrid ultrasonic tracking system, Hybrid Vocalization Localizer (HyVL), that synergistically integrates a high-resolution acoustic camera with high-quality ultrasonic microphones. HyVL is the first to achieve millimeter precision (~3.4-4.8 mm, 91% assigned) in localizing USVs, ~3× better than other systems, approaching the physical limits (mouse snout ~10 mm). We analyze mouse courtship interactions and demonstrate that males and females vocalize in starkly different relative spatial positions, and that the fraction of female vocalizations has likely been overestimated previously due to imprecise localization. Further, we find that when two male mice interact with one female, one of the males takes a dominant role in the interaction both in terms of the vocalization rate and the location relative to the female. HyVL substantially improves the precision with which social communication between rodents can be studied. It is also affordable, open-source, easy to set up, can be integrated with existing setups, and reduces the required number of experiments and animals.

Improving the workflow to crack Small, Unbalanced, Noisy, but Genuine (SUNG) datasets in bioacoustics: The case of bonobo calls

Despite the accumulation of data and studies, deciphering animal vocal communication remains challenging. In most cases, researchers must deal with the sparse recordings composing Small, Unbalanced, Noisy, but Genuine (SUNG) datasets. SUNG datasets are characterized by a limited number of recordings, most often noisy, and unbalanced in number between the individuals or categories of vocalizations. SUNG datasets therefore offer a valuable but inevitably distorted vision of communication systems. Adopting the best practices in their analysis is essential to effectively extract the available information and draw reliable conclusions. Here we show that the most recent advances in machine learning applied to a SUNG dataset succeed in unraveling the complex vocal repertoire of the bonobo, and we propose a workflow that can be effective with other animal species. We implement acoustic parameterization in three feature spaces and run a Supervised Uniform Manifold Approximation and Projection (S-UMAP) to evaluate how call types and individual signatures cluster in the bonobo acoustic space. We then implement three classification algorithms (Support Vector Machine, xgboost, neural networks) and their combination to explore the structure and variability of bonobo calls, as well as the robustness of the individual signature they encode. We underscore how classification performance is affected by the feature set and identify the most informative features. In addition, we highlight the need to address data leakage in the evaluation of classification performance to avoid misleading interpretations. Our results lead to identifying several practical approaches that are generalizable to any other animal communication system. To improve the reliability and replicability of vocal communication studies with SUNG datasets, we thus recommend: i) comparing several acoustic parameterizations; ii) visualizing the dataset with supervised UMAP to examine the species acoustic space; iii) adopting Support Vector Machines as the baseline classification approach; iv) explicitly evaluating data leakage and possibly implementing a mitigation strategy.

High-precision spatial analysis of mouse courtship vocalization behavior reveals sex and strain differences

Mice display a wide repertoire of vocalizations that varies with sex, strain, and context. Especially during social interaction, including sexually motivated dyadic interaction, mice emit sequences of ultrasonic vocalizations (USVs) of high complexity. As animals of both sexes vocalize, a reliable attribution of USVs to their emitter is essential. The state-of-the-art in sound localization for USVs in 2D allows spatial localization at a resolution of multiple centimeters. However, animals interact at closer ranges, e.g. snout-to-snout. Hence, improved algorithms are required to reliably assign USVs. We present a novel algorithm, SLIM (Sound Localization via Intersecting Manifolds), that achieves a 2-3-fold improvement in accuracy (13.1-14.3 mm) using only 4 microphones and extends to many microphones and localization in 3D. This accuracy allows reliable assignment of 84.3% of all USVs in our dataset. We apply SLIM to courtship interactions between adult C57Bl/6J wildtype mice and those carrying a heterozygous Foxp2 variant (R552H). The improved spatial accuracy reveals that vocalization behavior is dependent on the spatial relation between the interacting mice. Female mice vocalized more in close snout-to-snout interaction while male mice vocalized more when the male snout was in close proximity to the female's ano-genital region. Further, we find that the acoustic properties of the ultrasonic vocalizations (duration, Wiener Entropy, and sound level) are dependent on the spatial relation between the interacting mice as well as on the genotype. In conclusion, the improved attribution of vocalizations to their emitters provides a foundation for better understanding social vocal behaviors.

Male scent but not courtship vocalizations induce estrus in wild female house mice

Exposure to males or male urinary scent can induce and accelerate the rate of female estrous cycling in house mice ("Whitten effect"), and this response has been replicated many times since its discovery over 60 years ago. Here, we tested whether exposing female mice to recordings of male courtship ultrasonic vocalizations (USVs) induces estrous cycling, and whether exposure to both male scent and USVs has a stronger effect than to either of these stimuli alone. We conducted our study with 60 wild-derived female house mice (Mus musculus musculus). After singly housing females for 14 days, we monitored estrous stages via vaginal cytology for two weeks while isolated from males or male stimuli. We continued monitoring estrus for two more weeks during experimental exposure to one of four different types of stimuli: (1) clean bedding and background noise playback (negative control); (2) recordings of male USVs (16 min per day) and clean bedding (male USV treatment); (3) soiled male bedding and background noise playback (male odor treatment; positive control); or (4) male USVs and soiled male bedding (male odor and USV treatment). Females were then paired with males to test whether any of the four treatments influenced female reproduction (especially latency to birth). We confirmed that exposure to male odor increased female cycling, as expected, but exposure to recordings of male USVs had no effect on estrus. Females exposed to both USVs and odor went through more cycles compared to controls, but did not differ significantly from exposure to male odor (and background noise). After pairing females with a male, females showing male odor-induced cycling produced their first litter sooner than controls, whereas USVs did not have such an effect. This is the first study to our knowledge to show that male odor induces estrus in wild house mice and to show functional effects on reproduction. Our results do not support the hypothesis that male vocalizations induce female estrus, although we suggest other approaches that could be used to further test this hypothesis.

Playback of broadband vocalizations of female mice suppresses male ultrasonic calls

Although male vocalizations during opposite- sex interaction have been heavily studied as sexually selected signals, the understanding of the roles of female vocal signals produced in this context is more limited. During intersexual interactions between mice, males produce a majority of ultrasonic vocalizations (USVs), while females produce a majority of human-audible squeaks, also called broadband vocalizations (BBVs). BBVs may be produced in conjunction with defensive aggression, making it difficult to assess whether males respond to BBVs themselves. To assess the direct effect of BBVs on male behavior, we used a split-cage paradigm in which high rates of male USVs were elicited by female presence on the other side of a barrier, but which precluded extensive male-female contact and the spontaneous production of BBVs. In this paradigm, playback of female BBVs decreased USV production, which recovered after the playback period. Trials in which female vocalizations were prevented by the use of female bedding alone or of anesthetized females as stimuli also showed a decrease in response to BBV playback. No non-vocal behaviors declined during playback, although digging behavior increased. Similar to BBVs, WNs also robustly suppressed USV production, albeit to a significantly larger extent. USVs suppression had two distinct temporal components. When grouped in 5-second bins, USVs interleaved with bursts of stimulus BBVs. USV suppression also adapted to BBV playback on the order of minutes. Adaptation occurred more rapidly in males that were housed individually as opposed to socially for a week prior to testing, suggesting that the adaptation trajectory is sensitive to social experience. These findings suggest the possibility that vocal interaction between male and female mice, with males suppressing USVs in response to BBVs, may influence the dynamics of communicative behavior.

Beyond the three-chamber test: toward a multimodal and objective assessment of social behavior in rodents

MAIN: In recent years, substantial advances in social neuroscience have been realized, including the generation of numerous rodent models of autism spectrum disorder. Still, it can be argued that those methods currently being used to analyze animal social behavior create a bottleneck that significantly slows down progress in this field. Indeed, the bulk of research still relies on a small number of simple behavioral paradigms, the results of which are assessed without considering behavioral dynamics. Moreover, only few variables are examined in each paradigm, thus overlooking a significant portion of the complexity that characterizes social interaction between two conspecifics, subsequently hindering our understanding of the neural mechanisms governing different aspects of social behavior. We further demonstrate these constraints by discussing the most commonly used paradigm for assessing rodent social behavior, the three-chamber test. We also point to the fact that although emotions greatly influence human social behavior, we lack reliable means for assessing the emotional state of animals during social tasks. As such, we also discuss current evidence supporting the existence of pro-social emotions and emotional cognition in animal models. We further suggest that adequate social behavior analysis requires a novel multimodal approach that employs automated and simultaneous measurements of multiple behavioral and physiological variables at high temporal resolution in socially interacting animals. We accordingly describe several computerized systems and computational tools for acquiring and analyzing such measurements. Finally, we address several behavioral and physiological variables that can be used to assess socio-emotional states in animal models and thus elucidate intricacies of social behavior so as to attain deeper insight into the brain mechanisms that mediate such behaviors. CONCLUSIONS: In summary, we suggest that combining automated multimodal measurements with machine-learning algorithms will help define socio-emotional states and determine their dynamics during various types of social tasks, thus enabling a more thorough understanding of the complexity of social behavior.

Ultrasonic Vocalizations in Adult C57BL/6J Mice: The Role of Sex Differences and Repeated Testing

Ultrasonic vocalizations (USVs) are a major tool for assessing social communication in laboratory mice during their entire lifespan. At adulthood, male mice preferentially emit USVs toward a female conspecific, while females mostly produce ultrasonic calls when facing an adult intruder of the same sex. Recent studies have developed several sophisticated tools to analyze adult mouse USVs, especially in males, because of the increasing relevance of adult communication for behavioral phenotyping of mouse models of autism spectrum disorder (ASD). Little attention has been instead devoted to adult female USVs and impact of sex differences on the quantitative and qualitative characteristics of mouse USVs. Most of the studies have also focused on a single testing session, often without concomitant assessment of other social behaviors (e.g., sniffing), so little is still known about the link between USVs and other aspects of social interaction and their stability/variations across multiple encounters. Here, we evaluated the USVs emitted by adult male and female mice during 3 repeated encounters with an unfamiliar female, with equal or different pre-testing isolation periods between sexes. We demonstrated clear sex differences in several USVs' characteristics and other social behaviors, and these were mostly stable across the encounters and independent of pre-testing isolation. The estrous cycle of the tested females exerted quantitative effects on their vocal and non-vocal behaviors, although it did not affect the qualitative composition of ultrasonic calls. Our findings obtained in B6 mice, i.e., the strain most widely used for engineering of transgenic mouse lines, contribute to provide new guidelines for assessing ultrasonic communication in male and female adult mice.

Automatic segmentation and classification of mice ultrasonic vocalizations

This paper addresses the development of a system for classifying mouse ultrasonic vocalizations (USVs) present in audio recordings. The automatic labeling process for USVs is usually divided into two main steps: USV segmentation followed by the matching classification. Three main contributions can be highlighted: (i) a new segmentation algorithm, (ii) a new set of features, and (iii) the discrimination of a higher number of classes when compared to similar studies. The developed segmentation algorithm is based on spectral entropy analysis. This novel segmentation approach can detect USVs with 94% and 74% recall and precision, respectively. When compared to other methods/software, our segmentation algorithm achieves a higher recall. Regarding the classification phase, besides the traditional features from time, frequency, and time-frequency domains, a new set of contour-based features were extracted and used as inputs of shallow machine learning classification models. The contour-based features were obtained from the time-frequency ridge representation of USVs. The classification methods can differentiate among ten different syllable types with 81.1% accuracy and 80.5% weighted F1-score. The algorithms were developed and evaluated based on a large dataset, acquired on diverse social interaction conditions between the animals, to stimulate a varied vocal repertoire.

Ultrasonic vocalizations in laboratory mice: strain, age, and sex differences

Mice produce ultrasonic vocalizations (USVs) in different social contexts across lifespan. There is ethological evidence that pup USVs elicit maternal retrieval and adult USVs facilitate social interaction with a conspecific. Analysis of mouse vocal and social repertoire across strains, sex and contexts remains not well explored. To address these issues, in inbred (C57BL/6, FVB) and outbred (CD-1) mouse strains, we recorded and evaluated USVs as neonates and during adult social encounters (male-female and female-female social interaction). We showed significant strain differences in the quantitative (call rate and duration of USVs) and qualitative vocal analysis (spectrographic characterization) from early stage to adulthood, in line with specific patterns of social behaviors. Inbred C57BL/6 mice produced a lower number of calls with less internal changes and shorter duration; inbred FVB mice displayed more social behaviors and produced more syllables with repeated internal changes; outbred CD-1 mice had an intermediate profile. Our results suggest specific vocal signatures in each mouse strain, thus helping to better define socio-communicative profiles of mouse strains and to guide the choice of an appropriate strain according to the experimental settings.

Comparing methodologies for classification of zebra finch distance calls

Bioacoustic analysis has been used for a variety of purposes including classifying vocalizations for biodiversity monitoring and understanding mechanisms of cognitive processes. A wide range of statistical methods, including various automated methods, have been used to successfully classify vocalizations based on species, sex, geography, and individual. A comprehensive approach focusing on identifying acoustic features putatively involved in classification is required for the prediction of features necessary for discrimination in the real world. Here, we used several classification techniques, namely discriminant function analyses (DFAs), support vector machines (SVMs), and artificial neural networks (ANNs), for sex-based classification of zebra finch (Taeniopygia guttata) distance calls using acoustic features measured from spectrograms. We found that all three methods (DFAs, SVMs, and ANNs) correctly classified the calls to respective sex-based categories with high accuracy between 92 and 96%. Frequency modulation of ascending frequency, total duration, and end frequency of the distance call were the most predictive features underlying this classification in all of our models. Our results corroborate evidence of the importance of total call duration and frequency modulation in the classification of male and female distance calls. Moreover, we provide a methodological approach for bioacoustic classification problems using multiple statistical analyses.

Computational bioacoustics with deep learning: a review and roadmap

Animal vocalisations and natural soundscapes are fascinating objects of study, and contain valuable evidence about animal behaviours, populations and ecosystems. They are studied in bioacoustics and ecoacoustics, with signal processing and analysis an important component. Computational bioacoustics has accelerated in recent decades due to the growth of affordable digital sound recording devices, and to huge progress in informatics such as big data, signal processing and machine learning. Methods are inherited from the wider field of deep learning, including speech and image processing. However, the tasks, demands and data characteristics are often different from those addressed in speech or music analysis. There remain unsolved problems, and tasks for which evidence is surely present in many acoustic signals, but not yet realised. In this paper I perform a review of the state of the art in deep learning for computational bioacoustics, aiming to clarify key concepts and identify and analyse knowledge gaps. Based on this, I offer a subjective but principled roadmap for computational bioacoustics with deep learning: topics that the community should aim to address, in order to make the most of future developments in AI and informatics, and to use audio data in answering zoological and ecological questions.

Sensing the Sounds of Silence: A Pilot Study on the Detection of Model Mice of Autism Spectrum Disorder from Ultrasonic Vocalisations

Studying the animal models of human neuropsychiatric disorders can facilitate the understanding of mechanisms of symptoms both physiologically and genetically. Previous studies have shown that ultrasonic vocalisations (USVs) of mice might be efficient markers to distinguish the wild type group and the model of autism spectrum disorder (mASD). Nevertheless, in-depth analysis of these 'silence' sounds by leveraging the power of advanced computer audition technologies (e. g., deep learning) is limited. To this end, we propose a pilot study on using a large-scale pre-trained audio neural network to extract high-level representations from the USVs of mice for the task on detection of mASD. Experiments have shown a best result reaching an unweighted average recall of 79.2 % for the binary classification task in a rigorous subject-independent scenario. To the best of our knowledge, this is the first time to analyse the sounds that cannot be heard by human beings for the detection of mASD mice. The novel findings can be significant to motivate future works with according means on studying animal models of human patients.

Fast and accurate annotation of acoustic signals with deep neural networks

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, and fast.

We here introduce DeepAudioSegmenter (DAS), a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DAS using acoustic signals with diverse characteristics from insects, birds, and mammals. DAS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations. The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DAS annotates song with high throughput and low latency for experimental interventions in realtime. Overall, DAS is a universal, versatile, and accessible tool for annotating acoustic communication signals.

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

The reproducibility crisis (or replication crisis) in biomedical research is a particularly existential and under-addressed issue in the field of behavioral neuroscience, where, in spite of efforts to standardize testing and assay protocols, several known and unknown sources of confounding environmental factors add to variance. Human interference is a major contributor to variability both within and across laboratories, as well as novelty-induced anxiety. Attempts to reduce human interference and to measure more "natural" behaviors in subjects has led to the development of automated home-cage monitoring systems. These systems enable prolonged and longitudinal recordings, and provide large continuous measures of spontaneous behavior that can be analyzed across multiple time scales. In this review, a diverse team of neuroscientists and product developers share their experiences using such an automated monitoring system that combines Noldus PhenoTyper® home-cages and the video-based tracking software, EthoVision® XT, to extract digital biomarkers of motor, emotional, social and cognitive behavior. After presenting our working definition of a "home-cage", we compare home-cage testing with more conventional out-of-cage tests (e.g., the open field) and outline the various advantages of the former, including opportunities for within-subject analyses and assessments of circadian and ultradian activity. Next, we address technical issues pertaining to the acquisition of behavioral data, such as the fine-tuning of the tracking software and the potential for integration with biotelemetry and optogenetics. Finally, we provide guidance on which behavioral measures to emphasize, how to filter, segment, and analyze behavior, and how to use analysis scripts. We summarize how the PhenoTyper has applications to study neuropharmacology as well as animal models of neurodegenerative and neuropsychiatric illness. Looking forward, we examine current challenges and the impact of new developments. Examples include the automated recognition of specific behaviors, unambiguous tracking of individuals in a social context, the development of more animal-centered measures of behavior and ways of dealing with large datasets. Together, we advocate that by embracing standardized home-cage monitoring platforms like the PhenoTyper, we are poised to directly assess issues pertaining to reproducibility, and more importantly, measure features of rodent behavior under more ethologically relevant scenarios.

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Increases in the scale and complexity of behavioral data pose an increasing challenge for data analysis. A common strategy involves replacing entire behaviors with small numbers of handpicked, domain-specific features, but this approach suffers from several crucial limitations. For example, handpicked features may miss important dimensions of variability, and correlations among them complicate statistical testing. Here, by contrast, we apply the variational autoencoder (VAE), an unsupervised learning method, to learn features directly from data and quantify the vocal behavior of two model species: the laboratory mouse and the zebra finch. The VAE converges on a parsimonious representation that outperforms handpicked features on a variety of common analysis tasks, enables the measurement of moment-by-moment vocal variability on the timescale of tens of milliseconds in the zebra finch, provides strong evidence that mouse ultrasonic vocalizations do not cluster as is commonly believed, and captures the similarity of tutor and pupil birdsong with qualitatively higher fidelity than previous approaches. In all, we demonstrate the utility of modern unsupervised learning approaches to the quantification of complex and high-dimensional vocal behavior.

Automatic classification of mice vocalizations using Machine Learning techniques and Convolutional Neural Networks

Ultrasonic vocalizations (USVs) analysis is a well-recognized tool to investigate animal communication. It can be used for behavioral phenotyping of murine models of different disorders. The USVs are usually recorded with a microphone sensitive to ultrasound frequencies and they are analyzed by specific software. Different calls typologies exist, and each ultrasonic call can be manually classified, but the qualitative analysis is highly time-consuming. Considering this framework, in this work we proposed and evaluated a set of supervised learning methods for automatic USVs classification. This could represent a sustainable procedure to deeply analyze the ultrasonic communication, other than a standardized analysis. We used manually built datasets obtained by segmenting the USVs audio tracks analyzed with the Avisoft software, and then by labelling each of them into 10 representative classes. For the automatic classification task, we designed a Convolutional Neural Network that was trained receiving as input the spectrogram images associated to the segmented audio files. In addition, we also tested some other supervised learning algorithms, such as Support Vector Machine, Random Forest and Multilayer Perceptrons, exploiting informative numerical features extracted from the spectrograms. The performance showed how considering the whole time/frequency information of the spectrogram leads to significantly higher performance than considering a subset of numerical features. In the authors’ opinion, the experimental results may represent a valuable benchmark for future work in this research field.