Active vision during prey capture in wild marmoset monkeys

String-pulling by the common marmoset

Coordinated hand movements used to grasp and manipulate objects are crucial for many daily activities, such as tying shoelaces or opening jars. Recently, the string-pulling task, which involves cyclically reaching, grasping, and pulling a string, has been used to study coordinated hand movements in rodents and humans. Here, we characterize how adult common marmosets perform the string-pulling task and describe changes in performance across the lifespan. Marmosets (n = 15, 7 females) performed a string-pulling task for a food reward using an instrumented apparatus attached to their home-cage. Movement kinematics were acquired using markerless video tracking and we assessed individual hand movements and bimanual coordination using standard metrics. Marmosets oriented their gaze toward the string above their hands and readily performed the task regardless of sex or age. The task required little training and animals routinely engaged in multiple pulling trials per session, despite not being under water or food control. All marmosets showed consistent pulling speed and similar hand movements regardless of age. Adult marmosets exhibited a clear hand effect, performing straighter and faster movements with their right hand despite showing idiosyncratic hand preference according to a traditional food retrieval assay. Hand effects were also evident for younger animals but seemed attenuated in the older animals. In terms of bimanual coordination, all adult marmosets demonstrated alternating movement pattern for vertical hand positions. Two younger and two older marmosets exhibited idiosyncratic coordination patterns even after substantial experience. In general, younger and older animals exhibited higher variability in bimanual coordination than adults.NEW & NOTEWORTHY Bimanual coordination is crucial for daily activities. In this study, we characterized how common marmosets performed the string-pulling task without extensive training, regardless of sex or age, and naturally exhibited a cyclical alternating pattern of hand movements. Although the overall behavior was similar across ages, younger and older marmosets demonstrated higher variability in bimanual coordination. These results establish the string-pulling task as a reliable tool for studying bimanual coordination and its underlying neural substrates.

Active vision in freely moving marmosets using head-mounted eye tracking

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing-e.g., eyes, head, posture, movement, etc.-on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye-tracking system that performs Chair-free Eye-Recording using Backpack mounted micROcontrollers (CEREBRO) for small mammals, such as marmoset monkeys. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in vestibularocular reflex gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

PreyTouch: a touchscreen-based closed-loop system for studying predator-prey interactions

The ability to catch prey is crucial for survival and reproduction and is subject to strong natural selection across predators. Prey capture demands the orchestrated activation of multiple brain regions and the interplay between sensory processing, decision-making, and motor execution. These factors, together with the ubiquity of prey capture across species makes it appealing for comparative studies across neuroscience and ecology. However, despite recent technological advances, experimental approaches for studying natural behaviors such as prey catch are lagging behind. To bridge this gap, we created PreyTouch-a novel approach for performing prey capture experiments that incorporate flexible prey control, accurate monitoring of predator touchscreen strikes and automated rewarding. Further, its real-time processing enables coupling predator movement and prey dynamics for studying predator-prey interactions. Finally, PreyTouch is optimized for automated long-term experiments featuring a web UI for remote control and monitoring. We successfully validated PreyTouch by conducting long-term prey capture experiments on the lizard Pogona vitticeps. This revealed the existence of prey preferences, complex prey attack patterns, and fast learning of prey dynamics. PreyTouch's unique features and the importance of studying prey capture behavior make it a valuable platform for connecting natural behavior with cognitive studies across various species and disciplines.

Active vision in freely moving marmosets using head-mounted eye tracking

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing - e.g. eyes, head, posture, movement, etc. - on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye tracking system called CEREBRO for small mammals, such as marmoset monkeys. Our system performs Chair-free Eye-Recording using Backpack mounted micROcontrollers. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely-moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely-moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in VOR gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

Posture-dependent modulation of marmoset cortical motor maps detected via rapid multichannel epidural stimulation

Recent neuroimaging and electrophysiological studies have suggested substantial short-term plasticity in the topographic maps of the primary motor cortex (M1). However, previous methods lack the temporal resolution to detect rapid modulation of these maps, particularly in naturalistic conditions. To address this limitation, we previously developed a rapid stimulation mapping procedure with implanted cortical surface electrodes. In this study, employing our previously established procedure, we examined rapid topographical changes in forelimb M1 motor maps in three awake male marmoset monkeys. The results revealed that although the hotspot (the location in M1 that elicited a forelimb muscle twitch with the lowest stimulus intensity) remained constant across postures, the stimulus intensity required to elicit the forelimb muscle twitch in the perihotspot region and the size of motor representations were posture-dependent. Hindlimb posture was particularly effective in inducing these modulations. The angle of the body axis relative to the gravitational vertical line did not alter the motor maps. These results provide a proof of concept that a rapid stimulation mapping system with chronically implanted cortical electrodes can capture the dynamic regulation of forelimb motor maps in natural conditions. Moreover, they suggest that posture is a crucial variable to be controlled in future studies of motor control and cortical plasticity. Further exploration is warranted into the neural mechanisms regulating forelimb muscle representations in M1 by the hindlimb sensorimotor state.

Dynamic modulation of social gaze by sex and familiarity in marmoset dyads

Social communication relies on the ability to perceive and interpret the direction of others' attention, and is commonly conveyed through head orientation and gaze direction in humans and nonhuman primates. However, traditional social gaze experiments in nonhuman primates require restraining head movements, significantly limiting their natural behavioral repertoire. Here, we developed a novel framework for accurately tracking facial features and three-dimensional head gaze orientations of multiple freely moving common marmosets (Callithrix jacchus). By combining deep learning-based computer vision tools with triangulation algorithms, we were able to track the facial features of marmoset dyads within an arena. This method effectively generates dynamic 3D geometrical facial frames while overcoming common challenges like occlusion. To detect the head gaze direction, we constructed a virtual cone, oriented perpendicular to the facial frame. Using this pipeline, we quantified different types of interactive social gaze events, including partner-directed gaze and joint gaze to a shared spatial location. We observed clear effects of sex and familiarity on both interpersonal distance and gaze dynamics in marmoset dyads. Unfamiliar pairs exhibited more stereotyped patterns of arena occupancy, more sustained levels of social gaze across social distance, and increased social gaze monitoring. On the other hand, familiar pairs exhibited higher levels of joint gazes. Moreover, males displayed significantly elevated levels of gazes toward females' faces and the surrounding regions, irrespective of familiarity. Our study reveals the importance of two key social factors in driving the gaze behaviors of a prosocial primate species and lays the groundwork for a rigorous quantification of primate behaviors in naturalistic settings.

Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs

Nonhuman primates (NHPs) exhibit complex and diverse behavior that typifies advanced cognitive function and social communication, but quantitative and systematical measure of this natural nonverbal processing has been a technical challenge. Specifically, a method is required to automatically segment time series of behavior into elemental motion motifs, much like finding meaningful words in character strings. Here, we propose a solution called SyntacticMotionParser (SMP), a general-purpose unsupervised behavior parsing algorithm using a nonparametric Bayesian model. Using three-dimensional posture-tracking data from NHPs, SMP automatically outputs an optimized sequence of latent motion motifs classified into the most likely number of states. When applied to behavioral datasets from common marmosets and rhesus monkeys, SMP outperformed conventional posture-clustering models and detected a set of behavioral ethograms from publicly available data. SMP also quantified and visualized the behavioral effects of chemogenetic neural manipulations. SMP thus has the potential to dramatically improve our understanding of natural NHP behavior in a variety of contexts.

An ethologically motivated neurobiology of primate visually-guided reach-to-grasp behavior

The precision of primate visually guided reaching likely evolved to meet the many challenges faced by living in arboreal environments, yet much of what we know about the underlying primate brain organization derives from a set of highly constrained experimental paradigms. Here we review the role of vision to guide natural reach-to-grasp movements in marmoset monkey prey capture to illustrate the breadth and diversity of these behaviors in ethological contexts, the fast predictive nature of these movements [1,2], and the advantages of this particular primate model to investigate the underlying neural mechanisms in more naturalistic contexts [3]. In addition to their amenability to freely-moving neural recording methods for investigating the neural basis of dynamic ethological behaviors [4,5], marmosets have a smooth neocortical surface that facilitates imaging and array recordings [6,7] in all areas in the primate fronto-parietal network [8,9]. Together, this model organism offers novel opportunities to study the real-world interplay between primate vision and reach-to-grasp dynamics using ethologically motivated neuroscientific experimental designs.

Comparing eye-hand coordination between controller-mediated virtual reality, and a real-world object interaction task

Virtual reality (VR) technology has advanced significantly in recent years, with many potential applications. However, it is unclear how well VR simulations mimic real-world experiences, particularly in terms of eye-hand coordination. This study compares eye-hand coordination from a previously validated real-world object interaction task to the same task re-created in controller-mediated VR. We recorded eye and body movements and segmented participants' gaze data using the movement data. In the real-world condition, participants wore a head-mounted eye tracker and motion capture markers and moved a pasta box into and out of a set of shelves. In the VR condition, participants wore a VR headset and moved a virtual box using handheld controllers. Unsurprisingly, VR participants took longer to complete the task. Before picking up or dropping off the box, participants in the real world visually fixated the box about half a second before their hand arrived at the area of action. This 500-ms minimum fixation time before the hand arrived was preserved in VR. Real-world participants disengaged their eyes from the box almost immediately after their hand initiated or terminated the interaction, but VR participants stayed fixated on the box for much longer after it was picked up or dropped off. We speculate that the limited haptic feedback during object interactions in VR forces users to maintain visual fixation on objects longer than in the real world, altering eye-hand coordination. These findings suggest that current VR technology does not replicate real-world experience in terms of eye-hand coordination.

Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors

Deciding whether to forego immediate rewards or explore new opportunities is a key component of flexible behavior and is critical for the survival of the species. Although previous studies have shown that different cortical and subcortical areas, including the amygdala and ventral striatum (VS), are implicated in representing the immediate (exploitative) and future (explorative) value of choices, the effect of the motor system used to make choices has not been examined. Here, we tested male rhesus macaques with amygdala or VS lesions on two versions of a three-arm bandit task where choices were registered with either a saccade or an arm movement. In both tasks we presented the monkeys with explore-exploit tradeoffs by periodically replacing familiar options with novel options that had unknown reward probabilities. We found that monkeys explored more with saccades but showed better learning with arm movements. VS lesions caused the monkeys to be more explorative with arm movements and less explorative with saccades, although this may have been due to an overall decrease in performance. VS lesions affected the monkeys' ability to learn novel stimulus-reward associations in both tasks, while after amygdala lesions this effect was stronger when choices were made with saccades. Further, on average, VS and amygdala lesions reduced the monkeys' ability to choose better options only when choices were made with a saccade. These results show that learning reward value associations to manage explore-exploit behaviors is motor system dependent and they further define the contributions of amygdala and VS to reinforcement learning.

The neurobiology of vocal communication in marmosets

An increasingly popular animal model for studying the neural basis of social behavior, cognition, and communication is the common marmoset (Callithrix jacchus). Interest in this New World primate across neuroscience is now being driven by their proclivity for prosociality across their repertoire, high volubility, and rapid development, as well as their amenability to naturalistic testing paradigms and freely moving neural recording and imaging technologies. The complement of these characteristics set marmosets up to be a powerful model of the primate social brain in the years to come. Here, we focus on vocal communication because it is the area that has both made the most progress and illustrates the prodigious potential of this species. We review the current state of the field with a focus on the various brain areas and networks involved in vocal perception and production, comparing the findings from marmosets to other animals, including humans.

Ocular following responses of the marmoset monkey are dependent on postsaccadic delay, spatiotemporal frequency, and saccade direction

Ocular following is a short-latency, reflexive eye movement that tracks wide-field visual motion. It has been studied extensively in humans and macaques and is an appealing behavior for studying sensory-motor transformations in the brain because of its rapidity and rigidity. We explored ocular following in the marmoset, an emerging model in neuroscience because their lissencephalic brain allows direct access to most cortical areas for imaging and electrophysiological recordings. In three experiments, we tested ocular following responses in three adult marmosets. First, we varied the delay between saccade end and stimulus motion onset, from 10 to 300 ms. As in other species, tracking had shorter onset latencies and higher eye speeds with shorter postsaccadic delays. Second, using sine-wave grating stimuli, we explored the dependence of eye speed on spatiotemporal frequency. The highest eye speed was evoked at ∼16 Hz and ∼0.16 cycles per degree (cpd); however, the highest gain was elicited at ∼1.6 Hz and ∼1.2 cpd. The highest eye speed for each spatial frequency was observed at a different temporal frequency, but this interdependence was not consistent with complete speed tuning of the ocular following response. Finally, we found the highest eye speeds when saccade and stimulus motion directions were identical, although latencies were unaffected by direction difference. Our results showed qualitatively similar ocular following in marmosets, humans, and macaques, despite over an order of magnitude variation in body and eye size across species. This characterization will help future studies examining the neural basis of sensory-motor transformations.NEW & NOTEWORTHY Previous ocular following studies focused on humans and macaques. We examined the properties of ocular following responses in marmosets in three experiments, in which postsaccadic delay, spatial-temporal frequency of stimuli, and congruence of saccade and motion directions were manipulated. We have demonstrated short-latency ocular following in marmosets and discuss the similarities across three species that vary markedly in eye and head size. Our findings will help future studies examining the neural mechanism of sensory-motor transformations.

Fast prediction in marmoset reach-to-grasp movements for dynamic prey

Primates have evolved sophisticated, visually guided reaching behaviors for interacting with dynamic objects, such as insects, during foraging.1,2,3,4,5 Reaching control in dynamic natural conditions requires active prediction of the target's future position to compensate for visuo-motor processing delays and to enhance online movement adjustments.6,7,8,9,10,11,12 Past reaching research in non-human primates mainly focused on seated subjects engaged in repeated ballistic arm movements to either stationary targets or targets that instantaneously change position during the movement.13,14,15,16,17 However, those approaches impose task constraints that limit the natural dynamics of reaching. A recent field study in marmoset monkeys highlights predictive aspects of visually guided reaching during insect prey capture among wild marmoset monkeys.5 To examine the complementary dynamics of similar natural behavior within a laboratory context, we developed an ecologically motivated, unrestrained reach-to-grasp task involving live crickets. We used multiple high-speed video cameras to capture the movements of common marmosets (Callithrix jacchus) and crickets stereoscopically and applied machine vision algorithms for marker-free object and hand tracking. Contrary to estimates under traditional constrained reaching paradigms, we find that reaching for dynamic targets can operate at incredibly short visuo-motor delays around 80 ms, rivaling the speeds that are typical of the oculomotor systems during closed-loop visual pursuit.18 Multivariate linear regression modeling of the kinematic relationships between the hand and cricket velocity revealed that predictions of the expected future location can compensate for visuo-motor delays during fast reaching. These results suggest a critical role of visual prediction facilitating online movement adjustments for dynamic prey.

Ultra-high field fMRI identifies an action-observation network in the common marmoset

The observation of others' actions activates a network of temporal, parietal and premotor/prefrontal areas in macaque monkeys and humans. This action-observation network (AON) has been shown to play important roles in social action monitoring, learning by imitation, and social cognition in both species. It is unclear whether a similar network exists in New-World primates, which separated from Old-Word primates ~35 million years ago. Here we used ultra-high field fMRI at 9.4 T in awake common marmosets (Callithrix jacchus) while they watched videos depicting goal-directed (grasping food) or non-goal-directed actions. The observation of goal-directed actions activates a temporo-parieto-frontal network, including areas 6 and 45 in premotor/prefrontal cortices, areas PGa-IPa, FST and TE in occipito-temporal region and areas V6A, MIP, LIP and PG in the occipito-parietal cortex. These results show overlap with the humans and macaques' AON, demonstrating the existence of an evolutionarily conserved network that likely predates the separation of Old and New-World primates.

Are there disciplinary boundaries in the comparative study of primate cognition?

A view continues to gain momentum that regards investigation of the cognition of great apes in captive settings as affording us a model for human cognitive evolution. Researchers from disciplines such as comparative psychology, anthropology, and even archaeology, seem eager to put their theories to the test by using great apes as their chosen experimental model. Questions addressed currently by comparative psychologists have long been the object of attention by neurophysiologists, psychobiologists and neuroscientists, who, however, often use rodents and monkeys as the species of choice. Whereas comparative psychology has been influenced greatly by ethology, much neuroscience has developed against a background of physiology and medicine. This separation of the intellectual contexts wherein they have arisen and flourished has impeded the development of fluid interaction between comparative psychologists and researchers in the other disciplines. We feel that it would be beneficial for comparative psychologists and neuroscientists to combine research endeavours far more often, in order to address common questions of interest related to cognition. We regard interdisciplinary cross-pollination to be particularly desirable, even if many comparative psychologists lack deep expertise about the workings of the brain, and even if many neuroscientists lack expert knowledge about the behaviour of different species. Furthermore, we believe that anthropology, archaeology, human evolutionary studies, and related disciplines, may well provide us with significant contextual knowledge about the physical and temporal background to the evolution in humans of specific cognitive skills. To that end, we urge researchers to dismantle methodological, conceptual and historical disciplinary boundaries, in order to strengthen cross-disciplinary cooperation in order to broaden and deepen our insights into the cognition of nonhuman and human primates.

Computational cross-species views of the hippocampal formation

The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.

A binocular perception deficit characterizes prey pursuit in developing mice

Integration of binocular information at the cellular level has long been studied in the mouse model to uncover the fundamental developmental mechanisms underlying mammalian vision. However, we lack an understanding of the corresponding ontogeny of visual behavior in mice that relies on binocular integration. To address this major outstanding question, we quantified the natural visually guided behavior of postnatal day 21 (P21) and adult mice using a live prey capture assay and a computerized-spontaneous perception of objects task (C-SPOT). We found a robust and specific binocular visual field processing deficit in P21 mice as compared to adults that corresponded to a selective increase in c-Fos expression in the anterior superior colliculus (SC) of the juveniles after C-SPOT. These data link a specific binocular perception deficit in developing mice to activity changes in the SC.

An Attention-Refined Light-Weight High-Resolution Network for Macaque Monkey Pose Estimation

Macaque monkey is a rare substitute which plays an important role for human beings in relation to psychological and spiritual science research. It is essential for these studies to accurately estimate the pose information of macaque monkeys. Many large-scale models have achieved state-of-the-art results in pose macaque estimation. However, it is difficult to deploy when computing resources are limited. Combining the structure of high-resolution network and the design principle of light-weight network, we propose the attention-refined light-weight high-resolution network for macaque monkey pose estimation (HR-MPE). The multi-branch parallel structure is adopted to maintain high-resolution representation throughout the process. Moreover, a novel basic block is designed by a powerful transformer structure and polarized self-attention, where there is a simple structure and fewer parameters. Two attention refined blocks are added at the end of the parallel structure, which are composed of light-weight asymmetric convolutions and a triplet attention with almost no parameter, obtaining richer representation information. An unbiased data processing method is also utilized to obtain an accurate flipping result. The experiment is conducted on a macaque dataset containing more than 13,000 pictures. Our network has reached a 77.0 AP score, surpassing HRFormer with fewer parameters by 1.8 AP.