Fast prediction in marmoset reach-to-grasp movements for dynamic prey

String-pulling by the common marmoset

Coordinated hand movements used to grasp and manipulate objects are crucial for many daily activities, such as tying shoelaces or opening jars. Recently, the string-pulling task, which involves cyclically reaching, grasping, and pulling a string, has been used to study coordinated hand movements in rodents and humans. Here, we characterize how adult common marmosets perform the string-pulling task and describe changes in performance across the lifespan. Marmosets (n = 15, 7 females) performed a string-pulling task for a food reward using an instrumented apparatus attached to their home-cage. Movement kinematics were acquired using markerless video tracking and we assessed individual hand movements and bimanual coordination using standard metrics. Marmosets oriented their gaze toward the string above their hands and readily performed the task regardless of sex or age. The task required little training and animals routinely engaged in multiple pulling trials per session, despite not being under water or food control. All marmosets showed consistent pulling speed and similar hand movements regardless of age. Adult marmosets exhibited a clear hand effect, performing straighter and faster movements with their right hand despite showing idiosyncratic hand preference according to a traditional food retrieval assay. Hand effects were also evident for younger animals but seemed attenuated in the older animals. In terms of bimanual coordination, all adult marmosets demonstrated alternating movement pattern for vertical hand positions. Two younger and two older marmosets exhibited idiosyncratic coordination patterns even after substantial experience. In general, younger and older animals exhibited higher variability in bimanual coordination than adults.NEW & NOTEWORTHY Bimanual coordination is crucial for daily activities. In this study, we characterized how common marmosets performed the string-pulling task without extensive training, regardless of sex or age, and naturally exhibited a cyclical alternating pattern of hand movements. Although the overall behavior was similar across ages, younger and older marmosets demonstrated higher variability in bimanual coordination. These results establish the string-pulling task as a reliable tool for studying bimanual coordination and its underlying neural substrates.

Reduced latency in manual interception with anticipatory smooth eye movements

Delays in visuomotor processing can cause the image of a moving object to fall outside the limited range of central vision, resulting in a blurred perception. To overcome these delays, anticipatory smooth eye movements (ASEMs) occur in the expected direction of future object motion before the object begins to move. This study demonstrated the functional benefits of ASEMs in rapid visually guided behaviors, highlighting their role beyond merely compensating for visuomotor delays. By experimentally facilitating ASEMs independent of participants' predictions regarding future object motion, we showed that ASEMs themselves expedite the initiation of interception movements. Our results revealed a quantitative relationship between ASEM velocity and interception latency on a single-trial level, which was more pronounced in participants employing predictive control compared to those relying on reactive control. These findings suggest that ASEMs enhance the feedforward control of interception movements by providing extraretinal signals rather than retinal signals.

Active vision in freely moving marmosets using head-mounted eye tracking

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing-e.g., eyes, head, posture, movement, etc.-on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye-tracking system that performs Chair-free Eye-Recording using Backpack mounted micROcontrollers (CEREBRO) for small mammals, such as marmoset monkeys. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in vestibularocular reflex gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

AI-assisted 3D analysis of grasping and reaching behavior of squirrel monkeys during recovery from cervical spinal cord injury

We have previously demonstrated that machine learning-based video analysis, conducted via DeepLabCut, is more sensitive for detecting subtle deficits in hand grasping behavior than traditional end-point performance assessments. This superiority was observed in a nonhuman primate (NHP) model of cervical spinal cord injury, specifically a dorsal column lesion (DCL). The current study aims to further characterize the kinematic aspects of the deficits in hand reaching, grasping, and retrieving behavior from a 3D perspective following a DCL. Squirrel monkeys were trained to retrieve sugar pellets from eight wells, which were located either on a flat plate or a raised tube with varying well depths. This setup was designed to require coordinated finger movements during the task. Immediately after the DCL, the animals exhibited measurable behavioral deficits. These were characterized by significant increases in grasping speed squared and trial completion time, markedly widened movement trajectories of individual fingers, and abnormalities in inter-finger distance and orientation. Increased task difficulty was associated with more pronounced behavioral deficits. By three months post-DCL, video-based measurements indicated no significant recovery, even though global end-point performance had returned to baseline levels. Our findings demonstrate that deprivation of tactile information results in impaired dexterous hand behavior involving coordinated finger movements, and the impairment is sustained for 20 weeks. This spinal cord injury (SCI) model, along with DeepLapCut analysis, provides a valuable platform for separately evaluating sensory and motor functions and their contributions to dexterous hand behavior and may be used for evaluating therapeutic interventions using more sensitive behavioral outcome readouts.

Identification of movie encoding neurons enables movie recognition AI

Natural visual scenes are dominated by spatiotemporal image dynamics, but how the visual system integrates "movie" information over time is unclear. We characterized optic tectal neuronal receptive fields using sparse noise stimuli and reverse correlation analysis. Neurons recognized movies of ~200-600 ms durations with defined start and stop stimuli. Movie durations from start to stop responses were tuned by sensory experience though a hierarchical algorithm. Neurons encoded families of image sequences following trigonometric functions. Spike sequence and information flow suggest that repetitive circuit motifs underlie movie detection. Principles of frog topographic retinotectal plasticity and cortical simple cells are employed in machine learning networks for static image recognition, suggesting that discoveries of principles of movie encoding in the brain, such as how image sequences and duration are encoded, may benefit movie recognition technology. We built and trained a machine learning network that mimicked neural principles of visual system movie encoders. The network, named MovieNet, outperformed current machine learning image recognition networks in classifying natural movie scenes, while reducing data size and steps to complete the classification task. This study reveals how movie sequences and time are encoded in the brain and demonstrates that brain-based movie processing principles enable efficient machine learning.

Active vision in freely moving marmosets using head-mounted eye tracking

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing - e.g. eyes, head, posture, movement, etc. - on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye tracking system called CEREBRO for small mammals, such as marmoset monkeys. Our system performs Chair-free Eye-Recording using Backpack mounted micROcontrollers. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely-moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely-moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in VOR gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

Representing the dynamics of natural marmoset vocal behaviors in frontal cortex

Here, we tested the respective contributions of primate premotor and prefrontal cortex to support vocal behavior. We applied a model-based generalized linear model (GLM) analysis that better accounts for the inherent variance in natural, continuous behaviors to characterize the activity of neurons throughout the frontal cortex as freely moving marmosets engaged in conversational exchanges. While analyses revealed functional clusters of neural activity related to the different processes involved in the vocal behavior, these clusters did not map to subfields of prefrontal or premotor cortex, as has been observed in more conventional task-based paradigms. Our results suggest a distributed functional organization for the myriad neural mechanisms underlying natural social interactions and have implications for our concepts of the role that frontal cortex plays in governing ethological behaviors in primates.

Deciphering social traits and pathophysiological conditions from natural behaviors in common marmosets

Nonhuman primates (NHPs) are indispensable animal models by virtue of the continuity of behavioral repertoires across primates, including humans. However, behavioral assessment at the laboratory level has so far been limited. Employing the application of three-dimensional (3D) pose estimation and the optimal integration of subsequent analytic methodologies, we demonstrate that our artificial intelligence (AI)-based approach has successfully deciphered the ethological, cognitive, and pathological traits of common marmosets from their natural behaviors. By applying multiple deep neural networks trained with large-scale datasets, we established an evaluation system that could reconstruct and estimate the 3D poses of the marmosets, a small NHP that is suitable for analyzing complex natural behaviors in laboratory setups. We further developed downstream analytic methodologies to quantify a variety of behavioral parameters beyond motion kinematics. We revealed the distinct parental roles of male and female marmosets through automated detections of food-sharing behaviors using a spatial-temporal filter on 3D poses. Employing a recurrent neural network to analyze 3D pose time series data during social interactions, we additionally discovered that marmosets adjusted their behaviors based on others' internal state, which is not directly observable but can be inferred from the sequence of others' actions. Moreover, a fully unsupervised approach enabled us to detect progressively appearing symptomatic behaviors over a year in a Parkinson's disease model. The high-throughput and versatile nature of an AI-driven approach to analyze natural behaviors will open a new avenue for neuroscience research dealing with big-data analyses of social and pathophysiological behaviors in NHPs.

A dynamic scale-mixture model of motion in natural scenes

Some of the most important tasks of visual and motor systems involve estimating the motion of objects and tracking them over time. Such systems evolved to meet the behavioral needs of the organism in its natural environment, and may therefore be adapted to the statistics of motion it is likely to encounter. By tracking the movement of individual points in movies of natural scenes, we begin to identify common properties of natural motion across scenes. As expected, objects in natural scenes move in a persistent fashion, with velocity correlations lasting hundreds of milliseconds. More subtly, but crucially, we find that the observed velocity distributions are heavy-tailed and can be modeled as a Gaussian scale-mixture. Extending this model to the time domain leads to a dynamic scale-mixture model, consisting of a Gaussian process multiplied by a positive scalar quantity with its own independent dynamics. Dynamic scaling of velocity arises naturally as a consequence of changes in object distance from the observer, and may approximate the effects of changes in other parameters governing the motion in a given scene. This modeling and estimation framework has implications for the neurobiology of sensory and motor systems, which need to cope with these fluctuations in scale in order to represent motion efficiently and drive fast and accurate tracking behavior.

An ethologically motivated neurobiology of primate visually-guided reach-to-grasp behavior

The precision of primate visually guided reaching likely evolved to meet the many challenges faced by living in arboreal environments, yet much of what we know about the underlying primate brain organization derives from a set of highly constrained experimental paradigms. Here we review the role of vision to guide natural reach-to-grasp movements in marmoset monkey prey capture to illustrate the breadth and diversity of these behaviors in ethological contexts, the fast predictive nature of these movements [1,2], and the advantages of this particular primate model to investigate the underlying neural mechanisms in more naturalistic contexts [3]. In addition to their amenability to freely-moving neural recording methods for investigating the neural basis of dynamic ethological behaviors [4,5], marmosets have a smooth neocortical surface that facilitates imaging and array recordings [6,7] in all areas in the primate fronto-parietal network [8,9]. Together, this model organism offers novel opportunities to study the real-world interplay between primate vision and reach-to-grasp dynamics using ethologically motivated neuroscientific experimental designs.

Arbovirus impact on mosquito behavior: the jury is still out

Parasites can manipulate host behavior to enhance transmission, but our understanding of arbovirus-induced changes in mosquito behavior is limited. Here, we explore current knowledge on such behavioral alterations in mosquito vectors, focusing on host-seeking and blood-feeding behaviors. Reviewing studies on dengue, Zika, La Crosse, Sindbis, and West Nile viruses in Aedes or Culex mosquitoes reveals subtle yet potentially significant effects. However, assay heterogeneity and limited sample sizes challenge definitive conclusions. To enhance robustness, we propose using deep-learning tools for automated behavior quantification and stress the need for standardized assays. Additionally, conducting longitudinal studies across the extrinsic incubation period and integrating diverse traits into modeling frameworks are crucial for understanding the nuanced implications of arbovirus-induced behavioral changes for virus transmission dynamics.

Representing the dynamics of natural marmoset vocal behaviors in frontal cortex

Here we tested the respective contributions of primate premotor and prefrontal cortex to support vocal behavior. We applied a model-based GLM analysis that better accounts for the inherent variance in natural, continuous behaviors to characterize the activity of neurons throughout frontal cortex as freely-moving marmosets engaged in conversational exchanges. While analyses revealed functional clusters of neural activity related to the different processes involved in the vocal behavior, these clusters did not map to subfields of prefrontal or premotor cortex, as has been observed in more conventional task-based paradigms. Our results suggest a distributed functional organization for the myriad neural mechanisms underlying natural social interactions and has implications for our concepts of the role that frontal cortex plays in governing ethological behaviors in primates.