Contrasting action and posture coding with hierarchical deep neural network models of proprioception

Acquiring musculoskeletal skills with curriculum-based reinforcement learning

Efficient musculoskeletal simulators and powerful learning algorithms provide computational tools to tackle the grand challenge of understanding biological motor control. Our winning solution for the inaugural NeurIPS MyoChallenge leverages an approach mirroring human skill learning. Using a novel curriculum learning approach, we trained a recurrent neural network to control a realistic model of the human hand with 39 muscles to rotate two Baoding balls in the palm of the hand. In agreement with data from human subjects, the policy uncovers a small number of kinematic synergies, even though it is not explicitly biased toward low-dimensional solutions. However, selectively inactivating parts of the control signal, we found that more dimensions contribute to the task performance than suggested by traditional synergy analysis. Overall, our work illustrates the emerging possibilities at the interface of musculoskeletal physics engines, reinforcement learning, and neuroscience to advance our understanding of biological motor control.

Decoding the brain: From neural representations to mechanistic models

A central principle in neuroscience is that neurons within the brain act in concert to produce perception, cognition, and adaptive behavior. Neurons are organized into specialized brain areas, dedicated to different functions to varying extents, and their function relies on distributed circuits to continuously encode relevant environmental and body-state features, enabling other areas to decode (interpret) these representations for computing meaningful decisions and executing precise movements. Thus, the distributed brain can be thought of as a series of computations that act to encode and decode information. In this perspective, we detail important concepts of neural encoding and decoding and highlight the mathematical tools used to measure them, including deep learning methods. We provide case studies where decoding concepts enable foundational and translational science in motor, visual, and language processing.

A computational study of how an α- to γ-motoneurone collateral can mitigate velocity-dependent stretch reflexes during voluntary movement

The primary motor cortex does not uniquely or directly produce alpha motoneurone (α-MN) drive to muscles during voluntary movement. Rather, α-MN drive emerges from the synthesis and competition among excitatory and inhibitory inputs from multiple descending tracts, spinal interneurons, sensory inputs, and proprioceptive afferents. One such fundamental input is velocity-dependent stretch reflexes in lengthening muscles, which should be inhibited to enable voluntary movement. It remains an open question, however, the extent to which unmodulated stretch reflexes disrupt voluntary movement, and whether and how they are inhibited in limbs with numerous multiarticular muscles. We used a computational model of a Rhesus Macaque arm to simulate movements with feedforward α-MN commands only, and with added velocity-dependent stretch reflex feedback. We found that velocity-dependent stretch reflex caused movement-specific, typically large and variable disruptions to arm movements. These disruptions were greatly reduced when modulating velocity-dependent stretch reflex feedback (i) as per the commonly proposed (but yet to be clarified) idealized alpha-gamma (α-γ) coactivation or (ii) an alternative α-MN collateral projection to homonymous γ-MNs. We conclude that such α-MN collaterals are a physiologically tenable propriospinal circuit in the mammalian fusimotor system. These collaterals could still collaborate with α-γ coactivation, and the few skeletofusimotor fibers (β-MNs) in mammals, to create a flexible fusimotor ecosystem to enable voluntary movement. By locally and automatically regulating the highly nonlinear neuro-musculo-skeletal mechanics of the limb, these collaterals could be a critical low-level enabler of learning, adaptation, and performance via higher-level brainstem, cerebellar, and cortical mechanisms.

An alpha- to gamma-motoneurone collateral can mitigate velocity-dependent stretch reflexes during voluntary movement: A computational study

The primary motor cortex does not uniquely or directly produce alpha motoneurone (α-MN) drive to muscles during voluntary movement. Rather, α-MN drive emerges from the synthesis and competition among excitatory and inhibitory inputs from multiple descending tracts, spinal interneurons, sensory inputs, and proprioceptive afferents. One such fundamental input is velocity-dependent stretch reflexes in lengthening muscles, which should be inhibited to enable voluntary movement. It remains an open question, however, the extent to which unmodulated stretch reflexes disrupt voluntary movement, and whether and how they are inhibited in limbs with numerous multi-articular muscles. We used a computational model of a Rhesus Macaque arm to simulate movements with feedforward α-MN commands only, and with added velocity-dependent stretch reflex feedback. We found that velocity-dependent stretch reflex caused movement-specific, typically large and variable disruptions to arm movements. These disruptions were greatly reduced when modulating velocity-dependent stretch reflex feedback (i) as per the commonly proposed (but yet to be clarified) idealized alpha-gamma (α-γ) co-activation or (ii) an alternative α-MN collateral projection to homonymous γ-MNs. We conclude that such α-MN collaterals are a physiologically tenable, but previously unrecognized, propriospinal circuit in the mammalian fusimotor system. These collaterals could still collaborate with α-γ co-activation, and the few skeletofusimotor fibers (β-MNs) in mammals, to create a flexible fusimotor ecosystem to enable voluntary movement. By locally and automatically regulating the highly nonlinear neuro-musculo-skeletal mechanics of the limb, these collaterals could be a critical low-level enabler of learning, adaptation, and performance via higher-level brainstem, cerebellar and cortical mechanisms.

Task-driven neural network models predict neural dynamics of proprioception

Proprioception tells the brain the state of the body based on distributed sensory neurons. Yet, the principles that govern proprioceptive processing are poorly understood. Here, we employ a task-driven modeling approach to investigate the neural code of proprioceptive neurons in cuneate nucleus (CN) and somatosensory cortex area 2 (S1). We simulated muscle spindle signals through musculoskeletal modeling and generated a large-scale movement repertoire to train neural networks based on 16 hypotheses, each representing different computational goals. We found that the emerging, task-optimized internal representations generalize from synthetic data to predict neural dynamics in CN and S1 of primates. Computational tasks that aim to predict the limb position and velocity were the best at predicting the neural activity in both areas. Since task optimization develops representations that better predict neural activity during active than passive movements, we postulate that neural activity in the CN and S1 is top-down modulated during goal-directed movements.

Contrasting action and posture coding with hierarchical deep neural network models of proprioception

Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body's state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one's posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks' units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.