Sensorimotor feedback loops are selectively sensitive to reward

Bilateral integration in somatosensory cortex is controlled by behavioral relevance

Sensory perception requires the processing of stimuli from both sides of the body. Yet, how neurons bind stimulus information across the hemispheres to create a unified percept remains unknown. Here we perform large-scale recordings from neurons in the left and right primary somatosensory cortex (S1) in mice performing a task requiring active whisker touch to coordinate stimulus features across hemispheres. When mice touched reward-associated stimuli, their whiskers moved with greater bilateral symmetry, and synchronous spiking and enhanced spike-field coupling emerged between the hemispheres. This coordinated activity was absent in stimulus-matched naive animals, indicating that interhemispheric coupling involves a goal-directed, internal process. In S1 neurons, the addition of ipsilateral touch primarily facilitated the contralateral principal whisker response. This facilitation primarily emerged for reward-associated stimuli and was lost on trials where mice failed to respond. Silencing of callosal S1 signaling reduced bilateral facilitation and interhemispheric synchrony. These results reveal a state-dependent logic that augments the flow of tactile information through the corpus callosum.

Success-efficient/failure-safe strategy for hierarchical reinforcement motor learning

Our study explores how ecological aspects of motor learning enhance survival by improving movement efficiency and mitigating injury risks during task failures. Traditional motor control theories mainly address isolated body movements and often overlook these ecological factors. We introduce a novel computational motor control approach, incorporating ecological fitness and a strategy that alternates between success-driven movement efficiency and failure-driven safety, akin to win-stay/lose-shift tactics. In our experiments, participants performed squat-to-stand movements under novel force perturbations. They adapted effectively through various adaptive motor control mechanisms to avoid falls, reducing failure rates rapidly. The results indicate a high-level ecological controller in human motor learning that switches objectives between safety and movement efficiency, depending on failure or success. This approach is supported by policy learning, internal model adaptation, and adaptive feedback control. Our findings offer a comprehensive perspective on human motor control, integrating risk management in a hierarchical reinforcement learning framework for real-world environments.

Neurocomputational Mechanisms of Sense of Agency: Literature Review for Integrating Predictive Coding and Adaptive Control in Human-Machine Interfaces

BACKGROUND

The sense of agency (SoA)-the subjective experience of controlling one's own actions and their consequences-is a fundamental aspect of human cognition, volition, and motor control. Understanding how the SoA arises and is disrupted in neuropsychiatric disorders has significant implications for human-machine interface (HMI) design for neurorehabilitation. Traditional cognitive models of agency often fail to capture its full complexity, especially in dynamic and uncertain environments.

OBJECTIVE

This review synthesizes computational models-particularly predictive coding, Bayesian inference, and optimal control theories-to provide a unified framework for understanding the SoA in both healthy and dysfunctional brains. It aims to demonstrate how these models can inform the design of adaptive HMIs and therapeutic tools by aligning with the brain's own inference and control mechanisms.

METHODS

I reviewed the foundational and contemporary literature on predictive coding, Kalman filtering, the Linear-Quadratic-Gaussian (LQG) control framework, and active inference. I explored their integration with neurophysiological mechanisms, focusing on the somato-cognitive action network (SCAN) and its role in sensorimotor integration, intention encoding, and the judgment of agency. Case studies, simulations, and XR-based rehabilitation paradigms using robotic haptics were used to illustrate theoretical concepts.

RESULTS

The SoA emerges from hierarchical inference processes that combine top-down motor intentions with bottom-up sensory feedback. Predictive coding frameworks, especially when implemented via Kalman filters and LQG control, provide a mechanistic basis for modeling motor learning, error correction, and adaptive control. Disruptions in these inference processes underlie symptoms in disorders such as functional movement disorder. XR-based interventions using robotic interfaces can restore the SoA by modulating sensory precision and motor predictions through adaptive feedback and suggestion. Computer simulations demonstrate how internal models, and hypnotic suggestions influence state estimation, motor execution, and the recovery of agency.

CONCLUSIONS

Predictive coding and active inference offer a powerful computational framework for understanding and enhancing the SoA in health and disease. The SCAN system serves as a neural hub for integrating motor plans with cognitive and affective processes. Future work should explore the real-time modulation of agency via biofeedback, simulation, and SCAN-targeted non-invasive brain stimulation.

Reward influences movement vigor through multiple motor cortical mechanisms

The prospect of greater rewards often invigorates movements. What neural mechanisms support this increase of movement vigor for greater rewards? We had three rhesus monkeys perform reaching movements to targets worth different magnitudes of reward. We recorded neural population activity from primary motor and dorsal premotor cortex, brain areas at the output of cortical processing for voluntary movements, and asked how neural activity mediated the translation of reward into increased vigor. We identified features of neural activity during movement preparation, initiation, and execution that were both correlated with vigor and modulated by reward. We also found that the neural metrics that correlate with different aspects of movement vigor exhibit only limited correlation with one another, suggesting that there are multiple mechanisms through which reward modulates vigor. Finally, we note that the majority of reward's modulation of motor cortical activity cannot be accounted for by reward-mediated vigor differences in behavior, indicating that reward modulations within motor cortex may serve roles in addition to affecting vigor. Overall, our results provide insight into the neural mechanisms that link reward-driven motivation to the modulation of the details of movement.

Effect of Extrinsic Reward on Motor Plasticity during Skill Learning

Human motor skill acquisition is improved by performance feedback, and coupling such feedback with extrinsic reward (such as money) can enhance skill learning. However, the neurophysiology underlying such behavioral effect is unclear. To bridge this gap, we assessed the effects of reward on multiple forms of motor plasticity during skill learning. Sixty-five healthy participants divided into three groups performed a pinch-grip skill task with sensory feedback only, sensory and reinforcement feedback, or both feedback coupled with an extrinsic monetary reward during skill training. To probe motor plasticity, we applied transcranial magnetic stimulation at rest, on the left primary motor cortex before, at an early-training time point, and after training in the three groups and measured motor-evoked potentials from task-relevant muscle of the right arm. This allowed us to evaluate the amplitude and variability of corticospinal output, GABAergic short-intracortical inhibition, and use-dependent plasticity before training and at two additional time points (early and end training). At the behavioral level, monetary reward accelerated skill learning. In parallel, corticospinal output became less variable early on during training in the presence of extrinsic reward. Interestingly, this effect was particularly pronounced for participants who were more sensitive to reward, as evaluated in an independent questionnaire. Other measures of motor excitability remained comparable across groups. These findings highlight that a mechanism underlying the benefit of reward on motor skill learning is the fine-tuning of early-training resting-state corticospinal variability.

A neural basis of choking under pressure

Incentives tend to drive improvements in performance. But when incentives get too high, we can "choke under pressure" and underperform right when it matters most. What neural processes might lead to choking under pressure? We studied rhesus monkeys performing a challenging reaching task in which they underperformed when an unusually large "jackpot" reward was at stake, and we sought a neural mechanism that might result in that underperformance. We found that increases in reward drive neural activity during movement preparation into, and then past, a zone of optimal performance. We conclude that neural signals of reward and motor preparation interact in the motor cortex (MC) in a manner that can explain why we choke under pressure.

Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

Dopamine Increases Accuracy and Lengthens Deliberation Time in Explicit Motor Skill Learning

Although animal research implicates a central role for dopamine in motor skill learning, a direct causal link has yet to be established in neurotypical humans. Here, we tested if a pharmacological manipulation of dopamine alters motor learning, using a paradigm which engaged explicit, goal-directed strategies. Participants (27 females; 11 males; aged 18-29 years) first consumed either 100 mg of levodopa (n = 19), a dopamine precursor that increases dopamine availability, or placebo (n = 19). Then, during training, participants learnt the explicit strategy of aiming away from presented targets by instructed angles of varying sizes. Targets jumped mid-movement by the instructed aiming angle. Task success was thus contingent upon aiming accuracy and not speed. The effect of the dopamine manipulations on skill learning was assessed during training and after an overnight follow-up. Increasing dopamine availability at training improved aiming accuracy and lengthened reaction times, particularly for larger, more difficult aiming angles, both at training and, importantly, at follow-up, despite prominent session-by-session performance improvements in both accuracy and speed. Exogenous dopamine thus seems to result in a learnt, persistent propensity to better adhere to task goals. Results support the proposal that dopamine is important in engagement of instrumental motivation to optimize adherence to task goals, particularly when learning to execute goal-directed strategies in motor skill learning.

Human reaching control in dynamic environments

Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Exploring the role of task success in implicit motor adaptation

We learn to improve our motor skills using different forms of feedback: sensory-prediction error, task success, and reward/punishment. While implicit motor adaptation is driven by sensory-prediction errors, recent work has shown that task success modulates this process. Task success is often confounded with reward, so we sought to determine if the effects of these two signals on adaptation can be dissociated. To address this question, we conducted five experiments that isolated implicit learning using error-clamp visuomotor reach adaptation paradigms. Task success was manipulated by changing the size and position of the target relative to the cursor providing visual feedback, and reward expectation was established using monetary cues and auditory feedback. We found that neither monetary cues nor auditory feedback affected implicit adaptation, suggesting that task success influences implicit adaptation via mechanisms distinct from conventional reward-related processes. Additionally, we found that changes in target size, which caused the target to either exclude or fully envelop the cursor, only affected implicit adaptation for a narrow range of error sizes, while jumping the target to overlap with the cursor more reliably and robustly affected implicit adaptation. Taken together, our data indicate that, while task success exerts a small effect on implicit adaptation, these effects are susceptible to methodological variations and unlikely to be mediated by reward.