Vigor of reaching movements: reward discounts the cost of effort

Observed reaching speed signals stimulus value and informs foraging

Optimal foraging requires agents to strike a balance between potential costs and rewards of interacting with stimuli in the environment. Research on human and animal foraging shows that the value an agent assigns to a stimulus is correlated with the speed of their reaching movement towards that stimulus (Shadmehr et al., 2019). Humans and other animals learn about the value of stimuli in their environment by observing others acting (Pyke, 1984; Boyd, Richerson & Henrich, 2011). Considering that humans are able to derive specific mental states such as intentions, emotions or confidence from specific movement parameters (Becchio et al., 2012), we aimed to investigate whether observers can use an actor's movement speed to: 1) infer the value of a foraging stimulus; and 2) use such cues to inform their own foraging behavior. The current study first replicated the effect of stimulus value on reaching movements in a novel foraging task (Exp. 1, N = 34). In three further experiments, we demonstrate that, depending on the speed by which an actor reaches for stimuli, observers infer the value of these stimuli (Exp. 2, N = 54), express foraging preferences (Exp. 3, N = 54), and invest time and effort to forage (Exp. 4, N = 105). This demonstrates that observers optimize their own explore-exploit decisions by inferring the value of a stimulus from the manner by which an actor approaches it, highlighting the fundamental role that action understanding plays in successful foraging.

Reward influences movement vigor through multiple motor cortical mechanisms

The prospect of greater rewards often invigorates movements. What neural mechanisms support this increase of movement vigor for greater rewards? We had three rhesus monkeys perform reaching movements to targets worth different magnitudes of reward. We recorded neural population activity from primary motor and dorsal premotor cortex, brain areas at the output of cortical processing for voluntary movements, and asked how neural activity mediated the translation of reward into increased vigor. We identified features of neural activity during movement preparation, initiation, and execution that were both correlated with vigor and modulated by reward. We also found that the neural metrics that correlate with different aspects of movement vigor exhibit only limited correlation with one another, suggesting that there are multiple mechanisms through which reward modulates vigor. Finally, we note that the majority of reward's modulation of motor cortical activity cannot be accounted for by reward-mediated vigor differences in behavior, indicating that reward modulations within motor cortex may serve roles in addition to affecting vigor. Overall, our results provide insight into the neural mechanisms that link reward-driven motivation to the modulation of the details of movement.

Reward invigorates isometric gripping actions

Individuals exhibit a propensity to move faster toward more rewarding stimuli. Although this phenomenon has been observed in movements, the effect of reward on implicit control of isometric actions, such as gripping or grasping, is relatively unknown. How reward-related invigoration generalizes to other effortful actions is an important question. Reward invigorates reaching movements and saccades, supporting the idea that reward pays the additional effort cost of moving faster. Effort in isometric force generation is less understood, so here we ask whether and how reward-related invigoration generalizes to isometric force gripping. And if so, what implicit characteristics of gripping change when there is a prospect of reward? Participants (n = 19) gripped a force transducer and the force applied was mapped to radial position of an onscreen cursor. Each trial, a target appeared in one of four locations; increasing grip force moved the cursor toward the target. The gripping action was interchangeable for all target positions. In each block of 100 trials, one target was consistently rewarded, whereas the other targets were not. When gripping to acquire the rewarded target, participants reacted faster, generated force more rapidly and to a greater extent, without increasing variance and without increasing the rising force-time integral. These findings support the generalization of reward-related invigoration in isometric force tasks, and that the brain exquisitely trades-off reward and effort costs to obtain reward more rapidly without increasing variance and without more effort costs than necessary.NEW & NOTEWORTHY Gripping actions are important for day-to-day tasks, for medical diagnostics like strength and force control, and for choice selection in decision-making experiments. Comparing isometric gripping responses to reward and nonreward cues, we observed reward-based invigoration mediated by selective increases in effort. These findings can be leveraged to provide additional insight into the decision making process and better understand the effect of reward on movement vigor.

Reaching vigor tracks learned prediction error

Movement vigor across multiple modalities increases with reward, suggesting that the neural circuits that represent value influence the control of movement. Dopaminergic neuron (DAN) activity in the basal ganglia has been suggested as the potential mediator of this response. If DAN activity is the bridge between value and vigor, then vigor should track canonical mediators of this activity, namely reward expectation and reward prediction error. Here we ask if a similar time-locked response is present in vigor of reaching movements. We explore this link by leveraging the known phasic dopaminergic response to stochastic rewards, where activity is modulated by both reward expectation at cue and the prediction error at feedback. We used probabilistic rewards to create a reaching task rich in reward expectation, reward prediction error, and learning. In one experiment, target reward probabilities were explicitly stated, and in the other, were left unknown and to be learned by the participants. We included two stochastic rewards (probabilities 33% and 66%) and two deterministic ones (probabilities 100% and 0%). Outgoing peak velocity in both experiments increased with increasing reward expectation. Furthermore, we observed a short-latency response in the vigor of the ongoing movement, that tracked reward prediction error: either invigorating or enervating velocity consistent with the sign and magnitude of the error. Reaching kinematics also revealed the value-update process in a trial-to-trial fashion, similar to the effect of prediction error signals typical in dopamine-mediated striatal phasic activity. Lastly, reach vigor increased with reward history over trials, mirroring the motivational effects often linked to fluctuating dopamine levels. Taken together, our results demonstrate and exquisite link between known short-latency reward signals and the invigoration of both discrete and ongoing movements.

Mapping Eye, Arm, and Reward Information in Frontal Motor Cortices Using Electrocorticography in Nonhuman Primates

Goal-directed reaches give rise to dynamic neural activity across the brain as we move our eyes and arms and process outcomes. High spatiotemporal resolution mapping of multiple cortical areas will improve our understanding of how these neural computations are spatially and temporally distributed across the brain. In this study, we used micro-electrocorticography (µECoG) recordings in two male monkeys performing visually guided reaches to map information related to eye movements, arm movements, and receiving rewards over primary motor cortex, premotor cortex, frontal eye field, and dorsolateral prefrontal cortex. Time-frequency and decoding analyses revealed that eye and arm movement information shifts across brain regions during a reach, likely reflecting shifts from planning to execution. Although eye and arm movement temporally overlapped, phase clustering analyses enabled us to resolve differences in eye and arm information across brain regions. This analysis revealed that eye and arm information spatially overlapped in motor cortex, which we further confirmed by demonstrating that arm movement decoding performance from motor cortex activity was impacted by task-irrelevant eye movements. Phase clustering analyses also identified reward-related activity in the prefrontal and premotor cortex. Our results demonstrate µECoG's strengths for functional mapping and provide further detail on the spatial distribution of eye, arm, and reward information processing distributed across frontal cortices during reaching. These insights advance our understanding of the overlapping neural computations underlying coordinated movements and reveal opportunities to leverage these signals to enhance future brain-computer interfaces.

Computational mechanism underlying switching of motor actions

Survival of species in an ever-changing environment requires a flexibility that extends beyond merely selecting the most appropriate actions. It also involves readiness to stop or switch actions in response to environmental changes. Although considerable research has been devoted to understanding how the brain switches actions, the computations underlying the switching process and how it relates to the selecting and stopping processes remain elusive. A normative theory suggests that switching is simply an extension of the stopping process, during which a current action is first inhibited by an independent pause mechanism before a new action is generated. This theory was challenged by the affordance competition hypothesis, according to which the switching process is implemented through a competition between the current and new actions, without engaging an independent pause mechanism. To delineate the computations underlying these action regulation functions, we utilized a neurocomputational theory that models the process of selecting, stopping and switching reaching movements. We tested the model predictions in healthy individuals who performed reaches in dynamic and uncertain environments that often required stopping and switching actions. Our findings suggest that unlike the stopping process, switching does not necessitate a proactive pause mechanism to delay movement initiation. Hence, the switching and stopping processes seem to be implemented by different mechanisms at the planning phase of the reaching movement. However, once the reaching movement has been initiated, the switching process seems to involve an independent pause mechanism if the new target location is unknown prior to movement initiation. These findings offer a new understanding of the computations underlying action switching, contribute valuable insights into the fundamental neuroscientific mechanisms of action regulation, and open new avenues for future neurophysiological investigations.

The Use of Extrinsic Performance Feedback and Reward to Enhance Upper Limb Motor Behavior and Recovery Post-Stroke: A Scoping Review

BACKGROUND

During post-stroke motor rehabilitation, patients often receive feedback from therapists or via rehabilitation technologies. Research suggests that feedback may benefit motor performance, skill acquisition, and action selection. However, there is no consensus on how extrinsic feedback should be implemented during stroke rehabilitation to best leverage specific neurobehavioral mechanisms to optimize recovery.

OBJECTIVE

To identify the existing evidence and research gaps regarding the effects of extrinsic feedback on upper extremity motor function in stroke survivors, and to map the evidence onto neurobehavioral concepts of motor performance, motor learning, and action selection.

METHODS

The MEDLINE, PsychInfo, EMBASE, and CINHAL databases were searched for relevant articles. A sequential screening process and data extraction were performed by 2 independent reviewers, and the results were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews guidelines.

RESULTS

A total of 29 studies were identified that met the criteria for inclusion. Beneficial effects of feedback were reported for clinical outcomes of rehabilitation interventions as well as motor performance, motor learning, and action selection post-stroke. Three studies showed that the addition of rewarding elements to positive performance feedback benefited learning or recovery.

CONCLUSIONS

Extrinsic feedback has the potential to improve outcomes of stroke rehabilitation through effects on motor performance, motor learning, or action selection. To understand how these specific neurobehavioral processes contribute to recovery, clinical trials should include more granular behavioral measures. Rewarding feedback may be particularly beneficial, but more research is needed regarding the specific implementation of feedback.

Frequency-specific changes in prefrontal activity associated with maladaptive belief updating in volatile environments in euthymic bipolar disorder

Bipolar disorder (BD) involves altered reward processing and decision-making, with inconsistencies across studies. Here, we integrated hierarchical Bayesian modelling with magnetoencephalography (MEG) to characterise maladaptive belief updating in this condition. First, we determined if previously reported increased learning rates in BD stem from a heightened expectation of environmental changes. Additionally, we examined if this increased expectation speeds up belief updating in decision-making, associated with modulation of rhythmic neural activity within the prefrontal, orbitofrontal, and anterior cingulate cortex (PFC, OFC, ACC). Twenty-two euthymic BD and 27 healthy control (HC) participants completed a reward-based motor decision-making task in a volatile setting. Hierarchical Bayesian modelling revealed BD participants anticipated greater environmental volatility, resulting in a more stochastic mapping from beliefs to actions and paralleled by lower win rates and a reduced tendency to repeat rewarded actions than HC. Despite this, BD individuals adjusted their expectations of action-outcome contingencies more slowly, but both groups invigorated their actions similarly. On a neural level, while healthy individuals exhibited an alpha-beta suppression and gamma increase during belief updating, BD participants showed dampened effects, extending across the PFC, OFC, and ACC regions. This was accompanied by an abnormally increased beta-band directed information flow in BD. Overall, the results suggest euthymic BD individuals anticipate environmental change without adequately learning from it, contributing to maladaptive belief updating. Alterations in frequency-domain amplitude and functional connectivity within the PFC, OFC, and ACC during belief updating underlie the computational effects and could serve as potential indicators for predicting relapse in future research.

Rapid, systematic updating of movement by accumulated decision evidence

Acting in the natural world requires not only deciding among multiple options but also converting decisions into motor commands. How the dynamics of decision formation influence the fine kinematics of response movement remains, however, poorly understood. Here we investigate how the accumulation of decision evidence shapes the response orienting trajectories in a task where freely-moving rats combine prior expectations and auditory information to select between two possible options. Response trajectories and their motor vigor are initially determined by the prior. Rats movements then incorporate sensory information in less than 100 ms after stimulus onset by accelerating or slowing depending on how much the stimulus supports their initial choice. When the stimulus evidence is in strong contradiction, rats change their mind and reverse their initial trajectory. Human subjects performing an equivalent task display a remarkably similar behavior. We encapsulate these results in a computational model that maps the decision variable onto the movement kinematics at discrete time points, capturing subjects' choices, trajectories and changes of mind. Our results show that motor responses are not ballistic. Instead, they are systematically and rapidly updated, as they smoothly unfold over time, by the parallel dynamics of the underlying decision process.

Gravity support from a robotic exoskeleton increases spontaneous use of the nondominant upper extremity during a choice reaching task

The objective was to determine whether gravity support for the left arm of right-handed participants would increase left arm use during a three-dimensional (3-D) reaching task in virtual reality. Twelve healthy control participants each completed 630 reaching movements broken into six blocks. The majority of targets were placed close to the midsagittal plane at three heights, and participants were free to use either limb when reaching for targets. The hand had to stay in the target for a prescribed dwell time before the target disappeared. For all reaching tasks within a block, the left arm gravity support was set to either 0% or 75% of full arm support. The blocks also varied in the dwell time (2, 4, or 6 s). The order of blocks was balanced across participants in terms of gravity support level and dwell time. Electromyogram (EMG) level in the left medial deltoid decreased with increasing gravity support (P < 0.001) and was higher for higher targets compared to lower targets (P < 0.001). The odds of using the left arm were 1.95 times higher with gravity support compared to no support (P < 0.001). With gravity support, we expected greater shifts toward the left arm in tasks that were more energetically demanding. This was not the case, as the increased use was evenly distributed across all target heights, and use decreased or remained unchanged with increasing dwell time. Results are discussed relative to current models of limb choice and the potential use of robotic gravity support to overcome learned nonuse in stroke patients.NEW & NOTEWORTHY We have shown that gravity support from a robotic exoskeleton increases use of the left arm of right-handed healthy participants. Prior work has shown similar results when movements of one arm are amplified in a virtual environment. The advantage of this approach is the potential to apply the intervention during functional task practice outside of the VR environment or during performance of actual activities of daily living (ADL).

The affective response to positive performance feedback is associated with motor learning

Motor skill learning and performance are improved when successful actions are paired with extrinsic rewards, such as money. Positive feedback indicating successful task performance is thought to induce intrinsic reward associated with goal attainment, evidenced by increases in positive affect that correlate with neural reward signaling. However, it is not clear whether the subjective, internal reward processes elicited by positive feedback promote motor learning and performance.Here, we tested the hypothesis that intrinsic reward elicited by positive feedback promotes motor learning and performance. Participants practiced a visuomotor interception task using a joystick, and received feedback during practice indicating success or failure depending on their accuracy. During practice, the accuracy demands were adapted to control and vary the frequency of positive feedback across randomly ordered blocks of practice at either 50%, 70%, or 90%. Performance was measured for each condition as the average accuracy during practice. Learning was estimated by measuring the accuracy pre and post practice in the absence of feedback. We queried participants periodically on their enjoyment of the task to index affective responses to performance feedback.The intrinsic reward elicited by positive feedback, operationalized by the increase in enjoyment immediately following positive versus negative feedback, was positively correlated with learning from pre to post practice. However, increasing the overall amount of positive feedback by lower accuracy demands did not improve performance. These results suggest that experiencing intrinsic reward due to positive feedback benefits motor learning only when it is contingent on good performance.

Kinematic coding: Measuring information in naturalistic behaviour

Recent years have seen an explosion of interest in naturalistic behaviour and in machine learning tools for automatically tracking it. However, questions about what to measure, how to measure it, and how to relate naturalistic behaviour to neural activity and cognitive processes remain unresolved. In this Perspective, we propose a general experimental and computational framework - kinematic coding - for measuring how information about cognitive states is encoded in structured patterns of behaviour and how this information is read out by others during social interactions. This framework enables the design of new experiments and the generation of testable hypotheses that link behaviour, cognition, and neural activity at the single-trial level. Researchers can employ this framework to identify single-subject, single-trial encoding and readout computations and address meaningful questions about how information encoded in bodily motion is transmitted and communicated.

Evidence for interacting but decoupled controls of decisions and movements in nonhuman primates

Many recent studies indicate that control of decisions and actions is integrated during interactive behavior. Among these, several carried out in humans and monkeys conclude that there is a coregulation of choices and movements. Another perspective, based on human data only, proposes a decoupled control of decision duration and movement speed, allowing, for instance, trading decision duration for movement duration when time pressure increases. Crucially, it is not currently known whether this ability to flexibly dissociate decision duration from movement speed is specific to humans, whether it can vary depending on the context in which a task is performed, and whether it is stable over time. These are important questions to address, especially to rely on monkey electrophysiology to infer the neural mechanisms of decision-action coordination in humans. To do so, we trained two macaque monkeys in a perceptual decision-making task and analyzed data collected over multiple behavioral sessions. Our findings reveal a strong and complex relationship between decision duration and movement vigor. Decision duration and action duration can covary but also "compensate" each other. Such integrated but decoupled control of decisions and actions aligns with recent studies in humans, validating the monkey model in electrophysiology as a means of inferring neural mechanisms in humans. Crucially, we demonstrate for the first time that this control can evolve with experience, in an adapted manner. Together, the present findings contribute to deepening our understanding of the integrated control of decisions and actions during interactive behavior.NEW & NOTEWORTHY The mechanism by which the integrated control of decisions and actions occurs, coupled or interactive but decoupled, is debated. In the present study, we show in monkeys that decisions and actions influence each other in a decoupled way. For the first time, we also demonstrate that this control can evolve depending the subject's experience, allowing the trade of movement time for decision time and limiting the temporal discounting of reward value.

Reward invigorates isometric gripping actions

Individuals exhibit a propensity to move faster toward more rewarding stimuli. While this phenomenon has been observed in movements, the effect of reward on implicit control of isometric actions, like gripping or grasping, is relatively unknown. How reward-related invigoration generalizes to other effortful actions is an important question. Reward invigorates reaching movements and saccades, supporting the idea that reward pays the additional effort cost of moving faster. Effort in isometric force generation is less understood, so here we ask whether and how reward-related invigoration generalizes to isometric force gripping. And if so, what implicit characteristics of gripping change when there is a prospect of reward? Participants (N=19) gripped a force transducer and the force applied was mapped to radial position of an onscreen cursor. Each trial, a target appeared in one of four locations; increasing grip force moved the cursor toward the target. The gripping action was interchangeable for all target positions. In each block of 100 trials, one target was consistently rewarded, while the other targets were not. When gripping to acquire the rewarded target, participants reacted faster, generated force more rapidly and to a greater extent, while intriguingly maintaining the same accuracy and integral of force over time. These findings support the generalization of reward-related invigoration in isometric force tasks, and that the brain exquisitely trades-off reward and effort costs to obtain reward more rapidly without compromising accuracy or more effort costs than necessary.

NEW & NOTEWORTHY

Gripping actions are important for day-to-day tasks, for medical diagnostics like strength and force control, and for choice selection in decision-making experiments. Comparing isometric gripping responses to reward and nonreward cues, we observed reward-based invigoration mediated by selective increases in effort. These findings can be leveraged to provide additional insight into the decision making process, and better understand the effect of reward on movement vigor and the implicit control of accuracy.

A neural basis of choking under pressure

Incentives tend to drive improvements in performance. But when incentives get too high, we can "choke under pressure" and underperform right when it matters most. What neural processes might lead to choking under pressure? We studied rhesus monkeys performing a challenging reaching task in which they underperformed when an unusually large "jackpot" reward was at stake, and we sought a neural mechanism that might result in that underperformance. We found that increases in reward drive neural activity during movement preparation into, and then past, a zone of optimal performance. We conclude that neural signals of reward and motor preparation interact in the motor cortex (MC) in a manner that can explain why we choke under pressure.

Mapping eye, arm, and reward information in frontal motor cortices using electrocorticography in non-human primates

Goal-directed reaches give rise to dynamic neural activity across the brain as we move our eyes and arms, and process outcomes. High spatiotemporal resolution mapping of multiple cortical areas will improve our understanding of how these neural computations are spatially and temporally distributed across the brain. In this study, we used micro-electrocorticography (μECoG) recordings in two male monkeys performing visually guided reaches to map information related to eye movements, arm movements, and receiving rewards over a 1.37 cm2 area of frontal motor cortices (primary motor cortex, premotor cortex, frontal eye field, and dorsolateral pre-frontal cortex). Time-frequency and decoding analyses revealed that eye and arm movement information shifts across brain regions during a reach, likely reflecting shifts from planning to execution. We then used phase-based analyses to reveal potential overlaps of eye and arm information. We found that arm movement decoding performance was impacted by task-irrelevant eye movements, consistent with the presence of intermixed eye and arm information across much of motor cortices. Phase-based analyses also identified reward-related activity primarily around the principal sulcus in the pre-frontal cortex as well as near the arcuate sulcus in the premotor cortex. Our results demonstrate μECoG's strengths for functional mapping and provide further detail on the spatial distribution of eye, arm, and reward information processing distributed across frontal cortices during reaching. These insights advance our understanding of the overlapping neural computations underlying coordinated movements and reveal opportunities to leverage these signals to enhance future brain-computer interfaces.

Pre-movement muscle co-contraction associated with motor performance deterioration under high reward conditions

Reward usually enhances task performance, but exceptionally large rewards can impede performance due to psychological pressure. In this study, we investigated motor activity changes in high-reward situations and identified indicators for performance decline. Fourteen healthy adults practiced a velocity-dependent right-hand motor task for three days, followed by a test day with varying monetary reward for each trial. Participants were divided into low performers (LPs) and high performers (HPs) according to whether success rate decreased or increased, respectively, on the highest reward trials compared to lower reward trials. Both LPs and HPs demonstrated increased hand velocity during higher reward trials, but only LPs exhibited a significant increase in velocity variance. There was also a negative correlation between the pre-movement co-contraction index (CCI) of the biceps and triceps muscles and success rate on the highest reward trials. This correlation was confirmed in a second experiment with 12 newly recruited participants, suggesting that pre-movement CCI is a marker for performance decline caused by high reward. These findings suggest that interventions to reduce pre-movement CCI such as biofeedback training could be useful for preventing the paradoxical decline in motor performance associated with high rewards.

Lifelong learning of cognitive styles for physical problem-solving: The effect of embodied experience

'Embodied cognition' suggests that our bodily experiences broadly shape our cognitive capabilities. We study how embodied experience affects the abstract physical problem-solving styles people use in a virtual task where embodiment does not affect action capabilities. We compare how groups with different embodied experience - 25 children and 35 adults with congenital limb differences versus 45 children and 40 adults born with two hands - perform this task, and find that while there is no difference in overall competence, the groups use different cognitive styles to find solutions. People born with limb differences think more before acting but take fewer attempts to reach solutions. Conversely, development affects the particular actions children use, as well as their persistence with their current strategy. Our findings suggest that while development alters action choices and persistence, differences in embodied experience drive changes in the acquisition of cognitive styles for balancing acting with thinking.

Improving rehabilitation motivation and motor learning ability of stroke patients using different reward strategies: study protocol for a single-center, randomized controlled trial

Background

Stroke survivors often face challenges in motor learning and motivation during rehabilitation, which can impede their recovery progress. Traditional rehabilitation methods vary in effectiveness, prompting the exploration of novel approaches such as reward strategies. Previous research indicates that rewards can enhance rehabilitation motivation and facilitate motor learning. However, most reward paradigms have utilized fixed reward amounts, which also have limitations. Exploring alternative, more effective reward strategies, such as probabilistic rewards, is warranted to optimize stroke patient rehabilitation.

Methods

A total of 81 stroke patients will be recruited and randomly assigned to control, fixed reward, or probabilistic reward groups at a ratio of 1:1:1 using a randomized number table method. Participants will undergo 10 days of daily hand motor function rehabilitation training, with sessions lasting 20 min each. The training will involve pegboard tests and box and block tests. Control group participants will receive standard training, while fixed reward group members will receive monetary incentives for completing tests, and probabilistic reward group members will have the chance to win monetary rewards through a lottery box. Rehabilitation motivation and motor performance and functional near-infrared spectroscopy brain imaging will be conducted at designated time points. The primary outcome measure is the stroke rehabilitation motivation scale, and the second outcome measures include motor performance, simple test for evaluating hand function, motivation and pleasure scale self-report, and Pittsburgh rehabilitation participation scale.

Discussion

Reward-based training enhance rehabilitation participation and adherence, it also improve motor learning speed and memory retention of stroke patients. The fixed reward applied in the past studies could diminish the sensitivity of stroke patients to rewards, while probabilistic reward may provide unpredictable or variable incentives or reinforcements for motor rehabilitation. This study will compare the efficacy of different reward strategies in enhancing motor learning ability and rehabilitation motivation among stroke patients. By conducting a randomized controlled trial, the study seeks to provide valuable insights into optimizing stroke rehabilitation protocols and improving patient outcomes.Clinical Trial Registration:https://www.chictr.org.cn/, ChiCTR2400082419.

Disentangling the effects of metabolic cost and accuracy on movement speed

On any given day, we make countless reaching movements to objects around us. While such ubiquity may suggest uniformity, each movement's speed is unique-why is this? Reach speed is known to be influenced by accuracy; we slow down to sustain high accuracy. However, in other forms of movement like walking or running, metabolic cost is often the primary determinant of movement speed. Here we bridge this gap and ask: how do metabolic cost and accuracy interact to determine speed of reaching movements? First, we systematically measure the effect of increasing mass on the metabolic cost of reaching across a range of movement speeds. Next, in a sequence of three experiments, we examine how added mass affects preferred reaching speed across changing accuracy requirements. We find that, while added mass consistently increases metabolic cost thereby leading to slower metabolically optimal movement speeds, self-selected reach speeds are slower than those predicted by an optimization of metabolic cost alone. We then demonstrate how a single model that considers both accuracy and metabolic costs can explain preferred movement speeds. Together, our findings provide a unifying framework to illuminate the combined effects of metabolic cost and accuracy on movement speed and highlight the integral role metabolic cost plays in determining reach speed.

Slowing of Movements in Healthy Aging as a Rational Economic Response to an Elevated Effort Landscape

Why do we move slower as we grow older? The reward circuits of the brain, which tend to invigorate movements, decline with aging, raising the possibility that reduced vigor is due to the diminishing value that our brain assigns to movements. However, as we grow older, it also becomes more effortful to make movements. Is age-related slowing principally a consequence of increased effort costs from the muscles, or reduced valuation of reward by the brain? Here, we first quantified the cost of reaching via metabolic energy expenditure in human participants (male and female), and found that older adults consumed more energy than the young at a given speed. Thus, movements are objectively more costly for older adults. Next, we observed that when reward increased, older adults, like the young, responded by initiating their movements earlier. Yet, unlike the young, they were unwilling to increase their movement speed. Was their reluctance to reach quicker for rewards due to the increased effort costs, or because they ascribed less value to the movement? Motivated by a mathematical model, we next made the young experience a component of aging by making their movements more effortful. Now the young responded to reward by reacting faster but chose not to increase their movement speed. This suggests that slower movements in older adults are partly driven by an adaptive response to an elevated effort landscape. Moving slower may be a rational economic response the brain is making to mitigate the elevated effort costs that accompany aging.

Effects of reward and effort history on decision making and movement vigor during foraging

During foraging, animals explore a site and harvest reward and then abandon that site and travel to the next opportunity. One aspect of this behavior involves decision making, and the other involves movement control. These two aspects of behavior may be linked via an underlying desire to maximize a single normative utility: the sum of all rewards acquired, minus all efforts expended, divided by time. According to this theory, the history of rewards, and not just its immediate availability, should dictate how long one should stay and harvest reward and how vigorously one should travel to the next opportunity. We tested this theory in a series of experiments in which humans used their hand to harvest tokens at a reward patch and then used their arm to reach toward another patch. After a history of high rewards, the subjects not only shortened their harvest duration but also moved more vigorously toward the next reward opportunity. In contrast, after a history of high effort they lengthened their harvest duration but reduced their movement vigor, reaching more slowly to the next reward site. Thus, a history of high reward or low effort biased decisions by promoting early abandonment of the reward site and biased movements by promoting vigor.NEW & NOTEWORTHY Much of life is spent foraging. Whereas previous work has focused on the decision regarding time spent harvesting from a reward patch, here we test the idea that both decision making and movement control are tuned to optimize the net rate of reward in an environment. Our results show that movement patterns reflect not just immediate expectations but also past experiences in the environment, providing fundamental insight into the factors governing volitional control of arm movements.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Punishment Leads to Greater Sensorimotor Learning But Less Movement Variability Compared to Reward

When a musician practices a new song, hitting a correct note sounds pleasant while striking an incorrect note sounds unpleasant. Such reward and punishment feedback has been shown to differentially influence the ability to learn a new motor skill. Recent work has suggested that punishment leads to greater movement variability, which causes greater exploration and faster learning. To further test this idea, we collected 102 participants over two experiments. Unlike previous work, in Experiment 1 we found that punishment did not lead to faster learning compared to reward (n = 68), but did lead to a greater extent of learning. Surprisingly, we also found evidence to suggest that punishment led to less movement variability, which was related to the extent of learning. We then designed a second experiment that did not involve adaptation, allowing us to further isolate the influence of punishment feedback on movement variability. In Experiment 2, we again found that punishment led to significantly less movement variability compared to reward (n = 34). Collectively our results suggest that punishment feedback leads to less movement variability. Future work should investigate whether punishment feedback leads to a greater knowledge of movement variability and or increases the sensitivity of updating motor actions.

Responsiveness to exoskeleton loading during bimanual reaching is associated with corticospinal tract integrity in stroke

Background

Device-based rehabilitation of upper extremity impairment following stroke often employs one-sized-fits-all approaches that do not account for individual differences in patient characteristics.

Objective

Determine if corticospinal tract lesion load could explain individual differences in the responsiveness to exoskeleton loading of the arms in chronic stroke participants.

Methods

Fourteen stroke participants performed a bimanual shared cursor reaching task in virtual reality while exoskeletons decreased the effective weight of the more-impaired arm and increased the effective weight of the less-impaired arm. We calculated the change in relative displacement between the arms (RC) and the change in relative muscle activity (MC) between the arms from the biceps and deltoids. We calculated corticospinal tract lesion load (wCSTLL) in a subset of 10 participants.

Results

Exoskeleton loading did not change RC (p = 0.07) or MC (p = 0.47) at the group level, but significant individual differences emerged. Participants with little overlap between the lesion and corticospinal tract responded to loading by decreasing muscle activity in the more-impaired arm relative to the less-impaired arm. The change in deltoid MC was associated with smaller wCSTLL (R2 = 0.43, p = 0.039); there was no such relationship for biceps MC (R2 < 0.001, p = 0.98).

Conclusion

Here we provide evidence that corticospinal tract integrity is a critical feature that determines one's ability to respond to upper extremity exoskeleton loading. Our work contributes to the development of personalized device-based interventions that would allow clinicians and researchers to titrate constraint levels during bimanual activities.

Using virtual reality-based neurocognitive testing and eye tracking to study naturalistic cognitive-motor performance

Natural human behavior arises from continuous interactions between the cognitive and motor domains. However, assessments of cognitive abilities are typically conducted using pen and paper tests, i.e., in isolation from "real life" cognitive-motor behavior and in artificial contexts. In the current study, we aimed to assess cognitive-motor task performance in a more naturalistic setting while recording multiple motor and eye tracking signals. Specifically, we aimed to (i) delineate the contribution of cognitive and motor components to overall task performance and (ii) probe for a link between cognitive-motor performance and pupil size. To that end, we used a virtual reality (VR) adaptation of a well-established neurocognitive test for executive functions, the 'Color Trails Test' (CTT). The VR-CTT involves performing 3D reaching movements to follow a trail of numbered targets. To tease apart the cognitive and motor components of task performance, we included two additional conditions: a condition where participants only used their eyes to perform the CTT task (using an eye tracking device), incurring reduced motor demands, and a condition where participants manually tracked visually-cued targets without numbers on them, incurring reduced cognitive demands. Our results from a group of 30 older adults (>65) showed that reducing cognitive demands shortened completion times more extensively than reducing motor demands. Conditions with higher cognitive demands had longer target search time, as well as decreased movement execution velocity and head-hand coordination. We found larger pupil sizes in the more cognitively demanding conditions, and an inverse correlation between pupil size and completion times across individuals in all task conditions. Lastly, we found a possible link between VR-CTT performance measures and clinical signatures of participants (fallers versus non-fallers). In summary, performance and pupil parameters were mainly dependent on task cognitive load, while maintaining systematic interindividual differences. We suggest that this paradigm opens the possibility for more detailed profiling of individual cognitive-motor performance capabilities in older adults and other at-risk populations.

The Effects of Different Motor Teaching Strategies on Learning a Complex Motor Task

During the learning of a new sensorimotor task, individuals are usually provided with instructional stimuli and relevant information about the target task. The inclusion of haptic devices in the study of this kind of learning has greatly helped in the understanding of how an individual can improve or acquire new skills. However, the way in which the information and stimuli are delivered has not been extensively explored. We have designed a challenging task with nonintuitive visuomotor perturbation that allows us to apply and compare different motor strategies to study the teaching process and to avoid the interference of previous knowledge present in the naïve subjects. Three subject groups participated in our experiment, where the learning by repetition without assistance, learning by repetition with assistance, and task Segmentation Learning techniques were performed with a haptic robot. Our results show that all the groups were able to successfully complete the task and that the subjects' performance during training and evaluation was not affected by modifying the teaching strategy. Nevertheless, our results indicate that the presented task design is useful for the study of sensorimotor teaching and that the presented metrics are suitable for exploring the evolution of the accuracy and precision during learning.

Rapid, systematic updating of movement by accumulated decision evidence

Acting in the natural world requires not only deciding among multiple options but also converting decisions into motor commands. How the dynamics of decision formation influence the fine kinematics of response movement remains, however, poorly understood. Here we investigate how the accumulation of decision evidence shapes the response orienting trajectories in a task where freely-moving rats combine prior expectations and auditory information to select between two possible options. Response trajectories and their motor vigor are initially determined by the prior. Rats movements then incorporate sensory information as early as 60 ms after stimulus onset by accelerating or slowing depending on how much the stimulus supports their initial choice. When the stimulus evidence is in strong contradiction, rats change their mind and reverse their initial trajectory. Human subjects performing an equivalent task display a remarkably similar behavior. We encapsulate these results in a computational model that, by mapping the decision variable onto the movement kinematics at discrete time points, captures subjects' choices, trajectories and changes of mind. Our results show that motor responses are not ballistic. Instead, they are systematically and rapidly updated, as they smoothly unfold over time, by the parallel dynamics of the underlying decision process.

Sensorimotor-linked reward modulates smooth pursuit eye movements in monkeys

Reward is essential for shaping behavior. Using sensory cues to imply forthcoming rewards, previous studies have demonstrated powerful effects of rewards on behavior. Nevertheless, the impact of reward on the sensorimotor transformation, particularly when reward is linked to behavior remains uncertain. In this study, we investigated how reward modulates smooth pursuit eye movements in monkeys. Three distinct associations between reward and eye movements were conducted in independent blocks. Results indicated that reward increased eye velocity during the steady-state pursuit, rather than during the initiation. The influence depended on the particular association between behavior and reward: a faster eye velocity was linked with reward. Neither rewarding slower eye movements nor randomizing rewards had a significant effect on behavior. The findings support the existence of distinct mechanisms involved in the initiation and steady-state phases of pursuit, and contribute to a deeper understanding of how reward interacts with these two periods of pursuit.

Atomoxetine and reward size equally improve task engagement and perceptual decisions but differently affect movement execution

Our ability to engage and perform daily activities relies on balancing the associated benefits and costs. Rewards, as benefits, act as powerful motivators that help us stay focused for longer durations. The noradrenergic (NA) system is thought to play a significant role in optimizing our performance. Yet, the interplay between reward and the NA system in shaping performance remains unclear, particularly when actions are driven by external incentives (reward). To explore this interaction, we tested four female rhesus monkeys performing a sustained Go/NoGo task under two reward sizes (low/high) and three pharmacological conditions (saline and two doses of atomoxetine, a NA reuptake inhibitor: ATX-0.5 mg/kg and ATX-1 mg/kg). We found that increasing either reward or NA levels equally enhanced the animal's engagement in the task compared to low reward saline; the animals also responded faster and more consistently under these circumstances. Notably, we identified differences between reward size and ATX. When combined with ATX, high reward further reduced the occurrence of false alarms (i.e., incorrect go trials on distractors), implying that it helped further suppress impulsive responses. In addition, ATX (but not reward size) consistently increased movement duration dose-dependently, while high reward did not affect movement duration but decreased its variability. We conclude that noradrenaline and reward modulate performance, but their effects are not identical, suggesting differential underlying mechanisms. Reward might energize/invigorate decisions and action, while ATX might help regulate energy expenditure, depending on the context, through the NA system.

Effort cost of harvest affects decisions and movement vigor of marmosets during foraging

Our decisions are guided by how we perceive the value of an option, but this evaluation also affects how we move to acquire that option. Why should economic variables such as reward and effort alter the vigor of our movements? In theory, both the option that we choose and the vigor with which we move contribute to a measure of fitness in which the objective is to maximize rewards minus efforts, divided by time. To explore this idea, we engaged marmosets in a foraging task in which on each trial they decided whether to work by making saccades to visual targets, thus accumulating food, or to harvest by licking what they had earned. We varied the effort cost of harvest by moving the food tube with respect to the mouth. Theory predicted that the subjects should respond to the increased effort costs by choosing to work longer, stockpiling food before commencing harvest, but reduce their movement vigor to conserve energy. Indeed, in response to an increased effort cost of harvest, marmosets extended their work duration, but slowed their movements. These changes in decisions and movements coincided with changes in pupil size. As the effort cost of harvest declined, work duration decreased, the pupils dilated, and the vigor of licks and saccades increased. Thus, when acquisition of reward became effortful, the pupils constricted, the decisions exhibited delayed gratification, and the movements displayed reduced vigor.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.

Common Mechanisms of Learning in Motor and Cognitive Systems

Rapid progress in our understanding of the brain's learning mechanisms has been accomplished over the past decade, particularly with conceptual advances, including representing behavior as a dynamical system, large-scale neural population recordings, and new methods of analysis of neuronal populations. However, motor and cognitive systems have been traditionally studied with different methods and paradigms. Recently, some common principles, evident in both behavior and neural activity, that underlie these different types of learning have become to emerge. Here we review results from motor and cognitive learning, relying on different techniques and studying different systems to understand the mechanisms of learning. Movement is intertwined with cognitive operations, and its dynamics reflect cognitive variables. Training, in either motor or cognitive tasks, involves recruitment of previously unresponsive neurons and reorganization of neural activity in a low dimensional manifold. Mapping of new variables in neural activity can be very rapid, instantiating flexible learning of new tasks. Communication between areas is just as critical a part of learning as are patterns of activity within an area emerging with learning. Common principles across systems provide a map for future research.

Computational mechanism underlying switching of motor actions

Surviving in a constantly changing environment requires not only the ability to select actions, but also the flexibility to stop and switch actions when necessary. Extensive research has been devoted to understanding how the brain switches actions, yet the computations underlying switching and how it relates to selecting and stopping processes remain elusive. A central question is whether switching is an extension of the stopping process or involves different mechanisms. To address this question, we modeled action regulation tasks with a neurocomputational theory and evaluated its predictions on individuals performing reaches in a dynamic environment. Our findings suggest that, unlike stopping, switching does not necessitate a proactive pause mechanism to delay movement onset. However, switching engages a pause mechanism after movement onset, if the new target location is unknown prior to switch signal. These findings offer a new understanding of the action-switching computations, opening new avenues for future neurophysiological investigations.

Reinforcement-based processes actively regulate motor exploration along redundant solution manifolds

From a baby's babbling to a songbird practising a new tune, exploration is critical to motor learning. A hallmark of exploration is the emergence of random walk behaviour along solution manifolds, where successive motor actions are not independent but rather become serially dependent. Such exploratory random walk behaviour is ubiquitous across species' neural firing, gait patterns and reaching behaviour. The past work has suggested that exploratory random walk behaviour arises from an accumulation of movement variability and a lack of error-based corrections. Here, we test a fundamentally different idea-that reinforcement-based processes regulate random walk behaviour to promote continual motor exploration to maximize success. Across three human reaching experiments, we manipulated the size of both the visually displayed target and an unseen reward zone, as well as the probability of reinforcement feedback. Our empirical and modelling results parsimoniously support the notion that exploratory random walk behaviour emerges by utilizing knowledge of movement variability to update intended reach aim towards recently reinforced motor actions. This mechanism leads to active and continuous exploration of the solution manifold, currently thought by prominent theories to arise passively. The ability to continually explore muscle, joint and task redundant solution manifolds is beneficial while acting in uncertain environments, during motor development or when recovering from a neurological disorder to discover and learn new motor actions.

The value of time in the invigoration of human movements when interacting with a robotic exoskeleton

Time and effort are thought to be subjectively balanced during the planning of goal-directed actions, thereby setting the vigor of volitional movements. Theoretical models predicted that the value of time should then amount to high levels of effort. However, the time-effort trade-off has so far only been studied for a narrow range of efforts. To investigate the extent to which humans can invest in a time-saving effort, we used a robotic exoskeleton to substantially vary the energetic cost associated with a certain vigor during reaching movements. In this situation, minimizing the time-effort trade-off should lead to high and low human efforts for upward and downward movements, respectively. Consistently, all participants expended substantial amounts of energy upward and remained essentially inactive by harnessing the work of gravity downward, while saving time in both cases. A common time-effort trade-off may therefore determine the vigor of reaching movements for a wide range of efforts.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Moving, fast and slow: behavioural insights into bradykinesia in Parkinson's disease

The debilitating symptoms of Parkinson's disease, including the hallmark slowness of movement, termed bradykinesia, were described more than 100 years ago. Despite significant advances in elucidating the genetic, molecular and neurobiological changes in Parkinson's disease, it remains conceptually unclear exactly why patients with Parkinson's disease move slowly. To address this, we summarize behavioural observations of movement slowness in Parkinson's disease and discuss these findings in a behavioural framework of optimal control. In this framework, agents optimize the time it takes to gather and harvest rewards by adapting their movement vigour according to the reward that is at stake and the effort that needs to be expended. Thus, slow movements can be favourable when the reward is deemed unappealing or the movement very costly. While reduced reward sensitivity, which makes patients less inclined to work for reward, has been reported in Parkinson's disease, this appears to be related mainly to motivational deficits (apathy) rather than bradykinesia. Increased effort sensitivity has been proposed to underlie movement slowness in Parkinson's disease. However, careful behavioural observations of bradykinesia are inconsistent with abnormal computations of effort costs due to accuracy constraints or movement energetic expenditure. These inconsistencies can be resolved when considering that a general disability to switch between stable and dynamic movement states can contribute to an abnormal composite effort cost related to movement in Parkinson's disease. This can account for paradoxical observations such as the abnormally slow relaxation of isometric contractions or difficulties in halting a movement in Parkinson's disease, both of which increase movement energy expenditure. A sound understanding of the abnormal behavioural computations mediating motor impairment in Parkinson's disease will be vital for linking them to their underlying neural dynamics in distributed brain networks and for grounding future experimental studies in well-defined behavioural frameworks.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

Effort cost of reaching prompts vigor reduction in older adults

As people age, they move slower. Is age-related reduction in vigor a reflection of a reduced valuation of reward by the brain, or a consequence of increased effort costs by the muscles? Here, we quantified cost of movements objectively via the metabolic energy that young and old participants consumed during reaching and found that in order reach at a given speed, older adults expended more energy than the young. We next quantified how reward modulated movements in the same populations and found that like the young, older adults responded to increased reward by initiating their movements earlier. Yet, their movements were less sensitive to increased reward, resulting in little or no modulation of reach speed. Lastly, we quantified the effect of increased effort on how reward modulated movements in young adults. Like the effects of aging, when faced with increased effort the young adults responded to reward primarily by reacting faster, with little change in movement speed. Therefore, reaching required greater energetic expenditure in the elderly, suggesting that the slower movements and reactions exhibited in aging are partly driven by an adaptive response to an elevation in the energetic landscape of effort. That is, moving slower appears to be a rational economic consequence of aging.

Significance statement

Healthy aging coincides with a reduction in speed, or vigor, of walking, reaching, and eye movements. Here we focused on disentangling two opposing sources of aging-related movement slowing: reduced reward sensitivity due to loss of dopaminergic tone, or increased energy expenditure movements related to mitochondrial or muscular inefficiencies. Through a series of three experiments and construction of a computational model, here we demonstrate that transient changes in reaction time and movement speed together offer a quantifiable metric to differentiate between reward- and effort-based alterations in movement vigor. Further, we suggest that objective increases in the metabolic cost of moving, not reductions in reward valuation, are driving much of the movement slowing occurring alongside healthy aging.

Motor Decision-Making as a Common Denominator in Motor Pathology and a Possible Rehabilitation Target

Despite the substantial progress in motor rehabilitation, patient involvement and motivation remain major challenges. They are typically addressed with communicational and environmental strategies, as well as with improved goal-setting procedures. Here we suggest a new research direction and framework involving Neuroeconomics principles to investigate the role of Motor Decision-Making (MDM) parameters in motivational component and motor performance in rehabilitation. We argue that investigating NE principles could bring new approaches aimed at increasing active patient engagement in the rehabilitation process by introducing more movement choice, and adapting existing goal-setting procedures. We discuss possible MDM implementation strategies and illustrate possible research directions using examples of stroke and psychiatric disorders.

Evidence for a common mechanism supporting invigoration of action selection and action execution

The speed, or vigor, of our movements can vary depending on circumstances. For instance, the promise of a reward leads to faster movements. Reward also leads us to move with a lower reaction time, suggesting that the process of action selection can also be invigorated by reward. It has been proposed that invigoration of action selection and of action execution might occur through a common mechanism, and thus these aspects of behavior might be coupled. To test this hypothesis, we asked participants to make reaching movements to "shoot" through a target at varying speeds to assess whether moving more quickly was also associated with more rapid action selection. We found that, when participants were required to move with a lower velocity, the speed of their action selection was also significantly slowed. This finding was recapitulated in a further dataset in which participants determined their own movement speed, but had to move slowly to stop their movement inside the target. By reanalyzing a previous dataset, we also found evidence for the converse relationship between action execution and action selection; when pressured to select actions more rapidly, people also executed movements with higher velocity. Our results establish that invigoration of action selection and action execution vary in tandem with one another, supporting the hypothesis of a common underlying mechanism.NEW & NOTEWORTHY We show that voluntary increases in the vigor of action execution lead action selection to also occur more rapidly. Conversely, hastening action selection by imposing a deadline to act also leads to increases in movement speed. These findings provide evidence that these two distinct aspects of behavior are modulated by a common underlying mechanism.

From Motivation to Action: Action Cost Better Predicts Changes in Premovement Beta-Band Activity than Speed

Although premovement beta-band event-related desynchronization (β-ERD; 13-30 Hz) from sensorimotor regions is modulated by movement speed, current evidence does not support a strict monotonic association between the two. Given that β-ERD is thought to increase information encoding capacity, we tested the hypothesis that it might be related to the expected neurocomputational cost of movement, here referred to as action cost. Critically, action cost is greater both for slow and fast movements compared with a medium or "preferred" speed. Thirty-one right-handed participants performed a speed-controlled reaching task while recording their EEG. Results revealed potent modulations of beta power as a function of speed, with β-ERD being significantly greater both for movements performed at high and low speeds compared with medium speed. Interestingly, medium-speed movements were more often chosen by participants than low-speed and high-speed movements, suggesting that they were evaluated as less costly. In line with this, modeling of action cost revealed a pattern of modulation across speed conditions that strikingly resembled the one found for β-ERD. Indeed, linear mixed models showed that estimated action cost predicted variations of β-ERD significantly better than speed. This relationship with action cost was specific to beta power, as it was not found when averaging activity in the mu band (8-12 Hz) and gamma band (31-49 Hz) bands. These results demonstrate that increasing β-ERD may not merely speed up movements, but instead facilitate the preparation of high-speed and low-speed movements through the allocation of additional neural resources, thereby enabling flexible motor control.SIGNIFICANCE STATEMENT Heightened beta activity has been associated with movement slowing in Parkinson's disease, and modulations of beta activity are commonly used to decode movement parameters in brain-computer interfaces. Here we show that premovement beta activity is better explained by the neurocomputational cost of the action rather than its speed. Instead of being interpreted as a mere reflection of changes in movement speed, premovement changes in beta activity might therefore be used to infer the amount of neural resources that are allocated for motor planning.

Task-Related Differences in End-Point Kinematics in School-Age Children with Typical Development

Understanding whether and how children with typical development adapt their reaches for different functional tasks could inform a more targeted design of rehabilitation interventions to improve upper extremity function in children with motor disabilities. This prospective study compares timing and coordination of a reach-to-drink, reach-to-eat, and a bilateral reaching task in typically developing school-aged children. Average speed, straightness, and smoothness of hand movements were measured in a convenience sample of 71 children, mean age 8.77 ± 0.48 years. Linear mixed models for repeated measures compared the variables by task, phases of the reach, task x phase interactions, and dominant versus non-dominant hands. There were significant main effects for task and phase, significant task x phase interactions (p < 0.05), and a significant difference between the dominant and non-dominant hand for straightness. Hand movements were fastest and smoothest for the reach-to-eat task, and least straight for the bilateral reaching task. Hand movements were also straighter in the object transport phases than the prehension and withdrawal phases. These results indicate that children with typical development change their timing and coordination of reach based on the task they are performing. These results can inform the design of rehabilitation interventions targeting arm and hand function.

Cognitive effort and active inference

This paper aims to integrate some key constructs in the cognitive neuroscience of cognitive control and executive function by formalising the notion of cognitive (or mental) effort in terms of active inference. To do so, we call upon a task used in neuropsychology to assess impulse inhibition-a Stroop task. In this task, participants must suppress the impulse to read a colour word and instead report the colour of the text of the word. The Stroop task is characteristically effortful, and we unpack a theory of mental effort in which, to perform this task accurately, participants must overcome prior beliefs about how they would normally act. However, our interest here is not in overt action, but in covert (mental) action. Mental actions change our beliefs but have no (direct) effect on the outside world-much like deploying covert attention. This account of effort as mental action lets us generate multimodal (choice, reaction time, and electrophysiological) data of the sort we might expect from a human participant engaging in this task. We analyse how parameters determining cognitive effort influence simulated responses and demonstrate that-when provided only with performance data-these parameters can be recovered, provided they are within a certain range.

Metabolic costs of walking and arm reaching in persons with mild multiple sclerosis

Movement slowness is a common and disruptive symptom of multiple sclerosis (MS). A potential cause is that individuals with MS slow down to conserve energy as a behavioral adjustment to heightened metabolic costs of movement. To investigate this prospect, we measured the metabolic costs of both walking and seated arm reaching at five speeds in persons with mild MS (pwMS; n = 13; 46.0 ± 7.7 yr) and sex- and age-matched controls (HCs; n = 13; 45.8 ± 7.8 yr). Notably, the cohort of pwMS was highly mobile and no individuals required a cane or aid when walking. We found that the net metabolic power of walking was approximately 20% higher for pwMS across all speeds (P = 0.0185). In contrast, we found no differences in the gross power of reaching between pwMS and HCs (P = 0.492). Collectively, our results suggest that abnormal slowness of movement in MS-particularly reaching-is not the consequence of heightened effort costs and that other sensorimotor mechanisms are playing a considerable role in slowing.NEW & NOTEWORTHY Individuals with multiple sclerosis (MS) often move more slowly than those without the disease. A possible cause is that movements in MS are more energetically expensive and slowing is an adaptation to conserve metabolic resources. Here, we find that while walking is more costly for persons with MS, arm-reaching movements are not. These results bring into question the driving force of movement slowness in MS and implicate other motor-related networks contributing to slowing.

Modulation of Motor Vigor by Expectation of Reward Probability Trial-by-Trial Is Preserved in Healthy Ageing and Parkinson's Disease Patients

Motor improvements, such as faster movement times or increased velocity, have been associated with reward magnitude in deterministic contexts. Yet whether individual inferences on reward probability influence motor vigor dynamically remains undetermined. We investigated how dynamically inferring volatile action-reward contingencies modulated motor performance trial-by-trial. We conducted three studies that coupled a reversal learning paradigm with a motor sequence task and used a validated hierarchical Bayesian model to fit trial-by-trial data. In Study 1, we tested healthy younger [HYA; 37 (24 females)] and older adults [HOA; 37 (17 females)], and medicated Parkinson's disease (PD) patients [20 (7 females)]. We showed that stronger predictions about the tendency of the action-reward contingency led to faster performance tempo, commensurate with movement time, on a trial-by-trial basis without robustly modulating reaction time (RT). Using Bayesian linear mixed models, we demonstrated a similar invigoration effect on performance tempo in HYA, HOA, and PD, despite HOA and PD being slower than HYA. In Study 2 [HYA, 39 (29 females)], we additionally showed that retrospective subjective inference about credit assignment did not contribute to differences in motor vigor effects. Last, Study 3 [HYA, 33 (27 females)] revealed that explicit beliefs about the reward tendency (confidence ratings) modulated performance tempo trial-by-trial. Our study is the first to reveal that the dynamic updating of beliefs about volatile action-reward contingencies positively biases motor performance through faster tempo. We also provide robust evidence for a preserved sensitivity of motor vigor to inferences about the action-reward mapping in aging and medicated PD.SIGNIFICANCE STATEMENT Navigating a world rich in uncertainty relies on updating beliefs about the probability that our actions lead to reward. Here, we investigated how inferring the action-reward contingencies in a volatile environment modulated motor vigor trial-by-trial in healthy younger and older adults, and in Parkinson's disease (PD) patients on medication. We found an association between trial-by-trial predictions about the tendency of the action-reward contingency and performance tempo, with stronger expectations speeding the movement. We additionally provided evidence for a similar sensitivity of performance tempo to the strength of these predictions in all groups. Thus, dynamic beliefs about the changing relationship between actions and their outcome enhanced motor vigor. This positive bias was not compromised by age or Parkinson's disease.