Reward timing matters in motor learning

Tiny visual latencies can profoundly impair implicit sensorimotor learning

Short sub-100 ms visual feedback latencies are common in many types of human-computer interactions yet are known to markedly reduce performance in a wide variety of motor tasks from simple pointing to operating surgical robotics. It remains unclear, however, whether these latencies impair not only skilled motor performance but also the implicit sensorimotor learning that underlies its acquisition. Inspired by neurophysiological findings showing that cerebellar LTD and cortical LTP would both be disrupted by sub-100 ms latencies, we hypothesized that implicit sensorimotor learning may be particularly sensitive to these short latencies. Remarkably, we find that improving latency by just 60 ms, from 85 to 25 ms in continuous-feedback experiments, increases implicit learning by 50% and proportionally decreases explicit learning. This resulted in a dramatic reorganization of sensorimotor memory from a 45/55 to a 70/30 implicit/explicit ratio. This 70/30 ratio is more than double that observed in any previous study examining the effect of latency on sensorimotor learning, including a recent study which provided time-advanced visual feedback, suggesting that the low-latency continuous visual feedback we provided is critical for efficiently driving implicit learning. We go on to show that implicit sensorimotor learning is considerably more sensitive to latencies in the sub-100 ms range than to higher latencies, in line with the latency-specific neural plasticity that has been observed. This suggests a clear benefit for latency reduction in computer-based training that involves implicit sensorimotor learning and that across-study differences in computer-based experiments that have examined implicit sensorimotor learning might be explained by differences in unmeasured feedback latencies.

Effect of Extrinsic Reward on Motor Plasticity during Skill Learning

Human motor skill acquisition is improved by performance feedback, and coupling such feedback with extrinsic reward (such as money) can enhance skill learning. However, the neurophysiology underlying such behavioral effect is unclear. To bridge this gap, we assessed the effects of reward on multiple forms of motor plasticity during skill learning. Sixty-five healthy participants divided into three groups performed a pinch-grip skill task with sensory feedback only, sensory and reinforcement feedback, or both feedback coupled with an extrinsic monetary reward during skill training. To probe motor plasticity, we applied transcranial magnetic stimulation at rest, on the left primary motor cortex before, at an early-training time point, and after training in the three groups and measured motor-evoked potentials from task-relevant muscle of the right arm. This allowed us to evaluate the amplitude and variability of corticospinal output, GABAergic short-intracortical inhibition, and use-dependent plasticity before training and at two additional time points (early and end training). At the behavioral level, monetary reward accelerated skill learning. In parallel, corticospinal output became less variable early on during training in the presence of extrinsic reward. Interestingly, this effect was particularly pronounced for participants who were more sensitive to reward, as evaluated in an independent questionnaire. Other measures of motor excitability remained comparable across groups. These findings highlight that a mechanism underlying the benefit of reward on motor skill learning is the fine-tuning of early-training resting-state corticospinal variability.

Reward signals in the motor cortex: from biology to neurotechnology

Over the past decade, research has shown that the primary motor cortex (M1), the brain's main output for movement, also responds to rewards. These reward signals may shape motor output in its final stages, influencing movement invigoration and motor learning. In this Perspective, we highlight the functional roles of M1 reward signals and propose how they could guide advances in neurotechnologies for movement restoration, specifically brain-computer interfaces and non-invasive brain stimulation. Understanding M1 reward signals may open new avenues for enhancing motor control and rehabilitation.

The Use of Extrinsic Performance Feedback and Reward to Enhance Upper Limb Motor Behavior and Recovery Post-Stroke: A Scoping Review

BACKGROUND

During post-stroke motor rehabilitation, patients often receive feedback from therapists or via rehabilitation technologies. Research suggests that feedback may benefit motor performance, skill acquisition, and action selection. However, there is no consensus on how extrinsic feedback should be implemented during stroke rehabilitation to best leverage specific neurobehavioral mechanisms to optimize recovery.

OBJECTIVE

To identify the existing evidence and research gaps regarding the effects of extrinsic feedback on upper extremity motor function in stroke survivors, and to map the evidence onto neurobehavioral concepts of motor performance, motor learning, and action selection.

METHODS

The MEDLINE, PsychInfo, EMBASE, and CINHAL databases were searched for relevant articles. A sequential screening process and data extraction were performed by 2 independent reviewers, and the results were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews guidelines.

RESULTS

A total of 29 studies were identified that met the criteria for inclusion. Beneficial effects of feedback were reported for clinical outcomes of rehabilitation interventions as well as motor performance, motor learning, and action selection post-stroke. Three studies showed that the addition of rewarding elements to positive performance feedback benefited learning or recovery.

CONCLUSIONS

Extrinsic feedback has the potential to improve outcomes of stroke rehabilitation through effects on motor performance, motor learning, or action selection. To understand how these specific neurobehavioral processes contribute to recovery, clinical trials should include more granular behavioral measures. Rewarding feedback may be particularly beneficial, but more research is needed regarding the specific implementation of feedback.

The effects of haloperidol on motor vigour and movement fusion during sequential reaching

Reward is a powerful tool to enhance human motor behaviour with previous research showing that during a sequential reaching movement, a monetary incentive leads to increased speed of each movement (motor vigour effect), whilst reward-based performance feedback increases the speed of transition between movements (movement fusion effect). The neurotransmitter dopamine plays a central role in the processing of reward signals and has been implicated to modulate motor vigour and regulate movement fusion. However, in humans, it is unclear if the same dopaminergic mechanism underlies both processes. To address this, we used a complex sequential reaching task in which rewards were based on movement times (MT). Crucially, MTs could be reduced via: 1) enhanced speed of individual movements (motor vigour effect) and/or 2) enhanced speed of transition between movements (movement fusion effect). 95 participants were randomly assigned to a reward or no reward group and were given either 2.5mg of the dopamine antagonist haloperidol or a placebo (control group). An independent decision-making task performed prior to the main experiment suggested that haloperidol was active during the sequential reaching task (positive control). We did not find evidence that haloperidol affected the facilitatory effects of reward on movement fusion. However, we found that haloperidol negated the reward-based effects on motor vigour. Therefore, our results suggest that a D2-antagonist differentially influences reward-based effects on movement vigour and movement fusion, indicating that the dopaminergic mechanisms underlying these two processes may be distinct.

The affective response to positive performance feedback is associated with motor learning

Motor skill learning and performance are improved when successful actions are paired with extrinsic rewards, such as money. Positive feedback indicating successful task performance is thought to induce intrinsic reward associated with goal attainment, evidenced by increases in positive affect that correlate with neural reward signaling. However, it is not clear whether the subjective, internal reward processes elicited by positive feedback promote motor learning and performance.Here, we tested the hypothesis that intrinsic reward elicited by positive feedback promotes motor learning and performance. Participants practiced a visuomotor interception task using a joystick, and received feedback during practice indicating success or failure depending on their accuracy. During practice, the accuracy demands were adapted to control and vary the frequency of positive feedback across randomly ordered blocks of practice at either 50%, 70%, or 90%. Performance was measured for each condition as the average accuracy during practice. Learning was estimated by measuring the accuracy pre and post practice in the absence of feedback. We queried participants periodically on their enjoyment of the task to index affective responses to performance feedback.The intrinsic reward elicited by positive feedback, operationalized by the increase in enjoyment immediately following positive versus negative feedback, was positively correlated with learning from pre to post practice. However, increasing the overall amount of positive feedback by lower accuracy demands did not improve performance. These results suggest that experiencing intrinsic reward due to positive feedback benefits motor learning only when it is contingent on good performance.

Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

Subtle Visual Latency Can Profoundly Impair Implicit Sensorimotor Learning

Short sub-100ms visual feedback latencies are common in many types of human-computer interactions yet are known to markedly reduce performance in a wide variety of motor tasks from simple pointing to operating surgical robotics. These latencies are also present in the computer-based experiments used to study the sensorimotor learning that underlies the acquisition of motor performance. Inspired by neurophysiological findings showing that cerebellar LTD and cortical LTP would both be disrupted by sub-100ms latencies, we hypothesized that implicit sensorimotor learning may be particularly sensitive to these short latencies. Remarkably, we find that improving latency by just 60ms, from 85 to 25ms in latency-optimized experiments, increases implicit learning by 50% and proportionally decreases explicit learning, resulting in a dramatic reorganization of sensorimotor memory. We go on to show that implicit sensorimotor learning is considerably more sensitive to latencies in the sub-100ms range than at higher latencies, in line with the latency-specific neural plasticity that has been observed. This suggests a clear benefit for latency reduction in computer-based training that involves implicit sensorimotor learning and that across-study differences in implicit motor learning might often be explained by disparities in feedback latency.

The effect of reward on motor learning: different stage, different effect

Motor learning is a prominent and extensively studied subject in rehabilitation following various types of neurological disorders. Motor repair and rehabilitation often extend over months and years post-injury with a slow pace of recovery, particularly affecting the fine movements of the distal extremities. This extended period can diminish the motivation and persistence of patients, a facet that has historically been overlooked in motor learning until recent years. Reward, including monetary compensation, social praise, video gaming, music, and virtual reality, is currently garnering heightened attention for its potential to enhance motor motivation and improve function. Numerous studies have examined the effects and attempted to explore potential mechanisms in various motor paradigms, yet they have yielded inconsistent or even contradictory results and conclusions. A comprehensive review is necessary to summarize studies on the effects of rewards on motor learning and to deduce a central pattern from these existing studies. Therefore, in this review, we initially outline a framework of motor learning considering two major types, two major components, and three stages. Subsequently, we summarize the effects of rewards on different stages of motor learning within the mentioned framework and analyze the underlying mechanisms at the level of behavior or neural circuit. Reward accelerates learning speed and enhances the extent of learning during the acquisition and consolidation stages, possibly by regulating the balance between the direct and indirect pathways (activating more D1-MSN than D2-MSN) of the ventral striatum and by increasing motor dynamics and kinematics. However, the effect varies depending on several experimental conditions. During the retention stage, there is a consensus that reward enhances both short-term and long-term memory retention in both types of motor learning, attributed to the LTP learning mechanism mediated by the VTA-M1 dopaminergic projection. Reward is a promising enhancer to bolster waning confidence and motivation, thereby increasing the efficiency of motor learning and rehabilitation. Further exploration of the circuit and functional connections between reward and the motor loop may provide a novel target for neural modulation to promote motor behavior.

Motor Decision-Making as a Common Denominator in Motor Pathology and a Possible Rehabilitation Target

Despite the substantial progress in motor rehabilitation, patient involvement and motivation remain major challenges. They are typically addressed with communicational and environmental strategies, as well as with improved goal-setting procedures. Here we suggest a new research direction and framework involving Neuroeconomics principles to investigate the role of Motor Decision-Making (MDM) parameters in motivational component and motor performance in rehabilitation. We argue that investigating NE principles could bring new approaches aimed at increasing active patient engagement in the rehabilitation process by introducing more movement choice, and adapting existing goal-setting procedures. We discuss possible MDM implementation strategies and illustrate possible research directions using examples of stroke and psychiatric disorders.

Reflecting on what is "skill" in human motor skill learning

Humans have an exceptional ability to execute a variety of skilled movements. Researchers have been long interested in understanding behavioral and neurophysiological basis of human motor skill learning for advancing both fundamental neuroscientific knowledge and clinical outcomes. However, despite decades of work in this field there is a lack of consensus about what is meant by "skill" in skill learning. With an advent of various task paradigms testing human motor behavior and increasing heterogeneity in motor learning assessments methods, it is very crucial to identify key features of skill in order to avoid any ambiguity that may result in misinterpretation or over-generalization of findings, which could have serious implications for replication and translational research. In this review, we attempt to highlight the features of skill following a historical approach, considering the seminal work that led to the first definitions of skill and including some contemporary concepts emerging from human motor learning research. Overall, based on this literature, we emphasize that skill has some fundamental characteristics, such as- (i) optimal movement selection and execution, (ii) improved movement speed and accuracy, and (iii) reduced movement variability and error. These features of skill can emerge as a consequence of extensive practice/training/learning, thus resulting in an improved performance state beyond baseline levels. Finally we provide some examples of model tasks that can appropriately capture these features of skill, and conclude that any neuroscientific endeavor aimed at understanding the essence of skill in human motor skill learning should focus on these aspects.