Fractionation of the visuomotor feedback response to directions of movement and perturbation

Behavioral Motor Performance

The human sensorimotor control system has exceptional abilities to perform skillful actions. We easily switch between strenuous tasks that involve brute force, such as lifting a heavy sewing machine, and delicate movements such as threading a needle in the same machine. Using a structure with different control architectures, the motor system is capable of updating its ability to perform through our daily interaction with the fluctuating environment. However, there are issues that make this a difficult computational problem for the brain to solve. The brain needs to control a nonlinear, nonstationary neuromuscular system, with redundant and occasionally undesired degrees of freedom, in an uncertain environment using a body in which information transmission is subject to delays and noise. To gain insight into the mechanisms of motor control, here we survey movement laws and invariances that shape our everyday motion. We then examine the major solutions to each of these problems in the three parts of the sensorimotor control system, sensing, planning, and acting. We focus on how the sensory system, the control architectures, and the structure and operation of the muscles serve as complementary mechanisms to overcome deviations and disturbances to motor behavior and give rise to skillful motor performance. We conclude with possible future research directions based on suggested links between the operation of the sensorimotor system across the movement stages. © 2024 American Physiological Society. Compr Physiol 14:5179-5224, 2024.

A robot-based interception task to quantify upper limb impairments in proprioceptive and visual feedback after stroke

BACKGROUND

A key motor skill is the ability to rapidly interact with our dynamic environment. Humans can generate goal-directed motor actions in response to sensory stimulus within ~ 60-200ms. This ability can be impaired after stroke, but most clinical tools lack any measures of rapid feedback processing. Reaching tasks have been used as a framework to quantify impairments in generating motor corrections for individuals with stroke. However, reaching may be inadequate as an assessment tool as repeated reaching can be fatiguing for individuals with stroke. Further, reaching requires many trials to be completed including trials with and without disturbances, and thus, exacerbate fatigue. Here, we describe a novel robotic task to quantify rapid feedback processing in healthy controls and compare this performance with individuals with stroke to (more) efficiently identify impairments in rapid feedback processing.

METHODS

We assessed a cohort of healthy controls (n = 135) and individuals with stroke (n = 40; Mean 41 days from stroke) in the Fast Feedback Interception Task (FFIT) using the Kinarm Exoskeleton robot. Participants were instructed to intercept a circular white target moving towards them with their hand represented as a virtual paddle. On some trials, the arm could be physically perturbed, the target or paddle could abruptly change location, or the target could change colour requiring the individual to now avoid the target.

RESULTS

Most participants with stroke were impaired in reaction time (85%) and end-point accuracy (83%) in at least one of the task conditions, most commonly with target or paddle shifts. Of note, this impairment was also evident in most individuals with stroke when performing the task using their unaffected arm (75%). Comparison with upper limb clinical measures identified moderate correlations with the FFIT.

CONCLUSION

The FFIT was able to identify a high proportion of individuals with stroke as impaired in rapid feedback processing using either the affected or unaffected arms. The task allows many different types of feedback responses to be efficiently assessed in a short amount of time.

The sensorimotor system modulates muscular co-contraction relative to visuomotor feedback responses to regulate movement variability

The naturally occurring variability in our movements often poses a significant challenge when attempting to produce precise and accurate actions, which is readily evident when playing a game of darts. Two differing, yet potentially complementary, control strategies that the sensorimotor system may use to regulate movement variability are impedance control and feedback control. Greater muscular co-contraction leads to greater impedance that acts to stabilize the hand, while visuomotor feedback responses can be used to rapidly correct for unexpected deviations when reaching toward a target. Here, we examined the independent roles and potential interplay of impedance control and visuomotor feedback control when regulating movement variability. Participants were instructed to perform a precise reaching task by moving a cursor through a narrow visual channel. We manipulated cursor feedback by visually amplifying movement variability and/or delaying the visual feedback of the cursor. We found that participants decreased movement variability by increasing muscular co-contraction, aligned with an impedance control strategy. Participants displayed visuomotor feedback responses during the task but, unexpectedly, there was no modulation between conditions. However, we did find a relationship between muscular co-contraction and visuomotor feedback responses, suggesting that participants modulated impedance control relative to feedback control. Taken together, our results highlight that the sensorimotor system modulates muscular co-contraction, relative to visuomotor feedback responses, to regulate movement variability and produce accurate actions.NEW & NOTEWORTHY The sensorimotor system has the constant challenge of dealing with the naturally occurring variability in our movements. Here, we investigated the potential roles of muscular co-contraction and visuomotor feedback responses to regulate movement variability. When we visually amplified movements, we found that the sensorimotor system primarily uses muscular co-contraction to regulate movement variability. Interestingly, we found that muscular co-contraction was modulated relative to inherent visuomotor feedback responses, suggesting an interplay between impedance and feedback control.

The Effect of Target Velocity on the Fast Corrective Response during Reaching Movement

Online motor control is often required to correct errors in rapid adjustments during reaching movements. It has been established that the initial arm trajectory during reaching is corrected by a target displacement. Since this corrective response occurs without perception of target perturbation, this is regarded as an automatic response. However, an object rarely "jumps" in daily life, rather it often "moves" as a chronological change of the position that causes visual motion. Therefore, the purpose of this study was to investigate whether the implicit visuomotor response is induced by target motion stimuli and to clarify the effects of target motion velocity on initial arm trajectory. Participants were asked to move a cursor from a start circle to a visual target. The target moved either leftward or rightward when the cursor passed 20 mm from the start circle. Four target velocities (10, 20, 30, 40 deg/s) were randomly presented. Our results showed that the initial velocity (first 50 ms) of the fast corrective response increased with the target velocity. Therefore, it is indicated that the fast corrective response is induced by the target motion stimulus with a short latency and its amplitude is dependent on the target velocity.

Integration of proprioceptive and visual feedback during online control of reaching

Visual and proprioceptive feedback both contribute to perceptual decisions, but it remains unknown how these feedback signals are integrated together or consider factors such as delays and variance during online control. We investigated this question by having participants reach to a target with randomly applied mechanical and/or visual disturbances. We observed that the presence of visual feedback during a mechanical disturbance did not increase the size of the muscle response significantly but did decrease variance, consistent with a dynamic Bayesian integration model. In a control experiment, we verified that vision had a potent influence when mechanical and visual disturbances were both present but opposite in sign. These results highlight a complex process for multisensory integration, where visual feedback has a relatively modest influence when the limb is mechanically disturbed, but a substantial influence when visual feedback becomes misaligned with the limb.

NEW & NOTEWORTHY Visual feedback is more accurate, but proprioceptive feedback is faster. How should you integrate these sources of feedback to guide limb movement? As predicted by dynamic Bayesian models, the size of the muscle response to a mechanical disturbance was essentially the same whether visual feedback was present or not. Only under artificial conditions, such as when shifting the position of a cursor representing hand position, can one observe a muscle response from visual feedback.

Timescales of motor memory formation in dual-adaptation

The timescales of adaptation to novel dynamics are well explained by a dual-rate model with slow and fast states. This model can predict interference, savings and spontaneous recovery, but cannot account for adaptation to multiple tasks, as each new task drives unlearning of the previously learned task. Nevertheless, in the presence of appropriate contextual cues, humans are able to adapt simultaneously to opposing dynamics. Consequently this model was expanded, suggesting that dual-adaptation occurs through a single fast process and multiple slow processes. However, such a model does not predict spontaneous recovery within dual-adaptation. Here we assess the existence of multiple fast processes by examining the presence of spontaneous recovery in two experimental variations of an adaptation-de-adaptation-error-clamp paradigm within dual-task adaptation in humans. In both experiments, evidence for spontaneous recovery towards the initially learned dynamics (A) was found in the error-clamp phase, invalidating the one-fast-two-slow dual-rate model. However, as adaptation is not only constrained to two timescales, we fit twelve multi-rate models to the experimental data. BIC model comparison again supported the existence of two fast processes, but extended the timescales to include a third rate: the ultraslow process. Even within our single day experiment, we found little evidence for decay of the learned memory over several hundred error-clamp trials. Overall, we show that dual-adaptation can be best explained by a two-fast-triple-rate model over the timescales of adaptation studied here. Longer term learning may require even slower timescales, explaining why we never forget how to ride a bicycle.

Retaining motor skills is crucial to perform basic daily life tasks. However we still have limited understanding of the computational structure of these motor memories, an understanding that is critical for designing rehabilitation. Here we demonstrate that learning any task involves adaptation of independent fast, slow and ultraslow processes to build a motor memory. The selection of the appropriate motor memory is gated through a contextual cue. Together this work extends our understanding of the architecture of motor memories, by merging disparate computational theories to propose a new model.

Time-to-Target Simplifies Optimal Control of Visuomotor Feedback Responses

Visuomotor feedback responses vary in intensity throughout a reach, commonly explained by optimal control. Here, we show that the optimal control for a range of movements with the same goal can be simplified to a time-to-target dependent control scheme. We measure our human participants’ visuomotor responses in five reaching conditions, each with different hand or cursor kinematics. Participants only produced different feedback responses when these kinematic changes resulted in different times-to-target. We complement our experimental data with a range of finite and non-finite horizon optimal feedback control (OFC) models, finding that the model with time-to-target as one of the input parameters best replicates the experimental data. Overall, this suggests that time-to-target is a critical control parameter in online feedback control. Moreover, we propose that for a specific task and known dynamics, humans can instantly produce a control signal without any additional online computation allowing rapid response onset and close to optimal control.

Feedback Delay Changes the Control of an Inverted Pendulum

We recently developed a simulated inverted pendulum in order to examine human sensorimotor control strategies for stabilization. This simulated system allows us to manipulate the visual and haptic feedback independently from the physical dynamics of the task. Here we examine the effect of sensory delay in a balancing task. Human participants attempted to balance an inverted pendulum (simulated on a robotic manipulandum) with three different added delays (25, 50, and 75 ms), where the same delay was added to both the visual and haptic feedback. Increasing sensory delays decreased the ability of the participants to stabilize the pendulum. Investigation into the online control of the pendulum showed that with longer delays participants reduced their movement frequency but increased the amplitudes of their corrections.

Visual Feedback Processing of the Limb Involves Two Distinct Phases

Muscle responses to mechanical disturbances exhibit two distinct phases: a response starting at ~20 ms that is fairly stereotyped, and a response starting at ~60 ms modulated by many behavioral contexts including goal-redundancy and environmental obstacles. Muscle responses to disturbances of visual feedback of the hand arise within ~90 ms. However, little is known whether these muscle responses are sensitive to behavioral contexts. We had 49 human participants (27 male) execute goal-directed reaches with visual feedback of their hand presented as a cursor. On random trials, the cursor jumped laterally to the reach direction, and thus, required a correction to attain the goal. The first experiment demonstrated that the response amplitude starting at 90 ms scaled with jump magnitude, but only for jumps <2 cm. For larger jumps, the duration of the muscle response scaled with the jump size starting after 120 ms. The second experiment demonstrated that the early response was sensitive to goal redundancy as wider targets evoked a smaller corrective response. The third experiment demonstrated that the early response did not consider the presence of obstacles, as this response routinely drove participants directly to the goal even though this path was blocked by an obstacle. Instead, the appropriate muscle response to navigate around the obstacle started after 120 ms. Our findings highlight that visual feedback of the limb involves two distinct phases: a response starting at 90 ms with limited sensitivity to jump magnitude and sensitive to goal-redundancy, and a response starting at 120 ms with increased sensitivity to jump magnitude and environmental factors.SIGNIFICANCE STATEMENT The motor system can integrate proprioceptive feedback to guide an ongoing action in ~60 ms and is flexible to a broad range of behavioral contexts. In contrast, the present study identified that the motor response to a visual disturbance exhibits two distinct phases: an early response starting at 90 ms with limited scaling with disturbance size and sensitivity to goal-redundancy, and a slower response starting after 120 ms with increased sensitivity to disturbance size and sensitive to environmental obstacles. These data suggest visual feedback of the hand is processed through two distinct feedback processes.

A Technique for Measuring Visuomotor Feedback Contributions to the Control of an Inverted Pendulum

We developed a new technique to measure the contributions of rapid visuomotor feedback responses to the stabilization of a simulated inverted pendulum. Human participants balanced an inverted pendulum simulated on a robotic manipulandum. At a random time during the balancing task, the visual representation of the tip of the pendulum was shifted by a small displacement to the left or right while the motor response was measured. This response was either the exerted force against a fixation position, or the motion to re-stabilize the pendulum in the free condition. Our results demonstrate that rapid involuntary visuomotor feedback responses contribute to the stabilization of the pendulum.

Contribution of the Cerebellum to Predictive Motor Control and Its Evaluation in Ataxic Patients

Goal-directed movements are predictive and multimodal in nature, especially for moving targets. For instance, during a reaching movement for a moving target, humans need to predict both motion of the target and movement of the limb. Recent computational studies show that the cerebellum predicts current and future states of the body and its environment using internal forward models. Sensory feedback signals from the periphery have delays in reaching the central nervous system, ranging between tens to hundreds of milliseconds. It is well known in engineering that feedback control based on time-delayed inputs can result in oscillatory and often unstable movements. In contrast, the brain predicts a current state from a previous state using forward models. This predictive mechanism most likely underpins stable and dexterous control of reaching movements. Although the cerebro-cerebellum has long been suggested as loci of various forward models, few methods are available to evaluate accuracy of the forward models in patients with cerebellar ataxia. Recently, we developed a non-invasive method to analyze receipt of motor commands in terms of movement kinematics for the wrist joint (B_r/K_r ratio). In the present study, we have identified two components (F1 and F2) of the smooth pursuit movement. We found that the two components were in different control modes with different B_r/K_r ratios. The major F1 component in a lower frequency range encodes both velocity and position of the moving target (higher B_r/K_r ratio) to synchronize movement of the wrist joint with motion of the target in a predictive manner. The minor F2 component in a higher frequency range is biased to position control in order to generate intermittent small step-wise movements. In cerebellar patients, the F1 component shows a selective decrease in the B_r/K_r ratio, which is correlated with decrease in accuracy of the pursuit movement. We conclude that the B_r/K_r ratio of the F1 component provides a unique parameter to evaluate accuracy of the predictive control. We also discuss the pathophysiological and clinical implications for clinical ataxiology.

Rapid Visuomotor Responses Reflect Value-Based Decisions

Cognitive decision-making is known to be sensitive to the values of potential options, which are the probability and size of rewards associated with different choices. Here, we examine whether rapid motor responses to perturbations of visual feedback about movement, which mediate low-level and involuntary feedback control loops, reflect computations associated with high-level value-based decision-making. In three experiments involving human participants, we varied the value associated with different potential targets for reaching movements by controlling the distributions of rewards across the targets (Experiment 1), the probability with which each target could be specified (Experiment 2), or both (Experiment 3). We found that the size of rapid and involuntary feedback responses to movement perturbations was strongly influenced by the relative value between targets. A statistical model of relative value that includes a term for risk sensitivity provided the best fit to the visuomotor response data, illustrating that feedback control policies are biased to favor more frequent task success at the expense of the overall extrinsic reward accumulated through movement. Importantly however, the regulation of rapid feedback responses was associated with successful pursuit of high-value task outcomes. This implies that when we move, the brain specifies a set of feedback control gains that enable low-level motor areas not only to generate efficient and accurate movement, but also to rapidly and adaptively respond to evolving sensory information in a manner consistent with value-based decision-making.SIGNIFICANCE STATEMENT Current theories of sensorimotor control suggest that, rather than selecting and planning the details of movements in advance, the role of the brain is to set time-varying feedback gains that continuously transform sensory information into motor commands by feedback control. Here, we examine whether the fastest motor responses to perturbations of movement, which mediate low-level and involuntary feedback control loops (i.e., reflexes), reflect computations associated with high-level, value-based decision-making. We find that rapid feedback responses during reaching reflect the relative probabilities and rewards associated with target options. This suggests that low-order components of the sensorimotor control hierarchy, which generate rapid and automatic responses, can continuously evaluate evolving sensory evidence and initiate responses according to the prospect of reward.

Influence of Visual Feedback on the Sensorimotor Control of an Inverted Pendulum

We examine the visual influence of stabilization in human sensorimotor control using a simulated inverted pendulum. As the inverted pendulum is fully simulated, we are able to manipulate the visual feedback independently from the dynamics during the motor control task. Human subjects performed a balancing task of an upright pendulum on a robotic manipulandum in two different visual feedback conditions. First we examined how subjects perform a task where the visual feedback is congruent with the pendulum dynamics. Second we tested how subjects performed when the physical dynamics were fixed but the visual feedback of the pendulum length was modulated. Subjects exhibited deficits in the control of the pendulum when haptic and visual feedback did not match, even when the visual feedback provided more sensitive information about the state of the pendulum. Overall we demonstrate the importance of accurate feedback regarding task dynamics for stabilization.

Correlations Between Primary Motor Cortex Activity with Recent Past and Future Limb Motion During Unperturbed Reaching

Many studies highlight that human movements are highly successful yet display a surprising amount of variability from trial to trial. There is a consistent pattern of variability throughout movement: initial motor errors are corrected by the end of movement, suggesting the presence of a powerful online control process. Here, we analyze the trial-by-trial variability of goal-directed reaching in nonhuman primates (five male Rhesus monkeys) and demonstrate that they display a similar pattern of variability during reaching, including a strong negative correlation between initial and late hand motion. We then demonstrate that trial-to-trial neural variability of primary motor cortex (M1) is positively correlated with variability of future hand motion (τ = ∼160 ms) during reaching. Furthermore, the variability of M1 activity is also correlated with variability of past hand motion (τ = ∼90 ms), but in the opposite polarity (i.e., negative correlation). Partial correlation analysis demonstrated that M1 activity independently reflects the variability of both past and future hand motions. These findings provide support for the hypothesis that M1 activity is involved in online feedback control of motor actions.SIGNIFICANCE STATEMENT Previous studies highlight that primary motor cortex (M1) rapidly responds to either visual or mechanical disturbances, suggesting its involvement in online feedback control. However, these studies required external disturbances to the motor system and it is not clear whether a similar feedback process addresses internal noise/errors generated by the motor system itself. Here, we introduce a novel analysis that evaluates how variations in the activity of M1 neurons covary with variations in hand motion on a trial-to-trial basis. The analyses demonstrate that M1 activity is correlated with hand motion in both the near future and the recent past, but with opposite polarity. These results suggest that M1 is involved in online feedback motor control to address errors/noise within the motor system.

Rapid visuomotor feedback gains are tuned to the task dynamics

Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Adaptation to novel dynamics requires learning a motor memory, or a new pattern of predictive feedforward motor commands. Recently, we demonstrated the upregulation of rapid visuomotor feedback gains early in curl force field learning, which decrease once a predictive motor memory is learned. However, even after learning is complete, these feedback gains are higher than those observed in the null field trials. Interestingly, these upregulated feedback gains in the curl field were not observed in a constant force field. Therefore, we suggest that adaptation also involves selectively tuning the feedback sensitivity of the sensorimotor control system to the environment. Here, we test this hypothesis by measuring the rapid visuomotor feedback gains after subjects adapt to a variety of novel dynamics generated by a robotic manipulandum in three experiments. To probe the feedback gains, we measured the magnitude of the motor response to rapid shifts in the visual location of the hand during reaching. While the feedback gain magnitude remained similar over a larger than a fourfold increase in constant background load, the feedback gains scaled with increasing lateral resistance and increasing instability. The third experiment demonstrated that the feedback gains could also be independently tuned to perturbations to the left and right, depending on the lateral resistance, demonstrating the fractionation of feedback gains to environmental dynamics. Our results show that the sensorimotor control system regulates the gain of the feedback system as part of the adaptation process to novel dynamics, appropriately tuning them to the environment.

NEW & NOTEWORTHY Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Sensorimotor Learning: Neurocognitive Mechanisms and Individual Differences

Here we provide an overview of findings and viewpoints on the mechanisms of sensorimotor learning presented at the 2016 Biomechanics and Neural Control of Movement (BANCOM) conference in Deer Creek, OH. This field has shown substantial growth in the past couple of decades. For example it is now well accepted that neural systems outside of primary motor pathways play a role in learning. Frontoparietal and anterior cingulate networks contribute to sensorimotor adaptation, reflecting strategic aspects of exploration and learning. Longer term training results in functional and morphological changes in primary motor and somatosensory cortices. Interestingly, re-engagement of strategic processes once a skill has become well learned may disrupt performance.

Efforts to predict individual differences in learning rate have enhanced our understanding of the neural, behavioral, and genetic factors underlying skilled human performance. Access to genomic analyses has dramatically increased over the past several years. This has enhanced our understanding of cellular processes underlying the expression of human behavior, including involvement of various neurotransmitters, receptors, and enzymes. Surprisingly our field has been slow to adopt such approaches in studying neural control, although this work does require much larger sample sizes than are typically used to investigate skill learning. We advocate that individual differences approaches can lead to new insights into human sensorimotor performance. Moreover, a greater understanding of the factors underlying the wide range of performance capabilities seen across individuals can promote personalized medicine and refinement of rehabilitation strategies, which stand to be more effective than “one size fits all” treatments.

Temporal Evolution of Spatial Computations for Visuomotor Control

Goal-directed reaching movements are guided by visual feedback from both target and hand. The classical view is that the brain extracts information about target and hand positions from a visual scene, calculates a difference vector between them, and uses this estimate to control the movement. Here we show that during fast feedback control, this computation is not immediate, but evolves dynamically over time. Immediately after a change in the visual scene, the motor system generates independent responses to the errors in hand and target location. Only about 200 ms later, the changes in target and hand positions are combined appropriately in the response, slowly converging to the true difference vector. Therefore, our results provide evidence for the temporal evolution of spatial computations in the human visuomotor system, in which the accurate difference vector computation is first estimated by a fast approximation.

Computations underlying sensorimotor learning

The study of sensorimotor learning has a long history. With the advent of innovative techniques for studying learning at the behavioral and computational levels new insights have been gained in recent years into how the sensorimotor system acquires, retains, represents, retrieves and forgets sensorimotor tasks. In this review we highlight recent advances in the field of sensorimotor learning from a computational perspective. We focus on studies in which computational models are used to elucidate basic mechanisms underlying adaptation and skill acquisition in human behavior.

Error Detection is Critical for Visual-Motor Corrections

The target article (Smeets, Oostwoud Wijdenes, & Brenner, 2016) proposes that short latency responses to changes in target location during reaching reflect an unconscious, continuous, and incremental minimization of the distance between the hand and the target, which does not require detection of the change in target location. We, instead, propose that short-latency visuomotor responses invoke reflex- or startle-like mechanisms, an idea supported by evidence that such responses are both automatic and resistant to cognitive influences. In addition, the target article fails to address the biological underpinnings for the range of response latencies reported across the literature, including the circuits that might underlie the proposed sensorimotor loops. When considering the range of latencies reported in the literature, we propose that mechanisms grounded in neurophysiology should be more informative than the simple information processing perspective adopted by the target article.

Impedance control: Learning stability in human sensorimotor control

The human sensorimotor control system generates movement by adapting and controlling the mechanics of the musculoskeletal system. To generate skilful movements the sensorimotor control system must be able to predict and compensate for any disturbances generated either in our own body or in the external environment. While stable and repeatable perturbations can be easily adapted through iterative learning, instability and unpredictability require a different approach: impedance control. Here I outline the arguments for impedance control as a fundamental process of human adaptation as well as describe evidence suggesting the manner in which such impedance can be learned in order to ensure the stability of the neuro-mechanical system.

Computations in Sensorimotor Learning

Our cognitive abilities can only be expressed on the world through our actions. Here we review the computations underlying the way that the sensorimotor system converts both low-level sensory signals and high-level decisions into action, focusing on the behavioral evidence for the theoretical frameworks. We review recent work that determines how motor memories underlying sensorimotor learning are activated and protected from interference, the role of Bayesian decision theory in sensorimotor control including sources of suboptimality, the role of risk sensitivity in guiding action, and how rapid motor responses may underlie the robustness of the motor system to the vagaries of the world.