Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses

Human Stiffness Perception and Learning in Interacting With Compliant Environments

Humans are capable of adjusting their posture stably when interacting with a compliant surface. Their whole-body motion can be modulated in order to respond to the environment and reach to a stable state. In perceiving an uncertain external force, humans repetitively push it and learn how to produce a stable state. Research in human motor control has led to the hypothesis that the central nervous system integrates an internal model with sensory feedback in order to generate accurate movements. However, how the brain understands external force through exploration movements, and how humans accurately estimate a force from their experience of the force, is yet to be fully understood. To address these questions, we tested human behaviour in different stiffness profiles even though the force at the goal was the same. We generated one linear and two non-linear stiffness profiles, which required the same force at the target but different forces half-way to the target; we then measured the differences in the learning performance at the target and the differences in perception at the half-way point. Human subjects learned the stiffness profile through repetitive movements in reaching the target, and then indicated their estimation of half of the target value (position and force separately). This experimental design enabled us to probe how perception of the force experienced in different profiles affects the participants' estimations. We observed that the early parts of the learning curves were different for the three stiffness profiles. Secondly, the position estimates were accurate independent of the stiffness profile. The estimation in position was most likely influenced by the external environment rather than the profile itself. Interestingly, although visual information about the target had a large influence, we observed significant differences in accuracy of force estimation according to the stiffness profile.

Perturbation Training for Fall-Risk Reduction in Healthy Older Adults: Interference and Generalization to Opposing Novel Perturbations Post Intervention

This study examined the effects of perturbation training on the contextual interference and generalization of encountering a novel opposing perturbation. One hundred and sixty-nine community-dwelling healthy older adults (69.6 ± 6.4 years) were randomly assigned to one of the three groups: slip-perturbation training (St, n = 67) group received 24 slips, trip-perturbation training (Tt, n = 67) group received 24 trips, and control (Ctrl: n = 31) group received only non-perturbed walking trials (ClinicalTrials.gov NCT03199729; https://clinicaltrials.gov/ct2/show/NCT03199729). After training, all groups had 30 min of rest and three post-training non-perturbed walking trials, followed by a reslip and a novel trip trial for St, a retrip and a novel slip trial for Tt, and randomized novel slip and trip trials for Ctrl. The margin of stability (MOS), step length, and toe clearance of post-training walking trials were compared among three groups to examine interferences in proactive adjustment. Falls, MOS at the instant of recovery foot touchdown, and hip height of post-training perturbation trials were investigated to detect interferences and generalization in reactive responses. Results indicated that prior adaptation to slip perturbation training, resulting in walking with a greater MOS (more anterior) and a shorter step length (p < 0.01) than that of the Ctrl group, would be associated with a greater likelihood to forward balance loss if encountered with a trip. The trip adaptation training mainly induced a higher toe clearance during walking (p < 0.01) than the Ctrl group, which could lead to reduced effectiveness of the reactive response when encountered with a novel slip. However, there was no difference in the reactive MOS, limb support, and falls between the control group and the slip and trip training groups on their respective opposing novel perturbation post-training (MOS, limb support, and falls for novel slip: Tt = Ctrl; for the novel trip: St = Ctrl, both p > 0.05). Current findings suggested that, although perturbation training results in proactive adjustments that could worsen the reactive response (interference) when exposed to an unexpected opposing perturbation, older adults demonstrated the ability to immediately generalize the training-induced adaptive reactive control to maintain MOS, to preserve limb support control, and to reduce fall risk.

Shared internal models for feedforward and feedback control of arm dynamics in non-human primates

Previous work has shown that humans account for and learn novel properties or the arm's dynamics, and that such learning causes changes in both the predictive (i.e., feedforward) control of reaching and reflex (i.e., feedback) responses to mechanical perturbations. Here we show that similar observations hold in old-world monkeys (Macaca fascicularis). Two monkeys were trained to use an exoskeleton to perform a single-joint elbow reaching and to respond to mechanical perturbations that created pure elbow motion. Both of these tasks engaged robust shoulder muscle activity as required to account for the torques that typically arise at the shoulder when the forearm rotates around the elbow joint (i.e., intersegmental dynamics). We altered these intersegmental arm dynamics by having the monkeys generate the same elbow movements with the shoulder joint either free to rotate, as normal, or fixed by the robotic manipulandum, which eliminates the shoulder torques caused by forearm rotation. After fixing the shoulder joint, we found a systematic reduction in shoulder muscle activity. In addition, after releasing the shoulder joint again, we found evidence of kinematic aftereffects (i.e., reach errors) in the direction predicted if failing to compensate for normal arm dynamics. We also tested whether such learning transfers to feedback responses evoked by mechanical perturbations and found a reduction in shoulder feedback responses, as appropriate for these altered arm intersegmental dynamics. Demonstrating this learning and transfer in non-human primates will allow the investigation of the neural mechanisms involved in feedforward and feedback control of the arm's dynamics.

Online corrective responses following target jump in altered gravitoinertial force field point to nested feedforward and feedback control

Studies on goal-directed arm movements have shown a close link between feedforward and feedback control in protocols where both planning and online control processes faced a similar type of perturbation, either mechanical or visual. This particular context might have facilitated the use of an adapted internal model by feedforward and feedback control. Here, we considered this link in a context where, after feedforward control was adapted through proprioception-based processes, feedback control was tested under visual perturbation. We analyzed the response of the reaching hand to target displacements following adaptation to an altered force field induced by rotating participants at constant velocity. Reaching corrections were assessed through variables related to the accuracy (lateral and longitudinal end point errors) and kinematics (movement time, peak velocity) of the corrective movements. The electromyographic activity of different arm muscles (pectoralis, posterior deltoid, biceps brachii, and triceps brachii) was analyzed. Statistical analyses revealed that accuracy and kinematics of corrective movements were strikingly alike between normal and altered gravitoinertial force fields. However, pectoralis and biceps muscle activities recorded during corrective movements were significantly modified to counteract the effect of rotation-induced Coriolis and centrifugal forces on the arm. Remarkably, feedback control was functional from the very first time participants encountered a target jump in the altered force field. Overall, the present results demonstrate that feedforward control enables immediate functional feedback control even when applied to distinct sensorimotor processes.NEW & NOTEWORTHY We investigated the link between feedforward and feedback control when applying a double-step perturbation (visual target jump) during reaching movements performed in modified gravitoinertial environments. Altogether, kinematics and EMG analyses showed that movement corrections were highly effective in the different force fields, suggesting that, although feedforward and feedback control were driven by different sensory inputs, feedback control was remarkably functional from the very first time participants encountered a target jump in the altered force field.

Estimation of Involuntary Components of Human Arm Impedance in Multi-Joint Movements via Feedback Jerk Isolation

Stable and efficient coordination in physical human-robot interaction requires consideration of human feedback behavior. In unpredictable tasks, where voluntary cognitive feedback is too slow to guarantee desired task execution, the human must rely on involuntary intrinsic and reflexive feedback. The combined effects of these two feedback mechanisms and the inertial characteristics can be summarized in the involuntary impedance components. In this work, we present a method for the estimation of the involuntary impedance components of the human arm in multi-joint movements. We apply force perturbations to evoke feedback jerks that can be isolated using a high pass filter and limit the duration of the estimation interval to guarantee exclusion of voluntary cognitive feedback. Dynamic regressor representation of the rigid body dynamics of the arm and first order Taylor series expansion of the feedback behavior yield a model that is linear in the involuntary impedance components. The constant values of the inertial parameters are estimated in a static posture maintenance task and subsequently inserted to estimate the remaining components in a dynamic movement task. The method is validated with simulated data of a neuromechanical model of the human arm and its performance is compared to established methods from the literature. The results of the validation demonstrate superior estimation performance for moderate movement velocities, and less influence of the variability of the movements. The applicability to real data and the plausibility of the limited estimation interval are successfully demonstrated in an experiment with human participants.

Learning New Feedforward Motor Commands Based on Feedback Responses

Learning a new motor task modifies feedforward (i.e., voluntary) motor commands and such learning also changes the sensitivity of feedback responses (i.e., reflexes) to mechanical perturbations [1-9]. For example, after people learn to generate straight reaching movements in the presence of an external force field or learn to reduce shoulder muscle activity when generating pure elbow movements with shoulder fixation, evoked stretch reflex responses to mechanical perturbations reflect the learning expressed during self-initiated reaching. Such a transfer from feedforward motor commands to feedback responses is thought to take place because of shared neural circuits at the level of the spinal cord, brainstem, and cerebral cortex [10-13]. The presence of shared neural resources also predicts the transfer from feedback responses to feedforward motor commands. Little is known about such a transfer presumably because it is relatively hard to elicit learning in reflexes without engaging associated voluntary responses following mechanical perturbations. Here, we demonstrate such transfer by leveraging two approaches to elicit stretch reflexes while minimizing engagement of voluntary motor responses in the learning process: applying very short mechanical perturbations [14-19] and instructing participants to not respond to them [20-26]. Taken together, our work shows that transfer between feedforward and feedback control is bidirectional, furthering the notion that these processes share common neural circuits that underlie motor learning and transfer.

Rapid adaptation to Coriolis force perturbations of voluntary body sway

Studying adaptation to Coriolis perturbations of arm movements has advanced our understanding of motor control and learning. We have now applied this paradigm to two-dimensional postural sway. We measured how subjects (n = 8) standing at the center of a fully enclosed rotating room who made voluntary anterior-posterior swaying movements adapted to the Coriolis perturbations generated by their sway. Subjects underwent four voluntary sway trials prerotation, 20 per-rotation at 10 rpm counterclockwise, and 10 postrotation. Each trial lasted 20 s, and subjects were permitted normal vision. Their voluntary sway during rotation generated Coriolis forces that initially induced rightward deviations of their forward sway paths and leftward deviations of their backward sway. Sagittal plane sway was gradually restored over per-rotation trials, and a mirror image aftereffect occurred in postrotation trials. Dual force plate data analysis showed that subjects learned to counter the Coriolis accelerations during rotation by executing a bimodal torque pattern that was asymmetric across legs and contingent on forward vs. backward movement. The experience-dependent acquisition and washout of this compensation indicate that an internal, feedforward model underlies the leg-asymmetric bimodal torque compensation, contingent on forward vs. backward movement. The learned torque asymmetry we observed for forward vs. backward sway is not consistent with parallel two-leg models of postural control. NEW & NOTEWORTHY This paper describes adaptation to Coriolis force perturbations of voluntary sway in a rotating environment. During counterclockwise rotation, sway paths are deviated clockwise, but full restoration of fore-aft sway is regained in minutes. Negative aftereffects are briefly present postrotation. Current parallel leg models of postural control cannot account for these findings, which show that postural control, like arm movement control, can adapt rapidly and completely to the Coriolis forces generated in artificial gravity environments.

Feedforward and Feedback Control Share an Internal Model of the Arm's Dynamics

Recent work has shown that, when countering external forces, the nervous system adjusts not only predictive (i.e., feedforward) control of reaching but also reflex (i.e., feedback) responses to mechanical perturbations. Here we show that altering the physical properties of the arm (i.e., intersegmental dynamics) causes the nervous system to adjust feedforward control and that this learning transfers to feedback responses even though the latter were never directly trained. Forty-five human participants (30 females) performed a single-joint elbow reaching task and countered mechanical perturbations that created pure elbow motion. In our first experiment, we altered intersegmental dynamics by asking participants to generate pure elbow movements when the shoulder joint was either free to rotate or locked by the robotic manipulandum. With the shoulder unlocked, we found robust activation of shoulder flexor muscles for pure elbow flexion trials, as required to counter the interaction torques that arise at the shoulder because of forearm rotation. After locking the shoulder joint, which cancels these interaction torques, we found a substantial reduction in shoulder muscle activity over many trials. In our second experiment, we tested whether such learning transfers to feedback control. Mechanical perturbations applied to the arm with the shoulder unlocked revealed that feedback responses also account for intersegmental dynamics. After locking the shoulder joint, we found a substantial reduction in shoulder feedback responses, as appropriate for the altered intersegmental dynamics. Our work suggests that feedforward and feedback control share an internal model of the arm's dynamics.SIGNIFICANCE STATEMENT Here we show that altering the physical properties of the arm causes people to learn new motor commands and that this learning transfers to their reflex responses to unexpected mechanical perturbations, even though the reflex responses were never directly trained. Our results suggest that feedforward motor commands and reflex responses share an internal model of the arm's dynamics.

Muscle patterns underlying voluntary modulation of co-contraction

Manipulative actions involving unstable interactions with the environment require controlling mechanical impedance through muscle co-contraction. While much research has focused on how the central nervous system (CNS) selects the muscle patterns underlying a desired movement or end-point force, the coordination strategies used to achieve a desired end-point impedance have received considerably less attention. We recorded isometric forces at the hand and electromyographic (EMG) signals in subjects performing a reaching task with an external disturbance. In a virtual environment, subjects displaced a cursor by applying isometric forces and were instructed to reach targets in 20 spatial locations. The motion of the cursor was then perturbed by disturbances whose effects could be attenuated by increasing co-contraction. All subjects could voluntarily modulate co-contraction when disturbances of different magnitudes were applied. For most muscles, activation was modulated by target direction according to a cosine tuning function with an offset and an amplitude increasing with disturbance magnitude. Co-contraction was characterized by projecting the muscle activation vector onto the null space of the EMG-to-force mapping. Even in the baseline the magnitude of the null space projection was larger than the minimum magnitude required for non-negative muscle activations. Moreover, the increase in co-contraction was not obtained by scaling the baseline null space projection, scaling the difference between the null space projections in any block and the projection of the non-negative minimum-norm muscle vector, or scaling the difference between the null space projections in the perturbed blocks and the baseline null space projection. However, the null space projections in the perturbed blocks were obtained by linear combination of the baseline null space projection and the muscle activation used to increase co-contraction without generating any force. The failure of scaling rules in explaining voluntary modulation of arm co-contraction suggests that muscle pattern generation may be constrained by muscle synergies.

The Mechanical Representation of Temporal Delays

When we knock on a door, we perceive the impact as a collection of simultaneous events, combining sound, sight, and tactile sensation. In reality, information from different modalities but from a single source is flowing inside the brain along different pathways, reaching processing centers at different times. Therefore, interpreting different sensory modalities which seem to occur simultaneously requires information processing that accounts for these different delays. As in a computer-based robotic system, does the brain use some explicit estimation of the time delay, to realign the sensory flows? Or does it compensate for temporal delays by representing them as changes in the body/environment mechanics? Using delayed-state or an approximation for delayed-state manipulations between visual and proprioceptive feedback during a tracking task, we show that tracking errors, grip forces, and learning curves are consistent with predictions of a representation that is based on approximation for delay, refuting an explicit delayed-state representation. Delayed-state representations are based on estimating the time elapsed between the movement commands and their observed consequences. In contrast, an approximation for delay representations result from estimating the instantaneous relation between the expected and observed motion variables, without explicit reference to time.

Mechanical Impedance and Its Relations to Motor Control, Limb Dynamics, and Motion Biomechanics

The concept of mechanical impedance represents the interactive relationship between deformation kinematics and the resulting dynamics in human joints or limbs. A major component of impedance, stiffness, is defined as the ratio between the force change to the displacement change and is strongly related to muscle activation. The set of impedance components, including effective mass, inertia, damping, and stiffness, is important in determining the performance of the many tasks assigned to the limbs and in counteracting undesired effects of applied loads and disturbances. Specifically for the upper limb, impedance enables controlling manual tasks and reaching motions. In the lower limb, impedance is responsible for the transmission and attenuation of impact forces in tasks of repulsive loadings. This review presents an updated account of the works on mechanical impedance and its relations with motor control, limb dynamics, and motion biomechanics. Basic questions related to the linearity and nonlinearity of impedance and to the factors that affect mechanical impedance are treated with relevance to upper and lower limb functions, joint performance, trunk stability, and seating under dynamic conditions. Methods for the derivation of mechanical impedance, both those for within the system and material-structural approaches, are reviewed. For system approaches, special attention is given to methods aimed at revealing the correct and sufficient degree of nonlinearity of impedance. This is particularly relevant in the design of spring-based artificial legs and robotic arms. Finally, due to the intricate relation between impedance and muscle activity, methods for the explicit expression of impedance of contractile tissue are reviewed.

Convergent models of handedness and brain lateralization

The pervasive nature of handedness across human history and cultures is a salient consequence of brain lateralization. This paper presents evidence that provides a structure for understanding the motor control processes that give rise to handedness. According to the Dynamic Dominance Model, the left hemisphere (in right handers) is proficient for processes that predict the effects of body and environmental dynamics, while the right hemisphere is proficient at impedance control processes that can minimize potential errors when faced with unexpected mechanical conditions, and can achieve accurate steady-state positions. This model can be viewed as a motor component for the paradigm of brain lateralization that has been proposed by Rogers et al. (MacNeilage et al., 2009) that is based upon evidence from a wide range of behaviors across many vertebrate species. Rogers proposed a left-hemisphere specialization for well-established patterns of behavior performed in familiar environmental conditions, and a right hemisphere specialization for responding to unforeseen environmental events. The dynamic dominance hypothesis provides a framework for understanding the biology of motor lateralization that is consistent with Roger's paradigm of brain lateralization.

Effects of predictability of load magnitude on the response of the Flexor Digitorum Superficialis to a sudden fingers extension

Muscle reflexes, evoked by opposing a sudden joint displacement, may be modulated by several factors associated with the features of the mechanical perturbation. We investigated the variations of muscle reflex response in relation to the predictability of load magnitude during a reactive grasping task. Subjects were instructed to flex the fingers 2–5 very quickly after a stretching was exerted by a handle pulled by loads of 750 or 1250 g. Two blocks of trials, one for each load (predictable condition), and one block of trials with a randomized distribution of the loads (unpredictable condition) were performed. Kinematic data were collected by an electrogoniometer attached to the middle phalanx of the digit III while the electromyography of the Flexor Digitorum Superficialis muscle was recorded by surface electrodes. For each trial we measured the kinematics of the finger angular rotation, the latency of muscle response and the level of muscle activation recorded below 50 ms (short-latency reflex), between 50 and 100 ms (long-latency reflex) and between 100 and 140 ms (initial portion of voluntary response) from the movement onset. We found that the latency of the muscle response lengthened from predictable (35.5±1.3 ms for 750 g and 35.5±2.5 ms for 1250 g) to unpredictable condition (43.6±1.3 ms for 750 g and 40.9±2.1 ms for 1250 g) and the level of muscle activation increased with load magnitude. The parallel increasing of muscle activation and load magnitude occurred within the window of the long-latency reflex during the predictable condition, and later, at the earliest portion of the voluntary response, in the unpredictable condition. Therefore, these results indicate that when the amount of an upcoming perturbation is known in advance, the muscle response improves, shortening the latency and modulating the muscle activity in relation to the mechanical demand.

Priors engaged in long-latency responses to mechanical perturbations suggest a rapid update in state estimation

In every motor task, our brain must handle external forces acting on the body. For example, riding a bike on cobblestones or skating on irregular surface requires us to appropriately respond to external perturbations. In these situations, motor predictions cannot help anticipate the motion of the body induced by external factors, and direct use of delayed sensory feedback will tend to generate instability. Here, we show that to solve this problem the motor system uses a rapid sensory prediction to correct the estimated state of the limb. We used a postural task with mechanical perturbations to address whether sensory predictions were engaged in upper-limb corrective movements. Subjects altered their initial motor response in ∼60 ms, depending on the expected perturbation profile, suggesting the use of an internal model, or prior, in this corrective process. Further, we found trial-to-trial changes in corrective responses indicating a rapid update of these perturbation priors. We used a computational model based on Kalman filtering to show that the response modulation was compatible with a rapid correction of the estimated state engaged in the feedback response. Such a process may allow us to handle external disturbances encountered in virtually every physical activity, which is likely an important feature of skilled motor behaviour.

Preparing to reach: selecting an adaptive long-latency feedback controller

In a voluntary movement, the nervous system specifies not only the motor commands but also the gains associated with reaction to sensory feedback. For example, suppose that, during reaching, a perturbation tends to push the hand to the left. With practice, the brain not only learns to produce commands that predictively compensate for the perturbation but also increases the long-latency reflex gain associated with leftward displacements of the arm. That is, the brain learns a feedback controller. Here, we wondered whether, during the preparatory period before the reach, the brain engaged this feedback controller in anticipation of the upcoming movement. If so, its signature might be present in how the motor system responds to perturbations in the preparatory period. Humans trained on a reach task in which they adapted to a force field. During the preparatory period before the reach, we measured how the arm responded to a pulse to the hand that was either in the direction of the upcoming field, or in the opposite direction. Reach adaptation produced an increase in the long-latency (45-100 ms delay) feedback gains with respect to baseline, but only for perturbations that were in the same direction as the force field that subjects expected to encounter during the reach. Therefore, as the brain prepares for a reach, it loads a feedback controller specific to the upcoming reach. With adaptation, this feedback controller undergoes a change, increasing the gains for the expected sensory feedback.

Colored context cues can facilitate the ability to learn and to switch between multiple dynamical force fields

We tested the efficacy of color context cues during adaptation to dynamic force fields. Four groups of human subjects performed elbow flexion/extension movements to move a cursor between targets on a monitor while encountering a resistive (Vr) or assistive (Va) viscous force field. They performed two training sets of 256 trials daily, for 10 days. The monitor background color changed (red, green) every four successful trials but provided different degrees of force field context information to each group. For the irrelevant-cue groups, the color changed every four trials, but one group encountered only the Va field and the other only the Vr field. For the reliable-cue group, the force field alternated between Va and Vr each time the monitor changed color (Vr, red; Va, green). For the unreliable-cue group, the force field changed between Va and Vr pseudorandomly at each color change. All subjects made increasingly stereotyped movements over 10 training days. Reliable-cue subjects typically learned the association between color cues and fields and began to make predictive changes in motor output at each color change during the first day. Their performance continued to improve over the remaining days. Unreliable-cue subjects also improved their performance across training days but developed a strategy of probing the nature of the field at each color change by emitting a default motor response and then adjusting their motor output in subsequent trials. These findings show that subjects can extract explicit and implicit information from color context cues during force field adaptation.

A computational model of limb impedance control based on principles of internal model uncertainty

Efficient human motor control is characterized by an extensive use of joint impedance modulation, which is achieved by co-contracting antagonistic muscles in a way that is beneficial to the specific task. While there is much experimental evidence available that the nervous system employs such strategies, no generally-valid computational model of impedance control derived from first principles has been proposed so far. Here we develop a new impedance control model for antagonistic limb systems which is based on a minimization of uncertainties in the internal model predictions. In contrast to previously proposed models, our framework predicts a wide range of impedance control patterns, during stationary and adaptive tasks. This indicates that many well-known impedance control phenomena naturally emerge from the first principles of a stochastic optimization process that minimizes for internal model prediction uncertainties, along with energy and accuracy demands. The insights from this computational model could be used to interpret existing experimental impedance control data from the viewpoint of optimality or could even govern the design of future experiments based on principles of internal model uncertainty.

Evidence of gender-specific motor templates to resist valgus loading at the knee

Gender differences in neuromuscular control of the lower extremity may contribute to increased injury risk in females, but the neurophysiological mechanisms underlying these differences remain unclear. In this study, we sought to explore the effect of gender on volitional and reflex neuromuscular responses to a rapid valgus perturbation at the knee applied under "intervene" and "do not intervene" conditions. Multiple 7 degrees ramp-and-hold valgus perturbations were applied at the neutrally extended knee of 12 male and 12 female healthy subjects, while surface electromyography over the quadriceps and hamstrings recorded the neuromuscular response. Volitional responses did not vary between groups, perhaps reflecting the relative novelty of the loading direction. However, reflex responses observed under the "do not intervene" paradigm did vary by gender. Males demonstrated much more frequent and consistent reflex muscle activation than females. Moreover, muscle activation patterns were gender-specific. Diminished responses in female subjects may indicate that the position-based valgus perturbation did not produce the necessary mechanical stimulus to elicit reflexes. These gender differences in reflex control of the knee provide new insight into the control of frontal-plane knee joint movement and loading and may elucidate the neuromechanical underpinnings associated with neuromuscular control.

Long-Latency Responses During Reaching Account for the Mechanical Interaction Between the Shoulder and Elbow Joints

Although considerable research indicates that reaching movements rely on knowledge of the arm's mechanical properties and environment to anticipate and counter predictable loads, far less research has examined whether this degree of sophistication is present for on-line corrections during reaching. Here we examine the R2/3 response to mechanical perturbations (45–100 ms, also called the long-latency reflex), which is highly flexible and includes the fastest possible contribution from primary motor cortex, a key neural substrate for self-initiated action. Torque perturbations were occasionally and unexpectedly applied to the subject's shoulder and/or elbow in the course of performing reaching movements. Critically, these perturbations would evoke different patterns of feedback corrections from a shoulder extensor muscle if it countered only the local shoulder displacement relative to unperturbed motion or accounted for the mechanical interactions between the shoulder and elbow joints and countered the underlying shoulder torque. Our results show that the earliest response (R1: 20–45 ms) reflected local shoulder displacement, whereas the R2/3 response (45–100 ms) reflected knowledge of multijoint dynamics. Moreover, the same pattern of feedback occurred whether the shoulder muscle helped initiate the movement (during its agonist phase) or helped terminate the movement (during its antagonist phase). These results contribute to the accumulating evidence that highly sophisticated feedback control underlies motor behavior and are consistent with a shared neural substrate, such as primary motor cortex, for feedforward and feedback control.

Temporal development of anticipatory reflex modulation to dynamical interactions during arm movement

It is known that somatosensory reflex during voluntary arm movement is modulated anticipatorily according to given tasks or environments. However, when and how reflex amplitude is set remains controversial. Is the reflex modulation completed preparatorily before movement execution or does it vary with the movement? Is the reflex amplitude coded in a temporal manner or in a spatial (or state-dependent) manner? Here we studied these issues while subjects performed planar reaching movements with upcoming opposite (rightward/leftward) directions of force fields. Somatosensory reflex responses of shoulder muscles induced by a small force perturbation were evaluated at several points before the arm encountered predictable force fields after movement start. We found that the shoulder flexor reflex responses were generally higher for the rightward than for the leftward upcoming force fields, whereas the extensor reflex responses were higher for the leftward force field. This reflex amplitude depending on the upcoming force field direction became prominent as the reflex was evoked closer to the force fields, indicating continuous changes in reflex modulation during movement. An additional experiment further showed that the reflex modulation developed as a function of the temporal distance to the force fields rather than the spatial distance. Taken together, the results suggest that, in the force field interaction task, somatosensory reflex amplitude during the course of movement is set anticipatorily on the basis of an estimate of the time-to-contact rather than the state-to-contact, to upcoming dynamical interaction during voluntary movement.

Visual modulation of proprioceptive reflexes during movement

Previous research has demonstrated that feedback circuits such as reflexes can be tuned by setting their gains prior to movement onset during both posture and movement tasks. However, such a control strategy requires that perturbation contingencies be predicted during movement planning and that task goals remain fixed. Here we test the hypothesis that feedforward regulation of reflex circuits also occurs during the course of movement in response to changes in task goals. Participants reached to a visual target that was occasionally jumped on movement initiation, thus changing task goals. Reflex responses were elicited through a mechanical perturbation on the same trial, 100 ms after the target jump. Impedance to the perturbation was tuned to the direction of the preceding jump: reflex responses increased or decreased depending on whether the perturbation opposed or was consistent with the target jump. This modulation, although sensitive to the direction of the jump, was insensitive to jump amplitude, as tested in a follow-up experiment. Our findings thus suggest that modulation of reflex circuits occurs online, and is sensitive to changes in visual target information. In addition, our results suggest a two-level model for visuo-motor control that reflects hierarchical neural organization.

Shared internal models for feedforward and feedback control

A child often learns to ride a bicycle in the driveway, free of unforeseen obstacles. Yet when she first rides in the street, we hope that if a car suddenly pulls out in front of her, she will combine her innate goal of avoiding an accident with her learned knowledge of the bicycle, and steer away or brake. In general, when we train to perform a new motor task, our learning is most robust if it updates the rules of online error correction to reflect the rules and goals of the new task. Here we provide direct evidence that, after a new feedforward motor adaptation, motor feedback responses to unanticipated errors become precisely task appropriate, even when such errors were never experienced during training. To study this ability, we asked how, if at all, do online responses to occasional, unanticipated force pulses during reaching arm movements change after adapting to altered arm dynamics? Specifically, do they change in a task-appropriate manner? In our task, subjects learned novel velocity-dependent dynamics. However, occasional force-pulse perturbations produced unanticipated changes in velocity. Therefore, after adaptation, task-appropriate responses to unanticipated pulses should compensate corresponding changes in velocity-dependent dynamics. We found that after adaptation, pulse responses precisely compensated these changes, although they were never trained to do so. These results provide evidence for a smart feedback controller which automatically produces responses specific to the learned dynamics of the current task. To accomplish this, the neural processes underlying feedback control must (1) be capable of accurate real-time state prediction for velocity via a forward model and (2) have access to recently learned changes in internal models of limb dynamics.

Performance differences in visually and internally guided continuous manual tracking movements

Control of familiar visually guided movements involves internal plans as well as visual and other online sensory information, though how visual and internal plans combine for reaching movements remain unclear. Traditional motor sequence learning tasks, such as the serial reaction time task, use stereotyped movements and measure only reaction time. Here, we used a continuous sequential reaching task comprised of naturalistic movements, in order to provide detailed kinematic performance measures. When we embedded pre-learned trajectories (those presumably having an internal plan) within similar but unpredictable movement sequences, participants performed the two kinds of movements with remarkable similarity, and position error alone could not reliably identify the epoch. For such embedded movements, performance during pre-learned sequences showed statistically significant but trivial decreases in measures of kinematic error, compared to performance during novel sequences. However, different sets of kinematic error variables changed significantly between learned and novel sequences for individual participants, suggesting that each participant used distinct motor strategies favoring different kinematic variables during each of the two movement types. Algorithms that incorporated multiple kinematic variables identified transitions between the two movement types well but imperfectly. Hidden Markov model classification differentiated learned and novel movements on single trials based on the above kinematic error variables with 82 +/- 5% accuracy within 244 +/- 696 ms, despite the limited extent of changes in those errors. These results suggest that the motor system can achieve markedly similar performance whether or not an internal plan is present, as only subtle changes arise from any difference between the neural substrates involved in those two conditions.

Motor adaptation as a process of reoptimization

Adaptation is sometimes viewed as a process in which the nervous system learns to predict and cancel effects of a novel environment, returning movements to near baseline (unperturbed) conditions. An alternate view is that cancellation is not the goal of adaptation. Rather, the goal is to maximize performance in that environment. If performance criteria are well defined, theory allows one to predict the reoptimized trajectory. For example, if velocity-dependent forces perturb the hand perpendicular to the direction of a reaching movement, the best reach plan is not a straight line but a curved path that appears to overcompensate for the forces. If this environment is stochastic (changing from trial to trial), the reoptimized plan should take into account this uncertainty, removing the overcompensation. If the stochastic environment is zero-mean, peak velocities should increase to allow for more time to approach the target. Finally, if one is reaching through a via-point, the optimum plan in a zero-mean deterministic environment is a smooth movement but in a zero-mean stochastic environment is a segmented movement. We observed all of these tendencies in how people adapt to novel environments. Therefore, motor control in a novel environment is not a process of perturbation cancellation. Rather, the process resembles reoptimization: through practice in the novel environment, we learn internal models that predict sensory consequences of motor commands. Through reward-based optimization, we use the internal model to search for a better movement plan to minimize implicit motor costs and maximize rewards.

Visual and haptic feedback contribute to tuning and online control during object manipulation

The authors employed a virtual environment to investigate how humans use haptic and visual feedback in a simple, rhythmic object-manipulation task. The authors hypothesized that feedback would help participants identify the appropriate resonant frequency and perform online control adjustments. The 1st test was whether sensory feedback is needed at all; the 2nd was whether the motor system combines visual and haptic feedback to improve performance. Task performance was quantified in terms of work performed on the virtual inertia, ability to identify the correct rhythm, and variability of movement. Strict feedforward control was found to be ineffective for this task, even when participants had previous knowledge of the rhythm. Participants (N = 11) performed far better when feedback was available (11 times more work, 2.2 times more precise frequency, 30% less variability; p < .05 for all 3 performance measures). Using sensory feedback, participants were able to rapidly identify 4 different spring-inertia systems without foreknowledge of the corresponding resonant frequencies. They performed over 20% more work with 24% less variability when provided with both visual and haptic feedback than they did with either feedback channel alone (p < .05), providing evidence that they integrated online sensory channels. Whereas feedforward control alone led to poor performance, feedback control led to fast tuning or calibration of control according to the resonant frequency of the object, and to better control of the rhythmic movement itself.

Computational principles of sensorimotor control that minimize uncertainty and variability

Sensory and motor noise limits the precision with which we can sense the world and act upon it. Recent research has begun to reveal computational principles by which the central nervous system reduces the sensory uncertainty and movement variability arising from this internal noise. Here we review the role of optimal estimation and sensory filtering in extracting the sensory information required for motor planning, and the role of optimal control, motor adaptation and impedance control in the specification of the motor output signal.

Transcranial magnetic stimulation over sensorimotor cortex disrupts anticipatory reflex gain modulation for skilled action

Skilled interactions with new environments require flexible changes to the transformation from somatosensory signals to motor outputs. Transcortical reflex gains are known to be modulated according to task and environmental dynamics, but the mechanism of this modulation remains unclear. We examined reflex organization in the sensorimotor cortex. Subjects performed point-to-point arm movements into predictable force fields. When a small perturbation was applied just before the arm encountered the force field, reflex responses in the shoulder muscles changed according to the upcoming force field direction, indicating anticipatory reflex gain modulation. However, when a transcranial magnetic stimulation (TMS) was applied before the reflex response to such perturbations so that the silent period caused by TMS overlapped the reflex processing period, this modulation was abolished, while the reflex itself remained. Loss of reflex gain modulation could not be explained by reduced reflex amplitudes nor by peripheral effects of TMS on the muscles themselves. Instead, we suggest that TMS disrupted interneuronal networks in the sensorimotor cortex, which contribute to reflex gain modulation rather than reflex generation. We suggest that these networks normally provide the adaptability of rapid sensorimotor reflex responses by regulating reflex gains according to the current dynamical environment.

Adaptation and generalization in acceleration-dependent force fields

Any passive rigid inertial object that we hold in our hand, e.g., a tennis racquet, imposes a field of forces on the arm that depends on limb position, velocity, and acceleration. A fundamental characteristic of this field is that the forces due to acceleration and velocity are linearly separable in the intrinsic coordinates of the limb. In order to learn such dynamics with a collection of basis elements, a control system would generalize correctly and therefore perform optimally if the basis elements that were sensitive to limb velocity were not sensitive to acceleration, and vice versa. However, in the mammalian nervous system proprioceptive sensors like muscle spindles encode a nonlinear combination of all components of limb state, with sensitivity to velocity dominating sensitivity to acceleration. Therefore, limb state in the space of proprioception is not linearly separable despite the fact that this separation is a desirable property of control systems that form models of inertial objects. In building internal models of limb dynamics, does the brain use a representation that is optimal for control of inertial objects, or a representation that is closely tied to how peripheral sensors measure limb state? Here we show that in humans, patterns of generalization of reaching movements in acceleration-dependent fields are strongly inconsistent with basis elements that are optimized for control of inertial objects. Unlike a robot controller that models the dynamics of the natural world and represents velocity and acceleration independently, internal models of dynamics that people learn appear to be rooted in the properties of proprioception, nonlinearly responding to the pattern of muscle activation and representing velocity more strongly than acceleration.

Motor control and learning in altered dynamic environments

Dynamic perturbations of reaching movements are an important technique for studying motor learning and adaptation. Adaptation to non-contacting, velocity-dependent inertial Coriolis forces generated by arm movements during passive body rotation is very rapid, and when complete the Coriolis forces are no longer sensed. Adaptation to velocity-dependent forces delivered by a robotic manipulandum takes longer and the perturbations continue to be perceived even when adaptation is complete. These differences reflect adaptive self-calibration of motor control versus learning the behavior of an external object or 'tool'. Velocity-dependent inertial Coriolis forces also arise in everyday behavior during voluntary turn and reach movements but because of anticipatory feedforward motor compensations do not affect movement accuracy despite being larger than the velocity-dependent forces typically used in experimental studies. Progress has been made in understanding: the common features that determine adaptive responses to velocity-dependent perturbations of jaw and limb movements; the transfer of adaptation to mechanical perturbations across different contact sites on a limb; and the parcellation and separate representation of the static and dynamic components of multiforce perturbations.

The Human Motor Control System's Response to Mechanical Perturbation: Should It, Can It and Does It Ensure Stability?

From among the diverse meanings of stability, the one the author adopts here is that the effects of a perturbation are opposed, and therefore small effects remain small. Except in linear systems, however, instability need not lead to unbounded motion and may actually be desirable when maneuverability is important. Moreover, properties of nerves, muscles, and tendons present serious challenges to stabilization. A review of observations from the motor control literature reveals that responses to perturbations in many common situations assist rather than resist the perturbation and are therefore presumably destabilizing. The observations encompass situations of position maintenance as well as impending or ongoing movement. The author proposes that the motor control system responds to a sudden perturbation by a pattern of muscle activity that mimics an accustomed voluntary movement, oblivious of stability considerations. What prevents runaway motion in the face of short-term instability appears to be voluntary intervention.

Optimality principles in sensorimotor control

The sensorimotor system is a product of evolution, development, learning and adaptation-which work on different time scales to improve behavioral performance. Consequently, many theories of motor function are based on 'optimal performance': they quantify task goals as cost functions, and apply the sophisticated tools of optimal control theory to obtain detailed behavioral predictions. The resulting models, although not without limitations, have explained more empirical phenomena than any other class. Traditional emphasis has been on optimizing desired movement trajectories while ignoring sensory feedback. Recent work has redefined optimality in terms of feedback control laws, and focused on the mechanisms that generate behavior online. This approach has allowed researchers to fit previously unrelated concepts and observations into what may become a unified theoretical framework for interpreting motor function. At the heart of the framework is the relationship between high-level goals, and the real-time sensorimotor control strategies most suitable for accomplishing those goals.

Robotics, motor learning, and neurologic recovery

Robotic devices are helping shed light on human motor control in health and injury. By using robots to apply novel force fields to the arm, investigators are gaining insight into how the nervous system models its external dynamic environment. The nervous system builds internal models gradually by experience and uses them in combination with impedance and feedback control strategies. Internal models are robust to environmental and neural noise, generalized across space, implemented in multiple brain regions, and developed in childhood. Robots are also being used to assist in repetitive movement practice following neurologic injury, providing insight into movement recovery. Robots can haptically assess sensorimotor performance, administer training, quantify amount of training, and improve motor recovery. In addition to providing insight into motor control, robotic paradigms may eventually enhance motor learning and rehabilitation beyond the levels possible with conventional training techniques.

Internal models and contextual cues: encoding serial order and direction of movement

During reaching, the brain may rely on internal models to transform desired sensory outcomes into motor commands. This transformation depends on both the state of the limb and the cues that can identify the context of the movement. How are contextual cues and information about state of the limb combined in the computations of internal models? We considered a reaching task where forces on the hand depended on both the direction of movement (state of the limb) and order of that movement in a predefined sequence (contextual cue). When the cue was available, the motor system formed an internal model that used both serial order and target direction to program motor commands. Assuming that the internal model was formed by a population code through a combination of unknown basis elements, the sensitivity of the bases with respect to state of the limb and contextual cue should dictate how error in one type of movement affected all other movement types. Using a state-space theory, we estimated this generalization function and identified the adaptive system from trial-by-trial changes in performance. The results implied that the basis elements were tuned to direction of movement but output of each basis at its preferred direction was multiplicatively modulated by a weak tuning with respect to the contextual cue. Activity fields that multiplicatively encode diverse sources of information may serve as a general mechanism for a single network to produce context-dependent motor output.

Failure of motor learning for large initial errors

For certain complex motor tasks, humans may experience the frustration of a lack of improvement despite repeated practice. We investigate a computational basis for failure of motor learning when there is no prior information about the system to be controlled and when it is not practical to perform a thorough random exploration of the set of possible commands. In this case, if the desired movement has never yet been performed, then it may not be possible to learn the correct motor commands since there will be no appropriate training examples. We derive the mathematical basis for this phenomenon when the controller can be modeled as a linear combination of nonlinear basis functions trained using a gradient descent learning rule on the observed commands and their results. We show that there are two failure modes for which continued training examples will never lead to improvement in performance. We suggest that this may provide a model for the lack of improvement in human skills that can occur despite repeated practice of a complex task.

EMG responses to an unexpected load in fast movements are delayed with an increase in the expected movement time

When moving an object, the motor system estimates the dynamic properties of the object and then controls the movement using a combination of predictive feedforward control and proprioceptive feedback. In this study, we examined how the feedforward and proprioceptive feedback processes depend on the expected movement task. Subjects made fast elbow flexion movements from an initial position to a target. The experimental protocol included movements made over a short and a long distance against an expected light or heavy inertial load. In each task in a few randomly chosen trials, a motor applied an unexpected viscous load that produced a velocity error, defined as the difference between the expected and unexpected velocities, and electromyographic (EMG) responses. The EMG responses appeared not earlier than 170-250 ms from the agonist EMG onset. Our main finding is that the onset of the EMG responses was correlated with the expected time of peak velocity, which increased for longer distances and larger loads. An analysis of the latency of the EMG responses with respect to the velocity error suggested that the EMG responses were due to segmental reflexes. We conclude that segmental reflex gains are centrally modulated with the time course dependent on the expected movement task. According to this view, the control of fast point-to-point movement is feedforward from the agonist EMG onset until the expected time of peak velocity after which the segmental reflex feedback is briefly facilitated.

Different Mechanisms Involved in Adaptation to Stable and Unstable Dynamics

Recently, we demonstrated that humans can learn to make accurate movements in an unstable environment by controlling magnitude, shape, and orientation of the endpoint impedance. Although previous studies of human motor learning suggest that the brain acquires an inverse dynamics model of the novel environment, it is not known whether this control mechanism is operative in unstable environments. We compared learning of multijoint arm movements in a “velocity-dependent force field” (VF), which interacted with the arm in a stable manner, and learning in a “divergent force field” (DF), where the interaction was unstable. The characteristics of error evolution were markedly different in the 2 fields. The direction of trajectory error in the DF alternated to the left and right during the early stage of learning; that is, signed error was inconsistent from movement to movement and could not have guided learning of an inverse dynamics model. This contrasted sharply with trajectory error in the VF, which was initially biased and decayed in a manner that was consistent with rapid feedback error learning. EMG recorded before and after learning in the DF and VF are also consistent with different learning and control mechanisms for adapting to stable and unstable dynamics, that is, inverse dynamics model formation and impedance control. We also investigated adaptation to a rotated DF to examine the interplay between inverse dynamics model formation and impedance control. Our results suggest that an inverse dynamics model can function in parallel with an impedance controller to compensate for consistent perturbing force in unstable environments.

Neuromotor noise limits motor performance, but not motor adaptation, in children

Children do not typically appear to move with the same skill and dexterity as adults, although they can still improve their motor performance in specific tasks with practice. One possible explanation is that their motor performance is limited by an inherently higher level of movement variability, but that their motor adaptive ability is robust to this variability. To test this hypothesis, we examined motor adaptation of 43 children (ages 6-17) and 12 adults as they reached while holding the tip of a lightweight robot. The robot applied either a predictable, velocity-dependent field (the "mean field") or a similar field that incorporated stochastic variation (the "noise field"), thereby further enhancing the variability of the subjects' movements. We found that children exhibited greater initial trial-to-trial variability in their unperturbed movements but were still able to adapt comparably to adults in both the mean and noise fields. Furthermore, the youngest children (ages 6-8) were able to reduce their variability with practice to levels comparable to the remaining children groups although not as low as adults. These results indicate that children as young as age 6 possess adult-like neural systems for motor adaptation and internal model formation that allow them to adapt to novel dynamic environments as well as adults on average despite increased neuromotor or environmental noise. Performance after adaptation is still more variable than adults, however, indicating that movement inconsistency, not motor adaptation inability, ultimately limits motor performance by children and may thus account for their appearance of incoordination and more frequent motor accidents (e.g., spilling, tripping). The results of this study also suggest that movement variability in young children may arise from two sources--a relatively constant, intrinsic source related to fundamental physiological constraints of the developing motor system and a more rapidly modifiable source that is modulated depending on the current motor context.

Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics

This study compared the mechanisms of adaptation to stable and unstable dynamics from the perspective of changes in joint mechanics. Subjects were instructed to make point to point movements in force fields generated by a robotic manipulandum which interacted with the arm in either a stable or an unstable manner. After subjects adjusted to the initial disturbing effects of the force fields they were able to produce normal straight movements to the target. In the case of the stable interaction, subjects modified the joint torques in order to appropriately compensate for the force field. No change in joint torque or endpoint force was required or observed in the case of the unstable interaction. After adaptation, the endpoint stiffness of the arm was measured by applying displacements to the hand in eight different directions midway through the movements. This was compared to the stiffness measured similarly during movements in a null force field. After adaptation, the endpoint stiffness under both the stable and unstable dynamics was modified relative to the null field. Adaptation to unstable dynamics was achieved by selective modification of endpoint stiffness in the direction of the instability. To investigate whether the change in endpoint stiffness could be accounted for by change in joint torque or endpoint force, we estimated the change in stiffness on each trial based on the change in joint torque relative to the null field. For stable dynamics the change in endpoint stiffness was accurately predicted. However, for unstable dynamics the change in endpoint stiffness could not be reproduced. In fact, the predicted endpoint stiffness was similar to that in the null force field. Thus, the change in endpoint stiffness seen after adaptation to stable dynamics was directly related to changes in net joint torque necessary to compensate for the dynamics in contrast to adaptation to unstable dynamics, where a selective change in endpoint stiffness occurred without any modification of net joint torque.