Did We Get Sensorimotor Adaptation Wrong? Implicit Adaptation as Direct Policy Updating Rather than Forward-Model-Based Learning

Audiomotor prediction errors drive speech adaptation even in the absence of overt movement

Observed outcomes of our movements sometimes differ from our expectations. These sensory prediction errors recalibrate the brain's internal models for motor control, reflected in alterations to subsequent movements that counteract these errors (motor adaptation). While leading theories suggest that all forms of motor adaptation are driven by learning from sensory prediction errors, dominant models of speech adaptation argue that adaptation results from integrating time-advanced copies of corrective feedback commands into feedforward motor programs. Here, we tested these competing theories of speech adaptation by inducing planned, but not executed, speech. Human speakers (male and female) were prompted to speak a word and, on a subset of trials, were rapidly cued to withhold the prompted speech. On standard trials, speakers were exposed to real-time playback of their own speech with an auditory perturbation of the first formant to induce single-trial speech adaptation. Speakers experienced a similar sensory error on movement cancelation trials, hearing a perturbation applied to a recording of their speech from a previous trial at the time they would have spoken. Speakers adapted to auditory prediction errors in both contexts, altering the spectral content of spoken vowels to counteract formant perturbations even when no actual movement coincided with the perturbed feedback. These results build upon recent findings in reaching, and suggest that prediction errors, rather than corrective motor commands, drive adaptation in speech.

Working memory constraints for visuomotor retrieval strategies

Recent work has shown the fundamental role that cognitive strategies play in visuomotor adaptation. Although algorithmic strategies, such as mental rotation, are flexible and generalizable, they are computationally demanding. To avoid this computational cost, people can instead rely on memory retrieval of previously successful visuomotor solutions. However, such a strategy is likely subject to stimulus-response associations and rely heavily on working memory. In a series of five experiments, we sought to estimate the constraints in terms of capacity and precision of working memory retrieval for visuomotor adaptation. This was accomplished by leveraging different variations of visuomotor item-recognition and visuomotor rotation tasks where we associated unique rotations with specific targets in the workspace and manipulated the set size (i.e., number of rotation-target associations). Notably, from experiment 1 to 4, we found key signatures of working memory retrieval and not mental rotation. In particular, participants were less accurate and slower for larger set sizes and less recent items. Using a Bayesian latent-mixture model, we found that such decrease in performance was the result of increasing guessing behavior and less precise memories. In addition, we estimated that participants' working memory capacity was limited to two to five items, after which guessing increasingly dominated performance. Finally, in experiment 5, we showed how the constraints observed across experiments 1 to 4 can be overcome when relying on long-term memory retrieval. Our results point to the opportunity of studying other sources of memories where visuomotor solutions can be stored (e.g., episodic memories) to achieve successful adaptation.NEW & NOTEWORTHY We show that humans can adapt to feedback perturbations in different variations of the visuomotor rotation task by retrieving the successful solutions from working memory. In addition, using a Bayesian latent-mixture model, we reveal that guessing and low-precision memories are both responsible for the decrease in participants' performance as the number of solutions to memorize increases. These constraints can be overcome by relying on long-term memory retrieval resulting from extended practice with the visuomotor solutions.

Inductive biases of neural network modularity in spatial navigation

The brain may have evolved a modular architecture for daily tasks, with circuits featuring functionally specialized modules that match the task structure. We hypothesize that this architecture enables better learning and generalization than architectures with less specialized modules. To test this, we trained reinforcement learning agents with various neural architectures on a naturalistic navigation task. We found that the modular agent, with an architecture that segregates computations of state representation, value, and action into specialized modules, achieved better learning and generalization. Its learned state representation combines prediction and observation, weighted by their relative uncertainty, akin to recursive Bayesian estimation. This agent's behavior also resembles macaques' behavior more closely. Our results shed light on the possible rationale for the brain's modularity and suggest that artificial systems can use this insight from neuroscience to improve learning and generalization in natural tasks.

Distinct learning, retention, and generalization patterns in de novo learning versus motor adaptation

People correct for movement errors when acquiring new motor skills (de novo learning) or adapting well-known movements (motor adaptation). While de novo learning establishes new control policies, adaptation modifies existing ones, and previous work have distinguished behavioral and underlying brain mechanisms for each motor learning type. However, it is still unclear whether learning in each type interferes with the other. In study 1, we use a within-subjects design where participants train with both 30° visuomotor rotation and mirror reversal perturbations, to compare adaptation and de novo learning respectively. We find no perturbation order effects, and find no evidence for differences in learning rates and asymptotes for both perturbations. Explicit instructions also provide an advantage during early learning in both perturbations. However, mirror reversal learning shows larger inter-participant variability and slower movement initiation. Furthermore, we only observe reach aftereffects following rotation training. In study 2, we incorporate the mirror reversal in a browser-based task, to investigate under-studied de novo learning mechanisms like retention and generalization. Learning persists across three or more days, substantially transfers to the untrained hand, and to targets on both sides of the mirror axis. Our results extend insights for distinguishing motor skill acquisition from adapting well-known movements.

Explicit learning based on reward prediction error facilitates agile motor adaptations

Error based motor learning can be driven by both sensory prediction error and reward prediction error. Learning based on sensory prediction error is termed sensorimotor adaptation, while learning based on reward prediction error is termed reward learning. To investigate the characteristics and differences between sensorimotor adaptation and reward learning, we adapted a visuomotor paradigm where subjects performed arm movements while presented with either the sensory prediction error, signed end-point error, or binary reward. Before each trial, perturbation indicators in the form of visual cues were presented to inform the subjects of the presence and direction of the perturbation. To analyse the interconnection between sensorimotor adaptation and reward learning, we designed a computational model that distinguishes between the two prediction errors. Our results indicate that subjects adapted to novel perturbations irrespective of the type of prediction error they received during learning, and they converged towards the same movement patterns. Sensorimotor adaptations led to a pronounced aftereffect, while adaptation based on reward consequences produced smaller aftereffects suggesting that reward learning does not alter the internal model to the same degree as sensorimotor adaptation. Even though all subjects had learned to counteract two different perturbations separately, only those who relied on explicit learning using reward prediction error could timely adapt to the randomly changing perturbation. The results from the computational model suggest that sensorimotor and reward learning operate through distinct adaptation processes and that only sensorimotor adaptation changes the internal model, whereas reward learning employs explicit strategies that do not result in aftereffects. Additionally, we demonstrate that when humans learn motor tasks, they utilize both learning processes to successfully adapt to the new environments.

Human reaching control in dynamic environments

Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.

Visuomotor adaptation, internal modelling, and compensatory movements in children with developmental coordination disorder

BACKGROUND

Developmental coordination disorder (DCD) is one of the most prevalent developmental disorders in school-aged children. The mechanisms and etiology underlying DCD remain somewhat unclear. Altered visuomotor adaptation and internal model deficits are discussed in the literature.

AIMS

The study aimed to investigate visuomotor adaptation and internal modelling to determine whether and to what extent visuomotor learning might be impaired in children with DCD compared to typically developing children (TD). Further, possible compensatory movements during visuomotor learning were explored.

METHODS AND PROCEDURES

Participants were 12 children with DCD (age 12.4 ± 1.8, four female) and 18 age-matched TD (12.3 ± 1.8, five female). Visuomotor learning was measured with the Motor task manager. Compensatory movements were parameterized by spatial and temporal variables.

OUTCOMES AND RESULTS

Despite no differences in visuomotor adaptation or internal modelling, significant main effects for group were found in parameters representing movement accuracy, motor speed, and movement variability between DCD and TD.

CONCLUSIONS AND IMPLICATIONS

Children with DCD showed comparable performances in visuomotor adaptation and internal modelling to TD. However, movement variability was increased, whereas movement accuracy and motor speed were reduced, suggesting decreased motor acuity in children with DCD.

Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors-discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

A double dissociation between savings and long-term memory in motor learning

Memories are easier to relearn than learn from scratch. This advantage, known as savings, has been widely assumed to result from the reemergence of stable long-term memories. In fact, the presence of savings has often been used as a marker for whether a memory has been consolidated. However, recent findings have demonstrated that motor learning rates can be systematically controlled, providing a mechanistic alternative to the reemergence of a stable long-term memory. Moreover, recent work has reported conflicting results about whether implicit contributions to savings in motor learning are present, absent, or inverted, suggesting a limited understanding of the underlying mechanisms. To elucidate these mechanisms, we investigate the relationship between savings and long-term memory by experimentally dissecting the underlying memories based on short-term (60-s) temporal persistence. Components of motor memory that are temporally-persistent at 60 s might go on to contribute to stable, consolidated long-term memory, whereas temporally-volatile components that have already decayed away by 60 s cannot. Surprisingly, we find that temporally-volatile implicit learning leads to savings, whereas temporally-persistent learning does not, but that temporally-persistent learning leads to long-term memory at 24 h, whereas temporally-volatile learning does not. This double dissociation between the mechanisms for savings and long-term memory formation challenges widespread assumptions about the connection between savings and memory consolidation. Moreover, we find that temporally-persistent implicit learning not only fails to contribute to savings, but also that it produces an opposite, anti-savings effect, and that the interplay between this temporally-persistent anti-savings and temporally-volatile savings provides an explanation for several seemingly conflicting recent reports about whether implicit contributions to savings are present, absent, or inverted. Finally, the learning curves we observed for the acquisition of temporally-volatile and temporally-persistent implicit memories demonstrate the coexistence of implicit memories with distinct time courses, challenging the assertion that models of context-based learning and estimation should supplant models of adaptive processes with different learning rates. Together, these findings provide new insight into the mechanisms for savings and long-term memory formation.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Mesolimbic dopamine adapts the rate of learning from action

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions^1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction⁴; however, so far there has been little consideration of how direct policy learning might inform our understanding⁵. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning⁶.

Memory decay and generalization following distinct motor learning mechanisms

Motor skill learning is considered to arise out of contributions from multiple learning mechanisms, including error-based learning (EBL), use-dependent learning (UDL), and reinforcement learning (RL). These learning mechanisms exhibit dissociable roles and engage different neural circuits during skill acquisition. However, it remains largely unknown how a newly formed motor memory acquired through each learning mechanism decays over time and whether distinct learning mechanisms produce different generalization patterns. Here, we used variants of reaching paradigms that dissociated these learning mechanisms to examine the time course of memory decay following each learning and the generalization patterns of each learning. We found that motor memories acquired through these learning mechanisms decayed as a function of time. Notably, 15 min, 6 h, and 24 h after acquisition, the memory of EBL decayed much greater than that of RL. The memory acquired through UDL faded away within a few minutes. Motor memories formed through EBL and RL for given movement directions generalized to untrained movement directions, with the generalization of EBL being greater than that of RL. In contrast, motor memory of UDL could not generalize to untrained movement directions. These results suggest that distinct learning mechanisms exhibit different patterns of memory decay and generalization.NEW & NOTEWORTHY Motor skill learning is likely to involve error-based learning, use-dependent plasticity, and operant reinforcement. Here, we showed that these dissociable learning mechanisms exhibited distinct patterns of memory decay and generalization. With a better understanding of the characteristics of these learning mechanisms, it becomes possible to regulate each learning process separately to improve neurological rehabilitation.

Contextual effects in sensorimotor adaptation adhere to associative learning rules

Traditional associative learning tasks focus on the formation of associations between salient events and arbitrary stimuli that predict those events. This is exemplified in cerebellar-dependent delay eyeblink conditioning, where arbitrary cues such as a tone or light act as conditioned stimuli (CSs) that predict aversive sensations at the cornea (unconditioned stimulus [US]). Here, we ask if a similar framework could be applied to another type of cerebellar-dependent sensorimotor learning - sensorimotor adaptation. Models of sensorimotor adaptation posit that the introduction of an environmental perturbation results in an error signal that is used to update an internal model of a sensorimotor map for motor planning. Here, we take a step toward an integrative account of these two forms of cerebellar-dependent learning, examining the relevance of core concepts from associative learning for sensorimotor adaptation. Using a visuomotor adaptation reaching task, we paired movement-related feedback (US) with neutral auditory or visual contextual cues that served as CSs. Trial-by-trial changes in feedforward movement kinematics exhibited three key signatures of associative learning: differential conditioning, sensitivity to the CS-US interval, and compound conditioning. Moreover, after compound conditioning, a robust negative correlation was observed between responses to the two elemental CSs of the compound (i.e. overshadowing), consistent with the additivity principle posited by theories of associative learning. The existence of associative learning effects in sensorimotor adaptation provides a proof-of-concept for linking cerebellar-dependent learning paradigms within a common theoretical framework.

Motor learning without movement

Prediction errors guide many forms of learning, providing teaching signals that help us improve our performance. Implicit motor adaptation, for instance, is thought to be driven by sensory prediction errors (SPEs), which occur when the expected and observed consequences of a movement differ. Traditionally, SPE computation is thought to require movement execution. However, recent work suggesting that the brain can generate sensory predictions based on motor imagery or planning alone calls this assumption into question. Here, by measuring implicit motor adaptation during a visuomotor task, we tested whether motor planning and well-timed sensory feedback are sufficient for adaptation. Human participants were cued to reach to a target and were, on a subset of trials, rapidly cued to withhold these movements. Errors displayed both on trials with and without movements induced single-trial adaptation. Learning following trials without movements persisted even when movement trials had never been paired with errors and when the direction of movement and sensory feedback trajectories were decoupled. These observations indicate that the brain can compute errors that drive implicit adaptation without generating overt movements, leading to the adaptation of motor commands that are not overtly produced.

Implicit Adaptation Processes Promoted by Immediate Offline Visual and Numeric Feedback

In adaptation learning, visual feedback impacts how adaptation proceeds. With concurrent feedback, a more implicit/feedforward process is thought to be engaged, compared to feedback after movement, which promotes more explicit processes. Due to discrepancies across studies, related to timing and type of visual feedback, we isolated these conditions here. Four groups (N = 52) practiced aiming under rotated feedback conditions; feedback was provided concurrently, immediately after movement (visually or numerically), or visually after a 3 s delay. All groups adapted and only delayed feedback attenuated implicit adaptation as evidenced by post-practice after-effects. Contrary to some suggestions, immediately presented offline and numeric feedback resulted in implicit after-effects, potentially due to comparisons between feedforward information and seen or imagined feedback.

Human motor learning is robust to control-dependent noise

Noises are ubiquitous in sensorimotor interactions and contaminate the information provided to the central nervous system (CNS) for motor learning. An interesting question is how the CNS manages motor learning with imprecise information. Integrating ideas from reinforcement learning and adaptive optimal control, this paper develops a novel computational mechanism to explain the robustness of human motor learning to the imprecise information, caused by control-dependent noise that exists inherently in the sensorimotor systems. Starting from an initial admissible control policy, in each learning trial the mechanism collects and uses the noisy sensory data (caused by the control-dependent noise) to form an imprecise evaluation of the performance of the current policy and then constructs an updated policy based on the imprecise evaluation. As the number of learning trials increases, the generated policies mathematically provably converge to a (potentially small) neighborhood of the optimal policy under mild conditions, despite the imprecise information in the learning process. The mechanism directly synthesizes the policies from the sensory data, without identifying an internal forward model. Our preliminary computational results on two classic arm reaching tasks are in line with experimental observations reported in the literature. The model-free control principle proposed in the paper sheds more lights into the inherent robustness of human sensorimotor systems to the imprecise information, especially control-dependent noise, in the CNS.

A differential Hebbian framework for biologically-plausible motor control

In this paper we explore a neural control architecture that is both biologically plausible, and capable of fully autonomous learning. It consists of feedback controllers that learn to achieve a desired state by selecting the errors that should drive them. This selection happens through a family of differential Hebbian learning rules that, through interaction with the environment, can learn to control systems where the error responds monotonically to the control signal. We next show that in a more general case, neural reinforcement learning can be coupled with a feedback controller to reduce errors that arise non-monotonically from the control signal. The use of feedback control can reduce the complexity of the reinforcement learning problem, because only a desired value must be learned, with the controller handling the details of how it is reached. This makes the function to be learned simpler, potentially allowing learning of more complex actions. We use simple examples to illustrate our approach, and discuss how it could be extended to hierarchical architectures.

Evidence for an internal model of friction when controlling kinetic energy at impact to slide an object along a surface toward a target

Although the role of an internal model of gravity for the predictive control of the upper limbs is quite well established, evidence is lacking regarding an internal model of friction. In this study, 33 male and female human participants performed a striking movement (with the index finger) to slide a plastic cube-like object to a given target distance. The surface material (aluminum or balsa wood) on which the object slides, the surface slope (-10°, 0, or +10°) and the target distance (25 cm or 50 cm) varied across conditions, with ten successive trials in each condition. Analysis of the object speed at impact and spatial error suggests that: 1) the participants chose to impart a similar speed to the object in the first trial regardless of the surface material to facilitate the estimation of the coefficient of friction; 2) the movement is parameterized across repetitions to reduce spatial error; 3) an internal model of friction can be generalized when the slope changes. Biomechanical analysis showed interindividual variability in the recruitment of the upper limb segments and in the adjustment of finger speed at impact in order to transmit the kinetic energy required to slide the object to the target distance. In short, we provide evidence that the brain builds an internal model of friction that makes it possible to parametrically control a striking movement in order to regulate the amount of kinetic energy required to impart the appropriate initial speed to the object.

Revisiting the Role of the Medial Temporal Lobe in Motor Learning

Classic taxonomies of memory distinguish explicit and implicit memory systems, placing motor skills squarely in the latter branch. This assertion is in part a consequence of foundational discoveries showing significant motor learning in amnesics. Those findings suggest that declarative memory processes in the medial temporal lobe (MTL) do not contribute to motor learning. Here, we revisit this issue, testing an individual (L. S. J.) with severe MTL damage on four motor learning tasks and comparing her performance to age-matched controls. Consistent with previous findings in amnesics, we observed that L. S. J. could improve motor performance despite having significantly impaired declarative memory. However, she tended to perform poorly relative to age-matched controls, with deficits apparently related to flexible action selection. Further supporting an action selection deficit, L. S. J. fully failed to learn a task that required the acquisition of arbitrary action-outcome associations. We thus propose a modest revision to the classic taxonomic model: Although MTL-dependent memory processes are not necessary for some motor learning to occur, they play a significant role in the acquisition, implementation, and retrieval of action selection strategies. These findings have implications for our understanding of the neural correlates of motor learning, the psychological mechanisms of skill, and the theory of multiple memory systems.

Long-Term Motor Learning in the "Wild" With High Volume Video Game Data

Motor learning occurs over long periods of practice during which motor acuity, the ability to execute actions more accurately, precisely, and in less time, improves. Laboratory-based studies of motor learning are typically limited to a small number of participants and a time frame of minutes to several hours per participant. There is a need to assess the generalizability of theories and findings from lab-based motor learning studies on larger samples and time scales. In addition, laboratory-based studies of motor learning use relatively simple motor tasks which participants are unlikely to be intrinsically motivated to learn, limiting the interpretation of their findings in more ecologically valid settings ("in the wild"). We studied the acquisition and longitudinal refinement of a complex sensorimotor skill embodied in a first-person shooter video game scenario, with a large sample size (N = 7174, 682,564 repeats of the 60 s game) over a period of months. Participants voluntarily practiced the gaming scenario for up to several hours per day up to 100 days. We found improvement in performance accuracy (quantified as hit rate) was modest over time but motor acuity (quantified as hits per second) improved considerably, with 40-60% retention from 1 day to the next. We observed steady improvements in motor acuity across multiple days of video game practice, unlike most motor learning tasks studied in the lab that hit a performance ceiling rather quickly. Learning rate was a non-linear function of baseline performance level, amount of daily practice, and to a lesser extent, number of days between practice sessions. In addition, we found that the benefit of additional practice on any given day was non-monotonic; the greatest improvements in motor acuity were evident with about an hour of practice and 90% of the learning benefit was achieved by practicing 30 min per day. Taken together, these results provide a proof-of-concept in studying motor skill acquisition outside the confines of the traditional laboratory, in the presence of unmeasured confounds, and provide new insights into how a complex motor skill is acquired in an ecologically valid setting and refined across much longer time scales than typically explored.

How learning unfolds in the brain: toward an optimization view

How do changes in the brain lead to learning? To answer this question, consider an artificial neural network (ANN), where learning proceeds by optimizing a given objective or cost function. This "optimization framework" may provide new insights into how the brain learns, as many idiosyncratic features of neural activity can be recapitulated by an ANN trained to perform the same task. Nevertheless, there are key features of how neural population activity changes throughout learning that cannot be readily explained in terms of optimization and are not typically features of ANNs. Here we detail three of these features: (1) the inflexibility of neural variability throughout learning, (2) the use of multiple learning processes even during simple tasks, and (3) the presence of large task-nonspecific activity changes. We propose that understanding the role of these features in the brain will be key to describing biological learning using an optimization framework.

Hierarchical Control of Visually-Guided Movements in a 3D-Printed Robot Arm

The control architecture guiding simple movements such as reaching toward a visual target remains an open problem. The nervous system needs to integrate different sensory modalities and coordinate multiple degrees of freedom in the human arm to achieve that goal. The challenge increases due to noise and transport delays in neural signals, non-linear and fatigable muscles as actuators, and unpredictable environmental disturbances. Here we examined the capabilities of hierarchical feedback control models proposed by W. T. Powers, so far only tested in silico. We built a robot arm system with four degrees of freedom, including a visual system for locating the planar position of the hand, joint angle proprioception, and pressure sensing in one point of contact. We subjected the robot to various human-inspired reaching and tracking tasks and found features of biological movement, such as isochrony and bell-shaped velocity profiles in straight-line movements, and the speed-curvature power law in curved movements. These behavioral properties emerge without trajectory planning or explicit optimization algorithms. We then applied static structural perturbations to the robot: we blocked the wrist joint, tilted the writing surface, extended the hand with a tool, and rotated the visual system. For all of them, we found that the arm in machina adapts its behavior without being reprogrammed. In sum, while limited in speed and precision (by the nature of the do-it-yourself inexpensive components we used to build the robot from scratch), when faced with the noise, delays, non-linearities, and unpredictable disturbances of the real world, the embodied control architecture shown here balances biological realism with design simplicity.

Implicit adaptation to mirror reversal is in the correct coordinate system but the wrong direction

Learning in visuomotor adaptation tasks is the result of both explicit and implicit processes. Explicit processes, operationalized as reaiming an intended movement to a new goal, account for a significant proportion of learning. However, implicit processes, operationalized as error-dependent learning that gives rise to aftereffects, appear to be highly constrained. The limitations of implicit learning are highlighted in the mirror-reversal task, where implicit corrections act in opposition to performance. This is surprising given the mirror-reversal task has been viewed as emblematic of implicit learning. One potential issue not being considered in these studies is that both explicit and implicit processes were allowed to operate concurrently, which may interact, potentially in opposition. Therefore, we sought to further characterize implicit learning in a mirror-reversal task with a clamp design to isolate implicit learning from explicit strategies. We confirmed that implicit adaptation is in the wrong direction for mirror reversal and operates as if the perturbation were a rotation and only showed a moderate attenuation after 3 days of training. This result raised the question of whether implicit adaptation blindly operates as though perturbations were a rotation. In a separate experiment, which directly compared a mirror reversal and a rotation, we found that implicit adaptation operates in a proper coordinate system for different perturbations: adaptation to a mirror reversal and rotational perturbation is more consistent with Cartesian and polar coordinate systems, respectively. It remains an open question why implicit process would be flexible to the coordinate system of a perturbation but continue to be directed inappropriately.NEW & NOTEWORTHY Recent studies have found that implicit learning may operate inappropriately in some motor tasks, requiring explicit strategies to improve performance. However, this inappropriate adaptation could be attributable to competitive interactions between explicit and implicit processes. After isolating implicit processes, we found that implicit adaptation remained in the wrong direction for a mirror reversal, acting as if it were a rotation. Interestingly, however, the implicit system is sensitive to a particular coordinate system, treating mirror reversal and rotation differently.

Implicit Visuomotor Adaptation Remains Limited after Several Days of Training

Learning in sensorimotor adaptation tasks has been viewed as an implicit learning phenomenon. The implicit process affords recalibration of existing motor skills so that the system can adjust to changes in the body or environment without relearning from scratch. However, recent findings suggest that the implicit process is heavily constrained, calling into question its utility in motor learning and the theoretical framework of sensorimotor adaptation paradigms. These inferences have been based mainly on results from single bouts of training, where explicit compensation strategies, such as explicitly re-aiming the intended movement direction, contribute a significant proportion of adaptive learning. It is possible, however, that the implicit process supersedes explicit compensation strategies over repeated practice sessions. We tested this by dissociating the contributions of explicit re-aiming strategies and the implicit process in human participants over five consecutive days of training. Despite a substantially longer duration of training, the implicit process still plateaued at a value far short of complete learning and, as has been observed in previous studies, was inappropriate for a mirror-reversal task. Notably, we find significant between subject differences that call into question traditional interpretation of these group-level results.

De novo learning versus adaptation of continuous control in a manual tracking task

How do people learn to perform tasks that require continuous adjustments of motor output, like riding a bicycle? People rely heavily on cognitive strategies when learning discrete movement tasks, but such time-consuming strategies are infeasible in continuous control tasks that demand rapid responses to ongoing sensory feedback. To understand how people can learn to perform such tasks without the benefit of cognitive strategies, we imposed a rotation/mirror reversal of visual feedback while participants performed a continuous tracking task. We analyzed behavior using a system identification approach, which revealed two qualitatively different components of learning: adaptation of a baseline controller and formation of a new, task-specific continuous controller. These components exhibited different signatures in the frequency domain and were differentially engaged under the rotation/mirror reversal. Our results demonstrate that people can rapidly build a new continuous controller de novo and can simultaneously deploy this process with adaptation of an existing controller.

Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning

The neural mechanisms underlying stuttering remain poorly understood. A large body of work has focused on sensorimotor integration difficulties in individuals who stutter, including recently the capacity for sensorimotor learning. Typically, sensorimotor learning is assessed with adaptation paradigms in which one or more sensory feedback modalities are experimentally perturbed in real time. Our own previous work on speech with perturbed auditory feedback revealed substantial auditory-motor learning limitations in both children and adults who stutter (AWS). It remains unknown, however, which subprocesses of sensorimotor learning are impaired. Indeed, new insights from research on upper limb motor control indicate that sensorimotor learning involves at least two distinct components: (a) an explicit component that includes intentional strategy use and presumably is driven by target error and (b) an implicit component that updates an internal model without awareness of the learner and presumably is driven by sensory prediction error. Here, we attempted to dissociate these components for speech auditory-motor learning in AWS versus adults who do not stutter (AWNS). Our formant-shift auditory-motor adaptation results replicated previous findings that such sensorimotor learning is limited in AWS. Novel findings are that neither control nor stuttering participants reported any awareness of changing their productions in response to the auditory perturbation and that neither group showed systematic drift in auditory target judgments made throughout the adaptation task. These results indicate that speech auditory-motor adaptation to formant-shifted feedback relies exclusively on implicit learning processes. Thus, limited adaptation in AWS reflects poor implicit sensorimotor learning. Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning.

Response-based outcome predictions and confidence regulate feedback processing and learning

Influential theories emphasize the importance of predictions in learning: we learn from feedback to the extent that it is surprising, and thus conveys new information. Here, we explore the hypothesis that surprise depends not only on comparing current events to past experience, but also on online evaluation of performance via internal monitoring. Specifically, we propose that people leverage insights from response-based performance monitoring - outcome predictions and confidence - to control learning from feedback. In line with predictions from a Bayesian inference model, we find that people who are better at calibrating their confidence to the precision of their outcome predictions learn more quickly. Further in line with our proposal, EEG signatures of feedback processing are sensitive to the accuracy of, and confidence in, post-response outcome predictions. Taken together, our results suggest that online predictions and confidence serve to calibrate neural error signals to improve the efficiency of learning.

Combining Observation and Physical Practice: Benefits of an Interleaved Schedule for Visuomotor Adaptation and Motor Memory Consolidation

Visuomotor adaptation to novel environments can occur via non-physical means, such as observation. Observation does not appear to activate the same implicit learning processes as physical practice, rather it appears to be more strategic in nature. However, there is evidence that interspersing observational practice with physical practice can benefit performance and memory consolidation either through the combined benefits of separate processes or through a change in processes activated during observation trials. To test these ideas, we asked people to practice aiming to targets with visually rotated cursor feedback or engage in a combined practice schedule comprising physical practice and observation of projected videos showing successful aiming. Ninety-three participants were randomly assigned to one of five groups: massed physical practice (Act), distributed physical practice (Act+Rest), or one of 3 types of combined practice: alternating blocks (Obs_During), or all observation before (Obs_Pre) or after (Obs_Post) blocked physical practice. Participants received 100 practice trials (all or half were physical practice). All groups improved in adaptation trials and showed savings across the 24-h retention interval relative to initial practice. There was some forgetting for all groups, but the magnitudes were larger for physical practice groups. The Act and Obs_During groups were most accurate in retention and did not differ, suggesting that observation can serve as a replacement for physical practice if supplied intermittently and offers advantages above just resting. However, after-effects associated with combined practice were smaller than those for physical practice control groups, suggesting that beneficial learning effects as a result of observation were not due to activation of implicit learning processes. Reaction time, variable error, and post-test rotation drawings supported this conclusion that adaptation for observation groups was promoted by explicit/strategic processes.