Biologically informed NeuralODEs for genome-wide regulatory dynamics

BACKGROUND

Gene regulatory network (GRN) models that are formulated as ordinary differential equations (ODEs) can accurately explain temporal gene expression patterns and promise to yield new insights into important cellular processes, disease progression, and intervention design. Learning such gene regulatory ODEs is challenging, since we want to predict the evolution of gene expression in a way that accurately encodes the underlying GRN governing the dynamics and the nonlinear functional relationships between genes. Most widely used ODE estimation methods either impose too many parametric restrictions or are not guided by meaningful biological insights, both of which impede either scalability, explainability, or both.

RESULTS

We developed PHOENIX, a modeling framework based on neural ordinary differential equations (NeuralODEs) and Hill-Langmuir kinetics, that overcomes limitations of other methods by flexibly incorporating prior domain knowledge and biological constraints to promote sparse, biologically interpretable representations of GRN ODEs. We tested the accuracy of PHOENIX in a series of in silico experiments, benchmarking it against several currently used tools. We demonstrated PHOENIX's flexibility by modeling regulation of oscillating expression profiles obtained from synchronized yeast cells. We also assessed the scalability of PHOENIX by modeling genome-scale GRNs for breast cancer samples ordered in pseudotime and for B cells treated with Rituximab.

CONCLUSIONS

PHOENIX uses a combination of user-defined prior knowledge and functional forms from systems biology to encode biological "first principles" as soft constraints on the GRN allowing us to predict subsequent gene expression patterns in a biologically explainable manner.

Biologically informed NeuralODEs for genome-wide regulatory dynamics

Modeling dynamics of gene regulatory networks using ordinary differential equations (ODEs) allow a deeper understanding of disease progression and response to therapy, thus aiding in intervention optimization. Although there exist methods to infer regulatory ODEs, these are generally limited to small networks, rely on dimensional reduction, or impose non-biological parametric restrictions - all impeding scalability and explainability. PHOENIX is a neural ODE framework incorporating prior domain knowledge as soft constraints to infer sparse, biologically interpretable dynamics. Extensive experiments - on simulated and real data - demonstrate PHOENIX's unique ability to learn key regulatory dynamics while scaling to the whole genome.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

Error-independent effect of sensory uncertainty on motor learning when both feedforward and feedback control processes are engaged

Integrating sensory information during movement and adapting motor plans over successive movements are both essential for accurate, flexible motor behaviour. When an ongoing movement is off target, feedback control mechanisms update the descending motor commands to counter the sensed error. Over longer timescales, errors induce adaptation in feedforward planning so that future movements become more accurate and require less online adjustment from feedback control processes. Both the degree to which sensory feedback is integrated into an ongoing movement and the degree to which movement errors drive adaptive changes in feedforward motor plans have been shown to scale inversely with sensory uncertainty. However, since these processes have only been studied in isolation from one another, little is known about how they are influenced by sensory uncertainty in real-world movement contexts where they co-occur. Here, we show that sensory uncertainty may impact feedforward adaptation of reaching movements differently when feedback integration is present versus when it is absent. In particular, participants gradually adjust their movements from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error reduction. Riding on top of this slow envelope, participants exhibit large and abrupt changes in their initial movement vectors that are strongly correlated with the degree of sensory uncertainty present on the previous trial. However, these abrupt changes are insensitive to the magnitude and direction of the sensed movement error. These results prompt important questions for current models of sensorimotor learning under uncertainty and open up new avenues for future exploration in the field.

The Effect of Feedback Modality When Learning a Novel Wrist Sensorimotor Transformation Through a Body-Machine Interface

Body-Machine Interfaces (BoMIs) are promising assistive and rehabilitative tools for helping individuals with impaired motor abilities regain independence. When operating a BoMI, the user has to learn a novel sensorimotor transformation between the movement of certain body parts and the output of the device. In this study, we investigated how different feedback modalities impacted learning to operate a BoMI. Forty-seven able-bodied participants learned to control the velocity of a 1D cursor using the 3D rotation of their dominant wrist to reach as many targets as possible in a given amount of time. The map was designed to maximize cursor speed for movements around a predefined axis of wrist rotation. We compared the user's performance and control efficiency under three feedback modalities: i) visual feedback of the cursor position, ii) proprioceptive feedback of the cursor position delivered by a wrist manipulandum, iii) both i) and ii). We found that visual feedback led to a greater number of targets reached than proprioceptive feedback alone. Conversely, proprioceptive feedback yielded greater alignment between the axis of rotation of the wrist and the optimal axis represented by the map. These results suggest that proprioceptive feedback may be preferable over visual feedback when information about intrinsic task components, i.e. joint configurations, is of interest as in rehabilitative interventions aiming to promote more effective learning strategies.

Contextual inference in learning and memory

Context is widely regarded as a major determinant of learning and memory across numerous domains, including classical and instrumental conditioning, episodic memory, economic decision-making, and motor learning. However, studies across these domains remain disconnected due to the lack of a unifying framework formalizing the concept of context and its role in learning. Here, we develop a unified vernacular allowing direct comparisons between different domains of contextual learning. This leads to a Bayesian model positing that context is unobserved and needs to be inferred. Contextual inference then controls the creation, expression, and updating of memories. This theoretical approach reveals two distinct components that underlie adaptation, proper and apparent learning, respectively referring to the creation and updating of memories versus time-varying adjustments in their expression. We review a number of extensions of the basic Bayesian model that allow it to account for increasingly complex forms of contextual learning.

ReActLab: A Custom Framework for Sensorimotor Experiments “in-the-wild”

Over the last few years online platforms for running psychology experiments beyond simple questionnaires and surveys have become increasingly popular. This trend has especially increased after many laboratory facilities had to temporarily avoid in-person data collection following COVID-19-related lockdown regulations. Yet, while offering a valid alternative to in-person experiments in many cases, platforms for online experiments are still not a viable solution for a large part of human-based behavioral research. Two situations in particular pose challenges: First, when the research question requires design features or participant interaction which exceed the customization capability provided by the online platform; and second, when variation among hardware characteristics between participants results in an inadmissible confounding factor. To mitigate the effects of these limitations, we developed ReActLab (Remote Action Laboratory), a framework for programming remote, browser-based experiments using freely available and open-source JavaScript libraries. Since the experiment is run entirely within the browser, our framework allows for portability to any operating system and many devices. In our case, we tested our approach by running experiments using only a specific model of Android tablet. Using ReActLab with this standardized hardware allowed us to optimize our experimental design for our research questions, as well as collect data outside of laboratory facilities without introducing setup variation among participants. In this paper, we describe our framework and show examples of two different experiments carried out with it: one consisting of a visuomotor adaptation task, the other of a visual localization task. Through comparison with results obtained from similar tasks in in-person laboratory settings, we discuss the advantages and limitations for developing browser-based experiments using our framework.

Individual differences in error-related frontal midline theta activity during visuomotor adaptation

Post-feedback frontal midline EEG activity has been found to correlate with error magnitude during motor adaptation. However, the role of this neuronal activity remains to be elucidated. It has been hypothesized that post-feedback frontal midline activity may represent a prediction error, which in turn may be directly related to the adaptation process or to an unspecific orienting response. To address these hypotheses, we replicated a previous visuomotor adaptation experiment with very small perturbations, likely to invoke implicit adaptation, in a new group of 60 participants and combined it with EEG recordings. We found error-related peaks in the frontal midline electrodes in the time domain. However, these were best understood as modulations of frontal midline theta activity (FMT, 4-8 Hz). Trial-level differences in FMT correlated with error magnitude. This correlation was robust even for very small errors as well as in the absence of imposed perturbations, indicating that FMT does not depend on explicit or strategic re-aiming. Within participants, trial-level differences in FMT were not related to between-trial error corrections. Between participants, individual differences in FMT-error-sensitivity did not predict differences in adaptation rate. Taken together, these results imply that FMT does not drive implicit motor adaptation. Finally, individual differences in FMT-error-sensitivity negatively correlate to motor execution noise. This suggests that FMT reflects saliency: larger execution noise means a larger standard deviation of errors so that a fixed error magnitude is less salient. In conclusion, this study suggests that frontal midline theta activity represents a saliency signal and does not directly drive motor adaptation.

Pitfalls in quantifying exploration in reward-based motor learning and how to avoid them

When learning a movement based on binary success information, one is more variable following failure than following success. Theoretically, the additional variability post-failure might reflect exploration of possibilities to obtain success. When average behavior is changing (as in learning), variability can be estimated from differences between subsequent movements. Can one estimate exploration reliably from such trial-to-trial changes when studying reward-based motor learning? To answer this question, we tried to reconstruct the exploration underlying learning as described by four existing reward-based motor learning models. We simulated learning for various learner and task characteristics. If we simply determined the additional change post-failure, estimates of exploration were sensitive to learner and task characteristics. We identified two pitfalls in quantifying exploration based on trial-to-trial changes. Firstly, performance-dependent feedback can cause correlated samples of motor noise and exploration on successful trials, which biases exploration estimates. Secondly, the trial relative to which trial-to-trial change is calculated may also contain exploration, which causes underestimation. As a solution, we developed the additional trial-to-trial change (ATTC) method. By moving the reference trial one trial back and subtracting trial-to-trial changes following specific sequences of trial outcomes, exploration can be estimated reliably for the three models that explore based on the outcome of only the previous trial. Since ATTC estimates are based on a selection of trial sequences, this method requires many trials. In conclusion, if exploration is a binary function of previous trial outcome, the ATTC method allows for a model-free quantification of exploration.

Emergence of complex dynamics of choice due to repeated exposures to extinction learning

Extinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is not only essential for survival in a changing environment, but also plays a fundamental role in the treatment of pathological behaviors. During therapy and other forms of training involving extinction, subjects are typically exposed to several sessions with a similar structure. The effects of this repeated exposure are not well understood. Here, we studied the behavior of pigeons across several sessions of a discrimination-learning task in context A, extinction in context B, and a return to context A to test the context-dependent return of the learned responses (ABA renewal). By focusing on individual learning curves across animals, we uncovered a session-dependent variability of behavior: (1) during extinction, pigeons preferred the unrewarded alternative choice in one-third of the sessions, predominantly during the first one. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerged, and (3) the renewal effect decayed as sessions progressed. We show that the observed results can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the power of “simple” associative learning processes.

An analytical method reduces noise bias in motor adaptation analysis

When a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction, quantified as a trial-by-trial adaptation rate, provides insight into how the nervous system is operating, particularly regarding how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis has required carefully controlled laboratory conditions such as the application of perturbations or error clamping, limiting the usefulness of motor analysis in clinical and everyday environments. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, we show that the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner’s intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions.

Modeling inter-trial variability of pointing movements during visuomotor adaptation

Trial-to-trial variability during visuomotor adaptation is usually explained as the result of two different sources, planning noise and execution noise. The estimation of the underlying variance parameters from observations involving varying feedback conditions cannot be achieved by standard techniques (Kalman filter) because they do not account for recursive noise propagation in a closed-loop system. We therefore developed a method to compute the exact likelihood of the output of a time-discrete and linear adaptation system as has been used to model visuomotor adaptation (Smith et al. in PLoS Biol 4(6):e179, 2006), observed under closed-loop and error-clamp conditions. We identified the variance parameters by maximizing this likelihood and compared the model prediction of the time course of variance and autocovariance with empiric data. The observed increase in variability during the early training phase could not be explained by planning noise and execution noise with constant variances. Extending the model by signal-dependent components of either execution noise or planning noise showed that the observed temporal changes of the trial-to-trial variability can be modeled by signal-dependent planning noise rather than signal-dependent execution noise. Comparing the variance time course between different training schedules showed that the signal-dependent increase of planning variance was specific for the fast adapting mechanism, whereas the assumption of constant planning variance was sufficient for the slow adapting mechanisms.

Personalized Online Adaptation of Kinematic Synergies for Human-Prosthesis Interfaces

Synergies have been adopted in prosthetic limb applications to reduce the complexity of design, but typically involve a single synergy setting for a population and ignore individual preference or adaptation capacity. However, personalization of the synergy setting is necessary for the effective operation of the prosthetic device. Two major challenges hinder the personalization of synergies in human-prosthesis interfaces (HPIs). The first is related to the process of human motor adaptation and the second to the variation in motor learning dynamics of individuals. In this paper, a systematic personalization of kinematic synergies for HPIs using online measurements from each individual is proposed. The task of reaching using the upper limb is described by an objective function and the interface is parameterized by a kinematic synergy. Consequently, personalizing the interface for a given individual can be formulated as finding an optimal personalized parameter. A structure to model the observed motor behavior that allows for the personalized traits of motor preference and motor learning is proposed, and subsequently used in an online optimization scheme to identify the synergies for an individual. The knowledge of the common features contained in the model enables online adaptation of the HPI to happen concurrently to human motor adaptation without the need to retune the personalization algorithm for each individual. Human-in-the-loop experimental results with able-bodied subjects, performed in a virtual reality environment to emulate amputation and prosthesis use, show that the proposed personalization algorithm was effective in obtaining optimal synergies with a fast uniform convergence speed across a group of individuals.

Time course of changes in the long-latency feedback response parallels the fast process of short-term motor adaptation

Adapting to novel dynamics involves modifying both feedforward and feedback control. We investigated whether the motor system alters feedback responses during adaptation to a novel force field in a manner similar to adjustments in feedforward control. We simultaneously tracked the time course of both feedforward and feedback systems via independent probes during a force field adaptation task. Participants (n = 35) grasped the handle of a robotic manipulandum and performed reaches to a visual target while the hand and arm were occluded. We introduced an abrupt counterclockwise velocity-dependent force field during a block of reaching trials. We measured movement kinematics and shoulder and elbow muscle activity with surface EMG electrodes. We tracked the feedback stretch response throughout the task. Using force channel trials, we measured overall learning, which was later decomposed into a fast and slow process. We found that the long-latency feedback response (LLFR) was upregulated in the early stages of learning and was correlated with the fast component of feedforward adaptation. The change in feedback response was specific to the long-latency epoch (50-100 ms after muscle stretch) and was observed only in the triceps muscle, which was the muscle required to counter the force field during adaptation. The similarity in time course for the LLFR and the estimated time course of the fast process suggests both are supported by common neural circuits. While some propose that the fast process reflects an explicit strategy, we argue instead that it may be a proxy for the feedback controller.NEW & NOTEWORTHY We investigated whether changes in the feedback stretch response were related to the proposed fast and slow processes of motor adaptation. We found that the long-latency component of the feedback stretch response was upregulated in the early stages of learning and the time course was correlated with the fast process. While some propose that the fast process reflects an explicit strategy, we argue instead that it may be a proxy for the feedback controller.

Sensorimotor adaptation and cue reweighting compensate for distorted 3D shape information, accounting for paradoxical perception-action dissociations

Visually guided movements can show surprising accuracy even when the perceived three-dimensional (3D) shape of the target is distorted. One explanation of this paradox is that an evolutionarily specialized “vision-for-action” system provides accurate shape estimates by relying selectively on stereo information and ignoring less reliable sources of shape information like texture and shading. However, the key support for this hypothesis has come from studies that analyze average behavior across many visuomotor interactions where available sensory feedback reinforces stereo information. The present study, which carefully accounts for the effects of feedback, shows that visuomotor interactions with slanted surfaces are actually planned using the same cue-combination function as slant perception and that apparent dissociations can arise due to two distinct supervised learning processes: sensorimotor adaptation and cue reweighting. In two experiments, we show that when a distorted slant cue biases perception (e.g., surfaces appear flattened by a fixed amount), sensorimotor adaptation rapidly adjusts the planned grip orientation to compensate for this constant error. However, when the distorted slant cue is unreliable, leading to variable errors across a set of objects (i.e., some slants are overestimated, others underestimated), then relative cue weights are gradually adjusted to reduce the misleading effect of the unreliable cue, consistent with previous perceptual studies of cue reweighting. The speed and flexibility of these two forms of learning provide an alternative explanation of why perception and action are sometimes found to be dissociated in experiments where some 3D shape cues are consistent with sensory feedback while others are faulty.

NEW & NOTEWORTHY When interacting with three-dimensional (3D) objects, sensory feedback is available that could improve future performance via supervised learning. Here we confirm that natural visuomotor interactions lead to sensorimotor adaptation and cue reweighting, two distinct learning processes uniquely suited to resolve errors caused by biased and noisy 3D shape cues. These findings explain why perception and action are often found to be dissociated in experiments where some cues are consistent with sensory feedback while others are faulty.

The Origins of Anterograde Interference in Visuomotor Adaptation

Anterograde interference refers to the negative impact of prior learning on the propensity for future learning. There is currently no consensus on whether this phenomenon is transient or long lasting, with studies pointing to an effect in the time scale of hours to days. These inconsistencies might be caused by the method employed to quantify performance, which often confounds changes in learning rate and retention. Here, we aimed to unveil the time course of anterograde interference by tracking its impact on visuomotor adaptation at different intervals throughout a 24-h period. Our empirical and model-based approaches allowed us to measure the capacity for new learning separately from the influence of a previous memory. In agreement with previous reports, we found that prior learning persistently impaired the initial level of performance upon revisiting the task. However, despite this strong initial bias, learning capacity was impaired only when conflicting information was learned up to 1 h apart, recovering thereafter with passage of time. These findings suggest that when adapting to conflicting perturbations, impairments in performance are driven by two distinct mechanisms: a long-lasting bias that acts as a prior and hinders initial performance and a short-lasting anterograde interference that originates from a reduction in error sensitivity.

Memories of motor adaptation do not necessarily decay with behavioral unlearning

Motor adaptation reshapes behaviors to habituate novel predictable demands caused by dramatic changes in our body (or environment). In the absence of error signals, behaviors rapidly return to the manner before adaptation. It is still in debate whether this behavioral unlearning is due to memory decay. Recent studies suggested that unlearning may be related to the detection of a context change between adaptation phase and error-absent phase. This context-dependent idea is extended in the present study, which examined the motor adaptation in a ball-tossing task. To facilitate the manipulation of the task and the measurement of the behavior, this tossing task was conducted in a virtual environment. Experiment 1 found that unlearning was more likely to occur when the context in the adaptation phase was less similar to that in the error-absent phase. Experiment 2 further demonstrated that the memory of motor adaptation can bias behavior even after behavioral unlearning. Experiment 3 confirmed that the results in Experiment 1 and 2 were not artifacts. These findings indicate that memories of adaptation are independent of behavioral unlearning, and the contextual similarity between adaptation and error-absent phase determines the unlearning rate.

A generative learning model for saccade adaptation

Plasticity in the oculomotor system ensures that saccadic eye movements reliably meet their visual goals-to bring regions of interest into foveal, high-acuity vision. Here, we present a comprehensive description of sensorimotor learning in saccades. We induced continuous adaptation of saccade amplitudes using a double-step paradigm, in which participants saccade to a peripheral target stimulus, which then undergoes a surreptitious, intra-saccadic shift (ISS) as the eyes are in flight. In our experiments, the ISS followed a systematic variation, increasing or decreasing from one saccade to the next as a sinusoidal function of the trial number. Over a large range of frequencies, we confirm that adaptation gain shows (1) a periodic response, reflecting the frequency of the ISS with a delay of a number of trials, and (2) a simultaneous drift towards lower saccade gains. We then show that state-space-based linear time-invariant systems (LTIS) represent suitable generative models for this evolution of saccade gain over time. This state-equation algorithm computes the prediction of an internal (or hidden state-) variable by learning from recent feedback errors, and it can be compared to experimentally observed adaptation gain. The algorithm also includes a forgetting rate that quantifies per-trial leaks in the adaptation gain, as well as a systematic, non-error-based bias. Finally, we study how the parameters of the generative models depend on features of the ISS. Driven by a sinusoidal disturbance, the state-equation admits an exact analytical solution that expresses the parameters of the phenomenological description as functions of those of the generative model. Together with statistical model selection criteria, we use these correspondences to characterize and refine the structure of compatible state-equation models. We discuss the relation of these findings to established results and suggest that they may guide further design of experimental research across domains of sensorimotor adaptation.

The influence of task outcome on implicit motor learning

Recent studies have demonstrated that task success signals can modulate learning during sensorimotor adaptation tasks, primarily through engaging explicit processes. Here, we examine the influence of task outcome on implicit adaptation, using a reaching task in which adaptation is induced by feedback that is not contingent on actual performance. We imposed an invariant perturbation (rotation) on the feedback cursor while varying the target size. In this way, the cursor either hit or missed the target, with the former producing a marked attenuation of implicit motor learning. We explored different computational architectures that might account for how task outcome information interacts with implicit adaptation. The results fail to support an architecture in which adaptation operates in parallel with a model-free operant reinforcement process. Rather, task outcome may serve as a gain on implicit adaptation or provide a distinct error signal for a second, independent implicit learning process.

Editorial note

This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).

Both fast and slow learning processes contribute to savings following sensorimotor adaptation

Recent work suggests that the rate of learning in sensorimotor adaptation is likely not fixed, but rather can change based on previous experience. One example is savings, a commonly observed phenomenon whereby the relearning of a motor skill is faster than the initial learning. Sensorimotor adaptation is thought to be driven by sensory prediction errors, which are the result of a mismatch between predicted and actual sensory consequences. It has been proposed that during motor adaptation the generation of sensory prediction errors engages two processes (fast and slow) that differ in learning and retention rates. We tested the idea that a history of errors would influence both the fast and slow processes during savings. Participants were asked to perform the same force field adaptation task twice in succession. We found that adaptation to the force field a second time led to increases in estimated learning rates for both fast and slow processes. While it has been proposed that savings is explained by an increase in learning rate for the fast process, here we observed that the slow process also contributes to savings. Our work suggests that fast and slow adaptation processes are both responsive to a history of error and both contribute to savings. NEW & NOTEWORTHY We studied the underlying mechanisms of savings during motor adaptation. Using a two-state model to represent fast and slow processes that contribute to motor adaptation, we found that a history of error modulates performance in both processes. While previous research has attributed savings to only changes in the fast process, we demonstrated that an increase in both processes is needed to account for the measured behavioral data.

Does visuomotor adaptation contribute to illusion-resistant grasping?

Do illusory distortions of perceived object size influence how wide the hand is opened during a grasping movement? Many studies on this question have reported illusion-resistant grasping, but this finding has been contradicted by other studies showing that grasping movements and perceptual judgments are equally susceptible. One largely unexplored explanation for these contradictions is that illusion effects on grasping can be reduced with repeated movements. Using a visuomotor adaptation paradigm, we investigated whether an adaptation model could predict the time course of Ponzo illusion effects on grasping. Participants performed a series of trials in which they viewed a thin wooden target, manually reported an estimate of the target's length, then reached to grasp the target. Manual size estimates (MSEs) were clearly biased by the illusion, but maximum grip apertures (MGAs) of grasping movements were consistently accurate. Illusion-resistant MGAs were observed immediately upon presentation of the illusion, so there was no decrement in susceptibility for the adaptation model to explain. To determine whether online corrections based on visual feedback could have produced illusion-resistant MGAs, we performed an exploratory post hoc analysis of movement trajectories. Early portions of the illusion effect profile evolved as if they were biased by the illusion to the same magnitude as the perceptual responses (MSEs), but this bias was attenuated prior to the MGA. Overall, this preregistered study demonstrated that visuomotor adaptation of grasping is not the primary source of illusion resistance in closed-loop grasping.

Relative sensitivity of explicit reaiming and implicit motor adaptation

It has become increasingly clear that learning in visuomotor rotation tasks, which induce an angular mismatch between movements of the hand and visual feedback, largely results from the combined effort of two distinct processes: implicit motor adaptation and explicit reaiming. However, it remains unclear how these two processes work together to produce trial-by-trial learning. Previous work has found that implicit motor adaptation operates automatically, regardless of task relevance, and saturates for large errors. In contrast, little is known about the automaticity of explicit reaiming and its sensitivity to error magnitude. Here we sought to characterize the automaticity and sensitivity function of these two processes to determine how they work together to facilitate performance in a visuomotor rotation task. We found that implicit adaptation scales relative to the visual error but only for small perturbations-replicating prior work. In contrast, explicit reaiming scales linearly for all tested perturbation sizes. Furthermore, the consistency of the perturbation appears to diminish both implicit adaptation and explicit reaiming, but to different degrees. Whereas implicit adaptation always displayed a response to the error, explicit reaiming was only engaged when errors displayed a minimal degree of consistency. This comports with the idea that implicit adaptation is obligatory and less flexible, whereas explicit reaiming is volitional and flexible. NEW & NOTEWORTHY This paper provides the first psychometric sensitivity function for explicit reaiming. Additionally, we show that the sensitivities of both implicit adaptation and explicit reaiming are influenced by consistency of errors. The pattern of results across two experiments further supports the idea that implicit adaptation is largely inflexible, whereas explicit reaiming is flexible and can be suppressed when unnecessary.

Invariant errors reveal limitations in motor correction rather than constraints on error sensitivity

Implicit sensorimotor adaptation is traditionally described as a process of error reduction, whereby a fraction of the error is corrected for with each movement. Here, in our study of healthy human participants, we characterize two constraints on this learning process: the size of adaptive corrections is only related to error size when errors are smaller than 6°, and learning functions converge to a similar level of asymptotic learning over a wide range of error sizes. These findings are problematic for current models of sensorimotor adaptation, and point to a new theoretical perspective in which learning is constrained by the size of the error correction, rather than sensitivity to error.

Estimating properties of the fast and slow adaptive processes during sensorimotor adaptation

Experience of a prediction error recruits multiple motor learning processes, some that learn strongly from error but have weak retention and some that learn weakly from error but exhibit strong retention. These processes are not generally observable but are inferred from their collective influence on behavior. Is there a robust way to uncover the hidden processes? A standard approach is to consider a state space model where the hidden states change following experience of error and then fit the model to the measured data by minimizing the squared error between measurement and model prediction. We found that this least-squares algorithm (LMSE) often yielded unrealistic predictions about the hidden states, possibly because of its neglect of the stochastic nature of error-based learning. We found that behavioral data during adaptation was better explained by a system in which both error-based learning and movement production were stochastic processes. To uncover the hidden states of learning, we developed a generalized expectation maximization (EM) algorithm. In simulation, we found that although LMSE tracked the measured data marginally better than EM, EM was far more accurate in unmasking the time courses and properties of the hidden states of learning. In a power analysis designed to measure the effect of an intervention on sensorimotor learning, EM significantly reduced the number of subjects that were required for effective hypothesis testing. In summary, we developed a new approach for analysis of data in sensorimotor experiments. The new algorithm improved the ability to uncover the multiple processes that contribute to learning from error. NEW & NOTEWORTHY Motor learning is supported by multiple adaptive processes, each with distinct error sensitivity and forgetting rates. We developed a generalized expectation maximization algorithm that uncovers these hidden processes in the context of modern sensorimotor learning experiments that include error-clamp trials and set breaks. The resulting toolbox may improve the ability to identify the properties of these hidden processes and reduce the number of subjects needed to test the effectiveness of interventions on sensorimotor learning.

The Structure and Acquisition of Sensorimotor Maps

One of the puzzles of learning to talk or play a musical instrument is how we learn which movement produces a particular sound: an audiomotor map. Existing research has used mappings that are already well learned such as controlling a cursor using a computer mouse. By contrast, the acquisition of novel sensorimotor maps was studied by having participants learn arm movements to auditory targets. These sounds did not come from different directions but, like speech, were only distinguished by their frequencies. It is shown that learning involves forming not one but two maps: a point map connecting sensory targets with motor commands and an error map linking sensory errors to motor corrections. Learning a point map is possible even when targets never repeat. Thus, although participants make errors, there is no opportunity to correct them because the target is different on every trial, and therefore learning cannot be driven by error correction. Furthermore, when the opportunity for error correction is provided, it is seen that acquiring error correction is itself a learning process that changes over time and results in an error map. In principle, the error map could be derived from the point map, but instead, these two maps are independently acquired and jointly enable sensorimotor control and learning. A computational model shows that this dual encoding is optimal and simulations based on this architecture predict that learning the two maps results in performance improvements comparable with those observed empirically.

Computational rehabilitation of neglect: Using state-space models to understand the recovery mechanisms

Unilateral spatial neglect is a neuropsychological syndrome often observed in right hemisphere stroke patients. The symptoms differ from subject to subject. A few rehabilitation approaches, e.g. prism adaptation, have demonstrated some effect in reducing the symptoms, but the underlying mechanisms are still largely unclear. Recently, neural models have been proposed to qualitatively describe cortical lesions, the resulting neglect symptoms and the effects of treatment. However, these predictions are qualitative and cannot be used to compare different hypotheses or to interpret symptoms at individual subjects level. Here we propose a computational model of the trial-by-trial dynamics of training-induced recovery from neglect. Neglect is modelled in terms of an impaired internal representation of visual stimuli in the left hemispace. The model assumes that recovery is driven by the mismatch between defective representations of visual stimuli and the corresponding hand positions. The model reproduces the main observations of prism adaptation experiments. Using standard system identification techniques, we fitted the model to data from a rehabilitation trial based on a novel rehabilitation approach based on virtual reality, involving reaching movements within an adaptive environment. Our results suggest that the model can be used to interpret data from individual subjects and to formulate testable hypotheses on the mechanisms of recovery and directions for treatment.

Limited Plasticity of Prismatic Visuomotor Adaptation

Movements toward an object displaced optically through prisms adapt quickly, a striking example for the plasticity of neuronal visuomotor programs. We investigated the degree and time course of this system’s plasticity. Participants performed goal-directed throwing or pointing movements with terminal feedback before, during, and after wearing prism goggles shifting the visual world laterally either to the right or to the left. Prism adaptation was incomplete even after 240 throwing movements, still deviating significantly laterally by on average of 0.8° (CI = 0.20°) at the end of the adaptation period. The remaining lateral deviation was significant for pointing movements only with left shifting prisms. In both tasks, removal of the prisms led to an aftereffect which disappeared in the course of further training. This incomplete prism adaptation may be caused by movement variability combined with an adaptive neuronal control system exhibiting a finite capacity for evaluating movement errors.

Characteristics of Implicit Sensorimotor Adaptation Revealed by Task-irrelevant Clamped Feedback

Sensorimotor adaptation occurs when there is a discrepancy between the expected and actual sensory consequences of a movement. This learning can be precisely measured, but its source has been hard to pin down because standard adaptation tasks introduce two potential learning signals: task performance errors and sensory prediction errors. Here we employed a new method that induces sensory prediction errors without task performance errors. This method combines the use of clamped visual feedback that is angularly offset from the target and independent of the direction of motion, along with instructions to ignore this feedback while reaching to targets. Despite these instructions, participants unknowingly showed robust adaptation of their movements. This adaptation was similar to that observed with standard methods, showing sign dependence, local generalization, and cerebellar dependency. Surprisingly, adaptation rate and magnitude were invariant across a large range of offsets. Collectively, our results challenge current models of adaptation and demonstrate that behavior observed in many studies of adaptation reflect the composite effects of task performance and sensory prediction errors.

The role of auditory feedback in music-supported stroke rehabilitation: A single-blinded randomised controlled intervention

PURPOSE

Learning to play musical instruments such as piano was previously shown to benefit post-stroke motor rehabilitation. Previous work hypothesised that the mechanism of this rehabilitation is that patients use auditory feedback to correct their movements and therefore show motor learning. We tested this hypothesis by manipulating the auditory feedback timing in a way that should disrupt such error-based learning.

METHODS

We contrasted a patient group undergoing music-supported therapy on a piano that emits sounds immediately (as in previous studies) with a group whose sounds are presented after a jittered delay. The delay was not noticeable to patients. Thirty-four patients in early stroke rehabilitation with moderate motor impairment and no previous musical background learned to play the piano using simple finger exercises and familiar children's songs.

RESULTS

Rehabilitation outcome was not impaired in the jitter group relative to the normal group. Conversely, some clinical tests suggests the jitter group outperformed the normal group.

CONCLUSIONS

Auditory feedback-based motor learning is not the beneficial mechanism of music-supported therapy. Immediate auditory feedback therapy may be suboptimal. Jittered delay may increase efficacy of the proposed therapy and allow patients to fully benefit from motivational factors of music training. Our study shows a novel way to test hypotheses concerning music training in a single-blinded way, which is an important improvement over existing unblinded tests of music interventions.

Saccadic adaptation to a systematically varying disturbance

Saccadic adaptation maintains the correct mapping between eye movements and their targets, yet the dynamics of saccadic gain changes in the presence of systematically varying disturbances has not been extensively studied. Here we assessed changes in the gain of saccade amplitudes induced by continuous and periodic postsaccadic visual feedback. Observers made saccades following a sequence of target steps either along the horizontal meridian (Two-way adaptation) or with unconstrained saccade directions (Global adaptation). An intrasaccadic step-following a sinusoidal variation as a function of the trial number (with 3 different frequencies tested in separate blocks)-consistently displaced the target along its vector. The oculomotor system responded to the resulting feedback error by modifying saccade amplitudes in a periodic fashion with similar frequency of variation but lagging the disturbance by a few tens of trials. This periodic response was superimposed on a drift toward stronger hypometria with similar asymptotes and decay rates across stimulus conditions. The magnitude of the periodic response decreased with increasing frequency and was smaller and more delayed for Global than Two-way adaptation. These results suggest that-in addition to the well-characterized return-to-baseline response observed in protocols using constant visual feedback-the oculomotor system attempts to minimize the feedback error by integrating its variation across trials. This process resembles a convolution with an internal response function, whose structure would be determined by coefficients of the learning model. Our protocol reveals this fast learning process in single short experimental sessions, qualifying it for the study of sensorimotor learning in health and disease.

Noise Induces Biased Estimation of the Correction Gain

The detection of an error in the motor output and the correction in the next movement are critical components of any form of motor learning. Accordingly, a variety of iterative learning models have assumed that a fraction of the error is adjusted in the next trial. This critical fraction, the correction gain, learning rate, or feedback gain, has been frequently estimated via least-square regression of the obtained data set. Such data contain not only the inevitable noise from motor execution, but also noise from measurement. It is generally assumed that this noise averages out with large data sets and does not affect the parameter estimation. This study demonstrates that this is not the case and that in the presence of noise the conventional estimate of the correction gain has a significant bias, even with the simplest model. Furthermore, this bias does not decrease with increasing length of the data set. This study reveals this limitation of current system identification methods and proposes a new method that overcomes this limitation. We derive an analytical form of the bias from a simple regression method (Yule-Walker) and develop an improved identification method. This bias is discussed as one of other examples for how the dynamics of noise can introduce significant distortions in data analysis.

Development of a Portable Motor Learning Laboratory (PoMLab)

Most motor learning experiments have been conducted in a laboratory setting. In this type of setting, a huge and expensive manipulandum is frequently used, requiring a large budget and wide open space. Subjects also need to travel to the laboratory, which is a burden for them. This burden is particularly severe for patients with neurological disorders. Here, we describe the development of a novel application based on Unity3D and smart devices, e.g., smartphones or tablet devices, that can be used to conduct motor learning experiments at any time and in any place, without requiring a large budget and wide open space and without the burden of travel on subjects. We refer to our application as POrtable Motor learning LABoratory, or PoMLab. PoMLab is a multiplatform application that is available and sharable for free. We investigated whether PoMLab could be an alternative to the laboratory setting using a visuomotor rotation paradigm that causes sensory prediction error, enabling the investigation of how subjects minimize the error. In the first experiment, subjects could adapt to a constant visuomotor rotation that was abruptly applied at a specific trial. The learning curve for the first experiment could be modeled well using a state space model, a mathematical model that describes the motor leaning process. In the second experiment, subjects could adapt to a visuomotor rotation that gradually increased each trial. The subjects adapted to the gradually increasing visuomotor rotation without being aware of the visuomotor rotation. These experimental results have been reported for conventional experiments conducted in a laboratory setting, and our PoMLab application could reproduce these results. PoMLab can thus be considered an alternative to the laboratory setting. We also conducted follow-up experiments in university physical education classes. A state space model that was fit to the data obtained in the laboratory experiments could predict the learning curves obtained in the follow-up experiments. Further, we investigated the influence of vibration function, weight, and screen size on learning curves. Finally, we compared the learning curves obtained in the PoMLab experiments to those obtained in the conventional reaching experiments. The results of the in-class experiments show that PoMLab can be used to conduct motor learning experiments at any time and place.

Neural coordination during reach-to-grasp

When reaching to grasp, we coordinate how we preshape the hand with how we move it. To ask how motor cortical neurons participate in this coordination, we examined the interactions between reach- and grasp-related neuronal ensembles while monkeys reached to grasp a variety of different objects in different locations. By describing the dynamics of these two ensembles as trajectories in a low-dimensional state space, we examined their coupling in time. We found evidence for temporal compensation across many different reach-to-grasp conditions such that if one neural trajectory led in time the other tended to catch up, reducing the asynchrony between the trajectories. Granger causality revealed bidirectional interactions between reach and grasp neural trajectories beyond that which could be attributed to the joint kinematics that were consistently stronger in the grasp-to-reach direction. Characterizing cortical coordination dynamics provides a new framework for understanding the functional interactions between neural populations.

Model-Aided Navigation with Sea Current Estimation for an Autonomous Underwater Vehicle

This paper presents a strategy to improve the navigation solution of the HRC-AUV by deploying a model-aided inertial navigation system (MA-INS). Based on a simpler three-DOF linear dynamic model (DM) of the vehicle, and implemented through a Kalman filter (KF), the performance of the proposed MA-INS is compared to state-of-the-art solutions based on non-linear models. The model allows the online estimation of the sea current parameters before and during the navigation mission. Qualitative and quantitative evaluations as well as a statistical significance test are performed using both simulated and real data, demonstrating the usefulness of the proposed model-aided navigation.

A memory of errors in sensorimotor learning

The current view of motor learning suggests that when we revisit a task, the brain recalls the motor commands it previously learned. In this view, motor memory is a memory of motor commands, acquired through trial-and-error and reinforcement. Here we show that the brain controls how much it is willing to learn from the current error through a principled mechanism that depends on the history of past errors. This suggests that the brain stores a previously unknown form of memory, a memory of errors. A mathematical formulation of this idea provides insights into a host of puzzling experimental data, including savings and meta-learning, demonstrating that when we are better at a motor task, it is partly because the brain recognizes the errors it experienced before.

Similarities in error processing establish a link between saccade prediction at baseline and adaptation performance

Adaptive processes are crucial in maintaining the accuracy of body movements and rely on error storage and processing mechanisms. Although classically studied with adaptation paradigms, evidence of these ongoing error-correction mechanisms should also be detectable in other movements. Despite this connection, current adaptation models are challenged when forecasting adaptation ability with measures of baseline behavior. On the other hand, we have previously identified an error-correction process present in a particular form of baseline behavior, the generation of predictive saccades. This process exhibits long-term intertrial correlations that decay gradually (as a power law) and are best characterized with the tools of fractal time series analysis. Since this baseline task and adaptation both involve error storage and processing, we sought to find a link between the intertrial correlations of the error-correction process in predictive saccades and the ability of subjects to alter their saccade amplitudes during an adaptation task. Here we find just such a relationship: the stronger the intertrial correlations during prediction, the more rapid the acquisition of adaptation. This reinforces the links found previously between prediction and adaptation in motor control and suggests that current adaptation models are inadequate to capture the complete dynamics of these error-correction processes. A better understanding of the similarities in error processing between prediction and adaptation might provide the means to forecast adaptation ability with a baseline task. This would have many potential uses in physical therapy and the general design of paradigms of motor adaptation.

Laterality Differences in Cerebellar-Motor Cortex Connectivity

Lateralization of function is an important organizational feature of the motor system. Each effector is predominantly controlled by the contralateral cerebral cortex and the ipsilateral cerebellum. Transcranial magnetic stimulation studies have revealed hemispheric differences in the stimulation strength required to evoke a muscle response from the primary motor cortex (M1), with the dominant hemisphere typically requiring less stimulation than the nondominant. The current study assessed whether the strength of the connection between the cerebellum and M1 (CB-M1), known to change in association with motor learning, have hemispheric differences and whether these differences have any behavioral correlate. We observed, in right-handed individuals, that the connection between the right cerebellum and left M1 is typically stronger than the contralateral network. Behaviorally, we detected no lateralized learning processes, though we did find a significant effect on the amplitude of reaching movements across hands. Furthermore, we observed that the strength of the CB-M1 connection is correlated with the amplitude variability of reaching movements, a measure of movement precision, where stronger connectivity was associated with better precision. These findings indicate that lateralization in the motor system is present beyond the primary motor cortex, and points to an association between cerebellar M1 connectivity and movement execution.

Design and Analysis of Closed-Loop Decoder Adaptation Algorithms for Brain-Machine Interfaces

Closed-loop decoder adaptation (CLDA) is an emerging paradigm for achieving rapid performance improvements in online brain-machine interface (BMI) operation. Designing an effective CLDA algorithm requires making multiple important decisions, including choosing the timescale of adaptation, selecting which decoder parameters to adapt, crafting the corresponding update rules, and designing CLDA parameters. These design choices, combined with the specific settings of CLDA parameters, will directly affect the algorithm's ability to make decoder parameters converge to values that optimize performance. In this article, we present a general framework for the design and analysis of CLDA algorithms and support our results with experimental data of two monkeys performing a BMI task. First, we analyze and compare existing CLDA algorithms to highlight the importance of four critical design elements: the adaptation timescale, selective parameter adaptation, smooth decoder updates, and intuitive CLDA parameters. Second, we introduce mathematical convergence analysis using measures such as mean-squared error and KL divergence as a useful paradigm for evaluating the convergence properties of a prototype CLDA algorithm before experimental testing. By applying these measures to an existing CLDA algorithm, we demonstrate that our convergence analysis is an effective analytical tool that can ultimately inform and improve the design of CLDA algorithms.

Robotic assessment of upper limb motor function after stroke

Traditional assessment of a stroke subject's motor ability, carried out by a therapist who observes and rates the subject's motor behavior using ordinal measurements scales, is subjective, time consuming and lacks sensitivity. Rehabilitation robots, which have been the subject of intense inquiry over the last decade, are equipped with sensors that are used to develop objective measures of motor behaviors in a semiautomated way during therapy. This article reviews the current contributions of robot-assisted motor assessment of the upper limb. It summarizes the various measures related to movement performance, the models of motor recovery in stroke subjects and the relationship of robotic measures to standard clinical measures. It analyses the possibilities offered by current robotic assessment techniques and the aspects to address to make robotic assessment a mainstream motor assessment method.

Individuals with cerebellar degeneration show similar adaptation deficits with large and small visuomotor errors

The cerebellum has long been recognized to play an important role in motor adaptation. Individuals with cerebellar ataxia exhibit impaired learning in visuomotor adaptation tasks such as prism adaptation and force field learning. Both types of tasks involve the adjustment of an internal model to compensate for an external perturbation. This updating process is error driven, with the error signal based on the difference between anticipated and actual sensory information. This process may entail a credit assignment problem, with a distinction made between error arising from faulty representation of the environment and error arising from noise in the controller. We hypothesized that people with ataxia may perform poorly at visuomotor adaptation because they attribute a greater proportion of their error to their motor control difficulties. We tested this hypothesis using a computational model based on a Kalman filter. We imposed a 20-deg visuomotor rotation in either a single large step or in a series of smaller 5-deg steps. The ataxic group exhibited a comparable deficit in both conditions. The computational analyses indicate that the patients' deficit cannot be accounted for simply by their increased motor variability. Rather, the patients' deficit in learning may be related to difficulty in estimating the instability in the environment or variability in their motor system.

How does our motor system determine its learning rate?

Motor learning is driven by movement errors. The speed of learning can be quantified by the learning rate, which is the proportion of an error that is corrected for in the planning of the next movement. Previous studies have shown that the learning rate depends on the reliability of the error signal and on the uncertainty of the motor system's own state. These dependences are in agreement with the predictions of the Kalman filter, which is a state estimator that can be used to determine the optimal learning rate for each movement such that the expected movement error is minimized. Here we test whether not only the average behaviour is optimal, as the previous studies showed, but if the learning rate is chosen optimally in every individual movement. Subjects made repeated movements to visual targets with their unseen hand. They received visual feedback about their endpoint error immediately after each movement. The reliability of these error-signals was varied across three conditions. The results are inconsistent with the predictions of the Kalman filter because correction for large errors in the beginning of a series of movements to a fixed target was not as fast as predicted and the learning rates for the extent and the direction of the movements did not differ in the way predicted by the Kalman filter. Instead, a simpler model that uses the same learning rate for all movements with the same error-signal reliability can explain the data. We conclude that our brain does not apply state estimation to determine the optimal planning correction for every individual movement, but it employs a simpler strategy of using a fixed learning rate for all movements with the same level of error-signal reliability.

A long-memory model of motor learning in the saccadic system: a regime-switching approach

Maintenance of movement accuracy relies on motor learning, by which prior errors guide future behavior. One aspect of this learning process involves the accurate generation of predictions of movement outcome. These predictions can, for example, drive anticipatory movements during a predictive-saccade task. Predictive saccades are rapid eye movements made to anticipated future targets based on error information from prior movements. This predictive process exhibits long-memory (fractal) behavior, as suggested by inter-trial fluctuations. Here, we model this learning process using a regime-switching approach, which avoids the computational complexities associated with true long-memory processes. The resulting model demonstrates two fundamental characteristics. First, long-memory behavior can be mimicked by a system possessing no true long-term memory, producing model outputs consistent with human-subjects performance. In contrast, the popular two-state model, which is frequently used in motor learning, cannot replicate these findings. Second, our model suggests that apparent long-term memory arises from the trade-off between correcting for the most recent movement error and maintaining consistent long-term behavior. Thus, the model surprisingly predicts that stronger long-memory behavior correlates to faster learning during adaptation (in which systematic errors drive large behavioral changes); greater apparent long-term memory indicates more effective incorporation of error from the cumulative history across trials.

Sensitivity to prediction error in reach adaptation

It has been proposed that the brain predicts the sensory consequences of a movement and compares it to the actual sensory feedback. When the two differ, an error signal is formed, driving adaptation. How does an error in one trial alter performance in the subsequent trial? Here we show that the sensitivity to error is not constant but declines as a function of error magnitude. That is, one learns relatively less from large errors compared with small errors. We performed an experiment in which humans made reaching movements and randomly experienced an error in both their visual and proprioceptive feedback. Proprioceptive errors were created with force fields, and visual errors were formed by perturbing the cursor trajectory to create a visual error that was smaller, the same size, or larger than the proprioceptive error. We measured single-trial adaptation and calculated sensitivity to error, i.e., the ratio of the trial-to-trial change in motor commands to error size. We found that for both sensory modalities sensitivity decreased with increasing error size. A reanalysis of a number of previously published psychophysical results also exhibited this feature. Finally, we asked how the brain might encode sensitivity to error. We reanalyzed previously published probabilities of cerebellar complex spikes (CSs) and found that this probability declined with increasing error size. From this we posit that a CS may be representative of the sensitivity to error, and not error itself, a hypothesis that may explain conflicting reports about CSs and their relationship to error.