Task-relevant and task-irrelevant variability causally shape error-based motor learning

Sense of agency for a new motor skill emerges via the formation of a structural internal model

Sense of agency (SoA) refers to the feeling of controlling one's body and the external environment. The traditional comparator model posits that SoA arises from a match between predicted and actual action outcomes. However, when learning new motor skills, individuals initially lack outcome predictions and gradually develop an internal model of action-outcome mapping through trial-and-error, a process known as motor exploration. To investigate the development of SoA in such scenarios, we employed a de novo motor learning task that participants had never experienced before. Using a data glove, participants controlled a cursor on a screen through finger movements. In Experiment 1, participants learned a spatial hand-to-screen mapping from scratch via motor exploration. At different learning phases, we measured and compared participants' SoA for cursor movements that either conformed to the learned mapping or incorporated spatial or temporal biases. Initially, SoA was driven solely by temporal contiguity between finger and cursor movements. As learning progressed, SoA increased for cursor movements following the learned mapping compared to those following the spatially biased, unlearned mapping. In contrast, such changes did not occur in Experiment 2, where participants only imitated gesture images and memorized corresponding screen positions. The findings enhance existing SoA theories by elucidating the origins of the comparator process and highlighting the critical role of motor exploration.

Adaptation of functional gait parameters to a newly provided stiffness-optimized ankle-foot orthosis

BACKGROUND

Clinical decisions regarding ankle-foot-orthosis stiffness in people with calf muscle weakness are based on immediate evaluations, not taking gait adaptation into account. This study examined adaptation of step length, walking speed and energy cost of walking in the 3-months post-provision and whether individuals with higher gait variability adapt more compared to individuals with lower gait variability.

METHODS

We conducted a post-hoc analysis in eighteen stiffness-optimized ankle-foot-orthosis users with bilateral calf muscle weakness. Gait biomechanics, step length, walking speed and walking energy cost directly after provision (T1) and 3-months post-provision of the ankle-foot-orthosis (T2) were compared using paired sampled t-tests. Based on gait variability scores at T1, a high and low gait variability group was determined, and change scores in the functional gait parameters were compared using non-parametric independent sampled t-tests. A significance level of p ˂ 0.1 was used.

FINDINGS

No significant differences in step length, walking speed and energy cost of walking between T1 and T2 were found (p > 0.20). Step length increased more in people with high gait variability scores at T1 compared to those with low gait variability scores (High: +3.1 [-3.2 - +6.9], Low: +0.2 [-6.8 - +3.7] cm, p = 0.085), while no differences between groups were found for walking speed and energy cost of walking (p > 0.129).

INTERPRETATION

After provision of stiffness-optimized ankle-foot-orthoses in people with bilateral calf muscle weakness, no functional gait adaptations were found. However, people demonstrating high gait variability increased step length more compared to those demonstrating lower variability, which might be an indication that variability plays a role in adaptation.

Relationship between initial motor variability and learning and adaptive ability. A systematic review

Motor variability is an intrinsic feature of human beings that has been associated with the ability for learning and adaptation to specific tasks. The purpose of this review is to examine whether there is a possible direct relationship between individuals' initial variability in their ability for learning and adaptation in motor tasks. Eighteen articles examined the relationship between initial motor variability and the ability for learning or adaptation. Twelve found a direct relationship. In reward-based tasks, greater initial variability was associated with greater learning and adaption improvement when assessed using linear measures of dispersion, however, this association was not observed with temporal structure variability. While in error-based task associations were reported with both greater amount variability and more complexity temporal structure. Nevertheless, bias in initial performance related to the amount of variability was found, so the temporal structure of initial variability seems to be a better indicator of improvement in this type of task. Further research is needed for further research to better understand the potential relationship between initial motor variability and the ability for learning or adaptation in motor tasks.

Physio-avatar EB: aftereffects in error learning with EMG manipulation of first-person avatar experience

Introduction

Many studies have investigated the manipulation of a virtual upper arm using electromyogram (EMG); however, these studies primarily used a machine learning model or trigger control for this purpose. Furthermore, most of them could only display the constant motion of the virtual arm because the motion to be displayed was selected by pattern recognition or trigger control. In addition, these studies did not examine changes in the electromyographic signals after experiencing the virtual arm. By contrast, we propose a real-time, continuous, learning-free avatar that manipulates the virtual arm with electromyogram signals or physio-avatar EMG biofeedback (EB). The goal of the physio-avatar EB system is to induce physiological changes through experiential interactions.

Methods

We explored the possibility of changing motor control strategies by applying the system to healthy individuals as a case study. An intervention method that provided an experience of a body different from one's own was conducted on seven participants using a time-invariant calculation algorithm to determine the joint angles of the avatar. Control strategies for an indicator of the equilibrium point in the baseline and adaptation phases were determined to evaluate the physio-avatar EB intervention effect. The similarity of these BL and adaptation control strategies compared to those used during the washout period was assessed using the coefficient of determination. The accuracy and reliability of the virtual reality (VR) system were evaluated by comparison with existing studies and the required specs.

Results and Discussion

Changes in motor control strategies due to the physio-avatar EB system were observed in four experiments, where the participants gradually returned to their pre-intervention control strategies. This result can be attributed to the aftereffects caused by error learning. This implies that the developed system influenced their motor control strategies. The number of EMG acquisition bits was 16 bits, and the sampling rate was 1,000 Hz. The refresh rate of the head-mounted display was 90 Hz, and its resolution was 1440 × 1600 for a single eye. Additionally, the simulation frame rate was 30 FPS. These values were adequate compared to existing studies and required specs. The essential contribution of this study is the development of an avatar that is controlled by a different method than has been used in previous studies and the demonstration of changes in a subject's muscle activity after they experience an avatar. In the future, the clinical efficacy of the proposed system will be evaluated with actual patients.

Implicit sensorimotor learning in ballistic movement for transporting an object to a target

To enhance and sustain movement accuracy, humans make corrections in subsequent trials based on previous errors. Trial-by-trial learning occurs unconsciously and has mostly been studied using reaching movements. Goal-directed projection movements, such as archery, have an inherent delay between releasing an object and observing an outcome (e.g. the arrival position of the object), and this delay may prevent trial-by-trial implicit learning. We aimed to investigate the learning in the projection movement and the impacts of the inherent delay. During the experiment, a joystick was flicked once to transport a cursor from the starting location to a target. To manipulate the length of the delay between the cursor release and outcome observation, the speed of the cursor movement was varied: a fast speed can lead to a short delay. We found trial-by-trial implicit learning under all speed conditions, and the error sensitivity was not significantly different across speed conditions. Furthermore, the error sensitivity depended on the target location, that is, the movement direction. The results indicate that trial-by-trial implicit learning occurred in goal-directed projection movement, despite the length of the inherent delay. Additionally, the degree of this learning was affected by the movement direction.

Generalization in de novo learning of virtual upper limb movements is influenced by motor exploration

The acquisition of new motor skills from scratch, also known as de novo learning, is an essential aspect of motor development. In de novo learning, the ability to generalize skills acquired under one condition to others is crucial because of the inherently limited range of motor experiences available for learning. However, the presence of generalization in de novo learning and its influencing factors remain unclear. This study aimed to elucidate the generalization of de novo motor learning by examining the motor exploration process, which is the accumulation of motor experiences. To this end, we manipulated the exploration process during practice by changing the target shape using either a small circular target or a bar-shaped target. Our findings demonstrated that the amount of learning during practice was generalized across different conditions. Furthermore, the extent of generalization is influenced by movement variability in the control space, which is irrelevant to the task, rather than the target shapes themselves. These results confirmed the occurrence of generalization in de novo learning and suggest that the exploration process within the control space plays a significant role in facilitating this generalization.

Behavioral Motor Performance

The human sensorimotor control system has exceptional abilities to perform skillful actions. We easily switch between strenuous tasks that involve brute force, such as lifting a heavy sewing machine, and delicate movements such as threading a needle in the same machine. Using a structure with different control architectures, the motor system is capable of updating its ability to perform through our daily interaction with the fluctuating environment. However, there are issues that make this a difficult computational problem for the brain to solve. The brain needs to control a nonlinear, nonstationary neuromuscular system, with redundant and occasionally undesired degrees of freedom, in an uncertain environment using a body in which information transmission is subject to delays and noise. To gain insight into the mechanisms of motor control, here we survey movement laws and invariances that shape our everyday motion. We then examine the major solutions to each of these problems in the three parts of the sensorimotor control system, sensing, planning, and acting. We focus on how the sensory system, the control architectures, and the structure and operation of the muscles serve as complementary mechanisms to overcome deviations and disturbances to motor behavior and give rise to skillful motor performance. We conclude with possible future research directions based on suggested links between the operation of the sensorimotor system across the movement stages. © 2024 American Physiological Society. Compr Physiol 14:5179-5224, 2024.

Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance

Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.

Failure induces task-irrelevant exploration during a stencil task

During reward-based motor tasks, performance failure leads to an increase in movement variability along task-relevant dimensions. These increases in movement variability are indicative of exploratory behaviour in search of a better, more successful motor action. It is unclear whether failure also induces exploration along task-irrelevant dimensions that do not influence performance. In this study, we ask whether participants would explore the task-irrelevant dimension while they performed a stencil task. With a stylus, participants applied downward, normal force that influenced whether they received reward (task-relevant) as they simultaneously made erasing-like movement patterns along the tablet that did not influence performance (task-irrelevant). In this task, the movement pattern was analyzed as the distribution of movement directions within a movement. The results showed significant exploration of task-relevant force and task-irrelevant movement patterns. We conclude that failure can induce additional movement variability along a task-irrelevant dimension.

Bayesian hierarchical models and prior elicitation for fitting psychometric functions

Our previous articles demonstrated how to analyze psychophysical data from a group of participants using generalized linear mixed models (GLMM) and two-level methods. The aim of this article is to revisit hierarchical models in a Bayesian framework. Bayesian models have been previously discussed for the analysis of psychometric functions although this approach is still seldom applied. The main advantage of using Bayesian models is that if the prior is informative, the uncertainty of the parameters is reduced through the combination of prior knowledge and the experimental data. Here, we evaluate uncertainties between and within participants through posterior distributions. To demonstrate the Bayesian approach, we re-analyzed data from two of our previous studies on the tactile discrimination of speed. We considered different methods to include a priori knowledge in the prior distribution, not only from the literature but also from previous experiments. A special type of Bayesian model, the power prior distribution, allowed us to modulate the weight of the prior, constructed from a first set of data, and use it to fit a second one. Bayesian models estimated the probability distributions of the parameters of interest that convey information about the effects of the experimental variables, their uncertainty, and the reliability of individual participants. We implemented these models using the software Just Another Gibbs Sampler (JAGS) that we interfaced with R with the package rjags. The Bayesian hierarchical model will provide a promising and powerful method for the analysis of psychometric functions in psychophysical experiments.

The online and offline effects of changing movement timing variability during training on a finger-opposition task

In motor learning tasks, there is mixed evidence for whether increased task-relevant variability in early learning stages leads to improved outcomes. One problem is that there may be a connection between skill level and motor variability, such that participants who initially have more variability may also perform worse on the task, so will have more room to improve. To avoid this confound, we experimentally manipulated the amount of movement timing variability (MTV) during training to test whether it improves performance. Based on previous studies showing that most of the improvement in finger-opposition tasks comes from optimizing the relative onset time of the finger movements, we used auditory cues (beeps) to guide the onset times of sequential movements during a training session, and then assessed motor performance after the intervention. Participants were assigned to three groups that either: (a) followed a prescribed random rhythm for their finger touches (Variable MTV), (b) followed a fixed rhythm (Fixed control MTV), or (c) produced the entire sequence following a single beep (Unsupervised control MTV). While the intervention was successful in increasing MTV during training for the Variable group, it did not lead to improved outcomes post-training compared to either control group, and the use of fixed timing led to significantly worse performance compared to the Unsupervised control group. These results suggest that manipulating MTV through auditory cues does not produce greater learning than unconstrained training in motor sequence tasks.