Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings

Reinforcement feedback impairs locomotor adaptation and retention

Introduction

Locomotor adaptation is a motor learning process used to alter spatiotemporal elements of walking that are driven by prediction errors, a discrepancy between the expected and actual outcomes of our actions. Sensory and reward prediction errors are two different types of prediction errors that can facilitate locomotor adaptation. Reward and punishment feedback generate reward prediction errors but have demonstrated mixed effects on upper extremity motor learning, with punishment enhancing adaptation, and reward supporting motor memory. However, an in-depth behavioral analysis of these distinct forms of feedback is sparse in locomotor tasks.

Methods

For this study, three groups of healthy young adults were divided into distinct feedback groups [Supervised, Reward, Punishment] and performed a novel locomotor adaptation task where each participant adapted their knee flexion to 30 degrees greater than baseline, guided by visual supervised or reinforcement feedback (Adaptation). Participants were then asked to recall the new walking pattern without feedback (Retention) and after a washout period with feedback restored (Savings).

Results

We found that all groups learned the adaptation task with external feedback. However, contrary to our initial hypothesis, enhancing sensory feedback with a visual representation of the knee angle (Supervised) accelerated the rate of learning and short-term retention in comparison to monetary reinforcement feedback. Reward and Punishment displayed similar rates of adaptation, short-term retention, and savings, suggesting both types of reinforcement feedback work similarly in locomotor adaptation. Moreover, all feedback enhanced the aftereffect of locomotor task indicating changes to implicit learning.

Discussion

These results demonstrate the multi-faceted nature of reinforcement feedback on locomotor adaptation and demonstrate the possible different neural substrates that underly reward and sensory prediction errors during different motor tasks.

Dopamine Increases Accuracy and Lengthens Deliberation Time in Explicit Motor Skill Learning

Although animal research implicates a central role for dopamine in motor skill learning, a direct causal link has yet to be established in neurotypical humans. Here, we tested if a pharmacological manipulation of dopamine alters motor learning, using a paradigm which engaged explicit, goal-directed strategies. Participants (27 females; 11 males; aged 18-29 years) first consumed either 100 mg of levodopa (n = 19), a dopamine precursor that increases dopamine availability, or placebo (n = 19). Then, during training, participants learnt the explicit strategy of aiming away from presented targets by instructed angles of varying sizes. Targets jumped mid-movement by the instructed aiming angle. Task success was thus contingent upon aiming accuracy and not speed. The effect of the dopamine manipulations on skill learning was assessed during training and after an overnight follow-up. Increasing dopamine availability at training improved aiming accuracy and lengthened reaction times, particularly for larger, more difficult aiming angles, both at training and, importantly, at follow-up, despite prominent session-by-session performance improvements in both accuracy and speed. Exogenous dopamine thus seems to result in a learnt, persistent propensity to better adhere to task goals. Results support the proposal that dopamine is important in engagement of instrumental motivation to optimize adherence to task goals, particularly when learning to execute goal-directed strategies in motor skill learning.

Transcranial direct current stimulation to facilitate neurofunctional rehabilitation in children with autism spectrum disorder: a protocol for a randomized, sham-controlled, double-blind clinical trial

Background

Anodal transcranial direct current stimulation (tDCS) over the primary motor cortex and cerebellum is gaining prominence in the literature due to its potential to favor learning and motor performance. If administered during motor training, tDCS is capable of increasing the effect of training. Considering the motor impairment presented by children with Autism Spectrum Disorders (ASD), atDCS applied during motor training may contribute to the rehabilitation of these children. However, it is necessary to examine and compare the effects of atDCS over the motor cortex and the cerebellum on the motor skills of children with ASD. This information may benefit future clinical indications of tDCS for rehabilitation of children with ASD. The aim of the proposed study is to determine whether anodal tDCS over the primary motor cortex and cerebellum can enhance the effects of gait training and postural control on motor skills, mobility, functional balance, cortical excitability, cognitive aspects and behavioral aspects in children with ASD. Our hypothesis is the active tDCS combined with motor training will enhance the performance of the participants in comparison to sham tDCS.

Methods and design

A randomized, sham-controlled, double-blind clinical trial will be conducted involving 30 children with ASD that will be recruited to receive ten sessions of sham or ten sessions of active anodal tDCS (1 mA, 20 min) over the primary motor cortex or cerebellun combined with motor training. The participants will be assessed before as well as one, four and eight weeks after the interventions. The primary outcome will be gross and fine motor skills. The secondary outcomes will be mobility, functional balance, motor cortical excitability, cognitive aspects and behavioral aspects.

Discussion

Although abnormalities in gait and balance are not primary characteristics of ASD, such abnormalities compromise independence and global functioning during the execution of routine activities of childhood. If demonstrated that anodal tDCS administered over areas of the brain involved in motor control, such as the primary motor cortex and cerebellum, can enhance the effects of gait and balance training in only ten sessions in two consecutive weeks, the clinical applicability of this stimulation modality will be expanded as well as more scientifically founded.Clinical trial registration February 16, 2023 (https://ensaiosclinicos.gov.br/rg/RBR-3bskhwf).

Modulating motor learning with brain stimulation: Stage-specific perspectives for transcranial and transcutaneous delivery

Brain stimulation has been used in motor learning studies with success in improving aspects of task learning, retention, and consolidation. Using a variety of motor tasks and stimulus parameters, researchers have produced an array of literature supporting the efficacy of brain stimulation to modulate motor task learning. We discuss the use of transcranial direct current stimulation, transcranial alternating current stimulation, and peripheral nerve stimulation to modulate motor learning. In a novel approach, we review literature of motor learning modulation in terms of learning stage, categorizing learning into acquisition, consolidation, and retention. We endeavour to provide a current perspective on the stage-specific mechanism behind modulation of motor task learning, to give insight into how electrical stimulation improves or hinders motor learning, and how mechanisms differ depending on learning stage. Offering a look into the effectiveness of peripheral nerve stimulation for motor learning, we include potential mechanisms and overlapping features with transcranial stimulation. We conclude by exploring how peripheral stimulation may contribute to the results of studies that employed brain stimulation intracranially.

Timing is everything: Event-related transcranial direct current stimulation improves motor adaptation

BACKGROUND

There is a current discord between the foundational theories underpinning motor learning and how we currently apply transcranial direct current stimulation (TDCS): the former is dependent on tight coupling of events while the latter is conducted with very low temporal resolution.

OBJECTIVE

Here we aimed to investigate the temporal specificity of stimulation by applying TDCS in short epochs, and coincidentally with movement, during a motor adaptation task.

METHODS

Participants simultaneously adapted a reaching movement to two opposing velocity-dependent force-fields (clockwise and counter-clockwise), distinguished by a contextual leftward or rightward shift in the task display and cursor location respectively. Brief bouts (<3 s) of event-related TDCS (er-TDCS) were applied over M1 or the cerebellum during movements for only one of these learning contexts.

RESULTS

We show that when short duration stimulation is applied to the cerebellum and yoked to movement, only those reaching movements performed simultaneously with stimulation are selectively enhanced, whilst similar and interleaved movements are left unaffected. We found no evidence of improved adaptation following M1 er-TDCS, as participants displayed equivalent levels of error during both stimulated and unstimulated movements. Similarly, participants in the sham stimulation group adapted comparably during left and right-shift trials.

CONCLUSIONS

It is proposed that the coupling of cerebellar stimulation and movement influences timing-dependent (i.e. Hebbian-like) mechanisms of plasticity to facilitate enhanced learning in the stimulated context.

Anodal Transcranial Direct Current Stimulation Could Modulate Cortical Excitability and the Central Cholinergic System in Akinetic Rigid-Type Parkinson's Disease: Pilot Study

Background

Transcranial direct current stimulation (tDCS) is a non-invasive technique that has been widely studied as an alternative treatment for Parkinson's disease (PD). However, its clinical benefit remains unclear. In this study, we aimed to investigate the effect of tDCS on the central cholinergic system and cortical excitability in mainly akinetic rigid-type patients with PD.

Methods

In total, 18 patients with PD were prospectively enrolled and underwent 5 sessions of anodal tDCS on the M1 area, which is on the contralateral side of the dominant hand. We excluded patients with PD who had evident resting tremor of the hand to reduce the artifact of electrophysiologic findings. We compared clinical scales reflecting motor, cognitive, and mood symptoms between pre- and post-tDCS. Additionally, we investigated the changes in electrophysiologic parameters, such as short latency afferent inhibition (SAI) (%), which reflects the central cholinergic system.

Results

The United Parkinson's Disease Rating Scale Part 3 (UPDRS-III), the Korean-Montreal Cognitive Assessment (MoCA-K), and Beck Depression Inventory (BDI) scores were significantly improved after anodal tDCS (p < 0.01, p < 0.01, and p < 0.01). Moreover, motor evoked potential amplitude ratio (MEPAR) (%) and integrated SAI showed significant improvement after tDCS (p < 0.01 and p < 0.01). The mean values of the change in integrated SAI (%) were significantly correlated with the changes in UPDRS-III scores; however, the MoCA-K and BDI scores did not show differences.

Conclusions

Anodal tDCS could influence the central cholinergic system, such as frontal cortical excitability and depression in PD. This mechanism could underlie the clinical benefit of tDCS in patients with PD.

Reward and plasticity: Implications for neurorehabilitation

Neuroplasticity follows nervous system injury in the presence or absence of rehabilitative treatments. Rehabilitative interventions can be used to modulate adaptive neuroplasticity, reducing motor impairment and improving activities of daily living in patients with brain lesions. Learning principles guide some rehabilitative interventions. While basic science research has shown that reward combined with training enhances learning, this principle has been only recently explored in the context of neurorehabilitation. Commonly used reinforcers may be more or less rewarding depending on the individual or the context in which the task is performed. Studies in healthy humans showed that both reward and punishment can enhance within-session motor performance; but reward, and not punishment, improves consolidation and retention of motor skills. On the other hand, neurorehabilitative training after brain lesions involves complex tasks (e.g., walking and activities of daily living). The contribution of reward to neurorehabilitation is incompletely understood. Here, we discuss recent research on the role of reward in neurorehabilitation and the needed directions of future research.

Engagement of a parieto-cerebellar network in prism adaptation. A double-blind high-definition transcranial direct current stimulation study on healthy individuals

Prism Adaptation (PA) is a non-invasive method to investigate visuomotor control. Recent neurostimulation studies have proposed an interpretation of the mechanisms underlying PA based on functioning of brain networks, instead of focusing on single brain areas. To test the functioning of the network during a classical PA procedure, here we used for the first time High-Definition transcranial Direct Current Stimulation (HD-tDCS) to simultaneously inhibit or facilitate brain activity in two main nodes of the network, namely the parietal cortex and the cerebellum, in healthy individuals. The main results showed that simultaneous anodal HD-tDCS over the two regions reduced terminal errors during exposure to prism glasses as compared to cathodal and sham stimulation. Conversely, cathodal HD-tDCS reduced after-effect as compared to anodal and sham stimulation following prism removal. Overall, these results provide new insights on the network related to the deployment of PA mechanisms and demonstrate the feasibility of using non-invasive HD-tDCS to modulate the adaptive mechanisms of PA.

Modulation of neural activity in frontopolar cortex drives reward-based motor learning

The frontopolar cortex (FPC) contributes to tracking the reward of alternative choices during decision making, as well as their reliability. Whether this FPC function extends to reward gradients associated with continuous movements during motor learning remains unknown. We used anodal transcranial direct current stimulation (tDCS) over the right FPC to investigate its role in reward-based motor learning. Nineteen healthy human participants practiced novel sequences of finger movements on a digital piano with corresponding auditory feedback. Their aim was to use trialwise reward feedback to discover a hidden performance goal along a continuous dimension: timing. We additionally modulated the contralateral motor cortex (left M1) activity, and included a control sham stimulation. Right FPC-tDCS led to faster learning compared to lM1-tDCS and sham through regulation of motor variability. Bayesian computational modelling revealed that in all stimulation protocols, an increase in the trialwise expectation of reward was followed by greater exploitation, as shown previously. Yet, this association was weaker in lM1-tDCS suggesting a less efficient learning strategy. The effects of frontopolar stimulation were dissociated from those induced by lM1-tDCS and sham, as motor exploration was more sensitive to inferred changes in the reward tendency (volatility). The findings suggest that rFPC-tDCS increases the sensitivity of motor exploration to updates in reward volatility, accelerating reward-based motor learning.

The cost of correcting for error during sensorimotor adaptation

Learning from error is often a slow process. In machine learning, the learning rate depends on a loss function that specifies a cost for error. Here, we hypothesized that during motor learning, error carries an implicit cost for the brain because the act of correcting for error consumes time and energy. Thus, if this implicit cost could be increased, it may robustly alter how the brain learns from error. To vary the implicit cost of error, we designed a task that combined saccade adaptation with motion discrimination: movement errors resulted in corrective saccades, but those corrections took time away from acquiring information in the discrimination task. We then modulated error cost using coherence of the discrimination task and found that when error cost was large, pupil diameter increased and the brain learned more from error. However, when error cost was small, the pupil constricted and the brain learned less from the same error. Thus, during sensorimotor adaptation, the act of correcting for error carries an implicit cost for the brain. Modulating this cost affects how much the brain learns from error.

Cortical preparatory activity during motor learning reflects visuomotor retention deficits after punishment feedback

Previous studies have shown that reinforcement-based motor learning requires the brain to process feedback-related information after movement execution. However, whether reinforcement feedback changes how the brain processes motor preparation before movement execution is unclear. By using electroencephalography (EEG), this study investigates whether reinforcement feedback changes cortical preparatory activity to modulate motor learning and memory. Human subjects were divided in three groups [reward, punishment, control] to perform a visuomotor rotation task under different conditions that assess adaptation (learning) and retention (memory) during the task. Reinforcement feedback was provided in the form of points after each trial that signaled monetary gains (reward) or losses (punishment). EEG was utilized to evaluate the amplitude of movement readiness potentials (MRPs) at the beginning of each trial for each group during the adaptation and retention conditions of the task. The results show that punishment feedback significantly decreased MRPs amplitude during both task conditions compared to Reward and Control groups. Moreover, the punishment-related decrease in MRPs amplitude paralleled decreases in motor performance during the retention but not the adaptation condition. No changes in MRPs or motor performance were observed in the Reward group. These results support the idea that reinforcement feedback modulates motor preparation and suggest that changes in cortical preparatory activity contribute to the visuomotor retention deficits observed after punishment feedback.

Acoustic stimulation increases implicit adaptation in sensorimotor adaptation

Sensorimotor adaptation is an important part of our ability to perform novel motor tasks (i.e., learning of motor skills). Efforts to improve adaptation in healthy and clinical patients using non-invasive brain stimulation methods have been hindered by inter-individual and intra-individual variability in brain susceptibility to stimulation. Here, we explore unpredictable loud acoustic stimulation as an alternative method of modulating brain excitability to improve sensorimotor adaptation. In two experiments, participants moved a cursor towards targets, and adapted to a 30º rotation of cursor feedback, either with or without unpredictable acoustic stimulation. Acoustic stimulation improved initial adaptation to sensory prediction errors in Study 1, and improved overnight retention of adaptation in Study 2. Unpredictable loud acoustic stimulation might thus be a potent method of modulating sensorimotor adaptation in healthy adults.

Feasibility of combining functional near-infrared spectroscopy with electroencephalography to identify chronic stroke responders to cerebellar transcranial direct current stimulation-a computational modeling and portable neuroimaging methodological study

Feasibility of portable neuroimaging of cerebellar transcranial direct current stimulation (ctDCS) effects on the cerebral cortex has not been investigated vis-à-vis cerebellar lobular electric field strength. We studied functional near-infrared spectroscopy (fNIRS) in conjunction with electroencephalography (EEG) to measure changes in the brain activation at the prefrontal cortex (PFC) and the sensorimotor cortex (SMC) following ctDCS as well as virtual reality-based balance training (VBaT) before and after ctDCS treatment in 12 hemiparetic chronic stroke survivors. We performed general linear modeling (GLM) that putatively associated the lobular electric field strength with the changes in the fNIRS-EEG measures at the ipsilesional and contra-lesional PFC and SMC. Here, fNIRS-EEG measures were found in the latent space from canonical correlation analysis (CCA) between the changes in total hemoglobin (tHb) concentrations (0.01-0.07Hz and 0.07-0.13Hz bands) and log10-transformed EEG bandpower within 1-45 Hz where significant (Wilks' lambda>0.95) canonical correlations were found only for the 0.07-0.13-Hz band. Also, the first principal component (97.5% variance accounted for) of the mean lobular electric field strength was a good predictor of the latent variables of oxy-hemoglobin (O2Hb) concentrations and log10-transformed EEG bandpower. GLM also provided insights into non-responders to ctDCS who also performed poorly in the VBaT due to ideomotor apraxia. Future studies should investigate fNIRS-EEG joint-imaging in a larger cohort to identify non-responders based on GLM fitting to the fNIRS-EEG data.

Effect of Combined Therapy of Virtual Reality and Transcranial Direct Current Stimulation in Children and Adolescents With Cerebral Palsy: A Study Protocol for a Triple-Blinded Randomized Controlled Crossover Trial

Background: Transcranial direct current stimulation (tDCS) and therapy-based virtual reality (VR) have been investigated separately. They have shown promise as efficient and engaging new tools in the neurological rehabilitation of individuals with cerebral palsy (CP). However, the recent literature encourages investigation of the combination of therapy tools in order to potentiate clinic effects and its mechanisms.

Methods: A triple-blinded randomised sham-controlled crossover trial will be performed. Thirty-six individuals with gross motor function of levels I to IV (aged 4–14 years old) will be recruited. Individuals will be randomly assigned to Group A (active first) or S (sham first): Group A will start with ten sessions of active tDSC combined with VR tasks. After a 1-month washout, this group will be reallocated to another ten sessions with sham tDCS combined with VR tasks. In contrast, Group S will carry out the opposite protocol, starting with sham tDCS. For the active tDCS the protocol will use low frequency tDCS [intensity of 1 milliampere (mA)] over the primary cortex (M1) area on the dominant side of the brain. Clinical evaluations (reaction times and coincident timing through VR, functional scales: Abilhand-Kids, ACTIVLIM-CP, Paediatric Evaluation of Disability Inventory-PEDI- and heart rate variability-HRV) will be performed at baseline, during, and after active and sham tDCS.

Conclusion: tDCS has produced positive results in treating individuals with CP; thus, its combination with new technologies shows promise as a potential mechanism for improving neurological functioning. The results of this study may provide new insights into motor rehabilitation, thereby contributing to the better use of combined tDCS and VR in people with CP.

Trial Registration:ClinicalTrials.gov, NCT04044677. Registered on 05 August 2019.

Punishment Feedback Impairs Memory and Changes Cortical Feedback-Related Potentials During Motor Learning

Reward and punishment have demonstrated dissociable effects on motor learning and memory, which suggests that these reinforcers are differently processed by the brain. To test this possibility, we use electroencephalography to record cortical neural activity after the presentation of reward and punishment feedback during a visuomotor rotation task. Participants were randomly placed into Reward, Punishment, or Control groups and performed the task under different conditions to assess the adaptation (learning) and retention (memory) of the motor task. These conditions featured an incongruent position between the cursor and the target, with the cursor trajectory, rotated 30° counterclockwise, requiring the participant to adapt their movement to hit the target. Feedback based on error magnitude was provided during the Adaptation condition in the form of a positive number (Reward) or negative number (Punishment), each representing a monetary gain or loss, respectively. No reinforcement or visual feedback was provided during the No Vision condition (retention). Performance error and event-related potentials (ERPs) time-locked to feedback presentation were calculated for each participant during both conditions. Punishment feedback reduced performance error and promoted faster learning during the Adaptation condition. In contrast, punishment feedback increased performance error during the No Vision condition compared to Control and Reward groups, which suggests a diminished motor memory. Moreover, the Punishment group showed a significant decrease in the amplitude of ERPs during the No Vision condition compared to the Adaptation condition. The amplitude of ERPs did not change in the other two groups. These results suggest that punishment feedback impairs motor retention by altering the neural processing involved in memory encoding. This study provides a neurophysiological underpinning for the dissociative effects of punishment feedback on motor learning.

Multiple Motor Learning Processes in Humans: Defining Their Neurophysiological Bases

Learning new motor behaviors or adjusting previously learned actions to account for dynamic changes in our environment requires the operation of multiple distinct motor learning processes, which rely on different neuronal substrates. For instance, humans are capable of acquiring new motor patterns via the formation of internal model representations of the movement dynamics and through positive reinforcement. In this review, we will discuss how changes in human physiological markers, assessed with noninvasive brain stimulation techniques from distinct brain regions, can be utilized to provide insights toward the distinct learning processes underlying motor learning. We will summarize the findings from several behavioral and neurophysiological studies that have made efforts to understand how distinct processes contribute to and interact when learning new motor behaviors. In particular, we will extensively review two types of behavioral processes described in human sensorimotor learning: (1) a recalibration process of a previously learned movement and (2) acquiring an entirely new motor control policy, such as learning to play an instrument. The selected studies will demonstrate in-detail how distinct physiological mechanisms contributions change depending on the time course of learning and the type of behaviors being learned.