Human Midbrain Sensitivity to Cognitive Feedback and Uncertainty During Classification Learning

Transcranial electrical stimulation for procedural learning and rehabilitation

Procedural learning is the acquisition of motor and non-motor skills through a gradual process that increases with practice. Impairments in procedural learning have been consistently demonstrated in neurodevelopmental, neurodegenerative, and neuropsychiatric disorders. Considering that noninvasive brain stimulation modulates brain activity and boosts neuroplastic mechanisms, we reviewed the effects of coupling transcranial direct current stimulation (tDCS) with training methods for motor and non-motor procedural learning to explore tDCS potential use as a tool for enhancing implicit learning in healthy and clinical populations. The review covers tDCS effects over i. motor procedural learning, from basic to complex activities; ii. non-motor procedural learning; iii. procedural rehabilitation in several clinical populations. We conclude that targeting the primary motor cortex and prefrontal areas seems the most promising for motor and non-motor procedural learning, respectively. For procedural rehabilitation, the use of tDCS is yet at an early stage but some effectiveness has been reported for implicit motor and memory learning. Still, systematic comparisons of stimulation parameters and target areas are recommended for maximising the effectiveness of tDCS and its robustness for procedural rehabilitation.

The effect of emotional faces on reward-related probability learning in depressed patients

BACKGROUND

Existing research indicates that individuals with Major Depressive Disorder (MDD) exhibit a bias toward salient negative stimuli. However, the impact of such biased stimuli on concurrent cognitive and affective processes in individuals with depression remains inadequately understood. This study aimed to investigate the effects of salient environmental stimuli, specifically emotional faces, on reward-associated processes in MDD.

METHODS

Thirty-three patients with recurrent MDD and thirty-two healthy controls (HC) matched for age, sex, and education were included in the study. We used a reward-related associative learning (RRAL) task primed with emotional (happy, sad, neutral) faces to investigate the effect of salient stimuli on reward-related learning and decision-making in functional magnetic resonance imaging (fMRI). Participants were instructed to ignore emotional faces during the task. The fMRI data were analyzed using a full-factorial general linear model (GLM) in Statistical Parametric Mapping (SPM12).

RESULTS

In depressed patients, cues primed with sad faces were associated with reduced amygdala activation. However, both HC and MDD group exhibited reduced ventral striatal activity while learning reward-related cues and receiving rewards.

LIMITATIONS

The patients'medication usage was not standardized.

CONCLUSIONS

This study underscores the functional alteration of the amygdala in response to cognitive tasks presented with negative emotionally salient stimuli in the environment of MDD patients. The observed alterations in amygdala activity suggest potential interconnected effects with other regions of the prefrontal cortex. Understanding the intricate neural connections and their disruptions in depression is crucial for unraveling the complex pathophysiology of the disorder.

Corticostriatal activity related to performance during continuous de novo motor learning

Corticostriatal regions play a pivotal role in visuomotor learning. However, less research has been done on how fMRI activity in their subregions is related to task performance, which is provided as visual feedback during motor learning. To address this, we conducted an fMRI experiment in which participants acquired a complex de novo motor skill using continuous or binary visual feedback related to performance. We found a highly selective response related to performance in the entire striatum in both conditions and a relatively higher response in the caudate nucleus for the binary feedback condition. However, the ventromedial prefrontal cortex (vmPFC) response was significant only for the continuous feedback condition. Furthermore, we also found functional distinction of the striatal subregions in random versus goal-directed motor control. These findings underscore the substantial effects of the visual feedback indicating performance on distinct corticostriatal responses, thereby elucidating its significance in reinforcement-based motor learning.

Bayesian Surprise Predicts Human Event Segmentation in Story Listening

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.

Diffusion tensor imaging techniques show that parkin gene S/N167 polymorphism is responsible for extensive brain white matter damage in patients with Parkinson's disease

Objective

To explore the influence of disease and genetic factors on the white matter microstructure in patients with PD. The white matter microstructural changes in the substantia nigra-striatum system were detected by diffusion tensor imaging (DTI) using the region of interest (ROI) and diffusion tensor tracer (DTT) methods.

Methods

Patients with primary Parkinson's disease (PD) without a family history of PD were selected and divided into PD-G/G and PD-G/A groups according to their parkin S/N167 polymorphism. Control groups matched for age, sex, and gene type (G/G and G/A) were also included. Three-dimensional brain volume imaging (3D-BRAVO) and DTI were performed. The microstructural changes in the substantia nigra-striatum system were evaluated by the ROI and DTT methods. The Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), Hoehn-Yahr (H-Y) staging, and the third part of the Unified Parkinson's Disease Rating (UPDRS-III) scales evaluated the cognitive and motor function impairment in patients with PD. Independent samples t-test compared normally-distributed data, and the Wilcoxon rank sum test compared measurement or categorical non-normally distributed data. Multiple regression analysis was used to analyze the correlation between various DTI indicators and the MMSE, MoCA, UPDRS-III, and H-Y scores in the PD-G/G and PD-G/A groups. P < 0.05 was considered statistically significant.

Results

The white matter microstructural changes in the nigrostriatal pathway differed significantly between the PD or PD-G/A and the control group (P < 0.05)The ROI method showed that the left globus pallidus radial diffusivity (RD) value was negatively correlated with the MMSE score (r = -0.404, P = 0.040), and the left substantia nigra (LSN) fractional anisotropy (FA) value was positively correlated with the MoCA score (r = 0.405, P = 0.040) and negatively with the H-Y stage (r = -0.479, P = 0.013).The DTT method showed that the MMSE score was positively correlated with the right substantia nigra (RSN) FA value (r = 0.592, P = 0.001) and negatively with its RD value (r = -0.439, P = 0.025). The H-Y grade was negatively correlated with the number of fibers in the RSN (r = -0.406, P = 0.040). The UPDRS-Ⅲ score was positively correlated with the mean diffusivity (r = 0.420, P = 0.033) and RD (r = 0.396, P = 0.045) values of the LSN, and the AD value of the RSN (r = 0.439, P = 0.025).

Conclusion

The DTI technique detected extensive white matter fiber damage in patients with PD, primarily in those with the G/A genotype, that led to motor and cognitivesymptoms.

Anticipating social feedback involves basal forebrain and mesolimbic functional connectivity

The mesolimbic system and basal forebrain (BF) are implicated in processing rewards and punishment, but their interplay and functional properties of subregions with respect to future social outcomes remain unclear. Therefore, this study investigated regional responses and interregional functional connectivity of the lateral (l), medial (m), and ventral (v) Substantia Nigra (SN), Nucleus Accumbens (NAcc), Nucleus basalis of Meynert (NBM), and Medial Septum/Diagonal Band (MS/DB) during reward and punishment anticipation in a social incentive delay task with neutral, positive, and negative feedback using high-resolution fMRI (1.5mm³). Neuroimaging data (n=36 healthy humans) of the anticipation phase was analyzed using mass-univariate, functional connectivity, and multivariate-pattern analysis. As expected, participants responded faster when anticipating positive and negative compared to neutral social feedback. At the neural level, anticipating social information engaged valence-related and valence-unrelated functional connectivity patterns involving the BF and mesolimbic areas. Precisely, valence-related connectivity between the lSN and NBM was associated with anticipating neutral social feedback, while connectivity between the vSN and NBM was associated with anticipating positive social feedback. A more complex pattern was observed for anticipating negative social feedback, including connectivity between the lSN and MS/DB, lSN and NAcc, as well as mSN and NAcc. To conclude, behavioral responses are modulated by the possibility to obtain positive and avoid negative social feedback. The neural processing of feedback anticipation relies on functional connectivity patterns between the BF and mesolimbic areas associated with the emotional valence of the social information. As such, our findings give novel insights into the underlying neural processes of social information processing.

Altered functional connectivity during performance feedback processing in multiple sclerosis

Effective learning from performance feedback is vital for adaptive behavior regulation necessary for successful cognitive performance. Yet, how this learning operates in clinical groups that experience cognitive dysfunction is not well understood. Multiple sclerosis (MS) is an autoimmune, degenerative disease of the central nervous system characterized by physical and cognitive dysfunction. A highly prevalent impairment in MS is cognitive fatigue (CF). CF is associated with altered functioning within cortico-striatal regions that also facilitate feedback-based learning in neurotypical (NT) individuals. Despite this cortico-striatal overlap, research about feedback-based learning in MS, its associated neural underpinnings, and its sensitivity to CF, are all lacking. The present study investigated feedback-based learning ability in MS, as well as associated cortico-striatal function and connectivity. MS and NT participants completed a functional magnetic resonance imaging (fMRI) paired-word association task during which they received trial-by-trial monetary, non-monetary, and uninformative performance feedback. Despite reporting greater CF throughout the task, MS participants displayed comparable task performance to NTs, suggesting preserved feedback-based learning ability in the MS group. Both groups recruited the ventral striatum (VS), caudate nucleus, and ventromedial prefrontal cortex in response to the receipt of performance feedback, suggesting that people with MS also recruit cortico-striatal regions during feedback-based learning. However, compared to NT participants, MS participants also displayed stronger functional connectivity between the VS and task-relevant regions, including the left angular gyrus and right superior temporal gyrus, in response to feedback receipt. Results indicate that CF may not interfere with feedback-based learning in MS. Nonetheless, people with MS may recruit alternative connections with the striatum to assist with this form of learning. These findings have implications for cognitive rehabilitation treatments that incorporate performance feedback to remediate cognitive dysfunction in clinical populations.

Dorsal posterior cingulate cortex responds to negative feedback information supporting learning and relearning of response policies

Many challenges in life come without explicit instructions. Instead, humans need to test, select, and adapt their behavioral responses based on feedback from the environment. While reward-centric accounts of feedback processing primarily stress the reinforcing aspect of positive feedback, feedback’s central function from an information-processing perspective is to offer an opportunity to correct errors, thus putting a greater emphasis on the informational content of negative feedback. Independent of its potential rewarding value, the informational value of performance feedback has recently been suggested to be neurophysiologically encoded in the dorsal portion of the posterior cingulate cortex (dPCC). To further test this association, we investigated multidimensional categorization and reversal learning by comparing negative and positive feedback in an event-related functional magnetic resonance imaging experiment. Negative feedback, compared with positive feedback, increased activation in the dPCC as well as in brain regions typically involved in error processing. Only in the dPCC, subarea d23, this effect was significantly enhanced in relearning, where negative feedback signaled the need to shift away from a previously established response policy. Together with previous findings, this result contributes to a more fine-grained functional parcellation of PCC subregions and supports the dPCC’s involvement in the adaptation to behaviorally relevant information from the environment.

An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits

The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

Different types of uncertainty distinguished by monkey prefrontal neurons

To adapt one's behavior, in a timely manner, to an environment that changes in many different aspects, one must be sensitive to uncertainty about each aspect of the environment. Although the medial prefrontal cortex has been implicated in the representation and reduction of a variety of uncertainties, it is unknown whether different types of uncertainty are distinguished by distinct neuronal populations. To investigate how the prefrontal cortex distinguishes between different types of uncertainty, we recorded neuronal activities from the medial and lateral prefrontal cortices of monkeys performing a visual feedback-based action-learning task in which uncertainty of coming feedback and that of context change varied asynchronously. We found that the activities of two groups of prefrontal cells represented the two different types of uncertainty. These results suggest that different types of uncertainty are represented by distinct neural populations in the prefrontal cortex.

Impulsivity is related to overhasty risk learning in attention-deficit/hyperactivity disorder: A computational psychiatric approach

Attention-deficit/hyperactivity disorder (ADHD) is often accompanied by excessive risky behavior, and an impulsive trait has been proposed to be associated with risk-taking. However, the aspect of the cognitive process that impulsivity influences is not well understood. We hypothesized that impulsivity could be related to an overhasty shifting of beliefs during risk learning, thereby resulting in enhanced risk-taking behavior. In this study, we tested our hypothesis using the Bayesian modeling approach and predicted overhasty learning by a data-driven approach. We used an openly available task-based functional magnetic resonance imaging (fMRI) dataset. Participants with adult ADHD (n = 42) completed the balloon analog risk task (BART). By fitting our computational model that encapsulates the degree of overhasty learning, we estimated the degree of learning bias and investigated its relationship with Barratt Impulsiveness Scale (BIS) outcomes. Moreover, we created a connectome-based predictive model (CPM) based on fMRI data to predict the degree of risk-learning bias. The degree of overhasty learning in ADHD patients was significantly correlated with the BIS score (r = 0.424, p = 0.009). The CPM predicted the 'learning bias' parameter using negatively correlated edges (r = 0.341, p = 0.041; q² = 0.092). The 'hub nodes' in the predictive network were in the frontal lobe, including the orbitofrontal area. Our findings suggest that impulsivity in ADHD patients is associated with overhasty updating of beliefs during risk learning. Weak functional connectivity to the both dorso-lateral prefrontal and orbitofrontal lobes is predictive of the degree of overhasty learning.

Posterior brain sensorimotor recruitment for inhibition of delayed responses in children

Inhibitory control, the ability to suppress irrelevant thoughts or actions, is central to cognitive and social development. Protracted maturation of frontal brain networks has been reported as a major restraint for this ability, yet, young children, when motivated, successfully inhibit delayed responses. A better understanding of the age-dependent neural inhibitory mechanism operating during the awaiting-to-respond window in children may elucidate this conundrum. We recorded ERPs from children and parental adults to a visual-spatial working memory task with delayed responses. Cortical activation elicited during the first 1000 ms of the awaiting-to-respond window showed, as predicted by prior studies, early inhibitory effects in prefrontal ERPs (P200, 160-260 ms) associated with top-down attentional-biasing, and later effects in parietal/occipital ERPs (P300, 270-650 ms) associated with selective inhibition of task-irrelevant stimuli/responses and recurrent memory retrieval. Children successfully inhibited delayed responses and performed with a high level of accuracy (often over 90%), although, the prefrontal P200 displayed reduced amplitude and uniformly delayed peak latency, suggesting low efficacy of top-down attentional-biasing. P300, however, with no significant age-contrasts in latency was markedly elevated in children over the occipital/inferior parietal regions, with effects stronger in younger children. These results provide developmental evidence supporting the sensorimotor recruitment model of visual-spatial working memory relying on the occipital/parietal regions of the early maturing dorsal-visual network. The evidence is in line with the concept of age-dependent variability in the recruitment of cognitive inhibitory networks, complementing the former predominant focus on frontal lobes.

Reward impacts visual statistical learning

Humans automatically detect and remember regularities in the visual environment-a type of learning termed visual statistical learning (VSL). Many aspects of learning from reward resemble VSL in certain respects, yet whether and how reward learning impacts VSL is largely unexamined. In two studies, we found that reward contingencies affect VSL, with high-value associated with stronger behavioral and neural signatures of such learning than low-value images. In Experiment 1, participants learned values (high or low) of images through a trial-and-error risky choice task. Unbeknownst to them, images were paired as four types-High-High, High-Low, Low-High, and Low-Low. In subsequent recognition and reward memory tests, participants chose the more familiar of two pairs (a target and a foil) and recalled the value of images. We found better recognition when the first images of pairs have high-values, with High-High pairs showing the highest recognition rate. In Experiment 2, we provided evidence that both value and statistical contingencies affected brain responses. When we compared responses between the high-value first image and the low-value first image, greater activation in regions that included inferior frontal gyrus, anterior cingulate gyrus, hippocampus, among other regions, were found. These findings were driven by the interaction between statistically structured information and reward-the same value contrast yielded no regions for second-image contrasts and for singletons. Our results suggest that when reward information is embedded in stimulus-stimulus associations, it may alter the learning process; specifically, the higher-value first image potentially enables better memory for statistically learned pairs and reward information.

Corticostriatal Regulation of Language Functions

The role of corticostriatal circuits in language functions is unclear. In this review, we consider evidence from language learning, syntax, and controlled language production and comprehension tasks that implicate various corticostriatal circuits. Converging evidence from neuroimaging in healthy individuals, studies in populations with subcortical dysfunction, pharmacological studies, and brain stimulation suggests a domain-general regulatory role of corticostriatal systems in language operations. The role of corticostriatal systems in language operations identified in this review is likely to reflect a broader function of the striatum in responding to uncertainty and conflict which demands selection, sequencing, and cognitive control. We argue that this role is dynamic and varies depending on the degree and form of cognitive control required, which in turn will recruit particular corticostriatal circuits and components organised in a cognitive hierarchy.

Dorsal posterior cingulate cortex encodes the informational value of feedback in human-computer interaction

In communication between humans as well as in human–computer interaction, feedback is ubiquitous. It is essential for keeping up the dialogue between interaction partners, evaluating the adequacy of an action, or improving task performance. While the neuroscientific view on feedback has largely focused on its function as reward, more general definitions also emphasise its function as information about aspects of one’s task performance. Using fMRI in a computer-controlled auditory categorisation task, we studied the neural correlates of the informational value of computer-given feedback independent of reward. Feedback about the correctness of a decision, compared with feedback only indicating the registration of a decision, increases activation of the dorsal posterior cingulate cortex, supporting this region’s role in adapting to behaviourally relevant information. Both conditions elicit equally strong activation of the dorsal striatum which does not support an interpretation of feedback information as a type of reward. Instead, we suggest that it reflects a more fundamental aspect of human interaction behaviour, namely the establishment of a state that enables us to continue with the next step of the interaction.

Spatial migration of human reward processing with functional development: Evidence from quantitative meta-analyses

Functional magnetic resonance imaging (fMRI) studies have shown notable age‐dependent differences in reward processing. We analyzed data from a total of 554 children, 1,059 adolescents, and 1,831 adults from 70 articles. Quantitative meta‐analyses results show that adults engage an extended set of regions that include anterior and posterior cingulate gyri, insula, basal ganglia, and thalamus. Adolescents engage the posterior cingulate and middle frontal gyri as well as the insula and amygdala, whereas children show concordance in right insula and striatal regions almost exclusively. Our data support the notion of reorganization of function over childhood and adolescence and may inform current hypotheses relating to decision‐making across age.

Short-Term Classification Learning Promotes Rapid Global Improvements of Information Processing in Human Brain Functional Connectome

Classification learning is a preeminent human ability within the animal kingdom but the key mechanisms of brain networks regulating learning remain mostly elusive. Recent neuroimaging advancements have depicted human brain as a complex graph machinery where brain regions are nodes and coherent activities among them represent the functional connections. While long-term motor memories have been found to alter functional connectivity in the resting human brain, a graph topological investigation of the short-time effects of learning are still not widely investigated. For instance, classification learning is known to orchestrate rapid modulation of diverse memory systems like short-term and visual working memories but how the brain functional connectome accommodates such modulations is unclear. We used publicly available repositories (openfmri.org) selecting three experiments, two focused on short-term classification learning along two consecutive runs where learning was promoted by trial-by-trial feedback errors, while a further experiment was used as supplementary control. We analyzed the functional connectivity extracted from BOLD fMRI signals, and estimated the graph information processing in the cerebral networks. The information processing capability, characterized by complex network statistics, significantly improved over runs, together with the subject classification accuracy. Instead, null-learning experiments, where feedbacks came with poor consistency, did not provoke any significant change in the functional connectivity over runs. We propose that learning induces fast modifications in the overall brain network dynamics, definitely ameliorating the short-term potential of the brain to process and integrate information, a dynamic consistently orchestrated by modulations of the functional connections among specific brain regions.

Sparse Principal Component based High-Dimensional Mediation Analysis

Causal mediation analysis aims to quantify the intermediate effect of a mediator on the causal pathway from treatment to outcome. When dealing with multiple mediators, which are potentially causally dependent, the possible decomposition of pathway effects grows exponentially with the number of mediators. An existing approach incorporated the principal component analysis (PCA) to address this challenge based on the fact that the transformed mediators are conditionally independent given the orthogonality of the principal components (PCs). However, the transformed mediator PCs, which are linear combinations of original mediators, can be difficult to interpret. A sparse high-dimensional mediation analysis approach is proposed which adopts the sparse PCA method to the mediation setting. The proposed approach is applied to a task-based functional magnetic resonance imaging study, illustrating its ability to detect biologically meaningful results related to an identified mediator.

Power Spectral Density and Functional Connectivity Changes due to a Sensorimotor Neurofeedback Training: A Preliminary Study

Neurofeedback is a form of neuromodulation based on learning to modify some aspects of cortical activity. Sensorimotor rhythm (SMR) oscillation is one of the most used frequency bands in neurofeedback. Several studies have shown that subjects can learn to modulate SMR power to control output devices, but little is known about possible related changes in brain networks. The aim of this study was to investigate the enhanced performance and changes in EEG power spectral density at somatosensory cerebral areas due to a bidirectional modulation-based SMR neurofeedback training. Furthermore, we also analyzed the functional changes in somatosensory areas during resting state induced by the training as exploratory procedure. A six-session neurofeedback protocol based on learning to synchronize and desynchronize (modulate) the SMR was implemented. Moreover, half of the participants were enrolled in two functional magnetic resonance imaging resting-state sessions (before and after the training). At the end of the training, participants showed a successful performance enhancement, an increase in SMR power specific to somatosensory locations, and higher functional connectivity between areas associated with somatosensory activity in resting state. Our research increases the better understanding of the relation between EEG neuromodulation and functional changes and the use of SMR training in clinical practice.

Understanding the Neural Bases of Implicit and Statistical Learning

Both implicit learning and statistical learning focus on the ability of learners to pick up on patterns in the environment. It has been suggested that these two lines of research may be combined into a single construct of "implicit statistical learning." However, by comparing the neural processes that give rise to implicit versus statistical learning, we may determine the extent to which these two learning paradigms do indeed describe the same core mechanisms. In this review, we describe current knowledge about neural mechanisms underlying both implicit learning and statistical learning, highlighting converging findings between these two literatures. A common thread across all paradigms is that learning is supported by interactions between the declarative and nondeclarative memory systems of the brain. We conclude by discussing several outstanding research questions and future directions for each of these two research fields. Moving forward, we suggest that the two literatures may interface by defining learning according to experimental paradigm, with "implicit learning" reserved as a specific term to denote learning without awareness, which may potentially occur across all paradigms. By continuing to align these two strands of research, we will be in a better position to characterize the neural bases of both implicit and statistical learning, ultimately improving our understanding of core mechanisms that underlie a wide variety of human cognitive abilities.

Financial gain- and loss-related BOLD signals in the human ventral tegmental area and substantia nigra pars compacta

Neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNC) play central roles in reward-related behaviours. Nonhuman animal studies suggest that these neurons also process aversive events. However, our understanding of how the human VTA and SNC responds to such events is limited and has been hindered by the technical challenge of using functional magnetic resonance imaging (fMRI) to investigate a small structure where the signal is particularly vulnerable to physiological noise. Here we show, using methods optimized specifically for the midbrain (including high-resolution imaging, a novel registration protocol, and physiological noise modelling), a BOLD (blood-oxygen-level dependent) signal to both financial gain and loss in the VTA and SNC, along with a response to nil outcomes that are better or worse than expected in the VTA. Taken together, these findings suggest that the human VTA and SNC are involved in the processing of both appetitive and aversive financial outcomes in humans.

Dissociable neural mechanisms underlie value-driven and selection-driven attentional capture

Stimuli associated with reward acquire the ability to automatically capture attention. It is also the case that, with sufficient training, former targets can acquire the ability to capture attention in the absence of extrinsic rewards. It remains unclear whether these two experience-dependent attentional biases share a common underlying mechanism. The present study examined the influence of selection history on attentional capture, and compared its neural correlates with those of value-driven attentional capture reported in Anderson et al. (2014a). Participants completed a four-day training in visual search for a specific colour target. In a subsequent test phase, they performed visual search for a shape-defined target in which colour was task-irrelevant. Response times were slower when a former target-colour distractor was present than when it was absent, replicating attentional capture by unrewarded former targets. Neuroimaging results revealed preferential activation by a former target-colour distractor in sensory areas. A more right lateralised pattern of activation was observed, compared to attentional capture by reward cues. No distractor-evoked activity was found in the caudate tail. These results imply that attentional capture by selection history is primarily driven by plasticity in sensory areas, and that reward history and selection history influence attention via dissociable underlying mechanisms.

The benefits of negative yet informative feedback

We investigated whether negative feedback with information could benefit both behavioral and neural responses. Fifteen participants were scanned with functional magnetic resonance imaging (fMRI) while receiving various feedbacks in a novel perceptual task. Behavioral data showed that reaction times of task performance were faster after receiving negative informative feedback compared to negative confirmatory feedback. The fMRI analysis of the interaction contrast between feedback type (informative vs. confirmatory) and valence (negative vs. positive) showed greater activation in the ventrolateral prefrontal cortex (vlPFC) and the ventral striatum in response to negative informative compared to confirmatory feedback. The psychophysiological interactions (PPI) analyses showed that the vlPFC activation was positively correlated with the amygdala and the rostral cingulate zone (RCZ). The ventral striatum activation was negatively correlated with the dorsolateral prefrontal cortex (dlPFC). These results suggest that negative but informative feedback benefits subsequent performance and its primary function is to elicit positive prediction error (instructive signal) and to induce cognitive control to guide subsequent goal-directed behavior.

Dynamics of error-related activity in deterministic learning - an EEG and fMRI study

There is a close relationship between progress in learning and the error-monitoring system. EEG and fMRI studies using probabilistic learning have revealed the distinct dynamics of this system after response and feedback, i.e. an increase of error-related and a decrease of feedback-related activity in the anterior cingulate cortex (ACC). Based on the existing theories, it can be presumed that during deterministic learning feedback-related activity in ACC would also increase. Since these assumptions have not yet been confirmed, it can be only speculated based on the data from the probabilistic paradigms how the information is being integrated within the ACC during deterministic learning. Here we implemented the Paired Associate Deterministic Learning task to the EEG and fMRI experiments. The analysis of EEG data showed a significant increase in the amplitude for both ERN and FN. Similarly, the fMRI results showed an increase in response-related and feedback-related activity of the ACC in erroneous trials. Our findings are in line with the current theories of ACC function: increasing ACC activity can be linked to the detected discrepancy between expected and obtained outcomes. We argue that expectancy violations in the course of deterministic learning are signalled by both, internal and external evaluation system.

Impaired reward prediction error encoding and striatal-midbrain connectivity in depression

Anhedonia (hyposensitivity to rewards) and negative bias (hypersensitivity to punishments) are core features of major depressive disorder (MDD), which could stem from abnormal reinforcement learning. Emerging evidence highlights blunted reward learning and reward prediction error (RPE) signaling in the striatum in MDD, although inconsistencies exist. Preclinical studies have clarified that ventral tegmental area (VTA) neurons encode RPE and habenular neurons encode punishment prediction error (PPE), which are then transmitted to the striatum and cortex to guide goal-directed behavior. However, few studies have probed striatal activation, and functional connectivity between VTA-striatum and VTA-habenula during reward and punishment learning respectively, in unmedicated MDD. To fill this gap, we acquired fMRI data from 25 unmedicated MDD and 26 healthy individuals during a monetary instrumental learning task and utilized a computational modeling approach to characterize underlying neural correlates of RPE and PPE. Relative to controls, MDD individuals showed impaired reward learning, blunted RPE signal in the striatum and overall reduced VTA-striatal connectivity to feedback. Critically, striatal RPE signal was increasingly blunted with more major depressive episodes (MDEs). No group differences emerged in PPE signals in the habenula and VTA or in connectivity between these regions. However, PPE signals in the habenula correlated positively with number of MDEs. These results highlight impaired reward learning, disrupted RPE signaling in the striatum (particularly among individuals with more lifetime MDEs) as well as reduced VTA-striatal connectivity in MDD. Collectively, these findings highlight reward-related learning deficits in MDD and their underlying pathophysiology.

Knowledge acquisition is governed by striatal prediction errors

Discrepancies between expectations and outcomes, or prediction errors, are central to trial-and-error learning based on reward and punishment, and their neurobiological basis is well characterized. It is not known, however, whether the same principles apply to declarative memory systems, such as those supporting semantic learning. Here, we demonstrate with fMRI that the brain parametrically encodes the degree to which new factual information violates expectations based on prior knowledge and beliefs-most prominently in the ventral striatum, and cortical regions supporting declarative memory encoding. These semantic prediction errors determine the extent to which information is incorporated into long-term memory, such that learning is superior when incoming information counters strong incorrect recollections, thereby eliciting large prediction errors. Paradoxically, by the same account, strong accurate recollections are more amenable to being supplanted by misinformation, engendering false memories. These findings highlight a commonality in brain mechanisms and computational rules that govern declarative and nondeclarative learning, traditionally deemed dissociable.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Separating the effect of reward from corrective feedback during learning in patients with Parkinson's disease

Parkinson's disease (PD) is associated with procedural learning deficits. Nonetheless, studies have demonstrated that reward-related learning is comparable between patients with PD and controls (Bódi et al., Brain, 132(9), 2385-2395, 2009; Frank, Seeberger, & O'Reilly, Science, 306(5703), 1940-1943, 2004; Palminteri et al., Proceedings of the National Academy of Sciences of the United States of America, 106(45), 19179-19184, 2009). However, because these studies do not separate the effect of reward from the effect of practice, it is difficult to determine whether the effect of reward on learning is distinct from the effect of corrective feedback on learning. Thus, it is unknown whether these group differences in learning are due to reward processing or learning in general. Here, we compared the performance of medicated PD patients to demographically matched healthy controls (HCs) on a task where the effect of reward can be examined separately from the effect of practice. We found that patients with PD showed significantly less reward-related learning improvements compared to HCs. In addition, stronger learning of rewarded associations over unrewarded associations was significantly correlated with smaller skin-conductance responses for HCs but not PD patients. These results demonstrate that when separating the effect of reward from the effect of corrective feedback, PD patients do not benefit from reward.

Reducing failures of working memory with performance feedback

Fluctuations in attentional control can lead to failures of working memory (WM), in which the subject is no better than chance at reporting items from a recent display. In three experiments, we used a whole-report measure of visual WM to examine the impact of feedback on the rate of failures. In each experiment, subjects remembered an array of colored items across a blank delay, and then reported the identity of items using a whole-report procedure. In Experiment 1, we gave subjects simple feedback about the number of items they correctly identified at the end of each trial. In Experiment 2, we gave subjects additional information about the cumulative number of items correctly identified within each block. Finally, in Experiment 3, we gave subjects weighted feedback in which poor trials resulted in lost points and consistent successful performance received "streak" points. Surprisingly, simple feedback (Exp. 1) was ineffective at improving average performance or decreasing the rate of poor-performance trials. Simple cumulative feedback (Exp. 2) modestly decreased poor-performance trials (by 7 %). Weighted feedback produced the greatest benefits, decreasing the frequency of poor-performance trials by 28 % relative to baseline performance. This set of results demonstrates the usefulness of whole-report WM measures for investigating the effects of feedback on WM performance. Further, we showed that only a feedback structure that specifically discouraged lapses using negative feedback led to large reductions in WM failures.

Motor cortex inhibition by TMS reduces cognitive non-motor procedural learning when immediate incentives are present

Inhibitory repetitive transcranial magnetic stimulation (rTMS) of the primary motor area (M1) impairs motor sequence-learning, but not basic motor function. It is unknown if this is specific for motor forms of procedural learning or a more general effect. To investigate, we tested the effect of M1-inhibition on the weather prediction task (WPT), a learning task with minimal motor learning component. In the WPT, participants learn arbitrary, probabilistic, associations between sets of meaningless cues and fictional outcomes. In our "Feedback" (FB) condition, they received monetary rewards/punishments during learning. In the "paired associate" (PA) condition they learned the same information by passive observation of associations. The observational and feedback learning conditions were matched for their non-learning-specific motor demands. In each of two FB or PA sessions, we delivered Real (inhibitory) or Sham continuous theta-burst (cTBS) to the left-M1, before 150 training-trials. We then tested learning with 42 trials without feedback immediately after learning and again 1-h after cTBS. Compared to Sham, Real cTBS reduced performance during FB-learning, when learning was immediately reinforced, but not when knowledge was tested after PA learning. Furthermore, when FB-based memory was tested after learning without immediate incentive, there was no effect of TMS compared to post-PA test performance, showing the TMS effect operated only in the presence of incentive and feedback. We conclude that M1 is a node in a network underlying feedback-driven procedural learning and inhibitory rTMS there results in decreased network efficiency.

Neurobiological findings related to Internet use disorders

In the last 10 years, numerous neurobiological studies have been conducted on Internet addiction or Internet use disorder. Various neurobiological research methods - such as magnetic resonance imaging; nuclear imaging modalities, including positron emission tomography and single photon emission computed tomography; molecular genetics; and neurophysiologic methods - have made it possible to discover structural or functional impairments in the brains of individuals with Internet use disorder. Specifically, Internet use disorder is associated with structural or functional impairment in the orbitofrontal cortex, dorsolateral prefrontal cortex, anterior cingulate cortex, and posterior cingulate cortex. These regions are associated with the processing of reward, motivation, memory, and cognitive control. Early neurobiological research results in this area indicated that Internet use disorder shares many similarities with substance use disorders, including, to a certain extent, a shared pathophysiology. However, recent studies suggest that differences in biological and psychological markers exist between Internet use disorder and substance use disorders. Further research is required for a better understanding of the pathophysiology of Internet use disorder.

From Concrete Examples to Abstract Relations: The Rostrolateral Prefrontal Cortex Integrates Novel Examples into Relational Categories

The ability to form relational categories for objects that share few features in common is a hallmark of human cognition. For example, anything that can play a preventative role, from a boulder to poverty, can be a "barrier." However, neurobiological research has focused solely on how people acquire categories defined by features. The present functional magnetic resonance imaging study examines how relational and feature-based category learning compare in well-matched learning tasks. Using a computational model-based approach, we observed a cluster in left rostrolateral prefrontal cortex (rlPFC) that tracked quantitative predictions for the representational distance between test and training examples during relational categorization. Contrastingly, medial and dorsal PFC exhibited graded activation that tracked decision evidence during both feature-based and relational categorization. The results suggest that rlPFC computes an alignment signal that is critical for integrating novel examples during relational categorization whereas other PFC regions support more general decision functions.

Whole-Brain Neural Dynamics of Probabilistic Reward Prediction

Predicting future reward is paramount to performing an optimal action. Although a number of brain areas are known to encode such predictions, a detailed account of how the associated representations evolve over time is lacking. Here, we address this question using human magnetoencephalography (MEG) and multivariate analyses of instantaneous activity in reconstructed sources. We overtrained participants on a simple instrumental reward learning task where geometric cues predicted a distribution of possible rewards, from which a sample was revealed 2000 ms later. We show that predicted mean reward (i.e., expected value), and predicted reward variability (i.e., economic risk), are encoded distinctly. Early on, representations of mean reward are seen in parietal and visual areas, and later in frontal regions with orbitofrontal cortex emerging last. Strikingly, an encoding of reward variability emerges simultaneously in parietal/sensory and frontal sources and later than mean reward encoding. An orbitofrontal variability encoding emerged around the same time as that seen for mean reward. Crucially, cross-prediction showed that mean reward and variability representations are distinct and also revealed that instantaneous representations become more stable over time. Across sources, the best fitting metric for variability signals was coefficient of variation (rather than SD or variance), but distinct best metrics were seen for individual brain regions. Our data demonstrate how a dynamic encoding of probabilistic reward prediction unfolds in the brain both in time and space.

SIGNIFICANCE STATEMENT Predicting future reward is paramount to optimal behavior. To gain insight into the underlying neural computations, we investigate how reward representations in the brain arise over time. Using magnetoencephalography, we show that a representation of predicted mean reward emerges early in parietal/sensory regions and later in frontal cortex. In contrast, predicted reward variability representations appear in most regions at the same time, and slightly later than for mean reward. For both features, representations dynamically change >1000 ms before stabilizing. The best metric for encoding variability is coefficient of variation, with heterogeneity in this encoding seen between brain areas. The results provide novel insights into the emergence of predictive reward representations.

Striatal dopaminergic reward response relates to age of first drunkenness and feedback response in at-risk youth

Dopamine receptor concentrations, primarily in the striatum, are hypothesized to contribute to a developmental imbalance between subcortical and prefrontal control systems in emerging adulthood potentially biasing motivation and increasing risky behaviors. Positron emission tomography studies have found significant reductions in striatal dopamine D2 receptors, and blunted amphetamine-induced dopamine release, in substance users compared with healthy controls. Extant literature is limited and inconsistent concerning vulnerability associated with having a family history of substance abuse (FH+). Some studies have reported familial liability associated with higher dopamine receptor levels, reduced dopamine response to stimulant challenges and decreased response to oral alcohol. However, other reports have failed to find group differences based on family history. We explored the interaction of familial liability and behavioral risk with multi-modal molecular and neural imaging of the dopaminergic system. Forty-four young adult male subjects performed monetary incentive delay tasks during both [¹¹ C]raclopride positron emission tomography and functional magnetic resonance imaging scans. FH+ subjects were identified as low (n = 24) or high risk (n = 9) based on early initiation of drunkenness. FH+ high-risk subjects exhibited heightened striatal dopamine response to monetary reward but did not differ in neural activations compared with FH+ low risk subjects and controls with no familial loading (n = 11). Across all subjects, a negative relationship was found between dopamine release and age of first drunkenness and a positive relationship with neural response to reward receipt. These results suggest that in at-risk individuals, higher dopamine transmission associated with monetary reward may represent a particularly useful neurobiological phenotype.

Influence of COMT Val158Met polymorphism on emotional decision-making: A sex-dependent relationship?

The biological underpinnings of sex-related differences in decision-making are still under-explored. The COMT gene is related to sexual dimorphism and with different choices made under uncertainty, albeit no study has specifically investigated a moderation effect of sex on the association between the COMT gene and the performance on decision-making paradigms. In this study, we investigated the influence of the COMT Val¹⁵⁸Met polymorphism on Iowa Gambling Task (IGT) performance depending on sex in a healthy adult sample. Participants were 192 healthy adults (84 men and 108 women). The first 40 choices in the IGT were considered decisions under ambiguity and the last 60 choices decisions under risk. To test our moderation hypothesis we used a separate regressions approach. The results revealed a sex-dependent effect of COMT Va l¹⁵⁸Met polymorphism on decision-making as measured by the IGT. Val/Val women showed the best performance in the last trials of the IGT. Therefore, the COMT Val¹⁵⁸Met polymorphism may be considered a genetic marker underlying sex differences in decision-making.

No Apparent Influence of Reward upon Visual Statistical Learning

Humans are capable of detecting and exploiting a variety of environmental regularities, including stimulus-stimulus contingencies (e.g., visual statistical learning) and stimulus-reward contingencies. However, the relationship between these two types of learning is poorly understood. In two experiments, we sought evidence that the occurrence of rewarding events enhances or impairs visual statistical learning. Across all of our attempts to find such evidence, we employed a training stage during which we grouped shapes into triplets and presented triplets one shape at a time in an undifferentiated stream. Participants subsequently performed a surprise recognition task in which they were tested on their knowledge of the underlying structure of the triplets. Unbeknownst to participants, triplets were also assigned no-, low-, or high-reward status. In Experiments 1A and 1B, participants viewed shape streams while low and high rewards were “randomly” given, presented as low- and high-pitched tones played through headphones. Rewards were always given on the third shape of a triplet (Experiment 1A) or the first shape of a triplet (Experiment 1B), and high- and low-reward sounds were always consistently paired with the same triplets. Experiment 2 was similar to Experiment 1, except that participants were required to learn value associations of a subset of shapes before viewing the shape stream. Across all experiments, we observed significant visual statistical learning effects, but the strength of learning did not differ amongst no-, low-, or high-reward conditions for any of the experiments. Thus, our experiments failed to find any influence of rewards on statistical learning, implying that visual statistical learning may be unaffected by the occurrence of reward. The system that detects basic stimulus-stimulus regularities may operate independently of the system that detects reward contingencies.

Performance Pressure Enhances Speech Learning

Real-world speech learning often occurs in high pressure situations such as trying to communicate in a foreign country. However, the impact of pressure on speech learning success is largely unexplored. In this study, adult, native speakers of English learned non-native speech categories under pressure or no-pressure conditions. In the pressure conditions, participants were informed that they were paired with a (fictitious) partner, and that each had to independently exceed a performance criterion for both to receive a monetary bonus. They were then informed that their partner had exceeded the bonus and the fate of both bonuses depended upon the participant's performance. Our results demonstrate that pressure significantly enhanced speech learning success. In addition, neurobiologically-inspired computational modeling revealed that the performance advantage was due to faster and more frequent use of procedural learning strategies. These results integrate two well-studied research domains and suggest a facilitatory role of motivational factors in speech learning performance that may not be captured in traditional training paradigms.

Cerebellar tDCS does not improve performance in probabilistic classification learning

In this study, the role of the cerebellum in a cognitive learning task using transcranial direct current stimulation (tDCS) was investigated. Using a weather prediction task, subjects had to learn the probabilistic associations between a stimulus (a combination of cards) and an outcome (sun or rain). This task is a variant of a probabilistic classification learning task, for which it has been reported that prefrontal tDCS enhances performance. Using a between-subject design, all 30 subjects learned to improve their performance with increasing accuracies and shortened response times over a series of 500 trials. Subjects also became more confident in their prediction during the experiment. However, no differences in performance and learning were observed between subjects receiving sham stimulation (n = 10) or anodal stimulation (2 mA for 20 min) over either the right cerebellum (n = 10) or the left prefrontal cortex (n = 10). This suggests that stimulating the brain with cerebellar tDCS does not readily influence probabilistic classification performances, probably due to the rather complex nature of this cognitive task.

What motivates adolescents? Neural responses to rewards and their influence on adolescents' risk taking, learning, and cognitive control

Adolescence is characterized by pronounced changes in motivated behavior, during which emphasis on potential rewards may result in an increased tendency to approach things that are novel and bring potential for positive reinforcement. While this may result in risky and health-endangering behavior, it may also lead to positive consequences, such as behavioral flexibility and greater learning. In this review we will discuss both the maladaptive and adaptive properties of heightened reward-sensitivity in adolescents by reviewing recent cognitive neuroscience findings in relation to behavioral outcomes. First, we identify brain regions involved in processing rewards in adults and adolescents. Second, we discuss how functional changes in reward-related brain activity during adolescence are related to two behavioral domains: risk taking and cognitive control. Finally, we conclude that progress lies in new levels of explanation by further integration of neural results with behavioral theories and computational models. In addition, we highlight that longitudinal measures, and a better conceptualization of adolescence and environmental determinants, are of crucial importance for understanding positive and negative developmental trajectories.

Sex-Differences, Handedness, and Lateralization in the Iowa Gambling Task

In a widely used decision-making task, the Iowa Gambling Task (IGT), male performance is observed to be superior to that of females, and is attributed to right lateralization (i.e., right hemispheric dominance). It is as yet unknown whether sex-differences in affect and motor lateralization have implications for sex-specific lateralization in the IGT, and specifically, whether sex-difference in performance in the IGT changes with right-handedness or with affect lateralization (decision valence, and valence-directed motivation). The present study (N = 320; 160 males) examined the effects of right-handedness (right-handedness vs. non-right-handedness) as a measure of motor lateralization, decision valence (reward vs. punishment IGT), and valence-directedness of task motivation (valence-directed vs. non-directed instructions), as measures of affective lateralization on IGT decision making. Analyses of variance revealed that both male and female participants showed valence-induced inconsistencies in advantageous decision-making; however, right-handed females made more disadvantageous decisions in a reward IGT. These results suggest that IGT decision-making may be largely right-lateralized in right-handed males, and show that sex and lateralized differences (motor and affect) have implications for sex-differences in IGT decision-making. Implications of the results are discussed with reference to lateralization and sex-differences in cognition.

The Cognitive Neuroscience of Human Decision Making: A Review and Conceptual Framework

Decision making, the process of choosing between options, is a fundamental human behavior that has been studied intensively by disciplines ranging from cognitive psychology to economics. Despite the importance of this behavior, the neural substrates of decision making are only beginning to be understood. Impaired decision making is recognized in neuropsychiatric conditions such as dementia and drug addiction, and the inconsistencies and biases of healthy decision makers have been intensively studied. However, the tools of cognitive neuroscience have only recently been applied to understanding the brain basis of this complex behavior. This article reviews the literature on the cognitive neuroscience of human decision making, focusing on the roles of the frontal lobes, and provides a conceptual framework for organizing this disparate body of work.

Neural substrates of cognitive biases during probabilistic inference

Decision making often requires simultaneously learning about and combining evidence from various sources of information. However, when making inferences from these sources, humans show systematic biases that are often attributed to heuristics or limitations in cognitive processes. Here we use a combination of experimental and modelling approaches to reveal neural substrates of probabilistic inference and corresponding biases. We find systematic deviations from normative accounts of inference when alternative options are not equally rewarding; subjects' choice behaviour is biased towards the more rewarding option, whereas their inferences about individual cues show the opposite bias. Moreover, inference bias about combinations of cues depends on the number of cues. Using a biophysically plausible model, we link these biases to synaptic plasticity mechanisms modulated by reward expectation and attention. We demonstrate that inference relies on direct estimation of posteriors, not on combination of likelihoods and prior. Our work reveals novel mechanisms underlying cognitive biases and contributions of interactions between reward-dependent learning, decision making and attention to high-level reasoning.

Humans are often biased in estimating the precise influence of probabilistic events on their decisions. Here, Khorsand and colleagues report a behavioural task that produces these biases in inference and describe a biophysically-plausible model that captures these behavioural deviations from optimal decision making.

Brain Activation of Negative Feedback in Rule Acquisition Revealed in a Segmented Wisconsin Card Sorting Test

The present study is to investigate the brain activation associated with the informative value of negative feedback in rule acquisition. In each trial of a segmented Wisconsin Card Sorting Test, participants were provided with three reference cards and one target card, and were asked to match one of three reference cards to the target card based on a classification rule. Participants received feedback after each match. Participants would acquire the rule after one negative feedback (1-NF condition) or two successive negative feedbacks (2-NF condition). The functional magnetic resonance imaging (fMRI) results indicated that lateral prefrontal-to-parietal cortices were more active in the 2-NF condition than in the 1-NF condition. The activation in the right lateral prefrontal cortex and left posterior parietal cortex increased gradually with the amount of negative feedback. These results demonstrate that the informative value of negative feedback in rule acquisition might be modulated by the lateral prefronto-parietal loop.