Expectancy affects the feedback-related negativity (FRN) for delayed feedback in probabilistic learning

The impact of emotional feedback in learning easy and difficult tasks - an ERP study

Learning from the emotional reaction of others is crucial in our everyday lives. We assumed that additional emotional information could be especially beneficial, when a task is difficult and the limits of working memory capacity are reached. For this reason, we examined whether a potential benefit of emotional feedback during reinforcement learning is dependent on working memory load. In addition to learning performance, we analysed the neural mechanisms of reinforcement learning by examining two event-related potentials (ERPs): feedback-related negativity (FRN) and P3b. Participants were divided into two difficulty groups (with n = 21 in the difficult and n = 22 in the easy group), performing a learning task with emotional or non-emotional feedback. Task difficulty was manipulated by varying the number of stimulus-response associations. Participants' showed learning in all conditions. Emotional feedback led to increased accuracy and decreased reaction times in both groups. However, this benefit occurred earlier in the easy condition. The detection of unexpected events, as reflected in the peak-to-peak FRN, as well as working memory updating, as reflected in the P3b, were enhanced after emotional in contrast to non-emotional feedback for both groups. In contrast, task difficulty had no effect on the detection of unexpected events but led to a P3b that was more evenly distributed over the scalp, which could indicate that additional frontal resources were recruited to perform the difficult task. Our results suggest that working memory load and emotional information independently influence feedback processing without interacting.

Revisiting the electrophysiological correlates of valence and expectancy in reward processing - A multi-lab replication

Two event-related brain potential (ERP) components, the frontocentral feedback-related negativity (FRN) and the posterior P300, are key in feedback processing. The FRN typically exhibits greater amplitude in response to negative and unexpected outcomes, whereas the P300 is generally more pronounced for positive outcomes. In an influential ERP study, Hajcak et al., (2005) manipulated outcome valence and expectancy in a guessing task. They found the FRN was larger for negative outcomes regardless of expectancy, and the P300 larger for unexpected outcomes regardless of valence. These findings challenged the dominant Reinforcement Learning Theory of the ERN. We aimed to replicate these results within the #EEGManyLabs project (Pavlov et al., 2021) across thirteen labs. Our replication, including robustness tests, a PCA and Bayesian models, found that both FRN and P300 were significantly modulated by outcome valence and expectancy: FRN amplitudes (no-reward - reward) were largest for unexpected outcomes, and P300 amplitudes were largest for reward outcomes. These results were consistent across different methods and analyses. Although our findings only partially replicate the original study, they underscore the complexity of feedback processing and demonstrate how aspects of Reinforcement Learning Theory may apply to the P300 component, reinforcing the need for rigorous ERP research methodologies.

Are you worth the wait? Waiting time modulates the social feedback processing: Evidence from event-related potentials

Processing social feedback is essential for establishing appropriate social connections. However, social feedback is not always immediate, and the impact of waiting time on social feedback processing remains unexplored. Using electroencephalography (EEG) and event-related potentials (ERPs), the present study investigated how waiting time affects the N170, reward positivity (RewP), and P3. Participants (N = 36) completed a social evaluation task, awaiting feedback from liked and disliked peers with short (800-1200 ms) or long (5000-6000 ms) waiting times. Participants were more motivated to receive feedback from liked peers, and they rated acceptance from liked peers as more pleasant than rejection. Notably, participants found longer waits more worthwhile when receiving acceptance from liked peers, but less worthwhile when awaiting feedback from disliked peers. EEG results revealed that the RewP was increased for long waiting times for feedback from liked peers, and, conversely, reduced for long waiting times for feedback from disliked peers. Additionally, N170 and P3 were found to be sensitive to waiting time, with larger amplitudes for long compared to short waits. Overall, this study demonstrates that waiting time differentially affects social feedback processing, as reflected by changes in the N170, RewP, and P3. Our findings suggest that increased waiting time does not necessarily reduce reward value; it can enhance it depending on subjective social preferences. The increased N170 and P3 amplitudes during longer waits may indicate heightened attentional and memory demands. This study advances our understanding of the neurocognitive mechanisms underlying social decision-making.

The effect of feedback timing on category learning and feedback processing in younger and older adults

Introduction

Corrective feedback can be received immediately after an action or with a temporal delay. Neuroimaging studies suggest that immediate and delayed feedback are processed by the striatum and medial temporal lobes (MTL), respectively. Age-related changes in the striatum and MTL may influence the efficiency of feedback-based learning in older adults. The current study leverages event-related potentials (ERPs) to evaluate age-related differences in immediate and delayed feedback processing and consequences for learning. The feedback-related negativity (FRN) captures activity in the frontostriatal circuit while the N170 is hypothesized to reflect MTL activation.

Methods

18 younger (Myears  = 24.4) and 20 older (Myears  = 65.5) adults completed learning tasks with immediate and delayed feedback. For each group, learning outcomes and ERP magnitudes were evaluated across timing conditions.

Results

Younger adults learned better than older adults in the immediate timing condition. This performance difference was associated with a typical FRN signature in younger but not older adults. For older adults, impaired processing of immediate feedback in the striatum may have negatively impacted learning. Conversely, learning was comparable across groups when feedback was delayed. For both groups, delayed feedback was associated with a larger magnitude N170 relative to immediate feedback, suggesting greater MTL activation.

Discussion and conclusion

Delaying feedback may increase MTL involvement and, for older adults, improve category learning. Age-related neural changes may differentially affect MTL- and striatal-dependent learning. Future research can evaluate the locus of age-related learning differences and how feedback can be manipulated to optimize learning across the lifespan.

Exploration of the influence of the quantification method and reference scheme on feedback-related negativity and standardized measurement error of feedback-related negativity amplitudes in a trust game

Various approaches have been taken over the years to quantify event-related potential (ERP) responses and these approaches may vary in their utility connecting empirical research and scientific claims. In this work we compared different quantification methods as well as the influence of three reference methods (linked mastoids, average reference, and current source density) on the resulting ERP amplitude. We use the experimental effects and effect sizes (Cohen's d) to evaluate the different methodological variants and we calculate intraclass correlation coefficients (ICC). In addition, the bootstrapped standard error of the means (SME, Luck et al., 2021), which was recently suggested as a quality criterion for ERP research, is used for this purpose. Our example for an ERP is the feedback-related negativity (FRN) to feedback about trustee behavior in a trust game with participants in the trustor position. We found that the quantification methods concerning the FRN influenced the absolute value of condition effects in the experimental paradigm. Yet, the patterns of effects were detected by all chosen methods, except for the 'individual difference wave'-based peak window approach. In addition, our findings stress the importance of checking the reference electrodes concerning effects of the experimental conditions. Furthermore, interactions of topographical distribution and reference choice should be considered. Finally, we were able to show that the SME is lower for more datapoints that are given in the quantification period of the FRN, and higher for more negative FRN amplitudes. These biases may lead to divergence of SME and effect size detection. Therefore, if the SME was used to compare different processing choices one should consider controlling for these important aspects of the data and possibly include other quality criteria like effect sizes.

Prediction-error-dependent processing of immediate and delayed positive feedback

Learning often involves trial-and-error, i.e. repeating behaviours that lead to desired outcomes, and adjusting behaviour when outcomes do not meet our expectations and thus lead to prediction errors (PEs). PEs have been shown to be reflected in the reward positivity (RewP), an event-related potential (ERP) component between 200 and 350 ms after performance feedback which is linked to striatal processing and assessed via electroencephalography (EEG). Here we show that this is also true for delayed feedback processing, for which a critical role of the hippocampus has been suggested. We found a general reduction of the RewP for delayed feedback, but the PE was similarly reflected in the RewP and the later P300 for immediate and delayed positive feedback, while no effect was found for negative feedback. Our results suggest that, despite processing differences between immediate and delayed feedback, positive PEs drive feedback processing and learning irrespective of delay.

Feedback processing in cognitive and motor tasks: A meta-analysis on the feedback-related negativity

For motor learning, the processing of behavioral outcomes is of high significance. The feedback-related negativity (FRN) is an event-related potential, which is often described as a correlate of the reward prediction error in reinforcement learning. The number of studies examining the FRN in motor tasks is increasing. This meta-analysis summarizes the component in the motor domain and compares it to the cognitive domain. Therefore, a data set of a previous meta-analysis in the cognitive domain that comprised 47 studies  was reanalyzed and compared to additional 25 studies of the motor domain. Further, a moderator analysis for the studies in the motor domain was conducted. The FRN amplitude was higher in the motor domain than in the cognitive domain. This might be related to a higher task complexity and a higher feedback ambiguity of motor tasks. The FRN latency was shorter in the motor domain than in the cognitive domain. Given that sensory information can be used as an external feedback predictor prior to the presentation of the final feedback, reward processing in the motor domain may have been faster and reduced the FRN latency. The moderator variable analysis revealed that the feedback modality influenced the FRN latency, with shorter FRN latencies after bimodal than after visual feedback. Processing of outcome feedback seems to share basic principles in both domains; however, differences exist and should be considered in FRN studies. Future research is motivated to scrutinize the effects of bimodal feedback and other moderators within the motor domain.

Effects of feedback reliability on event-related potentials in an arrow flanker task

The processing of feedback is essential for learning, error detection, and correction. However, the underlying mechanisms of the feedback's characteristics, such as its reliability, valence, and expectations in the processing of error information, are not completely clear. The two degrees of feedback reliability, reliable feedback and unreliable feedback, respectively, were established by manipulating the feedback valence. The time course of event-related potentials (ERP) during the arrow flanker tasks was used to investigate the effects of feedback reliability and responses on brain activity. Three ERP components, the error-related negativity (ERN), feedback-related negativity (FRN), and P3, respectively, were measured. The impacts of feedback reliability and responses on ERN, FRN, and P3 had a different profile. Specifically, ERN and P3 are associated with the responses but not the feedback reliability, while FRN is associated with feedback reliability and feedback expectations but not the responses. The ERN, FRN, and P3 reflect distinct cognitive processes in the processing of error information.

Learning new words via feedback-Association between feedback-locked ERPs and recall performance-An exploratory study

Feedback learning is thought to involve the dopamine system and its projection sites in the basal ganglia and anterior cingulate cortex (ACC), regions associated with procedural learning. Under certain conditions, such as when feedback is delayed, feedback-locked activation is pronounced in the medial temporal lobe (MTL), which is associated with declarative learning. In event-related potential research, the feedback-related negativity (FRN) has been linked to immediate feedback processing, while the N170, possibly reflecting MTL activity, has been related to delayed feedback processing. In the current study, we performed an exploratory investigation on the relation between N170 and FRN amplitude and memory performance in a test for declarative memory (free recall), also exploring the role of feedback delay. To this end, we adapted a paradigm in which participants learned associations between non-objects and non-words with either immediate or delayed feedback, and added a subsequent free recall test. We indeed found that N170, but not FRN amplitudes, depended on later free recall performance, with smaller amplitudes for later remembered non-words. In an additional analysis with memory performance as dependent variable, the N170, but not the FRN amplitude predicted free recall, modulated by feedback timing and valence. This finding shows that the N170 reflects an important process during feedback processing, possibly related to expectations and their violation, but is distinct from the process reflected by the FRN.

Feedback-related negativity in perfectionists: An index of performance outcome evaluation

It has been suggested that maladaptive perfectionists are more prone to concern over their performance outcomes than adaptive perfectionists. Performance outcome evaluation is reflected in the amplitude of feedback-related negativity (FRN) in brain electroencephalography (EEG). Hence, the amplitude of the FRN after receiving unfavorable feedback indicating a negative performance outcome may reflect personality characteristics. In other words, EEG could be a better marker of personality characteristics than self-report measures. However, the FRN component has not yet been investigated between different types of perfectionists. In the present study, group differences in the FRN were examined between two groups of adaptive and maladaptive perfectionists and a group of non-perfectionists during a monetary gambling task. We observed a larger FRN amplitude for adaptive perfectionists than for maladaptive perfectionists. This finding is consistent with previous reports that reward prediction error is reflected in the amplitude of the FRN. This difference in FRN could be interpreted as the pessimistic outcome expectation biases in maladaptive perfectionists.

Age-related differences in ERP correlates of value-based decision making

This study evaluates age-related differences in the temporal dynamics underlying neural processing of value for decision-making in younger and older adults. We applied a lottery-choice task with event-related potentials to determine how and when brain activity during choice and outcome processing diverge between younger and older adults. Behaviorally, older adults accepted more losing stakes than younger adults. During choice, younger adults evinced higher P2 ERP-response positivity with a later P3 positivity that monotonically increased with low to middle to high win probability. Older adults evinced lower P2 responses and P3 amplitudes with more positivity for high and low relative to middle win probability. Both age groups showed similar feedback-related negativity and late parietal positivity, indicating intact reward prediction error representations and salience integration. Feedback-P3 showed more complex sensitivity to expectancy violations in older than younger adults, suggesting subjective uncertainty about reward expectations. Reduced early general neural processing of objective stimulus value with greater contribution of downstream subjective processes might underlie older adult risk-taking behaviors.

Dissociable feedback valence effects on frontal midline theta during reward gain versus threat avoidance learning

While frontal midline theta (FMθ) has been associated with threat processing, with cognitive control in the context of anxiety, and with reinforcement learning, most reinforcement learning studies on FMθ have used reward rather than threat-related stimuli as reinforcer. Accordingly, the role of FMθ in threat-related reinforcement learning is largely unknown. Here, n = 23 human participants underwent one reward-, and one punishment-, based reversal learning task, which differed only with regard to the kind of reinforcers that feedback was tied to (i.e., monetary gain vs. loud noise burst, respectively). In addition to single-trial EEG, we assessed single-trial feedback expectations based on both a reinforcement learning computational model and trial-by-trial subjective feedback expectation ratings. While participants' performance and feedback expectations were comparable between the reward and punishment tasks, FMθ was more reliably amplified to negative vs. positive feedback in the reward vs. punishment task. Regressions with feedback valence, computationally derived, and self-reported expectations as predictors and FMθ as criterion further revealed that trial-by-trial variations in FMθ specifically relate to reward-related feedback-valence and not to threat-related feedback or to violated expectations/prediction errors. These findings suggest that FMθ as measured in reinforcement learning tasks may be less sensitive to the processing of events with direct relevance for fear and anxiety.

The impact of social comparison and (un)fairness on upstream indirect reciprocity: Evidence from ERP

Event-related potential (ERP) technology and the dictator game paradigm are used to explore the formation mechanism of upstream indirect reciprocity behaviors. We design a within subject experiment of 3 (social comparison: upward versus parallel versus downward) × 2 (treatment: fair versus unfair) involving 49 subjects. In the first round of allocations, subjects are forced to accept a monetary amount allocated to them by another player. In the second round, subjects assume the role of allocator and divide a monetary amount between themselves and a third party. Our results show the following: 1) Having received fair treatment from someone else, individuals engaged in downward comparison are more inclined to reciprocate the fairness they had received to a third party compared to individuals in parallel and upward comparison conditions. If individuals receive unfair treatment, they tend to repeat this behavior to a third party regardless of which social comparison condition they are in; 2) Under the condition of upward comparison, individuals receiving unfair treatment exhibit greater FRN amplitude and less P300 amplitude, but in parallel and downward comparison conditions, there is no significance in FRN and P300 amplitude between individuals receiving fair and unfair treatment.

Investigating mechanisms of cognitive control training: neural signatures of PASAT performance in depressed patients

Major depression disorder (MDD) is characterized by cognitive control (CC) dysfunctions associated with increased attention toward negative information. The paced auditory serial addition task (PASAT) has been used as a targeted training of CC and studies show promising effects on depressive symptoms. However, neural mechanisms underlying its efficacy are still unclear. Based on previous findings of feedback-locked event-related potentials in healthy subjects, we investigated neural signatures during PASAT performance in 46 depressed patients. We found significantly larger amplitudes after negative than positive feedback for the P300 and late positive potential (LPP). However, this difference was not significant for the feedback-related negativity (FRN). Moreover, no associations of valence-specific ERPs and PASAT performance nor depressive symptoms were found. This indicates that depressed patients seem unable to use neural activation in late feedback processing stages (P300, LPP) to adapt accordingly. Moreover, lack of valence-specific neural reaction in early feedback processing stages (FRN) might point toward emotional indifference in depressed patients.Trial registration number: NCT03518749 Date of registration: May 8, 2018.

Cooperate or aggress? An opponent's tendency to cooperate modulates the neural dynamics of interpersonal cooperation

Humans are social animals and need to cooperate to survive. However, individuals are not cooperative in every social interaction, and their cooperation may depend on social context. The present study used a social dilemma game to investigate whether an opponent's tendency to be cooperative over time influenced a player's behavior and neural response to outcomes in the game. University students ("players") thought they were playing against other students ("opponents") in the Chicken Game but were actually playing against a programmed computer. Participants were randomly assigned to play with an opponent who tended to be competitive (cooperative 20% of the time) or who tended to be cooperative (cooperative 80% of the time). The results showed that early in the game, participants in both groups adopted a "tit-for-tat" strategy. However, as the game progressed and the opponent's behavioral tendency became more noticeable, players in the competitive-opponent group became generally more cooperative to limit their losses. ERPs analyses indicated that players had a higher P300 and larger theta power in response to the opponent's aggression but not to the opponent's cooperation when their opponent showed a tendency to be cooperative vs. competitive. The results suggest that people adjust their cooperative behavior based on their opponent's behavior in social interaction, and aggression captures more attention than cooperation in this process.

Delaying feedback compensates for impaired reinforcement learning in developmental dyslexia

A theoretical framework suggests that developmental dyslexia is characterized by abnormalities in brain structures underlying the procedural learning and memory systems while the declarative learning and memory systems are presumed to remain intact or even enhanced (Procedural Deficit Hypothesis). This notion has been supported by a substantial body of research, which focused on each system independently. However, less attention has been paid to interactions between these memory systems which may provide insights as to learning situations and conditions in which learning in dyslexia can be improved. The current study was undertaken to examine these important but unresolved issues. To this end, probabilistic reinforcement learning and episodic memory tasks were examined in participants with dyslexia and neurotypicals simultaneously within a single task. Feedback timing presentation was manipulated, building on prior research indicating that delaying feedback timing shifts striatal-based probabilistic learning, to become more hippocampal-dependent. It was hypothesized that if the procedural learning and memory systems are impaired in dyslexia, performance will be impaired under conditions that encourage procedural memory engagement (immediate feedback trials) but not under conditions that promote declarative memory processing (long delayed feedback trials). It was also predicted that the ability to incidentally acquire episodic information would be preserved in dyslexia. The results supported these predictions. Participants with dyslexia were impaired in probabilistic learning of cue-outcome associations compared to neurotypicals in an immediate feedback condition, but not when feedback on choices was presented after a long delay. Furthermore, participants with dyslexia demonstrated similar performance to neurotypicals in a task requiring incidental episodic memory formation. These findings attest to a dissociation between procedural-based and declarative-based learning in developmental dyslexia within a single task, a finding that adds discriminative validity to the Procedural Deficit Hypothesis. Just as important, the present findings suggest that training conditions designed to shift the load from midbrain/striatal systems to declarative memory mechanisms have the potential to compensate for impaired learning in developmental dyslexia.

Acting in Temporal Contexts: On the Behavioral and Neurophysiological Consequences of Feedback Delays

Feedback on success or failure is critical to increase rewards through behavioral adaptation or learning of dependencies from trial and error. Learning from reward feedback is thereby treated as embedded in a reinforcement learning framework. Due to temporal discounting of reward, learning in this framework is suspected to be vulnerable to feedback delay. Together, investigations of reinforcement learning in learned decision making tasks show that performance and learning impairments due to feedback delay vary as a function of task type. Performance in tasks that require implicit processing is affected by the delayed availability of feedback compared to tasks that can be accomplished with explicit processing. At the same time, the feedback related negativity, an event related potential component in the electroencephalogram that is associated with feedback processing, is affected by feedback delay similarly independent of task type. With the idea of fully implicit or explicit processing as opposite endpoints of a continuum of reciprocal shares of the implicit and explicit processing systems with feedback delay as the determinant of where on this continuum processing can be located, a common explanatory approach of both, behavioral and electrophysiological findings, is suggested.

Dissociating the effect of reward uncertainty and timing uncertainty on neural indices of reward prediction errors: A reward positivity (RewP) event-related potential (ERP) study

Accurate reward predictions include forecasting both what a reward will be and when a reward will occur. We tested how variations in the certainty of reward outcome and certainty in timing of feedback presentation modulate neural indices of reward prediction errors using the reward positivity (RewP) component of the scalp-recorded brain event-related potential (ERP). In a within-subjects design, seventy-three healthy individuals completed two versions of a cued doors task; one cued the probability of a reward outcome while the other cued the probability of a delay before feedback. Replicating previous results, RewP amplitude was larger for uncertain feedback compared to certain feedback. Additionally, RewP amplitude was differentially associated with uncertainty of presence/absence of reward, but not uncertainty of feedback timing. Findings suggest a dissociation in that RewP amplitude is modulated by reward prediction certainty but is less affected by certainty surrounding timing of feedback.

Variability in competitive decision-making speed and quality against exploiting and exploitative opponents

A presumption in previous work has been that sub-optimality in competitive performance following loss is the result of a reduction in decision-making time (i.e., post-error speeding). The main goal of this paper is to test the relationship between decision-making speed and quality, with the hypothesis that slowing down decision-making should increase the likelihood of successful performance in cases where a model of opponent domination can be implemented. Across Experiments 1–3, the speed and quality of competitive decision-making was examined in a zero-sum game as a function of the nature of the opponent (unexploitable, exploiting, exploitable). Performance was also examined against the nature of a credit (or token) system used as a within-experimental manipulation (no credit, fixed credit, variable credit). To compliment reaction time variation as a function of outcome, both the fixed credit and variable credit conditions were designed to slow down decision-making, relative to a no credit condition where the game could be played in quick succession and without interruption. The data confirmed that (a) self-imposed reductions in processing time following losses (post-error speeding) were causal factors in determining poorer-quality behaviour, (b) the expression of lose-shift was less flexible than the expression of win-stay, and, (c) the use of a variable credit system may enhance the perceived control participants have against exploitable opponents. Future work should seek to disentangle temporal delay and response interruption as determinants of decision-making quality against numerous styles of opponency.

Acute stress impairs reward positivity effect in probabilistic learning

Decision making based on feedback learning requires a series of cognitive processes, including estimating the probability of particular outcomes and modulating expectations between expected versus actual outcomes. It has been suggested that stress affects decision making and subsequent processing of feedback valence and magnitude. However, less is known about the effect of acute stress on reward expectancy. In the current study, participants performed a probabilistic learning task, in which they learned an association between response and feedback within different reward expectancy trials (30% and 70%) under the conditions of stress (threat of shock) and safety (no shock). We recorded event-related potentials (ERPs) to measure the reward positivity (RewP) which reflects reward prediction error signals during feedback processing. Behavioral data indicated that participants performed better in the 70% reward trials than in the 30% reward trials. However, no significant difference was observed between stress and safe conditions. Importantly, ERP results indicated that unexpected feedback elicited larger RewP than did expected feedback and this expectancy effect of RewP was reduced in the stress relative to safe condition. Moreover, the correlations between RewP and choice accuracy (70% - 30% reward condition) in the safe and stress conditions were in a similar pattern; yet, only in the stress condition the correlation reached significantly. This may indicate that blunted RewP was associated with impaired performance at an individual level. Our study provides ERP evidence that acute stress affects brain responses to reward prediction error processing, which may explain various abnormal learning behaviors associated with stress-related disorders.

Stress effects on learning and feedback-related neural activity depend on feedback delay

Depending on feedback timing, the neural structures involved in learning differ, with the dopamine system including the dorsal striatum and anterior cingulate cortex (ACC) being more important for learning from immediate than delayed feedback. As stress has been shown to promote striatum-dependent learning, the current study aimed to explore if stress differentially affects learning from and processing of immediate and delayed feedback. One group of male participants was stressed using the socially evaluated cold pressor test, and another group underwent a control condition. Subsequently, participants performed a reward learning task with immediate (500 ms) and delayed (6,500 ms) feedback while brain activity was assessed with electroencephalography (EEG). While stress enhanced the accuracy for delayed relative to immediate feedback, it reduced the feedback-related negativity (FRN) valence effect, which is the amplitude difference between negative and positive feedback. For the P300, a reduced valence effect was found in the stress group only for delayed feedback. Frontal theta power was most pronounced for immediate negative feedback and was generally reduced under stress. Moreover, stress reduced associations of FRN and theta power with trial-by-trial accuracy. Associations between stress-induced cortisol increases and EEG components were examined using linear mixed effects analyses, which showed that the described stress effects were accompanied by associations between the stress-induced cortisol increases and feedback processing. The results indicate that stress and cortisol affect different aspects of feedback processing. Instead of an increased recruitment of the dopamine system and the ACC, the results may suggest enhanced salience processing and reduced cognitive control under stress.

Subjective preferences differentially modulate the processing of rewards gained by own vs. observed choices

The present EEG study investigated the impact of subjective reward preferences and agency on outcome processing. 47 healthy adults (11 male; 36 female) with preferences for either milk or white chocolate completed two runs of a gambling task involving their preferred chocolate (high preference outcomes, HPOs), non-preferred chocolate (medium preference outcomes, MPOs), and a lesser liked non-chocolate reward (low preference outcomes, LPOs). In the 'active' run, subjects chose between three different response options to receive the outcomes. In the 'observational' run, they observed another person's choices and subsequent outcomes. Cluster-based permutation analyses of event-related potential (ERPs) revealed that early processing in the P2 time window reflected outcome salience, differentiating HPOs and MPOs from LPOs, especially for outcomes following own choices, while not distinguishing between HPOs and MPOs. In contrast, processing in later stages, i.e., the typical time windows for feedback-related negativity (FRN) and P300, showed evidence of differential coding of HPOs and MPOs and was also modulated by agency. ERPs clearly differentiated between all three outcome types in the FRN and P300 time windows for outcomes following active but not for observed choices. The present study adds to evidence for modulation of outcome processing by contextual and inter-individual factors. In particular, our findings suggest that subjective preferences are complementarily represented in subjective reward valuation and in motivational value representations indexed by the FRN and the P300.

Electrophysiological correlates of prediction formation in anticipation of reward- and punishment-related feedback signals

Feedback processing during decision making involves comparing anticipated and actual outcome. Although effects on ERPs of valence, magnitude, expectancy, and context during feedback processing have been extensively investigated, the electrophysiological processes underlying prediction formation in anticipation of feedback signals have received little attention. The aim of the present study was to explore these processes of prediction formation and their influence on subsequent feedback signals. Twenty healthy, right‐handed volunteers performed a forced‐choice task in which they had to indicate which of two presented objects was more expensive. After the volunteer's choice, an expert cue, which was accurate in 80% of trials, was presented to manipulate prediction formation about future reward and punishment. ERPs were recorded during presentation of the expert cue and during feedback. Results revealed that prediction formation of future rewards and punishments is accompanied by differences in the P2 component and a subsequent delay period. During feedback processing, the prediction‐related P2 was associated with the processing of valence reflected in the feedback‐related P2. Furthermore, the prediction‐related difference in the delay period was associated with error processing in feedback‐related medial frontal negativity. These findings suggest that prediction signals prior to feedback contain information about whether a prediction is correct or wrong (expectancy) and if the outcome will be a reward or punishment (valence).

During feedback processing, a prediction of future outcome is affected by contextual information. Here, we show that electrocortical signals related to future outcome following contextual information inversely predict reward signals during feedback processing. Additionally, signal differences after presentation of contextual information relate to future signal differences reflecting prediction error during feedback processing.

Modulation of feedback processing by social context in social anxiety disorder (SAD)-an event-related potentials (ERPs) study

The ability to learn from feedback, especially under social scrutiny, is an essential prerequisite for successful interaction with the environment. Patients suffering from social anxiety disorder (SAD) have been proposed to show altered processing of and learning from feedback, especially depending on social context. However, the neural basis and behavioral consequences of altered reinforcement learning in SAD are not clear yet. In the present event-related potentials (ERPs) study, 34 SAD patients and 30 healthy control subjects (HC) performed an adapted version of a probabilistic feedback learning task in two distinct social conditions. In the observation condition, participants were observed by a confederate; in the control condition, they performed the task without being observed. Patients as compared to healthy controls experienced more subjective discomfort under social observation. Moreover, they showed better learning from negative feedback in the control condition, but reduced learning from negative feedback in the observation condition. This effect correlated with reduced differentiation of positive and negative feedback in the time range of the feedback-related negativity (FRN) under high action-feedback contingency. In addition, SAD patients demonstrated increased FRN amplitudes in the first half of the observation condition, in particular to positive feedback. The present results demonstrate that processing of and learning from feedback are altered in SAD, especially under social scrutiny. In particular, it appears that SAD patients do not process positive information adequately on the neural level, which may impair their ability to differentiate between negative and positive outcomes.

Accuracy of Motor Error Predictions for Different Sensory Signals

Detecting and evaluating errors in action execution is essential for learning. Through complex interactions of the inverse and the forward model, the human motor system can predict and subsequently adjust ongoing or subsequent actions. Inputs to such a prediction are efferent and afferent signals from various sources. The aim of the current study was to examine the impact of visual as well as a combination of efferent and proprioceptive input signals to error prediction in a complex motor task. Predicting motor errors has been shown to be correlated with a neural signal known as the error-related negativity (Ne/ERN). Here, we tested how the Ne/ERN amplitude was modulated by the availability of different sensory signals in a semi-virtual throwing task where the action outcome (hit or miss of the target) was temporally delayed relative to movement execution allowing participants to form predictions about the outcome prior to the availability of knowledge of results. 19 participants practiced the task and electroencephalogram was recorded in two test conditions. In the Visual condition, participants received only visual input by passively observing the throwing movement. In the EffProp condition, participants actively executed the task while visual information about the real and the virtual effector was occluded. Hence, only efferent and proprioceptive signals were available. Results show a significant modulation of the Ne/ERN in the Visual condition while no effect could be observed in the EffProp condition. In addition, amplitudes of the feedback-related negativity in response to the actual outcome feedback were found to be inversely related to the Ne/ERN amplitudes. Our findings indicate that error prediction is modulated by the availability of input signals to the forward model. The observed amplitudes were found to be attenuated in comparison to previous studies, in which all efferent and sensory inputs were present. Furthermore, we assume that visual signals are weighted higher than proprioceptive signals, at least in goal-oriented tasks with visual targets.

Effects of feedback delay on learning from positive and negative feedback in patients with Parkinson's disease off medication

Phasic dopamine (DA) signals conveyed from the substantia nigra to the striatum and the prefrontal cortex crucially affect learning from feedback, with DA bursts facilitating learning from positive feedback and DA dips facilitating learning from negative feedback. Consequently, diminished nigro-striatal dopamine levels as in unmedicated patients suffering from Parkinson's Disease (PD) have been shown to lead to a negative learning bias. Recent studies suggested a diminished striatal contribution to feedback processing when the outcome of an action is temporally delayed. This study investigated whether the bias towards negative feedback learning induced by a lack of DA in PD patients OFF medication is modulated by feedback delay. To this end, PD patients OFF medication and healthy controls completed a probabilistic selection task, in which feedback was given immediately (after 800 ms) or delayed (after 6800 ms). PD patients were impaired in immediate but not delayed feedback learning. However, differences in the preference for positive/negative learning between patients and controls were seen for both learning from immediate and delayed feedback, with evidence of stronger negative learning in patients than controls. A Bayesian analysis of the data supports the conclusion that feedback timing did not affect the learning bias in the patients. These results hint at reduced, but still relevant nigro-striatal contribution to feedback learning, when feedback is delayed.

Neurophysiological mechanisms of interval timing dissociate inattentive and combined ADHD subtypes

It is far from conclusive what distinguishes the inattentive (ADD) and the combined (ADHD-C) subtype of ADHD on the neuronal level. Theoretical considerations suggest that especially interval timing processes may dissociate these subtypes from each other. Combining high-density EEG recordings with source localization analyses, we examine whether there are ADHD-subtype specific modulations of neurophysiological processes subserving interval timing in matched groups of ADD (n = 16), ADHD-C (n = 16) and controls (n = 16). Patients with ADD and ADHD-C show deficits in interval timing, which was correlated with the degree of inattention in ADD patients. Compared to healthy controls, patients with ADHD-C display a somewhat weaker, yet consistent response preparation process (contingent negative variation, CNV). In patients with ADD, the early CNV is interrupted, indicating an oscillatory disruption of the interval timing process. This is associated with activations in the supplemental motor areas and the middle frontal gyrus. Patients with ADD display adequate feedback learning mechanisms (feedback-related negativity, FRN), which is not the case in patients with ADHD-C. The results suggest that altered pacemaker-accumulation processes in medial frontal structures distinguish the ADD from the ADHD-C subtype. Particularly in patients with ADD phasic interruptions of preparatory neurophysiological processes are evident, making this a possible diagnostic feature.

Slow Is Also Fast: Feedback Delay Affects Anxiety and Outcome Evaluation

Performance-related feedback plays an important role in improving human being’s adaptive behavior. Using event-related potentials (ERPs), previous studies have associated a particular component, i.e., reward positivity (RewP), with outcome evaluation processing and found that this component was affected by waiting time before outcome evaluation. Prior research has also suggested that anxious individuals are more prone to detecting threats and susceptible to negative emotions, and show different patterns of brain activity in outcome evaluation. It is quite common that a decision-maker cannot receive feedback immediately; however, few studies have focused on the processing of delayed feedback, especially in subjects who exhibit trait anxiety. In this study, we recruited two groups of subjects with different trait anxiety levels and recorded ERPs when they conducted a time-estimation task with short (0.6–1 s) or long delayed (4–5 s) feedback. The ERP results during the cue phase showed that long waiting cues elicited more negative-going feedback-related negativity (FRN)-like component than short waiting cues in the high trait anxiety (HTA) group. More importantly, the two groups showed different patterns of ERP in the feedback condition. In the low trait anxiety (LTA) group, more positive-going RewP was found in the short-delayed than in the long-delayed condition. In contrast, no difference was found in the HTA group. This pattern may reflect the hyperactivity of the reward systems of HTA individuals in uncertain environments (e.g., the long-delay condition) compared with LTA individuals. Our results provide a direction for future research on the neural mechanisms of reinforcement learning and anxiety.

Temporal dissociation of salience and prediction error responses to appetitive and aversive taste

The feedback-related negativity (FRN), a frontocentral ERP occurring 200-350 ms after emotionally valued outcomes, has been posited as the neural correlate of reward prediction error, a key component of associative learning. Recent evidence challenged this interpretation and has led to the suggestion that this ERP expresses salience instead. Here, we distinguish between utility prediction error and salience by delivering or withholding hedonistically matched appetitive and aversive tastes, and measure ERPs to cues signaling each taste. We observed a typical FRN (computed as the loss-minus-gain difference wave) to appetitive taste, but a reverse FRN to aversive taste. When tested axiomatically, frontocentral ERPs showed a salience response across tastes, with a particularly early response to outcome delivery, supporting recent propositions of a fast, unsigned, and unspecific response to salient stimuli. ERPs also expressed aversive prediction error peaking at 285 ms, which conformed to the logic of an axiomatic model of prediction error. With stimuli that most resemble those used in animal models, we did not detect any frontocentral ERP signal for utility prediction error, in contrast with dominant views of the functional role of the FRN ERP. We link the animal and human literature and present a challenge for current perspectives on associative learning research using ERPs.