Feedback-related EEG dynamics separately reflect decision parameters, biases, and future choices

The cerebellum contributes to prediction error coding in reinforcement learning in humans

Recent rodent data suggest that the cerebellum - a region typically associated with processing sensory prediction errors (PEs) - also processes PEs in reinforcement learning (RL-PEs; i.e., learning from action outcomes). We tested whether cerebellar output is necessary for RL-PE processing in regions more traditionally associated with action-outcome processing, such as striatum and anterior cingulate cortex. The feedback-related negativity (FRN) was measured as a proxy of cerebral RL-PE processing in a probabilistic feedback learning task using electroencephalography. Two complementary experiments were performed in humans. First, patients with chronic cerebellar stroke (20 male, 6 female) and matched healthy controls (19 male, 7 female) were tested. Second, single-pulse cerebellar transcranial magnetic stimulation (TMS) was applied in healthy participants (7 male, 17 female), thus implementing a virtual lesion approach. Consistent with previous studies, learning of action-outcome associations was intact with only minor changes in behavioural flexibility. Importantly, no significant RL-PE processing was observed in the FRN in patients with cerebellar stroke, and in participants receiving cerebellar TMS. Findings in both experiments show that RL-PE processing in the forebrain depends on cerebellar output in humans, complementing and extending previous findings in rodents.Significance statement While processing of prediction errors in reinforcement learning (RL-PEs) is usually attributed to midbrain and forebrain, recent rodent studies have recorded RL-PE signals in the cerebellum. It is not yet clear whether these cerebellar RL-PE signals contribute to RL-PE processing in the forebrain/midbrain. In the current study, we could show that forebrain RL-PE coding is blunted when the cerebellum is affected across two complementary lesion models (patients with cerebellar stroke, cerebellar TMS). Our results support direct involvement of the cerebellum in RL-PE processing. We can further show that the cerebellum is necessary for RL-PE coding in the forebrain.

Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance

The ability to calibrate learning according to new information is a fundamental component of an organism's ability to adapt to changing conditions. Yet, the exact neural mechanisms guiding dynamic learning rate adjustments remain unclear. Catecholamines appear to play a critical role in adjusting the degree to which we use new information over time, but individuals vary widely in the manner in which they adjust to changes. Here, we studied the effects of a low dose of methamphetamine (MA), and individual differences in these effects, on probabilistic reversal learning dynamics in a within-subject, double-blind, randomized design. Participants first completed a reversal learning task during a drug-free baseline session to provide a measure of baseline performance. Then they completed the task during two sessions, one with MA (20 mg oral) and one with placebo (PL). First, we showed that, relative to PL, MA modulates the ability to dynamically adjust learning from prediction errors. Second, this effect was more pronounced in participants who performed moderately low at baseline. These results present novel evidence for the involvement of catecholaminergic transmission on learning flexibility and highlights that baseline performance modulates the effect of the drug.

Electrophysiological signatures of the effect of context on exploration: Greater attentional and learning signals when exploration is costly

Humans are excellent at modifying our behaviour depending on context. For example, we will change how we explore when losses are possible compared to when losses are not possible. However, it remains unclear what specific cognitive and neural processes are modulated when exploring in different contexts. Here, we had participants learn within two different contexts: in one the participants could lose points while in the other the participants could not. Our goal was to determine how the inclusion of losses impacted human exploratory behaviour (experiment one), and whether we could explain the neural basis of these effects using EEG (experiment two). In experiment one, we found that participants preferred less-variable choices and explored less often when losses were possible. In addition, computational modelling revealed that participants engaged in less random exploration, had a lower rate of learning, and showed lower choice stickiness when losses were possible. In experiment two, we replicated these effects while examining a series of neural signals involved in exploration. During exploration, signals tied to working memory and learning (P3b), attention orienting (P3a) and motivation (late positive potential; an exploratory analysis) were enhanced when losses were possible. These neural differences contribute to why exploratory behaviour is changed by different learning contexts and can be explained by the theoretical claim that losses recruit attention and lead to increased task focus. These results provide insight into the cognitive processes that underlie exploration, and how exploratory behaviour changes across contexts.

Beyond peaks and troughs: Multiplexed performance monitoring signals in the EEG

With the discovery of event-related potentials elicited by errors more than 30 years ago, a new avenue of research on performance monitoring, cognitive control, and decision making emerged. Since then, the field has developed and expanded fulminantly. After a brief overview on the EEG correlates of performance monitoring, this article reviews recent advancements based on single-trial analyses using independent component analysis, multiple regression, and multivariate pattern classification. Given the close interconnection between performance monitoring and reinforcement learning, computational modeling and model-based EEG analyses have made a particularly strong impact. The reviewed findings demonstrate that error- and feedback-related EEG dynamics represent variables reflecting how performance-monitoring signals are weighted and transformed into an adaptation signal that guides future decisions and actions. The model-based single-trial analysis approach goes far beyond conventional peak-and-trough analyses of event-related potentials and enables testing mechanistic theories of performance monitoring, cognitive control, and decision making.

An EEG Dataset of Neural Signatures in a Competitive Two-Player Game Encouraging Deceptive Behavior

Studying deception is vital for understanding decision-making and social dynamics. Recent EEG research has deepened insights into the brain mechanisms behind deception. Standard methods in this field often rely on memory, are vulnerable to countermeasures, yield false positives, and lack real-world relevance. Here, we present a comprehensive dataset from an EEG-monitored competitive, two-player card game designed to elicit authentic deception behavior. Our extensive dataset contains EEG data from 12 pairs (N = 24 participants with role switching), controlled for age, gender, and risk-taking, with detailed labels and annotations. The dataset combines standard event-related potential and microstate analyses with state-of-the-art decoding approaches of four scenarios: spontaneous/instructed truth-telling and lying. This demonstrates game-based methods' efficacy in studying deception and sets a benchmark for future research. Overall, our dataset represents a unique resource with applications in cognitive neuroscience and related fields for studying deception, competitive behavior, decision-making, inter-brain synchrony, and benchmarking of decoding frameworks in a difficult, high-level cognitive task.

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits.

Unraveling the influence of trial-based motivational changes on performance monitoring stages in a flanker task

Performance monitoring (PM) is a vital component of adaptive behavior and known to be influenced by motivation. We examined effects of potential gain (PG) and loss avoidance (LA) on neural correlates of PM at different processing stages, using a task with trial-based changes in these motivational contexts. Findings suggest more attention is allocated to the PG context, with higher amplitudes for respective correlates of stimulus and feedback processing. The PG context favored rapid responses, while the LA context emphasized accurate responses. Lower response thresholds in the PG context after correct responses derived from a drift-diffusion model also indicate a more approach-oriented response style in the PG context. This cognitive shift is mirrored in neural correlates: negative feedback in the PG context elicited a higher feedback-related negativity (FRN) and higher theta power, whereas positive feedback in the LA context elicited higher P3a and P3b amplitudes, as well as higher theta power. There was no effect of motivational context on response-locked brain activity. Given the similar frequency of negative feedback in both contexts, the elevated FRN and theta power in PG trials cannot be attributed to variations in reward prediction error. The observed variations in the FRN indicate that the effect of outcome valence is modulated by motivational salience.