Dorsal anterior cingulate cortex integrates reinforcement history to guide voluntary behavior

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavior strategies

Both adults and children learn through feedback to associate environmental events and choices with reward, a process known as reinforcement learning (RL). However, tasks to assess RL-related neurocognitive processes in children have been limited. This study validated a child version of the Probabilistic Reward Learning task in preadolescents (8-12 years) while recording event-related-potential (ERPs), focusing on: (1) reward-feedback sensitivity (frontal Reward-related Positivity, RewP), (2) late attention-related responses to feedback (parietal P300), and (3) attentional shifting toward favored stimuli (N2pc). Behaviorally, as expected, preadolescents could learn stimulus-reward outcome associations, but with varying performance levels. Poor learners showed greater RewP amplitudes compared to good learners. Learning strategies (i.e., Win-Lose-Stay-Shift) were reflected by feedback-elicited P300 amplitudes. Lastly, attention shifted toward to-be-chosen stimuli, as evidenced by the N2pc, but not toward more highly rewarded stimuli as in adults. These findings provide novel insights into the neural processes underlying RL in preadolescents.

Dysfunctional feedback processing in male methamphetamine abusers: Evidence from neurophysiological and computational approaches

Methamphetamine use disorder (MUD) as a major public health risk is associated with dysfunctional neural feedback processing. Although dysfunctional feedback processing in people who are substance dependent has been explored in several behavioral, computational, and electrocortical studies, this mechanism in MUDs requires to be well understood. Furthermore, the current understanding of latent components of their behavior such as learning speed and exploration-exploitation dilemma is still limited. In addition, the association between the latent cognitive components and the related neural mechanisms also needs to be explored. Therefore, in this study, the underlying neurocognitive mechanisms of feedback processing of such impairment, and age/gender-matched healthy controls are evaluated within a probabilistic learning task with rewards and punishments. Mathematical modeling results based on the Q-learning paradigm suggested that MUDs show less sensitivity in distinguishing optimal options. Additionally, it may be worth noting that MUDs exhibited a slight decrease in their ability to learn from negative feedback compared to healthy controls. Also through the lens of underlying neural mechanisms, MUDs showed lower theta power at the medial-frontal areas while responding to negative feedback. However, other EEG measures of reinforcement learning including feedback-related negativity, parietal-P300, and activity flow from the medial frontal to lateral prefrontal regions, remained intact in MUDs. On the other hand, the elimination of the linkage between value sensitivity and medial-frontal theta activity in MUDs was observed. The observed dysfunction could be due to the adverse effects of methamphetamine on the cortico-striatal dopamine circuit, which is reflected in the anterior cingulate cortex activity as the most likely region responsible for efficient behavior adjustment. These findings could help us to pave the way toward tailored therapeutic approaches.

The Neural Correlates of Individual Differences in Reinforcement Learning during Pain Avoidance and Reward Seeking

Organisms learn to gain reward and avoid punishment through action-outcome associations. Reinforcement learning (RL) offers a critical framework to understand individual differences in this associative learning by assessing learning rate, action bias, pavlovian factor (i.e., the extent to which action values are influenced by stimulus values), and subjective impact of outcomes (i.e., motivation to seek reward and avoid punishment). Nevertheless, how these individual-level metrics are represented in the brain remains unclear. The current study leveraged fMRI in healthy humans and a probabilistic learning go/no-go task to characterize the neural correlates involved in learning to seek reward and avoid pain. Behaviorally, participants showed a higher learning rate during pain avoidance relative to reward seeking. Additionally, the subjective impact of outcomes was greater for reward trials and associated with lower response randomness. Our imaging findings showed that individual differences in learning rate and performance accuracy during avoidance learning were positively associated with activities of the dorsal anterior cingulate cortex, midcingulate cortex, and postcentral gyrus. In contrast, the pavlovian factor was represented in the precentral gyrus and superior frontal gyrus (SFG) during pain avoidance and reward seeking, respectively. Individual variation of the subjective impact of outcomes was positively predicted by activation of the left posterior cingulate cortex. Finally, action bias was represented by the supplementary motor area (SMA) and pre-SMA whereas the SFG played a role in restraining this action tendency. Together, these findings highlight for the first time the neural substrates of individual differences in the computational processes during RL.

Substructure of the brain's Cingulo-Opercular network

The Cingulo-Opercular network (CON) is an executive network of the human brain that regulates actions. CON is composed of many widely distributed cortical regions that are involved in top-down control over both lower-level (i.e., motor) and higher-level (i.e., cognitive) functions, as well as in processing of painful stimuli. Given the topographical and functional heterogeneity of the CON, we investigated whether subnetworks within the CON support separable aspects of action control. Using precision functional mapping (PFM) in 15 participants with > 5 hours of resting state functional connectivity (RSFC) and task data, we identified three anatomically and functionally distinct CON subnetworks within each individual. These three distinct subnetworks were linked to Decisions, Actions, and Feedback (including pain processing), respectively, in convergence with a meta-analytic task database. These Decision, Action and Feedback subnetworks represent pathways by which the brain establishes top-down goals, transforms those goals into actions, implemented as movements, and processes critical action feedback such as pain.

Where Actions Meet Outcomes: Medial Prefrontal Cortex, Central Thalamus, and the Basal Ganglia

Medial prefrontal cortex (mPFC) interacts with distributed networks that give rise to goal-directed behavior through afferent and efferent connections with multiple thalamic nuclei and recurrent basal ganglia-thalamocortical circuits. Recent studies have revealed individual roles for different thalamic nuclei: mediodorsal (MD) regulation of signaling properties in mPFC neurons, intralaminar control of cortico-basal ganglia networks, ventral medial facilitation of integrative motor function, and hippocampal functions supported by ventral midline and anterior nuclei. Large scale mapping studies have identified functionally distinct cortico-basal ganglia-thalamocortical subnetworks that provide a structural basis for understanding information processing and functional heterogeneity within the basal ganglia. Behavioral analyses comparing functional deficits produced by lesions or inactivation of specific thalamic nuclei or subregions of mPFC or the basal ganglia have elucidated the interdependent roles of these areas in adaptive goal-directed behavior. Electrophysiological recordings of mPFC neurons in rats performing delayed non-matching-to position (DNMTP) and other complex decision making tasks have revealed populations of neurons with activity related to actions and outcomes that underlie these behaviors. These include responses related to motor preparation, instrumental actions, movement, anticipation and delivery of action outcomes, memory delay, and spatial context. Comparison of results for mPFC, MD, and ventral pallidum (VP) suggest critical roles for mPFC in prospective processes that precede actions, MD for reinforcing task-relevant responses in mPFC, and VP for providing feedback about action outcomes. Synthesis of electrophysiological and behavioral results indicates that different networks connecting mPFC with thalamus and the basal ganglia are organized to support distinct functions that allow organisms to act efficiently to obtain intended outcomes.

Covariations between pupil diameter and supplementary eye field activity suggest a role in cognitive effort implementation

In both human and nonhuman primates (NHP), the medial prefrontal region, defined as the supplementary eye field (SEF), can indirectly influence behavior selection through modulation of the primary selection process in the oculomotor structures. To perform this oculomotor control, SEF integrates multiple cognitive signals such as attention, memory, reward, and error. As changes in pupil responses can assess these cognitive efforts, a better understanding of the precise dynamics by which pupil diameter and medial prefrontal cortex activity interact requires thorough investigations before, during, and after changes in pupil diameter. We tested whether SEF activity is related to pupil dynamics during a mixed pro/antisaccade oculomotor task in 2 macaque monkeys. We used functional ultrasound (fUS) imaging to examine temporal changes in brain activity at the 0.1-s time scale and 0.1-mm spatial resolution concerning behavioral performance and pupil dynamics. By combining the pupil signals and real-time imaging of NHP during cognitive tasks, we were able to infer localized cerebral blood volume (CBV) responses within a restricted part of the dorsomedial prefrontal cortex, referred to as the SEF, an area in which antisaccade preparation activity is also recorded. Inversely, SEF neurovascular activity measured by fUS imaging was found to be a robust predictor of specific variations in pupil diameter over short and long-time scales. Furthermore, we directly manipulated pupil diameter and CBV in the SEF using reward modulations. These results bring a novel understanding of the physiological links between pupil and SEF, but it also raises questions about the role of anterior cingulate cortex (ACC), as CBV variations in the ACC seems to be negligible compared to CBV variations in the SEF.

Ultrafast functional imaging reveals short- and long-term covariations between pupil diameter and activity in the Supplementary Eye Field (SEF) of awake behaving non-human primates, yielding a novel understanding of the physiological links between the pupil and SEF.

Conflict monitoring and attentional adjustment during binocular rivalry

To make sense of ambiguous and, at times, fragmentary sensory input, the brain must rely on a process of active interpretation. At any given moment, only one of several possible perceptual representations prevails in our conscious experience. Our hypothesis is that the competition between alternative representations induces a pattern of neural activation resembling cognitive conflict, eventually leading to fluctuations between different perceptual outcomes in the case of steep competition. To test this hypothesis, we probed changes in perceptual awareness between competing images using binocular rivalry. We drew our predictions from the conflict monitoring theory, which holds that cognitive control is invoked by the detection of conflict during information processing. Our results show that fronto-medial theta oscillations (5-7 Hz), an established electroencephalography (EEG) marker of conflict, increases right before perceptual alternations and decreases thereafter, suggesting that conflict monitoring occurs during perceptual competition. Furthermore, to investigate conflict resolution via attentional engagement, we looked for a neural marker of perceptual switches as by parieto-occipital alpha oscillations (8-12 Hz). The power of parieto-occipital alpha displayed an inverse pattern to that of fronto-medial theta, reflecting periods of high interocular inhibition during stable perception, and low inhibition around moments of perceptual change. Our findings aim to elucidate the relationship between conflict monitoring mechanisms and perceptual awareness.

Cortical and subcortical contributions to interference resolution and inhibition - An fMRI ALE meta-analysis

Interacting with our environment requires the selection of appropriate responses and the inhibition of others. Such effortful inhibition is achieved by a number of interference resolution and global inhibition processes. This meta-analysis including 57 studies and 73 contrasts revisits the overlap and differences in brain areas supporting interference resolution and global inhibition in cortical and subcortical brain areas. Activation likelihood estimation was used to discern the brain regions subserving each type of cognitive control. Individual contrast analysis revealed a common activation of the bilateral insula and supplementary motor areas. Subtraction analyses demonstrated the voxel-wise differences in recruitment in a number of areas including the precuneus in the interference tasks and the frontal pole and dorsal striatum in the inhibition tasks. Our results display a surprising lack of subcortical involvement within these types of cognitive control, a finding that is likely to reflect a systematic gap in the field of functional neuroimaging.

The Potential Role of Dopamine in Mediating Motor Function and Interpersonal Synchrony

Motor functions in general and motor planning in particular are crucial for our ability to synchronize our movements with those of others. To date, these co-occurring functions have been studied separately, and as yet it is unclear whether they share a common biological mechanism. Here, we synthesize disparate recent findings on motor functioning and interpersonal synchrony and propose that these two functions share a common neurobiological mechanism and adhere to the same principles of predictive coding. Critically, we describe the pivotal role of the dopaminergic system in modulating these two distinct functions. We present attention deficit hyperactivity disorder (ADHD) as an example of a disorder that involves the dopaminergic system and describe deficits in motor and interpersonal synchrony. Finally, we suggest possible directions for future studies emphasizing the role of dopamine modulation as a link between social and motor functioning.

State anxiety biases estimates of uncertainty and impairs reward learning in volatile environments

Clinical and subclinical (trait) anxiety impairs decision making and interferes with learning. Less understood are the effects of temporary anxious states on learning and decision making in healthy populations, and whether these can serve as a model for clinical anxiety. Here we test whether anxious states in healthy individuals elicit a pattern of aberrant behavioural, neural, and physiological responses comparable with those found in anxiety disorders-particularly when processing uncertainty in unstable environments. In our study, both a state anxious and a control group learned probabilistic stimulus-outcome mappings in a volatile task environment while we recorded their electrophysiological (EEG) signals. By using a hierarchical Bayesian model of inference and learning, we assessed the effect of state anxiety on Bayesian belief updating with a focus on uncertainty estimates. State anxiety was associated with an underestimation of environmental uncertainty, and informational uncertainty about the reward tendency. Anxious individuals' beliefs about reward contingencies were more precise (had smaller uncertainty) and thus more resistant to updating, ultimately leading to impaired reward-based learning. State anxiety was also associated with greater uncertainty about volatility. We interpret this pattern as evidence that state anxious individuals are less tolerant to informational uncertainty about the contingencies governing their environment and more willing to be uncertain about the level of stability of the world itself. Further, we tracked the neural representation of belief update signals in the trial-by-trial EEG amplitudes. In control participants, lower-level precision-weighted prediction errors (pwPEs) about reward tendencies were represented in the ERP signals across central and parietal electrodes peaking at 496 ms, overlapping with the late P300 in classical ERP analysis. The state anxiety group did not exhibit a significant representation of low-level pwPEs, and there were no significant differences between the groups. Smaller variance in low-level pwPE about reward tendencies in state anxiety could partially account for the null results. Expanding previous computational work on trait anxiety, our findings establish that temporary anxious states in healthy individuals impair reward-based learning in volatile environments, primarily through changes in uncertainty estimates, which play a central role in current Bayesian accounts of perceptual inference and learning.

Differentiating clinically depressed adolescents with and without active suicidality: An examination of neurophysiological and self-report measures of reward responsiveness

BACKGROUND

Suicidality in youth is a major public health problem and objective methods for identifying those at greatest risk are critically needed. Suicidality has been associated with alterations in reward-related decision-making, but the extent to which measures of reward responsiveness (RR) can differentiate youth with and without suicidality in clinical samples remains unclear.

METHODS

We examined reliable and accessible neurophysiological (i.e., reward positivity [RewP] event-related potential) and self-report (Behavioral Activation System subscales) measures of RR in relation to active suicidality in 58 clinically depressed adolescents (14- to 18-year-olds).

RESULTS

Logistic regression analysis indicated that active suicidality in depressed adolescents was associated with heightened RR at both the self-report and neurophysiological levels. A relatively more positive RewP to win and a more negative RewP to loss uniquely predicted active suicidality beyond demographic, clinical, and self-report measures.

CONCLUSIONS

Results support the utility of neurophysiological measures in differentiating clinically depressed adolescents with and without suicidality. Although depression is commonly characterized by reduced RR, depressed adolescents with active suicidality exhibited relatively enhanced neural responses to reward and loss feedback. Results highlight the need for consideration of heterogeneity in RR in depression and research on personalized depression treatment.

Communication between the Anterior Cingulate Cortex and Ventral Tegmental Area during a Cost-Benefit Reversal Task

The anterior cingulate cortex (ACC) is implicated in value-based decision making, anticipation, and adaptation; however, how ACC activity modulates these behaviors is unclear. One possibility is via the ACC's connections with the ventral tegmental area (VTA), a dopaminergic region implicated in motivation and feedback processing. We tested this by monitoring ACC and VTA local field potentials in rats performing a cost-benefit reversal task that elicited both value-based and anticipatory choices. Partial directed coherence analyses revealed that elevated 4-Hz ACC-to-VTA signaling accompanied decisions that appeared to be anticipatory. ACC-to-VTA signaling also occurred post-reversal, consistent with it being involved in the initiation of non-default behavior. An analysis of 4-Hz signals in the other direction (VTA-to-ACC) revealed that it was elevated when the rats committed errors and that this signal was followed by behavioral adaptation. Together, these findings suggest that bidirectional communication between the ACC and VTA supports behavioral flexibility.

The tenacious brain: How the anterior mid-cingulate contributes to achieving goals

Tenacity-persistence in the face of challenge-has received increasing attention, particularly because it contributes to better academic achievement, career opportunities and health outcomes. We review evidence from non-human primate neuroanatomy and structural and functional neuroimaging in humans suggesting that the anterior mid cingulate cortex (aMCC) is an important network hub in the brain that performs the cost/benefit computations necessary for tenacity. Specifically, we propose that its position as a structural and functional hub allows the aMCC to integrate signals from diverse brain systems to predict energy requirements that are needed for attention allocation, encoding of new information, and physical movement, all in the service of goal attainment. We review and integrate research findings from studies of attention, reward, memory, affect, multimodal sensory integration, and motor control to support this hypothesis. We close by discussing the implications of our framework for educational achievement, exercise and eating disorders, successful aging, and neuropsychiatric disorders such as depression and dementia.

Striatal dopamine D2 binding correlates with locus of control: Preliminary evidence from [11C]raclopride Positron Emission Tomography

The ability to exert control has been widely investigated as a hallmark of adaptive behaviour. Dopamine is recognized as the key neuromodulator mediating various control-related processes. The neural mechanisms underlying the subjective perception of being in control, or Locus of Control (LOC) are however less clear. LOC indicates the subjective tendency to attribute environmental outcomes to one's actions (internal LOC) or instead to external incontrollable factors (external LOC). Here we hypothesized that dopamine levels also relate to LOC. Previous work shows that dopamine signaling mediates learning of action-outcome relationships, outcome predictability, and opportunity cost. Prominent theories propose dopamine dysregulation as the key pathogenetic mechanism in schizophrenia and depression. Critically, external LOC is a risk factor for schizophrenia and depression, and predicts increased vulnerability to stress. However, a direct link between LOC and dopamine levels in healthy control had not been demonstrated. The purpose of our study was to investigate this link. Using [¹¹C]raclopride Positron Emission Tomography we tested the relationship between D2 receptor binding in the striatum and LOC (measured with the Rotter Locus of Control scale) in 15 healthy volunteers. Our results show a large and positive correlation: increased striatal D2 binding was associated with External LOC. This finding opens promising avenues for the study of several psychological impairments that have been associated with both dopamine and LOC, such as addiction, schizophrenia, and depression.

Blunted Social Reward Responsiveness Moderates the Effect of Lifetime Social Stress Exposure on Depressive Symptoms

Exposure to social stress is a well-established risk factor for the development and recurrence of depression. Reduced neural responsiveness to monetary reward has been associated with greater symptoms following stress exposure. However, it remains unclear whether reduced reward responsiveness serves as a mediator or moderator of the effects of stress on internalizing symptoms or whether similar patterns emerge with responses to social reward. We addressed this issue by measuring lifetime stress exposure and event-related potentials (ERPs) to social reward in 231 emerging adults (M = 18.16, SD = 0.41 years old). Participants completed the Stress and Adversity Inventory (STRAIN) to assess severity of lifetime stressors and self-report measures of current internalizing symptoms. In addition, participants completed the Island Getaway task in which the reward positivity (RewP) ERP was recorded in response to social acceptance, adjusting for responses to rejection (RewP residual). In this task, participants vote to accept or reject peers and receive reward/acceptance and rejection feedback. Stressors were divided into social and non-social stress severity scores. Analyses were conducted to test social reward responsiveness as a mediator or moderator of the effects of social and non-social stress on internalizing symptoms. Both social and non-social stress exposure over the life course predicted symptoms of depression (ps < 0.001) and social anxiety (ps < 0.002). The effect of social stress on depression was moderated by the residual RewP to social reward, adjusting for responses to social rejection (p =0.024), such that greater lifetime social stress exposure and a relatively blunted RewP to social reward were associated with greater depressive symptoms. Social reward responsiveness did not mediate effects of stress on internalizing symptoms. Reduced processing of social reward may be a vulnerability for depression that increases risk for symptoms following exposure to social stress. Blunted social reward responsiveness appears to be a relatively unique vulnerability for depression, rather than social anxiety. Results support the utility of ERP measures in measuring individual differences in social reward processing that can be applied to better understand neural processes involved in the development of depression, and highlight the importance of considering specific dimensions of stressful life experiences.

A neurophysiological measure of reward sensitivity and its association with anhedonia in psychiatrically healthy adolescents and young adults

Anhedonia (i.e., the attenuated ability to enjoy pleasurable stimuli) characterizes multiple mood disorders, but its neurophysiological underpinnings are not yet clear. Here, we measured event-related potentials in 116 adolescents and young adults engaged in an asymmetric reinforcement procedure designed to objectively characterize the anhedonic phenotype. In line with previous studies, the behavioral results showed that approximately 35% of the sample did not develop a response bias towards the more frequently rewarded stimuli (a sign of low hedonic capacity). The event-related potentials (ERPs) evoked by the reward feedback stimuli delivered during the task showed that individuals that did not develop a response bias had less cortical positivity at Fz from 224 ms to 316 ms post feedback onset compared to those that developed a response bias during the task. However, further analyses showed that this between groups difference was relatively weak, as it disappeared when we controlled for response-locked ERPs. Furthermore, the response bias observed in the asymmetric reinforcement procedure was not strongly associated with self-reported ratings of hedonic capacity. We conclude that even though the asymmetric reinforcement procedure may be used as a reward sensitivity measure in neurotypical adolescents and young adults, this task may only be able to detect clinically significant levels of anhedonia in this particular population.

Dynamics of error-related activity in deterministic learning - an EEG and fMRI study

There is a close relationship between progress in learning and the error-monitoring system. EEG and fMRI studies using probabilistic learning have revealed the distinct dynamics of this system after response and feedback, i.e. an increase of error-related and a decrease of feedback-related activity in the anterior cingulate cortex (ACC). Based on the existing theories, it can be presumed that during deterministic learning feedback-related activity in ACC would also increase. Since these assumptions have not yet been confirmed, it can be only speculated based on the data from the probabilistic paradigms how the information is being integrated within the ACC during deterministic learning. Here we implemented the Paired Associate Deterministic Learning task to the EEG and fMRI experiments. The analysis of EEG data showed a significant increase in the amplitude for both ERN and FN. Similarly, the fMRI results showed an increase in response-related and feedback-related activity of the ACC in erroneous trials. Our findings are in line with the current theories of ACC function: increasing ACC activity can be linked to the detected discrepancy between expected and obtained outcomes. We argue that expectancy violations in the course of deterministic learning are signalled by both, internal and external evaluation system.

The anatomy of apathy: A neurocognitive framework for amotivated behaviour

Apathy is a debilitating syndrome associated with many neurological disorders, including several common neurodegenerative diseases such as Parkinson's disease and Alzheimer's disease, and focal lesion syndromes such as stroke. Here, we review neuroimaging studies to identify anatomical correlates of apathy, across brain disorders. Our analysis reveals that apathy is strongly associated with disruption particularly of dorsal anterior cingulate cortex (dACC), ventral striatum (VS) and connected brain regions. Remarkably, these changes are consistent across clinical disorders and imaging modalities. Review of the neuroimaging findings allows us to develop a neurocognitive framework to consider potential mechanisms underlying apathy. According to this perspective, an interconnected group of brain regions - with dACC and VS at its core - plays a crucial role in normal motivated behaviour. Specifically we argue that motivated behaviour requires a willingness to work, to keep working, and to learn what is worth working for. We propose that deficits in any one or more of these processes can lead to the clinical syndrome of apathy, and outline specific approaches to test this hypothesis. A richer neurobiological understanding of the mechanisms underlying apathy should ultimately facilitate development of effective therapies for this disabling condition.

Internet Game Overuse Is Associated With an Alteration of Fronto-Striatal Functional Connectivity During Reward Feedback Processing

Internet gaming disorder is associated with abnormal reward processing in the reward circuit, which is known to interact with other brain regions during feedback learning. Kim et al. (1) observed that individuals with internet game overuse (IGO) exhibit altered behavior and neural activity for non-monetary reward, but not for monetary reward. Here, we extend our analysis of IGO to the functional connectivity of the reward network. Functional MRI data were obtained during a stimulus-response association learning task from 18 young males with IGO and 20 age-matched controls, where either monetary or non-monetary rewards were given as positive feedback for a correct response. Group differences in task-dependent functional connectivity were examined for the ventromedial prefrontal cortex (vmPFC) and ventral striatum (VS), which are known for reward evaluation and hedonic response processing, respectively, using a generalized form of the psychophysiological interaction approach. For non-monetary reward processing, no differences in functional connectivity were found. In contrast, for monetary reward, connectivity of the vmPFC with the left caudate nucleus was weaker for the IGO group relative to controls, while vmPFC connectivity with the right nucleus accumbens (NAcc) was elevated. The strength of vmPFC-NAcc functional connectivity appeared to be behaviorally relevant, because individuals with stronger vmPFC-NAcc connectivity showed lower learning rates for monetary reward. In addition, the IGO group showed weaker ventral striatum functional connectivity with various brain regions, including the right ventrolateral prefrontal cortex, dorsal anterior cingulate regions, and left pallidum. Thus, for monetary reward, the IGO group exhibited stronger functional connectivity within the brain regions involved in motivational salience, whereas they showed reduced functional connectivity the widely distributed brain areas involved in learning or attention. These differences in functional connectivity of reward networks, along with related behavioral impairments of reward learning, suggest that internet gaming disorder is associated with the increased incentive salience or "wanting" of addiction disorders, and may serve as the neurobiological mechanisms underlying the impaired goal-directed behavior.

Stress Elevates Frontal Midline Theta in Feedback-based Category Learning of Exceptions

Adapting behavior based on category knowledge is a fundamental cognitive function, which can be achieved via different learning strategies relying on different systems in the brain. Whereas the learning of typical category members has been linked to implicit, prototype abstraction learning, which relies predominantly on prefrontal areas, the learning of exceptions is associated with explicit, exemplar-based learning, which has been linked to the hippocampus. Stress is known to foster implicit learning strategies at the expense of explicit learning. Procedural, prefrontal learning and cognitive control processes are reflected in frontal midline theta (4–8 Hz) oscillations during feedback processing. In the current study, we examined the effect of acute stress on feedback-based category learning of typical category members and exceptions and the oscillatory correlates of feedback processing in the EEG. A computational modeling procedure was applied to estimate the use of abstraction and exemplar strategies during category learning. We tested healthy, male participants who underwent either the socially evaluated cold pressor test or a nonstressful control procedure before they learned to categorize typical members and exceptions based on feedback. The groups did not differ significantly in their categorization accuracy or use of categorization strategies. In the EEG, however, stressed participants revealed elevated theta power specifically during the learning of exceptions, whereas the theta power during the learning of typical members did not differ between the groups. Elevated frontal theta power may reflect an increased involvement of medial prefrontal areas in the learning of exceptions under stress.

Effects of feedback delay on learning from positive and negative feedback in patients with Parkinson's disease off medication

Phasic dopamine (DA) signals conveyed from the substantia nigra to the striatum and the prefrontal cortex crucially affect learning from feedback, with DA bursts facilitating learning from positive feedback and DA dips facilitating learning from negative feedback. Consequently, diminished nigro-striatal dopamine levels as in unmedicated patients suffering from Parkinson's Disease (PD) have been shown to lead to a negative learning bias. Recent studies suggested a diminished striatal contribution to feedback processing when the outcome of an action is temporally delayed. This study investigated whether the bias towards negative feedback learning induced by a lack of DA in PD patients OFF medication is modulated by feedback delay. To this end, PD patients OFF medication and healthy controls completed a probabilistic selection task, in which feedback was given immediately (after 800 ms) or delayed (after 6800 ms). PD patients were impaired in immediate but not delayed feedback learning. However, differences in the preference for positive/negative learning between patients and controls were seen for both learning from immediate and delayed feedback, with evidence of stronger negative learning in patients than controls. A Bayesian analysis of the data supports the conclusion that feedback timing did not affect the learning bias in the patients. These results hint at reduced, but still relevant nigro-striatal contribution to feedback learning, when feedback is delayed.

Social and asocial prefrontal cortex neurons: a new look at social facilitation and the social brain

A fundamental aspect of behavior in many animal species is 'social facilitation', the positive effect of the mere presence of conspecifics on performance. To date, the neuronal counterpart of this ubiquitous phenomenon is unknown. We recorded the activity of single neurons from two prefrontal cortex regions, the dorsolateral part and the anterior cingulate cortex in monkeys as they performed a visuomotor task, either in the presence of a conspecific (Presence condition) or alone. Monkeys performed better in the presence condition than alone (social facilitation), and analyses of outcome-related activity of 342 prefrontal neurons revealed that most of them (86%) were sensitive to the performance context. Two populations of neurons were discovered: 'social neurons', preferentially active under social presence and 'asocial neurons', preferentially active under social isolation. The activity of these neurons correlated positively with performance only in their preferred context (social neurons under social presence; asocial neurons under social isolation), thereby providing a potential neuronal mechanism of social facilitation. More generally, the fact that identical tasks recruited either social or asocial neurons depending on the presence or absence of a conspecific also brings a new look at the social brain hypothesis.

Alcohol hangover impacts learning and reward processing within the medial-frontal cortex

It is common knowledge that alcohol intoxication impairs motor coordination, judgment, and decision making. Indeed, an abundance of literature links intoxication to impaired cognitive control that leads to accidents and injury. A broadening body of research, however, suggests that the impact of alcohol may continue beyond the point of intoxication and into the period of alcohol hangover. Here, we examined differences in the amplitude of reward positivity-a component of the human ERP associated with learning-between control and hangover participants. During performance of a learnable gambling task, we found a reduction in the reward positivity during alcohol hangover. Additionally, participants experiencing alcohol hangover demonstrated reduced performance in the experimental task in comparison to their nonhangover counterparts. Our results suggest that the neural systems that underlie performance monitoring and reward-based learning are impaired during alcohol hangover.

Differential effects of real versus hypothetical monetary reward magnitude on risk-taking behavior and brain activity

Human decisions are more easily affected by a larger amount of money than a smaller one. Although numerous studies have used hypothetical money as incentives to motivate human behavior, the validity of hypothetical versus real monetary rewards remains controversial. In the present study, we used event-related potential (ERP) with the balloon analogue risk task to investigate how magnitudes of real and hypothetical monetary rewards modulate risk-taking behavior and feedback-related negativity (FRN). Behavioral data showed that participants were more risk averse after negative feedback with increased magnitude of real monetary rewards, while no behavior differences were observed between large and small hypothetical monetary rewards. Similarly, ERP data showed a larger FRN in response to negative feedback during risk taking with large compared to small real monetary rewards, while no FRN differences were observed between large and small hypothetical monetary rewards. Moreover, FRN amplitude differences correlated with risk-taking behavior changes from small to large real monetary rewards, while such correlation was not observed for hypothetical monetary rewards. These findings suggest that the magnitudes of real and hypothetical monetary rewards have differential effects on risk-taking behavior and brain activity. Real and hypothetical money incentives may have different validity for modulating human decisions.

Slow Is Also Fast: Feedback Delay Affects Anxiety and Outcome Evaluation

Performance-related feedback plays an important role in improving human being’s adaptive behavior. Using event-related potentials (ERPs), previous studies have associated a particular component, i.e., reward positivity (RewP), with outcome evaluation processing and found that this component was affected by waiting time before outcome evaluation. Prior research has also suggested that anxious individuals are more prone to detecting threats and susceptible to negative emotions, and show different patterns of brain activity in outcome evaluation. It is quite common that a decision-maker cannot receive feedback immediately; however, few studies have focused on the processing of delayed feedback, especially in subjects who exhibit trait anxiety. In this study, we recruited two groups of subjects with different trait anxiety levels and recorded ERPs when they conducted a time-estimation task with short (0.6–1 s) or long delayed (4–5 s) feedback. The ERP results during the cue phase showed that long waiting cues elicited more negative-going feedback-related negativity (FRN)-like component than short waiting cues in the high trait anxiety (HTA) group. More importantly, the two groups showed different patterns of ERP in the feedback condition. In the low trait anxiety (LTA) group, more positive-going RewP was found in the short-delayed than in the long-delayed condition. In contrast, no difference was found in the HTA group. This pattern may reflect the hyperactivity of the reward systems of HTA individuals in uncertain environments (e.g., the long-delay condition) compared with LTA individuals. Our results provide a direction for future research on the neural mechanisms of reinforcement learning and anxiety.

Intranasal oxytocin, but not vasopressin, augments neural responses to toddlers in human fathers

This study investigates paternal brain function with the hope of better understanding the neural basis for variation in caregiving involvement among men. The neuropeptides oxytocin (OT) and vasopressin (AVP) are implicated in paternal caregiving in humans and other species. In a double-blind, placebo-controlled, within-subject pharmaco-functional MRI experiment, we randomized 30 fathers of 1-2year old children to receive either 24IU intranasal OT before one scan and placebo before the other scan (n=15) or 20IU intranasal AVP before one scan and placebo before the other scan (n=15). Brain function was measured with fMRI as the fathers viewed pictures of their children, unknown children and unknown adults, and as they listened to unknown infant cry stimuli. Intranasal OT, but not AVP, significantly increased the BOLD fMRI response to viewing pictures of own children within the caudate nucleus, a target of midbrain dopamine projections, as well as the dorsal anterior cingulate (dACC) and visual cortex, suggesting that intranasal oxytocin augments activation in brain regions involved in reward, empathy and attention in human fathers. OT effects also varied as a function of order of administration such that when OT was given before placebo, it increased activation within several reward-related structures (substantia nigra, ventral tegmental area, putamen) more than when it was given after placebo. Neither OT nor AVP had significant main effects on the neural response to cries. Our findings suggest that the hormonal changes associated with the transition to fatherhood are likely to facilitate increased approach motivation and empathy for children, and call for future research that evaluates the potential of OT to normalize deficits in paternal motivation, as might be found among men suffering from post-partum depression.

Computational Models of Anterior Cingulate Cortex: At the Crossroads between Prediction and Effort

In the last two decades the anterior cingulate cortex (ACC) has become one of the most investigated areas of the brain. Extensive neuroimaging evidence suggests countless functions for this region, ranging from conflict and error coding, to social cognition, pain and effortful control. In response to this burgeoning amount of data, a proliferation of computational models has tried to characterize the neurocognitive architecture of ACC. Early seminal models provided a computational explanation for a relatively circumscribed set of empirical findings, mainly accounting for EEG and fMRI evidence. More recent models have focused on ACC's contribution to effortful control. In parallel to these developments, several proposals attempted to explain within a single computational framework a wider variety of empirical findings that span different cognitive processes and experimental modalities. Here we critically evaluate these modeling attempts, highlighting the continued need to reconcile the array of disparate ACC observations within a coherent, unifying framework.

Performance monitoring in nicotine dependence: Considering integration of recent reinforcement history

INTRODUCTION

Impaired monitoring of errors and conflict (performance monitoring; PM) is well documented in substance dependence (SD) including nicotine dependence and may contribute to continued drug use. Contemporary models of PM and complementary behavioural evidence suggest that PM works by integrating recent reinforcement history rather than evaluating individual behaviours. Despite this, studies of PM in SD have typically used indices derived from reaction to task error or conflict on individual trials. Consequently impaired integration of reinforcement history during action selection tasks requiring behavioural control in SD populations has been underexplored.

METHODS

A reinforcement learning task assessed the ability of abstinent, satiated, former and never smokers (N=60) to integrate recent reinforcement history alongside a more typical behavioural index of PM reflecting the degree of reaction time slowing following an error (post-punishment slowing; PPS).

RESULTS

On both indices there was a consistent pattern in PM data: Former smokers had the greatest and satiated smokers the poorest PM. Specifically satiated smokers had poorer reinforcement integration than former (p=0.005) and never smokers (p=0.041) and had less post-punishment slowing than former (p<0.001), never (p=0.003) and abstinent smokers (p=0.026).

CONCLUSIONS

These are the first data examining the effects of smoking status on PM that use an integration of reinforcement history metric. The concordance of the reinforcement integration and PPS data suggest that this could be a promising method to interrogate PM in future studies. PM is influenced by smoking status. As PM is associated with adapting behaviour, poor PM in satiated smokers may contribute towards continued smoking despite negative consequences. Former smokers show elevated PM suggesting this may be a good relapse prevention target for individuals struggling to remain abstinent however prospective and intervention studies are needed. A better understanding of PM deficits in terms of reinforcement integration failure may stimulate development of novel treatment approaches.

Anxious Individuals Are Impulsive Decision-Makers in the Delay Discounting Task: An ERP Study

Impulsivity, which is linked to a wide range of psychiatric disorders, is often characterized by a preference for immediate but smaller rewards over delayed but larger rewards. However, debate exists on the relationship between anxiety and impulsivity. Here we use event-related potential (ERP) components as biomarkers in the temporal discounting task to examine the effect of anxiety on inter-temporal decision-making. Our behavioral results indicated that the high trait anxiety (HTA) group made significantly more immediate choices than the low trait anxiety (LTA) group. Compared with the LTA group, shorter response time was associated with immediate rewards in the HTA group. Furthermore, previous studies have demonstrated three ERP components that are associated with impulsivity and/or delay discounting. First, the N1 is an early sensory component involved in selective attention and attention processing for goal-directed actions. Second, the reward positivity (RewP) reflects reward-related dopaminergic activity and encodes reward values. Third, the P3 is regarded as a measure of motivational significance in the decision-making literature. Accordingly, this study found in the immediate-option-evoked ERPs that the HTA group had a larger N1 than the LTA group did. For the delayed-option-evoked ERPs, the HTA group had larger N1 and RewP for the immediate choice than the LTA group did, while the LTA group had a larger P3 for the delayed choice than the HTA group did. These results support the notion that anxiety individuals are impulsive decision-makers in the Delay Discounting Task.

Behavioral and Electrophysiological Alterations for Reinforcement Learning in Manic and Euthymic Patients with Bipolar Disorder

AIMS

Bipolar disorder is characterized by behavioral changes such as risk-taking and increasing goal-directed activities, which may result from altered reward processing. Patients with bipolar disorder show impaired reward learning in situations that require the integration of reinforced feedback over time. In this study, we examined the behavioral and electrophysiological characteristics of reward learning in manic and euthymic patients with bipolar disorder using a probabilistic reward task.

METHODS

Twenty-four manic and 20 euthymic patients with bipolar I disorder and 24 healthy control subjects performed the probabilistic reward task. We assessed response bias (RB) as a preference for the stimulus paired with the more frequent reward and feedback-related negativity (FRN) to correct identification of the rich stimulus.

RESULTS

Both manic and euthymic patients showed significantly lower RB scores in the early learning stage (block 1) in comparison with the late learning stage (block 2 or block 3) of the task, as well as significantly lower RB scores in the early stage compared to healthy subjects. Relatively more negative FRN amplitude is elicited by no presentation of an expected reward, compared to that elicited by presentation of expected feedback. The FRN became significantly more negative from the early (block 1) to the later stages (blocks 2 and 3) in both manic and euthymic patients, but not in healthy subjects. Changes in RB scores and FRN amplitudes between blocks 2 and 3 and block 1 correlated positively in healthy controls, but correlated negatively in manic and euthymic patients. The severity of manic symptoms correlated positively with reward learning scores and negatively with the FRN.

CONCLUSIONS

These findings suggest that patients with bipolar disorder during euthymic or manic states have behavioral and electrophysiological alterations in reward learning compared to healthy subjects. This dysfunctional reward processing may be related to the abnormal decision-making or altered goal-directed activities frequently seen in patients with bipolar disorder.

Social learning through prediction error in the brain

Learning about the world is critical to survival and success. In social animals, learning about others is a necessary component of navigating the social world, ultimately contributing to increasing evolutionary fitness. How humans and nonhuman animals represent the internal states and experiences of others has long been a subject of intense interest in the developmental psychology tradition, and, more recently, in studies of learning and decision making involving self and other. In this review, we explore how psychology conceptualizes the process of representing others, and how neuroscience has uncovered correlates of reinforcement learning signals to explore the neural mechanisms underlying social learning from the perspective of representing reward-related information about self and other. In particular, we discuss self-referenced and other-referenced types of reward prediction errors across multiple brain structures that effectively allow reinforcement learning algorithms to mediate social learning. Prediction-based computational principles in the brain may be strikingly conserved between self-referenced and other-referenced information.

Endophenotype best practices

This review examines the current state of electrophysiological endophenotype research and recommends best practices that are based on knowledge gleaned from the last decade of molecular genetic research with complex traits. Endophenotype research is being oversold for its potential to help discover psychopathology relevant genes using the types of small samples feasible for electrophysiological research. This is largely because the genetic architecture of endophenotypes appears to be very much like that of behavioral traits and disorders: they are complex, influenced by many variants (e.g., tens of thousands) within many genes, each contributing a very small effect. Out of over 40 electrophysiological endophenotypes covered by our review, only resting heart, a measure that has received scant advocacy as an endophenotype, emerges as an electrophysiological variable with verified associations with molecular genetic variants. To move the field forward, investigations designed to discover novel variants associated with endophenotypes will need extremely large samples best obtained by forming consortia and sharing data obtained from genome wide arrays. In addition, endophenotype research can benefit from successful molecular genetic studies of psychopathology by examining the degree to which these verified psychopathology-relevant variants are also associated with an endophenotype, and by using knowledge about the functional significance of these variants to generate new endophenotypes. Even without molecular genetic associations, endophenotypes still have value in studying the development of disorders in unaffected individuals at high genetic risk, constructing animal models, and gaining insight into neural mechanisms that are relevant to clinical disorder.

'Proactive' use of cue-context congruence for building reinforcement learning's reward function

BACKGROUND

Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model.

RESULTS

In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively.

CONCLUSIONS

Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.

Action Experience and Action Discovery in Medicated Individuals with Parkinson's Disease

Parkinson's disease (PD) is a neurodegenerative disorder that markedly affects voluntary action. While regular dopamine treatment can help restore motor function, dopamine also influences cognitive portions of the action system. Previous studies have demonstrated that dopamine medication boosts action-effect associations, which are crucial for the discovery of new voluntary actions. In the present study, we investigated whether neural processes involved in the discovery of new actions are altered in PD participants on regular dopamine treatment, compared to healthy age-matched controls. We recorded brain electroencephalography (EEG) activity while PD patients and age-matched controls performed action discovery (AD) and action control tasks. We found that the novelty P3, a component normally present when there is uncertainty about the occurrence of the sensory effect, was enhanced in PD patients. However, AD was maintained in PD patients, and the novelty P3 demonstrated normal learning-related reductions. Crucially, we found that in PD patients the causal association between an action and its resulting sensory outcome did not modulate the amplitude of the feedback correct-related positivity (fCRP), an EEG component sensitive to the association between an action and its resulting effect. Collectively, these preliminary results suggest that the formation of long-term action-outcome representations may be maintained in PD patients on regular dopamine treatment, but the initial experience of action-effect association may be affected.

Transdiagnostic Psychiatric Symptoms and Event-Related Potentials following Rewarding and Aversive Outcomes

There is a need for a better understanding of transdiagnostic psychiatric symptoms that relate to neurophysiological abnormalities following rewarding and aversive feedback in order to inform development of novel targeted treatments. To address this need, we examined a transdiagnostic sample of 44 adults (mean age: 35.52; 57% female), which consisted of individuals with broadly-defined schizophrenia-spectrum disorders (n = 16), bipolar disorders (n = 10), other mood and anxiety disorders (n = 5), and no history of a psychiatric disorder (n = 13). Participants completed a Pavlovian monetary reward prediction task during 32-channel electroencephalogram recording. We assessed the event-related potentials (ERPs) of feedback-related negativity (FRN), feedback-related positivity (FRP), and the late positive potential (LPP), following better and worse than expected outcomes. Examination of symptom relationships using stepwise regressions across the entire sample revealed that an increase in the clinician-rated Negative Symptoms factor score from the Positive and Negative Syndrome Scale, was related to a decreased LPP amplitude during better than expected (i.e., rewarding) outcomes. We also found that increased self-reported scores on the Schizotypal Personality Questionnaire (Brief-Revised) Disorganized factor related to an increased FRN amplitude during worse than expected (i.e., aversive) outcomes. Across the entire sample, the FRP component amplitudes did not show significant relationships to any of the symptoms examined. Analyses of the three diagnostic groups of schizophrenia-spectrum disorders, bipolar disorders, and nonpsychiatric controls did not reveal any statistically significant differences across the ERP amplitudes and conditions. These findings suggest relationships between specific neurophysiological abnormalities following rewarding and aversive outcomes and particular transdiagnostic psychiatric symptoms.

The Effects of Age on the Neural Correlates of Recollection Success, Recollection-Related Cortical Reinstatement, and Post-Retrieval Monitoring

Functional magnetic resonance imaging (fMRI) was used to investigate whether age-related differences in episodic memory performance are accompanied by a reduction in the specificity of recollected information. We addressed this question by comparing recollection-related cortical reinstatement in young and older adults. At study, subjects viewed objects and concrete words, making 1 of 2 different semantic judgments depending on the study material. Test items were words that corresponded to studied words or the names of studied objects. Subjects indicated whether each test item was recollected, familiar, or novel. Reinstatement of information differentiating the encoding tasks was quantified both with a univariate analysis of the fMRI signal and with a multivoxel pattern analysis, using a classifier that had been trained to discriminate between the 2 classes of study episode. The results of these analyses converged to suggest that reinstatement did not differ according to age. Thus, there was no evidence that specificity of recollected information was reduced in older individuals. Additionally, there were no age effects in the magnitude of recollection-related modulations in regional activity or in the neural correlates of post-retrieval monitoring. Taken together, the findings suggest that the neural mechanisms engaged during successful episodic retrieval can remain stable with advancing age.

Habenula responses to potential and actual loss in major depression: preliminary evidence for lateralized dysfunction

The habenula has been implicated in predicting negative events and in responding to unexpected negative outcomes. Animal models of depression have supported the hypothesis that perturbations in habenula activity contribute to the pathophysiology of Major Depressive Disorder (MDD), a psychiatric illness characterized by abnormalities in responding to negative feedback and by pessimism in evaluating the likelihood of future events. No research to date, however, has examined human habenula responses to potential and experienced negative outcomes in MDD. In this study, depressed and healthy control participants performed a probabilistic guessing task for monetary rewards and penalties during high-resolution functional magnetic resonance imaging of the habenula. In healthy adults, we observed a pattern of habenula activation consistent with its hypothesized role in predicting future losses and responding to suboptimal outcomes. In contrast, in depressed participants the left habenula was not activated significantly during the prediction or experience of monetary penalty. Complementing this group difference, attenuated habenula activation to negative feedback in control participants was associated with levels of shame and rumination. The results of this study suggest that depressed individuals are characterized by dysfunction in a neural system involved in generating expectations and comparing expectations with objective outcomes.

Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.

An event-related source localization study of response monitoring and social impairments in autism spectrum disorder

A number of studies suggest anterior cingulate cortex (ACC) abnormalities in autism spectrum disorder (ASD), which might underlie response monitoring and social impairments exhibited by children and adolescents with ASD. The goal of the present study was to extend this work by examining error and correct response monitoring using event-related potentials (ERN, Pe, CRN) and LORETA source localization in high functioning adults with ASD and controls. Adults with ASD showed reduced ERN and Pe amplitudes and reduced rostral ACC activation compared with controls. Adults with ASD also showed less differentiation between error and correct ERP components. Social impairments and higher overall autism symptoms were related to reduced rostral ACC activity at the time of the ERN, particularly in adults with ASD. These findings suggest that reduced ACC activity may reflect a putative brain mechanism involved in the origins and maintenance of social impairments and raise the possibility of the presence of stable brain-behavior relation impairment across development in some individuals with ASD.

The Effects of Acute Dopamine Precursor Depletion on the Cognitive Control Functions of Performance Monitoring and Conflict Processing: An Event-Related Potential (ERP) Study

Studies using medications and psychiatric populations implicate dopamine in cognitive control and performance monitoring processes. However, side effects associated with medication or studying psychiatric groups may confound the relationship between dopamine and cognitive control. To circumvent such possibilities, we utilized a randomized, double-blind, placebo-controlled, within-subjects design wherein participants were administered a nutritionally-balanced amino acid mixture (BAL) and an amino acid mixture deficient in the dopamine precursors tyrosine (TYR) and phenylalanine (PHE) on two separate occasions. Order of sessions was randomly assigned. Cognitive control and performance monitoring were assessed using response times (RT), error rates, the N450, an event-related potential (ERP) index of conflict monitoring, the conflict slow potential (conflict SP), an ERP index of conflict resolution, and the error-related negativity (ERN) and error positivity (Pe), ERPs associated with performance monitoring. Participants were twelve males who completed a Stroop color-word task while ERPs were collected four hours following acute PHE and TYR depletion (APTD) or balanced (BAL) mixture ingestion in two separate sessions. N450 and conflict SP ERP amplitudes significantly differentiated congruent from incongruent trials, but did not differ as a function of APTD or BAL mixture ingestion. Similarly, ERN and Pe amplitudes showed significant differences between error and correct trials that were not different between APTD and BAL conditions. Findings indicate that acute dopamine precursor depletion does not significantly alter cognitive control and performance monitoring ERPs. Current results do not preclude the role of dopamine in these processes, but suggest that multiple methods for dopamine-related hypothesis testing are needed.

A new computational account of cognitive control over reinforcement-based decision-making: Modeling of a probabilistic learning task

Recent work on decision-making field offers an account of dual-system theory for decision-making process. This theory holds that this process is conducted by two main controllers: a goal-directed system and a habitual system. In the reinforcement learning (RL) domain, the habitual behaviors are connected with model-free methods, in which appropriate actions are learned through trial-and-error experiences. However, goal-directed behaviors are associated with model-based methods of RL, in which actions are selected using a model of the environment. Studies on cognitive control also suggest that during processes like decision-making, some cortical and subcortical structures work in concert to monitor the consequences of decisions and to adjust control according to current task demands. Here a computational model is presented based on dual system theory and cognitive control perspective of decision-making. The proposed model is used to simulate human performance on a variant of probabilistic learning task. The basic proposal is that the brain implements a dual controller, while an accompanying monitoring system detects some kinds of conflict including a hypothetical cost-conflict one. The simulation results address existing theories about two event-related potentials, namely error related negativity (ERN) and feedback related negativity (FRN), and explore the best account of them. Based on the results, some testable predictions are also presented.

Using brain potentials to understand prism adaptation: the error-related negativity and the P300

Prism adaptation (PA) is both a perceptual-motor learning task as well as a promising rehabilitation tool for visuo-spatial neglect (VSN)—a spatial attention disorder often experienced after stroke resulting in slowed and/or inaccurate motor responses to contralesional targets. During PA, individuals are exposed to prism-induced shifts of the visual-field while performing a visuo-guided reaching task. After adaptation, with goggles removed, visuomotor responding is shifted to the opposite direction of that initially induced by the prisms. This visuomotor aftereffect has been used to study visuomotor learning and adaptation and has been applied clinically to reduce VSN severity by improving motor responding to stimuli in contralesional (usually left-sided) space. In order to optimize PA's use for VSN patients, it is important to elucidate the neural and cognitive processes that alter visuomotor function during PA. In the present study, healthy young adults underwent PA while event-related potentials (ERPs) were recorded at the termination of each reach (screen-touch), then binned according to accuracy (hit vs. miss) and phase of exposure block (early, middle, late). Results show that two ERP components were evoked by screen-touch: an error-related negativity (ERN), and a P300. The ERN was consistently evoked on miss trials during adaptation, while the P300 amplitude was largest during the early phase of adaptation for both hit and miss trials. This study provides evidence of two neural signals sensitive to visual feedback during PA that may sub-serve changes in visuomotor responding. Prior ERP research suggests that the ERN reflects an error processing system in medial-frontal cortex, while the P300 is suggested to reflect a system for context updating and learning. Future research is needed to elucidate the role of these ERP components in improving visuomotor responses among individuals with VSN.