Action-specific value signals in reward-related regions of the human brain

Reassessing the Neural Correlates of Social Exclusion: A Replication Study of the Cyberball Paradigm Using Arterial Spin Labeling

BACKGROUND/OBJECTIVES

The cyberball paradigm has been used in numerous neuroimaging studies to elicit activation in neural substrates of social exclusion, which have been interpreted in terms of activity associated with "social pain". The objectives of the study were to assess not only the replicability but also the specificity of the areas activated by this paradigm.

METHODS

Functional imaging with arterial spin labeling, an approach to image longer mental states.

RESULTS

We replicated findings of previous meta-analyses of this paradigm in the inferior frontal gyrus and ventral cingular cortex. However, these areas were also active in a watch condition (in which participants were not excluded), although less so.

CONCLUSIONS

These findings relativize a simple and specific interpretation of these areas as the neural substrates of social exclusion and social pain, as in previous studies. In a broader experimental context, similar activations have been reported by neuroimaging studies when semantic disambiguation and evaluation of action goals are required, an interpretation that may also apply to the effects elicited by this paradigm.

Functional Magnetic Resonance Imaging Signatures of Pavlovian and Instrumental Valuation Systems during a Modified Orthogonalized Go/No-go Task

Motivational (i.e., Pavlovian) values interfere with instrumental responding and can lead to suboptimal decision-making. In humans, task-based neuroimaging studies have only recently started illuminating the functional neuroanatomy of Pavlovian biasing of instrumental control. To provide a mechanistic understanding of the neural dynamics underlying the Pavlovian and instrumental valuation systems, analysis of neuroimaging data has been informed by computational modeling of conditioned behavior. Nonetheless, because of collinearities in Pavlovian and instrumental predictions, previous research failed to tease out hemodynamic activity that is parametrically and dynamically modulated by coexistent Pavlovian and instrumental value expectations. Moreover, neural correlates of Pavlovian to instrumental transfer effects have so far only been identified in extinction (i.e., in the absence of learning). In this study, we devised a modified version of the orthogonalized go/no-go paradigm, which introduced Pavlovian-only catch trials to better disambiguate trial-by-trial Pavlovian and instrumental predictions in both sexes. We found that hemodynamic activity in the ventromedial pFC covaried uniquely with the model-derived Pavlovian value expectations. Notably, modulation of neural activity encoding for instrumental predictions in the supplementary motor cortex was linked to successful action selection in conflict conditions. Furthermore, hemodynamic activity in regions pertaining to the limbic system and medial pFC was correlated with synergistic Pavlovian and instrumental predictions and improved conditioned behavior during congruent trials. Altogether, our results provide new insights into the functional neuroanatomy of decision-making and corroborate the validity of our variant of the orthogonalized go/no-go task as a behavioral assay of the Pavlovian and instrumental valuation systems.

Neural responses underlying extraordinary altruists' generosity for socially distant others

Most people are much less generous toward strangers than close others, a bias termed social discounting. But people who engage in extraordinary real-world altruism, like altruistic kidney donors, show dramatically reduced social discounting. Why they do so is unclear. Some prior research suggests reduced social discounting requires effortfully overcoming selfishness via recruitment of the temporoparietal junction. Alternatively, reduced social discounting may reflect genuinely valuing strangers' welfare more due to how the subjective value of their outcomes is encoded in regions such as rostral anterior cingulate cortex (ACC) and amygdala. We tested both hypotheses in this pre-registered study. We also tested the hypothesis that a loving-kindness meditation (LKM) training intervention would cause typical adults' neural and behavioral patterns to resemble altruists. Altruists and matched controls (N = 77) completed a social discounting task during functional magnetic resonance imaging; 25 controls were randomized to complete LKM training. Neither behavioral nor imaging analyses supported the hypothesis that altruists' reduced social discounting reflects effortfully overcoming selfishness. Instead, group differences emerged in social value encoding regions, including rostral ACC and amygdala. Activation in these regions corresponded to the subjective valuation of others' welfare predicted by the social discounting model. LKM training did not result in more generous behavioral or neural patterns, but only greater perceived difficulty during social discounting. Our results indicate extraordinary altruists' generosity results from the way regions involved in social decision-making encode the subjective value of others' welfare. Interventions aimed at promoting generosity may thus succeed to the degree they can increase the subjective valuation of others' welfare.

BDNF and its Role in the Alcohol Abuse Initiated During Early Adolescence: Evidence from Preclinical and Clinical Studies

Brain-derived neurotrophic factor (BDNF) is a crucial brain signaling protein that is integral to many signaling pathways. This neurotrophin has shown to be highly involved in brain plastic processes such as neurogenesis, synaptic plasticity, axonal growth, and neurotransmission, among others. In the first part of this review, we revise the role of BDNF in different neuroplastic processes within the central nervous system. On the other hand, its deficiency in key neural circuits is associated with the development of psychiatric disorders, including alcohol abuse disorder. Many people begin to drink alcohol during adolescence, and it seems that changes in BDNF are evident after the adolescent regularly consumes alcohol. Therefore, the second part of this manuscript addresses the involvement of BDNF during adolescent brain maturation and how this process can be negatively affected by alcohol abuse. Finally, we propose different BNDF enhancers, both behavioral and pharmacological, which should be considered in the treatment of problematic alcohol consumption initiated during the adolescence.

Global and Regional Structural Differences and Prediction of Autistic Traits during Adolescence

BACKGROUND

Autistic traits are commonly viewed as dimensional in nature, and as continuously distributed in the general population. In this respect, the identification of predictive values of markers such as subtle autism-related alterations in brain morphology for parameter values of autistic traits could increase our understanding of this dimensional occasion. However, currently, very little is known about how these traits correspond to alterations in brain morphology in typically developing individuals, particularly during a time period where changes due to brain development processes do not provide a bias. Therefore, in the present study, we analyzed brain volume, cortical thickness (CT) and surface area (SA) in a cohort of 14-15-year-old adolescents (N = 285, female: N = 162) and tested their predictive value for autistic traits, assessed with the social responsiveness scale (SRS) two years later at the age of 16-17 years, using a regression-based approach. We found that autistic traits were significantly predicted by volumetric changes in the amygdala (r = 0.181), cerebellum (r = 0.128) and hippocampus (r = -0.181, r = -0.203), both in boys and girls. Moreover, the CT of the superior frontal region was negatively correlated (r = -0.144) with SRS scores. Furthermore, we observed a significant association between the SRS total score and smaller left putamen volume, specifically in boys (r = -0.217), but not in girls. Our findings suggest that neural correlates of autistic traits also seem to lie on a continuum in the general population, are determined by limbic-striatal neuroanatomical brain areas, and are partly dependent on sex. As we imaged adolescents from a large population-based cohort within a small age range, these data may help to increase the understanding of autistic-like occasions in otherwise typically developing individuals.

Effects of the dopamine transporter gene on striatal functional connectivity in youths with attention-deficit/hyperactivity disorder

BACKGROUND

The dopamine transporter gene (DAT1), striatal network dysfunction, and visual memory deficits have been consistently reported to be associated with attention-deficit/hyperactivity disorder (ADHD). This study aimed to examine the effects of the DAT1 rs27048 (C)/rs429699 (T) haplotype on striatal functional connectivity and visual memory performance in youths with ADHD.

METHOD

After excluding those who had excessive head motion, a total of 96 drug-naïve youths with ADHD and 114 typically developing (TD) youths were assessed with the resting-state functional magnetic resonance imaging and the delayed matching to sample (DMS) task for visual memory. We examined the effects of ADHD, DAT1 CT haplotype, and the ADHD × CT haplotype interaction on the functional connectivity of five striatal seeds. We also correlated visual memory performance with the functional connectivity of striatal subregions, which showed significant diagnosis × genotype interactions.

RESULTS

Compared with TD youths, ADHD youths showed significant hypoconnectivity of the left dorsal caudate (DC) with bilateral sensorimotor clusters. Significant diagnosis × genotype interactions were found in the connectivity between the left DC and the right sensorimotor cluster, and between the right DC and the left dorsolateral prefrontal/bilateral anterior cingulate clusters. Furthermore, the connectivity of the left DC showing significant diagnosis × genotype interactions was associated with DMS performance in youths with ADHD who carried the DAT1 CT haplotype.

CONCLUSIONS

A novel gene-brain-behavior association between the left DC functional connectivity and visual memory performance in ADHD youths with the DAT1 rs27048 (C)/rs429699 (T) haplotype suggests a differential effect of DAT1 genotype altering specific brain function causing neuropsychological dysfunction in ADHD.

Dissociating brain systems that respond to contingency and valence during monetary loss avoidance in adolescence

Negative reinforcement processes allow individuals to avoid negative and/or harmful outcomes. They depend on the brain's ability to differentiate; (i) contingency from non-contingency, separately from (ii) judgements about positive and negative valence. Thirty-three males (8-18 years) performed a cued reaction-time task during fMRI scanning to differentiate the brain's responses to contingency and valence during loss avoidance. In two conditions, cues indicated no -contingency between participants' responses and monetary loss - (1) CERTAIN LOSS (negative valence) of €0.20, €1 or €5 or (2) CERTAIN LOSS AVOIDANCE (positive valence). In a third condition, cues indicated a contingency between short reaction times and avoidance of monetary loss. As expected participants had shorter reaction times in this latter condition where CONDITIONAL LOSS AVOIDANCE cues activated salience and motor-response-preparation brain networks - independent of the relative valence of the contrast (CERTAIN LOSS or CERTAIN LOSS AVOIDANCE). Effects of valence were seen toward the session's end where CERTAIN LOSS AVOIDANCE cues activated ventral striatum, medial-orbitofrontal cortex and medial-temporal areas more than CERTAIN LOSS. CONDITIONAL LOSS AVOIDANCE trials with feedback indicating "success" activated ventral striatum more than "failure feedback". The findings support the hypothesis that brain networks controlling contingency and valence processes during negative reinforcement are dissociable.

Reward modulates cortical representations of action

People are capable of rapid improvements in performance when they are offered a reward. The neural mechanism by which this performance enhancement occurs remains unclear. We investigated this phenomenon by offering people monetary reward for successful performance in a sequence production task. We found that people performed actions more quickly and accurately when they were offered large reward. Increasing reward magnitude was associated with elevated activity throughout the brain prior to movement. Multivariate patterns of activity in these reward-responsive regions encoded information about the upcoming action. Follow-up analyses provided evidence that action decoding in pre-SMA and other motor planning areas was improved for large reward trials and successful action decoding in SMA was associated with improved performance. These results suggest that reward may enhance performance by enhancing neural representations of action used in motor planning.

The Role of Heart Rate Variability in Mindfulness-Based Pain Relief

Mindfulness meditation is a self-regulatory practice premised on sustaining nonreactive awareness of arising sensory events that reliably reduces pain. Yet, the specific analgesic mechanisms supporting mindfulness have not been comprehensively disentangled from the potential nonspecific factors supporting this technique. Increased parasympathetic nervous system (PNS) activity is associated with pain relief corresponding to a number of cognitive manipulations. However, the relationship between the PNS and mindfulness-based pain attenuation remains unknown. The primary objective of the present study was to determine the role of high-frequency heart rate variability (HF HRV), a marker of PNS activity, during mindfulness-based pain relief as compared to a validated, sham-mindfulness meditation technique that served as a breathing-based control. Sixty-two healthy volunteers (31 females; 31 males) were randomized to a 4-session (25 min/session) mindfulness or sham-mindfulness training regimen. Before and after each group's respective training, participants were administered noxious (49°C) and innocuous (35°C) heat to the right calf. HF HRV and respiration rate were recorded during thermal stimulation and pain intensity and unpleasantness ratings were collected after each stimulation series. The primary analysis revealed that during mindfulness meditation, higher HF HRV was more strongly associated with lower pain unpleasantness ratings when compared to sham-mindfulness meditation (B = -.82, P = .04). This finding is in line with the prediction that mindfulness-based meditation engages distinct mechanisms from sham-mindfulness meditation to reduce pain. However, the same prediction was not confirmed for pain intensity ratings (B = -.41). Secondary analyses determined that mindfulness and sham-mindfulness meditation similarly reduced pain ratings, decreased respiration rate, and increased HF HRV (between group ps < .05). More mechanistic work is needed to reliably determine the role of parasympathetic activation in mindfulness-based pain relief as compared to other meditative techniques. Perspective: Mindfulness has been shown to engage multiple mechanisms to reduce pain. The present study extends on this work to show that higher HRV is associated with mindfulness-induced reductions in pain unpleasantness, but not pain intensity ratings, when compared to sham-mindfulness meditation. These findings warrant further investigation into the mechanisms engaged by mindfulness as compared to placebo.

Network Brain-Computer Interface (nBCI): An Alternative Approach for Cognitive Prosthetics

Brain computer interfaces (BCIs) have been applied to sensorimotor systems for many years. However, BCI technology has broad potential beyond sensorimotor systems. The emerging field of cognitive prosthetics, for example, promises to improve learning and memory for patients with cognitive impairment. Unfortunately, our understanding of the neural mechanisms underlying these cognitive processes remains limited in part due to the extensive individual variability in neural coding and circuit function. As a consequence, the development of methods to ascertain optimal control signals for cognitive decoding and restoration remains an active area of inquiry. To advance the field, robust tools are required to quantify time-varying and task-dependent brain states predictive of cognitive performance. Here, we suggest that network science is a natural language in which to formulate and apply such tools. In support of our argument, we offer a simple demonstration of the feasibility of a network approach to BCI control signals, which we refer to as network BCI (nBCI). Finally, in a single subject example, we show that nBCI can reliably predict online cognitive performance and is superior to certain common spectral approaches currently used in BCIs. Our review of the literature and preliminary findings support the notion that nBCI could provide a powerful approach for future applications in cognitive prosthetics.

Added value of money on motor performance feedback: Increased left central beta-band power for rewards and fronto-central theta-band power for punishments

Monetary rewards and punishments have been shown to respectively enhance retention of motor memories and short-term motor performance, but their underlying neural bases in the context of motor control tasks remain unclear. Using electroencephalography (EEG), the present study tested the hypothesis that monetary rewards and punishments are respectively reflected in post-feedback beta-band (20-30 Hz) and theta-band (3-8 Hz) oscillatory power. While participants performed upper limb reaching movements toward visual targets using their right hand, the delivery of monetary rewards and punishments was manipulated as well as their probability (i.e., by changing target size). Compared to unrewarded and unpunished trials, monetary rewards and the successful avoidance of punishments both entailed greater beta-band power at left central electrodes overlaying contralateral motor areas. In contrast, monetary punishments and reward omissions both entailed increased theta-band power at fronto-central scalp sites. Additional analyses revealed that beta-band power was further increased when rewards were lowly probable. In light of previous work demonstrating similar beta-band modulations in basal ganglia during reward processing, the present results may reflect functional communication of reward-related information between the basal ganglia and motor cortical regions. In turn, the increase in fronto-central theta-band power after monetary punishments may reflect an emphasized cognitive need for behavioral adjustments. Globally, the present work identifies possible neural substrates for the growing behavioral evidence showing beneficial effects of monetary feedback on motor learning and performance.

Striatal action-value neurons reconsidered

It is generally believed that during economic decisions, striatal neurons represent the values associated with different actions. This hypothesis is based on studies, in which the activity of striatal neurons was measured while the subject was learning to prefer the more rewarding action. Here we show that these publications are subject to at least one of two critical confounds. First, we show that even weak temporal correlations in the neuronal data may result in an erroneous identification of action-value representations. Second, we show that experiments and analyses designed to dissociate action-value representation from the representation of other decision variables cannot do so. We suggest solutions to identifying action-value representation that are not subject to these confounds. Applying one solution to previously identified action-value neurons in the basal ganglia we fail to detect action-value representations. We conclude that the claim that striatal neurons encode action-values must await new experiments and analyses.

The control of tonic pain by active relief learning

Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain. We used computational modelling of behavioural, physiological and neuroimaging data in two experiments in which subjects learned to terminate tonic pain in static and dynamic escape-learning paradigms. In both studies, we show that active relief-seeking involves a reinforcement learning process manifest by error signals observed in the dorsal putamen. Critically, this system uses an uncertainty ('associability') signal detected in pregenual anterior cingulate cortex that both controls the relief learning rate, and endogenously and parametrically modulates the level of tonic pain. The results define a self-organising learning circuit that reduces ongoing pain when learning about potential relief.

Alterations of Resting-State Static and Dynamic Functional Connectivity of the Dorsolateral Prefrontal Cortex in Subjects with Internet Gaming Disorder

Internet gaming disorder (IGD), a major behavior disorder, has gained increasing attention. Recent studies indicate altered resting-state static functional connectivity (FC) of the dorsolateral prefrontal cortex (DLPFC) in subjects with IGD. Whereas static FC often provides information on functional changes in subjects with IGD, investigations of temporal changes in FC between the DLPFC and the other brain regions may shed light on the dynamic characteristics of brain function associated with IGD. Thirty subjects with IGD and 30 healthy controls (HCs) matched for age, gender and education status were recruited. Using the bilateral DLPFC as seeds, static FC and dynamic FC maps were calculated and compared between groups. Correlations between alterations in static FC and dynamic FC and clinical variables were also investigated within the IGD group. The IGD group showed significantly lower static FC between the right DLPFC and the left rolandic operculum while higher static FC between the right DLPFC and the left pars triangularis when compared to HCs. The IGD group also had significantly decreased dynamic FC between the right DLPFC and the left insula, right putamen and left precentral gyrus, and increased dynamic FC in the left precuneus. Moreover, the dynamic FC between the right DLPFC and the left insula was negatively correlated with the severity of IGD. Dynamic FC can be used as a powerful supplement to static FC, helping us obtain a more comprehensive understanding of large-scale brain network activity in IGD and put forward new ideas for behavioral intervention therapy for it.

Separate neural representations of depression, anxiety and apathy in Parkinson's disease

Depression, anxiety and apathy are distinct neuropsychiatric symptoms that highly overlap in Parkinson’s disease (PD). It remains unknown whether each symptom is uniquely associated with a functional network dysfunction. Here, we examined whether individual differences in each neuropsychiatric symptom predict functional connectivity patterns in PD patients while controlling for all other symptoms and motor function. Resting-state functional connectivity MRI were acquired from 27 PD patients and 29 healthy controls. Widespread reduced functional connectivity was identified in PD patients and explained by either the neuropsychiatric or motor symptoms. Depression in PD predicted increased functional connectivity between the orbitofrontal, hippocampal complex, cingulate, caudate and thalamus. Apathy in PD predicted decreased caudate-thalamus and orbitofrontal-parahippocampal connectivity. Anxiety in PD predicted three distinct types of functional connectivity not described before: (i) increased limbic-orbitofrontal cortex; (ii) decreased limbic-dorsolateral prefrontal cortex and orbitofrontal-dorsolateral prefrontal cortices and (iii) decreased sensorimotor-orbitofrontal cortices. The first two types of functional connectivity suggest less voluntary and more automatic emotion regulation. The last type is argued to be specific to PD and reflect an impaired ability of the orbitofrontal cortex to guide goal-directed motor actions in anxious PD patients.

Multivariate Bayesian decoding of single-trial event-related fMRI responses for memory retrieval of voluntary actions

This study proposes a method for classifying event-related fMRI responses in a specialized setting of many known but few unknown stimuli presented in a rapid event-related design. Compared to block design fMRI signals, classification of the response to a single or a few stimulus trial(s) is not a trivial problem due to contamination by preceding events as well as the low signal-to-noise ratio. To overcome such problems, we proposed a single trial-based classification method of rapid event-related fMRI signals utilizing sparse multivariate Bayesian decoding of spatio-temporal fMRI responses. We applied the proposed method to classification of memory retrieval processes for two different classes of episodic memories: a voluntarily conducted experience and a passive experience induced by watching a video of others’ actions. A cross-validation showed higher classification performance of the proposed method compared to that of a support vector machine or of a classifier based on the general linear model. Evaluation of classification performances for one, two, and three stimuli from the same class and a correlation analysis between classification accuracy and target stimulus positions among trials suggest that presenting two target stimuli at longer inter-stimulus intervals is optimal in the design of classification experiments to identify the target stimuli. The proposed method for decoding subject-specific memory retrieval of voluntary behavior using fMRI would be useful in forensic applications in a natural environment, where many known trials can be extracted from a simulation of everyday tasks and few target stimuli from a crime scene.

Reduced functional connectivity between ventromedial prefrontal cortex and insula relates to longer corrected QT interval in HIV+ and HIV- individuals

OBJECTIVE

Prolongation of the QT interval, i.e., measure of the time between the start of the Q wave and the end of the T wave, is a precursor to fatal cardiac arrhythmias commonly observed in individuals infected with the Human Immunodeficiency Virus (HIV), and is related to dysregulation of the autonomic nervous system. We investigated the relationship between QT interval length and resting state functional connectivity (rsFC) of the ventromedial prefrontal cortex (VMPFC), a core region of the brain that is involved with cardio-autonomic regulation.

METHOD

Eighteen HIV+ men on antiretroviral therapy and with no history of heart disease were compared with 26 HIV-negative control subjects who had similar demographic and cardio-metabolic characteristics. A seed-based rsFC analysis of the right and left VMPFC was performed at the individual subject level, and 2nd-level analyses were conducted to identify the following: group differences in connectivity, brain regions correlating with corrected (QTc) interval length before and after controlling for those group differences, and regions where seed-based rsFC correlates with CD4 count and QTc interval within HIV+ individuals.

RESULTS

HIV-negative adults showed greater rsFC between the VMPFC seed regions and several default mode network structures. Across groups greater rsFC with the left anterior insula was associated with shorter QTc intervals, whereas right posterior insula connectivity with the VMPFC correlated with greater QTc intervals. HIV patients with lower CD4 counts and higher QTc intervals showed greater rsFC between the right VMPFC and the right posterior insula and dorsal cingulate gyrus.

CONCLUSIONS

This study demonstrates that QTc interval lengths are associated with distinct patterns of VMPFC rsFC with posterior and anterior insula. In HIV patients, longer QTc interval and lower CD4 count corresponded to weaker VMPFC connectivity with the dorsal striatrum.

SIGNIFICANCE

A forebrain control mechanism may be implicated in the suppression of cardiovagal influence that confers risk for ventricular arrhythmias and sudden cardiac death in HIV+ individuals.

Neural determinants of human goal-directed vs. habitual action control and their relation to trait motivation

Instrumental learning is mediated by goal-directed and habit systems in the brain. While rodent studies implicate distinct prefrontal/striatal regions in goal-directed and habit learning, neural systems underpinning these two processes in humans remain poorly understood. Here, using a validated discrimination learning task that distinguishes goal-directed learning from habit learning in 72 subjects in fMRI, we investigated the corticostriatal correlates of goal-directed learning and tested whether brain activation during learning is associated with trait motivation and behavioral performance in the post-learning test phase. Participants showed enhanced activation in medial prefrontal and posterior cingulate cortices during goal-directed action selection in the training phase, whereas habitual action selection activated bilateral insula, bilateral dorsal caudate and left precentral gyrus. In addition, early phase of learning was associated with increased activation in the frontoparietal control network and dorsal striatum, whereas default mode regions depicted increased activation in the late phase. Finally, avoidance motivation scores measured by Behavioral Inhibition/Activation System (BIS/BAS) correlated with accuracy during goal-directed learning and showed a nominally significant correlation with activation in dorsomedial prefrontal cortex during goal-directed acquisition of stimuli. These findings reveal the temporal dynamics of instrumental behavior and suggest that avoidance motivation predicts performance and brain activity during goal-directed learning.

Lower dorsal striatum activation in association with neuroticism during the acceptance of unfair offers

Unfair treatment may evoke more negative emotions in individuals scoring higher on neuroticism, thereby possibly impacting their decision-making in these situations. To investigate the neural basis of social decision-making in these individuals, we examined interpersonal reactions to unfairness in the Ultimatum Game (UG). We measured brain activation with fMRI in 120 participants selected based on their neuroticism score, while they made decisions to accept or reject proposals that were either fair or unfair. The anterior insula and anterior cingulate cortex were more activated during the processing of unfair offers, consistent with prior UG studies. Furthermore, we found more activation in parietal and temporal regions for the two most common decisions (fair accept and unfair reject), involving areas related to perceptual decision-making. Conversely, during the decision to accept unfair offers, individuals recruited more frontal regions previously associated with decision-making and the implementation of reappraisal in the UG. High compared to low neurotic individuals did not show differential activation patterns during the proposal of unfair offers; however, they did show lower activation in the right dorsal striatum (putamen) during the acceptance of unfair offers. This brain region has been involved in the formation of stimulus-action-reward associations and motivation/arousal. In conclusion, the findings suggest that both high and low neurotic individuals recruit brain regions signaling social norm violations in response to unfair offers. However, when it comes to decision-making, it seems that neural circuitry related to reward and motivation is altered in individuals scoring higher on neuroticism, when accepting an unfair offer.

Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.

Using fMRI to study reward processing in humans: past, present, and future

Functional magnetic resonance imaging (fMRI) is a noninvasive tool used to probe cognitive and affective processes. Although fMRI provides indirect measures of neural activity, the advent of fMRI has allowed for1) the corroboration of significant animal findings in the human brain, and2) the expansion of models to include more common human attributes that inform behavior. In this review, we briefly consider the neural basis of the blood oxygenation level dependent signal to set up a discussion of how fMRI studies have applied it in examining cognitive models in humans and the promise of using fMRI to advance such models. Specifically, we illustrate the contribution that fMRI has made to the study of reward processing, focusing on the role of the striatum in encoding reward-related learning signals that drive anticipatory and consummatory behaviors. For instance, we discuss how fMRI can be used to link neural signals (e.g., striatal responses to rewards) to individual differences in behavior and traits. While this functional segregation approach has been constructive to our understanding of reward-related functions, many fMRI studies have also benefitted from a functional integration approach that takes into account how interconnected regions (e.g., corticostriatal circuits) contribute to reward processing. We contend that future work using fMRI will profit from using a multimodal approach, such as combining fMRI with noninvasive brain stimulation tools (e.g., transcranial electrical stimulation), that can identify causal mechanisms underlying reward processing. Consequently, advancements in implementing fMRI will promise new translational opportunities to inform our understanding of psychopathologies.

Altered functioning of reward circuitry in youth offspring of parents with bipolar disorder

BACKGROUND

Offspring of parents with bipolar disorder (BD) (BO) are at higher risk of BD than offspring of parents with non-BD psychopathology (NBO), although both groups are at higher risk than offspring of psychiatrically healthy parents (HC) for other affective and psychiatric disorders. Abnormal functioning in reward circuitry has been demonstrated previously in individuals with BD. We aimed to determine whether activation and functional connectivity in this circuitry during risky decision-making differentiated BO, NBO and HC.

METHOD

BO (n = 29; mean age = 13.8 years; 14 female), NBO (n = 28; mean age = 13.9 years; 12 female) and HC (n = 23; mean age = 13.7 years; 11 female) were scanned while performing a number-guessing reward task. Of the participants, 11 BO and 12 NBO had current non-BD psychopathology; five BO and four NBO were taking psychotropic medications.

RESULTS

A 3 (group) × 2 (conditions: win-control/loss-control) analysis of variance revealed a main effect of group on right frontal pole activation: BO showed significantly greater activation than HC. There was a significant main effect of group on functional connectivity between the bilateral ventral striatum and the right ventrolateral prefrontal cortex (Z > 3.09, cluster-p < 0.05): BO showed significantly greater negative functional connectivity than other participants. These between-group differences remained after removing youth with psychiatric disorders and psychotropic medications from analyses.

CONCLUSIONS

This is the first study to demonstrate that reward circuitry activation and functional connectivity distinguish BO from NBO and HC. The fact that the pattern of findings remained when comparing healthy BO v. healthy NBO v. HC suggests that these neuroimaging measures may represent trait-level neurobiological markers conferring either risk for, or protection against, BD in youth.

Effects of social sustainability signaling on neural valuation signals and taste-experience of food products

Value-based decision making occurs when individuals choose between different alternatives and place a value on each alternative and its attributes. Marketing actions frequently manipulate product attributes, by adding, e.g., health claims on the packaging. A previous imaging study found that an emblem for organic products increased willingness to pay (WTP) and activity in the ventral striatum (VS). The current study investigated neural and behavioral processes underlying the influence of Fair Trade (FT) labeling on food valuation and choice. Sustainability is an important product attribute for many consumers, with FT signals being one way to highlight ethically sustainable production. Forty participants valuated products in combination with an FT emblem or no emblem and stated their WTP in a bidding task while in an MRI scanner. After that, participants tasted-objectively identical-chocolates, presented either as "FT" or as "conventionally produced". In the fMRI task, WTP was significantly higher for FT products. FT labeling increased activity in regions important for reward-processing and salience, that is, in the VS, anterior and posterior cingulate, as well as superior frontal gyrus. Subjective value, that is, WTP was correlated with activity in the ventromedial prefrontal cortex (vmPFC). We find that the anterior cingulate, VS and superior frontal gyrus exhibit task-related increases in functional connectivity to the vmPFC when an FT product was evaluated. Effective connectivity analyses revealed a highly probable directed modulation of the vmPFC by those three regions, suggesting a network which alters valuation processes. We also found a significant taste-placebo effect, with higher experienced taste pleasantness and intensity for FT labeled chocolates. Our results reveal a possible neural mechanism underlying valuation processes of certified food products. The results are important in light of understanding current marketing trends as well as designing future interventions that aim at positively influencing food choice.

A central role for the retrosplenial cortex in de novo environmental learning

With experience we become accustomed to the types of environments that we normally encounter as we navigate in the world. But how does this fundamental knowledge develop in the first place and what brain regions are involved? To examine de novo environmental learning, we created an ‘alien’ virtual reality world populated with landmarks of which participants had no prior experience. They learned about this environment by moving within it during functional MRI (fMRI) scanning while we tracked their evolving knowledge. Retrosplenial cortex (RSC) played a central and highly selective role by representing only the most stable, permanent features in this world. Subsequently, increased coupling was noted between RSC and hippocampus, with hippocampus then expressing knowledge of permanent landmark locations and overall environmental layout. Studying how environmental representations emerge from scratch provided a new window into the information processing underpinning the brain's navigation system, highlighting the key influence of the RSC.

DOI:http://dx.doi.org/10.7554/eLife.09031.001

Throughout our lives, we encounter novel environments that we must learn to find our way around, from a new office to a new city. Studies of brain activity in humans and rodents have revealed that many brain regions are involved in navigation, most notably the hippocampus. However, these experiments have typically involved humans navigating around environments filled with familiar objects and landmarks, and therefore tell us relatively little about how the brain builds up a map of a completely new environment in the first place.

To address this issue, Auger et al. scanned the brains of healthy human volunteers as they experienced an ‘alien’ virtual reality world called ‘Fog World’, so-named because of the dense fog used to precisely control what the volunteers could see. In contrast to previous virtual reality environments, which have contained houses, shops and other recognisable objects, Fog World contains only abstract landmarks that bear little resemblance to anything in the real world. The volunteers watched videos that simulated journeys through Fog World with the goal of learning the layout of the environment so that they could navigate within it. Of note, half of the landmarks in Fog World remained in fixed positions on all learning trials, while the other half changed location from one trial to the next.

After each block of trials, the volunteers were shown single landmarks—some from Fog World and others not—while their brains were scanned. A region called the retrosplenial cortex showed increasing activity that closely tracked the volunteers' growing knowledge of which landmarks had fixed, permanent locations in Fog World. In later trials towards the end of the learning period, the hippocampus also became active, and at this time communication between the retrosplenial cortex and hippocampus was also heightened. By the end of learning, the hippocampal activity was related to the volunteers' knowledge of the locations of the permanent landmarks across Fog World.

As well as revealing that the retrosplenial cortex may be essential for processing permanent landmarks, the work of Auger et al. shows how the hippocampus and retrosplenial cortex could work together to map new environments. These findings might also help us to better understand why some healthy individuals are bad navigators, and why disorientation is a common early symptom in neurodegenerative disorders such as Alzheimer's disease, where the retrosplenial cortex is often one of the first brain regions to become damaged.

DOI:http://dx.doi.org/10.7554/eLife.09031.002

Goal-directed, habitual and Pavlovian prosocial behavior

Although prosocial behaviors have been widely studied across disciplines, the mechanisms underlying them are not fully understood. Evidence from psychology, biology and economics suggests that prosocial behaviors can be driven by a variety of seemingly opposing factors: altruism or egoism, intuition or deliberation, inborn instincts or learned dispositions, and utility derived from actions or their outcomes. Here we propose a framework inspired by research on reinforcement learning and decision making that links these processes and explains characteristics of prosocial behaviors in different contexts. More specifically, we suggest that prosocial behaviors inherit features of up to three decision-making systems employed to choose between self- and other- regarding acts: a goal-directed system that selects actions based on their predicted consequences, a habitual system that selects actions based on their reinforcement history, and a Pavlovian system that emits reflexive responses based on evolutionarily prescribed priors. This framework, initially described in the field of cognitive neuroscience and machine learning, provides insight into the potential neural circuits and computations shaping prosocial behaviors. Furthermore, it identifies specific conditions in which each of these three systems should dominate and promote other- or self- regarding behavior.

Which motor cortical region best predicts imagined movement?

In brain-computer interfacing (BCI), motor imagery is used to provide a gateway to an effector action or behavior. However, in contrast to the main functional role of the primary motor cortex (M1) in motor execution, the M1's involvement in motor imagery has been debated, while the roles of secondary motor areas such as the premotor cortex (PMC) and supplementary motor area (SMA) in motor imagery have been proposed. We examined which motor cortical region had the greatest predictive ability for imagined movement among the primary and secondary motor areas. For two modes of motor performance, executed movement and imagined movement, in 12 healthy subjects who performed two types of motor task, hand grasping and hand rotation, we used the multivariate Bayes method to compare predictive ability between the primary and secondary motor areas (M1, PMC, and SMA) contralateral to the moved hand. With the distributed representation of activation, executed movement was best predicted from the M1 while imagined movement from the SMA, among the three motor cortical regions, in both types of motor task. In addition, the most predictive information about the distinction between executed movement and imagined movement was contained in the M1. The greater predictive ability of the SMA for imagined movement suggests its functional role that could be applied to motor imagery-based BCI.

BDNF Val66Met and reward-related brain function in adolescents: role for early alcohol consumption

Changes in reward processing have been identified as one important pathogenetic mechanism in alcohol addiction. The nonsynonymous single nucleotide polymorphism in the brain-derived neurotrophic factor (BDNF) gene (rs6265/Val66Met) modulates the central nervous system activity of neurotransmitters involved in reward processing such as serotonin, dopamine, and glutamate. It was identified as crucial for alcohol consumption in healthy adults and, in rats, specifically related to the function in the striatum, a region that is commonly involved in reward processing. However, studies in humans on the association of BDNF Val66Met and reward-related brain functions and its role for alcohol consumption, a significant predictor of later alcohol addiction, are missing. Based on an intermediate phenotype approach, we assessed the early orientation toward alcohol and alcohol consumption in 530 healthy adolescents that underwent a monetary incentive delay task during functional magnetic resonance imaging. We found a significantly lower response in the putamen to reward anticipation in adolescent Met carriers with high versus low levels of alcohol consumption. During reward feedback, Met carriers with low putamen reactivity were significantly more likely to orient toward alcohol and to drink alcohol 2 years later. This study indicates a possible effect of BDNF Val66Met on alcohol addiction-related phenotypes in adolescence.

Reward activates stimulus-specific and task-dependent representations in visual association cortices

Humans reliably learn which actions lead to rewards. One prominent question is how credit is assigned to environmental stimuli that are acted upon. Recent functional magnetic resonance imaging (fMRI) studies have provided evidence that representations of rewarded stimuli are activated upon reward delivery, providing possible eligibility traces for credit assignment. Our study sought evidence of postreward activation in sensory cortices satisfying two conditions of instrumental learning: postreward activity should reflect the stimulus category that preceded reward (stimulus specificity), and should occur only if the stimulus was acted on to obtain reward (task dependency). Our experiment implemented two tasks in the fMRI scanner. The first was a perceptual decision-making task on degraded face and house stimuli. Stimulus specificity was evident as rewards activated the sensory cortices associated with face versus house perception more strongly after face versus house decisions, respectively, particularly in the fusiform face area. Stimulus specificity was further evident in a psychophysiological interaction analysis wherein face-sensitive areas correlated with nucleus accumbens activity after face-decision rewards, whereas house-sensitive areas correlated with nucleus accumbens activity after house-decision rewards. The second task required participants to make an instructed response. The criterion of task dependency was fulfilled as rewards after face versus house responses activated the respective association cortices to a larger degree when faces and houses were relevant to the performed task. Our study is the first to show that postreward sensory cortex activity meets these two key criteria of credit assignment, and does so independently from bottom-up perceptual processing.

Arbitration between controlled and impulsive choices

The impulse to act for immediate reward often conflicts with more deliberate evaluations that support long-term benefit. The neural architecture that negotiates this conflict remains unclear. One account proposes a single neural circuit that evaluates both immediate and delayed outcomes, while another outlines separate impulsive and patient systems that compete for behavioral control. Here we designed a task in which a complex payout structure divorces the immediate value of acting from the overall long-term value, within the same outcome modality. Using model-based fMRI in humans, we demonstrate separate neural representations of immediate and long-term values, with the former tracked in the anterior caudate (AC) and the latter in the ventromedial prefrontal cortex (vmPFC). Crucially, when subjects' choices were compatible with long-run consequences, value signals in AC were down-weighted and those in vmPFC were enhanced, while the opposite occurred when choice was impulsive. Thus, our data implicate a trade-off in value representation between AC and vmPFC as underlying controlled versus impulsive choice.

Functional dysconnectivity of corticostriatal circuitry and differential response to methylphenidate in youth with attention-deficit/hyperactivity disorder

BACKGROUND

Brain frontostriatal circuits have been implicated in the pathophysiology of attention-deficit/hyperactivity disorder (ADHD). However, effects of methylphenidate on circuit-level functional connectivity are as yet unclear. The aim of the present study was to comprehensively investigate the functional connectivity of major striatal subregions in children with ADHD, including subanalyses directed at mapping cognitive and treatment response characteristics.

METHODS

Using a comprehensive seeding strategy, we examined resting-state functional connectivity of dorsal and ventral subdivisions of the caudate nucleus and putamen in children and adolescents with ADHD and in age- and sex-matched healthy controls.

RESULTS

We enrolled 83 patients with ADHD and 22 controls in our study. Patients showed significantly reduced dorsal caudate functional connectivity with the superior and middle prefrontal cortices as well as reduced dorsal putamen connectivity with the parahippocampal cortex. These connectivity measures were correlated in opposite directions in patients and controls with attentional performance, as assessed using the Continuous Performance Test. Patients showing a good response to methylphenidate had significantly reduced ventral caudate/nucleus accumbens connectivity with the inferior frontal cortices compared with poor responders.

LIMITATIONS

Possible confounding effects of age-related functional connectivity change were not excluded owing to the wide age range of participants.

CONCLUSION

We observed a region-specific effect of methylphenidate on resting-state functional connectivity, suggesting the pretreatment level of ventral frontostriatal functional connectivity as a possible methylphenidate response biomarker of ADHD.

A selective involvement of putamen functional connectivity in youth with internet gaming disorder

Brain cortico-striatal circuits have consistently been implicated in the pathology of addiction related disorders. We applied a reliable seed-based analysis of the resting-state brain activity to comprehensively delineate the subdivisions of striatal functional connectivity implicated in internet gaming disorder. Among twelve right-handed male adolescents with internet gaming disorder and 11 right-handed and gender-matched healthy controls, we examined group differences in the functional connectivity of dorsal and ventral subdivisions of the caudate nucleus and putamen, as well as the association of these connectivity indices with behavioral measures of internet use. Adolescents with internet gaming disorder showed significantly reduced dorsal putamen functional connectivity with the posterior insula-parietal operculum. More time spent playing online games predicted significantly greater functional connectivity between the dorsal putamen and bilateral primary somatosensory cortices in adolescents with internet gaming disorder, and significantly lower functional connectivity between the dorsal putamen and bilateral sensorimotor cortices in healthy controls. The dorsal putamen functional connectivity was significantly and specifically different in adolescents with internet gaming disorder. The findings suggest a possible biomarker of internet gaming disorder.

Processing of action- but not stimulus-related prediction errors differs between active and observational feedback learning

Learning of stimulus-response-outcome associations is driven by outcome prediction errors (PEs). Previous studies have shown larger PE-dependent activity in the striatum for learning from own as compared to observed actions and the following outcomes despite comparable learning rates. We hypothesised that this finding relates primarily to a stronger integration of action and outcome information in active learners. Using functional magnetic resonance imaging, we investigated brain activations related to action-dependent PEs, reflecting the deviation between action values and obtained outcomes, and action-independent PEs, reflecting the deviation between subjective values of response-preceding cues and obtained outcomes. To this end, 16 active and 15 observational learners engaged in a probabilistic learning card-guessing paradigm. On each trial, active learners saw one out of five cues and pressed either a left or right response button to receive feedback (monetary win or loss). Each observational learner observed exactly those cues, responses and outcomes of one active learner. Learning performance was assessed in active test trials without feedback and did not differ between groups. For both types of PEs, activations were found in the globus pallidus, putamen, cerebellum, and insula in active learners. However, only for action-dependent PEs, activations in these structures and the anterior cingulate were increased in active relative to observational learners. Thus, PE-related activity in the reward system is not generally enhanced in active relative to observational learning but only for action-dependent PEs. For the cerebellum, additional activations were found across groups for cue-related uncertainty, thereby emphasising the cerebellum's role in stimulus-outcome learning.

An interoceptive neuroanatomical perspective on feelings, energy, and effort

A homeostatic energy model of awareness proposes that the anterior insular cortex engenders feelings that provide an amodal valuation of homeostatic energy utilization in an opponent, bivalent emotional control system. Feelings are the "common currency" which enable optimal utilization in the physical and mental behavior of a highly social primate. This model offers a different perspective.

The human amygdala encodes value and space during decision making

Valuable stimuli are invariably localized in space. While our knowledge regarding the neural networks supporting value assignment and comparisons is considerable, we lack a basic understanding of how the human brain integrates motivational and spatial information. The amygdala is a key structure for learning and maintaining the value of sensory stimuli and a recent non-human primate study provided initial evidence that it also acts to integrate value with spatial location, a question we address here in a human setting. We measured haemodynamic responses (fMRI) in amygdala while manipulating the value and spatial configuration of stimuli in a simple stimulus–reward task. Subjects responded significantly faster and showed greater amygdala activation when a reward was dependent on a spatial specific response, compared to when a reward required less spatial specificity. Supplemental analysis supported this spatial specificity by demonstrating that the pattern of amygdala activity varied based on whether subjects responded to a motivational target presented in the ipsilateral or contralateral visual space. Our data show that the human amygdala integrates information about space and value, an integration of likely importance for assigning cognitive resources towards highly valuable stimuli in our environment.

•

Amygdala responds to valuable stimuli in a spatial specific manner.

•

Amygdala–dACC connectivity varies according to the spatial location of value cues.

•

Amygdala integrates information about stimulus value and its spatial representation.

•

Dorsal ACC may supply information about spatial location to the amygdala.

Action-value comparisons in the dorsolateral prefrontal cortex control choice between goal-directed actions

It is generally assumed that choice between different actions reflects the difference between their action values yet little direct evidence confirming this assumption has been reported. Here we assess whether the brain calculates the absolute difference between action values or their relative advantage, that is, the probability that one action is better than the other alternatives. We use a two-armed bandit task during functional magnetic resonance imaging and modelled responses to determine both the size of the difference between action values (D) and the probability that one action value is better (P). The results show haemodynamic signals corresponding to P in right dorsolateral prefrontal cortex (dlPFC) together with evidence that these signals modulate motor cortex activity in an action-specific manner. We find no significant activity related to D. These findings demonstrate that a distinct neuronal population mediates action-value comparisons, and reveals how these comparisons are implemented to mediate value-based decision-making.

In humans, choice between actions depends on the ability to compare action–outcome values. Here, the authors show that action–outcome values are compared on the basis of the relative advantage of a particular action over alternative actions, which takes place in the right dorsolateral prefrontal cortex of the brain.

CA3 size predicts the precision of memory recall

There is enduring interest in why some of us have clearer memories than others, given the substantial individual variation that exists in retrieval ability and the precision with which we can differentiate past experiences. Here we report novel evidence showing that variation in the size of human hippocampal subfield CA3 predicted the amount of neural interference between episodic memories within CA3, which in turn predicted how much retrieval confusion occurred between past memories. This effect was not apparent in other hippocampal subfields. This shows that subtle individual differences in subjective mnemonic experience can be accurately gauged from measurable variations in the anatomy and neural coding of hippocampal region CA3. Moreover, this mechanism may be relevant for understanding memory muddles in aging and pathological states.

Social equality in the number of choice options is represented in the ventromedial prefrontal cortex

A distinct aspect of the sense of fairness in humans is that we care not only about equality in material rewards but also about equality in nonmaterial values. One such value is the opportunity to choose freely among many options, often regarded as a fundamental right to economic freedom. In modern developed societies, equal opportunities in work, living, and lifestyle are enforced by antidiscrimination laws. Despite the widespread endorsement of equal opportunity, no studies have explored how people assign value to it. We used functional magnetic resonance imaging to identify the neural substrates for subjective valuation of equality in choice opportunity. Participants performed a two-person choice task in which the number of choices available was varied across trials independently of choice outcomes. By using this procedure, we manipulated the degree of equality in choice opportunity between players and dissociated it from the value of reward outcomes and their equality. We found that activation in the ventromedial prefrontal cortex (vmPFC) tracked the degree to which the number of options between the two players was equal. In contrast, activation in the ventral striatum tracked the number of options available to participants themselves but not the equality between players. Our results demonstrate that the vmPFC, a key brain region previously implicated in the processing of social values, is also involved in valuation of equality in choice opportunity between individuals. These findings may provide valuable insight into the human ability to value equal opportunity, a characteristic long emphasized in politics, economics, and philosophy.

Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

The ability to learn contingencies between actions and outcomes in a dynamic environment is critical for flexible, adaptive behavior. Goal-directed actions adapt to changes in action-outcome contingencies as well as to changes in the reward-value of the outcome. When networks involved in reward processing and contingency learning are maladaptive, this fundamental ability can be lost, with detrimental consequences for decision-making. Impaired decision-making is a core feature in a number of psychiatric disorders, ranging from depression to schizophrenia. The argument can be developed, therefore, that seemingly disparate symptoms across psychiatric disorders can be explained by dysfunction within common decision-making circuitry. From this perspective, gaining a better understanding of the neural processes involved in goal-directed action, will allow a comparison of deficits observed across traditional diagnostic boundaries within a unified theoretical framework. This review describes the key processes and neural circuits involved in goal-directed decision-making using evidence from animal studies and human neuroimaging. Select studies are discussed to outline what we currently know about causal judgments regarding actions and their consequences, action-related reward evaluation, and, most importantly, how these processes are integrated in goal-directed learning and performance. Finally, we look at how adaptive decision-making is impaired across a range of psychiatric disorders and how deepening our understanding of this circuitry may offer insights into phenotypes and more targeted interventions.

Neurocognitive mechanisms of perception-action coordination: a review and theoretical integration

The present analysis aims at a theoretical integration of, and a systems-neuroscience perspective on, a variety of historical and contemporary views on perception-action coordination (PAC). We set out to determine the common principles or lawful linkages between sensory and motor systems that explain how perception is action-oriented and how action is perceptually guided. To this end, we analyze the key ingredients to such an integrated framework, examine the architecture of dual-system conjectures of PAC, and endeavor in an historical analysis of the key characteristics, mechanisms, and phenomena of PACs. This analysis will reveal that dual-systems views are in need of fundamental re-thinking, and its elements will be amalgamated with current views on action-oriented predictive processing into a novel integrative theoretical framework (IMPPACT: Impetus, Motivation, and Prediction in Perception-Action Coordination theory). From this framework and its neurocognitive architecture we derive a number of non-trivial predictions regarding conative, motive-driven PAC. We end by presenting a brief outlook on how IMPPACT might present novel insights into certain pathologies and into action expertise.

Disentangling neural representations of value and salience in the human brain

A large body of evidence has implicated the posterior parietal and orbitofrontal cortex in the processing of value. However, value correlates perfectly with salience when appetitive stimuli are investigated in isolation. Accordingly, considerable uncertainty has remained about the precise nature of the previously identified signals. In particular, recent evidence suggests that neurons in the primate parietal cortex signal salience instead of value. To investigate neural signatures of value and salience, here we apply multivariate (pattern-based) analyses to human functional MRI data acquired during a noninstrumental outcome-prediction task involving appetitive and aversive outcomes. Reaction time data indicated additive and independent effects of value and salience. Critically, we show that multivoxel ensemble activity in the posterior parietal cortex encodes predicted value and salience in superior and inferior compartments, respectively. These findings reinforce the earlier reports of parietal value signals and reconcile them with the recent salience report. Moreover, we find that multivoxel patterns in the orbitofrontal cortex correlate with value. Importantly, the patterns coding for the predicted value of appetitive and aversive outcomes are similar, indicating a common neural scale for appetite and aversive values in the orbitofrontal cortex. Thus orbitofrontal activity patterns satisfy a basic requirement for a neural value signal.

Reward-related activity in ventral striatum is action contingent and modulated by behavioral relevance

Multiple features of the environment are often imbued with motivational significance, and the relative importance of these can change across contexts. The ability to flexibly adjust evaluative processes so that currently important features of the environment alone drive behavior is critical to adaptive routines. We know relatively little about the neural mechanisms involved, including whether motivationally significant features are obligatorily evaluated or whether current relevance gates access to value-sensitive regions. We addressed these questions using functional magnetic resonance imaging data and a task design where human subjects had to choose whether to accept or reject an offer indicated by visual and auditory stimuli. By manipulating, on a trial-by-trial basis, which stimulus determined the value of the offer, we show choice activity in the ventral striatum solely reflects the value of the currently relevant stimulus, consistent with a model wherein behavioral relevance modulates the impact of sensory stimuli on value processing. Choice outcome signals in this same region covaried positively with wins on accept trials, and negatively with wins on reject trials, consistent with striatal activity at feedback reflecting correctness of response rather than reward processing per se. We conclude that ventral striatum activity during decision making is dynamically modulated by behavioral context, indexed here by task relevance and action selection.

People are better at maximizing expected gain in a manual aiming task with rapidly changing probabilities than with rapidly changing payoffs

Previous research has shown that humans can select movements that achieve their goals, while avoiding negative outcomes, by selecting an “optimal movement endpoint.” This optimal endpoint is modeled based on the participants' endpoint variability and the payoffs associated with the target and penalty regions within the environment. Although the values associated with our goals vary on a moment-to-moment basis in our daily interactions, the adaptation of endpoint selection to changing payoffs in laboratory-based tasks has been examined by varying contexts between blocks of trials. The present study was designed to determine whether participants adjust endpoints and aim to optimal endpoints and whether performance differs when probability or payoff parameters change from trial to trial. Participants aimed to a target circle that was partially overlapped by a penalty circle. They received 100 points for hitting the target and lost points for hitting the penalty area. The magnitude of the penalty value or the distance between the centers of the circles (related to the probability of target and penalty contact) was changed randomly from trial to trial in separate blocks. Results revealed that participants shifted their endpoint and generally aimed optimally when the distance between the circles was varied but did not optimally shift their endpoints when the penalty value was varied. The results suggest that participants rapidly adapted endpoints when the probabilities associated with the task change, because the spatial parameters are an intrinsic property of the visual stimuli that are tightly linked with the motor system, whereas consistent feedback may be necessary to adjust to value parameters effectively.

Action versus valence in decision making

•

Pavlovian responses couple action and valence.

•

This coupling interferes with instrumental learning and performance.

•

Action dominates valence in the striatum and dopaminergic midbrain.

•

Boosting dopamine enhances the dominance of action over valence in the striatum.

•

Boosting dopamine decreases the extent of the behavioral coupling between action and valence.

The selection of actions, and the vigor with which they are executed, are influenced by the affective valence of predicted outcomes. This interaction between action and valence significantly influences appropriate and inappropriate choices and is implicated in the expression of psychiatric and neurological abnormalities, including impulsivity and addiction. We review a series of recent human behavioral, neuroimaging, and pharmacological studies whose key design feature is an orthogonal manipulation of action and valence. These studies find that the interaction between the two is subject to the critical influence of dopamine. They also challenge existing views that neural representations in the striatum focus on valence, showing instead a dominance of the anticipation of action.

Keep focussing: striatal dopamine multiple functions resolved in a single mechanism tested in a simulated humanoid robot

The effects of striatal dopamine (DA) on behavior have been widely investigated over the past decades, with “phasic” burst firings considered as the key expression of a reward prediction error responsible for reinforcement learning. Less well studied is “tonic” DA, where putative functions include the idea that it is a regulator of vigor, incentive salience, disposition to exert an effort and a modulator of approach strategies. We present a model combining tonic and phasic DA to show how different outflows triggered by either intrinsically or extrinsically motivating stimuli dynamically affect the basal ganglia by impacting on a selection process this system performs on its cortical input. The model, which has been tested on the simulated humanoid robot iCub interacting with a mechatronic board, shows the putative functions ascribed to DA emerging from the combination of a standard computational mechanism coupled to a differential sensitivity to the presence of DA across the striatum.

Hierarchical prediction errors in midbrain and basal forebrain during sensory learning

In Bayesian brain theories, hierarchically related prediction errors (PEs) play a central role for predicting sensory inputs and inferring their underlying causes, e.g., the probabilistic structure of the environment and its volatility. Notably, PEs at different hierarchical levels may be encoded by different neuromodulatory transmitters. Here, we tested this possibility in computational fMRI studies of audio-visual learning. Using a hierarchical Bayesian model, we found that low-level PEs about visual stimulus outcome were reflected by widespread activity in visual and supramodal areas but also in the midbrain. In contrast, high-level PEs about stimulus probabilities were encoded by the basal forebrain. These findings were replicated in two groups of healthy volunteers. While our fMRI measures do not reveal the exact neuron types activated in midbrain and basal forebrain, they suggest a dichotomy between neuromodulatory systems, linking dopamine to low-level PEs about stimulus outcome and acetylcholine to more abstract PEs about stimulus probabilities.

Goals and habits in the brain

An enduring and richly elaborated dichotomy in cognitive neuroscience is that of reflective versus reflexive decision making and choice. Other literatures refer to the two ends of what is likely to be a spectrum with terms such as goal-directed versus habitual, model-based versus model-free or prospective versus retrospective. One of the most rigorous traditions of experimental work in the field started with studies in rodents and graduated via human versions and enrichments of those experiments to a current state in which new paradigms are probing and challenging the very heart of the distinction. We review four generations of work in this tradition and provide pointers to the forefront of the field's fifth generation.