Corticostriatal Circuits Encode the Subjective Value of Perceived Control

Choice enhances touch pleasantness

We value what we choose more than what is imposed upon us. Choice-induced preferences are extensively demonstrated using behavioural and neural methods, mainly involving rewarding objects such as money or material goods. However, the impact of choice on experiences, especially in the realm of affective touch, remains less explored. In this study, we specifically investigate whether choice can enhance the pleasure derived from affective touch, thereby increasing its intrinsic rewarding value. We conducted an experiment in which participants were being touched by an experimenter and asked to rate how pleasant their experience of touch was. They were given either a choice or no choice over certain touch stimulus variables which differed in their relevance: some were of low relevance (relating to the colour of the glove that the experimenter would use to touch them), while others were of high relevance (relating to the location on their arm where they would be stroked). Before and during touching, pupillometry was used to measure the level of arousal. We found that having a choice over aspects of tactile stimuli-especially those relevant to oneself-enhanced the pleasant perception of the touch. In addition, having a choice increases arousal in anticipation of touch. Regardless of how relevant it is to the actual tactile stimulus, allowing one to choose may positively enhance a person's perception of the physical contact they receive.

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Motivation as a Lens for Understanding Information-seeking Behaviors

Most prior research characterizes information-seeking behaviors as serving utilitarian purposes, such as whether the obtained information can help solve practical problems. However, information-seeking behaviors are sensitive to different contexts (i.e., threat vs. curiosity), despite having equivalent utility. Furthermore, these search behaviors can be modulated by individuals' life history and personality traits. Yet the emphasis on utilitarian utility has precluded the development of a unified model, which explains when and how individuals actively seek information. To account for this variability and flexibility, we propose a unified information-seeking framework that examines information-seeking through the lens of motivation. This unified model accounts for integration across individuals' internal goal states and the salient features of the environment to influence information-seeking behavior. We propose that information-seeking is determined by motivation for information, invigorated either by instrumental utility or hedonic utility, wherein one's personal or environmental context moderates this relationship. Furthermore, we speculate that the final common denominator in guiding information-seeking is the engagement of different neuromodulatory circuits centered on dopaminergic and noradrenergic tone. Our framework provides a unified framework for information-seeking behaviors and generates several testable predictions for future studies.

Sensitivity to intrinsic rewards is domain general and related to mental health

Humans frequently engage in intrinsically rewarding activities (for example, consuming art, reading). Despite such activities seeming diverse, we show that sensitivity to intrinsic rewards is domain general and associated with mental health. In this cross-sectional study, participants online (N = 483) were presented with putative visual, cognitive and social intrinsic rewards as well as monetary rewards and neutral stimuli. All rewards elicited positive feelings (were 'liked'), generated consummatory behaviour (were 'wanted') and increased the likelihood of the action leading to them (were 'reinforcing'). Factor analysis revealed that ~40% of response variance across stimuli was explained by a general sensitivity to all rewards, but not to neutral stimuli. Affective aspects of mental health were associated with sensitivity to intrinsic, but not monetary, rewards. These results may help explain thriving and suffering: individuals with high reward sensitivity will engage in a variety of intrinsically rewarding activities, eventually finding those they excel at, whereas low sensitivity individuals will not.

Associations of lesion location, structural disconnection, and functional diaschisis with depressive symptoms post stroke

Introduction

Post-stroke depressive symptoms (PSDS) are common and relevant for patient outcome, but their complex pathophysiology is ill understood. It likely involves social, psychological and biological factors. Lesion location is a readily available information in stroke patients, but it is unclear if the neurobiological substrates of PSDS are spatially localized. Building on previous analyses, we sought to determine if PSDS are associated with specific lesion locations, structural disconnection and/or localized functional diaschisis.

Methods

In a prospective observational study, we examined 270 patients with first-ever stroke with the Hospital Anxiety and Depression Scale (HADS) around 6 months post-stroke. Based on individual lesion locations and the depression subscale of the HADS we performed support vector regression lesion-symptom mapping, structural-disconnection-symptom mapping and functional lesion network-symptom-mapping, in a reanalysis of this previously published cohort to infer structure-function relationships.

Results

We found that depressive symptoms were associated with (i) lesions in the right insula, right putamen, inferior frontal gyrus and right amygdala and (ii) structural disconnection in the right temporal lobe. In contrast, we found no association with localized functional diaschisis. In addition, we were unable to confirm a previously described association between depressive symptom load and a network damage score derived from functional disconnection maps.

Discussion

Based on our results, and other recent lesion studies, we see growing evidence for a prominent role of right frontostriatal brain circuits in PSDS.

Choice Boosts Curiosity

In our connected era, we spend significant time and effort satisfying our curiosity. Often, we choose which information we seek, but sometimes the selection is made for us. We hypothesized that humans exhibit enhanced curiosity in the context of choice. We designed a task in which healthy participants saw two lotteries on each trial. On some trials, participants chose which lottery to play. On other trials, the lottery was selected for them. Participants then indicated their curiosity about the outcome of the to-be-played lottery via self-report ratings (Experiment 1, N = 34) or willingness-to-wait decisions (Experiment 2, N = 34). We found that participants exhibited higher curiosity ratings and greater willingness to wait for the outcome of lotteries they had chosen than for lotteries that had been selected for them (controlling for initial preference). This demonstrates that choice boosts curiosity, which may have implications for boosting learning, memory, and motivation.

Choosing to choose or not

To what degree do people prefer to choose for themselves and what drives this preference? Is it memory-based and results from a life-long association between choices and better outcomes, or is the process of choice itself reinforcing? In a new paradigm, across 6 experiments, participants experienced both ’Own Choice’ and ’Computer Picks’ conditions with identical outcomes before selecting which condition to re-experience in the final part of the experiment. Consistent with previous work, an overwhelming majority ( 83%) preferred own-choice. Several variations of the paradigm reveal that (1) Preference For Choice (PFC) is reduced when thinking about the task without actually choosing in it, (2) PFC is substantially reduced by choice-unrelated cognitive load, and (3) Preference For Choice is further diminished when selection is based on criteria other than one’s preferences. Across experiments, participants’ self-rated enjoyment predicted a significant portion of their PFC, while their perceived gains had little to no predictive value. If PFC stems solely from past reinforcement learning (i.e., memory) then neither performing another few scores of choices nor adding cognitive load to that sequence of choices would be expected to dramatically affect it. Hence, our findings suggest that a significant part of this preference stems from the process of choice itself, and that the experience it confers can itself be reinforcing. We discuss the implications of the proposed mechanism for PFC, which leads us to the prediction that PFC may be muted or even reversed under specific conditions and what this means for when the ‘opposite’ effect – sticking with the default – will occur.

Reward enhances connectivity between the ventral striatum and the default mode network

The default mode network (DMN) has been theorized to participate in a range of social, cognitive, and affective functions. Yet, previous accounts do not consider how the DMN contributes to other brain regions depending on psychological context, thus rendering our understanding of DMN function incomplete. We addressed this gap by applying a novel network-based psychophysiological interaction (nPPI) analysis to the reward task within the Human Connectome Project. We first focused on the task-evoked responses of the DMN and other networks involving the prefrontal cortex, including the executive control network (salience network) and the left and right frontoparietal networks. Consistent with a host of prior studies, the DMN exhibited a relative decrease in activation during the task, while the other networks exhibited a relative increase during the task. Next, we used nPPI analyses to assess whether these networks exhibit task-dependent changes in connectivity with other brain regions. Strikingly, we found that the experience of reward enhances task-dependent connectivity between the DMN and the ventral striatum, an effect that was specific to the DMN. Surprisingly, the strength of DMN-VS connectivity was correlated with personality characteristics relating to openness. Taken together, these results advance models of DMN by demonstrating how it contributes to other brain systems during task performance and how those contributions relate to individual differences.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

A Computational Model of Hopelessness and Active-Escape Bias in Suicidality

Currently, psychiatric practice lacks reliable predictive tools and a sufficiently detailed mechanistic understanding of suicidal thoughts and behaviors (STB) to provide timely and personalized interventions. Developing computational models of STB that integrate across behavioral, cognitive and neural levels of analysis could help better understand STB vulnerabilities and guide personalized interventions. To that end, we present a computational model based on the active inference framework. With this model, we show that several STB risk markers - hopelessness, Pavlovian bias and active-escape bias - are interrelated via the drive to maximize one's model evidence. We propose four ways in which these effects can arise: (1) increased learning from aversive outcomes, (2) reduced belief decay in response to unexpected outcomes, (3) increased stress sensitivity and (4) reduced sense of stressor controllability. These proposals stem from considering the neurocircuits implicated in STB: how the locus coeruleus - norepinephrine (LC-NE) system together with the amygdala (Amy), the dorsal prefrontal cortex (dPFC) and the anterior cingulate cortex (ACC) mediate learning in response to acute stress and volatility as well as how the dorsal raphe nucleus - serotonin (DRN-5-HT) system together with the ventromedial prefrontal cortex (vmPFC) mediate stress reactivity based on perceived stressor controllability. We validate the model by simulating performance in an Avoid/Escape Go/No-Go task replicating recent behavioral findings. This serves as a proof of concept and provides a computational hypothesis space that can be tested empirically and be used to distinguish planful versus impulsive STB subtypes. We discuss the relevance of the proposed model for treatment response prediction, including pharmacotherapy and psychotherapy, as well as sex differences as it relates to stress reactivity and suicide risk.

Prefrontal cortex and depression

The prefrontal cortex (PFC) has emerged as one of the regions most consistently impaired in major depressive disorder (MDD). Although functional and structural PFC abnormalities have been reported in both individuals with current MDD as well as those at increased vulnerability to MDD, this information has not translated into better treatment and prevention strategies. Here, we argue that dissecting depressive phenotypes into biologically more tractable dimensions - negative processing biases, anhedonia, despair-like behavior (learned helplessness) - affords unique opportunities for integrating clinical findings with mechanistic evidence emerging from preclinical models relevant to depression, and thereby promises to improve our understanding of MDD. To this end, we review and integrate clinical and preclinical literature pertinent to these core phenotypes, while emphasizing a systems-level approach, treatment effects, and whether specific PFC abnormalities are causes or consequences of MDD. In addition, we discuss several key issues linked to cross-species translation, including functional brain homology across species, the importance of dissecting neural pathways underlying specific functional domains that can be fruitfully probed across species, and the experimental approaches that best ensure translatability. Future directions and clinical implications of this burgeoning literature are discussed.

Neurocomputational mechanism of controllability inference under a multi-agent setting

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.

How we perceive controllability over an outcome if there are multiple other agents who can simultaneously influence that outcome? Previous ‘single-agent’ studies showed that an agents’ inferred controllability depends on contingency between its own action and following outcome and this inference involves striatum. Here, we show that in a multi-agent setting, other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which was explained as a biased Bayesian inference. Notably, bias in inference played an adaptive role under volatile controllability and was associated with a perception of guilt. Striatum and temporoparietal junction (TPJ) were involved in this multi-agent Bayesian controllability inference and this controllability information was leveraged to increase motivated behavior in the vmPFC. Our results first provide a neurocomputational mechanism of multi-agent controllability inference.

The Interplay Between Affective Processing and Sense of Agency During Action Regulation: A Review

Sense of agency is the feeling of being in control of one's actions and their perceivable effects. Most previous research identified cognitive or sensory determinants of agency experience. However, it has been proposed that sense of agency is also bound to the processing of affective information. For example, during goal-directed actions or instrumental learning we often rely on positive feedback (e.g., rewards) or negative feedback (e.g., error messages) to determine our level of control over the current task. Nevertheless, we still lack a scientific model which adequately explains the relation between affective processing and sense of agency. In this article, we review current empirical findings on how affective information modulates agency experience, and, conversely, how sense of agency changes the processing of affective action outcomes. Furthermore, we discuss in how far agency-related changes in affective processing might influence the ability to enact cognitive control and action regulation during goal-directed behavior. A preliminary model is presented for describing the interplay between sense of agency, affective processing, and action regulation. We propose that affective processing could play a role in mediating the influence between subjective sense of agency and the objective ability to regulate one's behavior. Thus, determining the interrelation between affective processing and sense of agency will help us to understand the potential mechanistic basis of agency experience, as well as its functional significance for goal-directed behavior.

Reward prediction errors drive declarative learning irrespective of agency

Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms, RPEs drive declarative learning, with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant's own response, or whether instead, any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a nonagency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.

Self-choice preference: The propensity to under-delegate irrespective of sense of control

People are more likely to make choices themselves than delegate to an agent, even when it may not be the most optimal decision based on a cost-benefit analysis. Previous studies have demonstrated that retaining authority and controllability might be the primary reason for preferring self-choice. The current study asks whether impairment of controllability associated with self-choice can increase the rate of delegation and whether there are self-other discrepancies in self-choice preference. In three studies, we directly manipulated participants' controllability associated with choice through literal instructions (experiment 1) and visual presentation (experiments 2 and 3). We found that participants showed a robust propensity to under-delegate even when they were aware of their impaired controllability associated with self-choice. Moreover, only 40% impairment of controllability (but not 20%) can decrease the propensity to under-delegate. This trend differed between decision-for-self and decision-for-other. These findings suggest that pursuing a sense of control cannot fully explain self-choice preference and appears to occur equally in decisions for oneself as well as for others.

The Protective Effects of Perceived Control During Repeated Exposure to Aversive Stimuli

The ability to perceive and exercise control is a major contributor to our mental and physical wellbeing. When faced with uncontrollable aversive stimuli, organisms develop heightened anxiety and become unwilling to exert effort to avoid the stimuli. In contrast, when faced with controllable aversive stimuli, organisms demonstrate behavioral vigor via avoidance attempts toward trying to seek and exercise control over the environment. As such, controllability confers protective effects against reduced avoidance motivation trigged by aversive environments. These observations beg the question of whether controllability can be potent enough to reverse passivity following repeated exposure to uncontrollable aversive stimuli and how this protective effect is encoded neurally. Human participants performed a Control in Aversive Domain (CAD) task where they were first subjected to a series of repeated uncontrollable aversive stimuli (i.e., aversive tones) across several contexts that were followed by a series of controllable aversive stimuli in a novel context. Faced with persistent uncontrollability, participants significantly reduced their avoidance attempts over time and biased toward giving up. However, the subsequent presence of controllability rescued participants' avoidance behavior. Strikingly, participants who responded more strongly to the protective effects of control also had greater ventromedial prefrontal cortical (vmPFC) activation-a region previously observed to be associated with encoding the subjective value of control. Taken together, these findings highlighted the protective effect conferred by perceived control against passivity and offered insights into the potential role of the vmPFC in controllable environments, with implications for understanding the beneficial influence of perceived control on adaptive behavior.

Functional heterogeneity of perceived control in feedback processing

Perceived control is a fundamental psychological function that can either boost positive affect or buffer negative affect. The current study addressed the electrophysiological correlates underlying perceived control, as exercised by choice, in the processing of feedback valence. Thirty-six participants performed an EEG choice task during which they received positive or negative feedback following choices made either by themselves or by a computer. Perceived control resulted in an enhanced reward positivity for positive feedback but increased theta power for negative feedback. Further, perceived control led to greater feedback P3 amplitude and delta power, regardless of feedback valence. These results suggest functional heterogeneity of perceived control in feedback processing as diverse as magnifying the reward signal, enhancing the need for control and increasing the motivational salience of outcome irrespective of valence.

Internal control beliefs shape positive affect and associated neural dynamics during outcome valuation

Experiencing events as controllable is essential for human well-being. Based on classic psychological theory, we test how internal control beliefs impact the affective valuation of task outcomes, neural dynamics and ensuing behavioral preferences. In three consecutive studies we show that dynamics in positive affect increase, with a qualitative shift towards self-evaluative pride, when agents believe they caused a given outcome. We demonstrate that these outcomes engage brain networks processing self-referential information in the cortical midline. Here, activity in the ventromedial prefrontal cortex tracks outcome valence regarding both success as well as internal control, and covaries with positive affect in response to outcomes. These affective dynamics also relate to increased functional coupling between the ventral striatum and cortical midline structures. Finally, we show that pride predicts preferences for control, even at monetary costs. Our investigations extend recent models of positive affect and well-being, and emphasize that control beliefs drive intrinsic motivation.

Human affect is shaped in part by whether desirable states are achieved under personal control. Here, the authors show that control beliefs affect happiness and pride, and how those effects relate to neural responses in the prefrontal cortex and behavioral preferences for exerting control.

Neural activity in human ventromedial prefrontal cortex reflecting the intention to save reward

Saving behavior usually requires individuals to perform several consecutive choices before collecting the final reward. The overt behavior is preceded by an intention to perform an appropriate choice sequence. We studied saving sequences for which each participant rated the intention numerically as willingness to save. Each sequence resulted in a specific reward amount and thus had a particular value for the participant, which we assessed with a Becker-DeGroot-Marschak auction-like mechanism. Using functional MRI, we found that blood-oxygen-level-dependent signals in human ventromedial prefrontal cortex (vmPFC) correlated with the participant's stated intention before each choice sequence. An adjacent vmPFC region showed graded activation that reflected the value of the sequence. These results demonstrate an involvement of vmPFC in intentional processes preceding sequential economic choices.

Striatal dopamine D2 binding correlates with locus of control: Preliminary evidence from [11C]raclopride Positron Emission Tomography

The ability to exert control has been widely investigated as a hallmark of adaptive behaviour. Dopamine is recognized as the key neuromodulator mediating various control-related processes. The neural mechanisms underlying the subjective perception of being in control, or Locus of Control (LOC) are however less clear. LOC indicates the subjective tendency to attribute environmental outcomes to one's actions (internal LOC) or instead to external incontrollable factors (external LOC). Here we hypothesized that dopamine levels also relate to LOC. Previous work shows that dopamine signaling mediates learning of action-outcome relationships, outcome predictability, and opportunity cost. Prominent theories propose dopamine dysregulation as the key pathogenetic mechanism in schizophrenia and depression. Critically, external LOC is a risk factor for schizophrenia and depression, and predicts increased vulnerability to stress. However, a direct link between LOC and dopamine levels in healthy control had not been demonstrated. The purpose of our study was to investigate this link. Using [¹¹C]raclopride Positron Emission Tomography we tested the relationship between D2 receptor binding in the striatum and LOC (measured with the Rotter Locus of Control scale) in 15 healthy volunteers. Our results show a large and positive correlation: increased striatal D2 binding was associated with External LOC. This finding opens promising avenues for the study of several psychological impairments that have been associated with both dopamine and LOC, such as addiction, schizophrenia, and depression.