Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence

Lesions to Different Regions of the Frontal Cortex Have Dissociable Effects on Voluntary Persistence in Humans

Deciding how long to keep waiting for uncertain future rewards is a complex problem. Previous research has shown that choosing to stop waiting results from an evaluative process that weighs the subjective value of the awaited reward against the opportunity cost of waiting. Activity in the ventromedial prefrontal cortex (vmPFC) tracks the dynamics of this evaluation, while activation in the dorsomedial prefrontal cortex (dmPFC) and anterior insula (AI) ramps up before a decision to quit is made. Here, we provide causal evidence of the necessity of these brain regions for successful performance in a willingness-to-wait task. Twenty-eight participants (20 female and 8 male) with lesions to different regions of the frontal lobe were tested on their ability to adaptively calibrate how long they waited for monetary rewards. We found that participants with lesions to the vmPFC waited less overall, while participants with lesions to the dmPFC and anterior insula were specifically impaired at calibrating their level of persistence to the environment. These behavioral effects were accounted for by systematic differences in parameter estimates from a computational model of task performance.

The amygdala and the pursuit of future rewards

The successful pursuit of future rewards requires forming an internal goal, followed by planning, decision-making, and progress-tracking over multiple steps. The initial step-forming goals and the plans for obtaining them-involves the subjective valuation of an anticipated reward, considering both the reward's properties and associated delay and physical-effort costs. Recent findings indicate individuals similarly evaluate cognitive effort over time (Johnson and Most, 2023). Success and failure in these processes have been linked to differential life outcomes and psychiatric conditions. Here we review evidence from single-neuron recordings and neuroimaging studies that implicate the amygdala-a brain structure long associated with cue-reactivity and emotion-in decision-making and the planned pursuit of future rewards (Grabenhorst et al., 2012, 2016, 2019, 2023;Hernadi et al., 2015;Zangemeister et al., 2016). The main findings are that, in behavioral tasks in which future rewards can be pursued through planning and stepwise decision-making, amygdala neurons prospectively encode the value of anticipated rewards and related behavioral plans. Moreover, amygdala neurons predict the stepwise choices to pursue these rewards, signal progress toward goals, and distinguish internally generated (i.e., self-determined) choices from externally imposed actions. Importantly, amygdala neurons integrate the subjective value of a future reward with delay and effort costs inherent in pursuing it. This neural evidence identifies three key computations of the primate amygdala that underlie the pursuit of future rewards: (1) forming a self-determined internal goal based on subjective reward-cost valuations, (2) defining a behavioral plan for obtaining the goal, (3) executing this plan through stepwise decision-making and progress-tracking. Based on this framework, we suggest that amygdala neurons constitute vulnerabilities for dysfunction that contribute to maladaptive reward pursuit in psychiatric and behavioral conditions. Consequently, amygdala neurons may also represent potential targets for behavioral-change interventions that aim to improve individual decision-making.

The sinking platform test: a novel paradigm to measure persistence in animal models

Persistence is the propensity to maintain goal-directed actions despite adversities. While this temperamental trait is crucial to mitigate depression risk, its neurobiological foundations remain elusive. Developing behavioral tasks to capture persistence in animal models is crucial for understanding its molecular underpinnings. Here, we introduce the Sinking Platform Test (SPT), a novel high-throughput paradigm to measure persistence. Mice were trained to exit a water-filled tank by ascending onto a platform above water level. Throughout the training, mice were also occasionally exposed to "failure trials," during which an operator would submerge a platform right after the mouse climbed onto it, requiring the mouse to reach and ascend a newly introduced platform. Following training, mice were subjected to a 5-min test exclusively consisting of failure trials. Male and female mice exhibited comparable persistence, measured by the number of climbed platforms during the test. Furthermore, this index was increased by chronic administration of fluoxetine or imipramine; conversely, it was reduced by acute and chronic haloperidol. Notably, six weeks of social isolation reduced SPT performance, and this effect was rescued by imipramine treatment over the last two weeks. A 4-week regimen of voluntary wheel running also improved persistence in socially isolated mice. Finally, comparing transcriptomic profiles of the prefrontal cortex of mice with high and low SPT performance revealed significant enrichment of immediate-early genes known to shape susceptibility for chronic stress. These findings highlight the potential of SPT as a promising method to uncover the biological mechanisms of persistence and evaluate novel interventions to enhance this response.

Sex differences in change-of-mind neuroeconomic decision-making is modulated by LINC00473 in medial prefrontal cortex

Changing one's mind is a complex cognitive phenomenon involving a continuous re-appraisal of the trade-off between past costs and future value. Recent work modeling this behavior across species has established associations between aspects of this choice process and their contributions to altered decision-making in psychopathology. Here, we investigated the actions in medial prefrontal cortex (mPFC) neurons of long intergenic non-coding RNA, LINC00473, known to induce stress resilience in a striking sex-dependent manner, but whose role in cognitive function is unknown. We characterized complex decision-making behavior in male and female mice longitudinally in our neuroeconomic foraging paradigm, Restaurant Row, following virus-mediated LINC00473 expression in mPFC neurons. On this task, mice foraged for their primary source of food among varying costs (delays) and subjective value (flavors) while on a limited time-budget during which decisions to accept and wait for rewards were separated into discrete stages of primary commitments and secondary re-evaluations. We discovered important differences in decision-making behavior between female and male mice. LINC00473 expression selectively influenced multiple features of re-evaluative choices, without affecting primary decisions, in female mice only. These behavioral effects included changing how mice (i) cached the value of the passage of time and (ii) weighed their history of economically disadvantageous choices. Both processes were uniquely linked to change-of-mind decisions and underlie the computational bases of distinct aspects of counterfactual thinking. These findings reveal a key bridge between a molecular driver of stress resilience and psychological mechanisms underlying sex-specific decision-making proclivities.

Continuous decision to wait for a future reward is guided by fronto-hippocampal anticipatory dynamics

Deciding whether to wait for a future reward is crucial for surviving in an uncertain world. While seeking rewards, agents anticipate a reward in the present environment and constantly face a trade-off between staying in their environment or leaving it. It remains unclear, however, how humans make continuous decisions in such situations. Here, we show that anticipatory activity in the anterior prefrontal cortex, ventrolateral prefrontal cortex, and hippocampus underpins continuous stay-leave decision-making. Participants awaited real liquid rewards available after tens of seconds, and their continuous decision was tracked by dynamic brain activity associated with the anticipation of a reward. Participants stopped waiting more frequently and sooner after they experienced longer delays and received smaller rewards. When the dynamic anticipatory brain activity was enhanced in the anterior prefrontal cortex, participants remained in their current environment, but when this activity diminished, they left the environment. Moreover, while experiencing a delayed reward in a novel environment, the ventrolateral prefrontal cortex and hippocampus showed anticipatory activity. Finally, the activity in the anterior prefrontal cortex and ventrolateral prefrontal cortex was enhanced in participants adopting a leave strategy, whereas those remaining stationary showed enhanced hippocampal activity. Our results suggest that fronto-hippocampal anticipatory dynamics underlie continuous decision-making while anticipating a future reward.

Lesions to different regions of frontal cortex have dissociable effects on voluntary persistence

Deciding how long to keep waiting for uncertain future rewards is a complex problem. Previous research has shown that choosing to stop waiting results from an evaluative process that weighs the subjective value of the awaited reward against the opportunity cost of waiting. In functional neuroimaging data, activity in ventromedial prefrontal cortex (vmPFC) tracks the dynamics of this evaluation, while activation in the dorsomedial prefrontal cortex (dmPFC) and anterior insula (AI) ramps up before a decision to quit is made. Here, we provide causal evidence of the necessity of these brain regions for successful performance in a willingness-to-wait task. 28 participants with frontal lobe lesions were tested on their ability to adaptively calibrate how long they waited for monetary rewards. We grouped the participants based on the location of their lesions, which were primarily in ventromedial, dorsomedial, or lateral parts of their prefrontal cortex (vmPFC, dmPFC, and lPFC, respectively), or in the anterior insula. We compared the performance of each subset of lesion participants to behavior in a control group without lesions (n=18). Finally, we fit a newly developed computational model to the data to glean a more mechanistic understanding of how lesions affect the cognitive processes underlying choice. We found that participants with lesions to the vmPFC waited less overall, while participants with lesions to the dmPFC and anterior insula were specifically impaired at calibrating their level of persistence to the environment. These behavioral effects were accounted for by systematic differences in parameter estimates from a computational model of task performance: while the vmPFC group showed reduced initial willingness to wait, lesions to the dmPFC/anterior insula were associated with slower learning from negative feedback. These findings corroborate the notion that failures of persistence can be driven by sophisticated cost-benefit analyses rather than lapses in self-control. They also support the functional specialization of different parts of the prefrontal cortex in service of voluntary persistence.

Statistical information about reward timing is insufficient for promoting optimal persistence decisions

When deciding how long to keep waiting for delayed rewards that will arrive at an uncertain time, different distributions of possible reward times dictate different optimal strategies for maximizing reward. When reward timing distributions are heavy-tailed (e.g., waiting on hold) there is a point at which waiting is no longer advantageous because the opportunity cost of waiting is too high. Alternatively, when reward timing distributions have more predictable timing (e.g., uniform), it is advantageous to wait as long as necessary for the reward. Although people learn to approximate optimal strategies, little is known about how this learning occurs. One possibility is that people learn a general cognitive representation of the probability distribution that governs reward timing and then infer a strategy from that model of the environment. Another possibility is that they learn an action policy in a way that depends more narrowly on direct task experience, such that general knowledge of the reward timing distribution is insufficient for expressing the optimal strategy. Here, in a series of studies in which participants decided how long to persist for delayed rewards before quitting, we provided participants with information about the reward timing distribution in several ways. Whether the information was provided through counterfactual feedback (Study 1), previous exposure (Studies 2a and 2b), or description (Studies 3a and 3b), it did not obviate the need for direct, feedback-driven learning in a decision context. Therefore, learning when to quit waiting for delayed rewards might depend on task-specific experience, not solely on probabilistic reasoning.

Positive affect treatment targets reward sensitivity: A randomized controlled trial

OBJECTIVE

Determine whether a novel psychosocial treatment for positive affect improves clinical status and reward sensitivity more than a form of cognitive behavioral therapy that targets negative affect and whether improvements in reward sensitivity correlate with improvements in clinical status.

METHOD

In this assessor-blinded, parallel-group, multisite, two-arm randomized controlled clinical superiority trial, 85 treatment-seeking adults with severely low positive affect, moderate-to-severe depression or anxiety, and functional impairment received 15 weekly individual therapy sessions of positive affect treatment (PAT) or negative affect treatment (NAT). Clinical status measures were self-reported positive affect, interviewer-rated anhedonia, and self-reported depression and anxiety. Target measures were eleven physiological, behavioral, cognitive, and self-report measures of reward anticipation-motivation, response to reward attainment, and reward learning. All analyses were intent-to-treat.

RESULTS

Compared to NAT, individuals receiving PAT achieved superior improvements in the multivariate clinical status measures at posttreatment, b = .37, 95% CI [.15, .59], t(109) = 3.34, p = .001, q = .004, d = .64. Compared to NAT, individuals receiving PAT also achieved higher multivariate reward anticipation-motivation, b = .21, 95% CI [.05, .37], t(268) = 2.61, p = .010, q = .020, d = .32, and higher multivariate response to reward attainment, b = .24, 95% CI [.02, .45], t(266) = 2.17, p = .031, q = .041, d = .25, at posttreatment. Measures of reward learning did not differ between the two groups. Improvements in reward anticipation-motivation and in response to reward attainment correlated with improvements in the clinical status measures.

CONCLUSIONS

Targeting positive affect results in superior improvements in clinical status and reward sensitivity than targeting negative affect. This is the first demonstration of differential target engagement across two psychological interventions for anxious or depressed individuals with low positive affect. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Neural dynamics underlying self-control in the primate subthalamic nucleus

The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost-benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.

Pupillometric evidence for a temporal expectations-based account of persistence under temporal uncertainty

People often quit waiting for delayed rewards when the exact timing of those rewards is uncertain. This behavior often has been attributed to self-control failure. Another possibility is that quitting is the result of a rational decision-making process in the face of uncertainty, based on the decision-maker's expectations about the possible arrival times of the awaited reward. There are forms of temporal expectations (e.g., heavy-tailed) under which the expected time remaining until a reward arrives actually increases as time elapses. In those cases, the rational strategy is to quit waiting when the expected reward is no longer worth the expected time remaining. To arbitrate between the "limited self-control" and "temporal expectations" accounts of persistence, we measured pupil diameter during a persistence task, as a physiological marker of surprise (phasic responses) and effort (pre-decision diameter). Phasic pupil responses were elevated in response to reward receipt. Critically, the extent to which pupils dilated following rewards depended on the delay: people showed larger pupillary surprise responses the more delayed the reward was. This result suggests that people expect the reward less the longer they wait for it-a form of temporal expectations under which limiting persistence is rational. Moreover, predecision pupil diameter before quit events was not associated with how long the participant had been waiting, but rather, depended on how atypical the quit decision was compared with the participant's usual behavior. These data provide physiological evidence for a temporal expectations account of persistence under temporal uncertainty.

A New Explanation for the Frog-in-the-Pan Phenomenon Based on the Cognitive-Evolutionary Model of Surprise

The frog-in-the-pan (FIP) phenomenon suggests that investors are more sensitive to abrupt price changes than gradual price changes in the stock market. Based on the cognitive-evolutionary model of surprise and the reinforcement learning model, this paper provides a new explanation for the FIP phenomenon in that this phenomenon could be explained by the elicitation of surprise emotion. We predict that when a change substantially and abruptly occurs, the significant prediction error triggers participants' surprise, which makes participants more sensitive to the change. To ascertain these hypotheses, we recruited 109 participants and compared participants' learning rates and surprise responses under different contexts. We observed that participants' learning rate soared when the prediction error was large enough to trigger surprise emotion under abruptly changed conditions and confirmed that the FIP phenomenon could be explained by the elicitation of surprise emotion. In a word, this research demonstrates the significant role of surprise emotion in the decision-making process.

Revealing human sensitivity to a latent temporal structure of changes

Precisely timed behavior and accurate time perception plays a critical role in our everyday lives, as our wellbeing and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modeling, we found that human subjects' beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in the active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry.

Delay of gratification dissociates cognitive control and valuation brain regions in healthy young adults

Delay of gratification (DofG) refers to an inter-temporal choice phenomenon that is of great interest in many domains, including animal learning, cognitive development, economic decision-making, and executive control. Yet experimental tools for investigating DofG in human adults are almost non-existent, and as a consequence, very little is known regarding the brain basis of core DofG behaviors. Here, we utilize a novel DofG paradigm, adapted for use in neuroimaging contexts, to examine event-related changes in neural activity as healthy young adult participants made repeated choices to continue waiting for a delayed reward, rather than take an immediately available one of lesser value. On DofG trials, choose-to-wait events were associated with increased activation in fronto-parietal and cingulo-opercular regions associated with cognitive control. Activity in the right lateral prefrontal cortex (PFC) was also associated with individual variability in task performance and strategy. Fronto-parietal activity was clearly dissociable from that observed in ventromedial PFC, as this latter region exhibited a ramping-up pattern of activity during the waiting period prior to reward delivery. Ventromedial PFC ramping activity dynamics were further selective to DofG trials associated with increased future reward rate, consistent with the involvement of this region in subjective reward valuation that incorporates higher-order task structure. These results provide important initial validation of this experimental paradigm as a useful tool for investigating and isolating unique DofG neural mechanisms, which can now be utilized to study a wide-variety of populations and task factors.

Choices favoring cognitive effort in a foraging environment decrease when multiple forms of effort and delay are interleaved

Cognitive and physical effort are typically regarded as costly, but demands for effort also seemingly boost the appeal of prospects under certain conditions. One contextual factor that might influence choices for or against effort is the mix of different types of demand a decision maker encounters in a given environment. In two foraging experiments, participants encountered prospective rewards that required equally long intervals of cognitive effort, physical effort, or unfilled delay. Monetary offers varied per trial, and the two experiments differed in whether the type of effort or delay cost was the same on every trial, or varied across trials. When each participant faced only one type of cost, cognitive effort persistently produced the highest acceptance rate compared to trials with an equivalent period of either physical effort or unfilled delay. We theorized that if cognitive effort were intrinsically rewarding, we would observe the same pattern of preferences when participants foraged for varying cost types in addition to rewards. Contrary to this prediction, in the second experiment, an initially higher acceptance rate for cognitive effort trials disappeared over time amid an overall decline in acceptance rates as participants gained experience with all three conditions. Our results indicate that cognitive demands may reduce the discounting effect of delays, but not because decision makers assign intrinsic value to cognitive effort. Rather, the results suggest that a cognitive effort requirement might influence contextual factors such as subjective delay duration estimates, which can be recalibrated if multiple forms of demand are interleaved.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

Local and distributed cortical markers of effort expenditure during sustained goal pursuit

The adaptive adjustment of behavior in pursuit of desired goals is critical for survival. To accomplish this complex feat, individuals must weigh the potential benefits of a given action against time, energy, and resource costs. Here, we examine brain responses associated with willingness to exert physical effort during the sustained pursuit of desired goals. Our analyses reveal a distributed pattern of brain activity in aspects of ventral medial prefrontal cortex that tracks with trial-level variability in effort expenditure. Indicating the brain represents echoes of effort at the point of feedback, whole-brain searchlights identified signals reflecting past effort expenditure in medial and lateral prefrontal cortices, encompassing broad swaths of frontoparietal and dorsal attention networks. These data have important implications for our understanding of how the brain's valuation mechanisms contend with the complexity of real-world dynamic environments with relevance for the study of behavior across health and disease.

Prefrontal-Striatal Mechanisms of Behavioral Impulsivity During Consumption of Delayed Real Liquid Rewards

Intertemporal choice involves the evaluation of future rewards and reflects behavioral impulsivity. After choosing a delayed reward in an intertemporal choice, a behavioral agent waits for, receives, and then consumes the reward. The current study focused on the consumption of the delayed reward and examined the neural mechanisms of behavioral impulsivity. In humans consuming delayed real liquid rewards in an intertemporal choice, the ventral striatum (VS) showed differential activity between anterior (aVS) and posterior (pVS) regions depending on the degree of behavioral impulsivity. Additionally, impulsive individuals showed activity in the anterior prefrontal cortex (aPFC). An analysis of task-related effective connectivity based on psychophysiological interaction (PPI) revealed that PPI was robust from the aPFC to pVS, but not in the opposite direction. On the other hand, strong bidirectional PPIs were observed between the aVS and pVS, but PPIs from the pVS to aVS were enhanced in impulsive individuals. These results suggest that behavioral impulsivity is reflected in aPFC-VS mechanisms during the consumption of delayed real liquid rewards.

Statistical power or more precise insights into neuro-temporal dynamics? Assessing the benefits of rapid temporal sampling in fMRI

Functional magnetic resonance imaging (fMRI), a non-invasive and widely used human neuroimaging method, is most known for its spatial precision. However, there is a growing interest in its temporal sensitivity. This is despite the temporal blurring of neuronal events by the blood oxygen level dependent (BOLD) signal, the peak of which lags neuronal firing by 4-6 seconds. Given this, the goal of this review is to answer a seemingly simple question - "What are the benefits of increased temporal sampling for fMRI?". To answer this, we have combined fMRI data collected at multiple temporal scales, from 323 to 1000 milliseconds, with a review of both historical and contemporary temporal literature. After a brief discussion of technological developments that have rekindled interest in temporal research, we next consider the potential statistical and methodological benefits. Most importantly, we explore how fast fMRI can uncover previously unobserved neuro-temporal dynamics - effects that are entirely missed when sampling at conventional 1 to 2 second rates. With the intrinsic link between space and time in fMRI, this temporal renaissance also delivers improvements in spatial precision. Far from producing only statistical gains, the array of benefits suggest that the continued temporal work is worth the effort.

Quantifying the subjective cost of self-control in humans

Since Odysseus committed to resisting the Sirens, mechanisms to limit self-control failure have been a central feature of human behavior. Psychologists have long argued that the use of self-control is an effortful process and, more recently, that its failure arises when the cognitive costs of self-control outweigh its perceived benefits. In a similar way, economists have argued that sophisticated choosers can adopt "precommitment strategies" that tie the hands of their future selves in order to reduce these costs. Yet, we still lack an empirical tool to quantify and demonstrate the cost of self-control. Here, we develop and validate an economic decision-making task to quantify the subjective cost of self-control by determining the monetary cost a person is willing to incur in order to eliminate the need for self-control. We find that humans will pay to avoid having to exert self-control in a way that scales with increasing levels of temptation and that these costs appear to be modulated both by motivational incentives and stress exposure. Our psychophysical approach allows us to index moment-to-moment self-control costs at the within-subject level, validating important theoretical work across multiple disciplines and opening avenues of self-control research in healthy and clinical populations.

Adolescent Decision-Making Under Risk: Neural Correlates and Sex Differences

An increased propensity for risk taking is a hallmark of adolescent behavior with significant health and social consequences. Here, we elucidated cortical and subcortical regions associated with risky and risk-averse decisions and outcome evaluation using the Balloon Analog Risk Task in a large sample of adolescents (n = 256, 56% female, age 14 ± 0.6), including the level of risk as a parametric modulator. We also identified sex differences in neural activity. Risky decisions engaged regions that are parts of the salience, dorsal attention, and frontoparietal networks, but only the insula was sensitive to increasing risks in parametric analyses. During risk-averse decisions, the same networks covaried with parametric levels of risk. The dorsal striatum was engaged by both risky and risk-averse decisions, but was not sensitive to escalating risk. Negative-outcome processing showed greater activations than positive-outcome processing. Insula, lateral orbitofrontal cortex, middle, rostral, and superior frontal areas, rostral and caudal anterior cingulate cortex were activated only by negative outcomes, with a subset of regions associated with negative outcomes showing greater activation in females. Taken together, these results suggest that safe decisions are predicted by more accurate neural representation of increasing risk levels, whereas reward-related processes play a relatively minor role.

Dopamine and the interdependency of time perception and reward

Time is a fundamental dimension of our perception of the world and is therefore of critical importance to the organization of human behavior. A corpus of work - including recent optogenetic evidence - implicates striatal dopamine as a crucial factor influencing the perception of time. Another stream of literature implicates dopamine in reward and motivation processes. However, these two domains of research have remained largely separated, despite neurobiological overlap and the apothegmatic notion that "time flies when you're having fun". This article constitutes a review of the literature linking time perception and reward, including neurobiological and behavioral studies. Together, these provide compelling support for the idea that time perception and reward processing interact via a common dopaminergic mechanism.

Neural responses to negative events and subsequent persistence behavior differ in individuals recovering from opioid use disorder compared to controls

Background: Negative emotion is associated with substance craving and use in individuals recovering from substance use disorders, including prescription opioid use disorder (POUD). Decisions to abandon or persist towards a goal after negative emotion-eliciting events, and neural responses that shape such decisions, may be important in maintaining recovery from POUD.Objectives: We examined differences in neural responses to negative events and subsequent persistence decisions in individuals recovering from POUD without a history of a substance use disorder. Methods: 20 individuals with POUD (POUD group: 4 females, abstinent 2-3 weeks after admission to an inpatient treatment facility post-detoxification, no other substance use disorder), and 20 individuals with no substance use history (control group: 6 females) completed a persistence-after-setbacks task during functional magnetic resonance imaging. Participants advanced along a path toward a reward; after encountering each negative event (i.e., progress-erasing setback), participants made decisions to persist or abandon the path. Persistence decision rates were compared between groups and blood-oxygen-level-dependent signal to negative events was analyzed within a striatum region of interest (ROI) as well as whole-brain.Results: The POUD group persisted less (t(38) = 2.293, p = .028, d = .725) and showed lower striatum (left ventral putamen) signal to negative events compared to the control group (p < .05, corrected for striatum ROI).Conclusions: In POUD, neural and behavioral responses to negative events differ from controls. These differences are a target for research to address whether POUD treatment increases persistence and striatum responses to negative events and improves recovery outcomes.

From affective to cognitive processing: Functional organization of the medial frontal cortex

The medial wall of the primate frontal lobe encompasses multiple anatomical subregions. Based on distinct neurophysiological correlates and effects of lesions, individual areas are thought to play unique roles in behavior. Further, evidence suggests that dysfunction localized to specific subregions is commonly found in different neuropsychiatric disorders. The neurobiological underpinnings of these disorders, however, remain far from clear. Here, to better understand the functions of medial frontal cortex (MFC) and its role in psychiatric disease, we focus on its functional organization. We describe the emerging pattern in which more dorsal regions subserve temporally extended cognitive functions and more ventral regions predominantly subserve affective functions. We focus on two specific domains, decision-making and social cognition, that require integration across emotion and cognition. In each case, we discuss the current understanding of the functions believed to depend on subregions of MFC as a stepping-stone to speculate on how they might work in unison. We conclude with an overview of how symptoms of certain psychiatric disorders relate to our understanding of MFC functional organization and how further discovery could fuel advances in circuit-based therapies.

Self-Controlled Choice Arises from Dynamic Prefrontal Signals That Enable Future Anticipation

Self-control allows humans the patience necessary to maximize reward attainment in the future. Yet it remains elusive when and how the preference to self-controlled choice is formed. We measured brain activity while female and male humans performed an intertemporal choice task in which they first received delayed real liquid rewards (forced-choice trial), and then made a choice between the reward options based on the experiences (free-choice trial). We found that, while subjects were awaiting an upcoming reward in the forced-choice trial, the anterior prefrontal cortex (aPFC) tracked a dynamic signal reflecting the pleasure of anticipating the future reward. Importantly, this prefrontal signal was specifically observed in self-controlled individuals, and moreover, interregional negative coupling between the prefrontal region and the ventral striatum (VS) became stronger in those individuals. During consumption of the liquid rewards, reduced ventral striatal activity predicted self-controlled choices in the subsequent free-choice trials. These results suggest that a well-coordinated prefrontal-striatal mechanism during the reward experience shapes preferences regarding the future self-controlled choice.SIGNIFICANCE STATEMENT Anticipating future desirable events is a critical mental function that guides self-controlled behavior in humans. When and how are the self-controlled choices formed in the brain? We monitored brain activity while humans awaited a real liquid reward that became available in tens of seconds. We found that the frontal polar cortex tracked temporally evolving signals reflecting the pleasure of anticipating the future reward, which was enhanced in self-controlled individuals. Our results highlight the contribution of the fronto-polar cortex to the formation of self-controlled preferences, and further suggest that future prospect in the prefrontal cortex (PFC) plays an important role in shaping future choice behavior.

Regional Activity in the Rat Anterior Cingulate Cortex and Insula during Persistence and Quitting in a Physical-Effort Task

As animals carry out behaviors, particularly costly ones, they must constantly assess whether or not to persist in the behavior or quit. The anterior cingulate cortex (ACC) has been shown to assess the value of behaviors and to be especially sensitive to physical effort costs. Complimentary to these functions, the insula is thought to represent the internal state of the animal including factors such as hunger, thirst, and fatigue. Using a novel weight-lifting task for rats, we characterized the local field potential (LFP) activity of the ACC and anterior insula (AI) during effort expenditure. In the task, male rats are challenged to work for sucrose reward, which costs progressively more effort over time to obtain. Rats are able to quit the task at any point. We found modest shifts in LFP theta (7-9 Hz) activity as the task got progressively more difficult in terms of absolute effort expenditure. However, when the LFP data were analyzed based on the relative progress of the rat toward quitting the task, substantial shifts in LFP power in the theta and gamma (55-100 Hz) frequency bands were observed in ACC and AI. Both ACC and AI theta power decreased as the rats got closer to quitting, while ACC and AI gamma power increased. Furthermore, coherency between ACC and AI in the delta (2-4 Hz) range shifted alongside the performance state of the rat. Overall, we show that ACC and AI LFP activity changes correlate to the relative performance state of rats in an effort-based task.

Cingulum and abnormal psychological stress response in schizophrenia

Stress is implicated in many aspects of schizophrenia, including heightened distress intolerance. We examined how affect and microstructure of major brain tracts involved in regulating affect may contribute to distress intolerance in schizophrenia. Patients with schizophrenia spectrum disorders (n = 78) and community controls (n = 95) completed diffusion weighted imaging and performed psychological stress tasks. Subjective affect was collected pre and post stressors. Individuals who did not persist during one or both stress tasks were considered distress intolerant (DI), and otherwise distress tolerant (DT). Fractional anisotropy (FA) of the dorsal cingulum showed a significant diagnosis x DT/DI phenotype interaction (p = 0.003). Post-hoc tests showed dorsal cingulum FA was significantly lower in DI patients compared with DI controls (p < 0.001), but not different between DT groups (p = 0.27). Regarding affect responses to stress, irritability showed the largest stress-related change (p < 0.001), but irritability changes were significantly reduced in DI patients compared to DI controls (p = 0.006). The relationship between irritability change and performance errors also differed among patients (ρ = -0.29, p = 0.011) and controls (ρ = 0.21, p = 0.042). Further modeling highlighted the explanatory power of dorsal cingulum for predicting DI even after performance and irritability were taken into account. Distress intolerance during psychological stress exposure is related to microstructural properties of the dorsal cingulum, a key structure for cognitive control and emotion regulation. In schizophrenia, the affective response to psychological stressors is abnormal, and distress intolerant patients had significantly reduced dorsal cingulum FA compared to distress intolerant controls. The findings provide new insight regarding distress intolerance in schizophrenia.

Foraging optimally in social neuroscience: computations and methodological considerations

Research in social neuroscience has increasingly begun to use the tools of computational neuroscience to better understand behaviour. Such approaches have proven fruitful for probing underlying neural mechanisms. However, little attention has been paid to how the structure of experimental tasks relates to real-world decisions, and the problems that brains have evolved to solve. To go significantly beyond current understanding, we must begin to use paradigms and mathematical models from behavioural ecology, which offer insights into the decisions animals must make successfully in order to survive. One highly influential theory-marginal value theorem (MVT)-precisely characterises and provides an optimal solution to a vital foraging decision that most species must make: the patch-leaving problem. Animals must decide when to leave collecting rewards in a current patch (location) and travel somewhere else. We propose that many questions posed in social neuroscience can be approached as patch-leaving problems. A richer understanding of the neural mechanisms underlying social behaviour will be obtained by using MVT. In this 'tools of the trade' article, we outline the patch-leaving problem, the computations of MVT and discuss the application to social neuroscience. Furthermore, we consider the practical challenges and offer solutions for designing paradigms probing patch leaving, both behaviourally and when using neuroimaging techniques.

Altered subcallosal and posterior cingulate cortex-based functional connectivity during smoking cue and mental simulation processing in smokers

BACKGROUND

Long-term cigarette smoking induces sensitization of incentive salience and conditioning of contextual cues which involves brain function alteration across multiple regions. Understanding how nicotine affects hub-based functional connectivities involved in affective and cognitive function can help us determine the treatment strategy for nicotine dependence.

METHOD

Functional MRI was conducted on 30 smokers and 30 non-smokers while mentally simulating neutral and smoking hand movements. Smoking cue and mental simulation processing-related changes in functional connectivity strengths of the subcallosal and posterior cingulate cortex (SCC and PCC) with major brain network nodes were examined.

RESULTS

Compared to non-smokers, smokers showed cue-induced SCC functional connectivities which were enhanced with the intraparietal sulcus and reduced with the medial prefrontal cortex. The PCC activation and functional connectivity enhancements with the anterior insula cortex and rostro-lateral prefrontal cortex was found during smoking mental simulation. The PCC-lateral prefrontal cortex functional connectivity correlated with nicotine dependence severity.

CONCLUSION

The present results demonstrate that smokers can be identified by cue-induced SCC functional connectivity strength decline and increment in the default mode and dorsal attention network nodes. However, nicotine dependence was associated with smoking mental simulation-related PCC-lateral prefrontal cortex functional connectivity strength, suggesting that the development of nicotine dependence may depend on the strength of coupling between the default mode network and the central executive network at the cognitive level.

Neural activity in human ventromedial prefrontal cortex reflecting the intention to save reward

Saving behavior usually requires individuals to perform several consecutive choices before collecting the final reward. The overt behavior is preceded by an intention to perform an appropriate choice sequence. We studied saving sequences for which each participant rated the intention numerically as willingness to save. Each sequence resulted in a specific reward amount and thus had a particular value for the participant, which we assessed with a Becker-DeGroot-Marschak auction-like mechanism. Using functional MRI, we found that blood-oxygen-level-dependent signals in human ventromedial prefrontal cortex (vmPFC) correlated with the participant's stated intention before each choice sequence. An adjacent vmPFC region showed graded activation that reflected the value of the sequence. These results demonstrate an involvement of vmPFC in intentional processes preceding sequential economic choices.

Uncovering the 'state': Tracing the hidden state representations that structure learning and decision-making

We review the abstract concept of a 'state' - an internal representation posited by reinforcement learning theories to be used by an agent, whether animal, human or artificial, to summarize the features of the external and internal environment that are relevant for future behavior on a particular task. Armed with this summary representation, an agent can make decisions and perform actions to interact effectively with the world. Here, we review recent findings from the neurobiological and behavioral literature to ask: 'what is a state?' with respect to the internal representations that organize learning and decision making across a range of tasks. We find that state representations include information beyond a straightforward summary of the immediate cues in the environment, providing timing or contextual information from the recent or more distant past, which allows these additional factors to influence decision making and other goal-directed behaviors in complex and perhaps unexpected ways.

Sex differences in neural stress responses and correlation with subjective stress and stress regulation

Emotional stress responses, encompassing both stress reactivity and regulation, have been shown to differ between men and women, but the neural networks supporting these processes remain unclear. The current study used functional neuroimaging (fMRI) to investigate sex differences in neural responses during stress and the sex-specific relationships between these responses and emotional stress responses for men and women. A significant sex by condition interaction revealed that men showed greater stress responses in prefrontal cortex (PFC) regions, whereas women had stronger responses in limbic/striatal regions. Although men and women did not significantly differ in emotional stress reactivity or subjective reports of stress regulation, these responses were associated with distinct neural networks. Higher dorsomedial PFC responses were associated with lower stress reactivity in men, but higher stress reactivity in women. In contrast, while higher ventromedial PFC stress responses were associated with worse stress regulation in men (but better regulation in women), dynamic increases in vmPFC responses during stress were associated with lower stress reactivity in men. Finally, stress-induced hippocampal responses were more adaptive for women: for men, high and dynamically increasing responses in left hippocampus were associated with high stress reactivity, and dynamic increases in the left (but not right) hippocampus were associated with worse stress regulation. Together, these results reveal that men and women engage distinct neural networks during stress, and sex-specific neural stress responses facilitate optimal emotional stress responses.

Why has evolution not selected for perfect self-control?

Self-control refers to the ability to deliberately reject tempting options and instead select ones that produce greater long-term benefits. Although some apparent failures of self-control are, on closer inspection, reward maximizing, at least some self-control failures are clearly disadvantageous and non-strategic. The existence of poor self-control presents an important evolutionary puzzle because there is no obvious reason why good self-control should be more costly than poor self-control. After all, a rock is infinitely patient. I propose that self-control failures result from cases in which well-learned (and thus routinized) decision-making strategies yield suboptimal choices. These mappings persist in the decision-makers' repertoire because they result from learning processes that are adaptive in the broader context, either on the timescale of learning or of evolution. Self-control, then, is a form of cognitive control and the subjective feeling of effort likely reflects the true costs of cognitive control. Poor self-control, in this view, is ultimately a result of bounded optimality. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications.

Local field potentials in dorsal anterior cingulate sulcus reflect rewards but not travel time costs during foraging

To maximise long-term reward rates, foragers deciding when to leave a patch must compute a decision variable that reflects both the immediately available reward and the time costs associated with travelling to the next patch. Identifying the mechanisms that mediate this computation is central to understanding how brains implement foraging decisions. We previously showed that firing rates of dorsal anterior cingulate sulcus neurons incorporate both variables. This result does not provide information about whether integration of information reflected in dorsal anterior cingulate sulcus spiking activity arises locally or whether it is inherited from upstream structures. Here, we examined local field potentials gathered simultaneously with our earlier recordings. In the majority of recording sites, local field potential spectral bands - specifically theta, beta, and gamma frequency ranges - encoded immediately available rewards but not time costs. The disjunction between information contained in spiking and local field potentials can constrain models of foraging-related processing. In particular, given the proposed link between local field potentials and inputs to a brain area, it raises the possibility that local processing within dorsal anterior cingulate sulcus serves to more fully bind immediate reward and time costs into a single decision variable.

Effort and performance in a cooperative activity are boosted by perception of a partner's effort

In everyday life, people must often determine how much time and effort to allocate to cooperative activities. In the current study, we tested the hypothesis that the perception of others' effort investment in a cooperative activity may elicit a sense of commitment, leading people to allocate more time and effort to the activity themselves. We developed an effortful task in which participants were required to move an increasingly difficult bar slider on a screen while simultaneously reacting to the appearance of virtual coins and earn points to share between themselves and their partner. This design allowed us to operationalize commitment in terms of participants' investment of time and effort. Crucially, the cooperative activity could only be performed after a partner had completed a complementary activity which we manipulated to be either easy (Low Effort condition) or difficult (High Effort condition). Our results revealed participants invested more effort, persisted longer and performed better in the High Effort condition, i.e. when they perceived their partner to have invested more effort. These results support the hypothesis that the perception of a partner's effort boosts one's own sense of commitment to a cooperative activity, and consequently also one's willingness to invest time and effort.

Temporal multivariate pattern analysis (tMVPA): A single trial approach exploring the temporal dynamics of the BOLD signal

Background:

fMRI provides spatial resolution that is unmatched by non-invasive neuroimaging techniques. Its temporal dynamics however are typically neglected due to the sluggishness of the hemodynamic signal.

New Methods:

We present temporal multivariate pattern analysis (tMVPA), a method for investigating the temporal evolution of neural representations in fMRI data, computed on single-trial BOLD time-courses, leveraging both spatial and temporal components of the fMRI signal. We implemented an expanding sliding window approach that allows identifying the time-window of an effect.

Results:

We demonstrate that tMVPA can successfully detect condition-specific multivariate modulations over time, in the absence of mean BOLD amplitude differences. Using Monte-Carlo simulations and synthetic data, we quantified family-wise error rate (FWER) and statistical power. Both at the group and single-subject levels, FWER was either at or significantly below 5%. We reached the desired power with 18 subjects and 12 trials for the group level, and with 14 trials in the single-subject scenario.

Comparison with existing methods:

We compare the tMVPA statistical evaluation to that of a linear support vector machine (SVM). SVM outperformed tMVPA with large N and trial numbers. Conversely, tMVPA, leveraging on single trials analyses, outperformed SVM in low N and trials and in a single-subject scenario.

Conclusion:

Recent evidence suggesting that the BOLD signal carries finer-grained temporal information than previously thought, advocates the need for analytical tools, such as tMVPA, tailored to investigate BOLD temporal dynamics. The comparable performance between tMVPA and SVM, a powerful and reliable tool for fMRI, supports the validity of our technique.

The anatomy of apathy: A neurocognitive framework for amotivated behaviour

Apathy is a debilitating syndrome associated with many neurological disorders, including several common neurodegenerative diseases such as Parkinson's disease and Alzheimer's disease, and focal lesion syndromes such as stroke. Here, we review neuroimaging studies to identify anatomical correlates of apathy, across brain disorders. Our analysis reveals that apathy is strongly associated with disruption particularly of dorsal anterior cingulate cortex (dACC), ventral striatum (VS) and connected brain regions. Remarkably, these changes are consistent across clinical disorders and imaging modalities. Review of the neuroimaging findings allows us to develop a neurocognitive framework to consider potential mechanisms underlying apathy. According to this perspective, an interconnected group of brain regions - with dACC and VS at its core - plays a crucial role in normal motivated behaviour. Specifically we argue that motivated behaviour requires a willingness to work, to keep working, and to learn what is worth working for. We propose that deficits in any one or more of these processes can lead to the clinical syndrome of apathy, and outline specific approaches to test this hypothesis. A richer neurobiological understanding of the mechanisms underlying apathy should ultimately facilitate development of effective therapies for this disabling condition.

Dissociable neural mechanisms track evidence accumulation for selection of attention versus action

Decision-making is typically studied as a sequential process from the selection of what to attend (e.g., between possible tasks, stimuli, or stimulus attributes) to which actions to take based on the attended information. However, people often process information across these various levels in parallel. Here we scan participants while they simultaneously weigh how much to attend to two dynamic stimulus attributes and what response to give. Regions of the prefrontal cortex track information about the stimulus attributes in dissociable ways, related to either the predicted reward (ventromedial prefrontal cortex) or the degree to which that attribute is being attended (dorsal anterior cingulate cortex, dACC). Within the dACC, adjacent regions track correlates of uncertainty at different levels of the decision, regarding what to attend versus how to respond. These findings bridge research on perceptual and value-based decision-making, demonstrating that people dynamically integrate information in parallel across different levels of decision-making.

The neural systems for perceptual updating

In a constantly changing environment we must adapt to both abrupt and gradual changes to incoming information. Previously, we demonstrated that a distributed network (including the anterior insula and anterior cingulate cortex) was active when participants updated their initial representations (e.g., it's a cat) in a gradually morphing picture task (e.g., now it's a rabbit; Stöttinger et al., 2015). To shed light on whether these activations reflect the proactive decisions to update or perceptual uncertainty, we introduced two additional conditions. By presenting picture morphs twice we controlled for uncertainty in perceptual decision making. Inducing an abrupt shift in a third condition allowed us to differentiate between a proactive decision in uncertainty-driven updating and a reactive decision in surprise-based updating. We replicated our earlier result, showing the robustness of the effect. In addition, we found activation in the anterior insula (bilaterally) and the mid frontal area/ACC in all three conditions, indicative of the importance of these areas in updating of all kinds. When participants were naïve as to the identity of the second object, we found higher activations in the mid-cingulate cortex and cuneus - areas typically associated with task difficulty, in addition to higher activations in the right TPJ most likely reflecting the shift to a new perspective. Activations associated with the proactive decision to update to a new interpretation were found in a network including the dorsal ACC known to be involved in exploration and the endogenous decision to switch to a new interpretation. These findings suggest a general network commonly engaged in all types of perceptual decision making supported by additional networks associated with perceptual uncertainty or updating provoked by either proactive or reactive decision making.

The control of tonic pain by active relief learning

Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain. We used computational modelling of behavioural, physiological and neuroimaging data in two experiments in which subjects learned to terminate tonic pain in static and dynamic escape-learning paradigms. In both studies, we show that active relief-seeking involves a reinforcement learning process manifest by error signals observed in the dorsal putamen. Critically, this system uses an uncertainty ('associability') signal detected in pregenual anterior cingulate cortex that both controls the relief learning rate, and endogenously and parametrically modulates the level of tonic pain. The results define a self-organising learning circuit that reduces ongoing pain when learning about potential relief.

The effects of acute stress on the calibration of persistence

People frequently fail to wait for delayed rewards after choosing them. These preference reversals are sometimes thought to reflect self-control failure. Other times, however, continuing to wait for a delayed reward may be counterproductive (e.g., when reward timing uncertainty is high). Research has demonstrated that people can calibrate how long to wait for rewards in a given environment. Thus, the role of self-control might be to integrate information about the environment to flexibly adapt behavior, not merely to promote waiting. Here we tested effects of acute stress, which has been shown to tax control processes, on persistence, and the calibration of persistence, in young adult human participants. Half the participants (n = 60) performed a task in which persistence was optimal, and the other half (n = 60) performed a task in which it was optimal to quit waiting for reward soon after each trial began. Each participant completed the task either after cold pressor stress or no stress. Stress did not influence persistence or optimal calibration of persistence. Nevertheless, an exploratory analysis revealed an "inverted-U" relationship between cortisol increase and performance in the stress groups, suggesting that choosing the adaptive waiting policy may be facilitated with some stress and impaired with severe stress.

High monetary reward rates and caloric rewards decrease temporal persistence

Temporal persistence refers to an individual's capacity to wait for future rewards, while forgoing possible alternatives. This requires a trade-off between the potential value of delayed rewards and opportunity costs, and is relevant to many real-world decisions, such as dieting. Theoretical models have previously suggested that high monetary reward rates, or positive energy balance, may result in decreased temporal persistence. In our study, 50 fasted participants engaged in a temporal persistence task, incentivised with monetary rewards. In alternating blocks of this task, rewards were delivered at delays drawn randomly from distributions with either a lower or higher maximum reward rate. During some blocks participants received either a caloric drink or water. We used survival analysis to estimate participants' probability of quitting conditional on the delay distribution and the consumed liquid. Participants had a higher probability of quitting in blocks with the higher reward rate. Furthermore, participants who consumed the caloric drink had a higher probability of quitting than those who consumed water. Our results support the predictions from the theoretical models, and importantly, suggest that both higher monetary reward rates and physiologically relevant rewards can decrease temporal persistence, which is a crucial determinant for survival in many species.

Heuristic and optimal policy computations in the human brain during sequential decision-making

Optimal decisions across extended time horizons require value calculations over multiple probabilistic future states. Humans may circumvent such complex computations by resorting to easy-to-compute heuristics that approximate optimal solutions. To probe the potential interplay between heuristic and optimal computations, we develop a novel sequential decision-making task, framed as virtual foraging in which participants have to avoid virtual starvation. Rewards depend only on final outcomes over five-trial blocks, necessitating planning over five sequential decisions and probabilistic outcomes. Here, we report model comparisons demonstrating that participants primarily rely on the best available heuristic but also use the normatively optimal policy. FMRI signals in medial prefrontal cortex (MPFC) relate to heuristic and optimal policies and associated choice uncertainties. Crucially, reaction times and dorsal MPFC activity scale with discrepancies between heuristic and optimal policies. Thus, sequential decision-making in humans may emerge from integration between heuristic and optimal policies, implemented by controllers in MPFC.

Alhough humans often make a series of related decisions, it is unknown whether this is done by relying on optimal or heuristic strategies. Here, the authors show that humans rely on both the best heuristic and the optimal policy, and that these strategies are controlled by parts of the medial prefrontal cortex.

Rat mPFC and M2 Play a Waiting Game (at Different Timescales)

In this issue of Neuron, Murakami et al. (2017) relate neural activity in frontal cortex to stochastic and deterministic components of waiting behavior in rats; they find that mPFC biases waiting time, while M2 is ultimately responsible for trial-to-trial variability in decisions about how long to wait.

Self-Control as Value-Based Choice

Self-control is often conceived as a battle between "hot" impulsive processes and "cold" deliberative ones. Heeding the angel on one shoulder leads to success; following the demon on the other leads to failure. Self-control feels like a duality. What if that sensation is misleading, and, despite how they feel, self-control decisions are just like any other choice? We argue that self-control is a form of value-based choice wherein options are assigned a subjective value and a decision is made through a dynamic integration process. We articulate how a value-based choice model of self-control can capture its phenomenology and account for relevant behavioral and neuroscientific data. This conceptualization of self-control links divergent scientific approaches, allows for more robust and precise hypothesis testing, and suggests novel pathways to improve self-control.

Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice

Recent research has highlighted a distinction between sequential foraging choices and traditional economic choices between simultaneously presented options. This was partly motivated by observations in Kolling, Behrens, Mars, and Rushworth, Science, 336(6077), 95-98 (2012) (hereafter, KBMR) that these choice types are subserved by different circuits, with dorsal anterior cingulate (dACC) preferentially involved in foraging and ventromedial prefrontal cortex (vmPFC) preferentially involved in economic choice. To support this account, KBMR used fMRI to scan human subjects making either a foraging choice (between exploiting a current offer or swapping for potentially better rewards) or an economic choice (between two reward-probability pairs). This study found that dACC better tracked values pertaining to foraging, whereas vmPFC better tracked values pertaining to economic choice. We recently showed that dACC's role in these foraging choices is better described by the difficulty of choosing than by foraging value, when correcting for choice biases and testing a sufficiently broad set of foraging values (Shenhav, Straccia, Cohen, & Botvinick Nature Neuroscience, 17(9), 1249-1254, 2014). Here, we extend these findings in 3 ways. First, we replicate our original finding with a larger sample and a task modified to address remaining methodological gaps between our previous experiments and that of KBMR. Second, we show that dACC activity is best accounted for by choice difficulty alone (rather than in combination with foraging value) during both foraging and economic choices. Third, we show that patterns of vmPFC activity, inverted relative to dACC, also suggest a common function across both choice types. Overall, we conclude that both regions are similarly engaged by foraging-like and economic choice.

Value, search, persistence and model updating in anterior cingulate cortex

Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.

The Neural Representation of Prospective Choice during Spatial Planning and Decisions

We are remarkably adept at inferring the consequences of our actions, yet the neuronal mechanisms that allow us to plan a sequence of novel choices remain unclear. We used functional magnetic resonance imaging (fMRI) to investigate how the human brain plans the shortest path to a goal in novel mazes with one (shallow maze) or two (deep maze) choice points. We observed two distinct anterior prefrontal responses to demanding choices at the second choice point: one in rostrodorsal medial prefrontal cortex (rd-mPFC)/superior frontal gyrus (SFG) that was also sensitive to (deactivated by) demanding initial choices and another in lateral frontopolar cortex (lFPC), which was only engaged by demanding choices at the second choice point. Furthermore, we identified hippocampal responses during planning that correlated with subsequent choice accuracy and response time, particularly in mazes affording sequential choices. Psychophysiological interaction (PPI) analyses showed that coupling between the hippocampus and rd-mPFC increases during sequential (deep versus shallow) planning and is higher before correct versus incorrect choices. In short, using a naturalistic spatial planning paradigm, we reveal how the human brain represents sequential choices during planning without extensive training. Our data highlight a network centred on the cortical midline and hippocampus that allows us to make prospective choices while maintaining initial choices during planning in novel environments.

Using neuroimaging and computational modelling, this study explains how the human brain represents initial versus subsequent choices during spatial planning in novel environments.

We are remarkably adept at inferring the consequences of our actions, even in novel situations. However, the neuronal mechanisms that allow us to plan a sequence of novel choices remain a mystery. One hypothesis is that anterior prefrontal brain regions can jump ahead from an initial decision to evaluate subsequent choices. Here, we examine how the brain represents initial versus subsequent choices of varying difficulty during spatial planning in novel environments. Specifically, participants visually searched for the shortest path to a goal in pictures of novel mazes that contained one or two path junctions. We monitored the participants’ brain activity during the task with functional magnetic resonance imaging (fMRI). We observed, in the anterior prefrontal brain, two distinct responses to demanding choices at the second junction: one in the rostrodorsal medial prefrontal cortex (rd-mPFC), which also signalled less demanding initial choices, and another one in the lateral frontopolar cortex (lFPC), which was only engaged by demanding choices at the second junction. Notably, interactions of the rd-mPFC with the hippocampus, a region associated with memory, increased when planning required extensive deliberation and particularly when planning led to accurate choices. Our findings show how humans can rapidly formulate a plan in novel environments. More broadly, these data uncover potential neural mechanisms underlying how we make inferences about states beyond a current subjective state.