The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection

Volitional mental absorption in meditation: Toward a scientific understanding of advanced concentrative absorption meditation and the case of jhana

Meditation has been integral to human culture for millennia, deeply rooted in various spiritual and contemplative traditions. While the field of contemplative science has made significant steps toward understanding the effects of meditation on health and well-being, there has been little study of advanced meditative states, including those achieved through intense concentration and absorption. We refer to these types of states as advanced concentrative absorption meditation (ACAM), characterized by absorption with the meditation object leading to states of heightened attention, clarity, energy, effortlessness, and bliss. This review focuses on a type of ACAM known as jhana (ACAM-J) due to its well-documented history, systematic practice approach, recurring phenomenological themes, and growing popularity among contemplative scientists and more generally in media and society. ACAM-J encompasses eight layers of deep concentration, awareness, and internal experiences. Here, we describe the phenomenology of ACAM-J and present evidence from phenomenological and neuroscientific studies that highlight their potential applications in contemplative practices, psychological sciences, and therapeutics. We additionally propose theoretical ACAM-J frameworks grounded in current cognitive neuroscientific understanding of meditation and ancient contemplative traditions. We aim to stimulate further research on ACAM more broadly, encompassing advanced meditation including meditative development and meditative endpoints. Studying advanced meditation including ACAM, and specific practices such as ACAM-J, can potentially revolutionize our understanding of consciousness and applications for mental health.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Divergent Subregional Information Processing in Mouse Prefrontal Cortex During Working Memory

Working memory (WM) is a critical cognitive function allowing recent information to be temporarily held in mind to inform future action. This process depends on coordination between key subregions in prefrontal cortex (PFC) and other connected brain areas. However, few studies have examined the degree of functional specialization between these subregions throughout the phases of WM using electrophysiological recordings in freely-moving animals, particularly mice. To this end, we recorded single-units in three neighboring medial PFC (mPFC) subregions in mouse - supplementary motor area (MOs), dorsomedial PFC (dmPFC), and ventromedial (vmPFC) - during a freely-behaving non-match-to-position WM task. We found divergent patterns of task-related activity across the phases of WM. The MOs is most active around task phase transitions and encodes the starting sample location most selectively. Dorsomedial PFC contains a more stable population code, including persistent sample-location-specific firing during a five second delay period. Finally, the vmPFC responds most strongly to reward-related information during the choice phase. Our results reveal anatomically and temporally segregated computation of WM task information in mPFC and motivate more precise consideration of the dynamic neural activity required for WM.

Neural activity ramps in frontal cortex signal extended motivation during learning

Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.

Theta and alpha oscillatory signatures of auditory sensory and cognitive loads during complex listening

The neuronal signatures of sensory and cognitive load provide access to brain activities related to complex listening situations. Sensory and cognitive loads are typically reflected in measures like response time (RT) and event-related potentials (ERPs) components. It's, however, strenuous to distinguish the underlying brain processes solely from these measures. In this study, along with RT- and ERP-analysis, we performed time-frequency analysis and source localization of oscillatory activity in participants performing two different auditory tasks with varying degrees of complexity and related them to sensory and cognitive load. We studied neuronal oscillatory activity in both periods before the behavioral response (pre-response) and after it (post-response). Robust oscillatory activities were found in both periods and were differentially affected by sensory and cognitive load. Oscillatory activity under sensory load was characterized by decrease in pre-response (early) theta activity and increased alpha activity. Oscillatory activity under cognitive load was characterized by increased theta activity, mainly in post-response (late) time. Furthermore, source localization revealed specific brain regions responsible for processing these loads, such as temporal and frontal lobe, cingulate cortex and precuneus. The results provide evidence that in complex listening situations, the brain processes sensory and cognitive loads differently. These neural processes have specific oscillatory signatures and are long lasting, extending beyond the behavioral response.

Lights, fiber, action! A primer on in vivo fiber photometry

Fiber photometry is a key technique for characterizing brain-behavior relationships in vivo. Initially, it was primarily used to report calcium dynamics as a proxy for neural activity via genetically encoded indicators. This generated new insights into brain functions including movement, memory, and motivation at the level of defined circuits and cell types. Recently, the opportunity for discovery with fiber photometry has exploded with the development of an extensive range of fluorescent sensors for biomolecules including neuromodulators and peptides that were previously inaccessible in vivo. This critical advance, combined with the new availability of affordable "plug-and-play" recording systems, has made monitoring molecules with high spatiotemporal precision during behavior highly accessible. However, while opening exciting new avenues for research, the rapid expansion in fiber photometry applications has occurred without coordination or consensus on best practices. Here, we provide a comprehensive guide to help end-users execute, analyze, and suitably interpret fiber photometry studies.

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning

Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step,reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task

Working memory is a fundamental cognitive ability, allowing us to keep information in memory for the time needed to perform a given task. A complex neural circuit fulfills these functions, among which is the anterior cingulate cortex (CG). Functionally and anatomically connected to the medial prefrontal, retrosplenial, midcingulate and hippocampus, as well as motor cortices, CG has been implicated in retrieving appropriate information when needed to select and control appropriate behavior. The role of cingulate cortex in working memory-guided behaviors remains unclear due to the lack of studies reversibly interfering with its activity during specific epochs of working memory. We used eNpHR3.0 to silence cingulate neurons while animals perform a standard delayed non-match to trajectory task, and found that, while not causing an absolute impairment in working memory, silencing cingulate neurons during retrieval decreases the mean performance if compared to silencing during encoding. Such retrieval-associated changes are accompanied by longer delays observed when light is delivered to control animals, when compared to eNpHR3.0+ ones, consistent with an adaptive recruitment of additional cognitive resources.

A stress-sensitive frontostriatal circuit supporting effortful reward-seeking behavior

Effort valuation-a process for selecting actions based on the anticipated value of rewarding outcomes and expectations about the work required to obtain them-plays a fundamental role in decision-making. Effort valuation is disrupted in chronic stress states and is supported by the anterior cingulate cortex (ACC), but the circuit-level mechanisms by which the ACC regulates effort-based decision-making are unclear. Here, we show that ACC neurons projecting to the nucleus accumbens (ACC-NAc) play a critical role in effort valuation behavior in mice. Activity in ACC-NAc cells integrates both reward- and effort-related information, encoding a reward-related signal that scales with effort requirements and is necessary for supporting future effortful decisions. Chronic corticosterone exposure reduces motivation, suppresses effortful reward-seeking, and disrupts ACC-NAc signals. Together, our results delineate a stress-sensitive ACC-NAc circuit that supports effortful reward-seeking behavior by integrating reward and effort signals and reinforcing effort allocation in the service of maximizing reward.

Dopamine-independent effect of rewards on choices through hidden-state inference

Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.

State-transition-free reinforcement learning in chimpanzees (Pan troglodytes)

The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.

The relation between implicit statistical learning and proactivity as revealed by EEG

Environmental events often occur on a probabilistic basis but can sometimes be predicted based on specific cues and thus approached proactively. Incidental statistical learning enables the acquisition of knowledge about probabilistic cue-target contingencies. However, the neural mechanisms of statistical learning about contingencies (SLC), the required conditions for successful learning, and the role of implicit processes in the resultant proactive behavior are still debated. We examined changes in behavior and cortical activity during an SLC task in which subjects responded to visual targets. Unbeknown to them, there were three types of target cues associated with high-, low-, and zero target probabilities. About half of the subjects spontaneously gained explicit knowledge about the contingencies (contingency-aware group), and only they showed evidence of proactivity: shortened response times to predictable targets and enhanced event-related brain responses (cue-evoked P300 and contingent negative variation, CNV) to high probability cues. The behavioral and brain responses were strictly associated on a single-trial basis. Source reconstruction of the brain responses revealed activation of fronto-parietal brain regions associated with cognitive control, particularly the anterior cingulate cortex and precuneus. We also found neural correlates of SLC in the contingency-unaware group, but these were restricted to post-target latencies and visual association areas. Our results document a qualitative difference between explicit and implicit learning processes and suggest that in certain conditions, proactivity may require explicit knowledge about contingencies.

Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control

Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ-1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ-1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ-1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.

Effects of the number of competing responses on neural signatures of pre- and post-response conflict

The optimization of human performance requires the continuous monitoring of behavioral conflicts. According to conflict monitoring theory, the dorsal anterior cingulate cortex registers response conflict which is reflected by two electrophysiological signatures, the N2 and the Ne/ERN. The theory assumes that, if a stimulus activates an incorrect response that competes with the correct response, pre-response conflict on correct trials (reflected by the N2) is enhanced but post-response conflict on error trials (reflected by the Ne/ERN) is reduced. Here, we asked whether response conflict depends on the number of competing incorrect responses activated by a stimulus, that is, whether the N2 is further enhanced and the Ne/ERN is further reduced if two incorrect responses are activated as compared to one. To this end, we used a modified flanker paradigm, in which the two flankers were associated either with the same incorrect response or with different incorrect responses. Our results indicate an increased N2 on correct trials and a reduced Ne/ERN on error trials in the latter as compared to the former condition. These results confirm central predictions of conflict monitoring theory and demonstrate that response conflict is directly related to the number of competing incorrect responses.

Control of sustained attention and impulsivity by Gq-protein signalling in parvalbumin interneurons of the anterior cingulate cortex

The anterior cingulate cortex (ACC) has been implicated in attention deficit hyperactivity disorder (ADHD). More specifically, an appropriate balance of excitatory and inhibitory activity in the ACC may be critical for the control of impulsivity, hyperactivity, and sustained attention which are centrally affected in ADHD. Hence, pharmacological augmentation of parvalbumin- (PV) or somatostatin-positive (Sst) inhibitory ACC interneurons could be a potential treatment strategy. We, therefore, tested whether stimulation of G_q-protein-coupled receptors (G_qPCRs) in these interneurons could improve attention or impulsivity assessed with the 5-choice-serial reaction-time task in male mice. When challenging impulse control behaviourally or pharmacologically, activation of the chemogenetic G_qPCR hM3Dq in ACC PV-cells caused a selective decrease of active erroneous-i.e. incorrect and premature-responses, indicating improved attentional and impulse control. When challenging attention, in contrast, omissions were increased, albeit without extension of reward latencies or decreases of attentional accuracy. These effects largely resembled those of the ADHD medication atomoxetine. Additionally, they were mostly independent of each other within individual animals. G_qPCR activation in ACC PV-cells also reduced hyperactivity. In contrast, if hM3Dq was activated in Sst-interneurons, no improvement of impulse control was observed, and a reduction of incorrect responses was only induced at high agonist levels and accompanied by reduced motivational drive. These results suggest that the activation of G_qPCRs expressed specifically in PV-cells of the ACC may be a viable strategy to improve certain aspects of sustained attention, impulsivity and hyperactivity in ADHD.

Expertise increases planning depth in human gameplay

A hallmark of human intelligence is the ability to plan multiple steps into the future1,2. Despite decades of research3-5, it is still debated whether skilled decision-makers plan more steps ahead than novices6-8. Traditionally, the study of expertise in planning has used board games such as chess, but the complexity of these games poses a barrier to quantitative estimates of planning depth. Conversely, common planning tasks in cognitive science often have a lower complexity9,10 and impose a ceiling for the depth to which any player can plan. Here we investigate expertise in a complex board game that offers ample opportunity for skilled players to plan deeply. We use model fitting methods to show that human behaviour can be captured using a computational cognitive model based on heuristic search. To validate this model, we predict human choices, response times and eye movements. We also perform a Turing test and a reconstruction experiment. Using the model, we find robust evidence for increased planning depth with expertise in both laboratory and large-scale mobile data. Experts memorize and reconstruct board features more accurately. Using complex tasks combined with precise behavioural modelling might expand our understanding of human planning and help to bridge the gap with progress in artificial intelligence.

Predicting Alcohol-Related Memory Problems in Older Adults: A Machine Learning Study with Multi-Domain Features

Memory problems are common among older adults with a history of alcohol use disorder (AUD). Employing a machine learning framework, the current study investigates the use of multi-domain features to classify individuals with and without alcohol-induced memory problems. A group of 94 individuals (ages 50-81 years) with alcohol-induced memory problems (the memory group) were compared with a matched control group who did not have memory problems. The random forests model identified specific features from each domain that contributed to the classification of the memory group vs. the control group (AUC = 88.29%). Specifically, individuals from the memory group manifested a predominant pattern of hyperconnectivity across the default mode network regions except for some connections involving the anterior cingulate cortex, which were predominantly hypoconnected. Other significant contributing features were: (i) polygenic risk scores for AUD, (ii) alcohol consumption and related health consequences during the past five years, such as health problems, past negative experiences, withdrawal symptoms, and the largest number of drinks in a day during the past twelve months, and (iii) elevated neuroticism and increased harm avoidance, and fewer positive "uplift" life events. At the neural systems level, hyperconnectivity across the default mode network regions, including the connections across the hippocampal hub regions, in individuals with memory problems may indicate dysregulation in neural information processing. Overall, the study outlines the importance of utilizing multidomain features, consisting of resting-state brain connectivity data collected ~18 years ago, together with personality, life experiences, polygenic risk, and alcohol consumption and related consequences, to predict the alcohol-related memory problems that arise in later life.

The human cerebellum in reward anticipation and reward outcome processing: An activation likelihood estimation meta-analysis

The cerebellum generates internal prediction models and actively compares anticipated and actual outcomes in order to reach a desired end state. In this process, reward can serve as a reinforcer that shapes internal prediction models, enabling context-appropriate behavior. While the involvement of the cerebellum in reward processing has been established in animals, there is no detailed account of which cerebellar regions are involved in reward anticipation and reward outcome processing in humans. To this end, an activation likelihood estimation meta-analysis of functional neuroimaging studies was performed to investigate cerebellar functional activity patterns associated with reward anticipation and reward outcome processing in healthy adults. Results showed that reward anticipation (k=31) was associated with regional activity in the bilateral anterior lobe, bilateral lobule VI, left Crus I and the posterior vermis, while reward outcome (k=16) was associated with regional activity in the declive and left lobule VI. The findings of this meta-analysis show distinct involvement of the cerebellum in reward anticipation and reward outcome processing as part of a predictive coding routine.

Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning

Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition, there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.

Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia

Negative symptoms, such as lack of motivation or social withdrawal, are highly prevalent and debilitating in patients with schizophrenia. Underlying mechanisms of negative symptoms are incompletely understood, thereby preventing the development of targeted treatments. We hypothesized that in patients with schizophrenia during psychotic remission, impaired influences of both model-based and model-free reward predictions on decision-making ('reward prediction influence', RPI) underlie negative symptoms. We focused on psychotic remission, because psychotic symptoms might confound reward-based decision-making. Moreover, we hypothesized that impaired model-based/model-free RPIs depend on alterations of both associative striatum dopamine synthesis and storage (DSS) and executive functioning. Both factors influence RPI in healthy subjects and are typically impaired in schizophrenia. Twenty-five patients with schizophrenia with pronounced negative symptoms during psychotic remission and 24 healthy controls were included in the study. Negative symptom severity was measured by the Positive and Negative Syndrome Scale negative subscale, model-based/model-free RPI by the two-stage decision task, associative striatum DSS by 18F-DOPA positron emission tomography and executive functioning by the symbol coding task. Model-free RPI was selectively reduced in patients and associated with negative symptom severity as well as with reduced associative striatum DSS (in patients only) and executive functions (both in patients and controls). In contrast, model-based RPI was not altered in patients. Results provide evidence for impaired model-free reward prediction influence as a mechanism for negative symptoms in schizophrenia as well as for reduced associative striatum dopamine and executive dysfunction as relevant factors. Data suggest potential treatment targets for patients with schizophrenia and pronounced negative symptoms.

A neural substrate of sex-dependent modulation of motivation

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Differential processing of decision information in subregions of rodent medial prefrontal cortex

Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.

The association of spouse interactions and emotional learning in interference related to chronic back pain

Social interactions affect individual behaviours, preferences, and attitudes. This is also critical in the context of experiencing pain and expressing pain behaviours, and may relate to learned emotional responses. In this respect, individual variability in the medial prefrontal cortex (mPFC), which is involved in adjusting an organism's behaviour to its environment by evaluating and interpreting information within the context of past experiences, is important. It is critical for selecting suitable behavioural responses within a social environment and may reinforce maladaptation in chronic pain. In our study, we used brain imaging during appetitive and aversive pavlovian conditioning in persons with chronic back pain (CBP), subacute back pain (SABP), and healthy controls (HC), together with information on spouse responses to pain behaviours. We also examined the relationship of these responses with pain-related interference in the patients. Our findings yielded a significant negative association between mPFC responses to appetitive and aversive learning in CBP. We also observed a significant negative association for mPFC responses during aversive learning and distracting spouse responses, and a significant positive association between mPFC responses during appetitive learning and solicitous spouse responses in CBP. Both significantly predicted pain-related interference in the CBP group (explained variance up to 53%). Significant associations were not found for SABP or HC. Our findings support an association between appetitive and aversive pavlovian learning, related brain circuits and spouse responses to pain in CBP, where appetitive and aversive learning processes seem to be differentially involved. This can inform prevention and early intervention in a mechanistic approach.

Calcium activity is a degraded estimate of spikes

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.

Prospective and retrospective values integrated in frontal cortex drive predictive choice

To make a deliberate action in a volatile environment, the brain must frequently reassess the value of each action (action-value). Choice can be initially made from the experience of trial-and-errors, but once the dynamics of the environment is learned, the choice can be made from the knowledge of the environment. The action-values constructed from the experience (retrospective value) and the ones from the knowledge (prospective value) were identified in various regions of the brain. However, how and which neural circuit integrates these values and executes the chosen action remains unknown. Combining reinforcement learning and two-photon calcium imaging, we found that the preparatory activity of neurons in a part of the frontal cortex, the anterior-lateral motor (ALM) area, initially encodes retrospective value, but after extensive training, they jointly encode the retrospective and prospective value. Optogenetic inhibition of ALM preparatory activity specifically abolished the expert mice's predictive choice behavior and returned them to the novice-like state. Thus, the integrated action-value encoded in the preparatory activity of ALM plays an important role to bias the action toward the knowledge-dependent, predictive choice behavior.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

The interaction between depression diagnosis and BMI is related to altered activation pattern in the right inferior frontal gyrus and anterior cingulate cortex during food anticipation

BACKGROUND

Depression and overweight/obesity often cooccur but the underlying neural mechanisms for this bidirectional link are not well understood.

METHODS

In this functional magnetic resonance imaging study, we scanned 54 individuals diagnosed with depressive disorders (DD) and 48 healthy controls (HC) to examine how diagnostic status moderates the relationship between body mass index (BMI) and brain activation during anticipation and pleasantness rating of food versus nonfood stimuli.

RESULTS

We found a significant BMI-by-diagnosis interaction effect on activation in the right inferior frontal gyrus (RIFG) and anterior cingulate cortex (ACC) during food versus nonfood anticipation (p < .0125). Brain activation in these regions was greater in HC with higher BMI than in HC with lower BMI. Individuals with DD showed an opposite pattern of activation. Structural equation modeling revealed that the relationship between BMI, activation in the RIFG and ACC, and participants' desire to eat food items shown in the experiment depended on the diagnostic status.

CONCLUSIONS

Considering that food anticipation is an important component of appetitive behavior and that the RIFG and ACC are involved in emotion regulation, response inhibition and conflict monitoring necessary to control this behavior, we propose that future clinical trials targeting weight loss in DD should investigate whether adequate mental preparation positively affects subsequent food consumption behaviors in these individuals.

Cingulate-motor circuits update rule representations for sequential choice decisions

Anterior cingulate cortex mediates the flexible updating of an animal’s choice responses upon rule changes in the environment. However, how anterior cingulate cortex entrains motor cortex to reorganize rule representations and generate required motor outputs remains unclear. Here, we demonstrate that chemogenetic silencing of the terminal projections of cingulate cortical neurons in secondary motor cortex in the rat disrupts choice performance in trials immediately following rule switches, suggesting that these inputs are necessary to update rule representations for choice decisions stored in the motor cortex. Indeed, the silencing of cingulate cortex decreases rule selectivity of secondary motor cortical neurons. Furthermore, optogenetic silencing of cingulate cortical neurons that is temporally targeted to error trials immediately after rule switches exacerbates errors in the following trials. These results suggest that cingulate cortex monitors behavioral errors and updates rule representations in motor cortex, revealing a critical role for cingulate-motor circuits in adaptive choice behaviors.

The anterior cingulate cortex allows an animal to update its behaviour when the environment changes. In this work, the authors identify a pathway from cingulate to secondary motor cortex, critical for updating motor rules following behavioural errors.

Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task

In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.

Explicit knowledge of task structure is a primary determinant of human model-based action

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.

A Computational Model of Hopelessness and Active-Escape Bias in Suicidality

Currently, psychiatric practice lacks reliable predictive tools and a sufficiently detailed mechanistic understanding of suicidal thoughts and behaviors (STB) to provide timely and personalized interventions. Developing computational models of STB that integrate across behavioral, cognitive and neural levels of analysis could help better understand STB vulnerabilities and guide personalized interventions. To that end, we present a computational model based on the active inference framework. With this model, we show that several STB risk markers - hopelessness, Pavlovian bias and active-escape bias - are interrelated via the drive to maximize one's model evidence. We propose four ways in which these effects can arise: (1) increased learning from aversive outcomes, (2) reduced belief decay in response to unexpected outcomes, (3) increased stress sensitivity and (4) reduced sense of stressor controllability. These proposals stem from considering the neurocircuits implicated in STB: how the locus coeruleus - norepinephrine (LC-NE) system together with the amygdala (Amy), the dorsal prefrontal cortex (dPFC) and the anterior cingulate cortex (ACC) mediate learning in response to acute stress and volatility as well as how the dorsal raphe nucleus - serotonin (DRN-5-HT) system together with the ventromedial prefrontal cortex (vmPFC) mediate stress reactivity based on perceived stressor controllability. We validate the model by simulating performance in an Avoid/Escape Go/No-Go task replicating recent behavioral findings. This serves as a proof of concept and provides a computational hypothesis space that can be tested empirically and be used to distinguish planful versus impulsive STB subtypes. We discuss the relevance of the proposed model for treatment response prediction, including pharmacotherapy and psychotherapy, as well as sex differences as it relates to stress reactivity and suicide risk.

Decision Making as a Learned Skill in Mice and Humans

Attention is a necessary component in many forms of human and animal learning. Numerous studies have described how attention and memory interact when confronted with a choice point during skill learning. In both animal and human studies, pathways have been found that connect the executive and orienting networks of attention to the hippocampus. The anterior cingulate cortex, part of the executive attention network, is linked to the hippocampus via the nucleus reuniens of the thalamus. The parietal cortex, part of the orienting attention network, accesses the hippocampus via the entorhinal cortex. These studies have led to specific predictions concerning the functional role of each pathway in connecting the cortex to the hippocampus. Here, we review some of the predictions arising from these studies. We then discuss potential methods for manipulating the two pathways and assessing the directionality of their functional connection using viral expression techniques in mice. New studies may allow testing of a behavioral model specifying how the two pathways work together during skill learning.

Metastable dynamics of neural circuits and networks

Cortical neurons emit seemingly erratic trains of action potentials or "spikes," and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns, which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this Review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed "metastable" and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal, and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals; and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation, or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.

Open-source, Python-based, hardware and software for controlling behavioural neuroscience experiments

Laboratory behavioural tasks are an essential research tool. As questions asked of behaviour and brain activity become more sophisticated, the ability to specify and run richly structured tasks becomes more important. An increasing focus on reproducibility also necessitates accurate communication of task logic to other researchers. To these ends, we developed pyControl, a system of open-source hardware and software for controlling behavioural experiments comprising a simple yet flexible Python-based syntax for specifying tasks as extended state machines, hardware modules for building behavioural setups, and a graphical user interface designed for efficiently running high-throughput experiments on many setups in parallel, all with extensive online documentation. These tools make it quicker, easier, and cheaper to implement rich behavioural tasks at scale. As important, pyControl facilitates communication and reproducibility of behavioural experiments through a highly readable task definition syntax and self-documenting features. Here, we outline the system's design and rationale, present validation experiments characterising system performance, and demonstrate example applications in freely moving and head-fixed mouse behaviour.

Planning in the brain

Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.

Spatial and Temporal Distribution of Information Processing in the Human Dorsal Anterior Cingulate Cortex

The dorsal anterior cingulate cortex (dACC) is a key node in the human salience network. It has been ascribed motor, pain-processing and affective functions. However, the dynamics of information flow in this complex region and how it responds to inputs remain unclear and are difficult to study using non-invasive electrophysiology. The area is targeted by neurosurgery to treat neuropathic pain. During deep brain stimulation surgery, we recorded local field potentials from this region in humans during a decision-making task requiring motor output. We investigated the spatial and temporal distribution of information flow within the dACC. We demonstrate the existence of a distributed network within the anterior cingulate cortex where discrete nodes demonstrate directed communication following inputs. We show that this network anticipates and responds to the valence of feedback to actions. We further show that these network dynamics adapt following learning. Our results provide evidence for the integration of learning and the response to feedback in a key cognitive region.

Computational models of adaptive behavior and prefrontal cortex

The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus-action associations through rewards; (2) predictive models learning stimulus- and/or action-outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.

A low-cost open-source 5-choice operant box system optimized for electrophysiology and optophysiology in mice

Operant boxes enable the application of complex behavioural paradigms to support circuit neuroscience and drug discovery research. However, commercial operant box systems are expensive and often not optimised for combining behaviour with neurophysiology. Here we introduce a fully open-source Python-based operant-box system in a 5-choice design (pyOS-5) that enables assessment of multiple cognitive and affective functions. It is optimized for fast turn-over between animals, and for testing of tethered mice for simultaneous physiological recordings or optogenetic manipulation. For reward delivery, we developed peristaltic and syringe pumps based on a stepper motor and 3D-printed parts. Tasks are specified using a Python-based syntax implemented on custom-designed printed circuit boards that are commercially available at low cost. We developed an open-source graphical user interface (GUI) and task definition scripts to conduct assays assessing operant learning, attention, impulsivity, working memory, or cognitive flexibility, alleviating the need for programming skills of the end user. All behavioural events are recorded with millisecond resolution, and TTL-outputs and -inputs allow straightforward integration with physiological recordings and closed-loop manipulations. This combination of features realizes a cost-effective, nose-poke-based operant box system that allows reliable circuit-neuroscience experiments investigating correlates of cognition and emotion in large cohorts of subjects.

Alzheimer's Disease and Empathic Abilities: The Proposed Role of the Cingulate Cortex

In recent years there has been increasing interest in examining the role of empathic abilities in Alzheimer’s disease (AD). Empathy, the ability to understand and share another person’s feelings, implies the existence of emotional and cognitive processes and is a pivotal aspect for success in social interactions. In turn, self-empathy is oriented to one’s thoughts and feelings. Decline of empathy and self-empathy can occur during the AD continuum and can be linked to different neuroanatomical pathways in which the cingulate cortex may play a crucial role. Here, we will summarize the involvement of empathic abilities through the AD continuum and further discuss the potential neurocognitive mechanisms that contribute to decline of empathy and self-empathy in AD.

The anterior cingulate cortex directs exploration of alternative strategies

The ability to adjust one's behavioral strategy in complex environments is at the core of cognition. Doing so efficiently requires monitoring the reliability of the ongoing strategy and, when appropriate, switching away from it to evaluate alternatives. Studies in humans and non-human primates have uncovered signals in the anterior cingulate cortex (ACC) that reflect the pressure to switch away from the ongoing strategy, whereas other ACC signals relate to the pursuit of alternatives. However, whether these signals underlie computations that actually underpin strategy switching or merely reflect tracking of related variables remains unclear. Here we provide causal evidence that the rodent ACC actively arbitrates between persisting with the ongoing behavioral strategy and temporarily switching away to re-evaluate alternatives. Furthermore, by individually perturbing distinct output pathways, we establish that the two associated computations-determining whether to switch strategy and committing to the pursuit of a specific alternative-are segregated in the ACC microcircuitry.

The Ouroboros Model, Proposal for Self-Organizing General Cognition Substantiated

The Ouroboros Model has been proposed as a biologically-inspired comprehensive cognitive architecture for general intelligence, comprising natural and artificial manifestations. The approach addresses very diverse fundamental desiderata of research in natural cognition and also artificial intelligence, AI. Here, it is described how the postulated structures have met with supportive evidence over recent years. The associated hypothesized processes could remedy pressing problems plaguing many, and even the most powerful current implementations of AI, including in particular deep neural networks. Some selected recent findings from very different fields are summoned, which illustrate the status and substantiate the proposal.

Distinct roles for dopamine clearance mechanisms in regulating behavioral flexibility

Dopamine plays a crucial role in adaptive behavior, and dysfunctional dopamine is implicated in multiple psychiatric conditions characterized by inflexible or inconsistent choices. However, the precise relationship between dopamine and flexible decision making remains unclear. One reason is that, while many studies have focused on the activity of dopamine neurons, efficient dopamine signaling also relies on clearance mechanisms, notably the dopamine transporter (DAT), which predominates in striatum, and catechol-O-methyltransferase (COMT), which predominates in cortex. The exact locus, extent, and timescale of the effects of DAT and COMT are uncertain. Moreover, there is limited data on how acute disruption of either mechanism affects flexible decision making strategies mediated by cortico-striatal networks. To address these issues, we combined pharmacological modulation of DAT and COMT with electrochemistry and behavior in mice. DAT blockade, but not COMT inhibition, regulated sub-second dopamine release in the nucleus accumbens core, but surprisingly neither clearance mechanism affected evoked release in prelimbic cortex. This was not due to a lack of sensitivity, as both amphetamine and atomoxetine changed the kinetics of sub-second release. In a multi-step decision making task where mice had to respond to reversals in either reward probabilities or the choice sequence to reach the goal, DAT blockade selectively impaired, and COMT inhibition improved, performance after reward reversals, but neither manipulation affected the adaptation of choices after action-state transition reversals. Together, our data suggest that DAT and COMT shape specific aspects of behavioral flexibility by regulating different aspects of the kinetics of striatal and cortical dopamine, respectively.