The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection

Distributed Representations for Cognitive Control in Frontal Medial Cortex

In natural and artificial neural networks, modularity and distributed structure afford complementary but competing benefits. The former allows for hierarchical representations that can flexibly recombine modules to address novel problems, whereas the latter can benefit from less constrained training, potentially uncovering fruitful statistical regularities. Here, we investigate these competing demands in the context of human sequential behavior. First, we explore this setting by comparing the properties of several recurrent neural network models. We find that explicit hierarchical structure by itself fails to provide a critical performance advantage when compared with a "flat" model that does not incorporate hierarchical structure. However, hierarchy appears to facilitate cognitive control processes that support nonroutine behaviors and behaviors that are carried out under computational stress. Second, we compare these models against fMRI data using representational similarity analysis. We find that a model that incorporates so-called wiring costs in the cost function, which produces a hierarchically organized gradient of representational structure across the hidden layer of the neural network, best accounts for fMRI data collected from human participants in a previous study [Holroyd, C. B., Ribas-Fernandes, J. J. F., Shahnazian, D., Silvetti, M., & Verguts, T., Human midcingulate cortex encodes distributed representations of task progress. Proceedings of the National Academy of Sciences, U.S.A., 115, 6398-6403, 2018]. The results reveal that the ACC encodes distributed representations of sequential task context along a rostro-caudal gradient of abstraction: Rostral ACC encodes relatively abstract and temporally extended patterns of activity compared with those encoded by caudal ACC. These results provide insight into the role of ACC in motivation and cognitive control.

An Open-Source Joystick Platform for Investigating Forelimb Motor Control, Auditory-Motor Integration, and Value-Based Decision-Making in Head-Fixed Mice

Investigation of neural processes underlying motor control requires behavioral readouts that capture the richness of actions, including both categorical (choice-based) information and motor execution (kinematics). We present an open-source platform for behavioral training of head-fixed mice that combines a stationary or retractable forelimb-based joystick, sound-presentation system, capacitive lick sensor, and water reward dispenser. The setup allows for the creation of multiple behavioral paradigms, two of which are highlighted here: a two-alternative forced-choice auditory-motor discrimination paradigm and a two-armed bandit value-based decision-making task. In the auditory-motor paradigm, mice learn to report high- or low-frequency tones by pushing or pulling the joystick. In the value-based paradigm, mice learn to push or pull the joystick based on the history of rewarded trials. In addition to reporting categorical choices, this setup provides a rich dataset of motor parameters that reflect components of the underlying learning and decision processes in both of these tasks. These kinematic parameters (including joystick speed and displacement, Fréchet similarity of trajectories, tortuosity, angular standard deviation, and movement vigor) provide key additional insights into the motor execution of choices that are not as readily assessed in other paradigms. The system's flexibility of task design, joystick readout, and ease of construction represent an advance compared with currently available manipulandum tasks in mice. We provide detailed schematics for constructing the setup and protocols for behavioral training using both paradigms, with the hope that this open-source resource is readily adopted by neuroscientists interested in mechanisms of sensorimotor integration, motor control, and choice behavior.

Complementary roles for hippocampus and anterior cingulate in composing continuous choice

Naturalistic, goal directed behavior often requires continuous actions directed at dynamically changing goals. In this context, the closest analogue to choice is a strategic reweighting of multiple goal-specific control policies in response to shifting environmental pressures. To understand the algorithmic and neural bases of choice in continuous contexts, we examined behavior and brain activity in humans performing a continuous prey-pursuit task. Using a newly developed control-theoretic decomposition of behavior, we find pursuit strategies are well described by a meta-controller dictating a mixture of lower-level controllers, each linked to specific pursuit goals. Examining hippocampus and anterior cingulate cortex (ACC) population dynamics during goal switches revealed distinct roles for the two regions in parameterizing continuous controller mixing and meta-control. Hippocampal ensemble dynamics encoded the controller blending dynamics, suggesting it implements a mixing of goal-specific control policies. In contrast, ACC ensemble activity exhibited value-dependent ramping activity before goal switches, linking it to a meta-control process that accumulates evidence for switching goals. Our results suggest that hippocampus and ACC play complementary roles corresponding to a generalizable mixture controller and meta-controller that dictates value dependent changes in controller mixing.

Schemas, reinforcement learning and the medial prefrontal cortex

Schemas are rich and complex knowledge structures about the typical unfolding of events in a context; for example, a schema of a dinner at a restaurant. In this Perspective, we suggest that reinforcement learning (RL), a computational theory of learning the structure of the world and relevant goal-oriented behaviour, underlies schema learning. We synthesize literature about schemas and RL to offer that three RL principles might govern the learning of schemas: learning via prediction errors, constructing hierarchical knowledge using hierarchical RL, and dimensionality reduction through learning a simplified and abstract representation of the world. We then suggest that the orbitomedial prefrontal cortex is involved in both schemas and RL due to its involvement in dimensionality reduction and in guiding memory reactivation through interactions with posterior brain regions. Last, we hypothesize that the amount of dimensionality reduction might underlie gradients of involvement along the ventral-dorsal and posterior-anterior axes of the orbitomedial prefrontal cortex. More specific and detailed representations might engage the ventral and posterior parts, whereas abstraction might shift representations towards the dorsal and anterior parts of the medial prefrontal cortex.

Anterior cingulate cortex in complex associative learning: monitoring action state and action content

Environmental changes necessitate adaptive responses, and thus the ability to monitor one's actions and their connection to specific cues and outcomes is crucial for survival. The anterior cingulate cortex (ACC) is implicated in these processes, yet its precise role in action monitoring and outcome evaluation remains unclear. To investigate this, we developed a novel discrimination-avoidance task for mice, designed with clear temporal separation between actions and outcomes. Our findings show that ACC neurons primarily encode post-action variables over extended periods, reflecting the animal's preceding actions rather than the outcomes or values of those actions. Specifically, we identified two distinct subpopulations of ACC neurons: one encoding the action state (whether an action was taken) and the other encoding the action content (which action was taken). Importantly, increased post-action ACC activity was associated with better performance in subsequent trials. These findings suggest that the ACC supports complex associative learning through extended signaling of rich action-relevant information, thereby bridging cue, action, and outcome associations.

An Open-Source Joystick Platform for Investigating Forelimb Motor Control, Auditory-Motor Integration, and Value-Based Decision-Making in Head-Fixed Mice

Investigation of neural processes underlying motor control requires behavioral readouts that capture the richness of actions, including both categorical (choice-based) information and motor execution (kinematics). We present an open-source platform for behavioral training of head-fixed mice that combines a stationary or retractable forelimb-based joystick, sound-presentation system, capacitive lick sensor, and water reward dispenser. The setup allows for the creation of multiple behavioral paradigms, two of which are highlighted here: a two-alternative forced-choice auditory-motor discrimination paradigm, and a two-armed bandit value-based decision-making task. In the auditory-motor paradigm, mice learn to report high or low frequency tones by pushing or pulling the joystick. In the value-based paradigm, mice learn to push or pull the joystick based on the history of rewarded trials. In addition to reporting categorical choices, this setup provides a rich dataset of motor parameters that reflect components of the underlying learning and decision processes in both of these tasks. These kinematic parameters (including joystick speed and displacement, Fréchet similarity of trajectories, tortuosity, angular standard deviation, and movement vigor) provide key additional insights into the motor execution of choices that are not as readily assessed in other paradigms. The system's flexibility of task design, joystick readout, and ease of construction represent an advance compared to currently available manipulandum tasks in mice. We provide detailed schematics for constructing the setup and protocols for behavioral training using both paradigms, with the hope that this open-source resource is readily adopted by neuroscientists interested in mechanisms of sensorimotor integration, motor control, and choice behavior.

Significance Statement

Behavioral paradigms for experiments in head-restrained mice are important for investigating the relationship between neural activity and behavior. However, behavioral setups are often constrained by high cost, design complexity, and implementation challenges. Here, we present an open-source platform for behavioral training of head-fixed mice using a joystick manipulandum. The setup allows for the creation of multiple behavioral paradigms, including an auditory-motor discrimination paradigm, and a value-based decision-making task. We include detailed instructions for construction and implementation of the entire open-source behavioral platform.

An amygdala-cortical circuit for encoding generalized fear memories

Generalized learning is a fundamental process observed across species, contexts, and sensory modalities that enables animals to use past experiences to adapt to changing conditions. Evidence suggests that the prefrontal cortex (PFC) extracts general features of an experience that can be used across multiple situations. The anterior cingulate cortex (ACC), a region of the PFC, is implicated in generalized fear responses to novel contexts. However, the ACC's role in encoding contextual information is poorly understood, especially under increased threat intensity that promotes generalization. Here, we show that synaptic plasticity within the ACC and signaling from amygdala inputs during fear learning are necessary for generalized fear responses to novel encountered contexts. The ACC did not encode specific fear to the training context, suggesting this region extracts general features of a threatening experience rather than specific contextual information. Together with our previous work, our results demonstrate that generalized learning about threatening contexts is encoded, in part, within an ascending amygdala-cortical circuit, whereas descending ACC projections to the amygdala drive generalized fear responses during exposure to novel contexts. Our results further demonstrate that schematic learning can occur in the PFC after single-trial learning, a process typically attributed to learning over many repeated learning episodes.

Prefrontal transthalamic uncertainty processing drives flexible switching

Making adaptive decisions in complex environments requires appropriately identifying sources of error1,2. The frontal cortex is critical for adaptive decisions, but its neurons show mixed selectivity to task features3 and their uncertainty estimates4, raising the question of how errors are attributed to their most likely causes. Here, by recording neural responses from tree shrews (Tupaia belangeri) performing a hierarchical decision task with rule reversals, we find that the mediodorsal thalamus independently represents cueing and rule uncertainty. This enables the relevant thalamic population to drive prefrontal reconfiguration following a reversal by appropriately attributing errors to an environmental change. Mechanistic dissection of behavioural switching revealed a transthalamic pathway for cingulate cortical error monitoring5,6 to reconfigure prefrontal executive control7. Overall, our work highlights a potential role for the thalamus in demixing cortical signals while providing a low-dimensional pathway for cortico-cortical communication.

A cellular basis for mapping behavioural structure

To flexibly adapt to new situations, our brains must understand the regularities in the world, as well as those in our own patterns of behaviour. A wealth of findings is beginning to reveal the algorithms that we use to map the outside world1-6. However, the biological algorithms that map the complex structured behaviours that we compose to reach our goals remain unknown. Here we reveal a neuronal implementation of an algorithm for mapping abstract behavioural structure and transferring it to new scenarios. We trained mice on many tasks that shared a common structure (organizing a sequence of goals) but differed in the specific goal locations. The mice discovered the underlying task structure, enabling zero-shot inferences on the first trial of new tasks. The activity of most neurons in the medial frontal cortex tiled progress to goal, akin to how place cells map physical space. These 'goal-progress cells' generalized, stretching and compressing their tiling to accommodate different goal distances. By contrast, progress along the overall sequence of goals was not encoded explicitly. Instead, a subset of goal-progress cells was further tuned such that individual neurons fired with a fixed task lag from a particular behavioural step. Together, these cells acted as task-structured memory buffers, implementing an algorithm that instantaneously encoded the entire sequence of future behavioural steps, and whose dynamics automatically computed the appropriate action at each step. These dynamics mirrored the abstract task structure both on-task and during offline sleep. Our findings suggest that schemata of complex behavioural structures can be generated by sculpting progress-to-goal tuning into task-structured buffers of individual behavioural steps.

Hidden state inference requires abstract contextual representations in the ventral hippocampus

The ability to use subjective, latent contextual representations to influence decision-making is crucial for everyday life. The hippocampus is hypothesized to bind together otherwise abstract combinations of stimuli to represent such latent contexts, to support the process of hidden state inference. Yet evidence for a role of the hippocampus in hidden state inference remains limited. We found that the ventral hippocampus is required for mice to perform hidden state inference during a two-armed bandit task. Hippocampal neurons differentiate the two abstract contexts required for this strategy in a manner similar to the differentiation of spatial locations, and their activity is essential for appropriate dopamine dynamics. These findings offer insight into how latent contextual information is used to optimize decisions, and they emphasize a key role for the hippocampus in hidden state inference.

Global neural encoding of behavioral strategies in mice during perceptual decision-making task with two different sensory patterns

When a simple model-free strategy does not provide sufficient outcomes, an inference-based strategy estimating a hidden task structure becomes essential for optimizing choices. However, the neural circuitry involved in inference-based strategies is still unclear. We developed a tone frequency discrimination task in head-fixed mice in which the tone category of the current trial depended on the category of the previous trial. When the tone category was repeated, the mice continued using the default model-free strategy, as well as when the tone was randomly presented, to bias choices. In contrast, when the tone was alternated, the default strategy gradually shifted to a hybrid of model-free and inference-based strategies, although we did not observe distinct strategy changes. Brain-wide electrophysiological recording suggested that the neural activity of the frontal and sensory cortices, hippocampus, and striatum was correlated with the reward expectation in different task conditions, suggesting the global encoding of multiple strategies in the brain.

A normative framework dissociates need and motivation in hypothalamic neurons

Physiological needs evoke motivational drives that produce natural behaviors for survival. In previous studies, the temporally intertwined dynamics of need and motivation have made it challenging to differentiate these two components. On the basis of classic homeostatic theories, we established a normative framework to derive computational models for need-encoding and motivation-encoding neurons. By combining the model-based predictions and naturalistic experimental paradigms, we demonstrated that agouti-related peptide (AgRP) and lateral hypothalamic leptin receptor (LHLepR) neuronal activities encode need and motivation, respectively. Our model further explains the difference in the dynamics of appetitive behaviors induced by optogenetic stimulation of AgRP or LHLepR neurons. Our study provides a normative modeling framework that explains how hypothalamic neurons separately encode need and motivation in the mammalian brain.

An Active-Inference Approach to Second-Person Neuroscience

Social neuroscience has often been criticized for approaching the investigation of the neural processes that enable social interaction and cognition from a passive, detached, third-person perspective, without involving any real-time social interaction. With the emergence of second-person neuroscience, investigators have uncovered the unique complexity of neural-activation patterns in actual, real-time interaction. Social cognition that occurs during social interaction is fundamentally different from that unfolding during social observation. However, it remains unclear how the neural correlates of social interaction are to be interpreted. Here, we leverage the active-inference framework to shed light on the mechanisms at play during social interaction in second-person neuroscience studies. Specifically, we show how counterfactually rich mutual predictions, real-time bodily adaptation, and policy selection explain activation in components of the default mode, salience, and frontoparietal networks of the brain, as well as in the basal ganglia. We further argue that these processes constitute the crucial neural processes that underwrite bona fide social interaction. By placing the experimental approach of second-person neuroscience on the theoretical foundation of the active-inference framework, we inform the field of social neuroscience about the mechanisms of real-life interactions. We thereby contribute to the theoretical foundations of empirical second-person neuroscience.

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning

Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step, reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.

Divergent subregional information processing in mouse prefrontal cortex during working memory

Working memory (WM) is a critical cognitive function allowing recent information to be temporarily held in mind to inform future action. This process depends on coordination between prefrontal cortex (PFC) subregions and other connected brain areas. However, few studies have examined the degree of functional specialization between these subregions throughout WM using electrophysiological recordings in freely-moving mice. Here we record single-units in three neighboring mouse medial PFC (mPFC) subregions-supplementary motor area (MOs), dorsomedial PFC (dmPFC), and ventromedial (vmPFC)-during a freely-behaving non-match-to-position WM task. The MOs is most active around task phase transitions, when it transiently represents the starting sample location. Dorsomedial PFC contains a stable population code, including persistent sample-location-specific firing during the delay period. Ventromedial PFC responds most strongly to reward-related information during choices. Our results reveal subregionally segregated WM computation in mPFC and motivate more precise consideration of the dynamic neural activity required for WM.

Understanding dual process cognition via the minimum description length principle

Dual-process theories play a central role in both psychology and neuroscience, figuring prominently in domains ranging from executive control to reward-based learning to judgment and decision making. In each of these domains, two mechanisms appear to operate concurrently, one relatively high in computational complexity, the other relatively simple. Why is neural information processing organized in this way? We propose an answer to this question based on the notion of compression. The key insight is that dual-process structure can enhance adaptive behavior by allowing an agent to minimize the description length of its own behavior. We apply a single model based on this observation to findings from research on executive control, reward-based learning, and judgment and decision making, showing that seemingly diverse dual-process phenomena can be understood as domain-specific consequences of a single underlying set of computational principles.

Dynamic ensemble balance in direct- and indirect-pathway striatal projection neurons underlying decision-related action selection

The posterior dorsal striatum (pDS) plays an essential role in sensory-guided decision-making. However, it remains unclear how the antagonizing direct- and indirect-pathway striatal projection neurons (dSPNs and iSPNs) work in concert to support action selection. Here, we employed deep-brain two-photon imaging to investigate pathway-specific single-neuron and population representations during an auditory-guided decision-making task. We found that the majority of pDS projection neurons predominantly encode choice information. Both dSPNs and iSPNs comprise divergent subpopulations of comparable sizes representing competing choices, rendering a multi-ensemble balance between the two pathways. Intriguingly, such ensemble balance displays a dynamic shift during the decision period: dSPNs show a significantly stronger preference for the contraversive choice than iSPNs. This dynamic shift is further manifested in the inter-neuronal coactivity and population trajectory divergence. Our results support a balance-shift model as a neuronal population mechanism coordinating the direct and indirect striatal pathways for eliciting selected actions during decision-making.

Strategic stabilization of arousal boosts sustained attention

Arousal and motivation interact to profoundly influence behavior. For example, experience tells us that we have some capacity to control our arousal when appropriately motivated, such as staying awake while driving a motor vehicle. However, little is known about how arousal and motivation jointly influence decision computations, including if and how animals, such as rodents, adapt their arousal state to their needs. Here, we developed and show results from an auditory, feature-based, sustained-attention task with intermittently shifting task utility. We use pupil size to estimate arousal across a wide range of states and apply tailored signal-detection theoretic, hazard function, and accumulation-to-bound modeling approaches in a large cohort of mice. We find that pupil-linked arousal and task utility both have major impacts on multiple aspects of task performance. Although substantial arousal fluctuations persist across utility conditions, mice partially stabilize their arousal near an intermediate and optimal level when task utility is high. Behavioral analyses show that multiple elements of behavior improve during high task utility and that arousal influences some, but not all, of them. Specifically, arousal influences the likelihood and timescale of sensory evidence accumulation but not the quantity of evidence accumulated per time step while attending. In sum, the results establish specific decision-computational signatures of arousal, motivation, and their interaction in attention. So doing, we provide an experimental and analysis framework for studying arousal self-regulation in neurotypical brains and in diseases such as attention-deficit/hyperactivity disorder.

Striatal indirect pathway mediates hesitation

Determining the best possible action in an uncertain situation is often challenging, and organisms frequently need extra time to deliberate. This pause in behavior in response to uncertainty - also known as hesitation - commonly occurs in many aspects of daily life, yet its neural circuits are poorly understood. Here we present the first experimental paradigm that reliably evokes hesitation in mice. Using cell-type specific electrophysiology and optogenetics, we show that indirect, but not direct, pathway spiny projection neurons specifically in the dorsomedial striatum mediate hesitation. These data indicate that the basal ganglia circuits controlling the pausing involved in cognitive processes like hesitation are distinct from those that control other types of behavioral inhibition, such as cue-induced stopping.

Prediction-error signals in anterior cingulate cortex drive task-switching

Task-switching is a fundamental cognitive ability that allows animals to update their knowledge of current rules or contexts. Detecting discrepancies between predicted and observed events is essential for this process. However, little is known about how the brain computes cognitive prediction-errors and whether neural prediction-error signals are causally related to task-switching behaviours. Here we trained mice to use a prediction-error to switch, in a single trial, between responding to the same stimuli using two distinct rules. Optogenetic silencing and un-silencing, together with widefield and two-photon calcium imaging revealed that the anterior cingulate cortex (ACC) was specifically required for this rapid task-switching, but only when it exhibited neural prediction-error signals. These prediction-error signals were projection-target dependent and were larger preceding successful behavioural transitions. An all-optical approach revealed a disinhibitory interneuron circuit required for successful prediction-error computation. These results reveal a circuit mechanism for computing prediction-errors and transitioning between distinct cognitive states.

Updating predictions in a complex repertoire of actions and its neural representation

Even though actions we observe in everyday life seem to unfold in a continuous manner, they are automatically divided into meaningful chunks, that are single actions or segments, which provide information for the formation and updating of internal predictive models. Specifically, boundaries between actions constitute a hub for predictive processing since the prediction of the current action comes to an end and calls for updating of predictions for the next action. In the current study, we investigated neural processes which characterize such boundaries using a repertoire of complex action sequences with a predefined probabilistic structure. Action sequences consisted of actions that started with the hand touching an object (T) and ended with the hand releasing the object (U). These action boundaries were determined using an automatic computer vision algorithm. Participants trained all action sequences by imitating demo videos. Subsequently, they returned for an fMRI session during which the original action sequences were presented in addition to slightly modified versions thereof. Participants completed a post-fMRI memory test to assess the retention of original action sequences. The exchange of individual actions, and thus a violation of action prediction, resulted in increased activation of the action observation network and the anterior insula. At U events, marking the end of an action, increased brain activation in supplementary motor area, striatum, and lingual gyrus was indicative of the retrieval of the previously encoded action repertoire. As expected, brain activation at U events also reflected the predefined probabilistic branching structure of the action repertoire. At T events, marking the beginning of the next action, midline and hippocampal regions were recruited, reflecting the selected prediction of the unfolding action segment. In conclusion, our findings contribute to a better understanding of the various cerebral processes characterizing prediction during the observation of complex action repertoires.

Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing

The anterior cingulate cortex (ACC) has been implicated across multiple highly specialized cognitive functions-including task engagement, motivation, error detection, attention allocation, value processing, and action selection. Here, we ask if ACC lesions disrupt task performance and firing in dorsomedial striatum (DMS) during the performance of a reward-guided decision-making task that engages many of these cognitive functions. We found that ACC lesions impacted several facets of task performance-including decreasing the initiation and completion of trials, slowing reaction times, and resulting in suboptimal and inaccurate action selection. Reductions in movement times towards the end of behavioral sessions further suggested attenuations in motivation, which paralleled reductions in directional action selection signals in the DMS that were observed later in recording sessions. Surprisingly, however, beyond altered action signals late in sessions-neural correlates in the DMS were largely unaffected, even though behavior was disrupted at multiple levels. We conclude that ACC lesions result in overall deficits in task engagement that impact multiple facets of task performance during our reward-guided decision-making task, which-beyond impacting motivated action signals-arise from dysregulated attentional signals in the ACC and are mediated via downstream targets other than DMS.

Neural activity ramps in frontal cortex signal extended motivation during learning

Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward-associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection-specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward-associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.

Bad habits-good goals? Meta-analysis and translation of the habit construct to alcoholism

Excessive alcohol consumption remains a global public health crisis, with millions suffering from alcohol use disorder (AUD, or simply "alcoholism"), leading to significantly reduced life expectancy. This review examines the interplay between habitual and goal-directed behaviors and the associated neurobiological changes induced by chronic alcohol exposure. Contrary to a strict habit-goal dichotomy, our meta-analysis of the published animal experiments combined with a review of human studies reveals a nuanced transition between these behavioral control systems, emphasizing the need for refined terminology to capture the probabilistic nature of decision biases in individuals with a history of chronic alcohol exposure. Furthermore, we distinguish habitual responding from compulsivity, viewing them as separate entities with diverse roles throughout the stages of the addiction cycle. By addressing species-specific differences and translational challenges in habit research, we provide insights to enhance future investigations and inform strategies for combatting AUD.

Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation

Adjusting decision-making under uncertain and dynamic situations is the hallmark of intelligence. It requires a system capable of converting feedback information to renew the internal value. The anterior cingulate cortex (ACC) involves in error and reward events that prompt switching or maintenance of current decision strategies. However, it is unclear whether and how the changes of stimulus-action mapping during behavioral adaptation are encoded, nor how such computation drives decision adaptation. Here, we tracked ACC activity in male mice performing go/no-go auditory discrimination tasks with manipulated stimulus-reward contingencies. Individual ACC neurons integrate the outcome information to the value representation in the next-run trials. Dynamic recruitment of them determines the learning rate of error-guided value iteration and decision adaptation, forming a non-linear feedback-driven updating system to secure the appropriate decision switch. Optogenetically suppressing ACC significantly slowed down feedback-driven decision switching without interfering with the execution of the established strategy.

Volitional mental absorption in meditation: Toward a scientific understanding of advanced concentrative absorption meditation and the case of jhana

Meditation has been integral to human culture for millennia, deeply rooted in various spiritual and contemplative traditions. While the field of contemplative science has made significant steps toward understanding the effects of meditation on health and well-being, there has been little study of advanced meditative states, including those achieved through intense concentration and absorption. We refer to these types of states as advanced concentrative absorption meditation (ACAM), characterized by absorption with the meditation object leading to states of heightened attention, clarity, energy, effortlessness, and bliss. This review focuses on a type of ACAM known as jhana (ACAM-J) due to its well-documented history, systematic practice approach, recurring phenomenological themes, and growing popularity among contemplative scientists and more generally in media and society. ACAM-J encompasses eight layers of deep concentration, awareness, and internal experiences. Here, we describe the phenomenology of ACAM-J and present evidence from phenomenological and neuroscientific studies that highlight their potential applications in contemplative practices, psychological sciences, and therapeutics. We additionally propose theoretical ACAM-J frameworks grounded in current cognitive neuroscientific understanding of meditation and ancient contemplative traditions. We aim to stimulate further research on ACAM more broadly, encompassing advanced meditation including meditative development and meditative endpoints. Studying advanced meditation including ACAM, and specific practices such as ACAM-J, can potentially revolutionize our understanding of consciousness and applications for mental health.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Divergent Subregional Information Processing in Mouse Prefrontal Cortex During Working Memory

Working memory (WM) is a critical cognitive function allowing recent information to be temporarily held in mind to inform future action. This process depends on coordination between key subregions in prefrontal cortex (PFC) and other connected brain areas. However, few studies have examined the degree of functional specialization between these subregions throughout the phases of WM using electrophysiological recordings in freely-moving animals, particularly mice. To this end, we recorded single-units in three neighboring medial PFC (mPFC) subregions in mouse - supplementary motor area (MOs), dorsomedial PFC (dmPFC), and ventromedial (vmPFC) - during a freely-behaving non-match-to-position WM task. We found divergent patterns of task-related activity across the phases of WM. The MOs is most active around task phase transitions and encodes the starting sample location most selectively. Dorsomedial PFC contains a more stable population code, including persistent sample-location-specific firing during a five second delay period. Finally, the vmPFC responds most strongly to reward-related information during the choice phase. Our results reveal anatomically and temporally segregated computation of WM task information in mPFC and motivate more precise consideration of the dynamic neural activity required for WM.

Neural activity ramps in frontal cortex signal extended motivation during learning

Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.

Theta and alpha oscillatory signatures of auditory sensory and cognitive loads during complex listening

The neuronal signatures of sensory and cognitive load provide access to brain activities related to complex listening situations. Sensory and cognitive loads are typically reflected in measures like response time (RT) and event-related potentials (ERPs) components. It's, however, strenuous to distinguish the underlying brain processes solely from these measures. In this study, along with RT- and ERP-analysis, we performed time-frequency analysis and source localization of oscillatory activity in participants performing two different auditory tasks with varying degrees of complexity and related them to sensory and cognitive load. We studied neuronal oscillatory activity in both periods before the behavioral response (pre-response) and after it (post-response). Robust oscillatory activities were found in both periods and were differentially affected by sensory and cognitive load. Oscillatory activity under sensory load was characterized by decrease in pre-response (early) theta activity and increased alpha activity. Oscillatory activity under cognitive load was characterized by increased theta activity, mainly in post-response (late) time. Furthermore, source localization revealed specific brain regions responsible for processing these loads, such as temporal and frontal lobe, cingulate cortex and precuneus. The results provide evidence that in complex listening situations, the brain processes sensory and cognitive loads differently. These neural processes have specific oscillatory signatures and are long lasting, extending beyond the behavioral response.

Lights, fiber, action! A primer on in vivo fiber photometry

Fiber photometry is a key technique for characterizing brain-behavior relationships in vivo. Initially, it was primarily used to report calcium dynamics as a proxy for neural activity via genetically encoded indicators. This generated new insights into brain functions including movement, memory, and motivation at the level of defined circuits and cell types. Recently, the opportunity for discovery with fiber photometry has exploded with the development of an extensive range of fluorescent sensors for biomolecules including neuromodulators and peptides that were previously inaccessible in vivo. This critical advance, combined with the new availability of affordable "plug-and-play" recording systems, has made monitoring molecules with high spatiotemporal precision during behavior highly accessible. However, while opening exciting new avenues for research, the rapid expansion in fiber photometry applications has occurred without coordination or consensus on best practices. Here, we provide a comprehensive guide to help end-users execute, analyze, and suitably interpret fiber photometry studies.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task

Working memory is a fundamental cognitive ability, allowing us to keep information in memory for the time needed to perform a given task. A complex neural circuit fulfills these functions, among which is the anterior cingulate cortex (CG). Functionally and anatomically connected to the medial prefrontal, retrosplenial, midcingulate and hippocampus, as well as motor cortices, CG has been implicated in retrieving appropriate information when needed to select and control appropriate behavior. The role of cingulate cortex in working memory-guided behaviors remains unclear due to the lack of studies reversibly interfering with its activity during specific epochs of working memory. We used eNpHR3.0 to silence cingulate neurons while animals perform a standard delayed non-match to trajectory task, and found that, while not causing an absolute impairment in working memory, silencing cingulate neurons during retrieval decreases the mean performance if compared to silencing during encoding. Such retrieval-associated changes are accompanied by longer delays observed when light is delivered to control animals, when compared to eNpHR3.0+ ones, consistent with an adaptive recruitment of additional cognitive resources.

A stress-sensitive frontostriatal circuit supporting effortful reward-seeking behavior

Effort valuation-a process for selecting actions based on the anticipated value of rewarding outcomes and expectations about the work required to obtain them-plays a fundamental role in decision-making. Effort valuation is disrupted in chronic stress states and is supported by the anterior cingulate cortex (ACC), but the circuit-level mechanisms by which the ACC regulates effort-based decision-making are unclear. Here, we show that ACC neurons projecting to the nucleus accumbens (ACC-NAc) play a critical role in effort valuation behavior in mice. Activity in ACC-NAc cells integrates both reward- and effort-related information, encoding a reward-related signal that scales with effort requirements and is necessary for supporting future effortful decisions. Chronic corticosterone exposure reduces motivation, suppresses effortful reward-seeking, and disrupts ACC-NAc signals. Together, our results delineate a stress-sensitive ACC-NAc circuit that supports effortful reward-seeking behavior by integrating reward and effort signals and reinforcing effort allocation in the service of maximizing reward.

Dopamine-independent effect of rewards on choices through hidden-state inference

Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.

State-transition-free reinforcement learning in chimpanzees (Pan troglodytes)

The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.

The relation between implicit statistical learning and proactivity as revealed by EEG

Environmental events often occur on a probabilistic basis but can sometimes be predicted based on specific cues and thus approached proactively. Incidental statistical learning enables the acquisition of knowledge about probabilistic cue-target contingencies. However, the neural mechanisms of statistical learning about contingencies (SLC), the required conditions for successful learning, and the role of implicit processes in the resultant proactive behavior are still debated. We examined changes in behavior and cortical activity during an SLC task in which subjects responded to visual targets. Unbeknown to them, there were three types of target cues associated with high-, low-, and zero target probabilities. About half of the subjects spontaneously gained explicit knowledge about the contingencies (contingency-aware group), and only they showed evidence of proactivity: shortened response times to predictable targets and enhanced event-related brain responses (cue-evoked P300 and contingent negative variation, CNV) to high probability cues. The behavioral and brain responses were strictly associated on a single-trial basis. Source reconstruction of the brain responses revealed activation of fronto-parietal brain regions associated with cognitive control, particularly the anterior cingulate cortex and precuneus. We also found neural correlates of SLC in the contingency-unaware group, but these were restricted to post-target latencies and visual association areas. Our results document a qualitative difference between explicit and implicit learning processes and suggest that in certain conditions, proactivity may require explicit knowledge about contingencies.

Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control

Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ-1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ-1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ-1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.

Effects of the number of competing responses on neural signatures of pre- and post-response conflict

The optimization of human performance requires the continuous monitoring of behavioral conflicts. According to conflict monitoring theory, the dorsal anterior cingulate cortex registers response conflict which is reflected by two electrophysiological signatures, the N2 and the Ne/ERN. The theory assumes that, if a stimulus activates an incorrect response that competes with the correct response, pre-response conflict on correct trials (reflected by the N2) is enhanced but post-response conflict on error trials (reflected by the Ne/ERN) is reduced. Here, we asked whether response conflict depends on the number of competing incorrect responses activated by a stimulus, that is, whether the N2 is further enhanced and the Ne/ERN is further reduced if two incorrect responses are activated as compared to one. To this end, we used a modified flanker paradigm, in which the two flankers were associated either with the same incorrect response or with different incorrect responses. Our results indicate an increased N2 on correct trials and a reduced Ne/ERN on error trials in the latter as compared to the former condition. These results confirm central predictions of conflict monitoring theory and demonstrate that response conflict is directly related to the number of competing incorrect responses.

Control of sustained attention and impulsivity by Gq-protein signalling in parvalbumin interneurons of the anterior cingulate cortex

The anterior cingulate cortex (ACC) has been implicated in attention deficit hyperactivity disorder (ADHD). More specifically, an appropriate balance of excitatory and inhibitory activity in the ACC may be critical for the control of impulsivity, hyperactivity, and sustained attention which are centrally affected in ADHD. Hence, pharmacological augmentation of parvalbumin- (PV) or somatostatin-positive (Sst) inhibitory ACC interneurons could be a potential treatment strategy. We, therefore, tested whether stimulation of G_q-protein-coupled receptors (G_qPCRs) in these interneurons could improve attention or impulsivity assessed with the 5-choice-serial reaction-time task in male mice. When challenging impulse control behaviourally or pharmacologically, activation of the chemogenetic G_qPCR hM3Dq in ACC PV-cells caused a selective decrease of active erroneous-i.e. incorrect and premature-responses, indicating improved attentional and impulse control. When challenging attention, in contrast, omissions were increased, albeit without extension of reward latencies or decreases of attentional accuracy. These effects largely resembled those of the ADHD medication atomoxetine. Additionally, they were mostly independent of each other within individual animals. G_qPCR activation in ACC PV-cells also reduced hyperactivity. In contrast, if hM3Dq was activated in Sst-interneurons, no improvement of impulse control was observed, and a reduction of incorrect responses was only induced at high agonist levels and accompanied by reduced motivational drive. These results suggest that the activation of G_qPCRs expressed specifically in PV-cells of the ACC may be a viable strategy to improve certain aspects of sustained attention, impulsivity and hyperactivity in ADHD.

Expertise increases planning depth in human gameplay

A hallmark of human intelligence is the ability to plan multiple steps into the future1,2. Despite decades of research3-5, it is still debated whether skilled decision-makers plan more steps ahead than novices6-8. Traditionally, the study of expertise in planning has used board games such as chess, but the complexity of these games poses a barrier to quantitative estimates of planning depth. Conversely, common planning tasks in cognitive science often have a lower complexity9,10 and impose a ceiling for the depth to which any player can plan. Here we investigate expertise in a complex board game that offers ample opportunity for skilled players to plan deeply. We use model fitting methods to show that human behaviour can be captured using a computational cognitive model based on heuristic search. To validate this model, we predict human choices, response times and eye movements. We also perform a Turing test and a reconstruction experiment. Using the model, we find robust evidence for increased planning depth with expertise in both laboratory and large-scale mobile data. Experts memorize and reconstruct board features more accurately. Using complex tasks combined with precise behavioural modelling might expand our understanding of human planning and help to bridge the gap with progress in artificial intelligence.

Predicting Alcohol-Related Memory Problems in Older Adults: A Machine Learning Study with Multi-Domain Features

Memory problems are common among older adults with a history of alcohol use disorder (AUD). Employing a machine learning framework, the current study investigates the use of multi-domain features to classify individuals with and without alcohol-induced memory problems. A group of 94 individuals (ages 50-81 years) with alcohol-induced memory problems (the memory group) were compared with a matched control group who did not have memory problems. The random forests model identified specific features from each domain that contributed to the classification of the memory group vs. the control group (AUC = 88.29%). Specifically, individuals from the memory group manifested a predominant pattern of hyperconnectivity across the default mode network regions except for some connections involving the anterior cingulate cortex, which were predominantly hypoconnected. Other significant contributing features were: (i) polygenic risk scores for AUD, (ii) alcohol consumption and related health consequences during the past five years, such as health problems, past negative experiences, withdrawal symptoms, and the largest number of drinks in a day during the past twelve months, and (iii) elevated neuroticism and increased harm avoidance, and fewer positive "uplift" life events. At the neural systems level, hyperconnectivity across the default mode network regions, including the connections across the hippocampal hub regions, in individuals with memory problems may indicate dysregulation in neural information processing. Overall, the study outlines the importance of utilizing multidomain features, consisting of resting-state brain connectivity data collected ~18 years ago, together with personality, life experiences, polygenic risk, and alcohol consumption and related consequences, to predict the alcohol-related memory problems that arise in later life.

The human cerebellum in reward anticipation and reward outcome processing: An activation likelihood estimation meta-analysis

The cerebellum generates internal prediction models and actively compares anticipated and actual outcomes in order to reach a desired end state. In this process, reward can serve as a reinforcer that shapes internal prediction models, enabling context-appropriate behavior. While the involvement of the cerebellum in reward processing has been established in animals, there is no detailed account of which cerebellar regions are involved in reward anticipation and reward outcome processing in humans. To this end, an activation likelihood estimation meta-analysis of functional neuroimaging studies was performed to investigate cerebellar functional activity patterns associated with reward anticipation and reward outcome processing in healthy adults. Results showed that reward anticipation (k=31) was associated with regional activity in the bilateral anterior lobe, bilateral lobule VI, left Crus I and the posterior vermis, while reward outcome (k=16) was associated with regional activity in the declive and left lobule VI. The findings of this meta-analysis show distinct involvement of the cerebellum in reward anticipation and reward outcome processing as part of a predictive coding routine.

Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning

Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition, there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.

Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia

Negative symptoms, such as lack of motivation or social withdrawal, are highly prevalent and debilitating in patients with schizophrenia. Underlying mechanisms of negative symptoms are incompletely understood, thereby preventing the development of targeted treatments. We hypothesized that in patients with schizophrenia during psychotic remission, impaired influences of both model-based and model-free reward predictions on decision-making ('reward prediction influence', RPI) underlie negative symptoms. We focused on psychotic remission, because psychotic symptoms might confound reward-based decision-making. Moreover, we hypothesized that impaired model-based/model-free RPIs depend on alterations of both associative striatum dopamine synthesis and storage (DSS) and executive functioning. Both factors influence RPI in healthy subjects and are typically impaired in schizophrenia. Twenty-five patients with schizophrenia with pronounced negative symptoms during psychotic remission and 24 healthy controls were included in the study. Negative symptom severity was measured by the Positive and Negative Syndrome Scale negative subscale, model-based/model-free RPI by the two-stage decision task, associative striatum DSS by 18F-DOPA positron emission tomography and executive functioning by the symbol coding task. Model-free RPI was selectively reduced in patients and associated with negative symptom severity as well as with reduced associative striatum DSS (in patients only) and executive functions (both in patients and controls). In contrast, model-based RPI was not altered in patients. Results provide evidence for impaired model-free reward prediction influence as a mechanism for negative symptoms in schizophrenia as well as for reduced associative striatum dopamine and executive dysfunction as relevant factors. Data suggest potential treatment targets for patients with schizophrenia and pronounced negative symptoms.

A neural substrate of sex-dependent modulation of motivation

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Differential processing of decision information in subregions of rodent medial prefrontal cortex

Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.

The association of spouse interactions and emotional learning in interference related to chronic back pain

Social interactions affect individual behaviours, preferences, and attitudes. This is also critical in the context of experiencing pain and expressing pain behaviours, and may relate to learned emotional responses. In this respect, individual variability in the medial prefrontal cortex (mPFC), which is involved in adjusting an organism's behaviour to its environment by evaluating and interpreting information within the context of past experiences, is important. It is critical for selecting suitable behavioural responses within a social environment and may reinforce maladaptation in chronic pain. In our study, we used brain imaging during appetitive and aversive pavlovian conditioning in persons with chronic back pain (CBP), subacute back pain (SABP), and healthy controls (HC), together with information on spouse responses to pain behaviours. We also examined the relationship of these responses with pain-related interference in the patients. Our findings yielded a significant negative association between mPFC responses to appetitive and aversive learning in CBP. We also observed a significant negative association for mPFC responses during aversive learning and distracting spouse responses, and a significant positive association between mPFC responses during appetitive learning and solicitous spouse responses in CBP. Both significantly predicted pain-related interference in the CBP group (explained variance up to 53%). Significant associations were not found for SABP or HC. Our findings support an association between appetitive and aversive pavlovian learning, related brain circuits and spouse responses to pain in CBP, where appetitive and aversive learning processes seem to be differentially involved. This can inform prevention and early intervention in a mechanistic approach.

Calcium activity is a degraded estimate of spikes

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.