Neural circuits for inference-based decision-making

Neural mechanisms of credit assignment for delayed outcomes during contingent learning

Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (lFPC). Our results indicate that lFPC holds previous causal choices in a 'pending' state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.

Neural mechanisms of credit assignment for delayed outcomes during contingent learning

Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (lFPC). Our results indicate that lFPC holds previous causal choices in a "pending" state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.

Hippocampal and orbitofrontal neurons contribute to complementary aspects of associative structure

The ability to establish associations between environmental stimuli is fundamental for higher-order brain functions like state inference and generalization. Both the hippocampus and orbitofrontal cortex (OFC) play pivotal roles in this, demonstrating complex neural activity changes after associative learning. However, how precisely they contribute to representing learned associations remains unclear. Here, we train head-restrained mice to learn four 'odor-outcome' sequence pairs composed of several task variables-the past and current odor cues, sequence structure of 'cue-outcome' arrangement, and the expected outcome; and perform calcium imaging from these mice throughout learning. Sequence-splitting signals that distinguish between paired sequences are detected in both brain regions, reflecting associative memory formation. Critically, we uncover differential contents in represented associations by examining, in each area, how these task variables affect splitting signal generalization between sequence pairs. Specifically, the hippocampal splitting signals are influenced by the combination of past and current cues that define a particular sensory experience. In contrast, the OFC splitting signals are similar between sequence pairs that share the same sequence structure and expected outcome. These findings suggest that the hippocampus and OFC uniquely and complementarily organize the acquired associative structure.

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Lateral orbitofrontal cortex integrates predictive information across multiple cues to guide behavior

Individuals are often faced with multiple cues that concurrently predict the same outcome, and combining these predictions may benefit behavior. Previous work has studied the neural basis of decision-making, predominantly using isolated sensory stimuli, and so the mechanisms that allow us to leverage multiple cues remain unclear. In two experiments, we used neuroimaging and network-targeted brain stimulation to probe how the brain integrates outcome predictions to guide adaptive behavior. We identified neural signatures of outcome integration in the lateral orbitofrontal cortex (OFC), where concurrently presented cues evoke stronger pattern-based representations of expected outcomes. Moreover, perturbing lateral OFC network activity impairs subjects' ability to leverage predictions from multiple cues to facilitate responding. Intriguingly, we found similar behavioral and brain mechanisms for reward-predicting cues and for cues predicting the absence of reward. These findings highlight a causal role for the lateral OFC in utilizing outcome predictions from multiple cues to guide behavior.

Computationally Informed Interventions for Targeting Compulsive Behaviors

Compulsive behaviors are central to addiction and obsessive-compulsive disorder and can be understood as a failure of adaptive decision making. Particularly, they can be conceptualized as an imbalance in behavioral control, such that behavior is guided predominantly by learned rather than inferred outcome expectations. Inference is a computational process required for adaptive behavior, and recent work across species has identified the neural circuitry that supports inference-based decision making. This includes the orbitofrontal cortex, which has long been implicated in disorders of compulsive behavior. Inspired by evidence that modulating orbitofrontal cortex activity can alter inference-based behaviors, here we discuss noninvasive approaches to target these circuits in humans. Specifically, we discuss the potential of network-targeted transcranial magnetic stimulation and real-time neurofeedback to modulate the neural underpinnings of inference. Both interventions leverage recent advances in our understanding of the neurocomputational mechanisms of inference-based behavior and may be used to complement current treatment approaches for behavioral disorders.

Emotional learning retroactively promotes memory integration through rapid neural reactivation and reorganization

Neutral events preceding emotional experiences can be better remembered, likely by assigning them as significant to guide possible use in future. Yet, the neurobiological mechanisms of how emotional learning enhances memory for past mundane events remain unclear. By two behavioral studies and one functional magnetic resonance imaging study with an adapted sensory preconditioning paradigm, we show rapid neural reactivation and connectivity changes underlying emotion-charged retroactive memory enhancement. Behaviorally, emotional learning retroactively enhanced initial memory for neutral associations across the three studies. Neurally, emotional learning potentiated trial-specific reactivation of overlapping neural traces in the hippocampus and stimulus-relevant neocortex. It further induced rapid hippocampal-neocortical functional reorganization supporting such retroactive memory benefit, as characterized by enhanced hippocampal-neocortical coupling modulated by the amygdala during emotional learning, and a shift of hippocampal connectivity from stimulus-relevant neocortex to distributed transmodal prefrontal-parietal areas at post-learning rests. Together, emotional learning retroactively promotes memory integration for past neutral events through stimulating trial-specific reactivation of overlapping representations and reorganization of associated memories into an integrated network to foster its priority for future use.

Reward Uncertainty and Expected Value Enhance Generalization of Episodic Memory

Previous research has revealed some mechanisms underlying the generalization of reward expectation of generalization stimuli, but little is known about the generalization of episodic memory for rewarding events, its consolidation, and how reward components such as expected value and reward uncertainty affect it. Participants underwent a Pavlovian reward-conditioning task to test whether reward conditioning would enhance episodic memory generalization and which reward components would directly affect it. Counterbalanced across participants, one semantic category was paired with a reward, while the other was never paired. Following a delay of either 5 min or 24 h, participants took a memory test consisting of old, highly similar, and new items. We found that participants were more likely to falsely recognize lure items as old in the reward-paired category after 5 min and 24 h delays. These results indicate that reward conditioning enhanced the generalization of episodic memory, but this effect was not necessarily dependent on consolidation. The composite score and raw data of generalization further showed that the uncertainty and expected value enhanced generalization. Together, these findings revealed an effect of reward conditioning on episodic memory generalization and supported the enhancement effects of expected value and uncertainty.

The mediodorsal thalamus supports adaptive responding based on stimulus-outcome associations

The ability to engage into flexible behaviors is crucial in dynamic environments. We recently showed that in addition to the well described role of the orbitofrontal cortex (OFC), its thalamic input from the submedius thalamic nucleus (Sub) also contributes to adaptive responding during Pavlovian degradation. In the present study, we examined the role of the mediodorsal thalamus (MD) which is the other main thalamic input to the OFC. To this end, we assessed the effect of both pre- and post-training MD lesions in rats performing a Pavlovian contingency degradation task. Pre-training lesions mildly impeded the establishment of stimulus-outcome associations during the initial training of Pavlovian conditioning without interfering with Pavlovian degradation training when the sensory feedback provided by the outcome rewards were available to animals. However, we found that both pre- and post-training MD lesions produced a selective impairment during a test conducted under extinction conditions, during which only current mental representation could guide behavior. Altogether, these data suggest a role for the MD in the successful encoding and representation of Pavlovian associations.