Adaptive learning under expected and unexpected uncertainty

Distributed Intracranial Activity Underlying Human Decision-making Behavior

Value-based decision-making involves multiple cortical and subcortical brain areas, but the distributed nature of neurophysiological activity underlying economic choices in the human brain remains largely unexplored. Specifically, the nature of the neurophysiological representation of reward-guided choices, as well as whether they are represented in a subset of reward-related regions or in a more distributed fashion, is unknown. Here, we hypothesize that reward choices, as well as choice-related computations (win probability, risk), are primarily represented in high-frequency neural activity reflecting local cortical processing and that they are highly distributed throughout the human brain, engaging multiple brain regions. To test these hypotheses, we used intracranial recordings from multiple areas (including orbitofrontal, lateral prefrontal, parietal, cingulate cortices as well as subcortical regions such as the hippocampus and amygdala) from neurosurgical patients of both sexes playing a decision-making game. We show that high-frequency activity (HFA; ɣ and HFA) represents both individual choice-related computations (e.g., risk, win probability) and choice information with different prevalence and regional representation. Choice-related computations are locally and unevenly present in multiple brain regions, whereas choice information is widely distributed and more prevalent and appears later across all regions examined. These results suggest brain-wide reward processing, with local HFA reflecting the coalescence of choice-related information into a final choice, and shed light on the distributed nature of neural activity underlying economic choices in the human brain.

Chronic ethanol exposure produces sex-dependent impairments in value computations in the striatum

Value-based decision-making relies on the striatum, where neural plasticity can be altered by chronic ethanol (EtOH) exposure, but the effects of such plasticity on striatal neural dynamics during decision-making remain unclear. This study investigated the long-term impacts of EtOH on reward-driven decision-making and striatal neurocomputations in male and female rats using a dynamic probabilistic reversal learning task. Following a prolonged withdrawal period, EtOH-exposed male rats exhibited deficits in adaptability and exploratory behavior, with aberrant outcome-driven value updating that heightened preference for chosen action. These behavioral changes were linked to altered neural activity in the dorsomedial striatum (DMS), where EtOH increased outcome-related encoding and decreased choice-related encoding. In contrast, female rats showed minimal behavioral changes with distinct EtOH-evoked alterations of neural activity, revealing significant sex differences in the impact of chronic EtOH. Our findings underscore the impact of chronic EtOH exposure on adaptive decision-making, revealing enduring changes in neurocomputational processes in the striatum underlying cognitive deficits that differ by sex.

A roadmap for transformative translational research on gambling disorder in the UK

The UK has one of the highest rates of recreational gambling in the world. Some vulnerable individuals progressively lose control over gambling and develop at-risk gambling or gambling disorder (GD), characterised by the compulsive pursuit of gambling. GD destroys lives and incurs massive costs to societies, yet only a few treatments are available. Failure to develop a wider range of interventions is in part due to a lack of funding that has slowed progress in the translational research necessary to understand the individual vulnerability to switch from controlled to compulsive gambling. Current preclinical models of GD do not operationalise the key clinical features of the human condition. The so-called "gambling tasks" for non-human mammals almost exclusively assess probabilistic decision-making, which is not real-world gambling. While they have provided insights into the psychological and neural mechanisms involved in the processing of gains and losses, these tasks have failed to capture those underlying real-world gambling and its compulsive manifestation in humans. Here, we highlight the strengths and weaknesses of current gambling-like behaviour tasks and suggest how their translational validity may be improved. We then propose a theoretical framework, the incentive habit theory of GD, which may prove useful for the operationalisation of the biobehavioural mechanisms of GD in preclinical models. We conclude with a list of recommendations for the development of next-generation preclinical models of GD and discuss how modern techniques in animal behavioural experimentation can be deployed in the context of GD preclinical research to bolster the translational pipeline.

Violated predictions enhance the representational fidelity of visual features in perception

Predictive coding theories argue that recent experience establishes expectations that generate prediction errors when violated. In humans, brain imaging studies have revealed unique signatures of violated predictions in sensory cortex, but the perceptual consequences of these effects remain unknown. We had observers perform a dual-report task on the orientation of a briefly presented target grating within predictable or random sequences, while we recorded pupil size as an index of surprise. Observers first made a speeded response to categorize the orientation of the target grating (clockwise or counterclockwise from vertical), then reproduced its orientation without time pressure by rotating a bar. This allowed us to separately assess response speed and precision for the same stimuli. Critically, on half the trials, the target orientation deviated from the spatiotemporal structure established by the preceding gratings. Observers responded faster and more accurately to unexpected gratings, and pupillometry provided physiological evidence of observers' surprise in response to these events. In a second experiment, we cued the spatial location and timing of the grating and found the same pattern of results, demonstrating that unexpected orientation information is sufficient to produce faster and more precise responses, even when the location and timing of the relevant stimuli are fully expected. These findings indicate that unexpected events are prioritized by the visual system both in terms of processing speed and representational fidelity.

Temporal context modulates the recovery of the attentional blink

Humans usually adjust their attentional mode to tackle the challenges posed by environmental inputs. Depending on the uncertainty level, different attentional strategies may be adopted. As people face increasingly complicated daily situations-e.g., driving a car or chatting online-where intervals between significant events do not necessarily follow certain rules but are likely random, it appears important to understand how temporal contexts with different uncertainty levels affect temporal attention allocation when processing rapid serial inputs. We pursued this issue by employing a task examining the temporal limit of attention-the attentional blink (AB). The manipulation of temporal context was achieved by presenting trials with different inter-target intervals following either a "random-walk" or a "random" sequence. The results suggest a facilitated recovery from the AB deficit in the "random" compared to "random-walk" context, without a corresponding change in AB magnitude. Such effect is likely attributed to the higher perceived uncertainty in the former, and could be attenuated by a decrease in the temporal uncertainty level. These observations suggest that observers likely adopted a more flexible temporal attention allocation in the more unpredictable "random" context; they also support non-overlapping mechanisms responsible for AB width/duration and amplitude or lag-1 sparing. The flexibility of temporal attentional control may provide an evolutionary advantage for organisms to deal with unpredictable changes and is likely to be exploited for reference in the design of human-machine interacting platforms.

Uncertainty estimation with prediction-error circuits

Neural circuits continuously integrate noisy sensory stimuli with predictions that often do not perfectly match, requiring the brain to combine these conflicting feedforward and feedback inputs according to their uncertainties. However, how the brain tracks both stimulus and prediction uncertainty remains unclear. Here, we show that a hierarchical prediction-error network can estimate both the sensory and prediction uncertainty with positive and negative prediction-error neurons. Consistent with prior hypotheses, we demonstrate that neural circuits rely more on predictions when sensory inputs are noisy and the environment is stable. By perturbing inhibitory interneurons within the prediction-error circuit, we reveal their role in uncertainty estimation and input weighting. Finally, we link our model to biased perception, showing how stimulus and prediction uncertainty contribute to the contraction bias.

Volitional Regulation and Transferable Patterns of Midbrain Oscillations

Dopaminergic brain areas are crucial for cognition and their dysregulation is linked to neuropsychiatric disorders typically treated with pharmacological interventions. These treatments often have side effects and variable effectiveness, underscoring the need for alternatives. We introduce the first demonstration of neurofeedback using local field potentials (LFP) from the ventral tegmental area (VTA). This approach leverages the real-time temporal resolution of LFP and ability to target deep brain. In our study, two male rhesus macaque monkeys (Macaca mulatta) learned to regulate VTA beta power using a customized normalized metric to stably quantify VTA LFP signal modulation. The subjects demonstrated flexible and specific control with different strategies for specific frequency bands, revealing new insights into the plasticity of VTA neurons contributing to oscillatory activity that is functionally relevant to many aspects of cognition. Excitingly, the subjects showed transferable patterns, a key criterion for clinical applications beyond training settings. This work provides a foundation for neurofeedback-based treatments, which may be a promising alternative to conventional approaches and open new avenues for understanding and managing neuropsychiatric disorders.

Neural coding of choice and outcome are modulated by uncertainty in orbitofrontal but not secondary motor cortex

Orbitofrontal cortex (OFC) and secondary motor cortex (M2) are both implicated in flexible reward learning but the conditions that differentially recruit these regions are not fully understood. We imaged calcium activity from single neurons in OFC or M2 during de novo learning of uncertain reward probability schedules. Predictions of choice were decoded from M2 neurons with high accuracy under all certainty conditions, but were more accurately decoded from OFC neurons under greater uncertainty. In M2, the proportion of outcome-selective neurons decreased with uncertainty whereas this proportion remained stable in OFC, due to an increased recruitment of reward-selective neurons across levels of uncertainty. Decoding accuracy of both choice and outcome were predicted by indices of flexible strategy like Win-Stay and Lose-Shift in OFC, but not M2. When schedules were experienced in increasing and then decreasing uncertainty, chemogenetic perturbation of M2 and OFC neurons resulted in opposing roles in certain and uncertain conditions, respectively. Our results indicate that M2 neurons are causally involved in learning of more certain conditions, whereas OFC neurons preferentially encode choices and outcomes that foster greater reliance on adaptive strategies under conditions of uncertainty. This reveals a novel functional heterogeneity within frontal cortex in support of flexible learning.

Antifragile control systems in neuronal processing: a sensorimotor perspective

The stability-robustness-resilience-adaptiveness continuum in neuronal processing follows a hierarchical structure that explains interactions and information processing among the different time scales. Interestingly, using "canonical" neuronal computational circuits, such as Homeostatic Activity Regulation, Winner-Take-All, and Hebbian Temporal Correlation Learning, one can extend the behavior spectrum towards antifragility. Cast already in both probability theory and dynamical systems, antifragility can explain and define the interesting interplay among neural circuits, found, for instance, in sensorimotor control in the face of uncertainty and volatility. This perspective proposes a new framework to analyze and describe closed-loop neuronal processing using principles of antifragility, targeting sensorimotor control. Our objective is two-fold. First, we introduce antifragile control as a conceptual framework to quantify closed-loop neuronal network behaviors that gain from uncertainty and volatility. Second, we introduce neuronal network design principles, opening the path to neuromorphic implementations and transfer to technical systems.

Exploring replay

Animals face uncertainty about their environments due to initial ignorance or subsequent changes. They therefore need to explore. However, the algorithmic structure of exploratory choices in the brain still remains largely elusive. Artificial agents face the same problem, and a venerable idea in reinforcement learning is that they can plan appropriate exploratory choices offline, during the equivalent of quiet wakefulness or sleep. Although offline processing in humans and other animals, in the form of hippocampal replay and preplay, has recently been the subject of highly informative modelling, existing methods only apply to known environments. Thus, they cannot predict exploratory replay choices during learning and/or behaviour in the face of uncertainty. Here, we extend an influential theory of hippocampal replay and examine its potential role in approximately optimal exploration, deriving testable predictions for the patterns of exploratory replay choices in a paradigmatic spatial navigation task. Our modelling provides a normative interpretation of the available experimental data suggestive of exploratory replay. Furthermore, we highlight the importance of sequence replay, and license a range of new experimental paradigms that should further our understanding of offline processing.

Understanding learning through uncertainty and bias

Learning allows humans and other animals to make predictions about the environment that facilitate adaptive behavior. Casting learning as predictive inference can shed light on normative cognitive mechanisms that improve predictions under uncertainty. Drawing on normative learning models, we illustrate how learning should be adjusted to different sources of uncertainty, including perceptual uncertainty, risk, and uncertainty due to environmental changes. Such models explain many hallmarks of human learning in terms of specific statistical considerations that come into play when updating predictions under uncertainty. However, humans also display systematic learning biases that deviate from normative models, as studied in computational psychiatry. Some biases can be explained as normative inference conditioned on inaccurate prior assumptions about the environment, while others reflect approximations to Bayesian inference aimed at reducing cognitive demands. These biases offer insights into cognitive mechanisms underlying learning and how they might go awry in psychiatric illness.

Violation of auditory regularities is reflected in pupil dynamics

The brain builds and maintains internal models and uses them to make predictions. When predictions are violated, the current model can either be updated or replaced by a new model. The latter is accompanied by pupil dilation responses (PDRs) related to locus coeruleus activity/norepinephrine release (LC-NE). Following earlier research, we investigated PDRs associated with transitions between regular and random patterns of tones in auditory sequences. We presented these sequences to participants and instructed them to find gaps (to maintain attention). Transitions from regular to random patterns induced PDRs, suggesting that an internal model attuned to the regular pattern is reset. Transitions from one regular pattern to another regular pattern also induced PDRs, suggesting that they also led to a model reset. In contrast, transitions from random patterns to regular patterns did not induce PDRs, suggesting a gradual update of model parameters. We modelled these findings, using pupil response functions to show how ongoing PDRs and pupil event rates were sensitive to the trial-by-trial changes in the information content of the auditory sequences. Expanding on previous research, we suggest that PDRs-as biomarkers for LC-NE activation-may indicate the extent of prediction violations.

Secondary motor cortex tracks decision value during the learning of a non-instructed task

Optimal decision-making depends on interconnected frontal brain regions, enabling animals to adapt decisions based on internal states, experiences, and contexts. The secondary motor cortex (M2) is key in adaptive behaviors in expert rodents, particularly in encoding decision values guiding complex probabilistic tasks. However, its role in deterministic tasks during initial learning remains uncertain. Here, we describe a self-initiated deterministic task requiring mice to use their forepaws to make choices without guiding cues. Our findings reveal that spontaneous decisions follow a "race" model between actions, which uncovers underlying decision values. We use in vivo microscopy and modeling to show that M2 neurons in male mice exhibit persistent activity-encoding decision values that predict action-selection probabilities. Optogenetic inhibition of the M2 reduces the reversal performance and alters the decision value. Additionally, updates in decision values determine the rate at which learning is reversed. These results highlight the use of decision values by the M2 to adapt choice during initial learning without instructive cues.

Distinct Computational Mechanisms of Uncertainty Processing Explain Opposing Exploratory Behaviors in Anxiety and Apathy

BACKGROUND

Decision making in uncertain environments can lead to varied outcomes, and how we process those outcomes may depend on our emotional state. Understanding how individuals interpret the sources of uncertainty is crucial for understanding adaptive behavior and mental well-being. Uncertainty can be broadly categorized into 2 components: volatility and stochasticity. Volatility describes how quickly conditions change. Stochasticity, on the other hand, refers to outcome randomness. We investigated how anxiety and apathy influenced people's perceptions of uncertainty and how uncertainty perception shaped explore-exploit decisions.

METHODS

Participants (N = 1001, nonclinical sample) completed a restless 3-armed bandit task that was analyzed using both latent state and process models.

RESULTS

Individuals with anxiety perceived uncertainty as resulting more from volatility, leading to increased exploration and learning rates, especially after reward omission. Conversely, individuals with apathy viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility to stochasticity ratio mediated the anxiety-exploration relationship post adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy.

CONCLUSIONS

These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.

Navigating uncertainty: reward location variability induces reorganization of hippocampal spatial representations

Navigating uncertainty is crucial for survival, with the location and availability of reward varying in different and unsignalled ways. Hippocampal place cell populations over-represent salient locations in an animal's environment, including those associated with rewards; however, how the spatial uncertainties impact the cognitive map is unclear. We report a virtual spatial navigation task designed to test the impact of different levels and types of uncertainty about reward on place cell populations. When the reward location changed on a trial-by-trial basis, inducing expected uncertainty, a greater proportion of place cells followed along, and the reward and the track end became anchors of a warped spatial metric. When the reward location then unexpectedly moved, the fraction of reward place cells that followed was greater when starting from a state of expected, compared to low, uncertainty. Overall, we show that different forms of potentially interacting uncertainty generate remapping in parallel, task-relevant, reference frames.

Accumbal acetylcholine signals associative salience

Learning in dynamic environments requires animals to not only associate cues with outcomes but also to determine cue salience, which modulates how quickly related associations are updated. While dopamine (DA) in the nucleus accumbens core (NAcc) has been implicated in learning associations, the mechanisms of salience are less understood. Here, we tested the hypothesis that acetylcholine (ACh) in the NAcc encodes cue salience. We conducted four odor discrimination experiments in rats while simultaneously measuring accumbal ACh and DA. We found that ACh developed characteristic dips to cues over learning before DA signals differentiated cues by value, with these dips persisting through value decreases and developing faster during meta-learning. The dips reflected the cue salience across learning stages and tasks, as predicted by a hybrid attentional associative learning model that integrated principles from the Mackintosh and Pearce-Hall models, suggesting that accumbal ACh signals encode salience and potentially gate the learning process.

Sustained EEG responses to rapidly unfolding stochastic sounds reflect Bayesian inferred reliability tracking

How does the brain track and process rapidly changing sensory information? Current computational accounts suggest that our sensations and decisions arise from the intricate interplay between bottom-up sensory signals and constantly changing expectations regarding the statistics of the surrounding world. A significant focus of recent research is determining which statistical properties are tracked by the brain as it monitors the rapid progression of sensory information. Here, by combining EEG (three experiments N ≥ 22 each) and computational modelling, we examined how the brain processes rapid and stochastic sound sequences that simulate key aspects of dynamic sensory environments. Passively listening participants were exposed to structured tone-pip arrangements that contained transitions between a range of stochastic patterns. Predictions were guided by a Bayesian predictive inference model. We demonstrate that listeners automatically track the statistics of unfolding sounds, even when these are irrelevant to behaviour. Transitions between sequence patterns drove a shift in the sustained EEG response. This was observed to a range of distributional statistics, and even in situations where behavioural detection of these transitions was at floor. These observations suggest that the modulation of the EEG sustained response reflects a process of belief updating within the brain. By establishing a connection between the outputs of the computational model and the observed brain responses, we demonstrate that the dynamics of these transition-related responses align with the tracking of "precision" - the confidence or reliability assigned to a predicted sensory signal - shedding light on the intricate interplay between the brain's statistical tracking mechanisms and its response dynamics.

Broadscale dampening of uncertainty adjustment in the aging brain

The ability to prioritize among input features according to relevance enables adaptive behaviors across the human lifespan. However, relevance often remains ambiguous, and such uncertainty increases demands for dynamic control. While both cognitive stability and flexibility decline during healthy ageing, it is unknown whether aging alters how uncertainty impacts perception and decision-making, and if so, via which neural mechanisms. Here, we assess uncertainty adjustment across the adult lifespan (N = 100; cross-sectional) via behavioral modeling and a theoretically informed set of EEG-, fMRI-, and pupil-based signatures. On the group level, older adults show a broad dampening of uncertainty adjustment relative to younger adults. At the individual level, older individuals whose modulation more closely resembled that of younger adults also exhibit better maintenance of cognitive control. Our results highlight neural mechanisms whose maintenance plausibly enables flexible task-set, perception, and decision computations across the adult lifespan.

The influence of anxiety on exploration: A review of computational modeling studies

Exploratory behaviors can serve an adaptive role within novel or changing environments. Namely, they facilitate information gain, allowing an organism to maintain accurate beliefs about the environment and select actions that better maximize reward. However, finding the optimal balance between exploration and reward-seeking behavior - the so-called explore-exploit dilemma - can be challenging, as it requires sensitivity to one's own uncertainty and to the predictability of one's surroundings. Given the close relationship between uncertainty and anxiety, a body of work has now also emerged identifying associated effects on exploration. In particular, the field of computational psychiatry has begun to use cognitive computational models to characterize how anxiety may modulate underlying information processing mechanisms, such as estimation of uncertainty and the value of information, and how this might contribute to psychopathology. Here, we review computational modeling studies investigating how exploration is influenced by anxiety. While some apparent inconsistencies remain to be resolved, studies using reinforcement learning tasks suggest that directed (but not random) forms of exploration may be elevated by trait and/or cognitive anxiety, but reduced by state and/or somatic anxiety. Anxiety is also consistently associated with less exploration in foraging tasks. Some differences in exploration may further stem from how anxiety modulates changes in uncertainty over time (learning rates). Jointly, these results highlight important directions for future work in refining choice of tasks and anxiety measures and maintaining consistent methodology across studies.

Prenatal Stress and Maternal Role in Neurodevelopment

This review summarizes recent findings on stress-related programming of brain development in utero, with an emphasis on situating findings within the mothers' broader psychosocial experiences. Meta-analyses of observational studies on prenatal stress exposure indicate the direction and size of effects on child neurodevelopment are heterogeneous across studies. Inspired by lifespan and topological frameworks of adversity, we conceptualize individual variation in mothers' lived experience during and prior to pregnancy as a key determinant of these heterogeneous effects across populations. We structure our review to discuss experiential categories that may uniquely shape the psychological and biological influence of stress on pregnant mothers and their developing children, including current socioeconomic resources, exposure to chronic and traumatic stressors, culture and historical trauma, and the contours of prenatal stress itself. We conclude by identifying next steps that hold potential to meaningfully advance the field of fetal programming.

Grasp context-dependent uncertainty alters the relative contribution of anticipatory and feedback-based mechanisms in object manipulation

Predictive control within dexterous object manipulation while allowing for the choice of contact points has been shown to employ a predominantly feedback-based force modulation. The anticipation is thought to be facilitated through the internal representation of the object dynamics being integrated and updated on a trial-to-trial basis with the feedback of contact locations on the object. This is as opposed to the classically studied memory representation-based fingertip force control for grasping with pre-selected contact locations. We designed a study to examine this grasp context-dependent asymmetry in sensorimotor integration by introducing binary uncertainty about the grasp type before movement initiation within the framework of motor planning. An inverted T-shaped instrumented object was presented to 24 participants as the manipulandum, and they were asked to reach, grasp, and lift it while minimising the peak roll. We dissociated the planning and the execution phases by pseudo-randomly manipulating the availability of visual contact cues on the object after movement onset. We analysed both derived as well as direct kinetic and kinematic measures of the grasp during the loading phase to understand the anticipatory coordination. Our findings suggest that uncertainty about the grasp context during movement preparation resulted in a shift towards feedback-based mechanisms for grasp force modulation despite the persistence of visual cues.

Dopamine transients encode reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented by corticostriatal synaptic weights, which are updated by dopamine-dependent plasticity. This suggests that dopamine release reflects the product of the learning rate and RPE. Here, we characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc) in a volatile environment. Using a task with semi-observable states offering different rewards, we find that rats adjust how quickly they initiate trials across states using RPEs. Computational modeling and behavioral analyses show that learning rates are higher following state transitions and scale with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encodes RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

Computational processes of simultaneous learning of stochasticity and volatility in humans

Making adaptive decisions requires predicting outcomes, and this in turn requires adapting to uncertain environments. This study explores computational challenges in distinguishing two types of noise influencing predictions: volatility and stochasticity. Volatility refers to diffusion noise in latent causes, requiring a higher learning rate, while stochasticity introduces moment-to-moment observation noise and reduces learning rate. Dissociating these effects is challenging as both increase the variance of observations. Previous research examined these factors mostly separately, but it remains unclear whether and how humans dissociate them when they are played off against one another. In two large-scale experiments, through a behavioral prediction task and computational modeling, we report evidence of humans dissociating volatility and stochasticity solely based on their observations. We observed contrasting effects of volatility and stochasticity on learning rates, consistent with statistical principles. These results are consistent with a computational model that estimates volatility and stochasticity by balancing their dueling effects.

Contribution of amygdala to dynamic model arbitration under uncertainty

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes based on preceding cues or actions, commonly referred to as stimulus- and action-based learning. Ultimately, the brain also must adopt appropriate choice behavior using reliability of these models. Here, we combined multiple experimental and computational approaches to quantify concurrent learning in monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala. Specifically, we demonstrate that the amygdala adjusts the initial balance between the two learning systems, thereby altering the interaction between arbitration and learning that shapes the time course of both learning and choice behaviors. This novel role of the amygdala can account for existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

Multiple and subject-specific roles of uncertainty in reward-guided decision-making

Decision-making in noisy, changing, and partially observable environments entails a basic tradeoff between immediate reward and longer-term information gain, known as the exploration-exploitation dilemma. Computationally, an effective way to balance this tradeoff is by leveraging uncertainty to guide exploration. Yet, in humans, empirical findings are mixed, from suggesting uncertainty-seeking to indifference and avoidance. In a novel bandit task that better captures uncertainty-driven behavior, we find multiple roles for uncertainty in human choices. First, stable and psychologically meaningful individual differences in uncertainty preferences actually range from seeking to avoidance, which can manifest as null group-level effects. Second, uncertainty modulates the use of basic decision heuristics that imperfectly exploit immediate rewards: a repetition bias and win-stay-lose-shift heuristic. These heuristics interact with uncertainty, favoring heuristic choices under higher uncertainty. These results, highlighting the rich and varied structure of reward-based choice, are a step to understanding its functional basis and dysfunction in psychopathology.

Neural correlates of reward valuation in individuals with nonsuicidal self-injury under uncertainty

BACKGROUND

Attitudes toward risk and ambiguity significantly influence how individuals assess and value rewards. This fMRI study examines the reward valuation process under conditions of uncertainty and investigates the associated neural mechanisms in individuals who engage in nonsuicidal self-injury (NSSI) as a coping mechanism for psychological pain.

METHODS

The study involved 44 unmedicated individuals who reported five or more NSSI episodes in the past year, along with 42 age-, sex-, handedness-, IQ-, and socioeconomic status-matched controls. During the fMRI scans, all participants were presented with decision-making scenarios involving uncertainty, both in terms of risk (known probabilities) and ambiguity (unknown probabilities).

RESULTS

In the NSSI group, aversive attitudes toward ambiguity were correlated with increased emotion reactivity and greater method versatility. Whole-brain analysis revealed notable group-by-condition interactions in the right middle cingulate cortex and left hippocampus. Specifically, the NSSI group showed decreased neural activation under ambiguity v. risk compared to the control group. Moreover, reduced hippocampal activation under ambiguity in the NSSI group was associated with increased emotion regulation problems.

CONCLUSIONS

This study presents the first evidence of reduced brain activity in specific regions during value-based decision-making under conditions of ambiguity in individuals with NSSI. These findings have important clinical implications, particularly concerning emotion dysregulation in this population. This study indicates the need for interventions that support and guide individuals with NSSI to promote adaptive decision-making in the face of ambiguous uncertainty.

Same Same, But Different: Brain Areas Underlying the Learning from Repetitive Episodic Prediction Errors

Prediction errors (PEs) function as learning signals. It is yet unclear how varying compared to repetitive PEs affect episodic memory in brain and behavior. The current study investigated cerebral and behavioral effects of experiencing either multiple alternative versions ("varying") or one single alternative version ("repetitive") of a previously encoded episode. Participants encoded a set of episodes ("originals") by watching videos showing toy stories. During scanning, participants either experienced originals, one single, or multiple alternative versions of the previously encoded episodes. Participants' memory performance was tested through recall of original objects. Varying and repetitive PEs revealed typical brain responses to the detection of mismatching information including inferior frontal and posterior parietal regions, as well as hippocampus, which is further linked to memory reactivation, and the amygdala, known for modulating memory consolidation. Furthermore, experiencing varying and repetitive PEs triggered distinct brain areas as revealed by direct contrast. Among others, experiencing varying versions triggered activity in the caudate, a region that has been associated with PEs. In contrast, repetitive PEs activated brain areas that resembled more those for retrieval of originally encoded episodes. Thus, ACC and posterior cingulate cortex activation seemed to serve both reactivating old and integrating new but similar information in episodic memory. Consistent with neural findings, participants recalled original objects less accurately when only presented with the same, but not varying, PE during fMRI. The current findings suggest that repeated PEs interact more strongly with a recalled original episodic memory than varying PEs.

Dopamine transients encode reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc). We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

The Causes and Consequences of Drifting Expectations

Awaiting news of uncertain outcomes is distressing because the news might be disappointing. To prevent such disappointments, people often "brace for the worst," pessimistically lowering expectations before news arrives to decrease the possibility of surprising disappointment (a negative prediction error, or PE). Computational decision-making research commonly assumes that expectations do not drift within trials, yet it is unclear whether expectations pessimistically drift in real-world, high-stakes settings, what factors influence expectation drift, and whether it effectively buffers emotional responses to goal-relevant outcomes. Moreover, individuals learn from PEs to accurately anticipate future outcomes, but it is unknown whether expectation drift also impedes PE-based learning. In a sample of students awaiting exam grades (N = 625), we found that expectations often drift and tend to drift pessimistically. We demonstrate that bracing is preferentially modulated by uncertainty; it transiently buffers the initial emotional impact of negative PEs but impairs PE-based learning, counterintuitively sustaining uncertainty into the future.

Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing

The anterior cingulate cortex (ACC) has been implicated across multiple highly specialized cognitive functions-including task engagement, motivation, error detection, attention allocation, value processing, and action selection. Here, we ask if ACC lesions disrupt task performance and firing in dorsomedial striatum (DMS) during the performance of a reward-guided decision-making task that engages many of these cognitive functions. We found that ACC lesions impacted several facets of task performance-including decreasing the initiation and completion of trials, slowing reaction times, and resulting in suboptimal and inaccurate action selection. Reductions in movement times towards the end of behavioral sessions further suggested attenuations in motivation, which paralleled reductions in directional action selection signals in the DMS that were observed later in recording sessions. Surprisingly, however, beyond altered action signals late in sessions-neural correlates in the DMS were largely unaffected, even though behavior was disrupted at multiple levels. We conclude that ACC lesions result in overall deficits in task engagement that impact multiple facets of task performance during our reward-guided decision-making task, which-beyond impacting motivated action signals-arise from dysregulated attentional signals in the ACC and are mediated via downstream targets other than DMS.

Migrainous vertigo impairs adaptive learning as a function of uncertainty

Objective

In this study, we examined whether vestibular migraine, as a source of increased perceptual uncertainty due to the associated dizziness, interferes with adaptive learning.

Methods

The IOWA gambling task (IGT) was used to assess adaptive learning in both healthy controls and patients with migraine-related dizziness. Participants were presented with four decks of cards (A, B, C, and D) and requested to select a card over 100 trials. Participants received a monetary reward or a penalty with equal probability when they selected a card. Card decks A and B (high-risk decks) involved high rewards (win £100) and high penalties (lose £250), whereas C and D (low-risk decks; favorable reward-to-punishment ratio) involved lower rewards (win £50) and penalties (lose £50). Task success required participants to decide (i.e., adaptively learn) through the feedback they received that C and D were the advantageous decks.

Results

The study revealed that patients with vestibular migraine selected more high-risk cards than the control group. Chronic vestibular migraine patients showed delayed improvement in task performance than those with acute presentation. Only in acute vestibular migraine patients, we observed that impaired learning positively correlated with measures of dizzy symptoms.

Conclusion

The findings of this study have clinical implications for how vestibular migraine can affect behavioural adaption in patients, either directly through altered perception or indirectly by impacting cognitive processes that can result in maladaptive behavior.

Exploring the relationship between frequent cannabis use, belief updating under uncertainty and psychotic-like symptoms

Background

Cannabis users present an important group for investigating putative mechanisms underlying psychosis, as cannabis-use is associated with an increased risk of psychosis. Recent work suggests that alterations in belief-updating under uncertainty underlie psychosis. We therefore compared belief updating under uncertainty between cannabis and non-cannabis users.

Methods

49 regular cannabis users and 52 controls completed the Space Game, via an online platform used for behavioral testing. In the task, participants were asked to predict the location of the stimulus based on previous information, under different uncertainty conditions. Mixed effects models were used to identify significant predictors of mean score, confidence, performance error and learning rate.

Results

Both groups showed decreased confidence in high noise conditions, and increased belief updating in more volatile conditions, suggesting that they could infer the degree and sources of uncertainty. There were no significant effects of group on any of the performance indices. However, within the cannabis group, frequent users showed worse performance than less frequent users.

Conclusion

Belief updating under uncertainty is not affected by cannabis use status but could be impaired in those who use cannabis more frequently. This finding could show a similarity between frequent cannabis use and psychosis risk, as predictors for abnormal belief-updating.

Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy

Decision-making in uncertain environments often leads to varied outcomes. Understanding how individuals interpret the causes of unexpected feedback is crucial for adaptive behavior and mental well-being. Uncertainty can be broadly categorized into two components: volatility and stochasticity. Volatility is about how quickly conditions change, impacting results. Stochasticity, on the other hand, refers to outcomes affected by random chance or "luck". Understanding these factors enables individuals to have more effective environmental analysis and strategy implementation (explore or exploit) for future decisions. This study investigates how anxiety and apathy, two prevalent affective states, influence the perceptions of uncertainty and exploratory behavior. Participants (N = 1001) completed a restless three-armed bandit task that was analyzed using latent state models. Anxious individuals perceived uncertainty as more volatile, leading to increased exploration and learning rates, especially after reward omission. Conversely, apathetic individuals viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility-to-stochasticity ratio mediated the anxiety-exploration relationship post-adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy. These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision-making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.

Reinforcement learning of altruistic punishment differs between cultures and across the lifespan

Altruistic punishment is key to establishing cooperation and maintaining social order, yet its developmental trends across cultures remain unclear. Using computational reinforcement learning models, we provided the first evidence of how social feedback dynamically influences group-biased altruistic punishment across cultures and the lifespan. Study 1 (n = 371) found that Chinese participants exhibited higher learning rates than Americans when socially incentivized to punish unfair allocations. Additionally, Chinese adults showed slower learning and less exploration when punishing ingroups than outgroups, a pattern absent in American counterparts, potentially reflecting a tendency towards ingroup favoritism that may contribute to reinforcing collectivist values. Study 2 (n = 430, aged 12-52) further showed that such ingroup favoritism develops with age. Chinese participants' learning rates for ingroup punishment decreased from adolescence into adulthood, while outgroup rates stayed constant, implying a process of cultural learning. Our findings highlight cultural and age-related variations in altruistic punishment learning, with implications for social reinforcement learning and culturally sensitive educational practices promoting fairness and altruism.

Lesions to the mediodorsal thalamus, but not orbitofrontal cortex, enhance volatility beliefs linked to paranoia

Beliefs-attitudes toward some state of the environment-guide action selection and should be robust to variability but sensitive to meaningful change. Beliefs about volatility (expectation of change) are associated with paranoia in humans, but the brain regions responsible for volatility beliefs remain unknown. The orbitofrontal cortex (OFC) is central to adaptive behavior, whereas the magnocellular mediodorsal thalamus (MDmc) is essential for arbitrating between perceptions and action policies. We assessed belief updating in a three-choice probabilistic reversal learning task following excitotoxic lesions of the MDmc (n = 3) or OFC (n = 3) and compared performance with that of unoperated monkeys (n = 14). Computational analyses indicated a double dissociation: MDmc, but not OFC, lesions were associated with erratic switching behavior and heightened volatility belief (as in paranoia in humans), whereas OFC, but not MDmc, lesions were associated with increased lose-stay behavior and reward learning rates. Given the consilience across species and models, these results have implications for understanding paranoia.

Uncertainty of treatment efficacy moderates placebo effects on reinforcement learning

The placebo-reward hypothesis postulates that positive effects of treatment expectations on health (i.e., placebo effects) and reward processing share common neural underpinnings. Moreover, experiments in humans and animals indicate that reward uncertainty increases striatal dopamine, which is presumably involved in placebo responses and reward learning. Therefore, treatment uncertainty analogously to reward uncertainty may affect updating from rewards after placebo treatment. Here, we address whether different degrees of uncertainty regarding the efficacy of a sham treatment affect reward sensitivity. In an online between-subjects experiment with N = 141 participants, we systematically varied the provided efficacy instructions before participants first received a sham treatment that consisted of listening to binaural beats and then performed a probabilistic reinforcement learning task. We fitted a Q-learning model including two different learning rates for positive (gain) and negative (loss) reward prediction errors and an inverse gain parameter to behavioral decision data in the reinforcement learning task. Our results yielded an inverted-U-relationship between provided treatment efficacy probability and learning rates for gain, such that higher levels of treatment uncertainty, rather than of expected net efficacy, affect presumably dopamine-related reward learning. These findings support the placebo-reward hypothesis and suggest harnessing uncertainty in placebo treatment for recovering reward learning capabilities.

Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task

Our prior research has identified neural correlates of cognitive control in the anterior cingulate cortex (ACC), leading us to hypothesize that the ACC is necessary for increasing attention as rats flexibly learn new contingencies during a complex reward-guided decision-making task. Here, we tested this hypothesis by using optogenetics to transiently inhibit the ACC, while rats of either sex performed the same two-choice task. ACC inhibition had a profound impact on behavior that extended beyond deficits in attention during learning when expected outcomes were uncertain. We found that ACC inactivation slowed and reduced the number of trials rats initiated and impaired both their accuracy and their ability to complete sessions. Furthermore, drift-diffusion model analysis suggested that free-choice performance and evidence accumulation (i.e., reduced drift rates) were degraded during initial learning-leading to weaker associations that were more easily overridden in later trial blocks (i.e., stronger bias). Together, these results suggest that in addition to attention-related functions, the ACC contributes to the ability to initiate trials and generally stay on task.

What the eyes, confidence, and partner's identity can tell about change of mind

Perceptual confidence reflects the ability to evaluate the evidence that supports perceptual decisions. It is thought to play a critical role in guiding decision-making. However, only a few empirical studies have actually investigated the function of perceptual confidence. To address this issue, we designed a perceptual task in which participants provided a confidence judgment on the accuracy of their perceptual decision. Then, they viewed the response of a machine or human partner, and they were instructed to decide whether to keep or change their initial response. We observed that confidence predicted participants' changes of mind more than task difficulty and perceptual accuracy. Additionally, interacting with a machine, compared to a human, decreased confidence and increased participants tendency to change their initial decision, suggesting that both confidence and changes of mind are influenced by contextual factors, such as the identity of a partner. Finally, variations in confidence judgments but not change of mind were correlated with pre-response pupil dynamics, indicating that arousal changes are linked to confidence computations. This study contributes to our understanding of the factors influencing confidence and changes of mind and also evaluates the possibility of using pupil dynamics as a proxy of confidence.

Longitudinal trajectories of anterior cingulate glutamate and subclinical psychotic experiences in early adolescence: the impact of bullying victimization

Previous studies reported decreased glutamate levels in the anterior cingulate cortex (ACC) in non-treatment-resistant schizophrenia and first-episode psychosis. However, ACC glutamatergic changes in subjects at high-risk for psychosis, and the effects of commonly experienced environmental emotional/social stressors on glutamatergic function in adolescents remain unclear. In this study, adolescents recruited from the general population underwent proton magnetic resonance spectroscopy (MRS) of the pregenual ACC using a 3-Tesla scanner. We explored longitudinal data on the association of combined glutamate-glutamine (Glx) levels, measured by MRS, with subclinical psychotic experiences. Moreover, we investigated associations of bullying victimization, a risk factor for subclinical psychotic experiences, and help-seeking intentions, a coping strategy against stressors including bullying victimization, with Glx levels. Finally, path analyses were conducted to explore multivariate associations. For a contrast analysis, gamma-aminobutyric acid plus macromolecule (GABA+) levels were also analyzed. Negative associations were found between Glx levels and subclinical psychotic experiences at both Times 1 (n = 219, mean age 11.5 y) and 2 (n = 211, mean age 13.6 y), as well as for over-time changes (n = 157, mean interval 2.0 y). Moreover, effects of bullying victimization and bullying victimization × help-seeking intention interaction effects on Glx levels were found (n = 156). Specifically, bullying victimization decreased Glx levels, whereas help-seeking intention increased Glx levels only in bullied adolescents. Finally, associations among bullying victimization, help-seeking intention, Glx levels, and subclinical psychotic experiences were revealed. GABA+ analysis revealed no significant results. This is the first adolescent study to reveal longitudinal trajectories of the association between glutamatergic function and subclinical psychotic experiences and to elucidate the effect of commonly experienced environmental emotional/social stressors on glutamatergic function. Our findings may deepen the understanding of how environmental emotional/social stressors induce impaired glutamatergic neurotransmission that could be the underpinning of liability for psychotic experiences in early adolescence.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

Foraging Under Uncertainty Follows the Marginal Value Theorem with Bayesian Updating of Environment Representations

Foraging theory has been a remarkably successful approach to understanding the behavior of animals in many contexts. In patch-based foraging contexts, the marginal value theorem (MVT) shows that the optimal strategy is to leave a patch when the marginal rate of return declines to the average for the environment. However, the MVT is only valid in deterministic environments whose statistics are known to the forager; naturalistic environments seldom meet these strict requirements. As a result, the strategies used by foragers in naturalistic environments must be empirically investigated. We developed a novel behavioral task and a corresponding computational framework for studying patch-leaving decisions in head-fixed and freely moving mice. We varied between-patch travel time, as well as within-patch reward depletion rate, both deterministically and stochastically. We found that mice adopt patch residence times in a manner consistent with the MVT and not explainable by simple ethologically motivated heuristic strategies. Critically, behavior was best accounted for by a modified form of the MVT wherein environment representations were updated based on local variations in reward timing, captured by a Bayesian estimator and dynamic prior. Thus, we show that mice can strategically attend to, learn from, and exploit task structure on multiple timescales simultaneously, thereby efficiently foraging in volatile environments. The results provide a foundation for applying the systems neuroscience toolkit in freely moving and head-fixed mice to understand the neural basis of foraging under uncertainty.

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.

Controlling human causal inference through in silico task design

Learning causal relationships is crucial for survival. The human brain's functional flexibility allows for effective causal inference, underlying various learning processes. While past studies focused on environmental factors influencing causal inference, a fundamental question remains: can these factors be manipulated for strategic causal inference control? This paper presents a task control framework for orchestrating causal learning task design. It utilizes a two-player game setting where a neural network learns to manipulate task variables by interacting with a human causal inference model. Training the task controller to generate experimental designs, we confirm its ability to accommodate complexities of environmental causal structure. Experiments involving 126 human subjects successfully validate the impact of task control on performance and learning efficiency. Additionally, we find that task control policy reflects the intrinsic nature of human causal inference: one-shot learning. This framework holds promising potential for applications paving the way for targeted behavioral outcomes in humans.

Fast adaptation to rule switching using neuronal surprise

In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In a spiking network with several modules, previously learned rules are protected against overwriting, as long as the number of modules is larger than the total number of rules-making a step towards solving the stability-plasticity dilemma in neuroscience. Our model relates the subjective notion of surprise to specific predictions on the circuit level.

Synthetic surprise as the foundation of the psychedelic experience

Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Perseveration and Shifting in Obsessive-Compulsive Disorder as a Function of Uncertainty, Punishment, and Serotonergic Medication

Background

The nature of cognitive flexibility deficits in obsessive-compulsive disorder (OCD), which historically have been tested with probabilistic reversal learning tasks, remains elusive. Here, a novel deterministic reversal task and inclusion of unmedicated patients in the study sample illuminated the role of fixed versus uncertain rules/contingencies and of serotonergic medication. Additionally, our understanding of probabilistic reversal was enhanced through theoretical computational modeling of cognitive flexibility in OCD.

Methods

We recruited 49 patients with OCD, 21 of whom were unmedicated, and 43 healthy control participants matched for age, IQ, and gender. Participants were tested on 2 tasks: a novel visuomotor deterministic reversal learning task with 3 reversals (feedback rewarding/punishing/neutral) measuring accuracy/perseveration and a 2-choice visual probabilistic reversal learning task with uncertain feedback and a single reversal measuring win-stay and lose-shift. Bayesian computational modeling provided measures of learning rate, reinforcement sensitivity, and stimulus stickiness.

Results

Unmedicated patients with OCD were impaired on the deterministic reversal task under punishment only at the first and third reversals compared with both control participants and medicated patients with OCD, who had no deficit. Perseverative errors were correlated with OCD severity. On the probabilistic reversal task, unmedicated patients were only impaired at reversal, whereas medicated patients were impaired at both the learning and reversal stages. Computational modeling showed that the overall change was reduced feedback sensitivity in both OCD groups.

Conclusions

Both perseveration and increased shifting can be observed in OCD, depending on test conditions including the predictability of reinforcement. Perseveration was related to clinical severity and remediated by serotonergic medication.

Flexible learning in complex worlds

Cognitive flexibility can enhance the ability to adjust to changing environments. Here, we use learning simulations to investigate the possible advantages of flexible learning in volatile (changing) environments. We compare two established learning mechanisms, one with constant learning rates and one with rates that adjust to volatility. We study an ecologically relevant case of volatility, based on observations of developing cleaner fish Labroides dimidiatus that experience a transition from a simpler to a more complex foraging environment. There are other similar transitions in nature, such as migrating to a new and different habitat. We also examine two traditional approaches to volatile environments in experimental psychology and behavioral ecology: reversal learning, and learning set formation (consisting of a sequence of different discrimination tasks). These provide experimental measures of cognitive flexibility. Concerning transitions to a complex world, we show that both constant and flexible learning rates perform well, losing only a small proportion of available rewards in the period after a transition, but flexible rates perform better than constant rates. For reversal learning, flexible rates improve the performance with each successive reversal because of increasing learning rates, but this does not happen for constant rates. For learning set formation, we find no improvement in performance with successive shifts to new stimuli to discriminate for either flexible or constant learning rates. Flexible learning rates might thus explain increasing performance in reversal learning but not in learning set formation, and this can shed light on the nature of cognitive flexibility in a given system.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Dynamic Foraging Behavior Performance Is Not Affected by Scn2a Haploinsufficiency

Dysfunction in the gene SCN2A, which encodes the voltage-gated sodium channel Nav1.2, is strongly associated with neurodevelopmental disorders including autism spectrum disorder and intellectual disability (ASD/ID). This dysfunction typically manifests in these disorders as a haploinsufficiency, where loss of one copy of a gene cannot be compensated for by the other allele. Scn2a haploinsufficiency affects a range of cells and circuits across the brain, including associative neocortical circuits that are important for cognitive flexibility and decision-making behaviors. Here, we tested whether Scn2a haploinsufficiency has any effect on a dynamic foraging task that engages such circuits. Scn2a +/- mice and wild-type (WT) littermates were trained on a choice behavior where the probability of reward between two options varied dynamically across trials and where the location of the high reward underwent uncued reversals. Despite impairments in Scn2a-related neuronal excitability, we found that both male and female Scn2a +/- mice performed these tasks as well as wild-type littermates, with no behavioral difference across genotypes in learning or performance parameters. Varying the number of trials between reversals or probabilities of receiving reward did not result in an observable behavioral difference, either. These data suggest that, despite heterozygous loss of Scn2a, mice can perform relatively complex foraging tasks that make use of higher-order neuronal circuits.