Sex differences in learning from exploration

An Observational Study of Wood-Chewing in Mares and Their Foals Kept in Fenced Outdoor Paddocks with Variable Browsing Access

The natural behaviour of wood-chewing (WC) in horses may, under certain conditions, become excessive or it may precede crib-biting . However, the potential causes of WC, including frustrated foraging motivation) or developmental exploratory or teething behavior are poorly understood. We studied WC in 31 mares and their foals, examining associations with browsing access, stocking density, foal sex and performance of other non-nutritive oral behaviors. Behaviors (counts/hour) were recorded by all-occurrences sampling during focal, continuous observations of grazing,resting and suckling for 20  ×  10 min periods, for three days when foals were 3-5 months of age and kept at pasture with their mares. 17/31 mares and 28/31 foals exhibited WC, with foals performing more WC than mares (Mdn (IQR): foals: 0.95 (0.42,2.26); mares: 0.01 (0,0.19)). Colts exhibited significantly more WC than fillies (F1,13.07 = 4.855, p = 0.046), possibly because of sex-based differences in exploratory tendencies. The amount of browsing access had a differential effect on mares and foals (F1,40.29 = 3.720, p = 0.033). However, the amount of browsing access was partially confounded with foal age. Overall, we suggest that WC is associated more with developmental exploration than frustrated foraging access. Experimental work is needed to confirm our findings.

Disruptions in Reward-Guided Decision-Making Functions Are Predictive of Greater Oral Oxycodone Self-Administration in Male and Female Rats

Background

Problematic opioid use that emerges in a subset of individuals may be due to preexisting disruptions in the biobehavioral mechanisms that regulate drug use. The identity of these mechanisms is not known, but emerging evidence suggests that suboptimal decision making that is observable prior to drug use may contribute to the pathology of addiction.

Methods

The current study investigated the relationship between decision-making phenotypes and opioid-taking behaviors in male and female Long Evans rats. Adaptive decision-making processes were assessed using a probabilistic reversal learning task and oxycodone- (or vehicle, as a control) taking behaviors assessed daily in 32 sessions using a saccharin fading procedure that promoted dynamic intake of oxycodone. Tests of motivation, extinction, and reinstatement were also performed.

Results

Computational analyses of decision-making data identified data-driven metrics that predicted self-administration of oxycodone and addiction-relevant behaviors. Moreover, preexisting impairments in reward-guided decision making observed in female rats were associated with greater addiction-relevant behaviors when compared with males.

Conclusions

These results provide new insights into the biobehavioral mechanisms that regulate opiate-taking behaviors and offer a novel phenotypic approach for interrogating sex differences in addiction susceptibility and opioid use disorders.

Touchscreen Response Precision Is Sensitive to the Explore/Exploit Trade-off

The explore/exploit trade-off is a fundamental property of choice selection during reward-guided decision making, where the "same" choice can reflect either of these internal cognitive states. An unanswered question is whether the execution of a decision provides an underexplored measure of internal cognitive states. Touchscreens are increasingly used across species for cognitive testing and afford the ability to measure the precise location of choice touch responses. We examined how male and female mice in a restless bandit decision making task interacted with a touchscreen to determine if the explore/exploit trade-off, prior reward, and/or sex differences change the variability in the kinetics of touchscreen choices. During exploit states, successive touch responses are closer together than those made in an explore state, suggesting exploit states reflect periods of increased motor stereotypy. Although exploit decisions might be expected to be rewarded more frequently than explore decisions, we find that immediate past reward reduces choice variability independently of explore/exploit state. Male mice are more variable in their interactions with the touchscreen than females, even in low-variability trials such as exploit or following reward. These results suggest that as exploit behavior emerges in reward-guided decision making, all mice become less variable and more automated in both their choice and the actions taken to make that choice, but this occurs on a background of increased male variability. These data uncover the hidden potential for touchscreen decision making tasks to uncover the latent neural states that unite cognition and movement.

Beyond reward learning deficits: Exploration-exploitation instability reveals computational heterogeneity in value-based decision making in early psychosis

Psychosis spectrum illnesses are characterized by impaired goal-directed behavior and significant neurophysiological heterogeneity. To investigate the neurocomputational underpinnings of this heterogeneity, 75 participants with Early Psychosis (EP) and 68 controls completed a dynamic decision-making task. Consistent with prior studies, EP exhibited more choice switching, not explained by reward learning deficits, but instead by increased transition to exploration from exploitation. Bayesian modeling implicated elevated uncertainty intolerance and decision noise as independent contributors to suboptimal transition dynamics across individuals, which identified three computational subtypes with unique cognitive and symptom profiles. Replicating prior studies, a high decision-noise subtype emerged showing learning deficits and worse negative symptoms; our analyses further uncovered a normative subtype with worse mood symptoms and a novel uncertainty-intolerance subtype with higher hospitalization rates. These specific microcognitive disruptions underlying the distinct neurocomputational subtypes are individually measurable and may have the potential for targeted interventions.

Cumulative effect of stress on decisional exploration-to-exploitation switch assessed through a gambling task in female mice

Survival and well-being hinge on an organism's ability to evaluate options, weighing costs and benefits to make adaptive decisions. It has long been shown that stress influences cognition and reward-related behaviour, the nature of which depends on the stressor's type and duration as well as gene x environment interactions. However, how stress influence decision-making in females has not been completely elucidated. Here, we have developed a new mouse gambling task (mGT) adapted to assess decision-making under uncertainty and risk. Adult female C57BL/6JRj mice administered with corticosterone (CORT) for 5 or 8 weeks reached similar final performance in the mGT as vehicle-treated controls. All groups tended to learn to maximize gain as the task progressed. Our results revealed that individual choice kinetics is impacted by chronic exposure to CORT, showing an accentuated sensitivity to penalties in female mice. These results confirm the suitability of our new mGT to assess decision-making under uncertainty and risk and are in line with previous reports of the effect of chronic CORT treatment on decision-making in male mice. Thereby this study provides new insights into the influence of sex and stress on decision-making.

Individual differences in sequential decision-making

1People differ widely in how they make decisions in uncertain environments. While many studies leverage this variability to measure differences in specific cognitive processes and parameters, the key dimension(s) of individual variability in uncertain decision-making tasks has not been identified. Here, we analyzed behavioral data from 1001 participants performing a restless three-armed bandit task, where reward probabilities fluctuated unpredictably over time. Using a novel analytical approach that controlled for the stochasticity in this tasks, we identified a dominant nonlinear axis of individual variability. We found that this primary axis of variability was strongly and selectively correlated with the probability of exploration, as inferred by latent state modeling. This suggests that the major factor shaping individual differences in bandit task performance is the tendency to explore (versus exploit), rather than personality characteristics, reinforcement learning model parameters, or low-level strategies. Certain demographic characteristics also predicted variance along this principle axis: participants at the exploratory end tended to be younger than participants at the exploitative end, and self-identified men were overrepresented at both extremes. Together, these findings offer a principled framework for understanding individual differences in task behavior while highlighting the cognitive and demographic factors that shape individual differences in decision-making under uncertainty.

Chronic ethanol exposure produces sex-dependent impairments in value computations in the striatum

Value-based decision-making relies on the striatum, where neural plasticity can be altered by chronic ethanol (EtOH) exposure, but the effects of such plasticity on striatal neural dynamics during decision-making remain unclear. This study investigated the long-term impacts of EtOH on reward-driven decision-making and striatal neurocomputations in male and female rats using a dynamic probabilistic reversal learning task. Following a prolonged withdrawal period, EtOH-exposed male rats exhibited deficits in adaptability and exploratory behavior, with aberrant outcome-driven value updating that heightened preference for chosen action. These behavioral changes were linked to altered neural activity in the dorsomedial striatum (DMS), where EtOH increased outcome-related encoding and decreased choice-related encoding. In contrast, female rats showed minimal behavioral changes with distinct EtOH-evoked alterations of neural activity, revealing significant sex differences in the impact of chronic EtOH. Our findings underscore the impact of chronic EtOH exposure on adaptive decision-making, revealing enduring changes in neurocomputational processes in the striatum underlying cognitive deficits that differ by sex.

Sex-biased computations underlying differential set shift performance in mice

Cognitive flexibility can be defined as the ability to adaptively shift between choices or strategies based on environmental feedback and it is disrupted in numerous neuropsychiatric conditions. Individual differences in the computations supporting cognitive flexibility are poised to reveal mechanisms of neuropsychiatric risk and resilience. One critical variable well known to influence individual differences in neuropsychiatric risk is sex. While previous research has identified sex differences in value based decision making in mice, whether sex reflects a major source of variation in cognitive flexibility remains unknown. To directly assess sex-biased individual differences in cognitive flexibility, we developed a novel touchscreen Set Shift task that permits robust and continuous testing in mice. Using this task, we discovered that female mice completed significantly more rule shifts with fewer errors than males. We next employed a suite of computational models that revealed sex-biased individual differences in the computations underlying cognitive flexibility. Overall, our results suggest that following rule shifts, female mice learn the new rule faster and commit to exploiting rule choices sooner compared to males - sometimes because they commit to multiple rules simultaneously. This suggests that increased choice stability in female rodents enhances commitment to a strategy during periods of uncertainty and directly contributes to increased rule shifting. This supports the counterintuitive conclusion that a high degree of stable choice is a strong requirement for enhanced cognitive flexibility in the Set Shift task, one of the gold standard cognitive flexibility tasks.

The curious interpretation of novel object recognition tests

Novel object recognition tasks are commonly used to assess memory in rodents. These tests rely on an innate preference for exploring objects that are new or have been moved or changed. However, this preference, while normally seen in control conditions, is not immutable. Stressful experiences as well as lesions and genetic mutations can lead mice and rats to show clear preferences for exploring familiar objects and familiar locations. This opinion article discusses the evidence for changes in novelty preference, implications of this lability for assessing memory, and the significance of shifts in novelty preference as a readout of changes in curiosity with implications in approach-avoidance behavior and explore-exploit decision-making. Finally, we provide some recommendations for reporting and interpreting novelty preference task findings moving forward.

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic view is that we calculate the value of each option, compare them, and choose the most rewarding modulo exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold and then explore alternatives. Because the fields use incompatible methods, it remains unclear which view better describes human decision-making. Here, we found that humans use compare-to-threshold computations in classic compare-alternative tasks. Because compare-alternative computations are central to the reinforcement-learning (RL) models typically used in the cognitive and brain sciences, we developed a novel compare-to-threshold model ("foraging"). Compared to previous RL models, the foraging model better fit participant behavior, better predicted the tendency to repeat choices, and predicted held-out participants that were almost impossible under compare-alternative models. These results suggest that humans use compare-to-threshold computations in sequential decision-making.

Hippocampal neuroinflammation induced by lipopolysaccharide causes sex-specific disruptions in action selection, food approach memories, and neuronal activation

Hippocampal neuroinflammation is present in multiple diseases and disorders that impact motivated behaviour in a sex-specific manner, but whether neuroinflammation alone is sufficient to disrupt this behaviour is unknown. We investigated this question here using mice. First, the application of an endotoxin to primary cultures containing only hippocampal neurons did not affect their activation. However, when the same endotoxin was applied to mixed neuronal/glial cultures it did increase neuronal activation, providing initial indications of how it might be able to effect behavioural change. We next showed neuroinflammatory effects on behaviour directly, demonstrating that intra-hippocampal administration of the same endotoxin increased locomotor activity and accelerated goal-directed learning in both male and female mice. In contrast, lipopolysaccharide-induced hippocampal neuroinflammation caused sex-specific disruptions to the acquisition of instrumental actions and to Pavlovian food-approach memories. Finally, we showed that LPS-induced hippocampal neuroinflammation had a sexually dimorphic effect on neuronal activation: increasing it in females and decreasing it in males.

Distinct Computational Mechanisms of Uncertainty Processing Explain Opposing Exploratory Behaviors in Anxiety and Apathy

BACKGROUND

Decision making in uncertain environments can lead to varied outcomes, and how we process those outcomes may depend on our emotional state. Understanding how individuals interpret the sources of uncertainty is crucial for understanding adaptive behavior and mental well-being. Uncertainty can be broadly categorized into 2 components: volatility and stochasticity. Volatility describes how quickly conditions change. Stochasticity, on the other hand, refers to outcome randomness. We investigated how anxiety and apathy influenced people's perceptions of uncertainty and how uncertainty perception shaped explore-exploit decisions.

METHODS

Participants (N = 1001, nonclinical sample) completed a restless 3-armed bandit task that was analyzed using both latent state and process models.

RESULTS

Individuals with anxiety perceived uncertainty as resulting more from volatility, leading to increased exploration and learning rates, especially after reward omission. Conversely, individuals with apathy viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility to stochasticity ratio mediated the anxiety-exploration relationship post adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy.

CONCLUSIONS

These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.

HMM for discovering decision-making dynamics using reinforcement learning experiments

Major depressive disorder (MDD), a leading cause of years of life lived with disability, presents challenges in diagnosis and treatment due to its complex and heterogeneous nature. Emerging evidence indicates that reward processing abnormalities may serve as a behavioral marker for MDD. To measure reward processing, patients perform computer-based behavioral tasks that involve making choices or responding to stimulants that are associated with different outcomes, such as gains or losses in the laboratory. Reinforcement learning (RL) models are fitted to extract parameters that measure various aspects of reward processing (e.g. reward sensitivity) to characterize how patients make decisions in behavioral tasks. Recent findings suggest the inadequacy of characterizing reward learning solely based on a single RL model; instead, there may be a switching of decision-making processes between multiple strategies. An important scientific question is how the dynamics of strategies in decision-making affect the reward learning ability of individuals with MDD. Motivated by the probabilistic reward task within the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, we propose a novel RL-HMM (hidden Markov model) framework for analyzing reward-based decision-making. Our model accommodates decision-making strategy switching between two distinct approaches under an HMM: subjects making decisions based on the RL model or opting for random choices. We account for continuous RL state space and allow time-varying transition probabilities in the HMM. We introduce a computationally efficient Expectation-maximization (EM) algorithm for parameter estimation and use a nonparametric bootstrap for inference. Extensive simulation studies validate the finite-sample performance of our method. We apply our approach to the EMBARC study to show that MDD patients are less engaged in RL compared to the healthy controls, and engagement is associated with brain activities in the negative affect circuitry during an emotional conflict task.

Altered trial-to-trial responses to reward outcomes in KCNMA1 knockout mice during probabilistic learning tasks

The large-conductance calcium- and voltage-activated potassium (BK) channels, encoded by the KCNMA1 gene, play important roles in neuronal function. Mutations in KCNMA1 have been found in patients with various neurodevelopmental features, including intellectual disability, autism spectrum disorder (ASD), or attention deficit hyperactivity disorder (ADHD). Previous studies of KCNMA1 knockout mice have suggested altered activity patterns and behavioral flexibility, but it remained unclear whether these changes primarily affect immediate behavioral adaptation or longer-term learning processes. Using a 5-armed bandit task (5-ABT) and a novel Δrepeat rate analysis method that considers individual baseline choice tendencies, we investigated immediate trial-by-trial Win-Stay-Lose-Shift (WSLS) strategies and learning rates across multiple trials in KCNMA1 knockout (KCNMA1-/-) mice. Three key findings emerged: (1) Unlike wildtype mice, which showed increased Δrepeat rates after rewards and decreased rates after losses, KCNMA1-/- mice exhibited impaired WSLS behavior, (2) KCNMA1-/- mice displayed shortened response intervals after unrewarded trials, and (3) despite these short-term behavioral impairments, their learning rates and task accuracy remained comparable to wildtype mice, with significantly shorter task completion times. These results suggest that BK channel dysfunction primarily alters immediate behavioral responses to outcomes in the next trial rather than affecting long-term learning capabilities. These findings and our analytical method may help identify behavioral phenotypes in animal models of both BK channel-related and other neurodevelopmental disorders.

Sex-dependent effects of early life sensory overstimulation on later life behavioral function in rats

Children today are immersed in electronic technology shortly after birth as they now begin regularly watching television earlier than they did in the past. Many new programs geared towards infants contain lots of lights, color, and sounds, which may constitute a form of sensory overstimulation (SOS) that leads to cognitive and behavioral changes in children and adolescents. Here, we examined the impact of early life SOS exposure on later life behavioral and cognitive function in rodents by exposing developing male and female rats to excessive audiovisual stimulation from postnatal days (PND) 10-40 and assessing anxiety-like behavior, social motivation, compulsive behavior, and spatial learning/cognition from PND 50-60. To evaluate potential SOS effects on hypothalamic-pituitary-adrenal (HPA)-axis function, levels of the stress hormone corticosterone (CORT) were measured at 3 timepoints (e.g., PND 23, 41, 61) post-SOS exposure. Sensory overstimulated males exhibited reduced anxiety-like and compulsive behavior compared to controls, whereas females exhibited reduced social motivation but enhanced spatial learning/cognition compared to controls. No differences in baseline CORT levels were found at any age tested, suggesting no impact of early life SOS on later life basal HPA-axis function. Our results demonstrate sex-specific effects of early life SOS on distinct behavioral domains in early adult rats.

Dopamine and Norepinephrine Differentially Mediate the Exploration-Exploitation Tradeoff

Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision-making processes. Although two neuromodulators share a synthesis pathway and are coactivated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision-making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision-making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol) and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Touchscreen response precision is sensitive to the explore/exploit tradeoff

The explore/exploit tradeoff is a fundamental property of choice selection during reward-guided decision making. In perceptual decision making, higher certainty decisions are more motorically precise, even when the decision does not require motor accuracy. However, while we can parametrically control uncertainty in perceptual tasks, we do not know what variables - if any - shape motor precision and reflect subjective certainty during reward-guided decision making. Touchscreens are increasingly used across species to measure choice, but provide no tactile feedback on whether an action is precise or not, and therefore provide a valuable opportunity to determine whether actions differ in precision due to explore/exploit state, reward, or individual variables. We find all three of these factors exert independent drives towards increased precision. During exploit states, successive touches to the same choice are closer together than those made in an explore state, consistent with exploit states reflecting higher certainty and/or motor stereotypy in responding. However, exploit decisions might be expected to be rewarded more frequently than explore decisions. We find that exploit choice precision is increased independently of a separate increase in precision due to immediate past reward, suggesting multiple mechanisms regulating choice precision. Finally, we see evidence that male mice in general are less precise in their interactions with the touchscreen than females, even when exploiting a choice. These results suggest that as exploit behavior emerges in reward-guided decision making, individuals become more motorically precise reflecting increased certainty, even when decision choice does not require additional motor accuracy, but this is influenced by individual differences and prior reward. These data uncover the hidden potential for touchscreen tasks in any species to uncover the latent neural states that unite cognition and movement.

Sex differences in motivational biases over instrumental actions

Motivational (i.e., appetitive or aversive) cues can bias value-based decisions by affecting either direction and intensity of instrumental actions. Despite several findings describing important interindividual differences in these biases, whether biological sex can also play a role is still up to debate. By comparing females and males in both appetitive and aversive Pavlovian-to-Instrumental Transfer paradigms we found that, while motivational cues similarly bias the direction of instrumental actions in both sexes, the intensity of such actions is increased by the cue in male participants only. The present results constitute compelling evidence that a crucial motivational bias of daily actions directed to obtaining rewards or avoiding punishments is modulated by biological sex. This evidence sheds new light on the role of sex in motivational processes that underlie decision-making, highlighting the importance of considering sex as a crucial factor in future research on this topic.

Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety

In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.

Sex differences in neural representations of social and nonsocial reward in the medial prefrontal cortex

The reinforcing nature of social interactions is necessary for the maintenance of appropriate social behavior. However, the neural substrates underlying social reward processing and how they might differ based on the sex and internal state of the animal remains unknown. It is also unclear whether these neural substrates are shared with those involved in nonsocial rewarding processing. We developed a fully automated, two choice (social-sucrose) operant assay in which mice choose between social and nonsocial rewards to directly compare the reward-related behaviors associated with two competing stimuli. We performed cellular resolution calcium imaging of medial prefrontal cortex (mPFC) neurons in male and female mice across varying states of water restriction and social isolation. We found that mPFC neurons maintain largely non-overlapping, flexible representations of social and nonsocial reward that vary with internal state in a sex-dependent manner. Additionally, optogenetic manipulation of mPFC activity during the reward period of the assay disrupted reward-seeking behavior across male and female mice. Thus, using a two choice operant assay, we have identified sex-dependent, non-overlapping neural representations of social and nonsocial reward in the mPFC that vary with internal state and that are essential for appropriate reward-seeking behavior.

Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy

Decision-making in uncertain environments often leads to varied outcomes. Understanding how individuals interpret the causes of unexpected feedback is crucial for adaptive behavior and mental well-being. Uncertainty can be broadly categorized into two components: volatility and stochasticity. Volatility is about how quickly conditions change, impacting results. Stochasticity, on the other hand, refers to outcomes affected by random chance or "luck". Understanding these factors enables individuals to have more effective environmental analysis and strategy implementation (explore or exploit) for future decisions. This study investigates how anxiety and apathy, two prevalent affective states, influence the perceptions of uncertainty and exploratory behavior. Participants (N = 1001) completed a restless three-armed bandit task that was analyzed using latent state models. Anxious individuals perceived uncertainty as more volatile, leading to increased exploration and learning rates, especially after reward omission. Conversely, apathetic individuals viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility-to-stochasticity ratio mediated the anxiety-exploration relationship post-adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy. These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision-making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.

Dopaminergic lesions of the anterior cingulate cortex of rats increase vulnerability to salient distractors

The anterior cingulate cortex (ACC) has been shown to be critical to many aspects of executive function including filtering irrelevant information, updating response contingencies when reinforcement contingencies change and stabilizing task sets. Nonspecific lesions to this region in rats produce a vulnerability to distractors that have gained salience through prior associations with reinforcement. These lesions also exacerbate cognitive fatigue in tests of sustained attention but do not produce global attentional impairments nor do they produce distractibility to novel distractors that do not have a prior association with reinforcement. To determine the neurochemical basis of these cognitive impairments, dopaminergically selective lesions of the ACC were made in both male and female Long-Evans, hooded rats prior to assessment in two attentional tasks. Dopaminergic lesions of the ACC increase the vulnerability of subjects to previously reinforced distractors and impede formation of an attentional set. Lesioned rats were not more susceptible to the effects of novel, irrelevant stimuli in a test of sustained attention as has been previously shown. Additionally, the effects of dopaminergic lesions were found to differ based on sex. Lesioned female, but not male, rats were more vulnerable than sham-lesioned females to the effects of prolonged testing and the removal of reinforcement during a test of sustained attention. Together, these data support the hypothesis that dopamine in the ACC is critical to filtering distractors whose salience has been gained through reinforcement.

Sex mechanisms as nonbinary influences on cognitive diversity

Essentially all neuropsychiatric diagnoses show some degree of sex and/or gender differences in their etiology, diagnosis, or prognosis. As a result, the roles of sex-related variables in behavior and cognition are of strong interest to many, with several lines of research showing effects on executive functions and value-based decision making in particular. These findings are often framed within a sex binary, with behavior of females described as less optimal than male "defaults"-- a framing that pits males and females against each other and deemphasizes the enormous overlap in fundamental neural mechanisms across sexes. Here, we propose an alternative framework in which sex-related factors encompass just one subset of many sources of valuable diversity in cognition. First, we review literature establishing multidimensional, nonbinary impacts of factors related to sex chromosomes and endocrine mechanisms on cognition, focusing on value- based decision-making tasks. Next, we present two suggestions for nonbinary interpretations and analyses of sex-related data that can be implemented by behavioral neuroscientists without devoting laboratory resources to delving into mechanisms underlying sex differences. We recommend (1) shifting interpretations of behavior away from performance metrics and towards strategy assessments to avoid the fallacy that the performance of one sex is worse than another; and (2) asking how much variance sex explains in measures and whether any differences are mosaic rather than binary, to avoid assuming that sex differences in separate measures are inextricably correlated. Nonbinary frameworks in research on cognition will allow neuroscience to represent the full spectrum of brains and behaviors.

The explore/exploit trade-off: An ecologically valid and translational framework that can advance mechanistic understanding of eating disorders

The explore/exploit trade-off is a decision-making process that is conserved across species and balances exploring unfamiliar choices of unknown value with choosing familiar options of known value to maximize reward. This framework is rooted in behavioral ecology and has traditionally been used to study maladaptive versus adaptive non-human animal foraging behavior. Researchers have begun to recognize the potential utility of understanding human decision-making and psychopathology through the explore/exploit trade-off. In this article, we propose that explore/exploit trade-off holds promise for advancing our mechanistic understanding of decision-making processes that confer vulnerability for and maintain eating pathology due to its neurodevelopmental bases, conservation across species, and ability to be mathematically modeled. We present a model for how suboptimal explore/exploit decision-making can promote disordered eating and present recommendations for future research applying this framework to eating pathology. Taken together, the explore/exploit trade-off provides a translational framework for expanding etiologic and maintenance models of eating pathology, given developmental changes in explore/exploit decision-making that coincide in time with the emergence of eating pathology and evidence of biased explore/exploit decision-making in psychopathology. Additionally, understanding explore/exploit decision-making in eating disorders may improve knowledge of their underlying pathophysiology, informing targeted clinical interventions such as neuromodulation and pharmacotherapy. PUBLIC SIGNIFICANCE STATEMENT: The explore/exploit trade-off is a cross-species decision-making process whereby organisms choose between a known option with a known reward or sampling unfamiliar options. We hypothesize that imbalanced explore/exploit decision-making can promote disordered eating and present preliminary data. We propose that explore/exploit trade-off has significant potential to advance understanding of the neurocognitive and neurodevelopmental mechanisms of eating pathology, which could ultimately guide revisions of etiologic models and inform novel interventions.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

Perineuronal nets are associated with decision making under conditions of uncertainty in female but not male mice

Biological sex influences decision-making processes in significant ways, differentiating the responses animals choose when faced with a range of stimuli. The neurobiological underpinnings that dictate sex differences in decision-making tasks remains an important open question, yet single-sex studies of males form most studies in behavioural neuroscience. Here we used female and male BALB/c mice on two spatial learning and memory tasks and examined the expression of perineuronal nets (PNNs) and parvalbumin interneurons (PV) in regions correlated with spatial memory. Mice underwent the aversive active place avoidance (APA) task or the appetitive trial-unique nonmatching-to-location (TUNL) touchscreen task. Mice in the APA cohort learnt to avoid the foot-shock and no differences were observed on key measures of the task nor in the number and intensity of PNNs and PV. On the delay but not separation manipulation in the TUNL task, females received more incorrect trials and less correct trials compared to males. Furthermore, females in this cohort exhibited higher intensity PNNs and PV cells in the agranular and granular retrosplenial cortex, compared to males. These data show that female and male mice perform similarly on spatial learning tasks. However, sex differences in neural circuitry may underly differences in making decisions under conditions of uncertainty on an appetitive task. These data emphasise the importance of using mice of both sexes in studies of decision-making neuroscience.

Sex differences in the context dependency of episodic memory

Context contributes to multiple aspects of human episodic memory including segmentation and retrieval. The present studies tested if, in adult male and female mice, context influences the encoding of odors encountered in a single unsupervised sampling session of the type used for the routine acquisition of episodic memories. The three paradigms used differed in complexity (single vs. multiple odor cues) and period from sampling to testing. Results show that males consistently encode odors in a context-dependent manner: the mice discriminated novel from previously sampled cues when tested in the chamber of initial cue sampling but not in a distinct yet familiar chamber. This was independent of the interval between cue encounters or the latency from initial sampling to testing. In contrast, female mice acquired both single cues and the elements of multi-cue episodes, but recall of that information was dependent upon the surrounding context only when the cues were presented serially. These results extend the list of episodic memory features expressed by rodents and also introduce a striking and unexpected sex difference in context effects.

Appetitively motivated tasks in the IntelliCage reveal a higher motivational cost of spatial learning in male than female mice

The IntelliCage (IC) permits the assessment of the behavior and learning abilities of mice in a social home cage context. To overcome water deprivation as an aversive driver of learning, we developed protocols in which spatial learning is motivated appetitively by the preference of mice for sweetened over plain water. While plain water is available at all times, only correct task responses give access to sweetened water rewards. Under these conditions, C57BL/6J mice successfully mastered a corner preference task with the reversal and also learned a more difficult time-place task with reversal. However, the rate of responding to sweetened water decreased strongly with increasing task difficulty, indicating that learning challenges and reduced success in obtaining rewards decreased the motivation of the animals to seek sweetened water. While C57BL/6J mice of both sexes showed similar initial taste preferences and learned similarly well in simple learning tasks, the rate of responding to sweetened water and performance dropped more rapidly in male than in female mice in response to increasing learning challenges. Taken together, our data indicate that male mice can have a disadvantage relative to females in mastering difficult, appetitively motivated learning tasks, likely due to sex differences in value-based decision-making.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Refinement of IntelliCage protocols for complex cognitive tasks through replacement of drinking restrictions by incentive-disincentive paradigms

The IntelliCage allows automated testing of cognitive abilities of mice in a social home cage environment without handling by human experimenters. Restricted water access in combination with protocols in which only correct responses give access to water is a reliable learning motivator for hippocampus-dependent tasks assessing spatial memory and executive function. However, water restriction may negatively impact on animal welfare, especially in poor learners. To better comply with the 3R principles, we previously tested protocols in which water was freely available but additional access to sweetened water could be obtained by learning a task rule. While this purely appetitive motivation worked for simple tasks, too many mice lost interest in the sweet reward during more difficult hippocampus-dependent tasks. In the present study, we tested a battery of increasingly difficult spatial tasks in which water was still available without learning the task rule, but rendered less attractive either by adding bitter tasting quinine or by increasing the amount of work to obtain it. As in previous protocols, learning of the task rule provided access to water sweetened with saccharin. The two approaches of dual motivation were tested in two cohorts of female C57BL/6 N mice. Compared to purely appetitive motivation, both novel protocols strongly improved task engagement and increased task performance. Importantly, neither of the added disincentives had an adverse impact on liquid consumption, health status or body weight of the animals. Our results show that it is possible to refine test protocols in the IntelliCage so that they challenge cognitive functions without restricting access to water.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

The influence of sex difference on behavior and adult hippocampal neurogenesis in C57BL/6 mice

Animal models have been used extensively in in vivo studies, especially within the biomedical field. Traditionally, single-sex studies, mostly males, are used to avoid any potential confounding variation caused by sex difference and the female estrous cycle. Historically, female animal subjects are believed to exhibit higher variability, and this could increase the statistical power needed to test a hypothesis. This study sets out to evaluate whether a sex difference does exist in mouse behavior, and whether female mice featured higher variability. We assessed the sensorimotor skills, anxiety-like behavior, depression-like behavior, and cognitive abilities of mice through a series of commonly used behavioral tests. Except for the stronger grip force and lower tactile sensory sensitivity detected in male mice, there was no significant difference between males and females in other tests. Furthermore, immunolabeling of neurogenesis markers suggested no significant difference between sexes in adult hippocampal neurogenesis. Within group variances were equivalent; females did not exhibit higher variability than males. However, the overall negative results could be due to the limitation of small sample size. In conclusion, our study provides evidence that sex difference in mice does not significantly influence these commonly used behavioral tests nor adult neurogenesis under basal conditions. We suggest that female mice could also be considered for test inclusion in future experiment design.

Stress, associative learning, and decision-making

Exposure to acute and chronic stress has significant effects on the basic mechanisms of associative learning and memory. Stress can both impair and enhance associative learning depending on type, intensity, and persistence of the stressor, the subject's sex, the context that the stress and behavior is experienced in, and the type of associative learning taking place. In some cases, stress can cause or exacerbate the maladaptive behavior that underlies numerous psychiatric conditions including anxiety disorders, obsessive-compulsive disorder, post-traumatic stress disorder, substance use disorder, and others. Therefore, it is critical to understand how the varied effects of stress, which may normally facilitate adaptive behavior, can also become maladaptive and even harmful. In this review, we highlight several findings of associative learning and decision-making processes that are affected by stress in both human and non-human subjects and how they are related to one another. An emerging theme from this work is that stress biases behavior towards less flexible strategies that may reflect a cautious insensitivity to changing contingencies. We consider how this inflexibility has been observed in different associative learning procedures and suggest that a goal for the field should be to clarify how factors such as sex and previous experience influence this inflexibility.

Mixtures of strategies underlie rodent behavior during reversal learning

In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.

Sex-dependent noradrenergic modulation of premotor cortex during decision-making

Rodent premotor cortex (M2) integrates information from sensory and cognitive networks for action planning during goal-directed decision-making. M2 function is regulated by cortical inputs and ascending neuromodulators, including norepinephrine (NE) released from the locus coeruleus (LC). LC-NE has been shown to modulate the signal-to-noise ratio of neural representations in target cortical regions, increasing the salience of relevant stimuli. Using rats performing a two-alternative forced choice task after administration of a β-noradrenergic antagonist (propranolol), we show that β-noradrenergic signaling is necessary for effective action plan signals in anterior M2. Loss of β-noradrenergic signaling results in failure to suppress irrelevant action plans in anterior M2 disrupting decoding of cue-related information, delaying decision times, and increasing trial omissions, particularly in females. Furthermore, we identify a potential mechanism for the sex bias in behavioral and neural changes after propranolol administration via differential expression of β2 noradrenergic receptor RNA across sexes in anterior M2, particularly on local inhibitory neurons. Overall, we show a critical role for β-noradrenergic signaling in anterior M2 during decision-making by suppressing irrelevant information to enable efficient action planning and decision-making.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Prolonged Physiological Stress Is Associated With a Lower Rate of Exploratory Learning That Is Compounded by Depression

BACKGROUND

Stress is a major risk factor for depression, and both are associated with important changes in decision-making patterns. However, decades of research have only weakly connected physiological measurements of stress to the subjective experience of depression. Here, we examined the relationship between prolonged physiological stress, mood, and explore-exploit decision making in a population navigating a dynamic environment under stress: health care workers during the COVID-19 pandemic.

METHODS

We measured hair cortisol levels in health care workers who completed symptom surveys and performed an explore-exploit restless-bandit decision-making task; 32 participants were included in the final analysis. Hidden Markov and reinforcement learning models assessed task behavior.

RESULTS

Participants with higher hair cortisol exhibited less exploration (r = -0.36, p = .046). Higher cortisol levels predicted less learning during exploration (β = -0.42, false discovery rate [FDR]-corrected p [pFDR] = .022). Importantly, mood did not independently correlate with cortisol concentration, but rather explained additional variance (β = 0.46, pFDR = .022) and strengthened the relationship between higher cortisol and lower levels of exploratory learning (β = -0.47, pFDR = .022) in a joint model. These results were corroborated by a reinforcement learning model, which revealed less learning with higher hair cortisol and low mood (β = -0.67, pFDR = .002).

CONCLUSIONS

These results imply that prolonged physiological stress may limit learning from new information and lead to cognitive rigidity, potentially contributing to burnout. Decision-making measures link subjective mood states to measured physiological stress, suggesting that they should be incorporated into future biomarker studies of mood and stress conditions.

Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff

The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

A low dimensional manifold of human exploratory behavior reveals opposing roles for apathy and anxiety

Exploration-exploitation decision-making is a feature of daily life that is altered in a number of neuropsychiatric conditions. Humans display a range of exploration and exploitation behaviors, which can be affected by apathy and anxiety. It remains unknown how factors underlying decision-making generate the spectrum of observed exploration-exploitation behavior and how they relate to states of anxiety and apathy. Here, we report a latent structure underlying sequential exploration and exploitation decisions that explains variation in anxiety and apathy. 1001 participants in a gender-balanced sample completed a three-armed restless bandit task along with psychiatric symptom surveys. Using dimensionality reduction methods, we found that decision sequences reduced to a low-dimensional manifold. The axes of this manifold explained individual differences in the balance between states of exploration and exploitation and the stability of those states, as determined by a statistical mechanics model of decision-making. Position along the balance axis was correlated with opposing symptoms of behavioral apathy and anxiety, while position along the stability axis correlated with the level of emotional apathy. This result resolves a paradox over how these symptoms can be correlated in samples but have opposite effects on behavior. Furthermore, this work provides a basis for using behavioral manifolds to reveal relationships between behavioral dynamics and affective states, with important implications for behavioral measurement approaches to neuropsychiatric conditions.

Pupil size predicts the onset of exploration in brain and behavior

In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible.

Sex differences in learning and performing the Go/NoGo tasks

BACKGROUND

The quality of learning and post-learning performances is critical for daily life. The behavioral flexibility is equally important for adapting the changing circumstances. The learning process requires repeated practices, which enhances prompt and proper behavioral responses, in turn, which promotes habits formation as well. Despite the well-documented sex differences in learning and performances, contradictory results were reported. A possible cause might be a systematic analysis due to specific research interests, regardless of the continuity of natural acquisition process. Here, we investigate the potential sex differences in learning, performances and adjustments of habited behaviors with regular and reversal Go/NoGo tasks.

METHODS

Both male and female Sprague-Dawley rats were used in this study. All rats were trained for a regular rodent Go/NoGo task and a subset of rats were trained for a reversal rodent Go/NoGo task, both with strict elimination criteria. The behavioral performance data were stored in PC for off-line analysis. Multiple behavioral indices were analyzed for both passed and retired rats.

RESULTS

The ability of learning the regular the reversal Go/NoGo tasks was similar for both male and female rats, however, the female rats took longer time to master the task principles in later stages for both tasks. In the regular Go/NoGo task, the female rats spent more time on completing the trial in performance optimization phases, which implied female rats were more cautious than male rats. Along with the progression of training, both male and female rats developed Go-preference strategies to perform the regular Go/NoGo task, which induced failure to meet the setting success criteria. The retired male rats exhibited shorter RTs and MTs than the retired female rats after developing Go-preference. Moreover, the time needed to complete the Go trials was significantly prolonged for male rats in the reversal Go/NoGo task.

CONCLUSIONS

Overall, we conclude that distinctive strategies were employed in performing Go/NoGo tasks for both male and female rats. Male rats required less time to stabilize the performance in behavioral optimization phase. In addition, male rats were more accurate in estimating time elapsing. In contrast, female rats took more cautious considerations in performing the task, through which minimal influences were manifested in the reversal version of task.

Further Characterization of Multi-Organ DEARE and Protection by 16,16 Dimethyl Prostaglandin E2 in a Mouse Model of the Hematopoietic Acute Radiation Syndrome

Survivors of acute radiation exposure suffer from the delayed effects of acute radiation exposure (DEARE), a chronic condition affecting multiple organs, including lung, kidney, heart, gastrointestinal tract, eyes, and brain, and often causing cancer. While effective medical countermeasures (MCM) for the hematopoietic-acute radiation syndrome (H-ARS) have been identified and approved by the FDA, development of MCM for DEARE has not yet been successful. We previously documented residual bone marrow damage (RBMD) and progressive renal and cardiovascular DEARE in murine survivors of H-ARS, and significant survival efficacy of 16,16-dimethyl prostaglandin E2 (dmPGE2) given as a radioprotectant or radiomitigator for H-ARS. We now describe additional DEARE (physiological and neural function, progressive fur graying, ocular inflammation, and malignancy) developing after sub-threshold doses in our H-ARS model, and detailed analysis of the effects of dmPGE2 administered before (PGE-pre) or after (PGE-post) lethal total-body irradiation (TBI) on these DEARE. Administration of PGE-pre normalized the twofold reduction of white blood cells (WBC) and lymphocytes seen in vehicle-treated survivors (Veh), and increased the number of bone marrow (BM) cells, splenocytes, thymocytes, and phenotypically defined hematopoietic progenitor cells (HPC) and hematopoietic stem cells (HSC) to levels equivalent to those in non-irradiated age-matched controls. PGE-pre significantly protected HPC colony formation ex vivo by >twofold, long term-HSC in vivo engraftment potential up to ninefold, and significantly blunted TBI-induced myeloid skewing. Secondary transplantation documented continued production of LT-HSC with normal lineage differentiation. PGE-pre reduced development of DEARE cardiovascular pathologies and renal damage; prevented coronary artery rarefication, blunted progressive loss of coronary artery endothelia, reduced inflammation and coronary early senescence, and blunted radiation-induced increase in blood urea nitrogen (BUN). Ocular monocytes were significantly lower in PGE-pre mice, as was TBI-induced fur graying. Increased body weight and decreased frailty in male mice, and reduced incidence of thymic lymphoma were documented in PGE-pre mice. In assays measuring behavioral and cognitive functions, PGE-pre reduced anxiety in females, significantly blunted shock flinch response, and increased exploratory behavior in males. No effect of TBI was observed on memory in any group. PGE-post, despite significantly increasing 30-day survival in H-ARS and WBC and hematopoietic recovery, was not effective in reducing TBI-induced RBMD or any other DEARE. In summary, dmPGE2 administered as an H-ARS MCM before lethal TBI significantly increased 30-day survival and ameliorated RBMD and multi-organ and cognitive/behavioral DEARE to at least 12 months after TBI, whereas given after TBI, dmPGE2 enhances survival from H-ARS but has little impact on RBMD or other DEARE.

The effects of time horizon and guided choices on explore-exploit decisions in rodents

Humans and animals have to balance the need for exploring new options with exploiting known options that yield good outcomes. This tradeoff is known as the explore-exploit dilemma. To better understand the neural mechanisms underlying how humans and animals address the explore-exploit dilemma, a good animal behavioral model is critical. Most previous rodents explore-exploit studies used ethologically unrealistic operant boxes and reversal learning paradigms in which the decision to abandon a bad option is confounded by the need for exploring a novel option for information collection, making it difficult to separate different drives and heuristics for exploration. In this study, we investigated how rodents make explore-exploit decisions using a spatial navigation horizon task (Wilson et al., 2014) adapted to rats to address the above limitations. We compared the rats' performance to that of humans using identical measures. We showed that rats use prior information to effectively guide exploration. In addition, rats use information-driven directed exploration like humans, but the extent to which they explore has the opposite dependance on time horizon than humans. Moreover, we found that free choices and guided choices have different influences on exploration in rodents, a finding that has not yet been tested in humans. This study reveals that the explore-exploit spatial behavior of rats is more complex than previously thought. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A neural substrate of sex-dependent modulation of motivation

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Sex differences in contextual pattern separation, neurogenesis, and functional connectivity within the limbic system

Background

Females are more likely to present with anxiety disorders such as post-traumatic stress disorder (PTSD) compared to males, which are associated with disrupted hippocampal integrity. Sex differences in the structure and function of hippocampus exist. Here, we examined sex differences in contextual pattern separation, functional connectivity, and activation of new neurons during fear memory.

Methods

Two-month-old male and female Sprague-Dawley rats were injected with the DNA synthesis markers, iododeoxyuridine (IdU) and chlorodeoxyuridine (CldU) 3 weeks and 4 weeks before perfusion, respectively. One week after CldU injection, the rats underwent a context discrimination task in which rats were placed in context A (shock) and context A’ (no shock) every day for 12 days. On the test day, rats were placed in the shock context (context A) to measure fear memory and expression of zif268, an immediate early gene across 16 different limbic and reward regions. Repeated-measures or factorial analysis of variance was conducted on our variables of interest. Pearson product-moment calculations and principal component analyses on zif268 expression across regions were also performed.

Results

We found that females, but not males, showed contextual discrimination during the last days of training. On the test day, both sexes displayed similar levels of freezing, indicating equivalent fear memory for context A. Despite similar fear memory, males showed more positive correlations of zif268 activation between the limbic regions and the striatum, whereas females showed more negative correlations among these regions. Females showed greater activation of the frontal cortex, dorsal CA1, and 3-week-old adult-born dentate granular cells compared to males.

Conclusions

These results highlight the importance of studying sex differences in fear memory and the contribution of adult neurogenesis to the neuronal network and may contribute to differences in susceptibility to fear-related disorders such as post-traumatic stress disorder.

Highlights

Female rats, but not male rats, show faster discrimination during a contextual pattern separation task.

Three-week-old adult-born neurons are more active in response to fear memory in females compared to males.

Females had greater neural activation compared to males in the frontal cortex and dorsal CA1 region of the hippocampus in response to fear memory.

Males and females show distinct patterns in functional connectivity for fear memory across limbic regions.

Males have many positive correlations between activated new neurons of different ages between the dorsal and ventral hippocampus, while females show more correlations between activated new neurons and other limbic regions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13293-022-00450-2.

A Signal Detection Analysis of Olfactory Learning in 12-Month-Old 5xFAD Mice

Although Alzheimer’s disease is most often studied in terms of memory impairments, olfactory dysfunction begins in the early stages. We tested olfactory learning, sensitivity, and response bias using signal detection methods in 12-month-old male and female 5xFAD mice and their wildtype controls in the operant olfactometer. Odor detection was not reduced in the 5xFAD mice, but learning was, which was worse in female 5xFAD mice than in males. Female mice were more conservative in their response strategy. Signal detection analysis allows us to discriminate between cognitive and sensory deficits of male and female mouse models of AD.