Divergent Strategies for Learning in Males and Females

Nucleus Accumbens Dopamine Encodes the Trace Period during Appetitive Pavlovian Conditioning

Pavlovian conditioning tasks have been used to identify the neural systems involved with learning cue-outcome relationships. In delay conditioning, the conditioned stimulus (CS) overlaps or co-terminates with the unconditioned stimulus (US). Prior studies demonstrate that dopamine in the nucleus accumbens (NAc) regulates behavioral responding during delay conditioning. Furthermore, the dopamine response to the CS reflects the relative value of the upcoming reward in these tasks. In contrast to delay conditioning, trace conditioning involves a "trace" period separating the end of the CS and the US delivery. While dopamine has been implicated in trace conditioning, no studies have examined how NAc dopamine responds to reward-related stimuli in these tasks. Here, we developed a within-subject trace conditioning task where distinct CSs signaled either a short trace period (5 s) or a long trace period (55 s) prior to food reward delivery. Male rats exhibited greater conditioned responding and a faster response latency to the Short Trace CS relative to the Long Trace CS. Voltammetry recordings in the NAc found that the CS-evoked dopamine response increased on Short Trace trials but decreased on Long Trace trials. Conversely, US-evoked dopamine responses were greater on Long Trace trials relative to Short Trace trials. The CS dopamine response correlated with the response latency and not with conditioned responding. Furthermore, the relationship between CS dopamine and latency was best explained by an exponential function. Our results collectively illustrate that the trace period is encoded by the bidirectional NAc dopamine response to the CS during pavlovian conditioning.

Prediction of Clinical Response of Transcranial Magnetic Stimulation Treatment for Major Depressive Disorder Using Hyperdimensional Computing

Cognitive control dysregulation is nearly universal across disorders, including major depressive disorder (MDD). Achieving comparable response rates to medication, the transcranial magnetic stimulation (TMS) mechanism and its effect on cognitive control have not been well understood yet. This paper investigates the predictive capability of the clinical response to TMS treatment using 34 cognitive variables measured from TMS treatment of 22 MDD subjects over an eight-week period. We employ a novel brain-inspired computing paradigm, hyperdimensional computing (HDC), to classify the effectiveness of TMS using leave-one-subject-out cross-validation (LOSOCV). Four performance metrics-accuracy, sensitivity, specificity and AUC-are used, with AUC being the primary metric. Experimental results reveal that: i). Although SVM outperforms HDC in terms of accuracy, HDC achieves an AUC of 0.82, surpassing SVM by 0.07. ii). The optimal performance for both classifiers is obtained with feature selection using SelectKBest. iii) Among the top features selected by SelectKBest for the two classifiers, ws_MedRT (median rate for the Websurf task) shows a more distinguishable distribution between clinical responses ("1") and no clinical responses ("0"). In conclusion, these results highlight the potential of HDC for predicting clinical responses to TMS and underscore the importance of feature selection in improving classification performance.

In vivo calcium extrusion from accumbal astrocytes reduces anxiety-like behaviors but increases compulsive-like responses and compulsive ethanol drinking in mice

The ventral striatum is crucially involved in reward processing. The present study investigates the behavioral effects of astrocyte-specific calcium extrusion virus "CalEx" on perseverative responses in the operant five-choice serial reaction time task and ethanol-conditioned place preference. Mice were injected with CalEx via the GfaABC1D promoter to extrude cytosolic calcium from astrocytes within the ventral striatum. We found that CalEx transfection in the ventral striatum reduced evoked response duration, the maximum amplitude, and the response frequency to 500 μM ATP as measured by ΔF/F fluorescence intensity of the genetically encoded calcium indicator targeting astrocytes GCaMP6f. During the five-choice serial reaction time task, CalEx mice persisted in perseverative responses compared to their counterparts. Additionally, during ethanol-conditioned place preference, CalEx mice showed increased place preference for a low ethanol concentration compared to control group. Furthermore, we found that accumbal astrocytic calcium extrusion increased quinine adulterated ethanol drinking. Our findings suggest that diminishing ventral striatum astrocyte calcium activity contributes to compulsive behaviors, ethanol drinking, and enhanced ethanol drug reward.

Touchscreen Response Precision Is Sensitive to the Explore/Exploit Trade-off

The explore/exploit trade-off is a fundamental property of choice selection during reward-guided decision making, where the "same" choice can reflect either of these internal cognitive states. An unanswered question is whether the execution of a decision provides an underexplored measure of internal cognitive states. Touchscreens are increasingly used across species for cognitive testing and afford the ability to measure the precise location of choice touch responses. We examined how male and female mice in a restless bandit decision making task interacted with a touchscreen to determine if the explore/exploit trade-off, prior reward, and/or sex differences change the variability in the kinetics of touchscreen choices. During exploit states, successive touch responses are closer together than those made in an explore state, suggesting exploit states reflect periods of increased motor stereotypy. Although exploit decisions might be expected to be rewarded more frequently than explore decisions, we find that immediate past reward reduces choice variability independently of explore/exploit state. Male mice are more variable in their interactions with the touchscreen than females, even in low-variability trials such as exploit or following reward. These results suggest that as exploit behavior emerges in reward-guided decision making, all mice become less variable and more automated in both their choice and the actions taken to make that choice, but this occurs on a background of increased male variability. These data uncover the hidden potential for touchscreen decision making tasks to uncover the latent neural states that unite cognition and movement.

Individual differences in sequential decision-making

1People differ widely in how they make decisions in uncertain environments. While many studies leverage this variability to measure differences in specific cognitive processes and parameters, the key dimension(s) of individual variability in uncertain decision-making tasks has not been identified. Here, we analyzed behavioral data from 1001 participants performing a restless three-armed bandit task, where reward probabilities fluctuated unpredictably over time. Using a novel analytical approach that controlled for the stochasticity in this tasks, we identified a dominant nonlinear axis of individual variability. We found that this primary axis of variability was strongly and selectively correlated with the probability of exploration, as inferred by latent state modeling. This suggests that the major factor shaping individual differences in bandit task performance is the tendency to explore (versus exploit), rather than personality characteristics, reinforcement learning model parameters, or low-level strategies. Certain demographic characteristics also predicted variance along this principle axis: participants at the exploratory end tended to be younger than participants at the exploitative end, and self-identified men were overrepresented at both extremes. Together, these findings offer a principled framework for understanding individual differences in task behavior while highlighting the cognitive and demographic factors that shape individual differences in decision-making under uncertainty.

Sex-biased computations underlying differential set shift performance in mice

Cognitive flexibility can be defined as the ability to adaptively shift between choices or strategies based on environmental feedback and it is disrupted in numerous neuropsychiatric conditions. Individual differences in the computations supporting cognitive flexibility are poised to reveal mechanisms of neuropsychiatric risk and resilience. One critical variable well known to influence individual differences in neuropsychiatric risk is sex. While previous research has identified sex differences in value based decision making in mice, whether sex reflects a major source of variation in cognitive flexibility remains unknown. To directly assess sex-biased individual differences in cognitive flexibility, we developed a novel touchscreen Set Shift task that permits robust and continuous testing in mice. Using this task, we discovered that female mice completed significantly more rule shifts with fewer errors than males. We next employed a suite of computational models that revealed sex-biased individual differences in the computations underlying cognitive flexibility. Overall, our results suggest that following rule shifts, female mice learn the new rule faster and commit to exploiting rule choices sooner compared to males - sometimes because they commit to multiple rules simultaneously. This suggests that increased choice stability in female rodents enhances commitment to a strategy during periods of uncertainty and directly contributes to increased rule shifting. This supports the counterintuitive conclusion that a high degree of stable choice is a strong requirement for enhanced cognitive flexibility in the Set Shift task, one of the gold standard cognitive flexibility tasks.

Nucleus accumbens dopamine encodes the trace period during appetitive Pavlovian conditioning

Pavlovian conditioning tasks have been used to identify the neural systems involved with learning cue-outcome relationships. In delay conditioning, the conditioned stimulus (CS) overlaps or co-terminates with the delivery of the unconditioned stimulus (US). Prior studies demonstrate that dopamine in the nucleus accumbens (NAc) regulates behavioral responding during delay conditioning. Furthermore, the dopamine response to the CS reflects the relative value of the upcoming reward in these tasks. In contrast to delay conditioning, trace conditioning involves a 'trace' period separating the end of the CS and the US delivery. While dopamine has been implicated in trace conditioning, no studies have examined how NAc dopamine responds to reward-related stimuli in these tasks. Here, we developed a within-subject trace conditioning task where distinct CSs signaled either a short trace period (5s) or a long trace period (55s) prior to food reward delivery. Male rats exhibited greater conditioned responding and a faster response latency to the Short Trace CS relative to the Long Trace CS. Voltammetry recordings in the NAc found that the CS-evoked dopamine response increased on Short Trace trials but decreased on Long Trace trials. Conversely, US-evoked dopamine responses were greater on Long Trace trials relative to Short Trace trials. The CS dopamine response correlated with the response latency and not with conditioned responding. Furthermore, the relationship between CS dopamine and latency was best explained by an exponential function. Our results collectively illustrate that the trace period is encoded by the bidirectional NAc dopamine response to the CS during Pavlovian conditioning.

Significance statement

Learning to associate a cue with given outcome is a fundamental process underlying reward seeking behavior. Striatal dopamine is important for associating cues with rewards during Pavlovian conditioning. However, it is unclear how the dopamine system responds to cues during trace conditioning when there is temporal gap between the cue and reward. Here, we performed voltammetry recordings of striatal dopamine levels in male rats during trace conditioning. We find that cue-evoked dopamine signals encode the trace period and is related to the response latency. While prior reports find dopamine neurons signal the relative reward value by increases in dopamine levels, we demonstrate that the dopamine response to reward-predictive cues can signal the reward value through bidirectional changes in dopamine transmission.

Effects of sex and estrous cycle on action-outcome contingencies

Goal-directed and habitual-like behaviors are both necessary to efficiently and effectively navigate the environment. A dysregulation between these behaviors can lead to an overreliance on habitual-like behaviors and may contribute to symptoms experienced in some neuropsychiatric disorders such as substance use disorder. One behavioral task used to evaluate goal-directed and habitual-like behavior is an action-outcome task, contingency degradation, where an action (i.e., lever press) is degraded by decoupling the receipt of a reward from the action. However, little is known about how male and female rats and females across the estrous cycle respond during contingency degradation training and extinction testing. Here, we investigated how the variable of sex and estrous cycle influences contingency degradation training and extinction testing and the correlation between baseline anxiety-like behaviors and performance on contingency degradation extinction testing in adult male and female Long-Evans rats. We found that both males and females learned the contingency degradation task. However, during extinction testing, males respond more to the contingent lever than the non-contingent lever while females do not differ in their responses on the non-contingent and contingent levers. Lower baseline anxiety-like behavior predicted better performance on the contingency degradation test in males, but not females. Next, when we examined performance during extinction testing in females based on their estrous cycle stage on test day, we found that females in the proestrus and estrus stages of the estrous cycle do not differ in their responses on the non-contingent and contingent levers, while females in the metestrus and diestrus stages of the estrous cycle respond more on the contingent lever than the non-contingent lever on the extinction test day, similar to male rats. Our findings indicate that the estrous cycle influences how female rats respond during contingency degradation extinction testing that is dependent on their estrous cycle stage.

Interactions of sex and stress in modulation of ventral tegmental area dopaminergic activity

Dopaminergic (DA) neurons of the ventral tegmental area (VTA) have long been studied for their role in reward prediction and goal-directed behaviors. However, appreciation is growing for a complementary role of VTA DA neurons in responding to aversive stimuli and as critical substrates for behavioral sequelae of stressful experiences. As is the case across neuroscience, the majority of our knowledge about VTA DA neurons comes from studies in male subjects. Recent years have seen an increase in inclusion of female subjects and exploration of sex differences. There is now an emerging body of literature showing that although there are minimal basal structural and functional sex differences in VTA DA neurons, experience-dependent changes in these neurons can differ significantly between males and females. Here, we discuss potential implications of sex differences in VTA function and review recent data on sex differences and similarities of DA neurons at baseline and following stress.

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

RATIONALE

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

OBJECTIVE

We determined how 14 daily THC injections (5 mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

METHODS

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impact probabilistic discounting.

RESULTS

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

CONCLUSIONS

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine neurons or their projections to mPFC.

Data-Driven Analysis Identifies Novel Modulation of Social Behavior in Female Mice Witnessing Chronic Social Defeat Stress

BACKGROUND

Chronic social defeat stress is a widely used depression model in male mice. Several proposed adaptations extend this model to females with variable, often marginal effects. We examined if the widely used male-defined metrics of stress are suboptimal in females witnessing defeat.

METHODS

Using a data-driven method, we comprehensively classified social interaction behavior in 761 male and female mice after chronic social witness/defeat stress, examining social modulation of behavior and associations with conventional metrics (i.e., social interaction ratio).

RESULTS

Social stress induced distinct behavioral adaptation patterns in defeated males and witness females. Social interaction ratio led to underpowered analyses in witness females with limited utility to differentiate susceptibility/resilience. Data-driven analyses revealed changes in social adaptation in witness females that were captured in attenuated velocity change from no target to target trials. We explored the utility of this metric in 4 female social stress models and in male witnesses. Combining social interaction ratio and velocity change optimally differentiated susceptibility/resilience in witness females and revealed resilient-specific adaptation in a resilience-associated neural circuit in female mice.

CONCLUSIONS

Chronic witness stress induced behavioral changes in females that were qualitatively distinct from those observed in defeated males and not adequately sampled by standard male-defined metrics. Modulation of locomotion is a robust and easily implementable metric for rigorous research in witness female mice. Overall, our findings highlight the need to critically evaluate sex differences in behavior and implement sex-based considerations in preclinical model design.

Optogenetic activation of mesencephalic projections to the nucleus accumbens shell impairs probabilistic reversal learning by disrupting learning from negative reinforcement

Cognitive flexibility, the capacity to adapt behaviour to changes in the environment, is impaired in a range of brain disorders, including schizophrenia and Parkinson's disease. Putative neural substrates of cognitive flexibility include mesencephalic pathways to the ventral striatum (VS) and dorsomedial striatum (DMS), hypothesized to encode learning signals needed to maximize rewarded outcomes during decision-making. However, it is unclear whether mesencephalic projections to the ventral and dorsal striatum are distinct in their contribution to flexible reward-related learning. Here, rats acquired a two-choice spatial probabilistic reversal learning (PRL) task, reinforced on an 80%|20% basis (correct|incorrect responses) that assessed the flexibility of behaviour to repeated reversals of response-outcome contingencies. We report that optogenetic stimulation of projections from the ventral tegmental area (VTA) to the nucleus accumbens shell (NAcS) in the VS significantly impaired reversal learning when optical stimulation was temporally aligned with negative feedback (i.e., reward omission). VTA → NAcS stimulation during other phases of the behavioural task was without significant effect. Optogenetic stimulation of projection neurons from the substantia nigra (SN) to the DMS, aligned either with reward receipt or omission or prior to making a choice, had no significant effect on reversal learning. These findings are consistent with the notion that increased activity in the VTA → NAcS pathway disrupts behavioural flexibility by impairing learning from negative reinforcement.

Sex differences in sensitivity to dopamine receptor manipulations of risk-based decision making in rats

Risky decision making involves the ability to weigh risks and rewards associated with different options to make adaptive choices. Previous work has established a necessary role for the basolateral amygdala (BLA) in mediating effective decision making under risk of punishment, but the mechanisms by which the BLA mediates this process are less clear. Because this form of decision making is profoundly sensitive to dopaminergic (DA) manipulations, we hypothesized that DA receptors in the BLA may be involved in risk-taking behavior. To test this hypothesis, male and female Long-Evans rats were trained in a decision-making task in which rats chose between a small, safe food reward and a larger food reward that was associated with a variable risk of footshock punishment. Once behavioral stability emerged, rats received intra-BLA infusions of ligands targeting distinct dopamine receptor subtypes prior to behavioral testing. Intra-BLA infusions of the dopamine D2 receptor (D2R) agonist quinpirole decreased risk taking in females at all doses, and this reduction in risk taking was accompanied by an increase in sensitivity to punishment. In males, decreased risk taking was only observed at the highest dose of quinpirole. In contrast, intra-BLA manipulations of dopamine D1 or D3 receptors (D1R and D3R, respectively) had no effect on risk taking. Considered together, these data suggest that differential D2R sensitivity in the BLA may contribute to the well-established sex differences in risk taking. Neither D1Rs nor D3Rs, however, appear to contribute to risky decision making in either sex.

Dopamine and Norepinephrine Differentially Mediate the Exploration-Exploitation Tradeoff

Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision-making processes. Although two neuromodulators share a synthesis pathway and are coactivated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision-making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision-making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol) and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Touchscreen response precision is sensitive to the explore/exploit tradeoff

The explore/exploit tradeoff is a fundamental property of choice selection during reward-guided decision making. In perceptual decision making, higher certainty decisions are more motorically precise, even when the decision does not require motor accuracy. However, while we can parametrically control uncertainty in perceptual tasks, we do not know what variables - if any - shape motor precision and reflect subjective certainty during reward-guided decision making. Touchscreens are increasingly used across species to measure choice, but provide no tactile feedback on whether an action is precise or not, and therefore provide a valuable opportunity to determine whether actions differ in precision due to explore/exploit state, reward, or individual variables. We find all three of these factors exert independent drives towards increased precision. During exploit states, successive touches to the same choice are closer together than those made in an explore state, consistent with exploit states reflecting higher certainty and/or motor stereotypy in responding. However, exploit decisions might be expected to be rewarded more frequently than explore decisions. We find that exploit choice precision is increased independently of a separate increase in precision due to immediate past reward, suggesting multiple mechanisms regulating choice precision. Finally, we see evidence that male mice in general are less precise in their interactions with the touchscreen than females, even when exploiting a choice. These results suggest that as exploit behavior emerges in reward-guided decision making, individuals become more motorically precise reflecting increased certainty, even when decision choice does not require additional motor accuracy, but this is influenced by individual differences and prior reward. These data uncover the hidden potential for touchscreen tasks in any species to uncover the latent neural states that unite cognition and movement.

Sex differences in motivational biases over instrumental actions

Motivational (i.e., appetitive or aversive) cues can bias value-based decisions by affecting either direction and intensity of instrumental actions. Despite several findings describing important interindividual differences in these biases, whether biological sex can also play a role is still up to debate. By comparing females and males in both appetitive and aversive Pavlovian-to-Instrumental Transfer paradigms we found that, while motivational cues similarly bias the direction of instrumental actions in both sexes, the intensity of such actions is increased by the cue in male participants only. The present results constitute compelling evidence that a crucial motivational bias of daily actions directed to obtaining rewards or avoiding punishments is modulated by biological sex. This evidence sheds new light on the role of sex in motivational processes that underlie decision-making, highlighting the importance of considering sex as a crucial factor in future research on this topic.

Female rodents in behavioral neuroscience: Narrative review on the methodological pitfalls

Since the NIH 'Sex as biological variable' policy, the percentage of studies including female subjects have increased largely. Nonetheless, many researchers fail to adequate their protocols to include females. In this narrative review, we aim to discuss the methodological pitfalls of the inclusion of female rodents in behavioral neuroscience. We address three points to consider in studies: the manipulations conducted only in female animals (such as estrous cycle monitoring, ovariectomy, and hormone replacement), the consideration of males as the standard, and biases related to interpretation and publication of the results. In addition, we suggest guidelines and perspectives for the inclusion of females in preclinical research.

Sex differences in neural representations of social and nonsocial reward in the medial prefrontal cortex

The reinforcing nature of social interactions is necessary for the maintenance of appropriate social behavior. However, the neural substrates underlying social reward processing and how they might differ based on the sex and internal state of the animal remains unknown. It is also unclear whether these neural substrates are shared with those involved in nonsocial rewarding processing. We developed a fully automated, two choice (social-sucrose) operant assay in which mice choose between social and nonsocial rewards to directly compare the reward-related behaviors associated with two competing stimuli. We performed cellular resolution calcium imaging of medial prefrontal cortex (mPFC) neurons in male and female mice across varying states of water restriction and social isolation. We found that mPFC neurons maintain largely non-overlapping, flexible representations of social and nonsocial reward that vary with internal state in a sex-dependent manner. Additionally, optogenetic manipulation of mPFC activity during the reward period of the assay disrupted reward-seeking behavior across male and female mice. Thus, using a two choice operant assay, we have identified sex-dependent, non-overlapping neural representations of social and nonsocial reward in the mPFC that vary with internal state and that are essential for appropriate reward-seeking behavior.

Sex differences in spontaneous behavior and cognition in mice using an automated behavior monitoring system

Isolation of sex differences as a key characteristic underlying neurobehavioral differentiation is an essential component of studies in neuroscience. The current study sought to address this concern by observing behavioral differences using an automated home cage system for neurobehavioral assessment, a method rapidly increasing in use due to advances in technology and advantages such as reduced handling stress and cross-lab variability. Sex differences in C57BL/6 mice arose for motor activity and circadian-linked behavior, with females being more active compared to males, and males having a stronger anticipatory increase in activity leading up to the onset of the light phase compared to females. These activity differences were observed not only across the lifespan, but also in different genetic background mouse strains across different testing sites showing the generalizability and robustness of these observed effects. Activity differences were also observed in performance on a spatial learning and reversal task with females making more responses and receiving a corresponding elevation in reward pellets. Notably, there were no sex differences in learning nor achieved accuracy, suggesting these observed effects were predominantly in activity. The outcomes of this study align with previous reports showcasing differences in activity between males and females. The comparison across strains and testing sites showed robust and reproducible differences in behavior between female and male mice that are relevant to consider when designing behavioral studies. Furthermore, the observed sex differences in performance on the learning and reversal procedure raise concern for interpretation of behavior differences between sexes due to the attribution of these differences to motor activity rather than cognition.

Behavioral neuroscience's inevitable SABV growing pains

The field of rodent behavioral neuroscience is undergoing two major sea changes: an ever-growing technological revolution, and worldwide calls to consider sex as a biological variable (SABV) in experimental design. Both have enormous potential to improve the precision and rigor with which the brain can be studied, but the convergence of these shifts in scientific practice has exposed critical limitations in classic and widely used behavioral paradigms. While our tools have advanced, our behavioral metrics - mostly developed in males and often allowing for only binary outcomes - have not. This opinion article explores how this disconnect has presented challenges for the accurate depiction and interpretation of sex differences in brain function, arguing for the expansion of current behavioral constructs to better account for behavioral diversity.

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

Rationale

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

Objective

We determined how 14 daily THC injections (5mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

Methods

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impacts probabilistic discounting.

Results

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

Conclusions

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine neurons or their projections to mPFC.

Sex mechanisms as nonbinary influences on cognitive diversity

Essentially all neuropsychiatric diagnoses show some degree of sex and/or gender differences in their etiology, diagnosis, or prognosis. As a result, the roles of sex-related variables in behavior and cognition are of strong interest to many, with several lines of research showing effects on executive functions and value-based decision making in particular. These findings are often framed within a sex binary, with behavior of females described as less optimal than male "defaults"-- a framing that pits males and females against each other and deemphasizes the enormous overlap in fundamental neural mechanisms across sexes. Here, we propose an alternative framework in which sex-related factors encompass just one subset of many sources of valuable diversity in cognition. First, we review literature establishing multidimensional, nonbinary impacts of factors related to sex chromosomes and endocrine mechanisms on cognition, focusing on value- based decision-making tasks. Next, we present two suggestions for nonbinary interpretations and analyses of sex-related data that can be implemented by behavioral neuroscientists without devoting laboratory resources to delving into mechanisms underlying sex differences. We recommend (1) shifting interpretations of behavior away from performance metrics and towards strategy assessments to avoid the fallacy that the performance of one sex is worse than another; and (2) asking how much variance sex explains in measures and whether any differences are mosaic rather than binary, to avoid assuming that sex differences in separate measures are inextricably correlated. Nonbinary frameworks in research on cognition will allow neuroscience to represent the full spectrum of brains and behaviors.

Impact of the day/night cycle on functional connectome in ageing male and female mice

To elucidate how time of day, sex, and age affect functional connectivity (FC) in mice, we aimed to examine whether the mouse functional connectome varied with the day/night cycle and whether it depended on sex and age. We explored C57Bl6/J mice (6♀ and 6♂) at mature age (5 ± 1 months) and middle-age (14 ± 1 months). Each mouse underwent Blood Oxygen-Level-Dependent (BOLD) resting-state functional MRI (rs-fMRI) on a 7T scanner at four different times of the day, two under the light condition and two under the dark condition. Data processing consisted of group independent component analysis (ICA) and region-level analysis using resting-state networks (RSNs) derived from literature. Linear mixed-effect models (LMEM) were used to assess the effects of sex, lighting condition and their interactions for each RSN obtained with group-ICA (RSNs-GICA) and six bilateral RSNs adapted from literature (RSNs-LIT). Our study highlighted new RSNs in mice related to day/night alternation in addition to other networks already reported in the literature. In mature mice, we found sex-related differences in brain activation only in one RSNs-GICA comprising the cortical, hippocampal, midbrain and cerebellar regions of the right hemisphere. In males, brain activity was significantly higher in the left hippocampus, the retrosplenial cortex, the superior colliculus, and the cerebellum regardless of lighting condition; consistent with the role of these structures in memory formation and integration, sleep, and sex-differences in memory processing. Experimental constraints limited the analysis to the impact of light/dark cycle on the RSNs for middle-aged females. We detected significant activation in the pineal gland during the dark condition, a finding in line with the nocturnal activity of this gland. For the analysis of RSNs-LIT, new variables "sexage" (sex and age combined) and "edges" (pairs of RSNs) were introduced. FC was calculated as the Pearson correlation between two RSNs. LMEM revealed no effect of sexage or lighting condition. The FC depended on the edges, but there were no interaction effects between sexage, lighting condition and edges. Interaction effects were detected between i) sex and lighting condition, with higher FC in males under the dark condition, ii) sexage and edges with higher FC in male brain regions related to vision, memory, and motor action. We conclude that time of day and sex should be taken into account when designing, analyzing, and interpreting functional imaging studies in rodents.

Perineuronal nets are associated with decision making under conditions of uncertainty in female but not male mice

Biological sex influences decision-making processes in significant ways, differentiating the responses animals choose when faced with a range of stimuli. The neurobiological underpinnings that dictate sex differences in decision-making tasks remains an important open question, yet single-sex studies of males form most studies in behavioural neuroscience. Here we used female and male BALB/c mice on two spatial learning and memory tasks and examined the expression of perineuronal nets (PNNs) and parvalbumin interneurons (PV) in regions correlated with spatial memory. Mice underwent the aversive active place avoidance (APA) task or the appetitive trial-unique nonmatching-to-location (TUNL) touchscreen task. Mice in the APA cohort learnt to avoid the foot-shock and no differences were observed on key measures of the task nor in the number and intensity of PNNs and PV. On the delay but not separation manipulation in the TUNL task, females received more incorrect trials and less correct trials compared to males. Furthermore, females in this cohort exhibited higher intensity PNNs and PV cells in the agranular and granular retrosplenial cortex, compared to males. These data show that female and male mice perform similarly on spatial learning tasks. However, sex differences in neural circuitry may underly differences in making decisions under conditions of uncertainty on an appetitive task. These data emphasise the importance of using mice of both sexes in studies of decision-making neuroscience.

Sex and individual differences in the effect of chronic low-dose ethanol on behavioral strategy selection

BACKGROUND

The development of an alcohol use disorder (AUD) involves impaired behavioral control and flexibility. Behavioral inflexibility includes an inability to shift behavior in response to changes in behavioral outcomes. Low levels of ethanol drinking may promote the formation of inflexible, habitual reward seeking, but this may depend on the timing of ethanol exposure in relation to learning. The goal of this study was to determine whether a history of low-dose ethanol exposure promoted contingency-insensitive sucrose seeking and altered behavioral strategy selection.

METHODS

Male and female C57BL/6J mice were trained to perform a response (lever press) for sucrose on two different reinforcement schedules: one that is thought to promote inflexible responding (random interval) and one that maintains flexible responding (variable ratio [VR]). Following instrumental training each day, mice were exposed to saline or low-dose ethanol (0.5 g/kg; i.p.) either proximal (1 h after) or distal (4 h after) to learning. Mice were then tested for sensitivity to changes in contingency in a contingency degradation test.

RESULTS

A history of low-dose ethanol exposure shifted behavioral strategy selection, as measured by reward tracking behavior, but this depended on sex and reinforcement schedule history. Both male and female mice used different strategies depending on the reinforcement schedule, but only males exhibited ethanol-induced shifts in strategy selection. A history of low-dose ethanol exposure did not impact contingency sensitivity in males but promoted insensitivity in females specifically on the VR lever.

CONCLUSIONS

Female mice show distinct behavioral effects of repeated, low-dose ethanol exposure as compared to males, with sex differences in the use of reward tracking strategies to guide behavior. Future studies should investigate sex differences in the neural consequences of chronic low-dose ethanol exposure that may underlie behavioral changes.

Positive correlation between measures of habitual responding and motivated responding in mice

Habit and motivation are thought to be separate processes, with motivated behavior often considered to be goal directed, whereas habits are defined by the absence of goal-directed control over behavior. However, there has been increasing interrogation of the binary nature of habitual versus goal-directed behavior. Furthermore, although drug and alcohol exposure can promote the formation of habits, drug seeking itself can also be highly flexible, pointing toward the need for complex consideration of the parallel processes that drive behavior. The goal of the current study was to determine whether there was a relation between motivation-as measured by progressive ratio-and habit-as measured by contingency degradation-and whether this relation was affected by ethanol exposure history and sex. The results showed that these measures were positively correlated such that greater contingency insensitivity was associated with achieving higher break points on the progressive-ratio task. However, this relation depended on reinforcement schedule history, ethanol exposure history, and sex. These point to potential relations between measures of habit and motivation and stress the importance of carefully parsing behavioral findings and assays. These findings are also expected to inform future substance use research, as drug history may affect these relations.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Stress, associative learning, and decision-making

Exposure to acute and chronic stress has significant effects on the basic mechanisms of associative learning and memory. Stress can both impair and enhance associative learning depending on type, intensity, and persistence of the stressor, the subject's sex, the context that the stress and behavior is experienced in, and the type of associative learning taking place. In some cases, stress can cause or exacerbate the maladaptive behavior that underlies numerous psychiatric conditions including anxiety disorders, obsessive-compulsive disorder, post-traumatic stress disorder, substance use disorder, and others. Therefore, it is critical to understand how the varied effects of stress, which may normally facilitate adaptive behavior, can also become maladaptive and even harmful. In this review, we highlight several findings of associative learning and decision-making processes that are affected by stress in both human and non-human subjects and how they are related to one another. An emerging theme from this work is that stress biases behavior towards less flexible strategies that may reflect a cautious insensitivity to changing contingencies. We consider how this inflexibility has been observed in different associative learning procedures and suggest that a goal for the field should be to clarify how factors such as sex and previous experience influence this inflexibility.

Genetic deletion of α7 nAChRs reduces hippocampal granule and pyramidal cell number in both sexes but impairs pattern separation in males only

Introduction

Neurogenesis within the dentate gyrus is thought to play an important role in cognitive processes such as reversal learning and pattern separation. The α7 nicotinic acetylcholine receptor (α7 nAChR) is expressed early in newly formed granule cells of the dentate gyrus, though its role in neurogenesis and related cognitive function is not fully understood.

Methods

To better characterize relevant function of α7 nAChRs, we performed unbiased stereology to quantify hippocampal granule cells, pyramidal cells, and total volume and used a touchscreen operant spatial discrimination/reversal task to test pattern separation in a global α7 nAChR knockout mouse line.

Results

The knockout resulted in an ≈22% reduction in granule cells and a ≈ 20% reduction in pyramidal cells in both sexes, with no change in total hippocampal volume. However, the knockout impaired performance in the touchscreen task for males only. The sex-dependent difference in behavioral, but not stereological, results suggest a divergence in the structure-function relationship in males versus females. Detailed analyses revealed males were more biased by the initial reversal contingency relative to females indicating a potential source of the sex-specific interaction with the loss of α7 nAChRs.

Discussion

These findings argue that the α7 nAChR plays a critical role in hippocampal development, not just granule cell neurogenesis, and plays a sex-dependent role in cognitive function.

Mixtures of strategies underlie rodent behavior during reversal learning

In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.

Sex-dependent noradrenergic modulation of premotor cortex during decision-making

Rodent premotor cortex (M2) integrates information from sensory and cognitive networks for action planning during goal-directed decision-making. M2 function is regulated by cortical inputs and ascending neuromodulators, including norepinephrine (NE) released from the locus coeruleus (LC). LC-NE has been shown to modulate the signal-to-noise ratio of neural representations in target cortical regions, increasing the salience of relevant stimuli. Using rats performing a two-alternative forced choice task after administration of a β-noradrenergic antagonist (propranolol), we show that β-noradrenergic signaling is necessary for effective action plan signals in anterior M2. Loss of β-noradrenergic signaling results in failure to suppress irrelevant action plans in anterior M2 disrupting decoding of cue-related information, delaying decision times, and increasing trial omissions, particularly in females. Furthermore, we identify a potential mechanism for the sex bias in behavioral and neural changes after propranolol administration via differential expression of β2 noradrenergic receptor RNA across sexes in anterior M2, particularly on local inhibitory neurons. Overall, we show a critical role for β-noradrenergic signaling in anterior M2 during decision-making by suppressing irrelevant information to enable efficient action planning and decision-making.

Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff

The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Sounding the Alarm: Sex Differences in Rat Ultrasonic Vocalizations during Pavlovian Fear Conditioning and Extinction

Pavlovian fear conditioning is a prevalent tool in the study of aversive learning, which is a key component of stress-related psychiatric disorders. Adult rats can exhibit various threat-related behaviors, including freezing, motor responses, and ultrasonic vocalizations (USVs). While these responses can all signal aversion, we know little about how they relate to one another. Here we characterize USVs emitted by male and female rats during cued fear acquisition and extinction, and assess the relationship between different threat-related behaviors. We found that males consistently emitted >22 kHz calls (referred to here as "alarm calls") than females, and that alarm call frequency in males, but not females, related to the intensity of the shock stimulus. Interestingly, 25% of males and 45% of females did not emit any alarm calls at all. Males that did make alarm calls had significantly higher levels of freezing than males who did not, while no differences in freezing were observed between female Alarm callers and Non-alarm callers. Alarm call emission was also affected by the predictability of the shock; when unpaired from a tone cue, both males and females started emitting alarm calls significantly later. During extinction learning and retrieval sessions, males were again more likely than females to emit alarm calls, which followed an extinction-like reduction in frequency. Collectively these data suggest sex dependence in how behavioral readouts relate to innate and conditioned threat responses. Importantly, we suggest that the same behaviors can signal sex-dependent features of aversion.

Behavioral impulsivity is associated with pupillary alterations and hyperactivity in CDKL5 mutant mice

Cyclin-dependent kinase-like 5 (Cdkl5) deficiency disorder (CDD) is a severe neurodevelopmental condition caused by mutations in the X-linked Cdkl5 gene. CDD is characterized by early-onset seizures in the first month of life, intellectual disability, motor and social impairment. No effective treatment is currently available and medical management is only symptomatic and supportive. Recently, mouse models of Cdkl5 disorder have demonstrated that mice lacking Cdkl5 exhibit autism-like phenotypes, hyperactivity and dysregulations of the arousal system, suggesting the possibility to use these features as translational biomarkers. In this study, we tested Cdkl5 male and female mutant mice in an appetitive operant conditioning chamber to assess cognitive and motor abilities, and performed pupillometry to assess the integrity of the arousal system. Then, we evaluated the performance of artificial intelligence models to classify the genotype of the animals from the behavioral and physiological phenotype. The behavioral results show that CDD mice display impulsivity, together with low levels of cognitive flexibility and perseverative behaviors. We assessed arousal levels by simultaneously recording pupil size and locomotor activity. Pupillometry reveals in CDD mice a smaller pupil size and an impaired response to unexpected stimuli associated with hyperlocomotion, demonstrating a global defect in arousal modulation. Finally, machine learning reveals that both behavioral and pupillometry parameters can be considered good predictors of CDD. Since early diagnosis is essential to evaluate treatment outcomes and pupillary measures can be performed easily, we proposed the monitoring of pupil size as a promising biomarker for CDD.

Medial septum activation improves strategy switching once strategies are well-learned via bidirectional regulation of dopamine neuron population activity

Strategy switching is a form of cognitive flexibility that requires inhibiting a previously successful strategy and switching to a new strategy of a different categorical modality. It is dependent on dopamine (DA) receptor activation and release in ventral striatum and prefrontal cortex, two primary targets of ventral tegmental area (VTA) DA projections. Although the circuitry that underlies strategy switching early in learning has been studied, few studies have examined it after extended discrimination training. This may be important as DA activity and release patterns change across learning, with several studies demonstrating a critical role for substantia nigra pars compacta (SNc) DA activity and release once behaviors are well-learned. We have demonstrated that medial septum (MS) activation simultaneously increased VTA and decreased SNc DA population activity, as well as improved reversal learning via these actions on DA activity. We hypothesized that MS activation would improve strategy switching both early in learning and after extended training through its ability to increase VTA DA population activity and decrease SNc DA population activity, respectively. We chemogenetically activated the MS of male and female rats and measured their performance on an operant-based strategy switching task following 1, 10, or 15 days of discrimination training. Contrary to our hypothesis, MS activation did not affect strategy switching after 1 day of discrimination training. MS activation improved strategy switching after 10 days of training, but only in females. MS activation improved strategy switching in both sexes after 15 days of training. Infusion of bicuculline into the ventral subiculum (vSub) inhibited the MS-mediated decrease in SNc DA population activity and attenuated the improvement in strategy switching. Intra-vSub infusion of scopolamine inhibited the MS-mediated increase in VTA DA population activity but did not affect the improvement in strategy switching. Intra-vSub infusion of both bicuculline and scopolamine inhibited the MS-mediated effects on DA population activity in both the SNc and VTA and completely prevented the improvement in strategy switching. These data indicate that MS activation improves strategy switching once the original strategy has been sufficiently well-learned, and that this may occur via the MS's regulation of DA neuron responsivity.

Distinct forms of regret linked to resilience versus susceptibility to stress are regulated by region-specific CREB function in mice

Regret describes recognizing alternative actions could have led to better outcomes. It remains unclear whether regret derives from generalized mistake appraisal or instead comprises dissociable, action-specific processes. Using a neuroeconomic task, we found that mice were sensitive to fundamentally distinct types of regret following exposure to chronic social defeat stress or manipulations of CREB, a transcription factor implicated in stress action. Bias to make compensatory decisions after rejecting high-value offers (regret type I) was unique to stress-susceptible mice. Bias following the converse operation, accepting low-value offers (regret type II), was enhanced in stress-resilient mice and absent in stress-susceptible mice. CREB function in either the prefrontal cortex or nucleus accumbens was required to suppress regret type I but bidirectionally regulated regret type II. We provide insight into how maladaptive stress response traits relate to distinct forms of counterfactual thinking, which could steer therapy for mood disorders, such as depression, toward circuit-specific computations through a careful description of decision narrative.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Delay discounting and probability discounting decision making tasks in rodent models have high translational potential. However, it is unclear whether the discounted value of the large reward option is the main contributor to variability in animals' choices in either task, which may limit translation to humans. Male and female mice underwent sessions of delay and probability discounting in sequence to assess how choice behavior adapts over experience with each task. To control for "anchoring" (persistent choices based on the initial delay or probability), mice experienced "Worsening" schedules where the large reward was offered under initially favorable conditions that became less favorable during testing, followed by "Improving" schedules where the large reward was offered under initially unfavorable conditions that improved over a session. During delay discounting, both male and female mice showed elimination of anchoring effects over training. In probability discounting, both sexes of mice continued to show some anchoring even after months of training. One possibility is that "noisy", exploratory choices could contribute to these persistent anchoring effects, rather than constant fluctuations in value discounting. We fit choice behavior in individual animals using models that included both a value-based discounting parameter and a decision noise parameter that captured variability in choices deviating from value maximization. Changes in anchoring behavior over time were tracked by changes in both the value and decision noise parameters in delay discounting, but by the decision noise parameter in probability discounting. Exploratory decision making was also reflected in choice response times that tracked the degree of conflict caused by both uncertainty and temporal cost, but was not linked with differences in locomotor activity reflecting chamber exploration. Thus, variable discounting behavior in mice can result from changes in exploration of the decision options rather than changes in reward valuation.

Individual differences in behavioral flexibility predict future volitional ethanol consumption in mice

Cognitive control is key to regulating alcohol intake and preventing relapse. Behavioral inflexibility can prevent adaptive strategies such as mindfulness or other relapse-prevention behaviors. In a mouse model we investigated whether individual variability in behavioral flexibility (using attentional set-shifting task; ASST) predicts future alcohol intake. Adult male and female C57BL/6J mice were subjected to ASST using a bowl-digging paradigm where mice identify a baited bowl based on compound odor and textural cues. This was completed prior to any alcohol exposure. Individual performance across mice varied within the group. We integrated several metrics, specifically ASST stage completed, trials to completion, and errors performed to produce an individual performance index measure of behavioral flexibility. Afterward, ASST mice were trained to drink ethanol (15%, v/v, 1 h/day) for 3-4 weeks until intake stabilized. Using this prospective approach, we identified an inverse relationship between behavioral flexibility and drinking-less-flexible mice had a propensity to consume more alcohol. Similar relationships have been identified previously in non-human primates and rats. Our results show that the relationship between alcohol and behavioral flexibility is a robust trait that is conserved across species and can be used in mice to study neural substrates underlying these behaviors.

A Signal Detection Analysis of Olfactory Learning in 12-Month-Old 5xFAD Mice

Although Alzheimer’s disease is most often studied in terms of memory impairments, olfactory dysfunction begins in the early stages. We tested olfactory learning, sensitivity, and response bias using signal detection methods in 12-month-old male and female 5xFAD mice and their wildtype controls in the operant olfactometer. Odor detection was not reduced in the 5xFAD mice, but learning was, which was worse in female 5xFAD mice than in males. Female mice were more conservative in their response strategy. Signal detection analysis allows us to discriminate between cognitive and sensory deficits of male and female mouse models of AD.

Reinforcement History Dependent Effects of Low Dose Ethanol on Reward Motivation in Male and Female Mice

Alcohol use disorders (AUDs) are more prevalent in men than in women, though AUD diagnoses in women are growing rapidly, making an understanding of sex differences in alcohol-related behaviors increasingly important. The development of AUDs involves the transition from casual, low levels of alcohol drinking to higher, maladaptive levels. The ability of low dose alcohol to drive reward and drug seeking may differ in males and females, and this could underlie differences in susceptibility to AUD. In this study we sought to determine whether a history of chronic, low dose ethanol exposure (0.5 g/kg; i.p.) could drive sucrose reward seeking and motivation, and whether this differed between male and female mice. Adult mice were trained to lever press for a liquid sucrose reward on two reinforcement schedules: a random interval (RI) schedule and a variable ratio (VR) schedule. After training, mice were tested on each of these levers for reward motivation using a progressive ratio test. We found that a history of low dose ethanol exposure increased sucrose reward motivation in male mice, but only on the RI lever and only when exposure occurred proximal to learning. Female mice were more motivated for sucrose on the RI lever than the VR lever regardless of ethanol exposure condition. These findings indicate that training on different reinforcement schedules affects reward motivation. Further, we show that males are more susceptible to the effects of low dose ethanol on sucrose reward motivation than females. These data broaden our understanding of sex differences in reward seeking as a result of ethanol exposure.

Sex Differences in Behavioral Responding and Dopamine Release during Pavlovian Learning

Learning associations between cues and rewards require the mesolimbic dopamine system. The dopamine response to cues signals differences in reward value in well trained animals. However, these value-related dopamine responses are absent during early training sessions when cues signal differences in the reward rate. These findings suggest cue-evoked dopamine release conveys differences between outcomes only after extensive training, though it is unclear whether this is unique to when cues signal differences in reward rate, or whether this is also evident when cues signal differences in other value-related parameters such as reward size. To address this, we used a Pavlovian conditioning task in which one audio cue was associated with a small reward (one pellet) and another audio cue was associated with a large reward (three pellets). We performed fast-scan cyclic voltammetry to record changes in dopamine release in the nucleus accumbens of male and female rats throughout learning. While female rats exhibited higher levels of conditioned responding, a faster latency to respond, and elevated post-reward head entries relative to male rats, there were no sex differences in the dopamine response to cues. Multiple training sessions were required before cue-evoked dopamine release signaled differences in reward size. Reward-evoked dopamine release scaled with reward size, though females displayed lower reward-evoked dopamine responses relative to males. Conditioned responding related to the decrease in the peak reward-evoked dopamine response and not to cue-evoked dopamine release. Collectively, these data illustrate sex differences in behavioral responding as well as in reward-evoked dopamine release during Pavlovian learning.

Roadbumps at the Crossroads of Integrating Behavioral and In Vitro Approaches for Neurotoxicity Assessment

With the appreciation that behavior represents the integration and complexity of the nervous system, neurobehavioral phenotyping and assessment has seen a renaissance over the last couple of decades, resulting in a robust database on rodent performance within various testing paradigms, possible associations with human disorders, and therapeutic interventions. The interchange of data across behavior and other test modalities and multiple model systems has advanced our understanding of fundamental biology and mechanisms associated with normal functions and alterations in the nervous system. While there is a demonstrated value and power of neurobehavioral assessments for examining alterations due to genetic manipulations, maternal factors, early development environment, the applied use of behavior to assess environmental neurotoxicity continues to come under question as to whether behavior represents a sensitive endpoint for assessment. Why is rodent behavior a sensitive tool to the neuroscientist and yet, not when used in pre-clinical or chemical neurotoxicity studies? Applying new paradigms and evidence on the biological basis of behavior to neurobehavioral testing requires expertise and refinement of how such experiments are conducted to minimize variability and maximize information. This review presents relevant issues of methods used to conduct such test, sources of variability, experimental design, data analysis, interpretation, and reporting. It presents beneficial and critical limitations as they translate to the in vivo environment and considers the need to integrate across disciplines for the best value. It proposes that a refinement of behavioral assessments and understanding of subtle pronounced differences will facilitate the integration of data obtained across multiple approaches and to address issues of translation.

Sex differences in learning from exploration

Sex-based modulation of cognitive processes could set the stage for individual differences in vulnerability to neuropsychiatric disorders. While value-based decision making processes in particular have been proposed to be influenced by sex differences, the overall correct performance in decision making tasks often show variable or minimal differences across sexes. Computational tools allow us to uncover latent variables that define different decision making approaches, even in animals with similar correct performance. Here, we quantify sex differences in mice in the latent variables underlying behavior in a classic value-based decision making task: a restless 2-armed bandit. While male and female mice had similar accuracy, they achieved this performance via different patterns of exploration. Male mice tended to make more exploratory choices overall, largely because they appeared to get 'stuck' in exploration once they had started. Female mice tended to explore less but learned more quickly during exploration. Together, these results suggest that sex exerts stronger influences on decision making during periods of learning and exploration than during stable choices. Exploration during decision making is altered in people diagnosed with addictions, depression, and neurodevelopmental disabilities, pinpointing the neural mechanisms of exploration as a highly translational avenue for conferring sex-modulated vulnerability to neuropsychiatric diagnoses.

Dopamine Circuit Mechanisms of Addiction-Like Behaviors

Addiction is a complex disease that impacts millions of people around the world. Clinically, addiction is formalized as substance use disorder (SUD), with three primary symptom categories: exaggerated substance use, social or lifestyle impairment, and risky substance use. Considerable efforts have been made to model features of these criteria in non-human animal research subjects, for insight into the underlying neurobiological mechanisms. Here we review evidence from rodent models of SUD-inspired criteria, focusing on the role of the striatal dopamine system. We identify distinct mesostriatal and nigrostriatal dopamine circuit functions in behavioral outcomes that are relevant to addictions and SUDs. This work suggests that striatal dopamine is essential for not only positive symptom features of SUDs, such as elevated intake and craving, but also for impairments in decision making that underlie compulsive behavior, reduced sociality, and risk taking. Understanding the functional heterogeneity of the dopamine system and related networks can offer insight into this complex symptomatology and may lead to more targeted treatments.

The population doctrine in cognitive neuroscience

A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.

Electrophysiological biomarkers of behavioral dimensions from cross-species paradigms

There has been a fundamental failure to translate preclinically supported research into clinically efficacious treatments for psychiatric disorders. One of the greatest impediments toward improving this species gap has been the difficulty of identifying translatable neurophysiological signals that are related to specific behavioral constructs. Here, we present evidence from three paradigms that were completed by humans and mice using analogous procedures, with each task eliciting candidate a priori defined electrophysiological signals underlying effortful motivation, reinforcement learning, and cognitive control. The effortful motivation was assessed using a progressive ratio breakpoint task, yielding a similar decrease in alpha-band activity over time in both species. Reinforcement learning was assessed via feedback in a probabilistic learning task with delta power significantly modulated by reward surprise in both species. Additionally, cognitive control was assessed in the five-choice continuous performance task, yielding response-locked theta power seen across species, and modulated by difficulty in humans. Together, these successes, and also the teachings from these failures, provide a roadmap towards the use of electrophysiology as a method for translating findings from the preclinical assays to the clinical settings.

Syn3 Gene Knockout Negatively Impacts Aspects of Reversal Learning Performance

Behavioral flexibility enables the ability to adaptively respond to changes in contingency requirements to maintain access to desired outcomes, and deficits in behavioral flexibility have been documented in many psychiatric disorders. Previous research has shown a correlation between behavioral flexibility measured in a reversal learning test and Syn3, the gene encoding synapsin III, which negatively regulates phasic dopamine release. Syn3 expression in the hippocampus, striatum, and neocortex is reported to be negatively correlated with reversal learning performance, so here, we used a global knock-out line to investigate reversal learning in mice homozygous wild type, heterozygous null, and homozygous null for the Syn3 gene. Compared with wild-type animals, we found a reversal-specific effect of genetic Syn3 deficiency that resulted in a greater proportional increase in trials required to reach a preset performance criterion during contingency reversal, despite no observed genotype effects on the ability to acquire the initial discrimination. Behavioral flexibility scores, which quantified the likelihood of switching subsequent choice behavior following positive or negative feedback, became significantly more negative in reversal only for Syn3 homozygous-null mice, suggesting a substantial increase in perseverative behavior in the reversal phase. Syn3 ablation reduced the number of anticipatory responses made per trial, often interpreted as a measure of waiting impulsivity. Overall, Syn3 expression negatively affected behavioral flexibility in a reversal-specific manner but may have reduced waiting impulsivity.

Considering sex as a biological variable will require a global shift in science culture

For over half a century, male rodents have been the default model organism in preclinical neuroscience research, a convention that has likely contributed to higher rates of misdiagnosis and adverse side effects from drug treatment in women. Studying both sexes could help to rectify these public health problems, but incentive structures in publishing and career advancement deter many researchers from doing so. Moreover, funding agency directives to include male and female animals and human participants in grant proposals lack mechanisms to hold recipients accountable. In this Perspective, we highlight areas of behavioral, cellular and systems neuroscience in which fundamental sex differences have been identified, demonstrating that truly rigorous science must include males and females. We call for a cultural and structural change in how we conduct research and evaluate scientific progress, realigning our professional reward systems and experimental standards to produce a more equitable, representative and therefore translational body of knowledge.