Preferential involvement by nucleus accumbens shell in mediating probabilistic learning and reversal shifts

In relentless pursuit of the white whale: A role for the ventral midline thalamus in behavioral flexibility and adaption?

The reuniens (Re) nucleus is located in the ventral midline thalamus. It has fostered increasing interest, not only for its participation in a variety of cognitive functions (e.g., spatial working memory, systemic consolidation, reconsolidation, extinction of fear or generalization), but also for its neuroanatomical positioning as a bidirectional relay between the prefrontal cortex (PFC) and the hippocampus (HIP). In this review we compile and discuss recent studies having tackled a possible implication of the Re nucleus in behavioral flexibility, a major PFC-dependent executive function controlling goal-directed behaviors. Experiments considered explored a possible role for the Re nucleus in perseveration, reversal learning, fear extinction, and set-shifting. They point to a contribution of this nucleus to behavioral flexibility, mainly by its connections with the PFC, but possibly also by those with the hippocampus, and even with the amygdala, at least for fear-related behavior. As such, the Re nucleus could be a crucial crossroad supporting a PFC-orchestrated ability to cope with new, potentially unpredictable environmental contingencies, and thus behavioral flexibility and adaption.

Behavioral Interventions Can Improve Brain Injury-Induced Deficits in Behavioral Flexibility and Impulsivity Linked to Impaired Reward-Feedback Beta Oscillations

Traumatic brain injury (TBI) affects a large population, resulting in severe cognitive impairments. Although cognitive rehabilitation is an accepted treatment for some deficits, studies in patients are limited in ability to probe physiological and behavioral mechanisms. Therefore, animal models are needed to optimize strategies. Frontal TBI in a rat model results in robust and replicable cognitive deficits, making this an ideal candidate for investigating various behavioral interventions. In this study, we report three distinct frontal TBI experiments assessing behavior well into the chronic post-injury period using male Long-Evans rats. First, we evaluated the impact of frontal injury on local field potentials recorded simultaneously from 12 brain regions during a probabilistic reversal learning (PbR) task. Next, a set of rats were tested on a similar PbR task or an impulsivity task (differential reinforcement of low-rate behavior [DRL]) and half received salient cues associated with reinforcement contingencies to encourage engagement in the target behavior. After intervention on the PbR task, brains were stained for markers of activity. On the DRL task, cue relevance was decoupled from outcomes to determine if beneficial effects persisted on impulsive behavior. TBI decreased the ability to detect reinforced outcomes; this was evident in task performance and reward-feedback signals occurring at beta frequencies in lateral orbitofrontal cortex (OFC) and associated frontostriatal regions. The behavioral intervention improved flexibility and increased OFC activity. Intervention also reduced impulsivity, even after cues were decoupled, which was partially mediated by improvements in timing behavior. The current study established a platform to begin investigating cognitive rehabilitation in rats and identified a strong role for dysfunctional OFC signaling in probabilistic learning after frontal TBI.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Foraging Under Uncertainty Follows the Marginal Value Theorem with Bayesian Updating of Environment Representations

Foraging theory has been a remarkably successful approach to understanding the behavior of animals in many contexts. In patch-based foraging contexts, the marginal value theorem (MVT) shows that the optimal strategy is to leave a patch when the marginal rate of return declines to the average for the environment. However, the MVT is only valid in deterministic environments whose statistics are known to the forager; naturalistic environments seldom meet these strict requirements. As a result, the strategies used by foragers in naturalistic environments must be empirically investigated. We developed a novel behavioral task and a corresponding computational framework for studying patch-leaving decisions in head-fixed and freely moving mice. We varied between-patch travel time, as well as within-patch reward depletion rate, both deterministically and stochastically. We found that mice adopt patch residence times in a manner consistent with the MVT and not explainable by simple ethologically motivated heuristic strategies. Critically, behavior was best accounted for by a modified form of the MVT wherein environment representations were updated based on local variations in reward timing, captured by a Bayesian estimator and dynamic prior. Thus, we show that mice can strategically attend to, learn from, and exploit task structure on multiple timescales simultaneously, thereby efficiently foraging in volatile environments. The results provide a foundation for applying the systems neuroscience toolkit in freely moving and head-fixed mice to understand the neural basis of foraging under uncertainty.

Identification of genes regulated by trait sensitivity to negative feedback and prolonged alcohol consumption in rats

BACKGROUND

The results of our previous studies demonstrated that low sensitivity to negative feedback (NF) is associated with increased vulnerability to the development of compulsive alcohol-seeking in rats. In the present study, we investigated the molecular underpinnings of this relationship.

METHODS

Using TaqMan Gene Expression Array Cards, we analyzed the expression of the genes related to NF sensitivity and alcohol metabolism in three cortical regions (medial prefrontal cortex [mPFC], anterior cingulate cortex [ACC], orbitofrontal cortex [OFC]) and two subcortical regions (nucleus accumbens [Nacc], amygdala [Amy]). Gene expression differences were confirmed at the protein level with Western blot.

RESULTS

Sensitivity to NF was characterized by differences in Gad2, Drd2, and Slc6a4 expression in the ACC, Maoa in the mPFC, and Gria1, Htr3a, and Maoa in the OFC. Chronic alcohol consumption was associated with differences in the expression of Comt and Maoa in the ACC, Comt, Adh1, and Htr2b in the mPFC, Adh1, and Slc6a4 in the Nacc, Gad2, and Htr1a in the OFC, and Drd2 in the Amy. Interactions between the sensitivity to NF and alcohol consumption were observed in the expression of Gabra1, Gabbr2, Grin2a, Grin2b, and Grm3 in the ACC, and Grin2a in the OFC. The observed differences were confirmed at the protein level for MAO-A in the mPFC, and ADH1 in the mPFC and Nacc.

CONCLUSIONS

Our findings contribute to a better understanding of the molecular mechanisms underlying the relationship between trait sensitivity to NF and compulsive alcohol consumption.

Effects of reproductive experience on cost-benefit decision making in female rats

Many individuals undergo mating and/or other aspects of reproductive experience at some point in their lives, and pregnancy and childbirth in particular are associated with alterations in the prevalence of several psychiatric disorders. Research in rodents shows that maternal experience affects spatial learning and other aspects of hippocampal function. In contrast, there has been little work in animal models concerning how reproductive experience affects cost-benefit decision making, despite the relevance of this aspect of cognition for psychiatric disorders. To begin to address this issue, reproductively experienced (RE) and reproductively naïve (RN) female Long-Evans rats were tested across multiple tasks that assess different forms of cost-benefit decision making. In a risky decision-making task, in which rats chose between a small, safe food reward and a large food reward accompanied by variable probabilities of punishment, RE females chose the large risky reward significantly more frequently than RN females (greater risk taking). In an intertemporal choice task, in which rats chose between a small, immediate food reward and a large food reward delivered after a variable delay period, RE females chose the large reward less frequently than RN females. Together, these results show distinct effects of reproductive experience on different forms of cost-benefit decision making in female rats, and highlight reproductive status as a variable that could influence aspects of cognition relevant for psychiatric disorders.

Trait sensitivity to positive feedback is a predisposing factor for several aspects of compulsive alcohol drinking in male rats: behavioural, physiological, and molecular correlates

INTRODUCTION

Alcohol use disorder (AUD) is one of the most common psychiatric disorders and a leading cause of mortality worldwide. While the pathophysiology underlying AUD is relatively well known, the cognitive mechanisms of an individual's susceptibility to the development of alcohol dependence remain poorly understood. In this study, we investigated the theoretical claim that sensitivity to positive feedback (PF), as a stable and enduring behavioural trait, can predict individual susceptibility to the acquisition and maintenance of alcohol-seeking behaviour in rats.

METHODS

Trait sensitivity to PF was assessed using a series of probabilistic reversal learning tests. The escalation of alcohol intake in rats was achieved by applying a mix of intermittent free access and instrumental paradigms of alcohol drinking. The next steps included testing the influence of sensitivity to PF on the acquisition of compulsive alcohol-seeking behaviour in the seeking-taking punishment task, measuring motivation to seek alcohol, and comparing the speed of extinction and reinstatement of alcohol-seeking after a period of abstinence between rats expressing trait insensitivity and sensitivity to PF. Finally, trait differences in the level of stress hormones and in the expression of genes and proteins in several brain regions of interest were measured to identify potential physiological and neuromolecular mechanisms of the observed interactions.

RESULTS

We showed that trait sensitivity to PF in rats determines the level of motivation to seek alcohol following the experience of its negative consequences. They also revealed significant differences between animals classified as insensitive and sensitive to PF in their propensity to reinstate alcohol-seeking behaviours after a period of forced abstinence. The abovementioned effects were accompanied by differences in blood levels of stress hormones and differences in the cortical and subcortical expression of genes and proteins related to dopaminergic, serotonergic, and GABAergic neurotransmission.

CONCLUSION

Trait sensitivity to PF can determine the trajectory of alcohol addiction in rats. This effect is, at least partially, mediated via distributed physiological and molecular changes within cortical and subcortical regions of the brain.

Effects of fentanyl self-administration on risk-taking behavior in male rats

RATIONALE

Individuals with opioid use disorder (OUD) exhibit impaired decision making and elevated risk-taking behavior. In contrast to the effects of natural and semi-synthetic opioids, however, the impact of synthetic opioids on decision making is still unknown.

OBJECTIVES

The objective of the current study was to determine how chronic exposure to the synthetic opioid fentanyl alters risk-based decision making in adult male rats.

METHODS

Male rats underwent 14 days of intravenous fentanyl or oral sucrose self-administration. After 3 weeks of abstinence, rats were tested in a decision-making task in which they chose between a small, safe food reward and a large food reward accompanied by variable risk of footshock punishment. Following testing in the decision-making task, rats were tested in control assays that assessed willingness to work for food and shock reactivity. Lastly, rats were tested on a probabilistic reversal learning task to evaluate enduring effects of fentanyl on behavioral flexibility.

RESULTS

Relative to rats in the sucrose group, rats in the fentanyl group displayed greater choice of the large, risky reward (risk taking), an effect that was present as long as 7 weeks into abstinence. This increased risk taking was driven by enhanced sensitivity to the large rewards and diminished sensitivity to punishment. The fentanyl-induced elevation in risk taking was not accompanied by alterations in food motivation or shock reactivity or impairments in behavioral flexibility.

CONCLUSIONS

Results from the current study reveal that the synthetic opioid fentanyl leads to long-lasting increases in risk taking in male rats. Future experiments will extend this work to females and identify neural mechanisms that underlie these drug-induced changes in risk taking.

Statistical power and false positive rates for interdependent outcomes are strongly influenced by test type: Implications for behavioral neuroscience

Statistical errors in preclinical science are a barrier to reproducibility and translation. For instance, linear models (e.g., ANOVA, linear regression) may be misapplied to data that violate assumptions. In behavioral neuroscience and psychopharmacology, linear models are frequently applied to interdependent or compositional data, which includes behavioral assessments where animals concurrently choose between chambers, objects, outcomes, or types of behavior (e.g., forced swim, novel object, place/social preference). The current study simulated behavioral data for a task with four interdependent choices (i.e., increased choice of a given outcome decreases others) using Monte Carlo methods. 16,000 datasets were simulated (1000 each of 4 effect sizes by 4 sample sizes) and statistical approaches evaluated for accuracy. Linear regression and linear mixed effects regression (LMER) with a single random intercept resulted in high false positives (>60%). Elevated false positives were attenuated in an LMER with random effects for all choice-levels and a binomial logistic mixed effects regression. However, these models were underpowered to reliably detect effects at common preclinical sample sizes. A Bayesian method using prior knowledge for control subjects increased power by up to 30%. These results were confirmed in a second simulation (8000 datasets). These data suggest that statistical analyses may often be misapplied in preclinical paradigms, with common linear methods increasing false positives, but potential alternatives lacking power. Ultimately, using informed priors may balance statistical requirements with ethical imperatives to minimize the number of animals used. These findings highlight the importance of considering statistical assumptions and limitations when designing research studies.

EPD1504: a novel μ-opioid receptor partial agonist attenuates obsessive-compulsive disorder (OCD)-like behaviors

Low doses of μ-opioid receptor (MOR) agonists rapidly ameliorate symptoms in treatment-resistant obsessive-compulsive disorder (OCD) patients (10-50% of OCD patients). However, the utility of MOR agonists is limited by their safety liabilities. We developed a novel MOR partial agonist (EPD1540) that has an improved respiratory safety profile when compared to buprenorphine. Buprenorphine is a MOR partial agonist primarily used in the treatment of opiate-use disorder, which in investigator-led trials, has been shown to rapidly ameliorate symptoms in treatment-resistant OCD patients. In this study, we show that doses of EPD1504 and buprenorphine that occupy small fractions of MORs in the CNS (approximately 20%) are as effective as fluoxetine at ameliorating OCD-like behaviors in two different rat models (an operant probabilistic reversal task and marble burying). Importantly, effective doses of EPD1504 did not impair either locomotor activity, or respiration under normoxic or hypercapnic conditions. Additionally, EPD1504 had effects comparable to buprenorphine in the conditioned place preference assay. These results indicate that EPD1504 may provide a safer alternative to buprenorphine for the treatment of OCD patients.

Modulation of ventromedial orbitofrontal cortical glutamatergic activity affects the explore-exploit balance and influences value-based decision-making

The balance between exploration and exploitation is essential for decision-making. The present study investigated the role of ventromedial orbitofrontal cortex (vmOFC) glutamate neurons in mediating value-based decision-making by first using optogenetics to manipulate vmOFC glutamate activity in rats during a probabilistic reversal learning (PRL) task. Rats that received vmOFC activation during informative feedback completed fewer reversals and exhibited reduced reward sensitivity relative to rats. Analysis with a Q-learning computational model revealed that increased vmOFC activity did not affect the learning rate but instead promoted maladaptive exploration. By contrast, vmOFC inhibition increased the number of completed reversals and increased exploitative behavior. In a separate group of animals, calcium activity of vmOFC glutamate neurons was recorded using fiber photometry. Complementing our results above, we found that suppression of vmOFC activity during the latter part of rewarded trials was associated with improved PRL performance, greater win-stay responding and selecting the correct choice on the next trial. These data demonstrate that excessive vmOFC activity during reward feedback disrupted value-based decision-making by increasing the maladaptive exploration of lower-valued options. Our findings support the premise that pharmacological interventions that normalize aberrant vmOFC glutamate activity during reward feedback processing may attenuate deficits in value-based decision-making.

Age-Related Changes in Risky Decision Making and Associated Neural Circuitry in a Rat Model

Altered decision making at advanced ages can have a significant impact on an individual's quality of life and the ability to maintain personal independence. Relative to young adults, older adults make less impulsive and less risky choices; although these changes in decision making could be considered beneficial, they can also lead to choices with potentially negative consequences (e.g., avoidance of medical procedures). Rodent models of decision making have been invaluable for dissecting cognitive and neurobiological mechanisms that contribute to age-related changes in decision making, but they have predominantly used costs related to timing or probability of reward delivery and have not considered other equally important costs, such as the risk of adverse consequences. The current study therefore used a rat model of decision making involving risk of explicit punishment to examine age-related changes in this form of choice behavior in male rats, and to identify potential cognitive and neurobiological mechanisms that contribute to these changes. Relative to young rats, aged rats displayed greater risk aversion, which was not attributable to reduced motivation for food, changes in shock sensitivity, or impaired cognitive flexibility. Functional MRI analyses revealed that, overall, functional connectivity was greater in aged rats compared with young rats, particularly among brain regions implicated in risky decision making such as basolateral amygdala, orbitofrontal cortex, and ventral tegmental area. Collectively, these findings are consistent with greater risk aversion found in older humans, and reveal age-related changes in brain connectivity.

Sustained MK-801 induced deficit in a novel probabilistic reversal learning task

Cognitive flexibility, the ability to adapt to unexpected changes, is critical for healthy environmental and social interactions, and thus to everyday functioning. In neuropsychiatric diseases, cognitive flexibility is often impaired and treatment options are lacking. Probabilistic reversal learning (PRL) is commonly used to measure cognitive flexibility in rodents and humans. In PRL tasks, subjects must sample choice options and, from probabilistic feedback, find the current best choice which then changes without warning. However, in rodents, pharmacological models of human cognitive impairment tend to disrupt only the first (or few) of several contingency reversals, making quantitative assessment of behavioral effects difficult. To address this limitation, we developed a novel rat PRL where reversals occur at relatively long intervals in time that demonstrates increased sensitivity to the non-competitive NMDA receptor antagonist MK-801. Here, we quantitively compare behavior in time-based PRL with a widely used task where reversals occur based on choice behavior. In time-based PRL, MK-801 induced sustained reversal learning deficits both in time and across reversal blocks but, at the same dose, only transient weak effects in performance-based PRL. Moreover, time-based PRL yielded better estimates of behavior and reinforcement learning model parameters, which opens meaningful pharmacological windows to efficiently test and develop novel drugs preclinically with the goal of improving cognitive impairment in human patients.

Nucleus Accumbens Core Dopamine D2 Receptor-Expressing Neurons Control Reversal Learning but Not Set-Shifting in Behavioral Flexibility in Male Mice

The ability to use environmental cues to flexibly guide responses is crucial for adaptive behavior and is thought to be controlled within a series of cortico-basal ganglia-thalamo-cortical loops. Previous evidence has indicated that different prefrontal cortical regions control dissociable aspects of behavioral flexibility, with the medial prefrontal cortex (mPFC) necessary for the ability to shift attention to a novel strategy (set-shifting) and the orbitofrontal cortex (OFC) necessary for shifting attention between learned stimulus-outcome associations (reversal learning). The nucleus accumbens (NAc) is a major downstream target of both the mPFC and the OFC; however, its role in controlling reversal learning and set-shifting abilities is still unclear. Here we investigated the contribution of the two major NAc neuronal populations, medium spiny neurons expressing either dopamine D1 or D2 receptors (D1-/D2-MSNs), in guiding reversal learning and set-shifting in an attentional set-shifting task (ASST). Persistent inhibition of neurotransmitter release from NAc D2-MSNs, but not D1-MSNs, resulted in an impaired ability for reversal learning, but not set-shifting in male mice. These findings suggest that NAc D2-MSNs play a critical role in suppressing responding toward specific learned cues that are now associated with unfavorable outcomes (i.e., in reversal stages), but not in the suppression of more general learned strategies (i.e., in set-shifting). This study provides further evidence for the anatomical separation of reversal learning and set-shifting abilities within cortico-basal ganglia-thalamo-cortical loops.

Large-N Rat Data Enables Phenotyping of Risky Decision-Making: A Retrospective Analysis of Brain Injury on the Rodent Gambling Task

Decision-making is substantially altered after brain injuries. Patients and rats with brain injury are more likely to make suboptimal, and sometimes risky choices. Such changes in decision-making may arise from alterations in how sensitive individuals are to outcomes. To assess this, we compiled and harmonized a large dataset from four studies of TBI, each of which evaluated behavior on the Rodent Gambling Task (RGT). We then determined whether the following were altered: (1) sensitivity to overall contingencies, (2) sensitivity to immediate outcomes, or (3) general choice phenotypes. Overall sensitivity was evaluated using the matching law, immediate sensitivity by looking at the probability of switching choices given a win or loss, and choice phenotypes by k-means clustering. We found significant reductions in sensitivity to the overall outcomes and a bias toward riskier alternatives in TBI rats. However, the substantial individual variability led to poor overall fits in matching analyses. We also found that TBI caused a significant reduction in the tendency to repeatedly choose a given option, but no difference in win- or loss-specific sensitivity. Finally, clustering revealed 5 distinct decision-making phenotypes and TBI reduced membership in the “optimal” type. The current findings support a hypothesis that TBI reduces sensitivity to contingencies. However, in the case of tasks such as the RGT, this is not a simple shift to indiscriminate or less discriminate responding. Rather, TBI rats are more likely to develop suboptimal preferences and frequently switch choices. Treatments will have to consider how this behavior might be corrected.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

HIV Transgenic Rats Demonstrate Superior Task Acquisition and Intact Reversal Learning in the Within-Session Probabilistic Reversal Learning Task

The HIV transgenic (HIVtg) rat is a commonly used animal model of chronic HIV infection that exhibits a wide range of cognitive deficits. To date, relatively little work has been conducted on these rats' capacity for reversal learning, an assay of executive function and cognitive flexibility used in humans. The present study sought to determine the impact of HIV genotype on probabilistic reversal learning, effortful motivation, and spontaneous locomotion/exploration in rats. Male (n = 8) and female (n = 8) HIVtg rats and wildtype (WT) controls were utilized. Cognitive flexibility was assessed via the Probabilistic Reversal Learning Task (PRLT), which reinforced responses to two stimuli on differential probabilistic schedules that periodically reversed. Effortful motivation and locomotor/exploratory behavior were assessed via the Progressive Ratio Breakpoint Task (PRBT) and the Behavioral Pattern Monitor (BPM), respectively. Regardless of sex, HIVtg rats required fewer trials to ascertain initial PRLT reward schedules than WT rats, and completed the same number of reversals. Secondary behaviors suggested that HIVtg PRLT performance was facilitated by a speed-accuracy tradeoff strategy. No main or interactive effects of genotype were observed in the PRBT or BPM. Relative to WT controls, HIVtg rats exhibited superior probabilistic reinforcement learning. Reversal learning was unaffected by HIV genotype, as was effortful motivation and exploratory behavior. These findings contrast with previous characterizations of the HIVtg rat, thus indicating a nuanced cognitive profile that is dependent upon such task specifications as within- versus between-session assessment and probabilistic versus deterministic reward schedules.

Entropy-based metrics for predicting choice behavior based on local response to reward

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

Reinforcement-based cognitive biases as vulnerability factors in alcohol addiction: From humans to animal models

Alcohol use disorder (AUD) is one of the most common, but still poorly treated, psychiatric conditions. Developing new treatments requires a better understanding of the aetiology of symptoms and evaluation of novel therapeutic targets in preclinical studies. Recent developments in our understanding of the reinforcement-based cognitive biases (RBCBs) that contribute to the development of AUD and its treatment offer new opportunities for both clinical and preclinical research. In this review, we first briefly describe psychological and cognitive theories that involve various aspects of reinforcement sensitivity in the development, maintenance, and recurrence of alcohol addiction. Furthermore, in separate sections, we describe studies investigating RBCBs and their neural, neurochemical, and pharmacological correlates, and we discuss possible interactions between RBCBs and trajectories of AUD. Finally, we describe how recent translational studies using state-of-the-art animal models can facilitate our understanding of the role of reinforcement sensitivity and RBCBs in various aspects of AUD.

Effects of repeated adolescent exposure to cannabis smoke on cognitive outcomes in adulthood

BACKGROUND

Cannabis (marijuana) is the most widely used illicit drug in the USA, and consumption among adolescents is rising. Some animal studies show that adolescent exposure to delta 9-tetrahydrocannabinol or synthetic cannabinoid receptor 1 agonists causes alterations in affect and cognition that can persist into adulthood. It is less clear, however, whether similar alterations result from exposure to cannabis via smoke inhalation, which remains the most frequent route of administration in humans.

AIMS

To begin to address these questions, a rat model was used to determine how cannabis smoke exposure during adolescence affects behavioral and cognitive outcomes in adulthood.

METHODS

Adolescent male Long-Evans rats were assigned to clean air, placebo smoke, or cannabis smoke groups. Clean air or smoke exposure sessions were conducted daily during adolescence (from P29-P49 days of age ) for a total of 21 days, and behavioral testing began on P70.

RESULTS

Compared to clean air and placebo smoke conditions, cannabis smoke significantly attenuated the normal developmental increase in body weight, but had no effects on several measures of either affect/motivation (open field activity, elevated plus maze, instrumental responding under a progressive ratio schedule of reinforcement) or cognition (set shifting, reversal learning, intertemporal choice). Surprisingly, however, in comparison to clean air controls rats exposed to either cannabis or placebo smoke in adolescence exhibited enhanced performance on a delayed response working memory task.

CONCLUSIONS

These findings are consistent with a growing body of evidence for limited long-term adverse cognitive and affective consequences of adolescent exposure to relatively low levels of cannabinoids.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Differential effects of d- and l-enantiomers of govadine on distinct forms of cognitive flexibility and a comparison with dopaminergic drugs

RATIONALE

There is an urgent need for novel drugs for treating cognitive deficits that are defining features of schizophrenia. The individual d- and l-enantiomers of the tetrahydroprotoberberine (THPB) d,l-govadine have been proposed for the treatment of cognitive deficiencies and positive symptoms of schizophrenia, respectively.

OBJECTIVES

We examined the effects of d-, l-, or d,l-govadine on two distinct forms of cognitive flexibility perturbed in schizophrenia and compared them to those induced by a selective D1 receptor agonist and D2 receptor antagonist.

METHODS

Male rats received d-, l-, or d,l-govadine (0.3, 0.5, and 1.0 mg/kg), D1 agonist SKF81297(0.1, 0.3, and 1.0 mg/kg), or D2 antagonist haloperidol (0.1-0.2 mg/kg). Experiment 1 used a strategy set-shifting task (between-subjects). In experiment 2, well-trained rats were tested on a probabilistic reversal task (within-subjects).

RESULTS

d-Govadine improved set-shifting across all doses, whereas higher doses of l-govadine impaired set-shifting. SKF81297 reduced perseverative errors at the lowest dose. Low/high doses of haloperidol increased/decreased set-shifting errors, the latter "improvement" attributable to impaired retrieval of a previous acquired rule. Probabilistic reversal performance was less affected by these drugs, but d-govadine reduced errors during the first reversal, whereas l-govadine impaired initial discrimination learning. d,l-Govadine had no reliable cognitive effects but caused psychomotor slowing like l-govadine and haloperidol.

CONCLUSIONS

These findings further highlight differences between two enantiomers of d,l-govadine that may reflect differential modulation of D1 and D2 receptors. These preclinical findings give further impetus to formal clinical evaluation of d-govadine as a treatment for cognitive deficiencies related to schizophrenia.

Spectral signatures of L-DOPA-induced dyskinesia depend on L-DOPA dose and are suppressed by ketamine

L-DOPA-induced dyskinesias (LID) are debilitating motor symptoms of dopamine-replacement therapy for Parkinson's disease (PD) that emerge after years of L-DOPA treatment. While there is an abundance of research into the cellular and synaptic origins of LID, less is known about how LID impacts systems-level circuits and neural synchrony, how synchrony is affected by the dose and duration of L-DOPA exposure, or how potential novel treatments for LID, such as sub-anesthetic ketamine, alter this activity. Sub-anesthetic ketamine treatments have recently been shown to reduce LID, and ketamine is known to affect neural synchrony. To investigate these questions, we measured movement and local-field potential (LFP) activity from the motor cortex (M1) and the striatum of preclinical rodent models of PD and LID. In the first experiment, we investigated the effect of the LID priming procedures and L-DOPA dose on neural signatures of LID. Two common priming procedures were compared: a high-dose procedure that exposed unilateral 6-hydroxydopamine-lesioned rats to 12 mg/kg L-DOPA for 7 days, and a low-dose procedure that exposed rats to 7 mg/kg L-DOPA for 21 days. Consistent with reports from other groups, 12 mg/kg L-DOPA triggered LID and 80-Hz oscillations; however, these 80-Hz oscillations were not observed after 7 mg/kg administration despite clear evidence of LID, indicating that 80-Hz oscillations are not an exclusive signature of LID. We also found that weeks-long low-dose priming resulted in the emergence of non-oscillatory broadband gamma activity (> 30 Hz) in the striatum and theta-to-high-gamma cross-frequency coupling (CFC) in M1. In a second set of experiments, we investigated how ketamine exposure affects spectral signatures of low-dose L-DOPA priming. During each neural recording session, ketamine was delivered through 5 injections (20 mg/kg, i.p.) administered every 2 h. We found that ketamine exposure suppressed striatal broadband gamma associated with LID but enhanced M1 broadband activity. We also found that M1 theta-to-high-gamma CFC associated with the LID on-state was suppressed by ketamine. These results suggest that ketamine's therapeutic effects are region specific. Our findings also have clinical implications, as we are the first to report novel oscillatory signatures of the common low-dose LID priming procedure that more closely models dopamine replacement therapy in individuals with PD. We also identify neural correlates of the anti-dyskinetic activity of sub-anesthetic ketamine treatment.

Development and evaluation of an artificial intelligence system for COVID-19 diagnosis

Early detection of COVID-19 based on chest CT enables timely treatment of patients and helps control the spread of the disease. We proposed an artificial intelligence (AI) system for rapid COVID-19 detection and performed extensive statistical analysis of CTs of COVID-19 based on the AI system. We developed and evaluated our system on a large dataset with more than 10 thousand CT volumes from COVID-19, influenza-A/B, non-viral community acquired pneumonia (CAP) and non-pneumonia subjects. In such a difficult multi-class diagnosis task, our deep convolutional neural network-based system is able to achieve an area under the receiver operating characteristic curve (AUC) of 97.81% for multi-way classification on test cohort of 3,199 scans, AUC of 92.99% and 93.25% on two publicly available datasets, CC-CCII and MosMedData respectively. In a reader study involving five radiologists, the AI system outperforms all of radiologists in more challenging tasks at a speed of two orders of magnitude above them. Diagnosis performance of chest x-ray (CXR) is compared to that of CT. Detailed interpretation of deep network is also performed to relate system outputs with CT presentations. The code is available at https://github.com/ChenWWWeixiang/diagnosis_covid19 .

Adaptive behaviour under conflict: Deconstructing extinction, reversal, and active avoidance learning

In complex environments, organisms must respond adaptively to situations despite conflicting information. Under natural (i.e. non-laboratory) circumstances, it is rare that cues or responses are consistently paired with a single outcome. Inconsistent pairings are more common, as are situations where cues and responses are associated with multiple outcomes. Such inconsistency creates conflict, and a response that is adaptive in one scenario may not be adaptive in another. Learning to adjust responses accordingly is important for species to survive and prosper. Here we review the behavioural and brain mechanisms of responding under conflict by focusing on three popular behavioural procedures: extinction, reversal learning, and active avoidance. Extinction involves adapting from reinforcement to non-reinforcement, reversal learning involves swapping the reinforcement of cues or responses, and active avoidance involves performing a response to avoid an aversive outcome, which may conflict with other defensive strategies. We note that each of these phenomena relies on somewhat overlapping neural circuits, suggesting that such circuits may be critical for the general ability to respond appropriately under conflict.

Central CRF and acute stress differentially modulate probabilistic reversal learning in male and female rats

Acute stress can have variable and sometimes sex-dependent effects on different executive functions, including cognitive flexibility, some of which may be mediated by increased corticotropin releasing factor (CRF). Previous studies on the effects of stress and CRF on cognitive flexibility have used procedures entailing deterministic rewards, yet how they may alter behavior when outcomes are probabilistic is unclear. The present study examined how acute stress and increased CRF activity alters probabilistic reversal learning (PRL) in male and female rats. Rats learned to discriminate between a 'correct' lever rewarded on 80 % of trials, and an "incorrect" lever delivering reward on 20 % of trials, with reward contingencies reversed after 8 consecutive correct choices. Separate groups received either intracerebroventricular infusions of CRF (3 μg) or restraint stress prior to a PRL session. Experiments examined how these manipulations affected learning when given prior to a one-day acquisition test or during performance in well-trained rats. Exogenous CRF, and to a lesser extent acute stress, impaired motivation across sexes, slowing deliberation times and increasing the number of trials omitted, particularly following a switch in reward contingencies. Neither manipulation significantly altered errors or reversal performance. However, increased CRF activity reduced negative feedback sensitivity. Across manipulations, females showed increased omissions and choice latencies, and were less sensitive to feedback than males. These results reveal the complexity with which stress, CRF, sex, and experience interact to alter aspects of motivation and probabilistic reinforcement learning and provide insight into how CRF activity may contribute to symptoms of stress-related disorders.

Prefrontal cortical and nucleus accumbens contributions to discriminative conditioned suppression of reward-seeking

Fear can potently inhibit ongoing behavior, including reward-seeking, yet the neural circuits that underlie such suppression remain to be clarified. Prior studies have demonstrated that distinct subregions of the rodent medial prefrontal cortex (mPFC) differentially affect fear behavior, whereby fear expression is promoted by the more dorsal prelimbic cortex (PL) and inhibited by the more ventral infralimbic cortex (IL). These mPFC regions project to subregions of the nucleus accumbens, the core (NAcC) and shell (NAcS), that differentially contribute to reward-seeking as well as affective processes that may be relevant to fear expression. Here, we investigated how these mPFC and NAc subregions contribute to discriminative fear conditioning, assessed by conditioned suppression of reward-seeking. Bilateral inactivation of the NAcS or PL reduced the expression of conditioned suppression to a shock-associated CS+, whereas NAcC inactivation reduced reward-seeking without affecting suppression. IL inactivation caused a general reduction in conditioned suppression following discriminative conditioning, but not when using a single-stimulus design. Pharmacological disconnection of the PL → NAcS pathway revealed that this projection mediates conditioned suppression. These data add to a growing literature implicating discrete cortico-striatal pathways in the suppression of reward-seeking in response to aversive stimuli. Dysfunction within related structures may contribute to aberrant patterns of behavior in psychiatric illnesses including substance use disorders.

Probing relationships between reinforcement learning and simple behavioral strategies to understand probabilistic reward learning

BACKGROUND

Reinforcement learning (RL) and win stay/lose shift model accounts of decision making are both widely used to describe how individuals learn about and interact with rewarding environments. Though mutually informative, these accounts are often conceptualized as independent processes and so the potential relationships between win stay/lose shift tendencies and RL parameters have not been explored.

NEW METHOD

We introduce a methodology to directly relate RL parameters to behavioral strategy. Specifically, by calculating a truncated multivariate normal distribution of RL parameters given win stay/lose shift tendencies from simulating these tendencies across the parameter space, we maximize the normal distribution for a given set of win stay/lose shift tendencies to approximate reinforcement learning parameters.

RESULTS

We demonstrate novel relationships between win stay/lose shift tendencies and RL parameters that challenge conventional interpretations of lose shift as a metric of loss sensitivity. Further, we demonstrate in both simulated and empirical data that this method of parameter approximation yields reliable parameter recovery.

COMPARISON WITH EXISTING METHOD

We compare this method against the conventionally used maximum likelihood estimation method for parameter approximation in simulated noisy and empirical data. For simulated noisy data, we show that this method performs similarly to maximum likelihood estimation. For empirical data, however, this method provides a more reliable approximation of reinforcement learning parameters than maximum likelihood estimation.

CONCLUSIONS

We demonstrate the existence of relationships between win stay/lose shift tendencies and RL parameters and introduce a method that leverages these relationships to enable recovery of RL parameters exclusively from win stay/lose shift tendencies.

Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models

Schizophrenia is characterized by positive, negative and cognitive symptoms. All current antipsychotic treatments feature dopamine-receptor antagonism that is relatively effective at addressing the psychotic (positive) symptoms of schizophrenia. However, there is no clear evidence that these medications improve the negative or cognitive symptoms, which are the greatest predictors of functional outcomes. One of the most robust pathophysiological observations in patients with schizophrenia is increased subcortical dopamine neurotransmission, primarily in the associative striatum. This brain area has an important role in a range of cognitive processes. Dopamine is also known to play a major part in regulating a number of cognitive functions impaired in schizophrenia but much of this research has been focused on cortical dopamine. Emerging research highlights the strong influence subcortical dopamine has on a range of cognitive domains, including attention, reward learning, goal-directed action and decision-making. Nonetheless, the precise role of the associative striatum in the cognitive impairments observed in schizophrenia remains poorly understood, presenting an opportunity to revisit its contribution to schizophrenia. Without a better understanding of the mechanisms underlying cognitive dysfunction, treatment development remains at a standstill. For this reason, improved preclinical animal models are needed if we are to understand the complex relationship between subcortical dopamine and cognition. A range of new techniques are facillitating the discrete manipulation of dopaminergic neurotransmission and measurements of cognitive performance, which can be investigated using a variety of sensitive translatable tasks. This has the potential to aid the successful incorporation of recent clinical research to address the lack of treatment strategies for cognitive symptoms in schizophrenia. This review will give an overview on the current state of research focused on subcortical dopamine and cognition in the context of schizophrenia research. We also discuss future strategies and approaches aimed at improving the translational outcomes for the treatment of cognitive deficits in schizophrenia.

Aberrations in Incentive Learning and Responding to Heroin in Male Rats After Adolescent or Adult Chronic Binge-Like Alcohol Exposure

BACKGROUND AND PURPOSE

Binge drinking is a serious problem among adolescents and young adults despite its adverse consequences on the brain and behavior. One area that remains poorly understood concerns the impact of chronic intermittent ethanol (CIE) exposure on incentive learning.

METHODS

Here, we examined the effects of CIE exposure during different developmental stages on conditioned approach and conditioned reward learning in rats experiencing acute or protracted withdrawal from alcohol. Two or 21 days after adolescent or adult CIE exposure, male rats were exposed to pairings of a light stimulus (CS) and food pellets for 3 consecutive daily sessions (30 CS-food pellet pairings per session). This was followed by conditioned approach testing measuring responses (food trough head entries) to the CS-only presentations and by conditioned reward testing measuring responses on a lever producing the CS and on another producing a tone. We then measured behavioral sensitization to repeated injections of heroin (2 mg/kg/d for 9 days).

RESULTS

Adolescent and adult alcohol-treated rats showed significantly impaired conditioned reward learning regardless of withdrawal period (acute or prolonged). We found no evidence of changes to conditioned approach learning after adolescent or adult exposure to CIE. Finally, in addition to producing long-term impairments in incentive learning, CIE exposure enhanced locomotor activity in response to heroin and had no effect on behavioral sensitization to heroin regardless of age and withdrawal period.

CONCLUSIONS

Our work sets a framework for identifying CIE-induced alterations in incentive learning and inducing susceptibility to subsequent opioid effects.

Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning

Impaired cognitive flexibility in visual reversal-learning tasks has been observed in a wide range of neurological and neuropsychiatric disorders. Although both human and animal studies have implicated striatal D2-like and D1-like receptors (D2R; D1R) in this form of flexibility, less is known about the contribution they make within distinct sub-regions of the striatum and the different phases of visual reversal learning. The present study investigated the involvement of D2R and D1R during the early (perseverative) phase of reversal learning as well as in the intermediate and late stages (new learning) after microinfusions of D2R and D1R antagonists into the nucleus accumbens core and shell (NAcC; NAcS), the anterior and posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS) on a touchscreen visual serial reversal-learning task. Reversal learning was improved after dopamine receptor blockade in the nucleus accumbens; the D1R antagonist, SCH23390, in the NAcS and the D2R antagonist, raclopride, in the NAcC selectively reduced early, perseverative errors. In contrast, reversal learning was impaired by D2R antagonism, but not D1R antagonism, in the dorsal striatum: raclopride increased errors in the intermediate phase after DMS infusions, and increased errors across phases after DLS infusions. These findings indicate that D1R and D2R modulate different stages of reversal learning through effects localised to different sub-regions of the striatum. Thus, deficits in behavioral flexibility observed in disorders linked to dopamine perturbations may be attributable to specific D1R and D2R dysfunction in distinct striatal sub-regions.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

Dissociable roles for the ventral and dorsal medial prefrontal cortex in cue-guided risk/reward decision making

Converging evidence from studies with animals and humans have implicated separate regions of the medial prefrontal cortex (mPFC) corresponding to the anterior cingulate cortex (ACC), in mediating different aspects of reward-related decisions involving uncertainty or risk. However, the dissociable contributions of subregions of the ACC remain unclear, as discrepancies exist between human neuroimaging findings and preclinical rodent studies. To clarify how ventral vs. dorsal regions of the mPFC contribute to risk/reward decision making, the present study assessed the effects of inactivation of different subregions on performance of a "Blackjack task" that measured cue-guided decision making and shares similarities with paradigms used with humans. Male, Long-Evans rats were well-trained to choose between a Small/Certain reward vs a Large/Risky reward delivered with variable probabilities (i.e., good vs. poor-odds, 50% vs. 12.5%). The odds of obtaining the larger reward was signaled by auditory cues at the start of each trial. Inactivation of the ventral, infralimbic region of the mPFC increased risky choice selectively when the odds of winning were poor. By contrast, inactivation of the prelimbic and anterior cingulate regions of the dorsal mPFC led to suboptimal reductions in risky choice on good-odds trials. The effects of prelimbic vs anterior cingulate inactivations were associated with context-dependent alterations in reward vs negative feedback, respectively. These results further clarify the distinct yet complementary manners in which separate ACC regions promote optimal risk/reward decision making and complement neuroimaging findings that activity in human ventral vs dorsal ACC promotes risk aversion or risky choices.

Unilateral accumbal dopamine depletion affects decision-making in a side-specific manner

Mechanisms underlying affective and cognitive deficits in Parkinson's disease (PD) remain less studied than motor symptoms. Nucleus accumbens (NAc) is affected in PD and due to its well-known involvement in motivation is an interesting target in this context. Furthermore, PD is frequently asymmetrical, with side-specific deficits aligning with evidences of accumbal laterality. We therefore used a 6-hydroxydopamine (6-OHDA) model to study the role of left and right NAc dopamine depletion in a battery of behavioral tasks. 2 months old male rats were used in all experiments. Habitual-based and goal-directed decision-making, impulsivity, anxiety- and depressive-like behavior and motor performance were tested 3 weeks after left (6-OHDA L) or right (6-OHDA R) NAc lesion was induced. Upon contingency degradation, 6-OHDA R decrease their lever press rate less than Sham and 6-OHDA L, indicating an impairment in the shift from habit-based to goal-directed strategies. On the other hand, 6-OHDA L lesions lead to increased rates of premature responding when delays where increased in the variable delay-to-signal test. Importantly, in both paradigms task acquisition was similar between groups. In the same line we found no differences in the amount of sugared pellets eaten when freely available as well as in both general and fine motor behaviors. In conclusion, left and right NAc play distinct roles in the contingency degradation and impulsivity. More studies are needed to understand the mechanisms behind this functional lateralization and its implications for PD patients.

Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice

The exploration/exploitation tradeoff – pursuing a known reward vs. sampling from lesser known options in the hope of finding a better payoff – is a fundamental aspect of learning and decision making. In humans, this has been studied using multi-armed bandit tasks. The same processes have also been studied using simplified probabilistic reversal learning (PRL) tasks with binary choices. Our investigations suggest that protocols previously used to explore PRL in mice may prove beyond their cognitive capacities, with animals performing at a no-better-than-chance level. We sought a novel probabilistic learning task to improve behavioral responding in mice, whilst allowing the investigation of the exploration/exploitation tradeoff in decision making. To achieve this, we developed a two-lever operant chamber task with levers corresponding to different probabilities (high/low) of receiving a saccharin reward, reversing the reward contingencies associated with levers once animals reached a threshold of 80% responding at the high rewarding lever. We found that, unlike in existing PRL tasks, mice are able to learn and behave near optimally with 80% high/20% low reward probabilities. Altering the reward contingencies towards equality showed that some mice displayed preference for the high rewarding lever with probabilities as close as 60% high/40% low. Additionally, we show that animal choice behavior can be effectively modelled using reinforcement learning (RL) models incorporating learning rates for positive and negative prediction error, a perseveration parameter, and a noise parameter. This new decision task, coupled with RL analyses, advances access to investigate the neuroscience of the exploration/exploitation tradeoff in decision making.

Persistent Alterations of Accumbal Cholinergic Interneurons and Cognitive Dysfunction after Adolescent Intermittent Ethanol Exposure

Adolescent binge drinking renders young drinkers vulnerable to alcohol use disorders in adulthood; therefore, understanding alcohol-induced brain damage and associated cognitive dysfunctions is of paramount importance. Here we investigated the effects of binge-like adolescent intermittent ethanol (AIE) exposure on nonspatial working memory, behavioral flexibility and cholinergic alterations in the nucleus accumbens (NAc) in male and female rats. On postnatal days P25-57 rats were intubated with water or ethanol (at a dose of 5 g/kg) on a 2-day-on/2-day-off cycle and were then tested in adulthood on social recognition and probabilistic reversal learning tasks. During the social recognition task AIE-treated rats spent similar amounts of time interacting with familiar and novel juveniles, indicating an impaired ability to sustain memory of the familiar juvenile. During probabilistic reversal learning, AIE-treated male and female rats showed behavioral inflexibility as indicated by a higher number of trials needed to complete three reversals within a session, longer response latencies for lever selection, and for males, a higher number of errors as compared to water-treated rats. AIE exposure also reduced the number of cholinergic interneurons in the NAc in males and females. These findings indicate AIE-related pathologies of accumbal cholinergic interneurons and long lasting cognitive-behavioral deficits, which may be associated with cortico-striatal hypofunction.

Oxytocin improves probabilistic reversal learning but not effortful motivation in Brown Norway rats

Deficits in cognition and motivation are common and debilitating aspects of psychiatric disorders, yet still go largely untreated. The neuropeptide oxytocin (OT) is a potential novel therapeutic for deficits in social cognition and motivation in psychiatric patients. However, the effects of OT on clinically relevant domains of non-social cognition and motivation remain under studied. The present study investigated the effects of acute and chronic (21-day) administration of subcutaneous OT (0.04, 0.2, and 1 mg/kg) in cross-species translatable operant paradigms of reward learning and effortful motivation in male and female Brown Norway (BN) rats (n = 8-10/group). Reward learning was assessed using the probabilistic reversal learning task (PRLT) and effortful motivation was measured using the progressive ratio breakpoint task (PRBT). As predicted, BN rats exhibited baseline deficits in the detection of reversals of reward contingency in the PRLT relative to Long Evans (LE) rats. The two strains performed equally in the PRBT. Thirty minutes after a single OT injection (1 mg/kg), measures of both initial probabilistic learning (trials to first criterion) and subsequent reversal learning (contingency switches) were significantly improved to levels comparable with LE rats. The OT effect on switches persisted in male, but not female, BN rats 30 min, 24 h, and 6 days after long-term OT administration, suggesting the induction of neuroplastic changes. OT did not affect effortful motivation at any time-point. The beneficial effects of OT on reward learning in the absence of increased effortful motivation support the development of OT as a novel therapeutic to improve cognitive functioning.

Sensitivity to negative and positive feedback as a stable and enduring behavioural trait in rats

RATIONALE

According to psychological theories, cognitive distortions play a pivotal role in the aetiology and recurrence of mood disorders. Although clinical evidence for the coexistence of depression and altered sensitivity to performance feedback is relatively coherent, we still do not know whether increased or decreased sensitivity to positive or negative feedback is associated with 'pro-depressive' profile in healthy subjects.

OBJECTIVE

Our research has been designed to answer this question, and here, we present the first steps in that direction.

METHODS

Using a rat version of the probabilistic reversal-learning (PRL) paradigm, we evaluated how sensitivity to negative and positive feedback influences other cognitive processes associated with mood disorders, such as motivation in the progressive ratio schedule of reinforcement (PRSR) paradigm, hedonic status in the sucrose preference (SP) test, locomotor and exploratory activity in the open field (OF) test, and anxiety in the light/dark box (LDB) test.

RESULTS

The results of our study demonstrated for the first time that in rodents, sensitivity to negative and positive feedback could be considered a stable and enduring behavioural trait. Importantly, we also showed that these traits are independent of each other and that trait sensitivity to positive feedback is associated with cognitive flexibility in the PRL test. The computational modelling results also revealed that in animals classified as sensitive to positive feedback, the α learning rates for both positive and negative reward prediction errors were higher than those in animals classified as insensitive. We observed no statistically significant interactions between sensitivity to negative or positive feedback and the parameters measured in the PRSR, SP, OF or LDB tests.

CONCLUSIONS

Further studies using animal models of depression based on chronic stress should reveal whether sensitivity to feedback is a latent trait that when interacts with stressful life events, could produce correlates of depressive symptoms in rats.

Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules

The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions.SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been associated with neuropsychiatric disorders, such as attention deficit hyperactivity disorder and schizophrenia, which in turn has been linked to aberrant processing in the nucleus accumbens. However, many preclinical studies have often assessed risk/reward decision making in the absence of explicit cues. The current study fills that gap by using a novel task that allows for the assessment of cue-guided risk/reward decision making in rodents. Our findings identified distinct yet complementary roles for the medial versus lateral portions of this nucleus that provide a broader understanding of the differential contributions it makes to decision making and reward seeking guided by discriminative stimuli.

Optogenetic Inhibition Reveals Distinct Roles for Basolateral Amygdala Activity at Discrete Time Points during Risky Decision Making

Decision making is a multifaceted process, consisting of several distinct phases that likely require different cognitive operations. Previous work showed that the basolateral amygdala (BLA) is a critical substrate for decision making involving risk of punishment; however, it is unclear how the BLA is recruited at different stages of the decision process. To this end, the current study used optogenetics to inhibit the BLA during specific task phases in a model of risky decision making (risky decision-making task) in which rats choose between a small, "safe" reward and a large reward accompanied by varying probabilities of footshock punishment. Male Long-Evans rats received intra-BLA microinjections of viral vectors carrying either halorhodopsin (eNpHR3.0-mCherry) or mCherry alone (control) followed by optic fiber implants and were trained in the risky decision-making task. Laser delivery during the task occurred during intertrial interval, deliberation, or reward outcome phases, the latter of which was further divided into the three possible outcomes (small, safe; large, unpunished; large, punished). Inhibition of the BLA selectively during the deliberation phase decreased choice of the large, risky outcome (decreased risky choice). In contrast, BLA inhibition selectively during delivery of the large, punished outcome increased risky choice. Inhibition had no effect during the other phases, nor did laser delivery affect performance in control rats. Collectively, these data indicate that the BLA can either inhibit or promote choice of risky options, depending on the phase of the decision process in which it is active.SIGNIFICANCE STATEMENT To date, most behavioral neuroscience research on neural mechanisms of decision making has used techniques that preclude assessment of distinct phases of the decision process. Here we show that optogenetic inhibition of the BLA has opposite effects on choice behavior in a rat model of risky decision making, depending on the phase in which inhibition occurs. BLA inhibition during a period of deliberation between small, safe and large, risky outcomes decreased risky choice. In contrast, BLA inhibition during receipt of the large, punished outcome increased risky choice. These findings highlight the importance of temporally targeted approaches to understand neural substrates underlying complex cognitive processes. More importantly, they reveal novel information about dynamic BLA modulation of risky choice.

Ventral striatum lesions do not affect reinforcement learning with deterministic outcomes on slow time scales

A large body of work has implicated the ventral striatum (VS) in aspects of reinforcement learning (RL). However, less work has directly examined the effects of lesions in the VS, or other forms of inactivation, on 2-armed bandit RL tasks. We have recently found that lesions in the VS in macaque monkeys affect learning with stochastic schedules but have minimal effects with deterministic schedules. The reasons for this are not currently clear. Because our previous work used short intertrial intervals, one possibility is that the animals were using working memory to bridge stimulus-reward associations from 1 trial to the next. In the present study, we examined learning of 60 pairs of objects, in which the animals received only 1 trial per day with each pair. The large number of object pairs and the long interval (approximately 24 hr) between trials with a given pair minimized the chances that the animals could use working memory to bridge trials. We found that monkeys with VS lesions were unimpaired relative to controls, which suggests that animals with VS lesions can still learn to select rewarded objects even when they cannot make use of working memory. (PsycINFO Database Record