Neural substrates of habit

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

A dual-pathway architecture enables chronic stress to promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions by promoting the formation of inflexible, potentially maladaptive, habits. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted approach in male and female mice, we reveal a dual pathway, amygdala-striatal, neuronal circuit architecture by which a recent history of chronic stress shapes learning to disrupt flexible goal-directed behavior in favor of inflexible habits. Chronic stress inhibits activity of basolateral amygdala projections to the dorsomedial striatum to impede the action-outcome learning that supports flexible, goal-directed decisions. Stress also increases activity in direct central amygdala projections to the dorsomedial striatum to promote the formation of rigid, inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to promote premature habit formation. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and other psychiatric conditions.

Estradiol Receptors Inhibit Long-Term Potentiation in the Dorsomedial Striatum

Estradiol, a female sex hormone and the predominant form of estrogen, has diverse effects throughout the brain including in learning and memory. Estradiol modulates several types of learning that depend on the dorsomedial striatum (DMS), a subregion of the basal ganglia involved in goal-directed learning, cued action-selection, and motor skills. A cellular basis of learning is synaptic plasticity, and the presence of extranuclear estradiol receptors ERα, ERβ, and G-protein-coupled estrogen receptor (GPER) throughout the DMS suggests that estradiol may influence rapid cellular actions including those involved in plasticity. To test whether estradiol affects synaptic plasticity in the DMS, corticostriatal long-term potentiation (LTP) was induced using theta-burst stimulation (TBS) in ex vivo brain slices from intact male and female C57BL/6 mice. Extracellular field recordings showed that female mice in the diestrous stage of the estrous cycle exhibited LTP similar to male mice, while female mice in estrus did not exhibit LTP. Furthermore, antagonists of ERα or GPER rescued LTP in estrous females and agonists of ERα or GPER reduced LTP in diestrous females. In males, activating ERα but not GPER reduced LTP. These results uncover an inhibitory action of estradiol receptors on cellular learning in the DMS and suggest a cellular mechanism underlying the impairment in certain types of DMS-based learning observed in the presence of high estradiol. Because of the dorsal striatum's role in substance use disorders, these findings may provide a mechanism underlying an estradiol-mediated progression from goal-directed to habitual drug use.

Glutamate dynamics in the dorsolateral striatum of rats with goal-directed and habitual cocaine-seeking behavior

The shift from drug abuse to addiction is considered to arise from the transition between goal-directed and habitual control over drug behavior. Habitual responding for appetitive and skill-based behaviors is mediated by potentiated glutamate signaling in the dorsolateral striatum (DLS), but the state of the DLS glutamate system in the context of habitual drug-behavior remains undefined. Evidence from the nucleus accumbens of cocaine-experienced rats suggests that decreased transporter-mediated glutamate clearance and enhanced synaptic glutamate release contribute to the potentiated glutamate signaling that underlies the enduring vulnerability to relapse. Preliminary evidence from the dorsal striatum of cocaine-experienced rats suggests that this region exhibits similar alterations to glutamate clearance and release, but it is not known whether these glutamate dynamics are associated with goal-directed or habitual control over cocaine-seeking behavior. Therefore, we trained rats to self-administer cocaine in a chained cocaine-seeking and -taking paradigm, which yielded goal-directed, intermediate, and habitual cocaine-seeking rats. We then assessed glutamate clearance and release dynamics in the DLS of these rats using two different methods: synaptic transporter current (STC) recordings of patch-clamped astrocytes and the intensity-based glutamate sensing fluorescent reporter (iGluSnFr). While we observed a decreased rate of glutamate clearance in STCs evoked with single-pulse stimulation in cocaine-experienced rats, we did not observe any cocaine-induced differences in glutamate clearance rates from STCs evoked with high frequency stimulation (HFS) or iGluSnFr responses evoked with either double-pulse stimulation or HFS. Furthermore, GLT-1 protein expression in the DLS was unchanged in cocaine-experienced rats, regardless of their mode of control over cocaine-seeking behavior. Lastly, there were no differences in metrics of glutamate release between cocaine-experienced rats and yoked-saline controls in either assay. Together, these results suggest that glutamate clearance and release dynamics in the DLS are largely unaltered by a history of cocaine self-administration on this established cocaine seeking-taking paradigm, regardless of whether the control over the cocaine seeking behavior was habitual or goal directed.

Sex differences in learning and performing the Go/NoGo tasks

BACKGROUND

The quality of learning and post-learning performances is critical for daily life. The behavioral flexibility is equally important for adapting the changing circumstances. The learning process requires repeated practices, which enhances prompt and proper behavioral responses, in turn, which promotes habits formation as well. Despite the well-documented sex differences in learning and performances, contradictory results were reported. A possible cause might be a systematic analysis due to specific research interests, regardless of the continuity of natural acquisition process. Here, we investigate the potential sex differences in learning, performances and adjustments of habited behaviors with regular and reversal Go/NoGo tasks.

METHODS

Both male and female Sprague-Dawley rats were used in this study. All rats were trained for a regular rodent Go/NoGo task and a subset of rats were trained for a reversal rodent Go/NoGo task, both with strict elimination criteria. The behavioral performance data were stored in PC for off-line analysis. Multiple behavioral indices were analyzed for both passed and retired rats.

RESULTS

The ability of learning the regular the reversal Go/NoGo tasks was similar for both male and female rats, however, the female rats took longer time to master the task principles in later stages for both tasks. In the regular Go/NoGo task, the female rats spent more time on completing the trial in performance optimization phases, which implied female rats were more cautious than male rats. Along with the progression of training, both male and female rats developed Go-preference strategies to perform the regular Go/NoGo task, which induced failure to meet the setting success criteria. The retired male rats exhibited shorter RTs and MTs than the retired female rats after developing Go-preference. Moreover, the time needed to complete the Go trials was significantly prolonged for male rats in the reversal Go/NoGo task.

CONCLUSIONS

Overall, we conclude that distinctive strategies were employed in performing Go/NoGo tasks for both male and female rats. Male rats required less time to stabilize the performance in behavioral optimization phase. In addition, male rats were more accurate in estimating time elapsing. In contrast, female rats took more cautious considerations in performing the task, through which minimal influences were manifested in the reversal version of task.

Cocaine and habit training cause dendritic spine rearrangement in the prelimbic cortex

Successfully navigating dynamic environments requires organisms to learn the consequences of their actions. The prelimbic prefrontal cortex (PL) formulates action-consequence memories and is modulated by addictive drugs like cocaine. We trained mice to obtain food rewards and then unexpectedly withheld reinforcement, triggering new action-consequence memory. New memory was disrupted by cocaine when delivered immediately following non-reinforcement, but not when delayed, suggesting that cocaine disrupted memory consolidation. Cocaine also rapidly inactivated cofilin, a primary regulator of the neuronal actin cytoskeleton. This observation led to the discovery that cocaine also within the time of memory consolidation elevated dendritic spine elimination and blunted spine formation rates on excitatory PL neurons, culminating in thin-type spine attrition. Training drug-naive mice to utilize inflexible response strategies also eliminated thin-type dendritic spines. Thus, cocaine may disrupt action-consequence memory, at least in part, by recapitulating neurobiological sequalae occurring in the formation of inflexible habits.

Medial septum activation improves strategy switching once strategies are well-learned via bidirectional regulation of dopamine neuron population activity

Strategy switching is a form of cognitive flexibility that requires inhibiting a previously successful strategy and switching to a new strategy of a different categorical modality. It is dependent on dopamine (DA) receptor activation and release in ventral striatum and prefrontal cortex, two primary targets of ventral tegmental area (VTA) DA projections. Although the circuitry that underlies strategy switching early in learning has been studied, few studies have examined it after extended discrimination training. This may be important as DA activity and release patterns change across learning, with several studies demonstrating a critical role for substantia nigra pars compacta (SNc) DA activity and release once behaviors are well-learned. We have demonstrated that medial septum (MS) activation simultaneously increased VTA and decreased SNc DA population activity, as well as improved reversal learning via these actions on DA activity. We hypothesized that MS activation would improve strategy switching both early in learning and after extended training through its ability to increase VTA DA population activity and decrease SNc DA population activity, respectively. We chemogenetically activated the MS of male and female rats and measured their performance on an operant-based strategy switching task following 1, 10, or 15 days of discrimination training. Contrary to our hypothesis, MS activation did not affect strategy switching after 1 day of discrimination training. MS activation improved strategy switching after 10 days of training, but only in females. MS activation improved strategy switching in both sexes after 15 days of training. Infusion of bicuculline into the ventral subiculum (vSub) inhibited the MS-mediated decrease in SNc DA population activity and attenuated the improvement in strategy switching. Intra-vSub infusion of scopolamine inhibited the MS-mediated increase in VTA DA population activity but did not affect the improvement in strategy switching. Intra-vSub infusion of both bicuculline and scopolamine inhibited the MS-mediated effects on DA population activity in both the SNc and VTA and completely prevented the improvement in strategy switching. These data indicate that MS activation improves strategy switching once the original strategy has been sufficiently well-learned, and that this may occur via the MS's regulation of DA neuron responsivity.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.