Absence of cue-evoked firing in rat dorsolateral striatum neurons

Synchronization in the prefrontal-striatal circuit tracks behavioural choice in a go-no-go task in rats

Rodent striatum is involved in sensory-motor transformations and reward-related learning. Lesion studies suggest dorsolateral striatum, dorsomedial striatum and nucleus accumbens underlie stimulus-response transformations, goal-directed behaviour and reward expectation, respectively. In addition, prefrontal inputs likely control these functions. Here, we set out to study how reward-driven behaviour is mediated by the coordinated activity of these structures in the intact brain. We implemented a discrimination task requiring rats to either respond or suppress responding on a lever after the presentation of auditory cues in order to obtain rewards. Single unit activity in the striatal subregions and pre-limbic cortex was recorded using tetrode arrays. Striatal units showed strong onset responses to auditory cues paired with an opportunity to obtain reward. Cue-onset responses in both striatum and cortex were significantly modulated by previous errors suggesting a role of these structures in maintaining appropriate motivation or action selection during ongoing behaviour. Furthermore, failure to respond to the reward-paired tones was associated with higher pre-trial coherence among striatal subregions and between cortex and striatum suggesting a task-negative corticostriatal network whose activity may be suppressed to enable processing of reward-predictive cues. Our findings highlight that coordinated activity in a distributed network including both pre-limbic cortex and multiple striatal regions underlies reward-related decisions.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum

Evaluating outcomes of behavior is a central function of the striatum. In circuits engaging the dorsomedial striatum, sensitivity to goal value is accentuated during learning, whereas outcome sensitivity is thought to be minimal in the dorsolateral striatum and its habit-related corticostriatal circuits. However, a distinct population of projection neurons in the dorsolateral striatum exhibits selective sensitivity to rewards. Here, we evaluated the outcome-related signaling in such neurons as rats performed an instructional T-maze task for two rewards. As the rats formed maze-running habits and then changed behavior after reward devaluation, we detected outcome-related spike activity in 116 units out of 1,479 recorded units. During initial training, nearly equal numbers of these units fired preferentially either after rewarded runs or after unrewarded runs, and the majority were responsive at only one of two reward locations. With overtraining, as habits formed, firing in nonrewarded trials almost disappeared, and reward-specific firing declined. Thus error-related signaling was lost, and reward signaling became generalized. Following reward devaluation, in an extinction test, postgoal activity was nearly undetectable, despite accurate running. Strikingly, when rewards were then returned, postgoal activity reappeared and recapitulated the original early response pattern, with nearly equal numbers responding to rewarded and unrewarded runs and to single rewards. These findings demonstrate that outcome evaluation in the dorsolateral striatum is highly plastic and tracks stages of behavioral exploration and exploitation. These signals could be a new target for understanding compulsive behaviors that involve changes to dorsal striatum function.

Dorsolateral striatum is critical for the expression of surprise-induced enhancements in cue associability

The dorsolateral striatum (DLS) is frequently implicated in sensory-motor integration, including the performance of sensory orienting responses (ORs) and learned stimulus-response habits. Our laboratory previously identified a role for the DLS in rats' performance of conditioned ORs to Pavlovian cues for food delivery. Here, we considered whether DLS is also critical to another aspect of attention in associative learning, the surprise-induced enhancement of cue associability. A large behavioral literature shows that a cue present when an expected event is omitted enters into new associations more rapidly when that cue is subsequently paired with food. Research from our laboratory has shown that both cue associability enhancements and conditioned ORs depend on the function of a circuit that includes the amygdala central nucleus and the substantia nigra pars compacta. In three experiments, we explored the involvement of DLS in surprise-induced associability enhancements, using a three-stage serial prediction task that permitted separation of DLS function in registering surprise (prediction error) and enhancing cue associability, and in using that increased associability to learn more rapidly about that cue later. The results showed that DLS is critical to the expression, but not the establishment, of the enhanced cue associability normally produced by surprise in this task. They extend the role of DLS and the amygdalo-nigro-striatal circuit underlying learned orienting to more subtle aspects of attention in associative learning, but are consistent with the general notion that DLS is more important in the expression of previously acquired tendencies than in their acquisition.

Electrophysiological evidence of alterations to the nucleus accumbens and dorsolateral striatum during chronic cocaine self-administration

As drug use becomes chronic, aberrant striatal processing contributes to the development of perseverative drug-taking behaviors. Two particular portions of the striatum, the nucleus accumbens (NAc) and the dorsolateral striatum (DLS), are known to undergo neurobiological changes from acute to chronic drug use. However, little is known about the exact progression of changes in functional striatal processing as drug intake persists. We sampled single-unit activity in the NAc and DLS throughout 24 daily sessions of chronic long-access cocaine self-administration, and longitudinally tracked firing rates (FR) specifically during the operant response, an upward vertical head movement. A total of 103 neurons were held longitudinally and immunohistochemically localised to either NAc Medial Shell (n = 29), NAc Core (n = 30), or DLS (n = 54). We modeled changes representative of each category as a whole. Results demonstrated that FRs of DLS Head Movement neurons were significantly increased relative to baseline during all sessions, while FRs of DLS Uncategorised neurons were significantly reduced relative to baseline during all sessions. NAc Shell neurons' FRs were also significantly decreased relative to baseline during all sessions while FRs of NAc Core neurons were reduced relative to baseline only during training days 1-18 but were not significantly reduced on the remaining sessions (19-24). The data suggest that all striatal subregions show changes in FR during the operant response relative to baseline, but longitudinal changes in response firing patterns were observed only in the NAc Core, suggesting that this region is particularly susceptible to plastic changes induced by abused drugs.

Amygdalo-striatal interaction in the enhancement of stimulus salience in associative learning

Function of the central nucleus of the amygdala (CeA) is critical to 2 aspects of attention in associative learning: the conditioning of orienting responses (ORs) to cues paired with food, and the enhancement of cue salience by the surprising omission of expected events. Such salience enhancements have been found to depend on interactions within a circuit that includes CeA, the substantia nigra pars compacta (SNc), the substantia innominata (SI), and the posterior parietal cortex (PPC). The acquisition and expression of conditioned ORs requires interactions among CeA, SNc, and the dorsal lateral striatum (DLS), but not SI or PPC. Here, we considered whether CeA-DLS interactions are also important in surprise-induced salience enhancements in a serial prediction task. Rats received unilateral lesions of CeA and DLS, either contralaterally, which disrupted interactions between those structures, or ipsilaterally, which produced comparable damage to each structure but permitted interactions between them in 1 hemisphere. Rats with ipsilateral lesions of CeA and DLS showed the salience enhancements normally observed in this task, but rats with contralateral lesions of those structures did not. Thus, convergence of information processing by CeA and DLS is essential for surprise-induced salience enhancements, as well as for conditioned ORs.

Investigating habits: strategies, technologies and models

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying action-learning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look “under the hood” of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

A dual operator view of habitual behavior reflecting cortical and striatal dynamics

Habits are notoriously difficult to break and, if broken, are usually replaced by new routines. To examine the neural basis of these characteristics, we recorded spike activity in cortical and striatal habit sites as rats learned maze tasks. Overtraining induced a shift from purposeful to habitual behavior. This shift coincided with the activation of neuronal ensembles in the infralimbic neocortex and the sensorimotor striatum, which became engaged simultaneously but developed changes in spike activity with distinct time courses and stability. The striatum rapidly acquired an action-bracketing activity pattern insensitive to reward devaluation but sensitive to running automaticity. A similar pattern developed in the upper layers of the infralimbic cortex, but it formed only late during overtraining and closely tracked habit states. Selective optogenetic disruption of infralimbic activity during overtraining prevented habit formation. We suggest that learning-related spiking dynamics of both striatum and neocortex are necessary, as dual operators, for habit crystallization.

Evidence for learned skill during cocaine self-administration in rats

RATIONALE

It has been proposed that cocaine abuse results in skilled or "automatic" drug-taking behaviors. Brain regions important for skill learning are implicated in cocaine self-administration. However, the development of skill during self-administration has not been investigated.

OBJECTIVES

The present experiment investigated the development of skilled self-administration over extended drug use by employing a novel operant vertical head movement under discriminative stimulus (S(D)) control. In addition, the capacity of the head movement to serve as an operant was tested by manipulating drug levels above or below satiety drug levels via frequent noncontingent microinfusions (0.2 s) of cocaine.

RESULTS

Animals acquired the vertical head movement operant, which increased in number over days. Task learning was demonstrated by reduced reaction time in response to the S(D), increased propensity to self-administer upon S(D) presentation, and escalated drug consumption over days. Skill learning was demonstrated by (1) an increase over days in the velocity of operant movements, as a function of shorter duration but not altered distance, and (2) an increase over days in the probability of initiating the operant at the optimal starting position. Evidence that responding was specific to self-administration was revealed during periods of experimenter-manipulated drug level: maintaining drug levels above satiety decreased responding while maintaining drug levels below satiety increased responding.

CONCLUSIONS

Under the specific set of circumstances tested herein, cocaine self-administration became skilled over extended drug use. The vertical head movement can be used as an operant comparable to lever pressing with the additional benefit of quantifying skill learning.