Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection

Dynamic responses of striatal cholinergic interneurons control behavioral flexibility

Striatal cholinergic interneurons (CINs) are key to regulating behavioral flexibility, involving both extinguishing learned actions and adopting new ones. However, the mechanisms driving these processes remain elusive. In this study, we initially demonstrate that chronic alcohol consumption disrupts the burst-pause dynamics of CINs and impairs behavioral flexibility. We next aimed to elucidate the mechanisms by which CIN dynamics control behavioral flexibility. We found that extinction learning enhances acetylcholine (ACh) release and that mimicking this enhancement through optogenetic induction of CIN burst firing accelerates the extinction process. In addition, we demonstrate that disrupting CIN pauses via continuous optogenetic stimulation reversibly impairs the updating of goal-directed behaviors. Overall, we demonstrate that CIN burst firing, which increases ACh release, promotes extinction learning, aiding the extinguishment of learned behaviors. Conversely, CIN firing pauses, which lead to ACh dips, are crucial for reversal learning, facilitating the adaptation of new actions. These findings shed light on how CIN dynamics regulate behavioral flexibility.

In relentless pursuit of the white whale: A role for the ventral midline thalamus in behavioral flexibility and adaption?

The reuniens (Re) nucleus is located in the ventral midline thalamus. It has fostered increasing interest, not only for its participation in a variety of cognitive functions (e.g., spatial working memory, systemic consolidation, reconsolidation, extinction of fear or generalization), but also for its neuroanatomical positioning as a bidirectional relay between the prefrontal cortex (PFC) and the hippocampus (HIP). In this review we compile and discuss recent studies having tackled a possible implication of the Re nucleus in behavioral flexibility, a major PFC-dependent executive function controlling goal-directed behaviors. Experiments considered explored a possible role for the Re nucleus in perseveration, reversal learning, fear extinction, and set-shifting. They point to a contribution of this nucleus to behavioral flexibility, mainly by its connections with the PFC, but possibly also by those with the hippocampus, and even with the amygdala, at least for fear-related behavior. As such, the Re nucleus could be a crucial crossroad supporting a PFC-orchestrated ability to cope with new, potentially unpredictable environmental contingencies, and thus behavioral flexibility and adaption.

Corticotropin releasing factor alters the functional diversity of accumbal cholinergic interneurons

Cholinergic interneurons (ChIs) provide the main source of acetylcholine in the striatum and have emerged as a critical modulator of behavioral flexibility, motivation, and associative learning. In the dorsal striatum, ChIs display heterogeneous firing patterns. Here, we investigated the spontaneous firing patterns of ChIs in the nucleus accumbens (NAc) shell, a region of the ventral striatum. We identified four distinct ChI firing signatures: regular single-spiking, irregular single-spiking, rhythmic bursting, and a mixed-mode pattern composed of bursting activity and regular single spiking. ChIs from females had lower firing rates compared to males and had both a higher proportion of mixed-mode firing patterns and a lower proportion of regular single-spiking neurons compared to males. We further observed that across the estrous cycle, the diestrus phase was characterized by higher proportions of irregular ChI firing patterns compared to other phases. Using pooled data from males and females, we examined how the stress-associated neuropeptide corticotropin releasing factor (CRF) impacts these firing patterns. ChI firing patterns showed differential sensitivity to CRF. This translated into differential ChI sensitivity to CRF across the estrous cycle. Furthermore, CRF shifted the proportion of ChI firing patterns toward more regular spiking activity over bursting patterns. Finally, we found that repeated stressor exposure altered ChI firing patterns and sensitivity to CRF in the NAc core, but not the NAc shell. These findings highlight the heterogeneous nature of ChI firing patterns, which may have implications for accumbal-dependent motivated behaviors.

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Neural circuit models for evidence accumulation through choice-selective sequences

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.

Development and cadherin-mediated control of prefrontal corticostriatal projections in mice

Action-outcome associations depend on prefrontal cortex (PFC) projections to the dorsal striatum. To assess how these projections form, we measured PFC axon patterning, synapse formation, and functional maturation in the postnatally developing mouse striatum. Using Hotspot analysis, we show that PFC axons form an adult-like pattern of clustered terminations in the first postnatal week that remains largely stable thereafter. PFC-striatal synaptic strength is adult-like by P21, while excitatory synapse density increases until adulthood. We then tested how the targeted deletion of a candidate adhesion/guidance protein, Cadherin-8 (Cdh8), from corticostriatal neurons regulates pathway development. Mutant mice showed diminished PFC axon targeting and reduced spontaneous glutamatergic synaptic activity in the dorsal striatum. They also exhibited impaired behavioral performance in action-outcome learning. The data show that PFC-striatal axons form striatal territories through an early, directed growth model and they highlight essential contributions of Cdh8 to the anatomical and functional features critical for the formation of action-outcome associations.

Immp2l knockdown in male mice increases stimulus-driven instrumental behaviour but does not alter goal-directed learning or neuron density in cortico-striatal circuits in a model of Tourette syndrome and autism spectrum disorder

Cortico-striatal neurocircuits mediate goal-directed and habitual actions which are necessary for adaptive behaviour. It has recently been proposed that some of the core symptoms of autism spectrum disorder (ASD) and Gilles de la Tourette syndrome (GTS), such as tics and other repetitive behaviours, may emerge because of imbalances in these neurocircuits. We have recently developed a model of ASD and GTS by knocking down Immp2l, a mitochondrial gene frequently associated with these disorders. The current study sought to determine whether Immp2l knockdown (KD) in male mice alters flexible, goal- or cue- driven behaviour using procedures specifically designed to examine response-outcome and stimulus-response associations, which underlie goal-directed and habitual behaviour, respectively. Whether Immp2l KD alters neuron density in cortico-striatal neurocircuits known to regulate these behaviours was also examined. Immp2l KD mice and wild type-like mice (WT) were trained on Pavlovian and instrumental learning procedures where auditory cues predicted food delivery and lever-press responses earned a food outcome. It was demonstrated that goal-directed learning was not changed for Immp2l KD mice compared to WT mice, as lever-press responses were sensitive to changes in the value of the food outcome, and to contingency reversal and degradation. There was also no difference in the capacity of KD mice to form habitual behaviours compared to WT mice following extending training of the instrumental action. However, Immp2l KD mice were more responsive to auditory stimuli paired with food as indicated by a non-specific increase in lever response rates during Pavlovian-to-instrumental transfer. Finally, there were no alterations to neuron density in striatum or any prefrontal cortex or limbic brain structures examined. Thus, the current study suggests that Immp2l is not necessary for learned maladaptive goal or stimulus driven behaviours in ASD or GTS, but that it may contribute to increased capacity for external stimuli to drive behaviour. Alterations to stimulus-driven behaviour could potentially influence the expression of tics and repetitive behaviours, suggesting that genetic alterations to Immp2l may contribute to these core symptoms in ASD and GTS. Given that this is the first application of this battery of instrumental learning procedures to a mouse model of ASD or GTS, it is an important initial step in determining the contribution of known risk-genes to goal-directed versus habitual behaviours, which should be more broadly applied to other rodent models of ASD and GTS in the future.

The role of the parafascicular thalamic nucleus in action initiation and steering

The parafascicular (Pf) nucleus of the thalamus has been implicated in arousal and attention, but its contributions to behavior remain poorly characterized. Here, using in vivo and in vitro electrophysiology, optogenetics, and 3D motion capture, we studied the role of the Pf nucleus in behavior using a continuous reward-tracking task in freely moving mice. We found that many Pf neurons precisely represent vector components of velocity, with a strong preference for ipsiversive movements. Their activity usually leads velocity, suggesting that Pf output is critical for self-initiated orienting behavior. To test this hypothesis, we expressed excitatory or inhibitory opsins in VGlut2+ Pf neurons to manipulate neural activity bidirectionally. We found that selective optogenetic stimulation of these neurons consistently produced ipsiversive head turning, whereas inhibition stopped turning and produced downward movements. Taken together, our results suggest that the Pf nucleus can send continuous top-down commands that specify detailed action parameters (e.g., direction and speed of the head), thus providing guidance for orienting and steering during behavior.

Cognitive effects of thalamostriatal degeneration are ameliorated by normalizing striatal cholinergic activity

The loss of neurons in parafascicular thalamus (Pf) and their inputs to dorsomedial striatum (DMS) in Lewy body disease (LBD) and Parkinson's disease dementia (PDD) have been linked to the effects of neuroinflammation. We found that, in rats, these inputs were necessary for both the function of striatal cholinergic interneurons (CINs) and the flexible encoding of the action-outcome (AO) associations necessary for goal-directed action, producing a burst-pause pattern of CIN firing but only during the remapping elicited by a shift in AO contingency. Neuroinflammation in the Pf abolished these changes in CIN activity and goal-directed control after the shift in contingency. However, both effects were rescued by either the peripheral or the intra-DMS administration of selegiline, a monoamine oxidase B inhibitor that we found also enhances adenosine triphosphatase activity in CINs. These findings suggest a potential treatment for the cognitive deficits associated with neuroinflammation affecting the function of the Pf and related structures.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Long-term effects of THC exposure on reward learning and motivated behavior in adolescent and adult male rats

RATIONALE

The endocannabinoid system makes critical contributions to reward processing, motivation, and behavioral control. Repeated exposure to THC or other cannabinoid drugs can cause persistent adaptions in the endocannabinoid system and associated neural circuitry. It remains unclear how such treatments affect the way rewards are processed and pursued.

OBJECTIVE AND METHODS

We examined if repeated THC exposure (5 mg/kg/day for 14 days) during adolescence or adulthood led to long-term changes in rats' capacity to flexibly encode and use action-outcome associations for goal-directed decision making. Effects on hedonic feeding and progressive ratio responding were also assessed.

RESULTS

THC exposure had no effect on rats' ability to flexibly select actions following reward devaluation. However, instrumental contingency degradation learning, which involves avoiding an action that is unnecessary for reward delivery, was augmented in rats with a history of adult but not adolescent THC exposure. THC-exposed rats also displayed more vigorous instrumental behavior in this study, suggesting a motivational enhancement. A separate experiment found that while THC exposure had no effect on hedonic feeding behavior, it increased rats' willingness to work for food on a progressive ratio schedule, an effect that was more pronounced when THC was administered to adults. Adolescent and adult THC exposure had opposing effects on the CB1 receptor dependence of progressive ratio performance, decreasing and increasing sensitivity to rimonabant-induced behavioral suppression, respectively.

CONCLUSIONS

Our findings reveal that exposure to a translationally relevant THC exposure regimen induces long-lasting, age-dependent alterations in cognitive and motivational processes that regulate the pursuit of rewards.

Inhibition of noradrenergic signalling in rodent orbitofrontal cortex impairs the updating of goal-directed actions

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.

Influence of Rat Central Thalamic Neurons on Foraging Behavior in a Hazardous Environment

Foraging entails a complex balance between approach and avoidance alongside sensorimotor and homeostatic processes under the control of multiple cortical and subcortical areas. Recently, it has become clear that several thalamic nuclei located near the midline regulate motivated behaviors. However, one midline thalamic nucleus that projects to key nodes in the foraging network, the central medial thalamic nucleus (CMT), has received little attention so far. Therefore, the present study examined CMT contributions to foraging behavior using inactivation and unit recording techniques in male rats. Inactivation of CMT or the basolateral amygdala (BLA) with muscimol abolished the normally cautious behavior of rats in the foraging task. Moreover, CMT neurons showed large but heterogeneous activity changes during the foraging task, with many neurons decreasing or increasing their discharge rates, with a modest bias for the latter. A generalized linear model revealed that the nature (inhibitory vs excitatory) and relative magnitude of the activity modulations seen in CMT neurons differed markedly from those of principal BLA cells but were very similar to those of fast-spiking BLA interneurons. Together, these findings suggest that CMT is an important regulator of foraging behavior. In the Discussion, we consider how CMT is integrated into the network of structures that regulate foraging.SIGNIFICANCE STATEMENT Foraging entails a complex balance between approach and avoidance alongside sensorimotor and homeostatic processes under the control of multiple cortical and subcortical areas. Although the central medial thalamic nucleus (CMT) is connected to many nodes in this network, its role in the regulation of foraging behavior has not been investigated so far. Here, we examined CMT contributions to foraging behavior using inactivation and unit recording techniques. We found that CMT inactivation abolishes the normally cautious foraging behavior of rats and that CMT neurons show large but heterogeneous changes in firing rates during the foraging task. Together, these results suggest that CMT is an important regulator of foraging behavior.

Dissociable roles for the striatal cholinergic system in different flexibility contexts

The production of behavioural flexibility requires the coordination and integration of information from across the brain, by the dorsal striatum. In particular, the striatal cholinergic system is thought to be important for the modulation of striatal activity. Research from animal literature has shown that chemical inactivation of the dorsal striatum leads to impairments in reversal learning. Furthermore, proton magnetic resonance spectroscopy work has shown that the striatal cholinergic system is also important for reversal learning in humans. Here, we aim to assess whether the state of the dorsal striatal cholinergic system at rest is related to serial reversal learning in humans. We provide preliminary results showing that variability in choline in the dorsal striatum is significantly related to both the number of perseverative and regressive errors that participants make, and their rate of learning from positive and negative prediction errors. These findings, in line with previous work, suggest the resting state of dorsal striatal cholinergic system has important implications for producing flexible behaviour. However, these results also suggest the system may have heterogeneous functionality across different types of tasks measuring behavioural flexibility. These findings provide a starting point for further interrogation into understanding the functional role of the striatal cholinergic system in flexibility.

•

Striatal acetylcholine is important for behavioural flexibility in rodents & primates.

•

Nascent evidence the striatal cholinergic system is important for human flexibility.

•

¹H-MRS, reversal learning and reinforcement learning used to interrogate relationship.

•

Striatal cholinergic system at rest is associated with direct and latent performance.

•

Results specific to concentrations of striatal choline, and not other metabolites.

Opposing Roles of the Dorsolateral and Dorsomedial Striatum in the Acquisition of Skilled Action Sequencing in Rats

The shift in control from dorsomedial to dorsolateral striatum during skill and habit formation has been well established, but whether striatal subregions orchestrate this shift cooperatively or competitively remains unclear. Cortical inputs have also been implicated in the shift toward automaticity, but it is unknown whether they mirror their downstream striatal targets across this transition. We addressed these questions using a five step heterogeneous action sequencing task in male rats that is optimally performed by automated chains of actions. By optimizing automatic habitual responding, we discovered that loss of function in the dorsomedial striatum accelerated sequence acquisition. In contrast, loss of function in the dorsolateral striatum impeded acquisition of sequencing, demonstrating functional opposition within the striatum. Unexpectedly, the mPFC was not involved; however, the lateral orbitofrontal cortex was critical. These results shift current theories about striatal control of behavior to a model of competitive opposition, where the dorsomedial striatum interferes with the development of dorsolateral-striatum dependent behavior.SIGNIFICANCE STATEMENT We provide the most direct evidence to date that the dorsomedial and dorsolateral striatum compete for control in the acquisition of habitual action sequences. The dorsolateral striatum was critical for sequencing behavior, but loss of dorsomedial striatum function enhanced acquisition. In addition, we found that the mPFC was not required for the formation of automated actions. Using a task that optimizes habitual responding, we demonstrate that the arbitration of dorsomedial and dorsolateral control is not modulated by medial prefrontal cortical activity. However, we find evidence for the role of the lateral orbitofrontal cortex in action sequencing. These results have implications for our understanding of how habits and skills form.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

CRF-receptor1 modulation of the dopamine projection to prelimbic cortex facilitates cognitive flexibility after acute and chronic stress

Stress reduces cognitive flexibility and dopamine D1 receptor-related activity in the prelimbic cortex (PL), effects hypothesized to depend on reduced corticotropic releasing factor receptor type 1 (CRFr1) regulation of dopamine neurons in the ventral tegmental area (VTA). We assessed this hypothesis in rats by examining the effect of chronic unpredictable restraint stress (CUS), mild acute stress, or their combination on cognitive flexibility, CRFr1 expression in the VTA and D1-related activity in PL. In Experiment 1, rats received either CUS or equivalent handling for 14 days before being trained to press two levers to earn distinct food outcomes. Initial learning was assessed using an outcome devaluation test after which cognitive flexibility was assessed by reversing the outcomes earned by the actions. Prior to each reversal training session, half the CUS and controls receiving acute stress with action-outcome updating assessed using a second devaluation test and CRFr1 expression in the VTA assessed using in-situ hybridisation. Although CUS did not itself affect action-outcome learning, its combination with acute stress blocked reversal learning and decreased VTA CRFr1 expression after acute shock. The relationship between these latter two effects was assessed in Experiment 2 by pharmacologically disconnecting the VTA and PL, unilaterally blocking neurons expressing CRFr1 in the VTA and D1 receptors in the contralateral PL during reversal learning after acute stress. Acute stress again blocked reversal learning but only in the group with VTA-PL disconnection, demonstrating that VTA CRFr1-induced facilitation of dopaminergic activity in the PL is necessary for maintaining cognitive flexibility after acute stress. [250].

•

Acute stress increased CRF receptor1 expression in the VTA.

•

Chronic stress attenuated the effect of acute stress on CRFr1 expression.

•

Chronic stress plus acute stress produced a loss of cognitive flexibility.

•

Blocking VTA CFRr1 and dopamine D1r in PL reduced cognitive flexibility following stress.

Chronic alcohol drinking persistently suppresses thalamostriatal excitation of cholinergic neurons to impair cognitive flexibility

Exposure to addictive substances impairs flexible decision making. Cognitive flexibility is mediated by striatal cholinergic interneurons (CINs). However, how chronic alcohol drinking alters cognitive flexibility through CINs remains unclear. Here, we report that chronic alcohol consumption and withdrawal impaired reversal of instrumental learning. Chronic alcohol consumption and withdrawal also caused a long-lasting (21 days) reduction of excitatory thalamic inputs onto CINs and reduced pause responses of CINs in the dorsomedial striatum (DMS). CINs are known to inhibit glutamatergic transmission in dopamine D1 receptor–expressing medium spiny neurons (D1-MSNs) but facilitate this transmission in D2-MSNs, which may contribute to flexible behavior. We discovered that chronic alcohol drinking impaired CIN-mediated inhibition in D1-MSNs and facilitation in D2-MSNs. Importantly, in vivo optogenetic induction of long-term potentiation of thalamostriatal transmission in DMS CINs rescued alcohol-induced reversal learning deficits. These results demonstrate that chronic alcohol drinking reduces thalamic excitation of DMS CINs, compromising their regulation of glutamatergic transmission in MSNs, which may contribute to alcohol-induced impairment of cognitive flexibility. These findings provide a neural mechanism underlying inflexible drinking in alcohol use disorder.

Parafascicular Thalamic and Orbitofrontal Cortical Inputs to Striatum Represent States for Goal-Directed Action Selection

Several lines of evidence accrued over the last 5–10 years have converged to suggest that the parafascicular nucleus of the thalamus and the lateral orbitofrontal cortex each represent or contribute to internal state/context representations that guide action selection in partially observable task situations. In rodents, inactivations of each structure have been found to selectively impair performance in paradigms testing goal-directed action selection, but only when that action selection relies on state representations. Electrophysiological evidence has suggested that each structure achieves this function via inputs onto cholinergic interneurons (CINs) in the dorsomedial striatum. Here, we briefly review these studies, then point to anatomical evidence regarding the afferents of each structure and what they suggest about the specific features that each contribute to internal state representations. Finally, we speculate as to whether this role might be achieved interdependently through direct PF→OFC projections, or through the convergence of independent direct orbitofrontal cortex (OFC) and parafascicular nucleus of the thalamus (PF) inputs onto striatal targets.

Orbitofrontal State Representations Are Related to Choice Adaptations and Reward Predictions

Animals can categorize the environment into "states," defined by unique sets of available action-outcome contingencies in different contexts. Doing so helps them choose appropriate actions and make accurate outcome predictions when in each given state. State maps have been hypothesized to be held in the orbitofrontal cortex (OFC), an area implicated in decision-making and encoding information about outcome predictions. Here we recorded neural activity in OFC in 6 male rats to test state representations. Rats were trained on an odor-guided choice task consisting of five trial blocks containing distinct sets of action-outcome contingencies, constituting states, with unsignaled transitions between them. OFC neural ensembles were analyzed using decoding algorithms. Results indicate that the vast majority of OFC neurons contributed to representations of the current state at any point in time, independent of odor cues and reward delivery, even at the level of individual neurons. Across state transitions, these representations gradually integrated evidence for the new state; the rate at which this integration happened in the prechoice part of the trial was related to how quickly the rats' choices adapted to the new state. Finally, OFC representations of outcome predictions, often thought to be the primary function of OFC, were dependent on the accuracy of OFC state representations.SIGNIFICANCE STATEMENT A prominent hypothesis proposes that orbitofrontal cortex (OFC) tracks current location in a "cognitive map" of state space. Here we tested this idea in detail by analyzing neural activity recorded in OFC of rats performing a task consisting of a series of states, each defined by a set of available action-outcome contingencies. Results show that most OFC neurons contribute to state representations and that these representations are related to the rats' decision-making and OFC reward predictions. These findings suggest new interpretations of emotional dysregulation in pathologies, such as addiction, which have long been known to be related to OFC dysfunction.

The Contribution of Thalamic Nuclei in Salience Processing

The brain continuously receives diverse information about the external environment and changes in the homeostatic state. The attribution of salience determines which stimuli capture attention and, therefore, plays an essential role in regulating emotions and guiding behaviors. Although the thalamus is included in the salience network, the neural mechanism of how the thalamus contributes to salience processing remains elusive. In this mini-review, we will focus on recent advances in understanding the specific roles of distinct thalamic nuclei in salience processing. We will summarize the functional connections between thalamus nuclei and other key nodes in the salience network. We will highlight the convergence of neural circuits involved in reward and pain processing, arousal, and attention control in thalamic structures. We will discuss how thalamic activities represent salience information in associative learning and how thalamic neurons modulate adaptive behaviors. Lastly, we will review recent studies which investigate the contribution of thalamic dysfunction to aberrant salience processing in neuropsychiatric disorders, such as drug addiction, posttraumatic stress disorder (PTSD), and schizophrenia. Based on emerging evidence from both human and rodent research, we propose that the thalamus, different from previous studies that as an information relay, has a broader role in coordinating the cognitive process and regulating emotions.

Adaptive behaviour under conflict: Deconstructing extinction, reversal, and active avoidance learning

In complex environments, organisms must respond adaptively to situations despite conflicting information. Under natural (i.e. non-laboratory) circumstances, it is rare that cues or responses are consistently paired with a single outcome. Inconsistent pairings are more common, as are situations where cues and responses are associated with multiple outcomes. Such inconsistency creates conflict, and a response that is adaptive in one scenario may not be adaptive in another. Learning to adjust responses accordingly is important for species to survive and prosper. Here we review the behavioural and brain mechanisms of responding under conflict by focusing on three popular behavioural procedures: extinction, reversal learning, and active avoidance. Extinction involves adapting from reinforcement to non-reinforcement, reversal learning involves swapping the reinforcement of cues or responses, and active avoidance involves performing a response to avoid an aversive outcome, which may conflict with other defensive strategies. We note that each of these phenomena relies on somewhat overlapping neural circuits, suggesting that such circuits may be critical for the general ability to respond appropriately under conflict.

Cholinergic midbrain afferents modulate striatal circuits and shape encoding of action strategies

Assimilation of novel strategies into a consolidated action repertoire is a crucial function for behavioral adaptation and cognitive flexibility. Acetylcholine in the striatum plays a pivotal role in such adaptation, and its release has been causally associated with the activity of cholinergic interneurons. Here we show that the midbrain, a previously unknown source of acetylcholine in the striatum, is a major contributor to cholinergic transmission in the striatal complex. Neurons of the pedunculopontine and laterodorsal tegmental nuclei synapse with striatal cholinergic interneurons and give rise to excitatory responses. Furthermore, they produce uniform inhibition of spiny projection neurons. Inhibition of acetylcholine release from midbrain terminals in the striatum impairs the association of contingencies and the formation of habits in an instrumental task, and mimics the effects observed following inhibition of acetylcholine release from striatal cholinergic interneurons. These results suggest the existence of two hierarchically-organized modes of cholinergic transmission in the striatum, where cholinergic interneurons are modulated by cholinergic neurons of the midbrain.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Concurrent Thalamostriatal and Corticostriatal Spike-Timing-Dependent Plasticity and Heterosynaptic Interactions Shape Striatal Plasticity Map

The striatum integrates inputs from the cortex and thalamus, which display concomitant or sequential activity. The striatum assists in forming memory, with acquisition of the behavioral repertoire being associated with corticostriatal (CS) plasticity. The literature has mainly focused on that CS plasticity, and little remains known about thalamostriatal (TS) plasticity rules or CS and TS plasticity interactions. We undertook here the study of these plasticity rules. We found bidirectional Hebbian and anti-Hebbian spike-timing-dependent plasticity (STDP) at the thalamic and cortical inputs, respectively, which were driving concurrent changes at the striatal synapses. Moreover, TS- and CS-STDP induced heterosynaptic plasticity. We developed a calcium-based mathematical model of the coupled TS and CS plasticity, and simulations predict complex changes in the CS and TS plasticity maps depending on the precise cortex–thalamus–striatum engram. These predictions were experimentally validated using triplet-based STDP stimulations, which revealed the significant remodeling of the CS-STDP map upon TS activity, which is notably the induction of the LTD areas in the CS-STDP for specific timing regimes. TS-STDP exerts a greater influence on CS plasticity than CS-STDP on TS plasticity. These findings highlight the major impact of precise timing in cortical and thalamic activity for the memory engram of striatal synapses.

Thalamostriatal degeneration contributes to dystonia and cholinergic interneuron dysfunction in a mouse model of Huntington's disease

Huntington’s disease (HD) is an autosomal dominant trinucleotide repeat disorder characterized by choreiform movements, dystonia and striatal neuronal loss. Amongst multiple cellular processes, abnormal neurotransmitter signalling and decreased trophic support from glutamatergic cortical afferents are major mechanisms underlying striatal degeneration. Recent work suggests that the thalamostriatal (TS) system, another major source of glutamatergic input, is abnormal in HD although its phenotypical significance is unknown. We hypothesized that TS dysfunction plays an important role in generating motor symptoms and contributes to degeneration of striatal neuronal subtypes. Our results using the R6/2 mouse model of HD indicate that neurons of the parafascicular nucleus (PF), the main source of TS afferents, degenerate at an early stage. PF lesions performed prior to motor dysfunction or striatal degeneration result in an accelerated dystonic phenotype and are associated with premature loss of cholinergic interneurons. The progressive loss of striatal medium spiny neurons and parvalbumin-positive interneurons observed in R6/2 mice is unaltered by PF lesions. Early striatal cholinergic ablation using a mitochondrial immunotoxin provides evidence for increased cholinergic vulnerability to cellular energy failure in R6/2 mice, and worsens the dystonic phenotype. The TS system therefore contributes to trophic support of striatal interneuron subtypes in the presence of neurodegenerative stress, and TS deafferentation may be a novel cell non-autonomous mechanism contributing to the pathogenesis of HD. Furthermore, behavioural experiments demonstrate that the TS system and striatal cholinergic interneurons are key motor-network structures involved in the pathogenesis of dystonia. This work suggests that treatments aimed at rescuing the TS system may preserve important elements of striatal structure and function and provide symptomatic relief in HD.

Striatal glutamate delta-1 receptor regulates behavioral flexibility and thalamostriatal connectivity

Impaired behavioral flexibility and repetitive behavior is a common phenotype in autism and other neuropsychiatric disorders, but the underlying synaptic mechanisms are poorly understood. The trans-synaptic glutamate delta (GluD)-Cerebellin 1-Neurexin complex, critical for synapse formation/maintenance, represents a vulnerable axis for neuropsychiatric diseases. We have previously found that GluD1 deletion results in reversal learning deficit and repetitive behavior. In this study, we show that selective ablation of GluD1 from the dorsal striatum impairs behavioral flexibility in a water T-maze task. We further found that striatal GluD1 is preferentially found in dendritic shafts, and more frequently associated with thalamic than cortical glutamatergic terminals suggesting localization to projections from the thalamic parafascicular nucleus (Pf). Conditional deletion of GluD1 from the striatum led to a selective loss of thalamic, but not cortical, terminals, and reduced glutamatergic neurotransmission. Optogenetic studies demonstrated functional changes at thalamostriatal synapses from the Pf, but no effect on the corticostriatal system, upon ablation of GluD1 in the dorsal striatum. These studies suggest a novel molecular mechanism by which genetic variations associated with neuropsychiatric disorders may impair behavioral flexibility, and reveal a unique principle by which GluD1 subunit regulates forebrain circuits.

Thalamic degeneration in MPTP-treated Parkinsonian monkeys: impact upon glutamatergic innervation of striatal cholinergic interneurons

In both Parkinson's disease (PD) patients and MPTP-treated non-human primates, there is a profound neuronal degeneration of the intralaminar centromedian/parafascicular (CM/Pf) thalamic complex. Although this thalamic pathology has long been established in PD (and other neurodegenerative disorders), the impact of CM/Pf cell loss on the integrity of the thalamo-striatal glutamatergic system and its regulatory functions upon striatal neurons remain unknown. In the striatum, cholinergic interneurons (ChIs) are important constituents of the striatal microcircuitry and represent one of the main targets of CM/Pf-striatal projections. Using light and electron microscopy approaches, we have analyzed the potential impact of CM/Pf neuronal loss on the anatomy of the synaptic connections between thalamic terminals (vGluT2-positive) and ChIs neurons in the striatum of parkinsonian monkeys treated chronically with MPTP. The following conclusions can be drawn from our observations: (1) as reported in PD patients, and in our previous monkey study, CM/Pf neurons undergo profound degeneration in monkeys chronically treated with low doses of MPTP. (2) In the caudate (head and body) nucleus of parkinsonian monkeys, there is an increased density of ChIs. (3) Despite the robust loss of CM/Pf neurons, no significant change was found in the density of thalamostriatal (vGluT2-positive) terminals, and in the prevalence of vGluT2-positive terminals in contact with ChIs in parkinsonian monkeys. These findings provide new information about the state of thalamic innervation of the striatum in parkinsonian monkeys with CM/Pf degeneration, and bring up an additional level of intricacy to the consequences of thalamic pathology upon the functional microcircuitry of the thalamostriatal system in parkinsonism. Future studies are needed to assess the importance of CM/Pf neuronal loss, and its potential consequences on the neuroplastic changes induced in the synaptic organization of the thalamostriatal system, in the development of early cognitive impairments in PD.

Loss of striatal tyrosine-hydroxylase interneurons impairs instrumental goal-directed behavior

The striatum mediates a broad range of cognitive and motor functions. Within the striatum, recently discovered tyrosine hydroxylase expressing interneurons (THINs) provide a source of intrastriatal synaptic connectivity that is critical for regulating striatal activity, yet the role of THIN's in behavior remains unknown. Given the important role of the striatum in reward-based behaviors, we investigated whether loss of striatal THINs would impact instrumental behavior in mice. We selectively ablated striatal THINs in TH-Cre mice using chemogenetic techniques, and then tested THIN-lesioned or control mice on three reward-based striatal-dependent instrumental tests: (a) progressive ratio test; (b) choice test following selective-satiety induced outcome devaluation; (c) outcome reinstatement test. Both striatal-THIN-lesioned and control mice acquired an instrumental response for flavored food pellets, and their behavior did not differ in the progressive ratio test, suggesting intact effort to obtain rewards. However, striatal THIN lesions markedly impaired choice performance following selective-satiety induced outcome devaluation. Unlike control mice, THIN-lesioned mice did not adjust their choice of actions following a change in outcome value. In the outcome reinstatement test THIN-lesioned and control mice showed response invigoration by outcome presentation, suggesting the incentive properties of outcomes were not disrupted by THIN lesions. Overall, we found that striatal THIN lesions selectively impaired goal-directed behavior, while preserving motoric and appetitive behaviors. These findings are the first to describe a function of striatal THINs in reward-based behavior, and further illustrate the important role for intrastriatal interneuronal connectivity in behavioral functions ascribed to the striatum more generally.

The Thalamostriatal Pathway and the Hierarchical Control of Action

Sequential ordering of motor commands is required for the simplest of our daily activities. In this issue of Neuron, Díaz-Hernández et al. (2018) show that distinct thalamic inputs to different regions of the dorsal striatum critically modulate the initiation and execution of action sequences.

An Integrated Model of Action Selection: Distinct Modes of Cortical Control of Striatal Decision Making

Making decisions in environments with few choice options is easy. We select the action that results in the most valued outcome. Making decisions in more complex environments, where the same action can produce different outcomes in different conditions, is much harder. In such circumstances, we propose that accurate action selection relies on top-down control from the prelimbic and orbitofrontal cortices over striatal activity through distinct thalamostriatal circuits. We suggest that the prelimbic cortex exerts direct influence over medium spiny neurons in the dorsomedial striatum to represent the state space relevant to the current environment. Conversely, the orbitofrontal cortex is argued to track a subject's position within that state space, likely through modulation of cholinergic interneurons.

The Cognitive Thalamus as a Gateway to Mental Representations

Historically, the thalamus has been viewed as little more than a relay, simply transferring information to key players of the cast, the cortex and hippocampus, without providing any unique functional contribution. In recent years, evidence from multiple laboratories researching different thalamic nuclei has contradicted this idea of the thalamus as a passive structure. Dated models of thalamic functions are being pushed aside, revealing a greater and far more complex contribution of the thalamus for cognition. In this Viewpoints article, we show how recent data support novel views of thalamic functions that emphasize integrative roles in cognition, ranging from learning and memory to flexible adaption. We propose that these apparently separate cognitive functions may indeed be supported by a more general role in shaping mental representations. Several features of thalamocortical circuits are consistent with this suggested role, and we highlight how divergent and convergent thalamocortical and corticothalamic pathways may complement each other to support these functions. Furthermore, the role of the thalamus for subcortical integration is highlighted as a key mechanism for maintaining and updating representations. Finally, we discuss future areas of research and stress the importance of incorporating new experimental findings into existing knowledge to continue developing thalamic models. The presence of thalamic pathology in a number of neurological conditions reinforces the need to better understand the role of this region in cognition.

Thalamocortical and corticothalamic pathways differentially contribute to goal-directed behaviors in the rat

Highly distributed neural circuits are thought to support adaptive decision-making in volatile and complex environments. Notably, the functional interactions between prefrontal and reciprocally connected thalamic nuclei areas may be important when choices are guided by current goal value or action-outcome contingency. We examined the functional involvement of selected thalamocortical and corticothalamic pathways connecting the dorsomedial prefrontal cortex (dmPFC) and the mediodorsal thalamus (MD) in the behaving rat. Using a chemogenetic approach to inhibit projection-defined dmPFC and MD neurons during an instrumental learning task, we show that thalamocortical and corticothalamic pathways differentially support goal attributes. Both pathways participate in adaptation to the current goal value, but only thalamocortical neurons are required to integrate current causal relationships. These data indicate that antiparallel flow of information within thalamocortical circuits may convey qualitatively distinct aspects of adaptive decision-making and highlight the importance of the direction of information flow within neural circuits.

Planning and decision-making rely upon a region of the brain called the prefrontal cortex. But the prefrontal cortex does not act in isolation. Instead, it works together with a number of other brain regions. These include the thalamus, an area long thought to pass information on to the cortex for further processing. But signals also travel in the opposite direction, from the cortex back to the thalamus. Does the cortex-to-thalamus pathway carry the same information as the thalamus-to-cortex pathway?

To find out, Alcaraz et al. blocked each pathway in rats performing a decision-making task. The rats had learned that pressing a lever led to one type of reward, whereas moving a rod led to another. Alcaraz et al. reduced the desirability of one of the rewards by giving the rats free access to it for an hour. Afterwards, the rats opted mainly for the action associated with the reward that had remained desirable. However, blocking either the thalamus-to-cortex or cortex-to-thalamus pathway prevented this preference from emerging. This suggests that an information flow in both directions is necessary to update knowledge about the value of a reward.

In a second experiment, Alcaraz et al. removed the link between one of the actions and its reward. The reward instead appeared at random, irrespective of the rat’s own behavior. Control rats responded by focusing their efforts on the action that still delivered a reliable reward, and by performing the other action less often. Blocking the thalamus-to-cortex pathway prevented this response, but blocking the cortex-to-thalamus pathway did not. This suggests that only the former pathway is necessary to re-evaluate the relationship between an action and an outcome.

Two key aspects of goal-directed behavior – recognizing the value of a reward and the link between an action and an outcome – thus depend differently on the thalamus-to-cortex and cortex-to-thalamus pathways. This same principle may also be at work in other neural circuits with bidirectional connections. Understanding such principles may lead to better strategies for treating disorders of brain connectivity, such as schizophrenia.

Functional neurochemical imaging of the human striatal cholinergic system during reversal learning

Animal studies have shown that acetylcholine (ACh) levels in the dorsal striatum play a role in reversal learning. However, this has not been studied in humans due to a lack of appropriate non-invasive techniques. Proton magnetic resonance spectroscopy (¹ H-MRS) can be used to measure metabolite levels in humans in vivo. Although it cannot be used to study ACh directly, ¹ H-MRS can be used to study choline, an ACh precursor, which is linked to activity-dependent ACh release. The aim of this study was to use functional-¹ H-MRS (fMRS) to measure changes in choline levels in the human dorsal striatum during performance of a probabilistic reversal learning task. We demonstrate a task-dependent decrease in choline, specifically during reversal, but not initial, learning. We interpret this to reflect a sustained increase in ACh levels, which is in line with findings from the animal literature. This task-dependent change was specific to choline and was not observed in control metabolites. These findings provide support for the use of fMRS in the in vivo study of the human cholinergic system.