Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning

Orbitostriatal encoding of reward delayed gratification and impulsivity in chronic pain

Central robust network functional rearrangement is a characteristic of several neurological conditions, including chronic pain. Preclinical and clinical studies have shown the importance of pain-induced dysfunction in both orbitofrontal cortex (OFC) and nucleus accumbens (NAc) brain regions for the emergence of cognitive deficits. Outcome information processing recruits the orbitostriatal circuitry, a pivotal pathway regarding context-dependent reward value encoding. The current literature reveals the existence of structural and functional changes in the orbitostriatal crosstalk in chronic pain conditions, which have emerged as a possible underlying cause for reward and time discrimination impairments observed in individuals affected by such disturbances. However, more comprehensive investigations are needed to elucidate the underlying disturbances that underpin disease development. In this review article, we aim to provide a comprehensive view of the orbitostriatal mechanisms underlying time-reward dependent behaviors, and integrate previous findings on local and network malplasticity under the framework of the chronic pain sphere.

Chemogenetic inactivation of the nucleus reuniens and its projections to the orbital cortex produce deficits on discrete measures of behavioral flexibility in the attentional set-shifting task

The nucleus reuniens (RE) of the ventral midline thalamus is a critical node in the communication between the orbitomedial prefrontal cortex (OFC) and the hippocampus (HF). While RE has been shown to directly participate in memory-associated functions through its connections with the medial prefrontal cortex and HF, less is known regarding the role of RE in executive functioning. Here, we examined the involvement of RE and its projections to the orbital cortex (ORB) in attention and behavioral flexibility in male rats using the attentional set shifting task (AST). Rats expressing the hM4Di DREADD receptor in RE were implanted with indwelling cannulas in either RE or the ventromedial ORB to pharmacologically inhibit RE or its projections to the ORB with intracranial infusions of clozapine-N-oxide hydrochloride (CNO). Chemogenetic-induced suppression of RE resulted in impairments in reversal learning and set-shifting. This supports a vital role for RE in behavioral flexibility - or the ability to adapt behavior to changing reward or rule contingencies. Interestingly, CNO suppression of RE projections to the ventromedial ORB produced impairments in rule abstraction - or dissociable effects elicited with direct RE suppression. In summary, the present findings indicate that RE, mediated in part by actions on the ORB, serves a critical role in the flexible use of rules to drive goal directed behavior. The cognitive deficits of various neurological disorders with impaired communication between the HF and OFC, may be partly attributed to alterations of RE -- as an established intermediary between these cortical structures.

Perturbations in risk/reward decision making and frontal cortical catecholamine regulation induced by mild traumatic brain injury

Mild traumatic brain injury (mTBI) disrupts cognitive processes that influence risk taking behavior. Little is known regarding the effects of repetitive mild injury (rmTBI) or whether these outcomes are sex specific. Risk/reward decision making is mediated by the prefrontal cortex (PFC), which is densely innervated by catecholaminergic fibers. Aberrant PFC catecholamine activity has been documented following TBI and may underlie TBI-induced risky behavior. The present study characterized the effects of rmTBI on risk/reward decision making behavior and catecholamine transmitter regulatory proteins within the PFC. Rats were exposed to sham, single (smTBI), or three closed-head controlled cortical impact (CH-CCI) injuries and assessed for injury-induced effects on risk/reward decision making using a probabilistic discounting task (PDT). In the first week post-final surgery, mTBI increased risky choice preference. By the fourth week, males exhibited increased latencies to make risky choices following rmTBI, demonstrating a delayed effect on processing speed. When levels of tyrosine hydroxylase (TH) and the norepinephrine reuptake transporter (NET) were measured within subregions of the PFC, females exhibited dramatic increases of TH levels within the orbitofrontal cortex (OFC) following smTBI. However, both males and females demonstrated reduced levels of OFC NET following rmTBI. These results indicate the OFC is susceptible to catecholamine instability after rmTBI and suggests that not all areas of the PFC contribute equally to TBI-induced imbalances. Overall, the CH-CCI model of rmTBI has revealed time-dependent and sex-specific changes in risk/reward decision making and catecholamine regulation following repetitive mild head injuries.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Opportunities for risk-taking during play alters cognitive performance and prefrontal inhibitory signalling in rats of both sexes

Social play behaviour is a rewarding activity that can entail risks, thus allowing young individuals to test the limits of their capacities and to train their cognitive and emotional adaptability to challenges. Here, we tested in rats how opportunities for risk-taking during play affect the development of cognitive and emotional capacities and medial prefrontal cortex (mPFC) function, a brain structure important for risk-based decision making. Male and female rats were housed socially or social play-deprived (SPD) between postnatal day (P)21 and P42. During this period, half of both groups were daily exposed to a high-risk play environment. Around P85, all rats were tested for cognitive performance and emotional behaviour after which inhibitory currents were recorded in layer 5 pyramidal neurons in mPFC slices. We show that playing in a high-risk environment altered cognitive flexibility in both sexes and improved behavioural inhibition in males. High-risk play altered anxiety-like behaviour in the elevated plus maze in males and in the open field in females, respectively. SPD affected cognitive flexibility in both sexes and decreased anxiety-like behaviour in the elevated plus maze in females. We found that synaptic inhibitory currents in the mPFC were increased in male, but not female, rats after high-risk play, while SPD lowered prefrontal cortex (PFC) synaptic inhibition in both sexes. Together, our data show that exposure to risks during play affects the development of cognition, emotional behaviour and inhibition in the mPFC. Furthermore, our study suggests that the opportunity to take risks during play cannot substitute for social play behaviour.

Proactive Versus Reactive Control Strategies Differentially Mediate Alcohol Drinking in Male Wistars and P Rats

Problematic alcohol consumption is associated with deficits in decision-making and alterations in prefrontal cortex neural activity likely contribute. We hypothesized that the differences in cognitive control would be evident between male Wistars and a model of genetic risk: alcohol-preferring P rats. Cognitive control is split into proactive and reactive components. Proactive control maintains goal-directed behavior independent of a stimulus, whereas reactive control elicits goal-directed behavior at the time of a stimulus. We hypothesized that Wistars would show proactive control over alcohol seeking whereas P rats would show reactive control over alcohol seeking. Neural activity was recorded from the prefrontal cortex during an alcohol seeking task with two session types. On congruent sessions, the conditioned stimulus (CS+) was on the same side as alcohol access. Incongruent sessions presented alcohol opposite the CS+. Wistars, but not P rats, made more incorrect approaches during incongruent sessions, suggesting that Wistars utilized the previously learned rule. This motivated the hypothesis that neural activity reflecting proactive control would be observable in Wistars but not P rats. While P rats showed differences in neural activity at times of alcohol access, Wistars showed differences prior to approaching the sipper. These results support our hypothesis that Wistars are more likely to engage in proactive cognitive control strategies whereas P rats are more likely to engage in reactive cognitive control strategies. Although P rats were bred to prefer alcohol, the differences in cognitive control may reflect a sequela of behaviors that mirror those in humans at risk for an AUD.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

DREADD activation of the lateral orbitofrontal increases cocaine-taking and cocaine-seeking in male and female rats during intermittent access self-administration under risky conditions

Addiction is a disorder that can be characterized in part as the constant pursuit of a particular substance despite negative consequences. Although the orbitofrontal cortex (OFC) is known to regulate risk-taking more generally and be critical to the development of addiction, its role in regulating drug use under risk-taking conditions is unknown. To address this, we examined drug-taking and drug-seeking in male and female rats under conditions where cocaine infusions were paired with foot shock punishment 50% of the time and combined this paradigm with cFos immunohistochemistry. We found that rats that showed higher levels of drug-taking and drug-seeking prior to punishment showed decreased responding during self-administration sessions under risky conditions and lower levels of c-Fos expression in the lateral but not medial OFC. However, despite these initial differences in responses to infusions paired with foot shocks, all rats showed decreased responding with additional punishment sessions. We then used chemogenetic viral approaches to examine how altering activity of the lateral OFC affects drug-taking and drug-seeking during punished drug use. Although there was no effect of Gi/o DREADD-mediated inhibition of the lateral OFC on these behaviors, Gq DREADD-mediated activation increased drug-taking and drug-seeking when drug use was associated with foot shock 50% of the time. Interestingly, this manipulation had no effect on non-risky self-administration behavior. These results suggest that the involvement of lateral OFC in cocaine use is context-sensitive and influences decision-making based on negative outcomes.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Activity in the Dorsomedial Striatum Underlies Serial Reversal Learning Performance Under Probabilistic Uncertainty

Background

Corticostriatal circuits, particularly the dorsomedial striatum (DMS) and lateral orbitofrontal cortex, are critical for navigating reversal learning under probabilistic uncertainty. These same areas are implicated in the reversal learning impairments observed in individuals with psychosis as well as their psychotic symptoms, suggesting that they may share a common neurobiological substrate. To address this question, we used psychostimulant exposure and specific activation of the DMS during reversal learning in mice to assess corticostriatal activity.

Methods

We used amphetamine treatment to induce psychosis-relevant neurobiology in male mice during reversal learning and to examine pathway-specific corticostriatal activation. To determine the causal role of DMS activity, we used chemogenetics to drive midbrain inputs during a range of probabilistic contingencies.

Results

Mice treated with amphetamine showed altered punishment learning, which was associated with decreased shifting after losses and increased perseverative errors after reversals. Reversal learning performance and strategies were dependent on increased activity in lateral orbitofrontal cortex to DMS circuits as well as in the DMS itself. Specific activation of midbrain to DMS circuits also decreased shifting after losses and reversal learning performance. However, these alterations were dependent on the probabilistic contingency.

Conclusions

Our work suggests that the DMS plays a multifaceted role in reversal learning. Increasing DMS activity impairs multiple reversal learning processes dependent on the level of uncertainty, confirming its role in the maintenance and selection of incoming cortical inputs. Together, these outcomes suggest that elevated dopamine levels in the DMS could contribute to decision-making impairments in individuals with psychosis.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Proactive Versus Reactive Control Strategies Differentially Mediate Alcohol Drinking in Wistar and P rats

Problematic alcohol consumption is associated with deficits in decision-making, and alterations in prefrontal cortex neural activity likely contributes. We hypothesized that differences in cognitive control would be evident between male Wistar rats and a model for genetic risk for alcohol use disorder (alcohol-preferring P rats). Cognitive control can be split into proactive and reactive components. Proactive control maintains goal-directed behavior independent of a stimulus whereas reactive control elicits goal-directed behavior at the time of a stimulus. We hypothesized that Wistars would show proactive control over alcohol-seeking whereas P rats would show reactive control over alcohol-seeking. Neural ensembles were recorded from prefrontal cortex during an alcohol seeking task that utilized two session types. On congruent sessions the CS+ was on the same side as alcohol access. Incongruent sessions presented alcohol opposite the CS+. Wistars, but not P rats, exhibited an increase in incorrect approaches during incongruent sessions, suggesting that Wistars utilized the previously learned task-rule. This motivated the hypothesis that ensemble activity reflecting proactive control would be observable in Wistars but not P rats. While P rats showed differences in neural activity at times relevant for alcohol delivery, Wistars showed differences prior to approaching the sipper. These results support our hypothesis that Wistars are more likely to engage proactive cognitive-control strategies whereas P rats are more likely to engage reactive cognitive control strategies. Although P rats were bred to prefer alcohol, differences in cognitive control may reflect a sequela of behaviors that mirror those in humans at risk for an AUD.

Modulation of ventromedial orbitofrontal cortical glutamatergic activity affects the explore-exploit balance and influences value-based decision-making

The balance between exploration and exploitation is essential for decision-making. The present study investigated the role of ventromedial orbitofrontal cortex (vmOFC) glutamate neurons in mediating value-based decision-making by first using optogenetics to manipulate vmOFC glutamate activity in rats during a probabilistic reversal learning (PRL) task. Rats that received vmOFC activation during informative feedback completed fewer reversals and exhibited reduced reward sensitivity relative to rats. Analysis with a Q-learning computational model revealed that increased vmOFC activity did not affect the learning rate but instead promoted maladaptive exploration. By contrast, vmOFC inhibition increased the number of completed reversals and increased exploitative behavior. In a separate group of animals, calcium activity of vmOFC glutamate neurons was recorded using fiber photometry. Complementing our results above, we found that suppression of vmOFC activity during the latter part of rewarded trials was associated with improved PRL performance, greater win-stay responding and selecting the correct choice on the next trial. These data demonstrate that excessive vmOFC activity during reward feedback disrupted value-based decision-making by increasing the maladaptive exploration of lower-valued options. Our findings support the premise that pharmacological interventions that normalize aberrant vmOFC glutamate activity during reward feedback processing may attenuate deficits in value-based decision-making.

Optogenetic disruption of the prelimbic cortex alters long-term decision strategy but not valuation on a spatial delay discounting task

Rats demonstrate a preference for smaller, immediate rewards over larger, delayed ones, a phenomenon known as delay-discounting (DD). Behavior arises from the interaction of multiple decision-making systems, and the medial prefrontal cortex (mPFC) has been identified as a central component in the mediation between these decision systems. To investigate the role of the prelimbic (PL) subregion of mPFC on decision strategy interaction, we compared two cohorts of rats (ChR2-opsin-expressing 'Active' and opsin-absent 'Control') on a spatial delay-discounting task while delivering in-vivo light stimulation into PL at the choice point of select trials. By analyzing the overall delay-adjustment along with deliberative and procedural behavioral strategy markers, our study revealed differences in the decision strategies used between the active and control animals despite both groups showing similar valuations. Control animals developed the expected shift from deliberative to procedural decision strategy on this task (indicated by reaching delay-stability, particularly during late-session laps); however, active-virus animals repeatedly over-adjusted around their preferred delay throughout the entire session, suggesting a significant deficit in procedural decision-making on this task. Active animals showed a significant decrease in proportion of vicarious trial and error events (VTE, a behavior correlated with deliberative processes) on delay adjustment laps relative to control animals. This points to a more nuanced role for VTE, not just in executing deliberation, but in shifting from deliberative to procedural processes. This opto-induced change in VTE was especially pronounced for late-session adjustment laps. We found no other session-by-session or lap-by-lap effects, leaving a particular role for PL in the long-term development of procedural strategies on this task.

Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning

Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition, there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.

Medial orbitofrontal cortical regulation of different aspects of Pavlovian and instrumental reward seeking

RATIONALE

The medial subregion of the orbitofrontal cortex (mOFC) is thought to play an important role representing the expected outcome of a given course of action, as lesioning or inactivating this cortical region results in the adoption of choice strategies based more on observable (rather than previously learned) information. Despite this, its role in mediating basic associative learning remains to be fully clarified.

OBJECTIVE

The present series of experiments examined the role of the mOFC in (1) Pavlovian conditioned approach, (2) conditioned reinforcement, (3) extinction, and (4) cue-induced reinstatement of food-seeking behavior.

METHODS

Separate cohorts of rats went through Pavlovian or instrumental training. Intra-mOFC infusions of either saline or GABA agonists (to temporarily inactivate neural activity) were given prior to Pavlovian approach, conditioned reinforcement, first or second day of instrumental extinction training, or cue-induced reinstatement test days.

RESULTS

mOFC inactivation increased lever-CS contacts in Pavlovian conditioned approach and (2) had no effect on conditioned reinforcement. These manipulations (3) accelerated within-session instrumental extinction during the initial extinction session, but impaired subsequent extinction learning on drug-free days. (4) mOFC inactivation induced differential effects on reinstatement that depended on baseline performance. mOFC inactivation abolished reinstatement in "Reinstater" rats (who displayed robust responding under control conditions) and robustly increased reinstatement in "Non-Reinstater" rats (who showed little reinstatement under control conditions) suggesting that individual differences in reinstatement may be supported by differences in mOFC mediated representations of expected outcomes.

CONCLUSIONS

These findings have important implications for understanding how the mOFC uses stimulus-outcome and action-outcome expectancies to guide behavior, and how dysfunction within this region may contribute to pathological patterns of reward seeking.

The medial orbitofrontal cortex governs reward-related circuits in an age-dependent manner

Nucleus accumbens (NAc) neurons integrate excitatory inputs from cortical and limbic structures, contributing to critical cognitive functions, including decision-making. As these afferents mature from adolescence through adulthood, incoming signals to the NAc may summate differently between age groups. Decision-making evaluates both reward and risk before action selection, suggesting an interplay between reward- and risk-related circuits. Medial orbitofrontal cortex (MO)-NAc circuits permit risk assessment behaviors and likely underlie risk information incorporation. As adolescents make reward-centric choices regardless of risk, we hypothesized the impact of MO activity alters reward-related NAc circuits in an age-dependent manner. To test this hypothesis, we used single-unit electrophysiology to measure MO train stimulation's effect on reward-related pathways, specifically the basolateral amygdala (BLA)-NAc circuit, in adult and adolescent rats. MO train stimulation altered the strength but not the timing of BLA-NAc interactions in a frequency-dependent manner. In adults, MO train stimulation produced a frequency-dependent, bidirectional effect on BLA-evoked NAc AP probability. Contrastingly, MO train stimulation uniformly attenuated BLA-NAc interactions in adolescents. While the mature MO can govern reward-related circuits in an activity-dependent manner, perhaps to adapt to positive or negative decision-making outcomes, the adolescent MO may be less able to bidirectionally impact reward-related pathways resulting in biased decision-making.

Inhibition of noradrenergic signalling in rodent orbitofrontal cortex impairs the updating of goal-directed actions

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.

Projections from the five divisions of the orbital cortex to the thalamus in the rat

The orbital cortex (ORB) of the rat consists of five divisions: the medial (MO), ventral (VO), ventrolateral (VLO), lateral (LO), and dorsolateral (DLO) orbital cortices. No previous report has comprehensively examined and compared projections from each division of the ORB to the thalamus. Using the anterograde anatomical tracer, Phaseolus vulgaris leucoagglutinin, we describe the efferent projections from the five divisions of the ORB to the thalamus in the rat. We demonstrated that, with some overlap, each division of the ORB distributed in a distinct (and unique) manner to nuclei of the thalamus. Overall, ORB projected to a relatively restricted number of sites in the thalamus, and strikingly distributed entirely to structures of the medial/midline thalamus, while completely avoiding lateral regions or principal nuclei of the thalamus. The main termination sites in the thalamus were the paratenial nucleus (PT) and nucleus reuniens (RE) of the midline thalamus, the medial (MDm) and central (MDc) divisions of the mediodorsal nucleus, the intermediodorsal nucleus, the central lateral, paracentral, and central medial nuclei of the rostral intralaminar complex and the submedial nucleus (SM). With some exceptions, medial divisions of the ORB (MO, VO) mainly targeted "limbic-associated" nuclei such as PT, RE, and MDm, whereas lateral division (VLO, LO, DLO) primarily distributed to "sensorimotor-associated" nuclei including MDc, SM, and the rostral intralaminar complex. As discussed herein, the medial/midline thalamus may represent an important link (or bridge) between the orbital cortex and the hippocampus and between the ORB and medial prefrontal cortex. In summary, the present results demonstrate that each division of the orbital cortex projects in a distinct manner to nuclei of the thalamus which suggests unique functions for each division of the orbital cortex.

Differential processing of decision information in subregions of rodent medial prefrontal cortex

Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.

Reinforcement learning deficits exhibited by postnatal PCP-treated rats enable deep neural network classification

The ability to appropriately update the value of a given action is a critical component of flexible decision making. Several psychiatric disorders, including schizophrenia, are associated with impairments in flexible decision making that can be evaluated using the probabilistic reversal learning (PRL) task. The PRL task has been reverse-translated for use in rodents. Disrupting glutamate neurotransmission during early postnatal neurodevelopment in rodents has induced behavioral, cognitive, and neuropathophysiological abnormalities relevant to schizophrenia. Here, we tested the hypothesis that using the NMDA receptor antagonist phencyclidine (PCP) to disrupt postnatal glutamatergic transmission in rats would lead to impaired decision making in the PRL. Consistent with this hypothesis, compared to controls the postnatal PCP-treated rats completed fewer reversals and exhibited disruptions in reward and punishment sensitivity (i.e., win-stay and lose-shift responding, respectively). Moreover, computational analysis of behavior revealed that postnatal PCP-treatment resulted in a pronounced impairment in the learning rate throughout PRL testing. Finally, a deep neural network (DNN) trained on the rodent behavior could accurately predict the treatment group of subjects. These data demonstrate that disrupting early postnatal glutamatergic neurotransmission impairs flexible decision making and provides evidence that DNNs can be trained on behavioral datasets to accurately predict the treatment group of new subjects, highlighting the potential for DNNs to aid in the diagnosis of schizophrenia.

Social Play Behavior Is Critical for the Development of Prefrontal Inhibitory Synapses and Cognitive Flexibility in Rats

Sensory driven activity during early life is critical for setting up the proper connectivity of the sensory cortices. We ask here whether social play behavior, a particular form of social interaction that is highly abundant during postweaning development, is equally important for setting up connections in the developing prefrontal cortex (PFC). Young male rats were deprived from social play with peers during the period in life when social play behavior normally peaks [postnatal day 21-42] (SPD rats), followed by resocialization until adulthood. We recorded synaptic currents in layer 5 cells in slices from medial PFC of adult SPD and control rats and observed that inhibitory synaptic currents were reduced in SPD slices, while excitatory synaptic currents were unaffected. This was associated with a decrease in perisomatic inhibitory synapses from parvalbumin-positive GABAergic cells. In parallel experiments, adult SPD rats achieved more reversals in a probabilistic reversal learning (PRL) task, which depends on the integrity of the PFC, by using a more simplified cognitive strategy than controls. Interestingly, we observed that one daily hour of play during SPD partially rescued the behavioral performance in the PRL, but did not prevent the decrease in PFC inhibitory synaptic inputs. Our data demonstrate the importance of unrestricted social play for the development of inhibitory synapses in the PFC and cognitive skills in adulthood and show that specific synaptic alterations in the PFC can result in a complex behavioral outcome.SIGNIFICANCE STATEMENT This study addressed the question whether social play behavior in juvenile rats contributes to functional development of the prefrontal cortex (PFC). We found that rats that had been deprived from juvenile social play (social play deprivation - SPD) showed a reduction in inhibitory synapses in the PFC and a simplified strategy to solve a complex behavioral task in adulthood. Providing one daily hour of play during SPD partially rescued the cognitive skills in these rats, but did not prevent the reduction in PFC inhibitory synapses. Our results demonstrate a key role for unrestricted juvenile social play in PFC development and emphasize the complex relation between PFC circuit connectivity and cognitive function.

Sustained MK-801 induced deficit in a novel probabilistic reversal learning task

Cognitive flexibility, the ability to adapt to unexpected changes, is critical for healthy environmental and social interactions, and thus to everyday functioning. In neuropsychiatric diseases, cognitive flexibility is often impaired and treatment options are lacking. Probabilistic reversal learning (PRL) is commonly used to measure cognitive flexibility in rodents and humans. In PRL tasks, subjects must sample choice options and, from probabilistic feedback, find the current best choice which then changes without warning. However, in rodents, pharmacological models of human cognitive impairment tend to disrupt only the first (or few) of several contingency reversals, making quantitative assessment of behavioral effects difficult. To address this limitation, we developed a novel rat PRL where reversals occur at relatively long intervals in time that demonstrates increased sensitivity to the non-competitive NMDA receptor antagonist MK-801. Here, we quantitively compare behavior in time-based PRL with a widely used task where reversals occur based on choice behavior. In time-based PRL, MK-801 induced sustained reversal learning deficits both in time and across reversal blocks but, at the same dose, only transient weak effects in performance-based PRL. Moreover, time-based PRL yielded better estimates of behavior and reinforcement learning model parameters, which opens meaningful pharmacological windows to efficiently test and develop novel drugs preclinically with the goal of improving cognitive impairment in human patients.

Medial orbitofrontal neurotrophin systems integrate hippocampal input into outcome-specific value representations

In everyday life, we mentally represent possible consequences of our behaviors and integrate specific outcome values into existing knowledge to inform decisions. The medial orbitofrontal cortex (MO) is necessary to adapt behaviors when outcomes are not immediately available-when they and their values need to be envisioned. Nevertheless, neurobiological mechanisms remain unclear. We find that the neuroplasticity-associated neurotrophin receptor tropomyosin receptor kinase B (TrkB) is necessary for mice to integrate outcome-specific value information into choice behavior. This function appears attributable to memory updating (and not retrieval) and the stabilization of dendritic spines on excitatory MO neurons, which led us to investigate inputs to the MO. Ventral hippocampal (vHC)-to-MO projections appear conditionally necessary for value updating, involved in long-term aversion-based value memory updating. Furthermore, vHC-MO-mediated control of choice is TrkB dependent. Altogether, we reveal a vHC-MO connection by which specific value memories are updated, and we position TrkB within this functional circuit.

Role of Orbitofrontal Cortex and Differential Effects of Acute and Chronic Stress on Motor Impulsivity Measured With 1-Choice Serial Reaction Time Test in Male Rats

BACKGROUND

Deficits in motor impulsivity, that is, the inability to inhibit a prepotent response, are frequently observed in psychiatric conditions. Several studies suggest that stress often correlates with higher impulsivity. Among the brain areas affected by stress, the orbitofrontal cortex (OFC) is notable because of its role in impulse control. OFC subregions with unique afferent and efferent circuitry play distinct roles in impulse control, yet it is not clear what OFC subregions are engaged during motor impulsivity tasks.

METHODS

In this study we used a rodent test of motor impulsivity, the 1-choice serial reaction time test, to explore activation of OFC subregions either during a well-learned motor impulsivity task or in a challenge task with a longer wait time that increases premature responding. We also examined the effects of acute inescapable stress, chronic intermittent cold stress and chronic unpredictable stress on motor impulsivity.

RESULTS

Fos expression increased in the lateral OFC and agranular insular cortex during performance in both the mastered and challenge conditions. In the ventral OFC, Fos expression increased only during challenge, and within the medial OFC, Fos was not induced in either condition. Inescapable stress produced a transient effect on premature responses in the mastered task, whereas chronic intermittent cold stress and chronic unpredictable stress altered premature responses in both conditions in ways specific to each stressor.

CONCLUSIONS

These results suggest that different OFC subregions have different roles in motor impulse control, and the effects of stress vary depending on the nature and duration of the stressor.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Medial orbitofrontal cortex and nucleus accumbens mediation in risk assessment behaviors in adolescents and adults

Risk assessment behaviors are necessary for gathering risk information and guiding decision-making. Risky decision-making heightens during adolescence, possibly as a result of low risk awareness and an increase in sensitivity to reward-associated cues and experiences. Higher adolescent engagement in high-risk behaviors may be, in part, due to developing circuits that contribute to risk assessment behaviors. Nucleus accumbens (NAc) activity is linked to risky decision-making and receives inputs carrying sensory and emotional information. Namely, the medial orbitofrontal cortex (MO) contributes to behavior guided by reward probability and sends direct projections to the NAc (MO→NAc), which may permit risk assessment in a mature circuit. Here, we evaluated risk assessment behaviors in adult and adolescent rats during elevated plus maze (EPM) exploration, including stretch and attend postures, head dips, and rears. We found that adolescents exhibited fewer EPM risk assessment behaviors than adults. We also quantified MO→NAc projections using a fluorescent anterograde tracer, Fluoro-Ruby, in both age groups. Labeled MO→NAc pathways exhibited greater total fluorescence in adults than in adolescents, indicating MO→NAc fibers increase over development. Using a disconnection approach to measure the contribution of the MO-NAc pathway in adults, we found that ipsilateral inactivation of the MO-NAc did not alter risk assessment behavior; however, MO-NAc disconnection reduced the number of stretch-and-attend postures. Together, this work suggests that the development of MO-NAc pathways can contribute to age-dependent differences in risk assessment.

Neuromodulation of OCD: A review of invasive and non-invasive methods

Early research into neural correlates of obsessive compulsive disorder (OCD) has focused on individual components, several network-based models have emerged from more recent data on dysfunction within brain networks, including the the lateral orbitofrontal cortex (lOFC)-ventromedial caudate, limbic, salience, and default mode networks. Moreover, the interplay between multiple brain networks has been increasingly recognized. As the understanding of the neural circuitry underlying the pathophysiology of OCD continues to evolve, so will too our ability to specifically target these networks using invasive and noninvasive methods. This review discusses the rationale for and theory behind neuromodulation in the treatment of OCD.

Cortical reorganization of the glutamate synapse in the activity-based anorexia rat model: impact on cognition

Patients suffering from anorexia nervosa (AN) display altered neural activity, morphological, and functional connectivity in the fronto‐striatal circuit. In addition, hypoglutamatergic transmission and aberrant excitability of the medial prefrontal cortex (mPFC) observed in AN patients might underpin cognitive deficits that fuel the vicious cycle of dieting behavior. To provide a molecular mechanism, we employed the activity‐based anorexia (ABA) rat model, which combines the two hallmarks of AN (i.e., caloric restriction and intense physical exercise), to evaluate structural remodeling together with alterations in the glutamatergic signaling in the mPFC and their impact on temporal memory, as measured by the temporal order object recognition (TOOR) test. Our data indicate that the combination of caloric restriction and intense physical exercise altered the homeostasis of the glutamate synapse and reduced spine density in the mPFC. These events, paralleled by an impairment in recency discrimination in the TOOR test, are associated with the ABA endophenotype. Of note, after a 7‐day recovery period, body weight was recovered and the mPFC structure normalized but ABA rats still exhibited reduced post‐synaptic stability of AMPA and NMDA glutamate receptors associated with cognitive dysfunction. Taken together, these data suggest that the combination of reduced food intake and hyperactivity affects the homeostasis of the excitatory synapse in the mPFC contributing to maintain the aberrant behaviors observed in AN patients. Our findings, by identifying novel potential targets of AN, may contribute to more effectively direct the therapeutic interventions to ameliorate, at least, the cognitive effects of this psychopathology.

Cognitive performance during adulthood in a rat model of neonatal diffuse white matter injury

RATIONALE

Infants born prematurely risk developing diffuse white matter injury (WMI), which is associated with impaired cognitive functioning and an increased risk of autism spectrum disorder. Recently, our rat model of preterm diffuse WMI induced by combined fetal inflammation and postnatal hypoxia showed impaired motor performance, anxiety-like behaviour and autism-like behaviour in juvenile rats, especially males. Immunohistochemistry showed delayed myelination in the sensory cortex and impaired oligodendrocyte differentiation.

OBJECTIVE

To assess long-term cognitive deficits in this double-hit rat model of diffuse WMI, animals were screened on impulsivity, attention and cognitive flexibility in adulthood using the 5-choice serial reaction time task (5CSRTT) and a probabilistic reversal learning task, tests that require a proper functioning prefrontal cortex. Thereafter, myelination deficits were evaluated by immunofluorescent staining in adulthood.

RESULTS

Overall, little effect of WMI or sex was found in the cognitive tasks. WMI animals showed subtle differences in performance in the 5CSRTT. Manipulating 5CSRTT parameters resulted in performance patterns previously seen in the literature. Sex differences were found in perseverative responses and omitted trials: female WMI rats seem to be less flexible in the 5CSRTT but not in the reversal learning task. Males collected rewards faster in the probabilistic reversal learning task. These findings are explained by temporally rather than permanently affected myelination and by the absence of extensive injury to prefrontal cortical subregions, confirmed by immunofluorescent staining in both adolescence and adulthood.

CONCLUSION

This rat model of preterm WMI does not lead to long-term cognitive deficits as observed in prematurely born human infants.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

A selective serotonin reuptake inhibitor ameliorates obsessive-compulsive disorder-like perseverative behavior by attenuating 5-HT2C receptor signaling in the orbitofrontal cortex

Perseveration is a characteristic of patients with obsessive-compulsive disorder (OCD). Clinically, neuronal activity in the lateral orbitofrontal cortex (OFC) is increased in OCD patients. Successful treatment with selective serotonin reuptake inhibitors (SSRIs) reduces activity in the lateral OFC of OCD patients, but the precise mechanisms underlying this effect are unclear. Previously, we reported that repeated injection of the dopamine D₂ receptor agonist quinpirole (QNP) resulted in OCD-like deficits, including perseveration in a reversal learning task. QNP-treated mice showed hyperactivity in lateral OFC pyramidal neurons. The present study demonstrated that 4-week administration of an SSRI increased the rate of correct choice in a reversal learning task. Using the electrophysiological approach, we revealed that an SSRI decreased the activity of lateral OFC pyramidal neurons in QNP-treated mice by potentiating inhibitory inputs. The 4-week administration of an SSRI inhibited the potentiation of neuronal activity induced by a 5-HT_2C receptor agonist. Additionally, both 4-week administration of SSRI and acute application of 5-HT_2C receptor antagonist prevented the QNP-induced potentiation of inhibitory inputs to fast-spiking interneurons in the lateral OFC. Administration of a 5-HT_2C receptor antagonist to mice for 4 days increased the rate of correct choice in a reversal learning task. Collectively, these results indicate that chronic SSRI ameliorated perseverative behavior in QNP-treated mice by modulating inhibitory inputs in the lateral OFC. Short-term 5-HT_2C receptor blockade also ameliorated QNP-induced behavioral and neurological abnormalities by, at least in part, a common mechanism with chronic SSRI.

Suppression of Corticostriatal Circuit Activity Improves Cognitive Flexibility and Prevents Body Weight Loss in Activity-Based Anorexia in Rats

BACKGROUND

The ability to adapt behavior to changing environmental circumstances, or cognitive flexibility, is impaired in multiple psychiatric conditions, including anorexia nervosa (AN). Exaggerated prefrontal cortical activity likely underpins the inflexible thinking and rigid behaviors exhibited by patients with AN. A better understanding of the neural basis of cognitive flexibility is necessary to enable treatment approaches that may target impaired executive control.

METHODS

Utilizing the activity-based anorexia (ABA) model and touchscreen operant learning paradigms, we investigated the neurobiological link between pathological weight loss and cognitive flexibility. We used pathway-specific chemogenetics to selectively modulate activity in neurons of the medial prefrontal cortex (mPFC) projecting to the nucleus accumbens shell (AcbSh) in female Sprague Dawley rats.

RESULTS

DREADD (designer receptor exclusively activated by designer drugs)-based inhibition of the mPFC-AcbSh pathway prevented weight loss in ABA and improved flexibility during early reversal learning by reducing perseverative responding. Modulation of activity within the mPFC-AcbSh pathway had no effect on running, locomotor activity, or feeding under ad libitum conditions, indicating the specific involvement of this circuit in conditions of dysregulated reward.

CONCLUSIONS

Parallel attenuation of weight loss in ABA and improvement of cognitive flexibility following suppression of mPFC-AcbSh activity align with the relationship between disrupted prefrontal function and cognitive rigidity in AN patients. The identification of a neurobiological correlate between cognitive flexibility and pathological weight loss provides a unique insight into the executive control of feeding behavior. It also highlights the utility of the ABA model for understanding the biological bases of cognitive deficits in AN and provides context for new treatment strategies.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

HIV Transgenic Rats Demonstrate Superior Task Acquisition and Intact Reversal Learning in the Within-Session Probabilistic Reversal Learning Task

The HIV transgenic (HIVtg) rat is a commonly used animal model of chronic HIV infection that exhibits a wide range of cognitive deficits. To date, relatively little work has been conducted on these rats' capacity for reversal learning, an assay of executive function and cognitive flexibility used in humans. The present study sought to determine the impact of HIV genotype on probabilistic reversal learning, effortful motivation, and spontaneous locomotion/exploration in rats. Male (n = 8) and female (n = 8) HIVtg rats and wildtype (WT) controls were utilized. Cognitive flexibility was assessed via the Probabilistic Reversal Learning Task (PRLT), which reinforced responses to two stimuli on differential probabilistic schedules that periodically reversed. Effortful motivation and locomotor/exploratory behavior were assessed via the Progressive Ratio Breakpoint Task (PRBT) and the Behavioral Pattern Monitor (BPM), respectively. Regardless of sex, HIVtg rats required fewer trials to ascertain initial PRLT reward schedules than WT rats, and completed the same number of reversals. Secondary behaviors suggested that HIVtg PRLT performance was facilitated by a speed-accuracy tradeoff strategy. No main or interactive effects of genotype were observed in the PRBT or BPM. Relative to WT controls, HIVtg rats exhibited superior probabilistic reinforcement learning. Reversal learning was unaffected by HIV genotype, as was effortful motivation and exploratory behavior. These findings contrast with previous characterizations of the HIVtg rat, thus indicating a nuanced cognitive profile that is dependent upon such task specifications as within- versus between-session assessment and probabilistic versus deterministic reward schedules.

Differential Expression Patterns of Lynx Proteins and Involvement of Lynx1 in Prepulse Inhibition

Negative allosteric modulators, such as lynx1 and lynx2, directly interact with nicotinic acetylcholine receptors (nAChRs). The nAChRs are integral to cholinergic signaling in the brain and have been shown to mediate different aspects of cognitive function. Given the interaction between lynx proteins and these receptors, we examined whether these endogenous negative allosteric modulators are involved in cognitive behaviors associated with cholinergic function. We found both cell-specific and overlapping expression patterns of lynx1 and lynx2 mRNA in brain regions associated with cognition, learning, memory, and sensorimotor processing, including the prefrontal cortex (PFC), cingulate cortex, septum, hippocampus, amygdala, striatum, and pontine nuclei. Since lynx proteins are thought to play a role in conditioned associations and given the expression patterns across brain regions, we first assessed whether lynx knockout mice would differ in a cognitive flexibility task. We found no deficits in reversal learning in either the lynx1^–/– or lynx2^–/– knockout mice. Thereafter, sensorimotor gating was examined with the prepulse inhibition (PPI) assessment. Interestingly, we found that both male and female lynx1^–/– mice exhibited a deficit in the PPI behavioral response. Given the comparable expression of lynx2 in regions involved in sensorimotor gating, we then examined whether removal of the lynx2 protein would lead to similar behavioral effects. Unexpectedly, we found that while male lynx2^–/– mice exhibited a decrease in the baseline startle response, no differences were found in sensorimotor gating for either male or female lynx2^–/– mice. Taken together, these studies provide insight into the expression patterns of lynx1 and lynx2 across multiple brain regions and illustrate the modulatory effects of the lynx1 protein in sensorimotor gating.

Entropy-based metrics for predicting choice behavior based on local response to reward

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

Translational opportunities in animal and human models to study alcohol use disorder

Animal and human laboratory paradigms offer invaluable approaches to study the complex etiologies and mechanisms of alcohol use disorder (AUD). We contend that human laboratory models provide a "bridge" between preclinical and clinical studies of AUD by allowing for well-controlled experimental manipulations in humans with AUD. As such, examining the consilience between experimental models in animals and humans in the laboratory provides unique opportunities to refine the translational utility of such models. The overall goal of the present review is to provide a systematic description and contrast of commonly used animal paradigms for the study of AUD, as well as their human laboratory analogs if applicable. While there is a wide breadth of animal species in AUD research, the paradigms discussed in this review rely predominately on rodent research. The overarching goal of this effort is to provide critical analysis of these animal models and to link them to human laboratory models of AUD. By systematically contrasting preclinical and controlled human laboratory models, we seek to identify opportunities to enhance their translational value through forward and reverse translation. We provide future directions to reconcile differences between animal and human work and to improve translational research for AUD.

Chronic alcohol exposure during critical developmental periods differentially impacts persistence of deficits in cognitive flexibility and related circuitry

Cognitive flexibility in decision making depends on prefrontal cortical function and is used by individuals to adapt to environmental changes in circumstances. Cognitive flexibility can be measured in the laboratory using a variety of discrete, translational tasks, including those that involve reversal learning and/or set-shifting ability. Distinct components of flexible behavior rely upon overlapping brain circuits, including different prefrontal substructures that have separable impacts on decision making. Cognitive flexibility is impaired after chronic alcohol exposure, particularly during development when the brain undergoes rapid maturation. This review examines how cognitive flexibility, as indexed by reversal and set-shifting tasks, is impacted by chronic alcohol exposure in adulthood, adolescent, and prenatal periods in humans and animal models. We also discuss areas for future study, including mechanisms that may contribute to the persistence of cognitive deficits after developmental alcohol exposure and the compacting consequences from exposure across multiple critical periods.

Reinforcement-based cognitive biases as vulnerability factors in alcohol addiction: From humans to animal models

Alcohol use disorder (AUD) is one of the most common, but still poorly treated, psychiatric conditions. Developing new treatments requires a better understanding of the aetiology of symptoms and evaluation of novel therapeutic targets in preclinical studies. Recent developments in our understanding of the reinforcement-based cognitive biases (RBCBs) that contribute to the development of AUD and its treatment offer new opportunities for both clinical and preclinical research. In this review, we first briefly describe psychological and cognitive theories that involve various aspects of reinforcement sensitivity in the development, maintenance, and recurrence of alcohol addiction. Furthermore, in separate sections, we describe studies investigating RBCBs and their neural, neurochemical, and pharmacological correlates, and we discuss possible interactions between RBCBs and trajectories of AUD. Finally, we describe how recent translational studies using state-of-the-art animal models can facilitate our understanding of the role of reinforcement sensitivity and RBCBs in various aspects of AUD.

Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Much evidence suggests that reversal learning is mediated by cortico-striatal circuitries with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, but potential differential roles of lateral (lOFC) and medial (mOFC) portions in visual reversal learning have yet to be determined. We investigated the effects of pharmacological inactivation of mOFC and lOFC on a deterministic serial visual reversal learning task for rats. For reference, we also targeted other areas previously implicated in reversal learning: prelimbic (PrL) and infralimbic (IL) prefrontal cortex, and basolateral amygdala (BLA). Inactivating mOFC and lOFC produced opposite effects; lOFC impairing, and mOFC improving, performance in the early, perseverative phase specifically. Additionally, mOFC inactivation enhanced negative feedback sensitivity, while lOFC inactivation diminished feedback sensitivity in general. mOFC and lOFC inactivation also affected novel visual discrimination learning differently; lOFC inactivation paradoxically improved learning, and mOFC inactivation had no effect. We also observed dissociable roles of the OFC and the IL/PrL. Whereas the OFC inactivation affected only perseveration, IL/PrL inactivation improved learning overall. BLA inactivation did not affect perseveration, but improved the late phase of reversal learning. These results support opponent roles of the rodent mOFC and lOFC in deterministic visual reversal learning.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Distinct outcome processing in deterministic and probabilistic reversal learning

Anticipations that fail to happen are important drivers of behavioral adaptation. Their processing appears to depend on the context. In a deterministic environment, where a stimulus unequivocally predicts the outcome, processing of absent outcomes involves the posterior orbitofrontal cortex (OFC). Failure has been linked to reality confusion with confabulations and disorientation. In a probabilistic environment, absent outcomes appear to be processed by the anterior cingulate cortex (ACC) rather than the OFC. Failure has been associated with poor decision making and schizophrenia. These data suggest different mechanisms depending on the context. Here, healthy human subjects made two formally similar reversal learning tasks, but one with deterministic, the other with probabilistic instructions. Brain activity was monitored using high-density electroencephalography. We found that in the deterministic task, negative outcomes, which unequivocally call for a behavioral switch, induced a distinct frontal potential at 200-300 msec. Computational modeling indicated a strong association of evoked potentials with prediction error, surprise, and behavioral adaptation. In the probabilistic task, where behavioral adaptation follows the cumulated processing of outcomes, negative outcomes evoked potentials that were associated with prediction error and surprise, but had a weak link with subsequent behavior. Outcome processing in the probabilistic task induced stronger activation than the deterministic task of an extended network including the ACC, OFC and striatum at 300-400 msec. In both tasks, negative outcomes were processed differently from positive outcomes at 400-600 msec, possibly reflecting updating of the outcome record. We conclude that the brain disposes of at least two distinct systems processing outcomes with unequivocal or ambiguous behavioral significance. These systems differ along behavioral, clinical, electrophysiological and anatomical dimensions.

Whole-brain mapping of monosynaptic inputs to midbrain cholinergic neurons

The cholinergic midbrain is involved in a wide range of motor and cognitive processes. Cholinergic neurons of the pedunculopontine (PPN) and laterodorsal tegmental nucleus (LDT) send long-ranging axonal projections that target sensorimotor and limbic areas in the thalamus, the dopaminergic midbrain and the striatal complex following a topographical gradient, where they influence a range of functions including attention, reinforcement learning and action-selection. Nevertheless, a comprehensive examination of the afferents to PPN and LDT cholinergic neurons is still lacking, partly due to the neurochemical heterogeneity of this region. Here we characterize the whole-brain input connectome to cholinergic neurons across distinct functional domains (i.e. PPN vs LDT) using conditional transsynaptic retrograde labeling in ChAT::Cre male and female rats. We reveal that input neurons are widely distributed throughout the brain but segregated into specific functional domains. Motor related areas innervate preferentially the PPN, whereas limbic related areas preferentially innervate the LDT. The quantification of input neurons revealed that both PPN and LDT receive similar substantial inputs from the superior colliculus and the output of the basal ganglia (i.e. substantia nigra pars reticulata). Notably, we found that PPN cholinergic neurons receive preferential inputs from basal ganglia structures, whereas LDT cholinergic neurons receive preferential inputs from limbic cortical areas. Our results provide the first characterization of inputs to PPN and LDT cholinergic neurons and highlight critical differences in the connectome among brain cholinergic systems thus supporting their differential roles in behavior.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Defining an orbitofrontal compass: Functional and anatomical heterogeneity across anterior-posterior and medial-lateral axes

The orbitofrontal cortex (OFC) plays a critical role in the flexible control of behaviors and has been the focus of increasing research interest. However, there have been a number of controversies around the exact theoretical role of the OFC. One potential source of these issues is the comparison of evidence from different studies, particularly across species, which focus on different specific sub-regions within the OFC. Furthermore, there is emerging evidence that there may be functional diversity across the OFC which may account for these theoretical differences. Therefore, in this review we consider evidence supporting functional heterogeneity within the OFC and how it relates to underlying anatomical heterogeneity. We highlight the importance of anatomical and functional distinctions within the traditionally defined OFC subregions across the medial-lateral axis, which are often not differentiated for practical and historical reasons. We then consider emerging evidence of even finer-grained distinctions within these defined subregions along the anterior-posterior axis. These fine-grained anatomical considerations reveal a pattern of dissociable, but often complementary functions within the OFC. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Differential effects of d- and l-enantiomers of govadine on distinct forms of cognitive flexibility and a comparison with dopaminergic drugs

RATIONALE

There is an urgent need for novel drugs for treating cognitive deficits that are defining features of schizophrenia. The individual d- and l-enantiomers of the tetrahydroprotoberberine (THPB) d,l-govadine have been proposed for the treatment of cognitive deficiencies and positive symptoms of schizophrenia, respectively.

OBJECTIVES

We examined the effects of d-, l-, or d,l-govadine on two distinct forms of cognitive flexibility perturbed in schizophrenia and compared them to those induced by a selective D1 receptor agonist and D2 receptor antagonist.

METHODS

Male rats received d-, l-, or d,l-govadine (0.3, 0.5, and 1.0 mg/kg), D1 agonist SKF81297(0.1, 0.3, and 1.0 mg/kg), or D2 antagonist haloperidol (0.1-0.2 mg/kg). Experiment 1 used a strategy set-shifting task (between-subjects). In experiment 2, well-trained rats were tested on a probabilistic reversal task (within-subjects).

RESULTS

d-Govadine improved set-shifting across all doses, whereas higher doses of l-govadine impaired set-shifting. SKF81297 reduced perseverative errors at the lowest dose. Low/high doses of haloperidol increased/decreased set-shifting errors, the latter "improvement" attributable to impaired retrieval of a previous acquired rule. Probabilistic reversal performance was less affected by these drugs, but d-govadine reduced errors during the first reversal, whereas l-govadine impaired initial discrimination learning. d,l-Govadine had no reliable cognitive effects but caused psychomotor slowing like l-govadine and haloperidol.

CONCLUSIONS

These findings further highlight differences between two enantiomers of d,l-govadine that may reflect differential modulation of D1 and D2 receptors. These preclinical findings give further impetus to formal clinical evaluation of d-govadine as a treatment for cognitive deficiencies related to schizophrenia.

Enhanced synaptic properties of the prefrontal cortex and hippocampus after learning a spatial working memory task in adult male mice

Working memory (WM) is the ability to hold on-line and manipulate information. The prefrontal cortex (PFC) is a key brain region involved in WM, while the hippocampus is also involved, particularly, in spatial WM. Although several studies have investigated the neuronal substrates of WM in trained animals, the effects and the mechanisms underlying learning WM tasks have not been explored. In our study, we investigated the effects of learning WM tasks in mice on the function of PFC and hippocampus, by training mice in the delayed alternation task for 9 days (adaptive group). This group was compared to naïve mice (which stayed in their homecage) and mice trained in the alternation procedure only (non-adaptive). Following training, a cohort of mice (Experiment A) was tested in the left-right discrimination task and the reversal learning task, while another cohort (Experiment B) was tested in the attention set-shifting task (AST). The adaptive group performed significantly better in the reversal learning task (Experiment A) and AST (Experiment B), compared to non-adaptive and naïve groups. At the end of the behavioral experiments in Experiment A, field excitatory post-synaptic potential (fEPSP) recordings were performed in PFC and hippocampal brain slices. The adaptive group had enhanced the long-term potentiation (LTP) in the PFC, compared to the other groups. In the hippocampus, both the adaptive and the non-adaptive groups exhibited increased fEPSP compared to the naïve group, but no differences in LTP. In Experiment B, the dendritic spine density was measured, which, in the PFC, was found increased in the adaptive group, compared to the non-adaptive and naïve groups. In the hippocampus, there was an increase in mature dendritic spine density in the adaptive group, compared to the other two groups. Our results indicate a role for LTP and dendritic spine density in learning WM tasks.