Punishment insensitivity emerges from impaired contingency detection, not aversion insensitivity or reward dominance

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

Delay of punishment highlights differential vulnerability to developing addiction-like behavior toward sweet food

Resistance to punishment is commonly used to measure the difficulty in refraining from rewarding activities when negative consequences ensue, which is a hallmark of addictive behavior. We recently developed a progressive shock strength (PSS) procedure in which individual rats can titrate the amount of punishment that they are willing to tolerate to obtain food rewards. Here, we investigated the effects of a range of delays (0-12 s) on resistance to punishment measured by PSS break points. As expected from delay discounting principles, we found that delayed shock was less effective as a punisher, as revealed by higher PSS breakpoints. However, this discounting effect was not equally distributed in the population of rats, and the introduction of a delay highlighted the existence of two populations: rats that were sensitive to immediate punishment were also sensitive to delayed shock, whereas rats that were resistant to immediate punishment showed strong temporal discounting of delayed punishment. Importantly, shock-sensitive rats suppressed responding even in subsequent non-punishment sessions, and they differed from shock-resistant rats in anxiety-like behavior, but not in sensitivity to pain. These results show that manipulation of temporal contingencies of punishment in the PSS procedure provides a valuable tool to identify individuals with a double vulnerability to addiction: low sensitivity to aversion and excessive discounting of negative future consequences. Conversely, the shock-sensitive population may provide a model of humans who are vulnerable to opportunity loss due to excessive anxiety.

Orbitofrontal and Prelimbic Cortices Serve Complementary Roles in Adapting Reward Seeking to Learned Anxiety

BACKGROUND

Anxiety is a common symptom of several mental health disorders and adversely affects motivated behaviors. Anxiety can emerge from associating risk of future harm while engaged in goal-guided actions. Using a recently developed behavioral paradigm to model this aspect of anxiety, we investigated the role of 2 cortical subregions, the prelimbic medial frontal cortex (PL) and lateral orbitofrontal cortex (lOFC), which have been implicated in anxiety and outcome expectation, in flexible representation of actions associated with harm risk.

METHODS

A seek-take reward-guided instrumental task design was used to train animals (N = 8) to associate the seek action with a variable risk of punishment. After learning, animals underwent extinction training for this association. Fiber photometry was used to measure and compare neuronal activity in the PL and lOFC during learning and extinction.

RESULTS

Animals increased action suppression in response to punishment contingencies. This increase dissipated after extinction training. These behavioral changes were associated with region-specific changes in neuronal activity. PL neuronal activity preferentially adapted to the threat of punishment, whereas lOFC activity adapted to safe aspects of the task. Moreover, correlated activity between these regions was suppressed during actions associated with harm risk, suggesting that these regions may guide behavior independently under anxiety.

CONCLUSIONS

These findings suggest that the PL and lOFC serve distinct but complementary roles in the representation of learned anxiety. This dissociation may provide a mechanism to explain how overlapping cortical systems are implicated in reward-guided action execution during anxiety.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Pavlovian Fear Conditioning Is More than You Think It Is

A common neuroscience application of Pavlovian fear conditioning is to manipulate neuron-type activity, pair a cue with foot shock, then measure cue-elicited freezing in a novel context. If the manipulation reduces freezing, the neuron type is implicated in Pavlovian fear conditioning. This application reduces Pavlovian fear conditioning to a single concept. In this Viewpoint, I describe experiments supporting the view that Pavlovian fear conditioning refers to three distinct concepts: procedure, process, and behavior. An experimenter controls procedure, observes behavior, but infers process. Distinguishing these concepts is essential because: (1) a shock-paired cue can engage numerous processes and behaviors; (2) experimenter decisions about procedure influence the processes engaged and behaviors elicited; and (3) many processes are latent, imbuing the cue with properties that only manifest outside of the original conditioning setting. This means we could understand the complete neural basis of freezing, yet know little about the neural basis of fear. Neuroscientists can choose to use a variety of procedures to study a diversity of processes and behaviors. Manipulating neuron-type activity in multiple procedures can reveal specific, general, or complex neuron-type contributions to cue-elicited processes and behaviors. The results will be a broader and more detailed neural basis of fear with greater relevance to the spectrum of symptoms defining anxiety and stressor-related disorders.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Extended access to fentanyl vapor self-administration leads to addiction-like behaviors in mice: Blood chemokine/cytokine levels as potential biomarkers

Rodent models are useful for understanding the mechanisms that underlie opioid addiction, but most preclinical studies have focused on rewarding and consummatory aspects of opioids without components of dependence-induced escalation of drug taking or seeking. We characterized several opioid-related behaviors in mice using a model of vaporized fentanyl self-administration. Male and female C57BL/6J mice were assigned to short-access (ShA; 1 h, nondependent) or long-access (LgA; 6 h, dependent) fentanyl vapor self-administration and subsequently tested in a battery of behavioral tests, followed by blood collection during withdrawal. Compared with mice in the ShA group, mice in the LgA group escalated their fentanyl intake, were more motivated to work to obtain the drug, exhibited greater hyperalgesia, and exhibited greater signs of naloxone-precipitated withdrawal. Principal component analysis indicated the emergence of two independent behavioral constructs: "intake/motivation" and "hyperalgesia/punished seeking." In mice in the LgA condition only, "hyperalgesia/punished seeking" was associated with plasma levels of proinflammatory interleukin-17 (IL-17), chemokine (C-C motif) ligand 4 (CCL-4), and tumor necrosis factor α (TNF-α). Overall, the results suggest that extended access to opioids leads to addiction-like behavior, and some constructs that are associated with addiction-like behavior may be associated with levels of the proinflammatory cytokines/chemokines IL-17, TNF-α, and CCL-4 in blood.

Effect of real-world fear on risky decision-making in medical school-based students: A quasi-experimental study

Objective: To explore the effect of real-world fear on risky decision-making under certainty and uncertainty. Methods: This quasi-experimental study enrolled non-psychology undergraduate volunteers aged between 17 and 20 years old from the Preventive Medical Institute medical school in Xi'an. Participants were randomly divided into two groups, and each group received a two-stage crossover design intervention (of a calm and fearful situation) and completed the tasks of risky decision-making under uncertainty (the balloon analog risk task: BART) and certainty (the Cambridge gambling task: CGT), respectively. The primary outcomes included the behavioral impulsivity measured by the BART value, and the speed of decision-making, the quality of decisions, the adventure index, behavioral impulsivity, and risk adjustment measured by CGT. The secondary outcome was the concentration of cortisol in the saliva. Results: A total of 60 questionnaires and data were obtained from 60 participants (28 males and 32 females, aged 19.55 ± 0.75). Compared with the calm situation, participants were more likely to have a lower BART value (p = 0.013), slower speed of decision-making (p < 0.05), and higher adventure index (p = 0.018) in the fearful situation. The quality of decisions (p = 0.189), behavioral impulsivity index (p = 0.182), and risk adjustment (p = 0.063) between subjects in the fearful and calm situations were comparable. Furthermore, the mean value of the adventure index of CGT in male subjects was significantly higher than that in female subjects (p < 0.05), and the cortisol concentration in saliva during the fearful situation was significantly higher compared to the calm situation (p < 0.05). Conclusion: Fear might reduce behavioral impulsivity under uncertainty, and increase the adventure index under certainty in risky decision-making. Risky behavior might be influenced by gender: under certainty in risky decision-making, men were more adventurous. Additionally, fear increased the secretion of cortisol in saliva.

A self-adjusting, progressive shock strength procedure to investigate resistance to punishment: Characterization in male and female rats

Indifference to harmful consequences is one of the main characteristics of compulsive behaviors and addiction. Animal models that provide a rapid and effective measure of resistance to punishment could be critical for the investigation of mechanisms underlying these maladaptive behaviors. Here, analogous to the progressive ratio (PR) procedure widely used to evaluate appetitive motivation as the response requirement is increased, we developed a self-adjusting, progressive shock strength (PSS) procedure. The PSS provides, within a single session, a break point that quantifies the propensity to work for a reward in spite of receiving electric footshock that progressively increases in duration. In both male and female rats, the PSS break point was sensitive to 1) hunger; and 2) changes in the qualitative, but not quantitative, incentive value of the reward. In systematic comparisons between PSS and PR procedures in the same rats, we found that both measures are sensitive to manipulations of motivational states, but they are not intercorrelated, suggesting that they measure overlapping but partially distinct processes. Importantly, the PSS procedure represents a refinement in the 3Rs principles of animal research because animals can control the strength of shock that they are willing to tolerate. This self-adjusting PSS procedure may represent a useful tool to investigate mechanisms underlying maladaptive behavior that persists in certain individuals despite harmful consequences.

Learning of probabilistic punishment as a model of anxiety produces changes in action but not punisher encoding in the dmPFC and VTA

Previously, we developed a novel model for anxiety during motivated behavior by training rats to perform a task where actions executed to obtain a reward were probabilistically punished and observed that after learning, neuronal activity in the ventral tegmental area (VTA) and dorsomedial prefrontal cortex (dmPFC) represent the relationship between action and punishment risk (Park and Moghaddam, 2017). Here, we used male and female rats to expand on the previous work by focusing on neural changes in the dmPFC and VTA that were associated with the learning of probabilistic punishment, and anxiolytic treatment with diazepam after learning. We find that adaptive neural responses of dmPFC and VTA during the learning of anxiogenic contingencies are independent from the punisher experience and occur primarily during the peri-action and reward period. Our results also identify peri-action ramping of VTA neural calcium activity, and VTA-dmPFC correlated activity, as potential markers for the anxiolytic properties of diazepam.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

Diet-induced inflammation in the anterior paraventricular thalamus induces compulsive sucrose-seeking

Overconsumption of palatable food may initiate neuroadaptive responses in brain reward circuitry that may contribute to eating disorders. Here we report that high-fat diet (HFD) consumption impedes threat-cue-induced suppression of sucrose-seeking in mice. This compulsive sucrose-seeking was due to enhanced cue-triggered neuronal activity in the anterior paraventricular thalamus (aPVT) resulting from HFD-induced microglia activation. Thus, metabolic inflammation in the aPVT produces an adaptive response to threat cues, leading to compulsive food-seeking.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Linking drug and food addiction via compulsive appetite

BACKGROUND AND PURPOSE

"Food addiction" is the subject of intense public and research interest. However, this nosology based on neurobehavioral similarities among obese individuals and patients with eating disorders and drug addiction remains controversial. We thus sought to determine which aspects of disordered eating are causally linked to preclinical models of drug addiction. We hypothesized that extensive drug histories, known to cause addiction-like brain changes and drug motivation in rats, would also cause addiction-like food motivation.

EXPERIMENTAL APPROACH

Rats underwent extensive cocaine, alcohol, caffeine or obesogenic diet histories, and were subsequently tested for punishment-resistant food self-administration or "compulsive appetite", as a measure of addiction-like food motivation.

KEY RESULTS

Extensive cocaine and alcohol (but not caffeine) histories caused compulsive appetite that persisted long after the last drug exposure. Extensive obesogenic diet histories also caused compulsive appetite, although neither cocaine nor alcohol histories caused excess calorie intake and bodyweight during abstinence. Hence, compulsive appetite and obesity appear to be dissociable, with the former sharing common mechanisms with preclinical drug addiction models.

CONCLUSION AND IMPLICATIONS

Compulsive appetite, as seen in subsets of obese individuals and patients with binge-eating disorder and bulimia nervosa (eating disorders that do not necessarily result in obesity), appears to epitomize "food addiction". Because different drug and obesogenic diet histories caused compulsive appetite, overlapping dysregulations in the reward circuits, which control drug and food motivation independently of energy homeostasis, may offer common therapeutic targets for treating addictive behaviors across drug addiction, eating disorders and obesity.

Neural correlates and determinants of approach-avoidance conflict in the prelimbic prefrontal cortex

The recollection of environmental cues associated with threat or reward allows animals to select the most appropriate behavioral responses. Neurons in the prelimbic (PL) cortex respond to both threat- and reward-associated cues. However, it remains unknown whether PL regulates threat-avoidance vs. reward-approaching responses when an animals’ decision depends on previously associated memories. Using a conflict model in which male Long–Evans rats retrieve memories of shock- and food-paired cues, we observed two distinct phenotypes during conflict: (1) rats that continued to press a lever for food (Pressers) and (2) rats that exhibited a complete suppression in food seeking (Non-pressers). Single-unit recordings revealed that increased risk-taking behavior in Pressers is associated with persistent food-cue responses in PL, and reduced spontaneous activity in PL glutamatergic (PL^GLUT) neurons during conflict. Activating PL^GLUT neurons in Pressers attenuated food-seeking responses in a neutral context, whereas inhibiting PL^GLUT neurons in Non-pressers reduced defensive responses and increased food approaching during conflict. Our results establish a causal role for PL^GLUT neurons in mediating individual variability in memory-based risky decision-making by regulating threat-avoidance vs. reward-approach behaviors.

Alcohol Seeking Under Risk of Punishment Is Associated With Activation of Cortical and Subcortical Brain Regions

In humans, stimuli associated with alcohol availability can provoke relapse during abstinence. In this study, we investigated the role of discriminative stimuli (DS) in the control of alcohol seeking in two types of behavioral tests. The first test examined the ability of an alcohol-associated DS to promote alcohol seeking (relapse) after punishment-imposed abstinence in the presence of a different DS. Following this, we tested whether the differentially associated DS can promote and suppress alcohol self-administration in a within-session discrimination task. During the within-session discrimination task, we also tested the rate of alcohol self-administration when two DS are presented in a compound. We first trained Long-Evans male rats (n = 24) to self-administer alcohol in the presence of one DS (reward-associated discriminative stimulus, rewDS) and then punished that behavior in the presence of a different DS (punishment-associated discriminative stimulus, punDS). On the test, we found that rats tested with the rewDS showed higher alcohol seeking than rats tested with the punDS. This result shows that a single Cue DS can promote alcohol seeking in a manner comparable to contexts. Subsequently, we trained 16 of these rats in a within-session trial-based discrimination task, comprised of intervening 2-min trials of rewDS, punDS, or conflict with rewDS and punDS in compound and a reduced probability of punishment. We found that alcohol self-administration is bi-directionally regulated by the rewDS and punDS. In conflict trials, alcohol self-administration was at a rate that was intermediate between the rewDS and punDS trials. In a final test, rats were presented with one of the three trial conditions and perfused for Fos immunohistochemistry. We found Fos expression was higher in the rats tested in the conflict condition in three interconnected sub-cortical brain regions. This study demonstrated the important role that alcohol-associated DS plays an important role in promoting relapse to alcohol seeking after punishment-imposed abstinence. We also implemented a within-session discrimination task that allows for the study of alcohol seeking under motivational conflict, which may be relevant for alcohol use despite negative consequences. The results from the Fos data suggest that higher alcohol seeking in approach-avoidance motivational conflict is associated with activation of sub-cortical regions but not cortical regions.

The lesser evil: Pavlovian-instrumental transfer & aversive motivation

While our understanding of appetitive motivation includes accounts of rich cognitive phenomena, such as choice, sensory-specificity and outcome valuation, the same is not true in aversive processes. A highly sophisticated picture has emerged of Pavlovian fear conditioning and extinction, but progress in aversive motivation has been somewhat limited to these fundamental behaviors. Many differences between appetitive and aversive stimuli permit different kinds of analyses; a widely used procedure in appetitive studies that can expand the scope of aversive motivation is Pavlovian-instrumental transfer (PIT). Recently, this motivational transfer effect has been used to examine issues pertaining to sensory-specificity and the nature of defensive control in avoidance learning. Given enduring controversies and unresolved criticisms surrounding avoidance research, PIT offers a valuable, well-controlled procedure with which to similarly probe this form of motivation. Furthermore, while avoidance itself can be criticized as artificial, PIT can be an effective model for how skills learned through avoidance can be practically applied to encounters with threatening or fearful stimuli and stress. Despite sensory-related challenges presented by the limited aversive unconditioned stimuli typically used in research, transfer testing can nevertheless provide valuable information on the psychological nature of this historically controversial phenomenon.

Punishment insensitivity in humans is due to failures in instrumental contingency learning

Punishment maximises the probability of our individual survival by reducing behaviours that cause us harm, and also sustains trust and fairness in groups essential for social cohesion. However, some individuals are more sensitive to punishment than others and these differences in punishment sensitivity have been linked to a variety of decision-making deficits and psychopathologies. The mechanisms for why individuals differ in punishment sensitivity are poorly understood, although recent studies of conditioned punishment in rodents highlight a key role for punishment contingency detection (Jean-Richard-Dit-Bressel et al., 2019). Here, we applied a novel ‘Planets and Pirates’ conditioned punishment task in humans, allowing us to identify the mechanisms for why individuals differ in their sensitivity to punishment. We show that punishment sensitivity is bimodally distributed in a large sample of normal participants. Sensitive and insensitive individuals equally liked reward and showed similar rates of reward-seeking. They also equally disliked punishment and did not differ in their valuation of cues that signalled punishment. However, sensitive and insensitive individuals differed profoundly in their capacity to detect and learn volitional control over aversive outcomes. Punishment insensitive individuals did not learn the instrumental contingencies, so they could not withhold behaviour that caused punishment and could not generate appropriately selective behaviours to prevent impending punishment. These differences in punishment sensitivity could not be explained by individual differences in behavioural inhibition, impulsivity, or anxiety. This bimodal punishment sensitivity and these deficits in instrumental contingency learning are identical to those dictating punishment sensitivity in non-human animals, suggesting that they are general properties of aversive learning and decision-making.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Motivation States for Physical Activity and Sedentary Behavior: Desire, Urge, Wanting, and Craving

To better explain daily fluctuations in physical activity and sedentary behavior, investigations of motivation are turning from social cognitive frameworks to those centered on affect, emotion and automaticity, such as the Affect and Health Behavior Framework (AHBF), Integrated Framework and Affective-Reflective Theory (ART). This shift has necessitated: (a) re-examination of older theories and their constructs, such as drives, needs and tensions and (b) an inspection of competing theories from other fields that also attempt to explain dynamic changes in health behaviors. The Dynamical Model of Desire, Elaborated Intrusion Theory and others commonly share with AHBF the idea that human behavior is driven strongly by desires and/or the similar concepts of wants, urges, and cravings. These affectively-charged motivation states (ACMS) change quickly and may better explain physical activity behavior from one moment to the next. Desires for movement predominantly derive from negative but also positive reinforcement. Data from clinical populations with movement dysfunction or psychiatric disorders provides further evidence of these drivers of movement. Those with Restless Legs Syndrome, akathisia, tic disorders and exercise dependence all report strong urges to move and relief when it is accomplished. Motor control research has identified centers of the brain responsible for wants and urges for muscular movement. Models elaborated herein differentiate between wants, desires, urges and cravings. The WANT model (Wants and Aversions for Neuromuscular Tasks) conceptualizes desires for movement and rest as varying by magnitude, approach or avoidance-orientation (wants versus aversions) and as occupying independent dimensions instead of opposite ends of the same axis. For instance, one hypothetically might be in a state of both high desire for movement and rest simultaneously. Variations in motivation states to move and rest may also be associated with various stress states, like freezing or fight and flight. The first validated instrument to measure feelings of desire/want for movement and rest, the CRAVE Scale (Cravings for Rest and Volitional Energy Expenditure) is already shedding light on the nature of these states. With these advances in theory, conceptual modeling and instrumentation, future investigations may explore the effects of desires and urges for movement and sedentary behavior in earnest.

Individual differences in causal structures inferred during feature negative learning

Traditional associative learning theories predict that training with feature negative (A+/AB-) contingencies leads to the feature B acquiring negative associative strength and becoming a conditioned inhibitor (i.e., prevention learning). However, feature negative training can sometimes result in negative occasion setting, where B modulates the effect of A. Other studies suggest that participants learn about configurations of cues rather than their individual elements. In this study, we administered simultaneous feature negative training to participants in an allergist causal learning task and tested whether evidence for these three types of learning (prevention, modulation, configural) could be captured via self-report in the absence of any procedural manipulation. Across two experiments, we show that only a small subset of participants endorse the prevention option, suggesting that traditional associative models that predict conditioned inhibition do not completely capture how humans learn about negative contingencies. We also show that the degree of transfer in a summation test corresponds to the implied causal structure underlying conditioned inhibition, occasion-setting, and configural learning, and that participants are only partially sensitive to explicit hints about causal structure. We conclude that feature negative training is an ambiguous causal scenario that reveals individual differences in the representation of inhibitory associations, potentially explaining the modest group-level inhibitory effects often found in humans.

A molecular insight into the dissociable regulation of associative learning and motivation by the synaptic protein neuroligin-1

Background

In a changing environment, a challenge for the brain is to flexibly guide adaptive behavior towards survival. Complex behavior and the underlying neural computations emerge from the structural components of the brain across many levels: circuits, cells, and ultimately the signaling complex of proteins at synapses. In line with this logic, dynamic modification of synaptic strength or synaptic plasticity is widely considered the cellular level implementation for adaptive behavior such as learning and memory. Predominantly expressed at excitatory synapses, the postsynaptic cell-adhesion molecule neuroligin-1 (Nlgn1) forms trans-synaptic complexes with presynaptic neurexins. Extensive evidence supports that Nlgn1 is essential for NMDA receptor transmission and long-term potentiation (LTP), both of which are putative synaptic mechanisms underlying learning and memory. Here, employing a comprehensive battery of touchscreen-based cognitive assays, we asked whether impaired NMDA receptor transmission and LTP in mice lacking Nlgn1 does in fact disrupt decision-making. To this end, we addressed two key decision problems: (i) the ability to learn and exploit the associative structure of the environment and (ii) balancing the trade-off between potential rewards and costs, or positive and negative utilities of available actions.

Results

We found that the capacity to acquire complex associative structures and adjust learned associations was intact. However, loss of Nlgn1 alters motivation leading to a reduced willingness to overcome effort cost for reward and an increased willingness to exert effort to escape an aversive situation. We suggest Nlgn1 may be important for balancing the weighting on positive and negative utilities in reward-cost trade-off.

Conclusions

Our findings update canonical views of this key synaptic molecule in behavior and suggest Nlgn1 may be essential for regulating distinct cognitive processes underlying action selection. Our data demonstrate that learning and motivational computations can be dissociated within the same animal model, from a detailed behavioral dissection. Further, these results highlight the complexities in mapping synaptic mechanisms to their behavioral consequences, and the future challenge to elucidate how complex behavior emerges through different levels of neural hardware.

Medial Orbitofrontal Cortex Regulates Instrumental Conditioned Punishment, but not Pavlovian Conditioned Fear

Bidirectionally aberrant medial orbitofrontal cortical (mOFC) activity has been consistently linked with compulsive disorders and related behaviors. Although rodent studies have established a causal link between mOFC excitation and compulsive-like actions, no such link has been made with mOFC inhibition. Here, we use excitotoxic lesions of mOFC to investigate its role in sensitivity to punishment; a core characteristic of many compulsive disorders. In our first experiment, we demonstrated that mOFC lesions prevented rats from learning to avoid a lever that was punished with a stimulus that coterminated with footshock. Our second experiment demonstrated that retrieval of punishment learning is also somewhat mOFC-dependent, as lesions prevented the extended retrieval of punishment contingencies relative to shams. In contrast, mOFC lesions did not prevent rats from reacquiring the ability to avoid a punished lever when it was learned prior to lesions being administered. In both experiments, Pavlovian fear conditioning to the stimulus was intact for all animals. Together, these results reveal that the mOFC regulates punishment learning and retrieval in a manner that is separate from any role in Pavlovian fear conditioning. These results imply that aberrant mOFC activity may contribute to the punishment insensitivity that is observed across multiple compulsive disorders.