The impact of emotional valence on generalization gradients

Generalization enables individuals to respond to novel stimuli based on previous experiences. The degree to which organisms respond is determined by their physical resemblance to the original conditioned stimulus (CS+), with a stronger response elicited by more similar stimuli, resulting in similarity-based generalization gradients. Recent research showed that cognitive or conceptual dimensions also result in gradients similar to those observed with manipulations of physical dimensions. Such findings suggest that attributes beyond physical similarity play a role in shaping generalization gradients. However, despite its adaptive relevance for survival, there is no study exploring the effectiveness of affective dimensions in shaping generalization gradients. In two experiments (135 Spanish and 150 English participants, respectively), we used an online predictive learning task, in which different stimuli (words and Gabor patches) were paired with the presence - or absence - of a fictitious shock. After training, we assessed whether valence (i.e., hedonic experience) conveyed by words shape generalization gradients. In Experiment 1, the outcome expectancy decreased monotonically with variations in valence of Spanish words, mirroring the gradient obtained with the physical dimension (line orientation). In Experiment 2, conducted with English words, a similar gradient was observed when non-trained (i.e., generalization) words varied along the valence dimension, but not when words were of neutral valence. The consistency of these findings across two different languages strengthens the reliability and validity of the affective dimension as a determinant of generalization gradients. Furthermore, our data highlight the importance of considering the role of affective features in generalization responses, advancing the interplay between emotion, language, and learning.

Contiguity and overshadowing interactions in the rapid-streaming procedure

When multiple cues are associated with the same outcome, organisms tend to select between the cues, with one revealing greater behavioral control at the expense of the others (i.e., cue competition). However, non-human and human studies have not always observed this competition, creating a puzzling scenario in which the interaction between cues can result in competition, no interaction, or facilitation as a function of several learning parameters. In five experiments, we assessed whether temporal contiguity and overshadowing effects are reliably observed in the streamed-trial procedure, and whether there was an interaction between them. We anticipated that weakening temporal contiguity (ranging from 500 to 1,000 ms) should attenuate competition. Using within-subject designs, participants experienced independent series of rapid streams in which they had to learn the relationship between visual cues (presented either alone or with another cue) and an outcome, with the cue-outcome pairings being presented with either a delay or trace relationship. Across experiments, we observed overshadowing (Experiments 1, 2, 4, and 5) and temporal contiguity effects (Experiments 2, 3, and 4). Despite the frequent occurrence of both effects, we did not find that trace conditioning abolished competition between cues. Overall, these results suggest that the extent to which contiguity determines cue interactions depends on multiple variables, some of which we address in the General discussion.

The effects of goal-landmark distance on overshadowing: A replication in humans (Homo sapiens) of Goodyear and Kamil (2004)

Goodyear and Kamil (2004) assessed the ability of Clark's nutcrackers to find buried food based on a cross-shaped array of landmarks at different distances from the goal. Their findings suggested that proximal landmarks overshadowed learning about distal landmarks, and this was attenuated when assessing the effect of distal landmarks on learning about proximal landmarks. In this study, we aimed to replicate their findings in human spatial navigation by using a virtual environment. Three groups of participants were trained in an open environment featuring orientation cues, and they had to find a hidden goal with reference to four landmarks that were arranged in the shape of a cross and placed at different distances from the goal. Two of the four landmark distances were common across all three groups to allow a comparison of the extent of overshadowing under comparable conditions. Following training, all participants were tested with each of the four landmarks individually. Consistent with the results in birds, we observed better performance in the groups with more distal landmarks, suggesting that overshadowing was greater in the groups with closer landmarks and thus dependent on the spatial distance between the landmarks and the goal. Landmarks near the goal more effectively overshadowed landmarks far from the goal. A second experiment, in which landmarks and orientation cues were misaligned in order to prevent the use of a straightforward solution to the task, replicated the results. The results are discussed in terms of a modification of Pearce's configural model. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Further evidence for the role of temporal contiguity as a determinant of overshadowing

Three experiments explored whether weakening temporal contiguity between auditory cues and an aversive outcome attenuated cue competition in an avoidance learning task with human participants. Overall, with strong temporal contiguity between auditory cues and the outcome during training (the offset of the predictive auditory signals concurred with the onset of the outcome), the target cue trained as part of a compound yielded less avoidance behaviour than the control cue trained alone, an instance of overshadowing. However, weakening temporal contiguity during training (inserting a 5-s trace) attenuated overshadowing, resulting in similar avoidance behaviour in response to the control and target cues. These results provide evidence that, as predicted by a recent modification of Pearce's configural theory, temporal contiguity is critical for determining cue competition.

Social overshadowing: Revisiting cue-competition in social interactions

In a large variety of contexts, it is essential to use the available information to extract patterns and behave accordingly. When it comes to social interactions for instance, the information gathered about interaction partners across multiple encounters (e.g., trustworthiness) is crucial in guiding one's own behavior (e.g., approach the trustworthy and avoid the untrustworthy), a process akin to trial-by-trial learning. Building on associative learning and social cognition literatures, the present research adopts a domain-general approach to learning and explores whether the principles underlying associative learning also govern learning in social contexts. In particular, we examined whether overshadowing, a well-established cue-competition phenomenon, impacts learning of the cooperative behaviors of unfamiliar interaction partners. Across three experiments using an adaptation of the iterated Trust Game, we consistently observed a 'social overshadowing' effect, that is, a better learning about the cooperative tendencies of partners presented alone compared to those presented in a pair. This robust effect was not modulated by gender stereotypes or beliefs about the internal communication dynamics within a pair of partners. Drawing on these results, we argue that examining domain-general learning processes in social contexts is a useful approach to understanding human social cognition.

Category relevance attenuates overshadowing in human predictive learning

In situations in which multiple predictors anticipate the presence or absence of an outcome, cues compete to anticipate the outcome, resulting in a loss of associative strength compared to control conditions without additional cues. Critically, there are multiple factors modulating the magnitude and direction of such competition, although in some scenarios the effect of these factors remains unexplored. We sought to assess whether the relative salience of the elements in a compound of cues modulates the magnitude of the overshadowing effect in human predictive learning. Two separable categories (i.e., colors and symbols) were used in a predictive learning task. In Experiment 1, different groups of participants were granted with different time of exposure to a compound of cues belonging to different categories (color and symbol) to evaluate potential differences in the magnitude of overshadowing. Furthermore, we used posttest questionnaires to assess whether participants used either only one or both categories during training, and assessed if this impacted the magnitude of overshadowing. In general, overshadowing was not modulated by the time of exposition, except in the case of very short time of exposition with prominent learning about the most salient category. In Experiment 2, the relative salience of a category was biased via prior experience either with a biconditional discrimination or attending only the relevant category (either color or symbol). The previously relevant category was less prone to overshadowing, but not the alternative one. Results are discussed in light of attentional and configural theories of associative learning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Genetic and environmental influences on one-trial conditioned context aversion in mice

Anticipatory nausea (AN) is caused by an association between contextual cues and the experience of nausea (the side effects of chemotherapy or radiation treatment) and it develops predominantly in female patients undergoing chemotherapy. Preclinical studies in rodents show that the administration of an illness-inducing agent in the presence of novel contextual cues can cause conditioned context aversion (CCA) and this has been proposed to model AN. The literature also suggests that brief pre-exposure to a novel context prior to shock delivery is critical in the development of contextual fear conditioning in rodents (a phenomenon known as Immediate Shock Deficit), but this has not been assessed in CCA. The aim of present study was to develop a CCA paradigm to assess this in outbred (CD1) and inbred (C57BL/6J) mice and evaluate potential sex differences. The results revealed that a single conditioning trial in which a distinctive context was paired with LiCl-induced illness was sufficient to elicit a conditioned response in both female and male CD1 outbred mice, but not in C57BL/6J inbred mice. In addition, CCA was facilitated when animals had prior experience with the context. Finally, outbred female mice showed longer and more robust retention of CCA than male mice, which parallels clinical findings. The results indicate the importance of using CD1 outbred mice as an animal model of AN as well as examining sex differences in the CCA paradigm. Similar findings in humans encourage the future use of this novel CCA preclinical mouse model.

Conditioned Place Avoidance in the Planaria Schmidtea mediterranea: A Pre-clinical Invertebrate Model of Anxiety-Related Disorders

The objective of the present study was to develop a model of avoidance learning and its extinction in planarians (Schmidtea mediterranea). Based on previous experiments showing conditioned place preference, we developed a procedure to investigate conditioned place avoidance (CPA) using shock as an unconditioned stimulus (US) and an automated tracking system to record the animals' behaviour. In Experiment 1, we assessed the unconditioned properties of different shock intensities by measuring post shock activity. In two subsequent experiments we investigated CPA using different designs, surfaces as conditioned stimuli (CSs; rough and smooth), and different US intensities (5V and 10V). In general, we observed the successful development of CPA. However, CPA was stronger with higher shock intensities, and we found that, in our preparation, a rough surface is best at entering into an association with the shock than a smooth surface. Finally, we also observed extinction of CPA. The evidence of CPA and its extinction in flatworms validates the planaria as a pre-clinical model for the study of avoidance learning, a hallmark of anxiety disorders.

Signal detection analysis of contingency assessment: Associative interference and nonreinforcement impact cue-outcome contingency sensitivity, whereas cue density affects bias

In a signal detection theory approach to associative learning, the perceived (i.e., subjective) contingency between a cue and an outcome is a random variable drawn from a Gaussian distribution. At the end of the sequence, participants report a positive cue-outcome contingency provided the subjective contingency is above some threshold. Some researchers have suggested that the mean of the subjective contingency distributions and the threshold are controlled by different variables. The present data provide empirical support for this claim. In three experiments, participants were exposed to rapid streams of trials at the end of which they had to indicate whether a target outcome O1 was more likely following a target cue X. Interfering treatments were incorporated in some streams to impend participants' ability to identify the objective X-O1 contingency: interference trials (X was paired with an irrelevant outcome O2), nonreinforced trials (X was presented alone), plus control trials (an irrelevant cue W was paired with O2). Overall, both interference and nonreinforced trials impaired participants' sensitivity to the contingencies as measured by signal detection theory's d', but they also enhanced detection of positive contingencies through a cue density effect, with nonreinforced trials being more susceptible to this effect than interference trials. These results are explicable if one assumes interference and nonreinforced trials impact the mean of the associative strength distribution, while the cue density influences the threshold. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Temporal and spatial contiguity are necessary for competition between events

Over the last 50 years, cue competition phenomena have shaped theoretical developments in animal and human learning. However, recent failures to observe competition effects in standard conditioning procedures, as well as the lengthy and ongoing debate surrounding cue competition in the spatial learning literature, have cast doubts on the generality of these phenomena. In the present study, we manipulated temporal contiguity between simultaneously trained predictors and outcomes (Experiments 1-4), and spatial contiguity between landmarks and goals in spatial learning (Supplemental Experiments 1 and 2; Experiment 5). Across different parametric variations, we observed overshadowing when temporal and spatial contiguity were strong, but no overshadowing when contiguity was weak. Thus, across temporal and spatial domains, we observed that contiguity is necessary for competition to occur, and that competition between cues presented simultaneously during learning is absent when these cues were either spatially or temporally discontiguous from the outcome. Consequently, we advance a model in which the contiguity between events is accounted for and which explains these results and reconciles the previously contradictory findings observed in spatial learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Nicotine chronic tolerance development and withdrawal in the planaria (Schmidtea mediterranea)

Chronic nicotine exposure reduces sensitivity to the effects of nicotine, which then results in behavioural changes and tolerance development. In the planaria, a valuable first-stage preclinical model for addictive behaviour, acute nicotine administration has been shown to steadily alter the motility of the animals, a result that has been interpreted as evidence of tolerance and withdrawal effects; however, chronic exposure - typically regarded as a condition for the development of tolerance - and the role of the contextual cues have not been systematically assessed. The present study assessed the acute and chronic effects of nicotine on the motility of planarians (Schmidtea mediterranea). The animals in the experimental groups received long chronic exposure to nicotine (ten daily 30 min exposures); a control group was exposed to water in the same context but in the absence of the drug. The motility of the animals was closely monitored on every exposure. Following this phase, all the animals were subject to three different tests: in the presence of the exposure context (without the drug, Test 1); in the presence of nicotine in the exposure context (Test 2); and in the presence of the drug in a novel context (Test 3). Exposure to nicotine consistently reduced motility; the motility in the presence of nicotine increased with repeated exposures to the drug, an instance of tolerance development. Tolerance development was dependent on nicotinic receptor activation, because it was blocked by the co-administration of mecamylamine. However, this tolerance was found to be independent of the contextual cues where the effects of the drug had been experienced. The results are discussed by reference to the existent theories of tolerance development to drugs.

Short- and long-term habituation of photonegative and exploratory responses in the flatworm planaria (Dugesia)

Two experiments address the habituation of photonegative and exploratory responses in the flatworm planaria (Dugesia). Planarians possess a well-documented photonegative response; Experiment 1 showed that repeated exposures to a bright light source with short inter trial intervals (ITIs) within 1 experimental session gradually weakens the unconditioned photonegative response. In addition, it was found that presentation of an unexpected arousal-increasing stimulus (dropped water or a shock) temporarily re-establishes the photonegative response. Experiment 2 addressed the development of long-term habituation; we recorded the locomotor activity of the animals exposed to an inescapable bright light. Experiments 2A and 2B showed that planarians develop long-term habituation but only when they were trained in relatively novel contexts; when they were trained in familiar contexts (in surfaces similar to the ones in the home) the development of habituation was severely impaired. The results are discussed by reference to the theory of short- and long-term habituation put forward by Allan R. Wagner (Wagner, 1976), highlighting the impact that this theory has had in the research of invertebrate learning. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Heat shock disrupts expression of excitatory and extinction memories in planaria: Interaction with amount of exposure

In planarians, as seen in rodents, natural reinforcers (sucrose) and drugs of abuse support Conditioned Place Preference (CPP), which is a form of Pavlovian learning to examine the rewarding effects of natural reinforcers and drugs of abuse. Using this preparation, we have previously observed acquisition, extinction and reinstatement of sucrose CPP. In the present experiments, we used planaria to investigate the amnestic effects of Heat Shock (HS, a known stressor in planaria) following different amounts of CPP extinction sessions. Experiment 1 showed that planarians developed a CPP response to a sucrose-paired surface. Heat shock, when given in conjunction with exposure to the sucrose-paired surface, produced amnesia as assessed by a subsequent sucrose reinstatement test. We interpreted that the amnesic effect of HS was due to HS affecting the dominant excitatory memory at the time of HS exposure. Thus, we hypothesized that after extensive extinction training (10 exposures), HS would lead to recovery from extinction (when the new inhibitory memory is dominant at the time of HS exposure). Experiment 2 explored this possibility and showed that given HS following 10 extinction sessions had no amnestic effect on the excitatory CPP response. In Experiment 3, we hypothesized that 16 extinction sessions would produce a stronger (and hence dominant) extinction inhibitory trace, which then would be vulnerable to HS. We observed that HS impaired the expression of the extinction memory following 16 exposures. These results reveal different effects of HS on CPP memories depending on the amount of extinction, and are fully consistent with the literature using rodents and humans. In addition, they suggest that planaria is a promising pre-clinical model to assess fundamental memory processes.

Delayed rewards facilitate habit formation

The transition from goal-directed to habitual forms of instrumental behavior is determined by variables such as the amount of training, schedules of reinforcement, the availability of choices, and exposure to drugs of abuse. Less is known about the control of goal-directed behavior when reinforcement is delayed rather than immediate. In these experiments, we investigated in rats the role of response-outcome contiguity on the control of goal-directed action, assessed through satiety-specific outcome devaluation tests. In Experiment 1 using a within-subjects design we observed goal-directed behavior after 6 days of FR1 training when the outcome was presented immediately following the lever press, but not when it was delayed for 20 s, revealing habit formation with delayed outcomes. Experiment 2 revealed that the habitual control observed with 20-s delays of reinforcement can be prevented if, immediately before each instrumental training session, the rats were exposed to the experimental context in the absence of both the lever and reinforcement. In summary, these experiments suggest that response-outcome contiguity plays an important role in the control of goal-directed actions and habits. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Excitatory second-order conditioning using a backward first-order conditioned stimulus: A challenge for prediction error reduction

Prével and colleagues reported excitatory learning with a backward conditioned stimulus (CS) in a conditioned reinforcement preparation. Their results add to existing evidence of backward CSs sometimes being excitatory and were viewed as challenging the view that learning is driven by prediction error reduction, which assumes that only predictive (i.e., forward) relationships are learned. The results instead were consistent with the assumptions of both Miller’s Temporal Coding Hypothesis and Wagner’s Sometimes Opponent Processes (SOP) model. The present experiment extended the conditioned reinforcement preparation developed by Prével et al. to a backward second-order conditioning preparation, with the aim of discriminating between these two accounts. We tested whether a second-order CS can serve as an effective conditioned reinforcer, even when the first-order CS with which it was paired is a backward CS that elicits no responding. Evidence of conditioned reinforcement was found, despite no conditioned response (CR) being elicited by the first-order backward CS. The evidence of second-order conditioning in the absence of excitatory conditioning to the first-order CS is interpreted as a challenge to SOP. In contrast, the present results are consistent with the Temporal Coding Hypothesis and constitute a conceptual replication in humans of previous reports of excitatory second-order conditioning in rodents with a backward CS. The proposal is made that learning is driven by “discrepancy” with prior experience as opposed to “ prediction error.”

Competition and facilitation in compound conditioning

Despite the generality and theoretical relevance of cue competition phenomena such as blocking and overshadowing, recent findings suggest that these observations may be due to some degree of publication bias, and that we lack insight into the boundary conditions of these phenomena. The present commentary does not question the existence of cue competition phenomena. Rather, I review findings showing that 3 variables, namely (a) relative stimulus duration, (b) contingency, and (c) contiguity parametrically determine not only whether cue competition is observed, but also whether no cue interaction, or cue facilitation occur. I discuss theoretical interpretations and implications of these findings, which may provide illuminating insights into the generality and functional significance of the commonly cited "principles of learning." (PsycINFO Database Record

Thalamic inputs to dorsomedial striatum are involved in inhibitory control: evidence from the five-choice serial reaction time task in rats

RATIONALE

Corticostriatal circuits are widely implicated in the top-down control of attention including inhibitory control and behavioural flexibility. However, recent neurophysiological evidence also suggests a role for thalamic inputs to striatum in behaviours related to salient, reward-paired cues.

OBJECTIVES

Here, we used designer receptors exclusively activated by designer drugs (DREADDs) to investigate the role of parafascicular (Pf) thalamic inputs to the dorsomedial striatum (DMS) using the five-choice serial reaction time task (5CSRTT) in rats.

METHODS

The 5CSRTT requires sustained attention in order to detect spatially and temporally distributed visual cues and provides measures of inhibitory control related to impulsivity (premature responses) and compulsivity (perseverative responses). Rats underwent bilateral Pf injections of the DREADD vector, AAV2-CaMKIIa-HA-hM4D(Gi)-IRES-mCitrine. The DREADD agonist, clozapine N-oxide (CNO; 1 μl bilateral; 3 μM) or vehicle, was injected into DMS 1 h before behavioural testing. Task parameters were manipulated to increase attention load or reduce stimulus predictability respectively.

RESULTS

We found that inhibition of the Pf-DMS projection significantly increased perseverative responses when stimulus predictability was reduced but had no effect on premature responses or response accuracy, even under increased attentional load. Control experiments showed no effects on locomotor activity in an open field.

CONCLUSIONS

These results complement previous lesion work in which the DMS and orbitofrontal cortex were similarly implicated in perseverative responses and suggest a specific role for thalamostriatal inputs in inhibitory control.

A comparison of behavioral and pharmacological interventions to attenuate reactivated fear memories

Two experiments using rats in a contextual fear memory preparation compared two approaches to reduce conditioned fear: (1) pharmacological reconsolidation blockade and (2) reactivation-plus-extinction training. In Experiment 1, we explored different combinations of reactivation-plus-extinction parameters to reduce conditioned fear and attenuate reacquisition. In Experiment 2, memory reactivation was followed by extinction training or administration of midazolam (MDZ) (vs. vehicle) to reduce conditioned fear and attenuate spontaneous recovery. We found both treatments to be equally effective in both experiments. This study suggests that parameters leading to memory destabilization during reactivation are critical to observe long-lasting effects of MDZ or reactivation plus extinction.

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

Prediction error and trace dominance determine the fate of fear memories after post-training manipulations

Different mnemonic outcomes have been observed when associative memories are reactivated by CS exposure and followed by amnestics. These outcomes include mere retrieval, destabilization-reconsolidation, a transitional period (which is insensitive to amnestics), and extinction learning. However, little is known about the interaction between initial learning conditions and these outcomes during a reinforced or nonreinforced reactivation. Here we systematically combined temporally specific memories with different reactivation parameters to observe whether these four outcomes are determined by the conditions established during training. First, we validated two training regimens with different temporal expectations about US arrival. Then, using Midazolam (MDZ) as an amnestic agent, fear memories in both learning conditions were submitted to retraining either under identical or different parameters to the original training. Destabilization (i.e., susceptibly to MDZ) occurred when reactivation was reinforced, provided the occurrence of a temporal prediction error about US arrival. In subsequent experiments, both treatments were systematically reactivated by nonreinforced context exposure of different lengths, which allowed to explore the interaction between training and reactivation lengths. These results suggest that temporal prediction error and trace dominance determine the extent to which reactivation produces the different outcomes.

Functional neuroimaging of avoidance habits in obsessive-compulsive disorder

OBJECTIVE

The purpose of this study was to determine the neural correlates of excessive habit formation in obsessive-compulsive disorder (OCD). The authors aimed to test for neurobiological convergence with the known pathophysiology of OCD and to infer, based on abnormalities in brain activation, whether these habits arise from dysfunction in the goal-directed or habit system.

METHOD

Thirty-seven OCD patients and 33 healthy comparison subjects learned to avoid shocks while undergoing a functional MRI scan. Following four blocks of training, the authors tested whether the avoidance response had become a habit by removing the threat of shock and measuring continued avoidance. Task-related differences in brain activity in three regions of interest (the caudate, the putamen, and the medial orbitofrontal cortex) were tested at a statistical threshold set at <0.05 (family-wise-error corrected).

RESULTS

Excessive habit formation in OCD patients, which was associated with hyperactivation in the caudate, was observed. Activation in this region was also associated with subjective ratings of increased urge to perform habits. The OCD group, as a whole, showed hyperactivation in the medial orbitofrontal cortex during the acquisition of avoidance; however, this did not relate directly to habit formation.

CONCLUSIONS

OCD patients exhibited excessive habits that were associated with hyperactivation in a key region implicated in the pathophysiology of OCD, the caudate nucleus. Previous studies indicate that this region is important for goal-directed behavior, suggesting that habit-forming biases in OCD may be a result of impairments in this system, rather than differences in the buildup of stimulus-response habits themselves.

Avoidance behavior: a free-operant lever-press avoidance task for the assessment of the effects of safety signals

This protocol details a free-operant avoidance paradigm that has been developed to evaluate the relative contribution of different sources of reinforcement of avoidance behavior that may play an important role in the development and maintenance of human anxiety disorders. The task enables the assessment of the effects of safety cues that signal a period free from danger on lever-press avoidance behavior. Avoidance behavior trained using this protocol has been shown to be sensitive to both behavioral and pharmacological manipulations and has been optimized so that it takes approximately 1 month for rats to perform at high levels of stable avoidance responding.

The dopamine D2/D3 receptor agonist quinpirole increases checking-like behaviour in an operant observing response task with uncertain reinforcement: a novel possible model of OCD

Excessive checking is a common, debilitating symptom of obsessive-compulsive disorder (OCD). In an established rodent model of OCD checking behaviour, quinpirole (dopamine D2/3-receptor agonist) increased checking in open-field tests, indicating dopaminergic modulation of checking-like behaviours. We designed a novel operant paradigm for rats (observing response task (ORT)) to further examine cognitive processes underpinning checking behaviour and clarify how and why checking develops. We investigated i) how quinpirole increases checking, ii) dependence of these effects on D2/3 receptor function (following treatment with D2/3 receptor antagonist sulpiride) and iii) effects of reward uncertainty. In the ORT, rats pressed an 'observing' lever for information about the location of an 'active' lever that provided food reinforcement. High- and low-checkers (defined from baseline observing) received quinpirole (0.5mg/kg, 10 treatments) or vehicle. Parametric task manipulations assessed observing/checking under increasing task demands relating to reinforcement uncertainty (variable response requirement and active-lever location switching). Treatment with sulpiride further probed the pharmacological basis of long-term behavioural changes. Quinpirole selectively increased checking, both functional observing lever presses (OLPs) and non-functional extra OLPs (EOLPs). The increase in OLPs and EOLPs was long-lasting, without further quinpirole administration. Quinpirole did not affect the immediate ability to use information from checking. Vehicle and quinpirole-treated rats (VEH and QNP respectively) were selectively sensitive to different forms of uncertainty. Sulpiride reduced non-functional EOLPs in QNP rats but had no effect on functional OLPs. These data have implications for treatment of compulsive checking in OCD, particularly for serotonin-reuptake-inhibitor treatment-refractory cases, where supplementation with dopamine receptor antagonists may be beneficial.

Enhanced avoidance habits in obsessive-compulsive disorder

BACKGROUND

Obsessive-compulsive disorder (OCD) is a psychiatric condition that typically manifests in compulsive urges to perform irrational or excessive avoidance behaviors. A recent account has suggested that compulsivity in OCD might arise from excessive stimulus-response habit formation, rendering behavior insensitive to goal value. We tested if OCD patients have a bias toward habits using a novel shock avoidance task. To explore how habits, as a putative model of compulsivity, might relate to obsessions and anxiety, we recorded measures of contingency knowledge, explicit fear, and physiological arousal.

METHODS

Twenty-five OCD patients and 25 control subjects completed a shock avoidance task designed to induce habits through overtraining, which were identified using goal-devaluation. The relationship between habitual behavior, erroneous cognitions, and physiological arousal was assessed using behavior, questionnaires, subjective report, and skin conductance responses.

RESULTS

A devaluation sensitivity test revealed that both groups could inhibit unnecessary behavioral responses before overtraining. Following overtraining, OCD patients showed greater avoidance habits than control subjects. Groups did not differ in conditioned arousal (skin conductance responses) at any stage. Additionally, groups did not differ in contingency knowledge or explicit ratings of shock expectancy following the habit test. Habit responses were associated with a subjective urge to respond.

CONCLUSIONS

These data indicate that OCD patients have a tendency to develop excessive avoidance habits, providing support for a habit account of OCD. Future research is needed to fully characterize the causal role of physiological arousal and explicit fear in habit formation in OCD.

The functions of contexts in associative learning

Although contexts play many roles during training and also during testing, over the last four decades theories of learning have predominantly focused on one or the other of the two families of functions served by contexts. In this selective review, we summarize recent data concerning these two functions and their interrelationship. The first function is similar to that of discrete cues, and allows contexts to elicit conditioned responses and compete with discrete events for behavioral control. The second function is modulatory, and similar to that of discrete occasion setters in that in this role contexts do not elicit conditioned responses by themselves, but rather modulate instrumental responding or responding to Pavlovian cues. We first present evidence for these two functions, and then suggest that the spacing of trials, amount of training, and contiguity are three determinants of the degree to which the context will play each function. We also conclude that these two functions are not mutually exclusive, and that future research would benefit from identifying the conditions under which their functions dominate behavioral control. We close by discussing some misconceptions concerning contexts. This article is part of a Special Issue entitled: SQAB 2013: Contextual Con.

Memory destabilization is critical for the success of the reactivation-extinction procedure

It has been suggested that, unlike pure extinction which typically results in the return of the fear response under a variety of circumstances, memory reactivation followed by extinction can attenuate the reemergence of conditioned fear. The reactivation-extinction procedure has attracted the attention of basic and clinical researchers due to its potential clinical value for the treatment of psychiatric conditions, such as anxiety and drug abuse disorders. However, mixed results have been achieved so far in replicating and understanding this paradigm. It has been proposed that memory destabilization could be critical in this sense. Using contextual fear conditioning in rats and midazolam as an amnesic agent, we first determined what reactivation conditions are necessary to destabilize the mnemonic trace. After establishing the conditions for memory destabilization, a series of experiments was conducted to determine if destabilization is critical for the success of the reactivation-extinction procedure. Data confirmed the importance of memory destabilization prior to extinction inside the reconsolidation window to attenuate spontaneous recovery and retard reacquisition of conditioned fear. The present report offers a candidate explanation of the discrepancy in results obtained with the reactivation-extinction procedure by different laboratories.

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

Linking ADHD, impulsivity, and drug abuse: a neuropsychological perspective

In this chapter, we consider the relevance of impulsivity as both a psychological construct and endophenotype underlying attention-deficit/hyperactivity disorder (ADHD) and drug addiction. The case for executive dysfunction in ADHD and drug addiction is critically reviewed in the context of dissociable cognitive control processes mediated by the dorsolateral prefrontal cortex (DLPFC), the orbital and ventral medial prefrontal cortex (VMPFC). We argue that such neuroanatomical divisions within the prefrontal cortex are likely to account for the multidimensional basis of impulsivity conceptually categorized in terms of "motoric" and "choice" impulsivity. The relevance of this distinction for the etiology of ADHD and drug addiction is integrated within a novel theoretical framework. This scheme embraces animal learning theory to help explain the heterogeneity of impulse control disorders, which are exemplified by ADHD as a vulnerability disorder for drug addiction.

Impulsive behaviour induced by both NMDA receptor antagonism and GABAA receptor activation in rat ventromedial prefrontal cortex

RATIONALE

Previous work has demonstrated a profound effect of N-methyl-D: -aspartic acid receptor (NMDAR) antagonism in the infralimbic cortex (IL) to selectively elevate impulsive responding in a rodent reaction time paradigm. However, the mechanism underlying this effect is unclear.

OBJECTIVES

This series of experiments investigated the pharmacological basis of this effect in terms of excitatory and inhibitory neurotransmission. We tested several pharmacological mechanisms that might produce the effect of NMDAR antagonism via disruption or dampening of IL output.

METHODS

Drugs known to affect brain GABA or glutamate function were tested in rats pre-trained on a five-choice serial reaction time task (5-CSRTT) following either their systemic administration or direct administration into the IL.

RESULTS

Systemic lamotrigine administration (15 mg/kg), which attenuates excess glutamate release, did not counteract the ability of the intra-IL NMDAR antagonist 3-((R)-2-carboxypiperazin-4-yl)-propyl-L: -phosphonic acid ((R)-CPP) to increase premature responding on the 5-CSRTT. Putative elevation of local extracellular glutamate via intra-IL infusions of the selective glutamate reuptake inhibitor DL: -threo-β-benzyloxyaspartate as well as local α-amino-3-hydroxy-5-methyl-4-isoxazole propionic acid receptor antagonism also had no effect on this task. However, intra-IL infusions of the GABA(A) receptor agonist muscimol produced qualitatively but not quantitatively comparable increases in impulsive responding to those elicited by (R)-CPP. Moreover, the GABA(A) receptor antagonist bicuculline blocked the increase in impulsivity produced by (R)-CPP when infused in the IL.

CONCLUSIONS

These findings implicate glutamatergic and GABAergic mechanisms in the IL in the expression of impulsivity and suggest that excessive glutamate release may not underlie increased impulsivity induced by local NMDA receptor antagonism.

Using context to resolve temporal ambiguity

Three conditioned lick suppression experiments with rats examined the role of the context in the selection and integration of independently acquired interval relationships. In Experiment 1, rats were exposed to separate conditioned stimuli 1 and 2 (CS1-CS2) pairings with 2 different interval relationships, each in its own distinctive context, X or Y. The resultant integration was determined by the training context (X or Y) in which unconditioned stimulus (US)-CS2 backward pairings occurred, as assessed in a third neutral context (Z). In Experiment 2, rats experienced CS1-CS2 pairings with 2 different interval relationships as in Experiment 1, and then received US-CS2 pairings in both contexts X and Y. The testing context (i.e., X or Y) determined the resultant integration. In Experiment 3, rats were exposed to CS1-CS2 pairings in 2 different interval relationships each in different phases (i.e., Phases 1 and 2), and then in Phase 3 received US-CS2 pairings. The temporal context of testing (i.e., short or long retention interval) determined the resultant integration. Thus, both physical and temporal context can be used to disambiguate conflicting temporal information.

Two roles of the context in Pavlovian fear conditioning

At both empirical and theoretical levels, multiple functional roles of contextual information upon memory performance have been proposed without a clear dissociation of these roles. Some theories have assumed that contexts are functionally similar to cues, whereas other views emphasize the retrieval facilitating properties of contextual information. In Experiment 1, we observed that one critical parameter, the spacing of trials, could determine whether the context would function as a conditioned stimulus or as a retrieval cue for memories trained in different phases. Experiments 2 and 3 doubly dissociated these functions by selectively disrupting one role but not the other, and vice versa. Overall, these observations identify one determinant of different functions of contextual information and pose a major challenge to theories of learning that assume exclusively one or the other function of the context. Moreover, these data emphasize the importance of parametric variations on behavioral control, which has critical implications for studies designed to understand the role of the hippocampus in processing of contextual attributes.

Expanding the intertrial interval during extinction: response cessation and recovery

We examined trial spacing during extinction following a human contingency learning task. Specifically, we assessed if an expanding retrieval practice schedule (Bjork & Bjork, 1992, 2006), in which the spacing between extinction trials was progressively increased, would result in faster immediate extinction and less recovery from extinction than uniformly spaced extinction trials. We used an ABB vs. ABA renewal design and observed that, whereas the expanding group extinguished faster during extinction treatment, the expanding and constant groups showed the same level of extinction with an immediate test in the extinction context (ABB) and the two groups showed equivalent ABA renewal at test in the training context. We conclude that the faster extinction observed in the expanding groups could be misleading in clinical treatment, if the therapist used the absence of fear during extinction as the basis for terminating treatment.

Protection from latent inhibition provided by a conditioned inhibitor

Two conditioned suppression experiments with rats investigated the influence on latent inhibition of compounding a Pavlovian conditioned inhibitor with the target cue during preexposure treatment. Results were compared with those of subjects that received conventional latent inhibition training, no preexposure, or preexposure to the target cue in compound with a neutral stimulus. In Experiment 1, greater attenuation of the latent inhibition effect was observed in subjects that received target preexposure in compound with a Pavlovian conditioned inhibitor relative to subjects that received preexposure with a neutral stimulus or to the target alone. In Experiment 2, this protection from latent inhibition was attenuated if the excitor that was used to train the conditioned inhibitor was extinguished between preexposure and target training. The results are consistent with an account offered by the extended comparator hypothesis.

On the generality and limits of abstraction in rats and humans

In this review, we address the question, central to cognition, of whether nonhuman animals such as rats are capable of extracting and extending information from a given learning situation to a new learning situation without generalizing through a physical dimension of the stimuli.This capacity underlies abstraction, which is a hallmark of human cognition and necessary for complex information processing such as language acquisition. We selectively review recent experiments with rats in which systematic changes in information processing of new stimuli are observed after training with different stimuli. These results strongly suggest that this capacity is present in rats. We also review two articles in which clear limitations to this capacity are detected. We conclude that, within specified limits, rats are capable of using prior experience when faced with a learning situation that involves new stimuli.We interpret this ability as a rudimentary form of abstraction. In the face of these provocative results, new theories of learning should be designed to account for these findings.

Constraints on Enhanced Extinction Resulting from Extinction Treatment in the Presence of an Added Excitor

Three Pavlovian fear conditioning experiments with rats as subjects explored the effect of extinction in the presence of a concurrent excitor. Our aim was to explore this particular treatment, documented in previous studies to deepen extinction, with novel control groups to shed light on the processes involved in extinction. Relative to subjects extinguished on the target CS alone, Experiments 1 and 2 found across a range of parameters that any appreciable effect of facilitated extinction due to the concurrent excitor was submerged by generalization decrement going from extinction to testing. In Experiment 3 we used different durations for the target and concurrent stimuli in order to discourage configuring and an ABC renewal design to increase sensitivity, and observed diminished renewal resulting from extinction in the presence of a second excitor. Our findings suggest that there are distinct limits to the observation of extinction in the presence of an excitor and identifies some of the sources of these limitations.

Potentiation and overshadowing in Pavlovian fear conditioning

The present experiments addressed a fundamental discrepancy in the Pavlovian conditioning literature concerning responding to a target cue following compound reinforced training with another cue of higher salience. Experiment 1 identified one determinant of whether the target cue will be overshadowed or potentiated by the more salient cue, namely contiguity between compound CS termination and US presentation. Overshadowing and potentiation were observed with delay and trace procedures, respectively. Experiments 2 and 3 contrasted elemental and configural explanations of potentiation. Both experiments supported a configural account. Experiments 3 and 4, by manipulating prior learning experiences to bias subjects to encode the same compound elementally or configurally, demonstrated decreased potentiation and overshadowing, respectively. Overall, these experiments demonstrate potentiation with nontaste stimuli and identify one variable that determines whether overshadowing or potentiation will occur. Moreover, they show that prior experiences can determine how a compound is encoded and are compatible with the idea of flexible encoding as a principle of information processing.

Overshadowing and CS duration: counteraction and a reexamination of the role of within-compound associations in cue competition

In the present experiments, we examined the role of within-compound associations in the interaction of the overshadowing procedure with conditioned stimulus (CS) duration, using a conditioned suppression procedure with rats. In Experiment 1, we found that, with elemental reinforced training, conditioned suppression to the target stimulus decreased as CS duration increased (i.e., the CS duration effect), whereas, with compound reinforced training (i.e., the overshadowing procedure), conditioned suppression to the target stimulus increased as CS duration increased. In subsequent experiments, we replicated these findings with sensory preconditioning and demonstrated that extinction of the overshadowing stimulus results in retrospective revaluation with short CSs and in mediated extinction with long CSs. These results highlight the role of the duration of the stimulus in behavioral control. Moreover, these results illuminate one cause (the CS duration) of whether retrospective revaluation or mediated extinction will be observed.

Spacing extinction trials alleviates renewal and spontaneous recovery

Studies of extinction in classical conditioning situations can reveal techniques that maximize the effectiveness of exposure-based behavior therapies. In three experiments, we investigated the effect of varying the intertrial interval during an extinction treatment in a fear-conditioning preparation with rats as subjects. In Experiment 1, we found less fear at test (i.e., more effective extinction) when extinction trials were widely spaced, relative to intermediate or massed extinction trials. In Experiment 2, we used an ABA renewal procedure and observed that spaced trials attenuated renewal of conditioned fear relative to massed trials. In Experiment 3, we used a similar design, but instead of changing the physical context at the time of testing, we interposed a retention interval after the extinction treatment to produce a change in the temporal context. The results showed less spontaneous recovery of fear after spaced than after massed extinction trials. These results suggest that extinction is more enduring when the extinction trials are spaced rather than massed. Although the benefits of spacing trials are small when there is no contextual change from extinction to testing, a change in either physical or temporal context following massed extinction trials leads to a recovery from extinction, which is reduced when the trials are spaced.

Pavlovian backward conditioned inhibition in humans: summation and retardation tests

Two experiments using human participants investigated whether a Pavlovian backward inhibitory treatment (nonreinforced trials in phase 1 followed by reinforced trials in phase 2; i.e., AX- followed by A+) produces a stimulus which can pass summation and retardation tests for inhibition. The rationale for conducting these experiments was that previous demonstrations of Pavlovian backward inhibition informed participants about the nature of the outcome before starting the experiment. According to some theoretical views, this is a potential confound. In the present experiments we used a predictive task in which participants had no knowledge about the outcome until phase 2, when reinforcement occurred. The results of Experiment 1 (summation test) and Experiment 2 (retardation test) provide a clear demonstration of backward conditioned inhibition.

A comparator view of Pavlovian and differential inhibition

In 3 experiments using rats as subjects, the authors varied trial spacing to investigate the conditions under which Pavlovian and differential inhibition are observed. Experiment 1 compared Pavlovian and differential inhibition with spaced training trials. Spaced trials resulted in only the Pavlovian inhibitor passing both summation and retardation tests. Conversely, Experiment 2 compared these 2 types of inhibition with massed training trials. This training resulted in only the differential inhibitor passing both tests for conditioned inhibition. Finally, in Experiment 3 all subjects experienced Pavlovian inhibition training with massed trials. Although this training by itself did not result in behavior indicative of inhibition, subjects that also experienced posttraining extinction of the training context did pass both tests for inhibition. Overall, these results are anticipated by the extended comparator hypothesis (Denniston, Savastano, & Miller, 2001) but are problematic for most contemporary associative learning theories.

Counteraction between overshadowing and degraded contingency treatments: support for the extended comparator hypothesis

Four experiments using rats in a Pavlovian lick-suppression preparation investigated the effects of combining 2 treatments known for their response-decrementing effects, namely, overshadowing and degraded contingency. Contrary to most contemporary learning theories, the extended comparator hypothesis predicts that these 2 treatments will counteract each other, and therefore, less of a decrement in conditioned responding should be observed than with either treatment alone. Experiments 1 and 2 confirmed this prediction in first-order conditioning and sensory preconditioning preparations, respectively. Experiment 3 demonstrated that posttraining extinction of the training context resulted in a recovery from degraded contingency and reversed the counteractive effect on overshadowing. Finally, Experiment 4 demonstrated that posttraining extinction of the overshadowing stimulus resulted in recovery from simple overshadowing and also reversed the counteractive effect on degraded contingency. These results are consistent with the extended comparator hypothesis but not traditional or recent acquisition-focused models.