Triggering Avoidance: Dissociable Influences of Aversive Pavlovian Conditioned Stimuli on Human Instrumental Behavior

Cortical Beta Power Reflects the Influence of Pavlovian Cues on Human Decision-Making

Reward-predictive cues can affect decision-making by enhancing instrumental responses toward the same (specific transfer) or similar (general transfer) rewards. The main theories on cue-guided decision-making consider specific transfer as driven by the activation of previously learned instrumental actions induced by cues sharing the sensory-specific properties of the reward they are associated with. However, to date, such theoretical assumption has never been directly investigated at the neural level. We hypothesize that such reactivation occurs within the premotor system and could be mapped by lateralized beta (12-30 Hz) desynchronization, a widely used marker of action selection and decision-making policy. To test this hypothesis, 42 participants (22 females) performed a pavlovian-to-instrumental transfer paradigm, while electroencephalographic activity was recorded. We anticipated increased beta desynchronization during the transfer phase when cues promoting specific transfer were presented, compared with cues predicting general transfer and neutral cues. The evidence collected confirmed our hypothesis, thus providing the first neural evidence in favor of the theorized reactivation of instrumental actions and corroborating the presence of two dissociable neural pathways underpinning specific and general transfer.

Sex differences in motivational biases over instrumental actions

Motivational (i.e., appetitive or aversive) cues can bias value-based decisions by affecting either direction and intensity of instrumental actions. Despite several findings describing important interindividual differences in these biases, whether biological sex can also play a role is still up to debate. By comparing females and males in both appetitive and aversive Pavlovian-to-Instrumental Transfer paradigms we found that, while motivational cues similarly bias the direction of instrumental actions in both sexes, the intensity of such actions is increased by the cue in male participants only. The present results constitute compelling evidence that a crucial motivational bias of daily actions directed to obtaining rewards or avoiding punishments is modulated by biological sex. This evidence sheds new light on the role of sex in motivational processes that underlie decision-making, highlighting the importance of considering sex as a crucial factor in future research on this topic.

Unraveling the influence of Pavlovian cues on decision-making: A pre-registered meta-analysis on Pavlovian-to-instrumental transfer

Amidst the replicability crisis, promoting transparency and rigor in research becomes imperative. The Pavlovian-to-Instrumental Transfer (PIT) paradigm is increasingly used in human studies to offer insights into how Pavlovian cues, by anticipating rewards or punishments, influence decision-making and potentially contribute to the development of clinical conditions. However, research on this topic faces challenges, including methodological variability and the need for standardized approaches, which can undermine the quality and robustness of experimental findings. Hence, we conducted a meta-analysis to unravel the methodological, task-related, individual, training, and learning factors that can modulate PIT. By scrutinizing these factors, the present meta-analysis reviews the current literature on human PIT, provides practical guidelines for future research to enhance study outcomes and refine methodologies, and identifies knowledge gaps that can serve as a direction for future studies aiming to advance the comprehension of how Pavlovian cues shape decision-making.

Compulsive avoidance in youths and adults with OCD: an aversive pavlovian-to-instrumental transfer study

Compulsive behaviour may often be triggered by Pavlovian cues. Assessing how Pavlovian cues drive instrumental behaviour in obsessive-compulsive disorder (OCD) is therefore crucial to understand how compulsions develop and are maintained. An aversive Pavlovian-to-Instrumental transfer (PIT) paradigm, particularly one involving avoidance/cancellation of negative outcomes, can enable such investigation and has not previously been studied in clinical-OCD. Forty-one participants diagnosed with OCD (21 adults; 20 youths) and 44 controls (21 adults; 23 youths) completed an aversive PIT task. Participants had to prevent the delivery of unpleasant noises by moving a joystick in the correct direction. They could infer these correct responses by learning appropriate response-outcome (instrumental) and stimulus-outcome (Pavlovian) associations. We then assessed whether Pavlovian cues elicited specific instrumental avoidance responses (specific PIT) and induced general instrumental avoidance (general PIT). We investigated whether task learning and confidence indices influenced PIT strength differentially between groups. There was no overall group difference in PIT performance, although youths with OCD showed weaker specific PIT than youth controls. However, urge to avoid unpleasant noises and preference for safe over unsafe stimuli influenced specific and general PIT respectively in OCD, while PIT in controls was more influenced by confidence in instrumental and Pavlovian learning. Thus, in OCD, implicit motivational factors, but not learnt knowledge, may contribute to the successful integration of aversive Pavlovian and instrumental cues. This implies that compulsive avoidance may be driven by these automatic processes. Youths with OCD show deficits in specific PIT, suggesting cue integration impairments are only apparent in adolescence. These findings may be clinically relevant as they emphasise the importance of targeting such implicit motivational processes when treating OCD.

Modifying Pavlovian-to-instrumental transfer by approach avoidance training in healthy subjects: a proof of concept study

The modulation of instrumental action by conditioned Pavlovian cues is hypothesized to play a role in the emergence and maintenance of maladaptive behavior. The Pavlovian to Instrumental transfer task (PIT) is designed to examine the magnitude of the influence of cues on behavior and we aim to manipulate the motivational value of Pavlovian cues to reduce their effect on instrumental responding. To this end, we utilized a joystick-based modification of approach and avoidance propensities that has shown success in clinical populations. To examine changes in PIT, we subjected 35 healthy participants to a series of experimental procedures: (1) Instrumental training was followed by (2) Pavlovian conditioning of neutral stimuli that were associated with monetary reward or loss. (3) In a subsequent joystick task, approach and avoidance tendencies toward conditioned cues were assessed. (4) In a transfer test, the PIT effect as the impact of conditioned cues on instrumental behavior was measured. (5) The explicit knowledge of cue-reward contingencies was assessed in a forced-choice phase. (6, 7) systematic joystick training was followed by a posttest (8) the transfer task and forced-choice test were repeated. We found no effect of training on approach-avoidance propensities in the context of this proof of concept study. A higher response rate towards negative stimuli during PIT after systematic training compared to sham training was seen. On the other hand, we saw an increased PIT effect after sham training. These results contribute to the understanding of the strength of the influence of cues on instrumental behavior. Our findings further stress the importance of context, instructions and operationalization of instrumental behavior in the framework of transfer effects.

Pavlovian threat learning shapes the kinematics of action

Prompt response to environmental threats is critical to survival. Previous research has revealed mechanisms underlying threat-conditioned physiological responses, but little is known about how threats shape action. Here we tested if threat learning shapes the kinematics of reaching in human adults. In two different experiments conducted on independent samples of participants, after Pavlovian threat learning, in which a stimulus anticipated the delivery of an aversive shock, whereas another did not, the peak velocity and acceleration of reaching increased for the shocked-paired stimulus, relative to the unpaired one. These kinematic changes appeared as a direct consequence of learning, emerging even in absence of an actual threat to body integrity, as no shock occurred during reaching. Additionally, they correlated with the strength of sympathetic response during threat learning, establishing a direct relationship between previous learning and subsequent changes in action. The increase in velocity and acceleration of action following threat learning may be adaptive to facilitate the implementation of defensive responses. Enhanced action invigoration may be maladaptive, however, when defensive responses are inappropriately enacted in safe contexts, as exemplified in a number of anxiety-related disorders.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Susceptibility to interference between Pavlovian and instrumental control is associated with early hazardous alcohol use

Pavlovian-to-instrumental transfer (PIT) tasks examine the influence of Pavlovian stimuli on ongoing instrumental behaviour. Previous studies reported associations between a strong PIT effect, high-risk drinking and alcohol use disorder. This study investigated whether susceptibility to interference between Pavlovian and instrumental control is linked to risky alcohol use in a community sample of 18-year-old male adults. Participants (N = 191) were instructed to 'collect good shells' and 'leave bad shells' during the presentation of appetitive (monetary reward), aversive (monetary loss) or neutral Pavlovian stimuli. We compared instrumental error rates (ER) and functional magnetic resonance imaging (fMRI) brain responses between the congruent and incongruent conditions, as well as among high-risk and low-risk drinking groups. On average, individuals showed a substantial PIT effect, that is, increased ER when Pavlovian cues and instrumental stimuli were in conflict compared with congruent trials. Neural PIT correlates were found in the ventral striatum and the dorsomedial and lateral prefrontal cortices (lPFC). Importantly, high-risk drinking was associated with a stronger behavioural PIT effect, a decreased lPFC response and an increased neural response in the ventral striatum on the trend level. Moreover, high-risk drinkers showed weaker connectivity from the ventral striatum to the lPFC during incongruent trials. Our study links interference during PIT to drinking behaviour in healthy, young adults. High-risk drinkers showed higher susceptibility to Pavlovian cues, especially when they conflicted with instrumental behaviour, indicating lower interference control abilities. Increased activity in the ventral striatum (bottom-up), decreased lPFC response (top-down), and their altered interplay may contribute to poor interference control in the high-risk drinkers.

Neural correlates of cue-induced changes in decision-making distinguish subjects with gambling disorder from healthy controls

In addiction, there are few human studies on the neural basis of cue-induced changes in value-based decision making (Pavlovian-to-instrumental transfer, PIT). It is especially unclear whether neural alterations related to PIT are due to the physiological effects of substance abuse or rather related to learning processes and/or other etiological factors related to addiction. We have thus investigated whether neural activation patterns during a PIT task help to distinguish subjects with gambling disorder (GD), a nonsubstance-based addiction, from healthy controls (HCs). Thirty GD and 30 HC subjects completed an affective decision-making task in a functional magnetic resonance imaging (fMRI) scanner. Gambling-associated and other emotional cues were shown in the background during the task. Data collection and feature modeling focused on a network of nucleus accumbens (NAcc), amygdala, and orbitofrontal cortex (OFC) (derived from PIT and substance use disorder [SUD] studies). We built and tested a linear classifier based on these multivariate neural PIT signatures. GD subjects showed stronger PIT than HC subjects. Classification based on neural PIT signatures yielded a significant area under the receiver operating curve (AUC-ROC) (0.70, p = 0.013). GD subjects showed stronger PIT-related functional connectivity between NAcc and amygdala elicited by gambling cues, as well as between amygdala and OFC elicited by negative and positive cues. HC and GD subjects were thus distinguishable by PIT-related neural signatures including amygdala-NAcc-OFC functional connectivity. Neural PIT alterations in addictive disorders might not depend on the physiological effect of a substance of abuse but on related learning processes or even innate neural traits.

Fear-related signals are prioritised in visual, somatosensory and spatial systems

The human brain has evolved a multifaceted fear system, allowing threat detection to enable rapid adaptive responses crucial for survival. Although many cortical and subcortical brain areas are believed to be involved in the survival circuits detecting and responding to threat, the amygdala has reportedly a crucial role in the fear system. Here, we review evidence demonstrating that fearful faces, a specific category of salient stimuli indicating the presence of threat in the surrounding, are preferentially processed in the fear system and in the connected sensory cortices, even when they are presented outside of awareness or are irrelevant to the task. In the visual domain, we discuss evidence showing in hemianopic patients that fearful faces, via a subcortical colliculo-pulvinar-amygdala pathway, have a privileged visual processing even in the absence of awareness and facilitate responses towards visual stimuli in the intact visual field. Moreover, evidence showing that somatosensory cortices prioritise fearful-related signals, to the extent that tactile processing is enhanced in the presence of fearful faces, will be also reported. Finally, we will review evidence revealing that fearful faces have a pivotal role in modulating responses in peripersonal space, in line with the defensive functional definition of PPS.

Cue-induced effects on decision-making distinguish subjects with gambling disorder from healthy controls

While an increased impact of cues on decision-making has been associated with substance dependence, it is yet unclear whether this is also a phenotype of non-substance-related addictive disorders, such as gambling disorder (GD). To better understand the basic mechanisms of impaired decision-making in addiction, we investigated whether cue-induced changes in decision-making could distinguish GD from healthy control (HC) subjects. We expected that cue-induced changes in gamble acceptance and specifically in loss aversion would distinguish GD from HC subjects. Thirty GD subjects and 30 matched HC subjects completed a mixed gambles task where gambling and other emotional cues were shown in the background. We used machine learning to carve out the importance of cue dependency of decision-making and of loss aversion for distinguishing GD from HC subjects. Cross-validated classification yielded an area under the receiver operating curve (AUC-ROC) of 68.9% (p = .002). Applying the classifier to an independent sample yielded an AUC-ROC of 65.0% (p = .047). As expected, the classifier used cue-induced changes in gamble acceptance to distinguish GD from HC. Especially, increased gambling during the presentation of gambling cues characterized GD subjects. However, cue-induced changes in loss aversion were irrelevant for distinguishing GD from HC subjects. To our knowledge, this is the first study to investigate the classificatory power of addiction-relevant behavioral task parameters when distinguishing GD from HC subjects. The results indicate that cue-induced changes in decision-making are a characteristic feature of addictive disorders, independent of a substance of abuse.

Motivational factors underlying aversive Pavlovian-instrumental transfer

While interest in active avoidance has recently been resurgent, many concerns relating to the nature of this form of learning remain unresolved. By separating stimulus and response acquisition, aversive Pavlovian-instrumental transfer can be used to measure the effect of avoidance learning on threat processing with more control than typical avoidance procedures. However, the motivational substrates that contribute to the aversive transfer effect have not been thoroughly examined. In three studies using rodents, the impact of a variety of aversive signals on shock-avoidance responding (i.e., two-way shuttling) was evaluated. Fox urine, as well as a tone paired with the delivery of the predator odor were insufficient modulatory stimuli for the avoidance response. Similarly, a signal for the absence of food did not generate appropriate aversive motivation to enhance shuttling. Only conditioned Pavlovian stimuli that had been paired with unconditioned threats were capable of augmenting shock-avoidance responding. This was true whether the signaled outcome was the same (e.g., shock) or different (e.g., klaxon) from the avoidance outcome (i.e., shock). These findings help to characterize the aversive transfer effect and provide a more thorough analysis of its generalization to warning signals for different kinds of threats. This feature of aversive motivation has not been demonstrated using conventional avoidance procedures and could be potentially useful for applying avoidance in treatment settings.

The effect of conditioned inhibitors and preexposed cues on the outcome-specific Pavlovian-to-instrumental transfer effect in humans

Using a human Pavlovian-to-instrumental transfer (PIT) task, Alarcón and Bonardi showed that the selective elevation of instrumental responding produced by excitatory transfer cues was reduced when these cues were presented with a conditioned inhibitor (CI), relative to a control cue that was simply preexposed. However, previous research has shown that preexposed cues might also acquire inhibitor-like properties. This study aimed to contrast the inhibitory properties of CIs and preexposed cues, using novel stimuli as controls, in summation and PIT tests. Participants were trained to perform two actions, each reinforced with a distinct outcome (O₁ or O₂). Two images were trained as CIs, each signalling the absence of one of the outcomes, by presenting them with a cue that was otherwise followed by that outcome (e.g., A→O₁, AI→no O₁). In contrast, the preexposed cues were simply presented in the absence of the outcomes. In the summation test, participants rated the likelihood of the outcomes in the presence of two independently trained excitatory cues, each presented with a CI, a preexposed cue, or a novel stimulus. Similarly, in the PIT test, participants performed both actions in the presence and absence of these compounds. In the summation test, the CIs and the preexposed cues reduced participants' expectations of the outcomes more than the novel stimuli. However, in the PIT test, only the CIs reduced the selective elevation of responding produced by the transfer cues. These results might reflect distinct properties of stimuli trained as CIs and those simply preexposed.

Single-response appetitive Pavlovian to instrumental transfer is suppressed by aversive counter-conditioning

Environmental stimuli, when paired with reward, can influence behaviour in maladaptive ways, for example, by encouraging overeating or addiction. Such behaviour can be sensitive to reward value manipulations, under circumscribed conditions, but whether reward-seeking is also sensitive to stimulus value manipulations remains unclear. Thus, the current experiment investigated whether reducing the hedonic value of a reward-paired stimulus would reduce reward-seeking behaviour. In total, 36 participants successfully completed a single-response Pavlovian-instrumental transfer (PIT) task with a counter-conditioning procedure. The Pavlovian phase associated three conditioned stimuli (CSs) with money at 100%, 50%, or 0% contingency. Counter-conditioning then followed for the experimental group, who saw the 100% CS paired with unpleasant pictures, while the control group saw only neutral images. Instrumental training required participants to learn a button-pressing response to win money. The transfer phase contrasted instrumental responding during baseline and CS presentation. Both experimental and control groups liked the 100% CS more than the other CSs after Pavlovian training, but counter-conditioning reduced this 100% CS liking. In transfer, the experimental group showed an abolition of appetitive PIT, while the control group showed maintenance of appetitive PIT. However, this group difference was only evident in response vigour, not response initiation. In summary, CS hedonic value influences cue-potentiated instrumental responding. More specifically, hedonic value of a reward-paired cue influences the vigour of instrumental responses, but not the decision to initiate a response. These data may have relevance to smoking cessation policies, where the introduction of health warnings may be viewed as a real-world example of counter conditioning.

Individual differences in working memory capacity and cue-guided behavior in humans

Information gathered via Pavlovian and Instrumental learning can be integrated to guide behavior, in a phenomenon experimentally known as Pavlovian-to-Instrumental Transfer (PIT). In particular, in appetitive PIT, a reward-associated cue is able to enhance the instrumental response previously associated with the same (outcome-specific PIT), or a similar (general PIT), reward. The PIT effect is increasingly investigated for its numerous implications in clinical contexts as well as daily life situations. Nevertheless, the precise mechanism behind it is not yet clear. The relation between the PIT effect and high-level cognitive abilities - like working memory - is still unknown, but potentially relevant to unveil its functioning. The present study aims to examine the precise relationship between individual differences in working memory and the two forms of PIT effect, namely outcome-specific and general. For this purpose, 100 participants underwent a classical PIT paradigm. Results showed a relationship between individual working memory and outcome-specific PIT, but not general PIT. Importantly, the role of working memory was not related to the acquisition of the learning contingencies, but rather linked to an imbalance between congruent and incongruent choices. The results are discussed in terms of the adaptive and maladaptive implications for human behavior.

Appetitive Pavlovian-to-Instrumental Transfer in Participants with Normal-Weight and Obesity

Altered eating behavior due to modern, food-enriched environments has a share in the recent obesity upsurge, though the exact mechanisms remain unclear. This study aims to assess whether higher weight or weight gain are related to stronger effects of external cues on motivation-driven behavior. 51 people with and without obesity completed an appetitive Pavlovian-to-Instrumental Transfer (PIT) paradigm. During training, button presses as well as presentation of fractal images resulted in three palatable and one neutral taste outcome. In the subsequent test phase, outcome-specific and general behavioral bias of the positively associated fractal images on deliberate button press were tested under extinction. While all participants showed signs of specific transfer, general transfer was not elicited. Contrary to our expectations, there was no main effect of weight group on PIT magnitude. Participants with obesity exhibited higher scores in the Three-Factor Eating Questionnaire Disinhibition scale, replicating a very robust effect from previous literature. Individual Restraint scores were able to predict body-mass index (BMI) change after a three-year period. Our data indicate that PIT is an important player in how our environment influences the initiation of food intake, but its effects alone cannot explain differences in-or future development of-individual weight.

Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms

BACKGROUND

Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown.

METHODS

To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations.

RESULTS

Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms.

CONCLUSION

Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine.

Motivational sensitivity of outcome-response priming: Experimental research and theoretical models

Outcome-response (O-R) priming is at the core of various associative theories of human intentional action. This is a simple and parsimonious mechanism by which activation of outcome representations (e.g. thinking about the light coming on) leads to activation of the associated motor patterns required to achieve it (e.g. pushing the light switch). In the current manuscript, we review the evidence for such O-R associative links demonstrated by converging (yet until now, separate) strands of research. While there is a wealth of evidence that both the perceptual and motivational properties of an outcome can be encoded in the O-R association and mediate O-R priming, we critically examine the integration of these mechanisms and the conditions under which motivational factors constrain the sensory O-R priming effect. We discuss the clinical relevance of this O-R priming mechanism, whether it can satisfactorily account for human goal-directed behaviour, and the implications for theories of human action control.