Anxiety-Like Behavioural Inhibition Is Normative under Environmental Threat-Reward Correlations

Hippocampal damage disrupts the latent decision-making processes underlying approach-avoidance conflict processing in humans

Rodent and human data implicate the hippocampus in the arbitration of approach-avoidance conflict (AAC), which arises when an organism is confronted with a stimulus associated simultaneously with reward and punishment. Yet, the precise contributions of this structure are underexplored, particularly with respect to the decision-making processes involved. We assessed humans with hippocampal damage and matched neurologically healthy controls on a computerized AAC paradigm in which participants first learned whether individual visual images were associated with the reward or loss of game points and were then asked to approach or avoid pairs of stimuli with non-conflicting or conflicting valences. To assess hippocampal involvement more broadly in response conflict, we also administered a Stroop and a Go/No-go task. On the AAC paradigm, following similar learning outcomes in individuals with hippocampal damage and matched controls, both participant groups approached positive and negative image pairs at the same rate but critically, those with hippocampal damage approached conflict pairs more often than controls. Choice and response AAC data were interrogated using the hierarchical drift diffusion model, which revealed that, compared to controls, individuals with hippocampal damage were more biased towards approach, required less evidence to make a decision during conflict trials, and were slower to accumulate evidence towards avoidance when confronted with conflicting image pairs. No significant differences were found between groups in performance accuracy or response time on the response conflict tasks. Taken together, these findings demonstrate the importance of the hippocampus to the evidence accumulation processes supporting value-based decision-making under motivational conflict.

High stakes slow responding, but do not help overcome Pavlovian biases in humans

"Pavlovian" or "motivational" biases are the phenomenon that the valence of prospective outcomes modulates action invigoration: the prospect of reward invigorates actions, while the prospect of punishment suppresses actions. Effects of the valence of prospective outcomes are well established, but it remains unclear how the magnitude of outcomes ("stake magnitude") modulates these biases. In this preregistered study (N = 55), we manipulated stake magnitude (high vs. low) in an orthogonalized Motivational Go/NoGo Task. We tested whether higher stakes (a) strengthen biases or (b) elicit cognitive control recruitment, enhancing the suppression of biases in motivationally incongruent conditions. Confirmatory tests showed that high stakes slowed down responding, especially in motivationally incongruent conditions. However, high stakes did not affect whether a response was made or not, and did not change the magnitude of Pavlovian biases. Reinforcement-learning drift-diffusion models (RL-DDMs) fit to the data suggested that response slowing was best captured by stakes prolonging the non-decision time. There was no effect of the stakes on the response threshold (as in typical speed-accuracy trade-offs). In sum, these results suggest that high stakes slow down responses without affecting the expression of Pavlovian biases in behavior. We speculate that this slowing under high stakes might reflect heightened cognitive control, which is however ineffectively used, or reflect positive conditioned suppression, i.e., the interference between goal-directed and consummatory behaviors, a phenomenon previously observed in rodents that might also exist in humans. Pavlovian biases and slowing under high stakes may arise in parallel to each other.

The expression of decision and learning variables in movement patterns related to decision actions

Decisions are not necessarily easy to separate into a planning and an execution phase and the decision-making process can often be reflected in the movement associated with the decision. Here, we used formalized definitions of concepts relevant in decision-making and learning to explore if and how these concepts correlate with decision-related movement paths, both during and after a choice is made. To this end, we let 120 participants (46 males, mean age = 24.5 years) undergo a repeated probabilistic two-choice task with changing probabilities where we used mouse-tracking, a simple non-invasive technique, to study the movements related to decisions. The decisions of the participants were modelled using Bayesian inference which enabled the computation of variables related to decision-making and learning. Analyses of the movement during the decision showed effects of relevant decision variables, such as confidence, on aspects related to, for instance, timing and pausing, range of movement and deviation from the shortest distance. For the movements after a decision there were some effects of relevant learning variables, mainly related to timing and speed. We believe our findings can be of interest for researchers within several fields, spanning from social learning to experimental methods and human-machine/robot interaction.

The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Out of control: computational dynamic control dysfunction in stress- and anxiety-related disorders

Control theory, which has played a central role in technological progress over the last 150 years, has also yielded critical insights into biology and neuroscience. Recently, there has been a surging interest in integrating control theory with computational psychiatry. Here, we review the state of the field of using control theory approaches in computational psychiatry and show that recent research has mapped a neural control circuit consisting of frontal cortex, parietal cortex, and the cerebellum. This basic feedback control circuit is modulated by estimates of reward and cost via the basal ganglia as well as by arousal states coordinated by the insula, dorsal anterior cingulate cortex, amygdala, and locus coeruleus. One major approach within the broader field of control theory, known as proportion-integral-derivative (PID) control, has shown promise as a model of human behavior which enables precise and reliable estimates of underlying control parameters at the individual level. These control parameters correlate with self-reported fear and with both structural and functional variation in affect-related brain regions. This suggests that dysfunctional engagement of stress and arousal systems may suboptimally modulate parameters of domain-general goal-directed control algorithms, impairing performance in complex tasks involving movement, cognition, and affect. Future directions include clarifying the causal role of control deficits in stress- and anxiety-related disorders and developing clinically useful tools based on insights from control theory.

Evaluation of behavioural selection processes in conflict scenarios using a newly developed mouse behavioural paradigm

Selecting an appropriate behaviour is critical for survival in conflict scenarios, wherein animals face both appetitive and aversive stimuli. Behavioural selection consists of multiple processes: (1) animals remain quiet in a safe place to avoid aversive stimuli (suspension), (2) once they decide to take risks to approach appetitive stimuli, they assess the risks (risk assessment), and (3) they act to reach the reward. However, most studies have not addressed these distinct behavioural processes separately. Here, we developed a new experimental paradigm called the three-compartment conflict task to quantitatively evaluate conflict processes. Our apparatus consisted of start, flat, and grid compartments. Mice needed to explore the grid compartment, where they might receive foot shocks while trying to obtain sucrose. Applying foot shocks increased sucrose acquisition latency in subsequent trials, reflecting elevated conflict levels throughout trials. The time spent in the start compartment and the number of retreats were determined to measure the conflict levels in suspension and risk assessment, respectively. Foot shocks increased these parameters, whereas diazepam decreased them. Our new paradigm is valuable for quantitatively evaluating distinct behavioural processes and contributes to developing effective treatments for psychiatric disorders associated with maladaptive behaviours in conflict scenarios.

Perceptual control or action-selection? Comment on: a perceptual control theory of emotional action

The Perceptual Control Theory of Emotional Action provides a compelling view of the synergy between action and perception in the context of emotion. In this invited response, we outline three suggestions to further clarify and concretesise the theory in the hope that it can provide a solid basis for the theoretical, empirical, and clinical fields of emotion and emotion regulation. First, we emphasise the importance of concretesising these ideas in a way that is biologically plausible and testable in terms of its neuronal implementation, which has not been addressed in the main manuscript. Secondly, we highlight the challenges for this account to effectively describe core symptoms in emotional disorders, an essential step if the theory aims to foster the development of better-tuned neurocognitively grounded interventions. Finally, we take a leap on what action-oriented accounts of emotion can mean for the field of emotion regulation.

Emotion regulation from an action-control perspective

Despite increasing interest in emotional processes in cognitive science, theories on emotion regulation have remained rather isolated, predominantly focused on cognitive regulation strategies such as reappraisal. However, recent neurocognitive evidence suggests that early emotion regulation may involve sensorimotor control in addition to other emotion-regulation processes. We propose an action-oriented view of emotion regulation, in which feedforward predictions develop from action-selection mechanisms. Those can account for acute emotional-action control as well as more abstract instances of emotion regulation such as cognitive reappraisal. We argue the latter occurs in absence of overt motor output, yet in the presence of full-blown autonomic, visceral, and subjective changes. This provides an integrated framework with testable neuro-computational predictions and concrete starting points for intervention to improve emotion control in affective disorders.

Computational perspectives on human fear and anxiety

Fear and anxiety are adaptive emotions that serve important defensive functions, yet in excess, they can be debilitating and lead to poor mental health. Computational modelling of behaviour provides a mechanistic framework for understanding the cognitive and neurobiological bases of fear and anxiety, and has seen increasing interest in the field. In this brief review, we discuss recent developments in the computational modelling of human fear and anxiety. Firstly, we describe various reinforcement learning strategies that humans employ when learning to predict or avoid threat, and how these relate to symptoms of fear and anxiety. Secondly, we discuss initial efforts to explore, through a computational lens, approach-avoidance conflict paradigms that are popular in animal research to measure fear- and anxiety-relevant behaviours. Finally, we discuss negative biases in decision-making in the face of uncertainty in anxiety.

Preclinical and clinical evidence on the approach-avoidance conflict evaluation as an integrative tool for psychopathology

The approach-avoidance conflict (AAC), i.e. the competing tendencies to undertake goal-directed actions or to withdraw from everyday life challenges, stands at the basis of humans' existence defining behavioural and personality domains. Gray's Reinforcement Sensitivity Theory posits that a stable bias toward approach or avoidance represents a psychopathological trait associated with excessive sensitivity to reward or punishment. Optogenetic studies in rodents and imaging studies in humans associated with cross-species AAC paradigms granted new emphasis to the hippocampus as a hub of behavioural inhibition. For instance, recent functional neuroimaging studies show that functional brain activity in the human hippocampus correlates with threat perception and seems to underlie passive avoidance. Therefore, our commentary aims to (i) discuss the inhibitory role of the hippocampus in approach-related behaviours and (ii) promote the integration of functional neuroimaging with cross-species AAC paradigms as a means of diagnostic, therapeutic, follow up and prognosis refinement in psychiatric populations.

Humans perseverate on punishment avoidance goals in multigoal reinforcement learning

Managing multiple goals is essential to adaptation, yet we are only beginning to understand computations by which we navigate the resource demands entailed in so doing. Here, we sought to elucidate how humans balance reward seeking and punishment avoidance goals, and relate this to variation in its expression within anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). We constructed a computational model of multigoal pursuit to quantify the degree to which participants could disengage from the pursuit goals when instructed to, as well as devote less model-based resources toward goals that were less abundant. In general, participants (n = 192) were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, participants often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. In a similar vein, participants showed no significant downregulation of avoidance when punishment avoidance goals were less abundant in the task. Importantly, we show preliminary evidence that individuals with chronic worry may have difficulty disengaging from punishment avoidance when instructed to seek reward. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward. Future studies should test in larger samples whether a difficulty to disengage from punishment avoidance contributes to chronic worry.

The prefrontal cortex, pathological anxiety, and anxiety disorders

Anxiety is experienced in response to threats that are distal or uncertain, involving changes in one's subjective state, autonomic responses, and behavior. Defensive and physiologic responses to threats that involve the amygdala and brainstem are conserved across species. While anxiety responses typically serve an adaptive purpose, when excessive, unregulated, and generalized, they can become maladaptive, leading to distress and avoidance of potentially threatening situations. In primates, anxiety can be regulated by the prefrontal cortex (PFC), which has expanded in evolution. This prefrontal expansion is thought to underlie primates' increased capacity to engage high-level regulatory strategies aimed at coping with and modifying the experience of anxiety. The specialized primate lateral, medial, and orbital PFC sectors are connected with association and limbic cortices, the latter of which are connected with the amygdala and brainstem autonomic structures that underlie emotional and physiological arousal. PFC pathways that interface with distinct inhibitory systems within the cortex, the amygdala, or the thalamus can regulate responses by modulating neuronal output. Within the PFC, pathways connecting cortical regions are poised to reduce noise and enhance signals for cognitive operations that regulate anxiety processing and autonomic drive. Specialized PFC pathways to the inhibitory thalamic reticular nucleus suggest a mechanism to allow passage of relevant signals from thalamus to cortex, and in the amygdala to modulate the output to autonomic structures. Disruption of specific nodes within the PFC that interface with inhibitory systems can affect the negative bias, failure to regulate autonomic arousal, and avoidance that characterize anxiety disorders.

Cross-species anxiety tests in psychiatry: pitfalls and promises

Behavioural anxiety tests in non-human animals are used for anxiolytic drug discovery, and to investigate the neurobiology of threat avoidance. Over the past decade, several of them were translated to humans with three clinically relevant goals: to assess potential efficacy of candidate treatments in healthy humans; to develop diagnostic tests or biomarkers; and to elucidate the pathophysiology of anxiety disorders. In this review, we scrutinise these promises and compare seven anxiety tests that are validated across species: five approach-avoidance conflict tests, unpredictable shock anticipation, and the social intrusion test in children. Regarding the first goal, three tests appear suitable for anxiolytic drug screening in humans. However, they have not become part of the drug development pipeline and achieving this may require independent confirmation of predictive validity and cost-effectiveness. Secondly, two tests have shown potential to measure clinically relevant individual differences, but their psychometric properties, predictive value, and clinical applicability need to be clarified. Finally, cross-species research has not yet revealed new evidence that the physiology of healthy human behaviour in anxiety tests relates to the physiology of anxiety symptoms in patients. To summarise, cross-species anxiety tests could be rendered useful for drug screening and for development of diagnostic instruments. Using these tests for aetiology research in healthy humans or animals needs to be queried and may turn out to be unrealistic.

Interoception of breathing and its relationship with anxiety

Interoception, the perception of internal bodily states, is thought to be inextricably linked to affective qualities such as anxiety. Although interoception spans sensory to metacognitive processing, it is not clear whether anxiety is differentially related to these processing levels. Here we investigated this question in the domain of breathing, using computational modeling and high-field (7 T) fMRI to assess brain activity relating to dynamic changes in inspiratory resistance of varying predictability. Notably, the anterior insula was associated with both breathing-related prediction certainty and prediction errors, suggesting an important role in representing and updating models of the body. Individuals with low versus moderate anxiety traits showed differential anterior insula activity for prediction certainty. Multi-modal analyses of data from fMRI, computational assessments of breathing-related metacognition, and questionnaires demonstrated that anxiety-interoception links span all levels from perceptual sensitivity to metacognition, with strong effects seen at higher levels of interoceptive processes.

Defensive freezing and its relation to approach-avoidance decision-making under threat

Successful responding to acutely threatening situations requires adequate approach-avoidance decisions. However, it is unclear how threat-induced states-like freezing-related bradycardia-impact the weighing of the potential outcomes of such value-based decisions. Insight into the underlying computations is essential, not only to improve our models of decision-making but also to improve interventions for maladaptive decisions, for instance in anxiety patients and first-responders who frequently have to make decisions under acute threat. Forty-two participants made passive and active approach-avoidance decisions under threat-of-shock when confronted with mixed outcome-prospects (i.e., varying money and shock amounts). Choice behavior was best predicted by a model including individual action-tendencies and bradycardia, beyond the subjective value of the outcome. Moreover, threat-related bradycardia (high-vs-low threat) interacted with subjective value, depending on the action-context (passive-vs-active). Specifically, in action-contexts incongruent with participants' intrinsic action-tendencies, stronger bradycardia related to diminished effects of subjective value on choice across participants. These findings illustrate the relevance of testing approach-avoidance decisions in relatively ecologically valid conditions of acute and primarily reinforced threat. These mechanistic insights into approach-avoidance conflict-resolution may inspire biofeedback-related techniques to optimize decision-making under threat. Critically, the findings demonstrate the relevance of incorporating internal psychophysiological states and external action-contexts into models of approach-avoidance decision-making.

Approach-Avoidance Decisions Under Threat: The Role of Autonomic Psychophysiological States

Acutely challenging or threatening situations frequently require approach-avoidance decisions. Acute threat triggers fast autonomic changes that prepare the body to freeze, fight or flee. However, such autonomic changes may also influence subsequent instrumental approach-avoidance decisions. Since defensive bodily states are often not considered in value-based decision-making models, it remains unclear how they influence the decision-making process. Here, we aim to bridge this gap by discussing the existing literature on the potential role of threat-induced bodily states on decision making and provide a new neurocomputational framework explaining how these effects can facilitate or bias approach-avoid decisions under threat. Theoretical accounts have stated that threat-induced parasympathetic activity is involved in information gathering and decision making. Parasympathetic dominance over sympathetic activity is particularly seen during threat-anticipatory freezing, an evolutionarily conserved response to threat demonstrated across species and characterized by immobility and bradycardia. Although this state of freezing has been linked to altered information processing and action preparation, a full theoretical treatment of the interactions with value-based decision making has not yet been achieved. Our neural framework, which we term the Threat State/Value Integration (TSI) Model, will illustrate how threat-induced bodily states may impact valuation of competing incentives at three stages of the decision-making process, namely at threat evaluation, integration of rewards and threats, and action initiation. Additionally, because altered parasympathetic activity and decision biases have been shown in anxious populations, we will end with discussing how biases in this system can lead to characteristic patterns of avoidance seen in anxiety-related disorders, motivating future pre-clinical and clinical research.

Threat of shock promotes passive avoidance, but not active avoidance

Anxiety and stress are adaptive responses to threat that promote harm avoidance. In particular, prior work has shown that anxiety induced in humans using threat of unpredictable shock promotes behavioral inhibition in the face of harm. This is consistent with the idea that anxiety promotes passive avoidance-that is, withholding approach actions that could lead to harm. However, harm can also be avoided through active avoidance, where a (withdrawal) action is taken to avoid harm. Here, we provide the first direct within-study comparison of the effects of threat of shock on active and passive avoidance. We operationalize passive avoidance as withholding a button press response in the face of negative outcomes, and active avoidance as lifting/releasing a button press in the face of negative outcomes. We explore the impact of threat of unpredictable shock on the learning of these behavioral responses (alongside matched responses to rewards) within a single cognitive task. We predicted that threat of shock would promote both active and passive avoidance, and that this would be driven by increased reliance on Pavlovian bias, as parameterized within reinforcement-learning models. Consistent with our predictions, we provide evidence that threat of shock promotes passive avoidance as conceptualized by our task. However, inconsistent with predictions, we found no evidence that threat of shock promoted active avoidance, nor evidence of elevated Pavlovian bias in any condition. One hypothetical framework with which to understand these findings is that anxiety promotes passive over active harm avoidance strategies in order to conserve energy while avoiding harm.

Motivational competition and the paraventricular thalamus

Although significant progress has been made in understanding the behavioral and brain mechanisms for motivational systems, much less is known about competition between motivational states or motivational conflict (e.g., approach - avoidance conflict). Despite being produced under diverse conditions, behavior during motivational competition has two signatures: bistability and metastability. These signatures reveal the operation of positive feedback mechanisms in behavioral selection. Different neuronal architectures can instantiate this selection to achieve bistability and metastability in behavior, but each relies on circuit-level inhibition to achieve rapid and stable selection between competing tendencies. Paraventricular thalamus (PVT) is identified as critical to this circuit level inhibition, resolving motivational competition via its extensive projections to local inhibitory networks in the ventral striatum and extended amygdala, enabling adaptive responding under motivational conflict.

Hippocampal Representation of Threat Features and Behavior in a Human Approach-Avoidance Conflict Anxiety Task

Decisions under threat are crucial to survival and require integration of distinct situational features, such as threat probability and magnitude. Recent evidence from human lesion and neuroimaging studies implicated anterior hippocampus (aHC) and amygdala in approach-avoidance decisions under threat, and linked their integrity to cautious behavior. Here we sought to elucidate how threat dimensions and behavior are represented in these structures. Twenty human participants (11 female) completed an approach-avoidance conflict task during high-resolution fMRI. Participants could gather tokens under threat of capture by a virtual predator, which would lead to token loss. Threat probability (predator wake-up rate) and magnitude (amount of token loss) varied on each trial. To disentangle effects of threat features, and ensuing behavior, we performed a multifold parametric analysis. We found that high threat probability and magnitude related to BOLD signal in left aHC/entorhinal cortex. However, BOLD signal in this region was better explained by avoidance behavior than by these threat features. A priori ROI analysis confirmed the relation of aHC BOLD response with avoidance. Exploratory subfield analysis revealed that this relation was specific to anterior CA2/3 but not CA1. Left lateral amygdala responded to low and high, but not intermediate, threat probability. Our results suggest that aHC BOLD signal is better explained by avoidance behavior than by threat features in approach-avoidance conflict. Rather than representing threat features in a monotonic manner, it appears that aHC may compute approach-avoidance decisions based on integration of situational threat features represented in other neural structures.SIGNIFICANCE STATEMENT An effective threat anticipation system is crucial to survival across species. Natural threats, however, are diverse and have distinct features. To be able to adapt to different modes of danger, the brain needs to recognize these features, integrate them, and use them to modify behavior. Our results disclose the human anterior hippocampus as a likely arbiter of approach-avoidance decisions harnessing compound environmental information while partially replicating previous findings and blending into recent efforts to illuminate the neural basis of approach-avoidance conflict in humans.

Representation of probabilistic outcomes during risky decision-making

Goal-directed behaviour requires prospectively retrieving and evaluating multiple possible action outcomes. While a plethora of studies suggested sequential retrieval for deterministic choice outcomes, it remains unclear whether this is also the case when integrating multiple probabilistic outcomes of the same action. We address this question by capitalising on magnetoencephalography (MEG) in humans who made choices in a risky foraging task. We train classifiers to distinguish MEG field patterns during presentation of two probabilistic outcomes (reward, loss), and then apply these to decode such patterns during deliberation. First, decoded outcome representations have a temporal structure, suggesting alternating retrieval of the outcomes. Moreover, the probability that one or the other outcome is being represented depends on loss magnitude, but not on loss probability, and it predicts the chosen action. In summary, we demonstrate decodable outcome representations during probabilistic decision-making, which are sequentially structured, depend on task features, and predict subsequent action.

Computational Evidence for Underweighting of Current Error and Overestimation of Future Error in Anxious Individuals

BACKGROUND

Real-time control of goal-directed actions requires continuous adjustments in response to both current error (i.e., distance from goal state) and predicted future error. Proportion-integral-derivative control models, which are extensively used in the automated control of industrial processes, formalize this intuition. Previous computational approaches to anxiety have separately addressed behavioral inhibition and exaggerated error processing, but a proportion-integral-derivative control approach that decomposes error processing into current and anticipated error could integrate these accounts and extend them to a real-time sensorimotor control domain.

METHODS

We applied a simplified proportion-derivative control model to a virtual driving task in a transdiagnostic psychiatric sample of 317 individuals and computed a drive parameter (weighting of current error) and a damping parameter (weighting of the rate of change of error, enabling adjustment based on future error).

RESULTS

Self-reported fear, but not negative affect, was selectively associated with lower drive and lower damping. Those individuals that were characterized by lower drive and damping also exhibited lower caudal anterior cingulate cortex, but not insula, volume in a structural magnetic resonance imaging analysis.

CONCLUSIONS

The proportion-derivative control approach reveals that fear is specifically associated with reduced weighting of current error and overestimation of future error, resulting in both approach inhibition and overcorrecting overshoots around a goal state.

Anxiety, avoidance, and sequential evaluation

Anxiety disorders are characterized by a range of aberrations in the processing of and response to threat, but there is little clarity what core pathogenesis might underlie these symptoms. Here we propose that a particular set of unrealistically pessimistic assumptions can distort an agent's behavior and underlie a host of seemingly disparate anxiety symptoms. We formalize this hypothesis in a decision theoretic analysis of maladaptive avoidance and a reinforcement learning model, which shows how a localized bias in beliefs can formally explain a range of phenomena related to anxiety. The core observation, implicit in standard decision theoretic accounts of sequential evaluation, is that the potential for avoidance should be protective: if danger can be avoided later, it poses less threat now. We show how a violation of this assumption - via a pessimistic, false belief that later avoidance will be unsuccessful - leads to a characteristic, excessive propagation of fear and avoidance to situations far antecedent of threat. This single deviation can explain a range of features of anxious behavior, including exaggerated threat appraisals, fear generalization, and persistent avoidance. Simulations of the model reproduce laboratory demonstrations of abnormal decision making in anxiety, including in situations of approach-avoid conflict and planning to avoid losses. The model also ties together a number of other seemingly disjoint phenomena in anxious disorders. For instance, learning under the pessimistic bias captures a hypothesis about the role of anxiety in the later development of depression. The bias itself offers a new formalization of classic insights from the psychiatric literature about the central role of maladaptive beliefs about control and self-efficacy in anxiety. This perspective also extends previous computational accounts of beliefs about control in mood disorders, which neglected the sequential aspects of choice.

Driven by Pain, Not Gain: Computational Approaches to Aversion-Related Decision Making in Psychiatry

Although it is well known that "losses loom larger than gains," computational approaches to aversion-related decision making (ARDM) for psychiatric disorders is an underdeveloped area. Computational models of ARDM have been implemented primarily as state-dependent reinforcement learning models with bias parameters to quantify Pavlovian associations, and differential learning rates to quantify instrumental updating have been shown to depend on context, involve complex cost calculations, and include the consideration of counterfactual outcomes. Little is known about how individual differences influence these models relevant to anxiety-related conditions or addiction-related dysfunction. It is argued that model parameters reflecting 1) Pavlovian biases in the context of reinforcement learning or 2) hyperprecise prior beliefs in the context of active inference play an important role in the emergence of dysfunctional avoidance behaviors. The neural implementation of ARDM includes brain areas that are important for valuation (ventromedial prefrontal cortex) and positive reinforcement-related prediction errors (ventral striatum), but also aversive processing (insular cortex and cerebellum). Computational models of ARDM will help to establish a quantitative explanatory account of the development of anxiety disorders and addiction, but such models also face several challenges, including limited evidence for stability of individual differences, relatively low reliability of tasks, and disorder heterogeneity. Thus, it will be necessary to develop robust, reliable, and model-based experimental probes; recruit larger sample sizes; and use single case experimental designs for better pragmatic and explanatory biological models of psychiatric disorders.

Striatopallidal neurons control avoidance behavior in exploratory tasks

The dorsal striatum has been linked to decision-making under conflict, but the mechanism by which striatal neurons contribute to approach-avoidance conflicts remains unclear. We hypothesized that striatopallidal dopamine D2 receptor (D2R)-expressing neurons promote avoidance, and tested this hypothesis in two exploratory approach-avoidance conflict paradigms in mice: the elevated zero maze and open field. Genetic elimination of D2Rs on striatopallidal neurons (iMSNs), but not other neural populations, increased avoidance of the open areas in both tasks, in a manner that was dissociable from global changes in movement. Population calcium activity of dorsomedial iMSNs was disrupted in mice lacking D2Rs on iMSNs, suggesting that disrupted output of iMSNs contributes to heightened avoidance behavior. Consistently, artificial disruption of iMSN output with optogenetic stimulation heightened avoidance of open areas of these tasks, while inhibition of iMSN output reduced avoidance. We conclude that dorsomedial striatal iMSNs control approach-avoidance conflicts in exploratory tasks, and highlight this neural population as a potential target for reducing avoidance in anxiety disorders.

Altered learning under uncertainty in unmedicated mood and anxiety disorders

Anxiety is characterized by altered responses under uncertain conditions, but the precise mechanism by which uncertainty changes the behaviour of anxious individuals is unclear. Here we probe the computational basis of learning under uncertainty in healthy individuals and individuals suffering from a mix of mood and anxiety disorders. Participants were asked to choose between four competing slot machines with fluctuating reward and punishment outcomes during safety and stress. We predicted that anxious individuals under stress would learn faster about punishments and exhibit choices that were more affected by those punishments, thus formalizing our predictions as parameters in reinforcement learning accounts of behaviour. Overall, the data suggest that anxious individuals are quicker to update their behaviour in response to negative outcomes (increased punishment learning rates). When treating anxiety, it may therefore be more fruitful to encourage anxious individuals to integrate information over longer horizons when bad things happen, rather than try to blunt their responses to negative outcomes.

Disentangling Hippocampal and Amygdala Contribution to Human Anxiety-Like Behavior

Anxiety comprises a suite of behaviors to deal with potential threat and is often modeled in approach–avoidance conflict tasks. Collectively, these tests constitute a predominant preclinical model of anxiety disorder. A body of evidence suggests that both ventral hippocampus and amygdala lesions impair anxiety-like behavior, but the relative contribution of these two structures is unclear. A possible reason is that approach–avoidance conflict tasks involve a series of decisions and actions, which may be controlled by distinct neural mechanisms that are difficult to disentangle from behavioral readouts. Here, we capitalize on a human approach–avoidance conflict test, implemented as computer game, that separately measures several action components. We investigate three patients of both sexes with unspecific unilateral medial temporal lobe (MTL) damage, one male with selective bilateral hippocampal (HC), and one female with selective bilateral amygdala lesions, and compare them to matched controls. MTL and selective HC lesions, but not selective amygdala lesions, increased approach decision when possible loss was high. In contrast, MTL and selective amygdala lesions, but not selective HC lesions, increased return latency. Additionally, selective HC and selective amygdala lesions reduced approach latency. In a task targeted at revealing subjective assumptions about the structure of the computer game, MTL and selective HC lesions impacted on reaction time generation but not on the subjective task structure. We conclude that deciding to approach reward under threat relies on hippocampus but not amygdala, whereas vigor of returning to safety depends on amygdala but not on hippocampus.

SIGNIFICANCE STATEMENT Approach–avoidance conflict tests are widely investigated in rodents, and increasingly in humans, to understand the neural basis of anxiety-like behavior. However, the contribution of the most relevant brain regions, ventral hippocampus and amygdala, is incompletely understood. We use a human computerized test that separates different action components and find that hippocampus, but not amygdala, lesions impair approach decisions, whereas amygdala, but not hippocampus, lesions impair the vigor of return to safety.

Establishing operant conflict tests for the translational study of anxiety in mice

RATIONALE

In conflict-based anxiety tests, rodents decide between actions with simultaneous rewarding and aversive outcomes. In humans, computerised operant conflict tests have identified response choice, latency, and vigour as distinct behavioural components. Animal operant conflict tests for measurement of these components would facilitate translational study.

OBJECTIVES

In C57BL/6 mice, two operant conflict tests for measurement of response choice, latency, and vigour were established, and effects of chlordiazepoxide (CDZ) thereon investigated.

METHODS

Mice were moderately diet-restricted to increase sucrose reward salience. A 1-lever test required responding under medium-effort reward/threat conditions of variable ratio 2-10 resulting in sucrose at p = 0.7 and footshock at p = 0.3. A 2-lever test mandated a choice between low-effort reward/threat with a fixed-ratio (FR) 2 lever yielding sucrose at p = 0.7 and footshock at p = 0.3 versus high-effort reward/no threat with a FR 20 lever yielding sucrose at p = 1.

RESULTS

In the 1-lever test, CDZ (7.5 or 15 mg/kg i.p.) reduced post-trial pause (response latency) following either sucrose or footshock and reduced inter-response interval (increased response vigour) after footshock. In the 2-lever test, mice favoured the FR2 lever and particularly at post-reward trials. CDZ increased choice of FR2 and FR20 responding after footshock, reduced response latency overall, and increased response vigour at the FR2 lever and after footshock specifically.

CONCLUSIONS

Mouse operant conflict tests, especially 2-lever choice, allow for the translational study of distinct anxiety components. CDZ influences each component by ameliorating the impact of both previous punishment and potential future punishment.

Minimizing threat via heuristic and optimal policies recruits hippocampus and medial prefrontal cortex

Jointly minimizing multiple threats over extended time horizons enhances survival. Consequently, many tests of approach-avoidance conflicts incorporate multiple threats for probing corollaries of animal and human anxiety. To facilitate computations necessary for threat minimization, the human brain may concurrently harness multiple decision policies and associated neural controllers, but it is unclear which. We combine a task that mimics foraging under predation with behavioural modelling and functional neuroimaging. Human choices rely on immediate predator probability-a myopic heuristic policy-and on the optimal policy, which integrates all relevant variables. Predator probability relates positively and the associated choice uncertainty relates negatively to activations in the anterior hippocampus, amygdala and dorsolateral prefrontal cortex. The optimal policy is positively associated with dorsomedial prefrontal cortex activity. We thus provide a decision-theoretic outlook on the role of the human hippocampus, amygdala and prefrontal cortex in resolving approach-avoidance conflicts relevant for anxiety and integral for survival.

Stress, aggression, and the balance of approach and avoidance

Stress is a well-established risk factor for many mental disorders including anxiety disorders or substance abuse. A hallmark of these disorders is an imbalance between behavioral approach and avoidance in situations with approach-avoidance conflicts and unclear outcomes. However, if and how stress affects human behavior in approach-avoidance conflicts is largely unknown. To investigate the effects of stress on approach-avoidance behavior, 80 participants underwent a stress or control manipulation before performing an approach-avoidance conflict task. Stress markedly increased behavioral inhibition when threats were distant and accelerated responses when threats were close; suggesting that stress amplifies the importance of threat distance. However, participants high in trait aggression showed increased approach behavior, particularly when stressed. These findings indicate that stress generally leads to enhanced avoidance, but induces approach in individuals prone to aggression, with important implications for stress-related psychopathologies.

Pavlovian-to-instrumental transfer after human threat conditioning

Threat conditioning is a common associative learning model with translational relevance. How threat-conditioned cues impact on formally unrelated instrumental behavior in humans is not well known. Such an effect is known as Pavlovian-to-instrumental transfer (PIT). While PIT with aversive primary Pavlovian reinforcers is established in nonhuman animals, this is less clear in humans, where secondary reinforcers or instructed instrumental responses are most often investigated. We modified an existing human PIT procedure to include primary reinforcers. Participants first learned to obtain (or avoid losing) appetitive instrumental reinforcement (chocolate) by appropriate approach or avoidance actions. They either had to act (Go) or to withhold an action (NoGo), and in the Go condition either to approach a reward target to collect it or to withdraw from the reward target to avoid losing it. Then they learned to associate screen color (CS) with aversive Pavlovian reinforcement (electric shock US). In the transfer phase, we conducted the instrumental task during the presence of Pavlovian CS. In a first experiment, we show that the aversive Pavlovian CS+, compared to CS−, increased response rate in Go-Withdraw trials, i.e., induce conditioned facilitation of avoidance responses. This finding was confirmed in a second and independent experiment with an increased number of Go-Withdraw trials. Notably, we observed no appreciable conditioned suppression of approach responses. Effect size to distinguish CS+/CS− in Go-Withdraw trials was d = 0.42 in the confirmation sample. This would require n = 37 participants to demonstrate threat learning with 80% power. Thus, the effect size is on a practically useful scale although smaller than for model-based analysis of autonomic measures. In summary, our results indicate conditioned facilitation of formally unrelated instrumental avoidance behavior in humans and provide a novel behavioral threat learning measure that requires only key presses.

Computational Models of Anxiety: Nascent Efforts and Future Directions

Computational approaches to understanding the algorithms of the mind are just beginning to pervade the field of clinical psychology. In the present article, we seek to explain in simple terms why this approach is indispensable to pursuing explanations of psychological phenomena broadly, and we review nascent efforts to use this lens to understand anxiety. We conclude with future directions that will be required to advance algorithmic accounts of anxiety. Ultimately, the surplus explanatory value of computational models of anxiety, above and beyond existing neurobiological models of anxiety, impugns the naively reductionist claim that neurobiological models are sufficient to explain anxiety.

A competitive model for striatal action selection

The direct and indirect pathway striatal medium spiny neurons (dMSNs and iMSNs) have long been linked to action selection, but the precise roles of these neurons in this process remain unclear. Here, we review different models of striatal pathway function, focusing on the classic "go/no-go" model which posits that dMSNs facilitate movement while iMSNs inhibit movement, and the "complementary" model, which argues that dMSNs facilitate the selection of specific actions while iMSNs inhibit potentially conflicting actions. We discuss the merits and shortcomings of these models and propose a "competitive" model to explain the contribution of these two pathways to behavior. The "competitive" model argues that rather than inhibiting conflicting actions, iMSNs are tuned to the same actions that dMSNs facilitate, and the two populations "compete" to determine the animal's behavioral response. This model provides a theoretical explanation for how these pathways work together to select actions. In addition, it provides a link between action selection and behavioral reinforcement, via modulating synaptic strength at inputs onto dMSNs and iMSNs. Finally, this model makes predictions about how imbalances in the activity of these pathways may underlie behavioral traits associated with psychiatric disorders. Understanding the roles of these striatal pathways in action selection may help to clarify the neuronal mechanisms of decision-making under normal and pathological conditions.

Effect of valproate and pregabalin on human anxiety-like behaviour in a randomised controlled trial

Valproate is an anticonvulsant drug with strong preclinical evidence for reducing anxiety behaviour in rodents but no clear clinical evidence. To motivate clinical trials, we here investigate the use of valproate in a translational human model of anxiety behaviour. In a double-blind, randomised, placebo-controlled trial, n = 118 healthy participants played a previously validated approach/avoidance conflict computer game to measure anxiety-like behaviour, while under 400 mg valproate, under 200 mg of the established anxiolytic/anticonvulsant pregabalin, or under placebo. Saccadic peak velocity and subjective ratings were assessed to control for drug-induced sedation. Compared to placebo, valproate and pregabaline were anxiolytic in the primary outcome, and several secondary outcomes. Bayesian model comparison decisively demonstrated no differences between the two drugs. Subjective and objective sedation was significantly more pronounced under pregabalin than valproate, but did not explain anxiolytic effects. We demonstrate acute anxiolytic properties of valproate in healthy humans. Both drugs have similar anxiolytic properties at the doses used. Valproate is less sedative than pregabalin. Our results suggest clinical trials on the use of valproate in anxiolytic treatment. More generally, we propose a strategy of screening drugs in human preclinical models that can directly be compared across species, such as the approach/avoidance conflict computer game used here. This approach could thus facilitate translational anxiety research.

Heuristic and optimal policy computations in the human brain during sequential decision-making

Optimal decisions across extended time horizons require value calculations over multiple probabilistic future states. Humans may circumvent such complex computations by resorting to easy-to-compute heuristics that approximate optimal solutions. To probe the potential interplay between heuristic and optimal computations, we develop a novel sequential decision-making task, framed as virtual foraging in which participants have to avoid virtual starvation. Rewards depend only on final outcomes over five-trial blocks, necessitating planning over five sequential decisions and probabilistic outcomes. Here, we report model comparisons demonstrating that participants primarily rely on the best available heuristic but also use the normatively optimal policy. FMRI signals in medial prefrontal cortex (MPFC) relate to heuristic and optimal policies and associated choice uncertainties. Crucially, reaction times and dorsal MPFC activity scale with discrepancies between heuristic and optimal policies. Thus, sequential decision-making in humans may emerge from integration between heuristic and optimal policies, implemented by controllers in MPFC.

Alhough humans often make a series of related decisions, it is unknown whether this is done by relying on optimal or heuristic strategies. Here, the authors show that humans rely on both the best heuristic and the optimal policy, and that these strategies are controlled by parts of the medial prefrontal cortex.

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning

BACKGROUND

Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear.

METHODS

Reinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock.

RESULTS

We show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress.

CONCLUSIONS

This formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders.

Amygdala Lesions Reduce Anxiety-like Behavior in a Human Benzodiazepine-Sensitive Approach-Avoidance Conflict Test

BACKGROUND

Rodent approach-avoidance conflict tests are common preclinical models of human anxiety disorder. Their translational validity mainly rests on the observation that anxiolytic drugs reduce rodent anxiety-like behavior. Here, we capitalized on a recently developed approach-avoidance conflict computer game to investigate the impact of benzodiazepines and of amygdala lesions on putative human anxiety-like behavior. In successive epochs of this game, participants collect monetary tokens on a spatial grid while under threat of virtual predation.

METHODS

In a preregistered, randomized, double-blind, placebo-controlled trial, we tested the effect of a single dose (1 mg) of lorazepam (n = 59). We then compared 2 patients with bilateral amygdala lesions due to Urbach-Wiethe syndrome with age- and gender-matched control participants (n = 17). Based on a previous report, the primary outcome measure was the effect of intra-epoch time (i.e., an adaptation to increasing potential loss) on presence in the safe quadrant of the spatial grid. We hypothesized reduced loss adaptation in this measure under lorazepam and in patients with amygdala lesions.

RESULTS

Lorazepam and amygdala lesions reduced loss adaptation in the primary outcome measure. We found similar results in several secondary outcome measures. The relative reduction of anxiety-like behavior in patients with amygdala lesions was qualitatively and quantitatively indistinguishable from an impact of anterior hippocampus lesions found in a previous report.

CONCLUSIONS

Our results establish the translational validity of human approach-avoidance conflict tests in terms of anxiolytic drug action. We identified the amygdala, in addition to the hippocampus, as a critical structure in human anxiety-like behavior.

Dissecting the Function of Hippocampal Oscillations in a Human Anxiety Model

Neural oscillations in hippocampus and medial prefrontal cortex (mPFC) are a hallmark of rodent anxiety models that build on conflict between approach and avoidance. Yet, the function of these oscillations, and their expression in humans, remain elusive. Here, we used magnetoencephalography (MEG) to investigate neural oscillations in a task that simulated approach–avoidance conflict, wherein 23 male and female human participants collected monetary tokens under a threat of virtual predation. Probability of threat was signaled by color and learned beforehand by direct experience. Magnitude of threat corresponded to a possible monetary loss, signaled as a quantity. We focused our analyses on an a priori defined region-of-interest, the bilateral hippocampus. Oscillatory power under conflict was linearly predicted by threat probability in a location consistent with right mid-hippocampus. This pattern was specific to the hippocampus, most pronounced in the gamma band, and not explained by spatial movement or anxiety-like behavior. Gamma power was modulated by slower theta rhythms, and this theta modulation increased with threat probability. Furthermore, theta oscillations in the same location showed greater synchrony with mPFC theta with increased threat probability. Strikingly, these findings were not seen in relation to an increase in threat magnitude, which was explicitly signaled as a quantity and induced similar behavioral responses as learned threat probability. Thus, our findings suggest that the expression of hippocampal and mPFC oscillatory activity in the context of anxiety is specifically linked to threat memory. These findings resonate with neurocomputational accounts of the role played by hippocampal oscillations in memory.

SIGNIFICANCE STATEMENT We use a biologically relevant approach–avoidance conflict test in humans while recording neural oscillations with magnetoencephalography to investigate the expression and function of hippocampal oscillations in human anxiety. Extending nonhuman studies, we can assign a possible function to hippocampal oscillations in this task, namely threat memory communication. This blends into recent attempts to elucidate the role of brain synchronization in defensive responses to threat.

Algorithms for survival: a comparative perspective on emotions

The nature and neural implementation of emotions is the subject of vigorous debate. Here, we use Bayesian decision theory to address key complexities in this field and conceptualize emotions in terms of their relationship to survival-relevant behavioural choices. Decision theory indicates which behaviours are optimal in a given situation; however, the calculations required are radically intractable. We therefore conjecture that the brain uses a range of pre-programmed algorithms that provide approximate solutions. These solutions seem to produce specific behavioural manifestations of emotions and can also be associated with core affective dimensions. We identify principles according to which these algorithms are implemented in the brain and illustrate our approach by considering decision making in the face of proximal threat.

Modeling Trait Anxiety: From Computational Processes to Personality

Computational methods are increasingly being applied to the study of psychiatric disorders. Often, this involves fitting models to the behavior of individuals with subclinical character traits that are known vulnerability factors for the development of psychiatric conditions. Anxiety disorders can be examined with reference to the behavior of individuals high in "trait" anxiety, which is a known vulnerability factor for the development of anxiety and mood disorders. However, it is not clear how this self-report measure relates to neural and behavioral processes captured by computational models. This paper reviews emerging computational approaches to the study of trait anxiety, specifying how interacting processes susceptible to analysis using computational models could drive a tendency to experience frequent anxious states and promote vulnerability to the development of clinical disorders. Existing computational studies are described in the light of this perspective and appropriate targets for future studies are discussed.

The cognitive architecture of anxiety-like behavioral inhibition

The combination of reward and potential threat is termed approach/avoidance conflict and elicits specific behaviors, including passive avoidance and behavioral inhibition (BI). Anxiety-relieving drugs reduce these behaviors, and a rich psychological literature has addressed how personality traits dominated by BI predispose for anxiety disorders. Yet, a formal understanding of the cognitive inference and planning processes underlying anxiety-like BI is lacking. Here, we present and empirically test such formalization in the terminology of reinforcement learning. We capitalize on a human computer game in which participants collect sequentially appearing monetary tokens while under threat of virtual “predation.” First, we demonstrate that humans modulate BI according to experienced consequences. This suggests an instrumental implementation of BI generation rather than a Pavlovian mechanism that is agnostic about action outcomes. Second, an internal model that would make BI adaptive is expressed in an independent task that involves no threat. The existence of such internal model is a necessary condition to conclude that BI is under model-based control. These findings relate a plethora of human and nonhuman observations on BI to reinforcement learning theory, and crucially constrain the quest for its neural implementation.

Emergence in extinction of enhanced and persistent responding to ambiguous aversive cues is associated with high MAOA activity in the prelimbic cortex

There is a great deal of individual variability in the emotional outcomes of potentially traumatic events, and the underlying mechanisms are only beginning to be understood. In order to further our understanding of individual trajectories to trauma, its vulnerability and resilience, we adapted a model of fear expression to ambiguous vs perfect cues in adult male rats, and examined long-term fear extinction, 2, 3, and 50 days from acquisition. After the final conditioned fear test, mitochondrial enzyme monoamine oxidase A (MAOA) function was examined. In order to identify associations between this function and behavioral expression, an a posteri median segregation approach was adopted, and animals were classified as high or low responding according to level of freezing to the ambiguous cue at remote testing, long after the initial extinction. Those individuals characterized by their higher response showed a freezing pattern that persisted from their previous extinction sessions, in spite of their acquisition levels being equivalent to the low-freezing group. Furthermore, unlike more adaptive individuals, freezing levels of high-freezing animals even increased at initial extinction, to almost double their acquisition session levels. Controlling for perfect cue response at remote extinction, greater ambiguous threat cue response was associated with enhanced prelimbic cortex MAOA functional activity. These findings underscore MAOA as a potential target for the development of interventions to mitigate the impact of traumatic experiences.

•

Potentially traumatic event outcomes vary and mechanisms are poorly understood.

•

We examined fear extinction of perfect or ambiguous cues in adult male rats.

•

Higher freezing to ambiguous cue in extinction yet followed equivalent acquisition.

•

Ambiguous cue response was associated with higher prelimbic cortex MAOA function.

•

These findings support targeting MAOA to mitigate impact of traumatic experiences.

Safety out of control: dopamine and defence

We enjoy a sophisticated understanding of how animals learn to predict appetitive outcomes and direct their behaviour accordingly. This encompasses well-defined learning algorithms and details of how these might be implemented in the brain. Dopamine has played an important part in this unfolding story, appearing to embody a learning signal for predicting rewards and stamping in useful actions, while also being a modulator of behavioural vigour. By contrast, although choosing correct actions and executing them vigorously in the face of adversity is at least as important, our understanding of learning and behaviour in aversive settings is less well developed. We examine aversive processing through the medium of the role of dopamine and targets such as D2 receptors in the striatum. We consider critical factors such as the degree of control that an animal believes it exerts over key aspects of its environment, the distinction between 'better' and 'good' actual or predicted future states, and the potential requirement for a particular form of opponent to dopamine to ensure proper calibration of state values.