Goals and habits in the brain

Biased competition between action representations

We propose a generalized version of the biased competition account of attention that may be applied to all domains of cognition. Based on our Generalized Biased Competition account, we propose a formal race model of selection of action representations. The model explains how action representations stored in long-term memory are competing for selection based on their match to the current environmental context and their importance weight. We then present results and model fits from three experiments using a recently developed multiple cue paradigm where several attention shifts with different associated reward values are competing. We show that participants were surprisingly efficient at selecting both when the number of cues and the number of possible reward values were increased. Only when we manipulated reward contingencies and knowledge of these, did the participants show suboptimal performance. The new Generalized Biased Competition account can also explain failures of executive control exemplified by goal neglect where instructions fail to influence behavior despite being retrievable. Finally, we argue that our model may provide a unified understanding of intentions, routines, and habits. Specifically, intentions, routines, and habits may be understood as a continuous range of the same fundamental form of action representation but with variation in their strength of long-term memory traces and importance weights.

Similar failures of consideration arise in human and machine planning

Humans are remarkably efficient at decision making, even in "open-ended" problems where the set of possible actions is too large for exhaustive evaluation. Our success relies, in part, on processes for calling to mind the right candidate actions. When these processes fail, the result is a kind of puzzle in which the value of a solution would be obvious once it is considered, but never gets considered in the first place. Recently, machine learning (ML) architectures have attained or even exceeded human performance on open-ended decision making tasks such as playing chess and Go. We ask whether the broad architectural principles that underlie ML success in these domains generate similar consideration failures to those observed in humans. We demonstrate a case in which they do, illuminating how humans make open-ended decisions, how this relates to ML approaches to similar problems, and how both architectures lead to characteristic patterns of success and failure.

Critical intelligence: Computing defensive behaviour

Characterising the mechanisms underlying naturalistic defensive behavior remains a significant challenge. While substantial progress has been made in unravelling the neural basis of tightly constrained behaviors, a critical gap persists in our comprehension of the circuits that implement algorithms capable of generating the diverse defensive responses observed outside experimental restrictions. Recent advancements in neuroscience technology now allow for an unprecedented examination of naturalistic behaviour. To help provide a theoretical grounding for this nascent experimental programme, we summarise the main computational and statistical challenges of defensive decision making, encapsulated in the concept of critical intelligence. Next, drawing from an extensive literature in biology, machine learning, and decision theory, we explore a range of candidate solutions to these challenges. While the proposed solutions offer insights into potential adaptive strategies, they also present inherent trade-offs and limitations in their applicability across different biological contexts. Ultimately, we propose series of experiments designed to differentiate between these candidate solutions, providing a roadmap for future investigations into the fundamental defensive algorithms utilized by biological agents and their neural implementation. Thus, our work aims to provide a roadmap towards broader understanding of how complex defensive behaviors are orchestrated in the brain, with implications for both neuroscience research and the development of more sophisticated artificial intelligence systems.

Studying human habit formation through motor sequence learning

Habits are automatic behaviors triggered by specific cues and are thought to optimize daily activities by reducing cognitive effort and enabling efficient and fast performance. Yet, they can also lead to inflexibility, preventing individuals from adapting to environmental changes. Since it has been difficult to examine habit formation in humans with traditional outcome devaluation paradigms, we applied a motor sequence learning task (MSLT) to study this process. Thirty-one participants (16 female, 28.4 ± 5.3 years old) completed the MSLT on two consecutive days. They implicitly learned to execute a 12-item motor sequence using four fingers, each corresponding to one of four distinct visual stimulus locations. Test blocks introduced sequence deviations by intermittently omitting one item of the sequence. We measured whether participants were able to flexibly adapt their behavior or would incorrectly execute the omitted response - a so-called action slip. Action slips serve as an indicator of automatization or behavioral inflexibility. Findings indicate that prolonged training led to faster response times and lower error rates in learning compared to random blocks, suggesting successful sequence learning and the emergence of automatic behaviors. Action slips increased with extensive training, demonstrating the shift towards automatic and inflexible responding, indicative of habit formation. The results highlight the utility of the MSLT in studying habit formation in humans and emphasize the role of extensive training, motor skills, and automaticity. The task offers a promising framework for investigating the neural and cognitive mechanisms underlying habitual behavior, providing new insights into the balance between habitual and goal-directed control.

Touchscreen Response Precision Is Sensitive to the Explore/Exploit Trade-off

The explore/exploit trade-off is a fundamental property of choice selection during reward-guided decision making, where the "same" choice can reflect either of these internal cognitive states. An unanswered question is whether the execution of a decision provides an underexplored measure of internal cognitive states. Touchscreens are increasingly used across species for cognitive testing and afford the ability to measure the precise location of choice touch responses. We examined how male and female mice in a restless bandit decision making task interacted with a touchscreen to determine if the explore/exploit trade-off, prior reward, and/or sex differences change the variability in the kinetics of touchscreen choices. During exploit states, successive touch responses are closer together than those made in an explore state, suggesting exploit states reflect periods of increased motor stereotypy. Although exploit decisions might be expected to be rewarded more frequently than explore decisions, we find that immediate past reward reduces choice variability independently of explore/exploit state. Male mice are more variable in their interactions with the touchscreen than females, even in low-variability trials such as exploit or following reward. These results suggest that as exploit behavior emerges in reward-guided decision making, all mice become less variable and more automated in both their choice and the actions taken to make that choice, but this occurs on a background of increased male variability. These data uncover the hidden potential for touchscreen decision making tasks to uncover the latent neural states that unite cognition and movement.

Erasing "bad memories": reversing aberrant synaptic plasticity as therapy for neurological and psychiatric disorders

Dopamine modulates corticostriatal plasticity in both the direct and indirect pathways of the cortico-striato-thalamo-cortical (CSTC) loops. These gradual changes in corticostriatal synaptic strengths produce long-lasting changes in behavioral responses. Under normal conditions, these mechanisms enable the selection of the most appropriate responses while inhibiting others. However, under dysregulated dopamine conditions, including a lack of dopamine release or dopamine signaling, these mechanisms could lead to the selection of maladaptive responses and/or the inhibition of appropriate responses in an experience-dependent and task-specific manner. In this review, we propose that preventing or reversing such maladaptive synaptic strengths and erasing such aberrant "memories" could be a disease-modifying therapeutic strategy for many neurological and psychiatric disorders. We review evidence from Parkinson's disease, drug-induced parkinsonism, L-DOPA-induced dyskinesia, obsessive-compulsive disorder, substance use disorders, and depression as well as research findings on animal disease models. Altogether, these studies allude to an emerging theme in translational neuroscience and promising new directions for therapy development. Specifically, we propose that combining pharmacotherapy with behavioral therapy or with deep brain stimulation (DBS) could potentially cause desired changes in specific neural circuits. If successful, one important advantage of correcting aberrant synaptic plasticity is long-lasting therapeutic effects even after treatment has ended. We will also discuss the potential molecular targets for these therapeutic approaches, including the cAMP pathway, proteins involved in synaptic plasticity as well as pathways involved in new protein synthesis. We place special emphasis on RNA binding proteins and epitranscriptomic mechanisms, as they represent a new frontier with the distinct advantage of rapidly and simultaneously altering the synthesis of many proteins locally.

Dorsal-Ventral Reinforcement Learning Network Connectivity and Incentive-Driven Changes in Exploration

Probabilistic reinforcement learning (RL) tasks assay how individuals make decisions under uncertainty. The use of internal models (model-based) or direct learning from experiences (model-free), and the degree of choice stochasticity across alternatives (i.e., exploration), can all be influenced by the state space of the decision-making task. There is considerable individual variation in the balance between model-based and model-free control during decision-making, and this balance is affected by incentive motivation. The effect of variable reward incentives on the arbitration between model-based and model-free learning remains understudied, and individual differences in neural signatures and cognitive traits that moderate the effect of reward on model-free/model-based control are unknown. Here we combined a two-stage decision-making task utilizing differing reward incentives with computational modeling, neuropsychological tests, and neuroimaging to address these questions. Results showed the prospect of greater reward decreased exploration of alternative options and increased the balance toward model-based learning. These behavioral effects were replicated across two independent datasets including both sexes. Individual differences in processing speed and analytical thinking style affected how reward altered the dependence on both systems. Using a systems neuroscience-inspired approach to resting-state functional connectivity, we found reduced exploration of the options during the first stage of our task under high relative to low incentives was predicted by increased cross-network coupling between ventral and dorsal RL circuitry. These findings suggest that integrity of functional connections between stimulus valuation (ventral) and action valuation (dorsal) RL networks is associated with changes in the balance between explore-exploit decisions under changing reward incentives.

Adaptive planning depth in human problem-solving

We humans are capable of solving challenging planning problems, but the range of adaptive strategies that we use to address them is not yet fully characterized. Here, we designed a series of problem-solving tasks that require planning at different depths. After systematically comparing the performance of participants and planning models, we found that when facing problems that require planning to a certain number of subgoals (from 1 to 8), participants make an adaptive use of their cognitive resources-namely, they tend to select an initial plan having the minimum required depth, rather than selecting the same depth for all problems. These results support the view of problem-solving as a bounded rational process, which adapts costly cognitive resources to task demands.

The role of psychological distress in understanding the relationship between habitual decision-making and addictive behaviors

Existing models are currently inadequate in explaining the relationship between habitual decision-making and different expressions of addictive behaviors. The current study investigates the role of psychological distress as a key factor in disrupting decision-making processes in the context of substance and behavioral addictions. A large community sample (N = 668) completed a gamified two-stage task to investigate the link between model-free (habitual) task behavior and a wide range of addictive behaviors. Addictive behaviors included substance use (alcohol, nicotine, and illicit drug use) and behavioral addictions (problematic use of the internet, addictive eating, shopping, and gambling). The relationship between habitual task behavior and addictive engagement was investigated using structural equation modelling with a bifactor latent variable structure, which was modeled and tested; one for substance use and one for behavioral addictions. For participants with higher levels of psychological distress, greater habitual task behavior was a significant predictor of behavioral addiction risk (β = -0.188, SE = 2.061, p = .016), specifically problematic use of the internet (β = -0.148, SE = 0.045, p = .018) and eating behaviors (β = -0.191, SE = 0.016, p < .001). These findings support our proposition that psychological distress disrupts cognitive control, leading to a greater reliance on habitual decision-making and non-substance addictive behaviors. This highlights the potential importance of habit-based interventions in combination with stress-reduction and mental health-promoting techniques to improve outcomes and minimize harm, especially in the context of behavioral addictions.

Location- and feature-based selection histories make independent, qualitatively distinct contributions to urgent visuomotor performance

Attention mechanisms guide visuomotor behavior by weighing physical salience and internal goals to prioritize stimuli as choices for action. Although less well studied, selection history, which reflects multiple facets of experience with recent events, is increasingly recognized as a distinct source of attentional bias. To examine how selection history impacts saccadic choices, we trained two macaque monkeys to perform an urgent version of an oddball search task in which a red target appeared among three green distracters, or vice versa. By imposing urgency, performance could be tracked continuously as it transitioned from uninformed guesses to informed choices as a function of processing time. This, in turn, permitted assessment of attentional control as manifest in motor biases, processing speed, and asymptotic accuracy. Here, we found that the probability of making a correct choice was strongly modulated by the histories of preceding target locations and target colors. Crucially, although both effects were gated by success (or reward), their dynamics were clearly distinct: whereas location history promoted a motor bias, color history modulated perceptual sensitivity, and these influences acted independently. Thus, combined selection histories can give rise to enormous swings in visuomotor performance even in simple tasks with highly discriminable stimuli.

Increasing the smoking cessation success rate by enhancing improvement of self-control through sleep-amplified memory consolidation: protocol of a randomized controlled, functional magnetic resonance study

BACKGROUND

Tobacco use disorder (TUD) remains a global health crisis characterized by high relapse rates despite extensive cessation efforts. This study aims to enhance treatment outcomes by addressing the cognitive and neural imbalances associated with habitual and goal-directed behaviours among individuals with TUD. We hypothesise that by integrating high-intensity interval training (HIIT), cognitive remediation treatment (CRT) via app-based chess training and a standard smoking cessation program (SCP) for cognitive control and sleep quality will be improved, thereby facilitating smoking cessation.

METHODS

The study will enrol 140 treatment-seeking smokers aged 18-65 years who meet the DSM-5 criteria for TUD. The participants will be randomly assigned to four groups: CRT + HIIT in the morning, CRT + HIIT in the evening, HIIT alone in the morning, and HIIT alone in the evening. Assessments will be conducted at baseline (T1), postintervention (T2), and at a three-month follow-up (T3) at the Central Institute of Mental Health in Mannheim, Germany. The primary outcomes include abstinence days or amount of alcohol consumed in cases of relapse, as well as craving reduction. Secondary outcomes include improvements in cognitive functions (working memory, response inhibition, and cognitive control), measured through neuropsychological tasks, functional magnetic resonance imaging (fMRI), polysomnography, and self-report questionnaires. The repeated-measures design allows for within-subject comparisons to evaluate intervention effectiveness.

DISCUSSION

This study aims to provide insights into the mechanisms through which combined CRT and evening HIIT, alongside improvements in sleep quality, can enhance smoking cessation outcomes. The hypothesised benefits on cognitive control and neural activity changes are expected to support better treatment adherence and reduced relapse rates among individuals with TUD. Addressing potential challenges such as high dropout rates through comprehensive participant support is crucial for the study's success. Findings from this research could inform future therapeutic strategies for TUD, potentially advancing addiction treatment approaches. The integration of novel interventions with established cessation programs underscores the study's significance in exploring holistic approaches to improving public health outcomes related to tobacco addiction.

TRIAL REGISTRATION

Registered at clinicaltrials.gov/ct2/show/NCT05726045 (Date 04.04.2024).

Proactive and reactive construction of memory-based preferences

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

Signatures of Perseveration and Heuristic-Based Directed Exploration in Two-Step Sequential Decision Task Behaviour

Processes formalized in classic Reinforcement Learning (RL) theory, such as model-based (MB) control, habit formation, and exploration have proven fertile in cognitive and computational neuroscience, as well as computational psychiatry. Dysregulations in MB control and exploration and their neurocomputational underpinnings play a key role across several psychiatric disorders. Yet, computational accounts mostly study these processes in isolation. The current study extended standard hybrid models of a widely-used sequential RL-task (two-step task; TST) employed to measure MB control. We implemented and compared different computational model extensions for this task to quantify potential exploration and perseveration mechanisms. In two independent data sets spanning two different variants of the task, an extended hybrid RL model with a higher-order perseveration and heuristic-based exploration mechanism provided the best fit. While a simpler model with complex perseveration only, was equally well equipped to describe the data, we found a robust positive effect of directed exploration on choice probabilities in stage one of the task. Posterior predictive checks further showed that the extended model reproduced choice patterns present in both data sets. Results are discussed with respect to implications for computational psychiatry and the search for neurocognitive endophenotypes.

Single-unit activity in the anterior claustrum during memory retrieval after trace fear conditioning

We have recently identified a group of claustral neurons that continuously maintain information associated with a fear-conditioned stimulus (CS) for at least tens of seconds, even after the CS has ceased. This "online state" refers to the persistent maintenance of threat-associated information, enabling it to be actively processed even after the threat has terminated. This state may involve reciprocal interactions of the claustral neurons with brain regions involved in decision-making, motor preparation, and adaptive behavioral responses. If these claustral neurons truly encode the online state, their function should remain independent of the modality of the threat stimulus or the specific defensive behavior exhibited. In this study, we used a tone cue and monitored freezing behavior in trace conditioning, in contrast to the light cue and escape behavior used in our recent study. During the retrieval test of trace conditioning, a subset of rostral-to-striatum claustrum (rsCla) neurons exhibited sustained activity in response to the CS, particularly during the trace interval. Importantly, we found a positive correlation between the activity of rsCla neurons and the magnitude of freezing during the trace interval, when intervals without freezing were excluded. Thus, this subset of rsCla neurons appears to exhibit the characteristics of 'online neurons' during memory retrieval following trace conditioning.

Cortical Beta Power Reflects the Influence of Pavlovian Cues on Human Decision-Making

Reward-predictive cues can affect decision-making by enhancing instrumental responses toward the same (specific transfer) or similar (general transfer) rewards. The main theories on cue-guided decision-making consider specific transfer as driven by the activation of previously learned instrumental actions induced by cues sharing the sensory-specific properties of the reward they are associated with. However, to date, such theoretical assumption has never been directly investigated at the neural level. We hypothesize that such reactivation occurs within the premotor system and could be mapped by lateralized beta (12-30 Hz) desynchronization, a widely used marker of action selection and decision-making policy. To test this hypothesis, 42 participants (22 females) performed a pavlovian-to-instrumental transfer paradigm, while electroencephalographic activity was recorded. We anticipated increased beta desynchronization during the transfer phase when cues promoting specific transfer were presented, compared with cues predicting general transfer and neutral cues. The evidence collected confirmed our hypothesis, thus providing the first neural evidence in favor of the theorized reactivation of instrumental actions and corroborating the presence of two dissociable neural pathways underpinning specific and general transfer.

Control adjustment costs limit goal flexibility: Empirical evidence and a computational account

A cornerstone of human intelligence is the ability to flexibly adjust our cognition and behavior as our goals change. For instance, achieving some goals requires efficiency, while others require caution. Different goals require us to engage different control processes, such as adjusting how attentive and cautious we are. Here, we show that performance incurs control adjustment costs when people adjust control to meet changing goals. Across four experiments, we provide evidence of these costs, and validate a dynamical systems model explaining the source of these costs. Participants performed a single cognitively demanding task under varying performance goals (e.g., being fast or accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model accounts for our empirical finding that people under-shoot their target control state more (i.e., exhibit larger adjustment costs) when goals switch rather than remain fixed (Study 1). Further validating our model, we show that the magnitude of this cost is increased when: distances between target states are larger (Study 2), there is less time to adjust to the new goal (Study 3), and goal switches are more frequent (Study 4). Our findings characterize the costs of adjusting control to meet changing goals, and show that these costs emerge directly from cognitive control dynamics. In so doing, they shed new light on the sources of and constraints on flexibility of goal-directed behavior.

Reward signals in the motor cortex: from biology to neurotechnology

Over the past decade, research has shown that the primary motor cortex (M1), the brain's main output for movement, also responds to rewards. These reward signals may shape motor output in its final stages, influencing movement invigoration and motor learning. In this Perspective, we highlight the functional roles of M1 reward signals and propose how they could guide advances in neurotechnologies for movement restoration, specifically brain-computer interfaces and non-invasive brain stimulation. Understanding M1 reward signals may open new avenues for enhancing motor control and rehabilitation.

Cognitive maps and schizophrenia

Structured internal representations ('cognitive maps') shape cognition, from imagining the future and counterfactual past, to transferring knowledge to new settings. Our understanding of how such representations are formed and maintained in biological and artificial neural networks has grown enormously. The cognitive mapping hypothesis of schizophrenia extends this enquiry to psychiatry, proposing that diverse symptoms - from delusions to conceptual disorganization - stem from abnormalities in how the brain forms structured representations. These abnormalities may arise from a confluence of neurophysiological perturbations (excitation-inhibition imbalance, resulting in attractor instability and impaired representational capacity) and/or environmental factors such as early life psychosocial stressors (which impinge on representation learning). This proposal thus links knowledge of neural circuit abnormalities, environmental risk factors, and symptoms.

Striatal cell-type specific stability and reorganization underlying agency and habit

Adaptive decision making requires agency, knowledge that actions produce particular outcomes. For well-practiced routines, agency is relinquished in favor of habit. Here, we asked how dorsomedial striatum D1+ and D2/A2A+ neurons contribute to agency and habit. We imaged calcium activity of these neurons as mice learned to lever press with agency and formed habits with overtraining. Whereas many D1+ neurons stably encoded actions throughout learning and developed encoding of reward outcomes, A2A+ neurons reorganized their encoding of actions from initial action-outcome learning to habit formation. Chemogenetic manipulations indicated that both D1+ and A2A+ neurons support action-outcome learning, but only D1+ neurons enable the use of such agency for adaptive, goal-directed decision making. These data reveal coordinated dorsomedial striatum D1+ and A2A+ function for the development of agency, cell-type specific stability and reorganization underlying agency and habit, and important insights into the neuronal circuits of how we learn and decide.

Maturation of striatal dopamine supports the development of habitual behavior through adolescence

Developmental trajectories during the transition from adolescence to adulthood contribute to the establishment of stable, adult forms of operation. Understanding the neural mechanisms underlying this transition is crucial for identifying variability in normal development and the onset of psychiatric disorders, which typically emerge during this time. Habitual behaviors can serve as a model for understanding brain mechanisms underlying the stabilization of adult behavior, while also conferring risk for psychopathologies. Dopaminergic (DA) processes in the basal ganglia are thought to facilitate the formation of habits; however, developmental trajectories of habits and the brain systems supporting them have not been characterized in vivo in developing humans. The current study examined trajectories of habitual behavior from adolescence to adulthood and sought to understand how the maturing striatal DA system may act as a potential mechanism in the process of habit formation. We used data from two longitudinal studies (combined n = 217, 10 - 32 years of age, 1-3 visits each, 320 total sessions) to characterize normative developmental trajectories of basal ganglia tissue iron concentration (a proxy for DA-related neurophysiology) and goal-direct and habitual control behaviors in a two-stage decision-making task. Tissue iron concentrations across the basal ganglia and habitual responding during the two-stage sequential decision-making task both increased with age (all p < 0.001). Importantly, habitual responding was associated with tissue iron concentrations in the putamen (F = 4.34, p = 0.014), such that increases in habitual responding were supported by increases in putamen tissue iron concentration during childhood through late adolescence. Exploratory analyses of further subdivisions of anatomical regions found that this association was specific to the posterior putamen. These results provide novel evidence in humans that habitual behavior continues to mature into adulthood and may be supported by increased specialization of reward systems.

Impaired arbitration between reward-related decision-making strategies in Alcohol Users compared to Alcohol Non-Users: a computational modeling study

Reinforcement learning studies propose that decision-making is guided by a tradeoff between computationally cheaper model-free (habitual) control and costly model-based (goal-directed) control. Greater model-based control is typically used under highly rewarding conditions to minimize risk and maximize gain. Although prior studies have shown impairments in sensitivity to reward value in individuals with frequent alcohol use, it is unclear how these individuals arbitrate between model-free and model-based control based on the magnitude of reward incentives. In this study, 81 individuals (47 frequent Alcohol Users and 34 Alcohol Non-Users) performed a modified 2-step learning task where stakes were sometimes high, and other times they were low. Maximum a posteriori fitting of a dual-system reinforcement-learning model was used to assess the degree of model-based control, and a utility model was used to assess risk sensitivity for the low- and high-stakes trials separately. As expected, Alcohol Non-Users showed significantly higher model-based control in higher compared to lower reward conditions, whereas no such difference between the two conditions was observed for the Alcohol Users. Additionally, both groups were significantly less risk-averse in higher compared to lower reward conditions. However, Alcohol Users were significantly less risk-averse compared to Alcohol Non-Users in the higher reward condition. Lastly, greater model-based control was associated with a less risk-sensitive approach in Alcohol Users. Taken together, these results suggest that frequent Alcohol Users may have impaired metacontrol, making them less flexible to varying monetary rewards and more prone to risky decision-making, especially when the stakes are high.

Leveraging cognitive neuroscience for making and breaking real-world habits

Habits are the behavioral output of two brain systems. A stimulus-response (S-R) system that encourages us to efficiently repeat well-practiced actions in familiar settings, and a goal-directed system concerned with flexibility, prospection, and planning. Getting the balance between these systems right is crucial: an imbalance may leave people vulnerable to action slips, impulsive behaviors, and even compulsive behaviors. In this review we examine how recent advances in our understanding of these competing brain mechanisms can be harnessed to increase the control over both making and breaking habits. We discuss applications in everyday life, as well as validated and emergent interventions for clinical populations affected by the balance between these systems. As research in this area accelerates, we anticipate a rapid influx of new insights into intentional behavioral change and clinical interventions, including new opportunities for personalization of these interventions based on the neurobiology, environmental context, and personal preferences of an individual.

Study Protocol for 'PsilOCD: A Pharmacological Challenge Study Evaluating the Effects of the 5-HT2A Agonist Psilocybin on the Neurocognitive and Clinical Correlates of Compulsivity'

BACKGROUND

Obsessive-compulsive disorder (OCD) is a complex condition marked by persistent distressing thoughts and repetitive behaviours. Despite its prevalence, the mechanisms behind OCD remain elusive, and current treatments are limited. This protocol outlines an investigative study for individuals with OCD, exploring the potential of psilocybin to improve key components of cognition implicated in the disorder. The PsilOCD study strives to assess the effects of low-moderate psilocybin treatment (10 mg) alongside non-interventional therapy on several facets of OCD. The main focus points of PsilOCD are cognitive flexibility, measured with cognitive tests, and neuroplasticity, assessed through electroencephalography (EEG).

METHODS

20 blinded participants with OCD will complete two dosing sessions, separated by four weeks, where they will receive 1 mg of psilocybin on the first and 10 mg on the second. The first dose serves as an active placebo, and the latter is a low-moderate dose that induces relatively mild-moderate emotional and perceptual effects. Participants will be supported by trained psychedelic therapists, who will sit with them during each dosing session and provide virtual preparation and integration sessions over the 12-week study period. Therapeutic support will be the same for both the 1 mg and 10 mg sessions. PsilOCD's primary outcomes include scores in the intradimensional-extradimensional (ID-ED) shift task, which is an established measure of cognitive flexibility, and neuroplasticity as quantified by a visual long-term potentiation (vLTP) task. This task is delivered as part of an EEG paradigm and measures acute quantified changes in neuroplasticity in the brain's visual system. The ID-ED task will be conducted twice, two days after each dosing session, and the EEG recordings will also be taken twice, immediately after each session. Secondary outcome assessments will include OCD and affective symptom severity, as well as an array of patient-reported outcome measures (PROMs), in the form of questionnaires designed to assess well-being, dissociable and well-established mood-related (affective) measures, and participants' subjective experience of the psilocybin experience.

DISCUSSION

This study's results are expected to offer critical insights into the neural mechanisms underlying the effects of psilocybin-assisted therapy in treating OCD, and whether these correlate with changes in the cognitive features of the condition. As a secondary aim, it will ascertain whether a low, tolerable dose is a feasible and efficacious clinical treatment, and will provide crucial data to guide the design of a potential follow-up randomised control trial (RCT).

Cost-benefit tradeoff mediates the transition from rule-based to memory-based processing during practice

Practice not only improves task performance but also changes task execution from rule- to memory-based processing by incorporating experiences from practice. However, how and when this change occurs is unclear. We test the hypothesis that strategy transitions in task learning can result from decision-making guided by cost-benefit analysis. Participants learn 2 task sequences and are then queried about the task type at a cued sequence and position. Behavioral improvement with practice can be accounted for by a computational model implementing cost-benefit analysis and the model-predicted strategy transition points align with the observed behavioral slowing. Model comparisons using behavioral data show that strategy transitions are better explained by a cost-benefit analysis across alternative strategies rather than solely on memory strength. Model-guided fMRI findings suggest that the brain encodes a decision variable reflecting the cost-benefit analysis and that different strategy representations are double-dissociated. Further analyses reveal that strategy transitions are associated with activation patterns in the dorsolateral prefrontal cortex and increased pattern separation in the ventromedial prefrontal cortex. Together, these findings support cost-benefit analysis as a mechanism of practice-induced strategy shift.

Contextualizing predictive minds

The structure of human memory seems to be optimized for efficient prediction, planning, and behavior. We propose that these capacities rely on a tripartite structure of memory that includes concepts, events, and contexts-three layers that constitute the mental world model. We suggest that the mechanism that critically increases adaptivity and flexibility is the tendency to contextualize. This tendency promotes local, context-encoding abstractions, which focus event- and concept-based planning and inference processes on the task and situation at hand. As a result, cognitive contextualization offers a solution to the frame problem-the need to select relevant features of the environment from the rich stream of sensorimotor signals. We draw evidence for our proposal from developmental psychology and neuroscience. Adopting a computational stance, we present evidence from cognitive modeling research which suggests that context sensitivity is a feature that is critical for maximizing the efficiency of cognitive processes. Finally, we turn to recent deep-learning architectures which independently demonstrate how context-sensitive memory can emerge in a self-organized learning system constrained by cognitively-inspired inductive biases.

Choosing the right frame: how context preferences facilitate subsequent decisions

Context shapes how we perceive choices and, therefore, how we decide between them. For instance, a large body of literature on the "framing effect" demonstrates that people become more risk-seeking when choices are framed in terms of losses. Despite this research, it remains unknown how people make choices between contexts and how these choices affect subsequent decision making. To address these questions, we designed the Frame Selection Task (FST). On each trial in the FST, participants first choose how risky and safe options are framed, either in terms of gains or losses, and then select between them. We found that participants exhibited frame preferences, with a predominant preference for the gain frame, and that they were willing to incur costs to select options within their preferred frame. Moreover, participants selected frames that aligned with their risk preferences: people with stronger risk aversion displayed a stronger gain-frame preference. These results demonstrate how people choose between contexts and that they can combine these preferences with cognitive biases to facilitate decision making.

Decision-making using the Iowa gambling test in unaffected first-degree relatives of obsessive-compulsive disorder: Comparison with healthy controls and patients with obsessive-compulsive disorder

Decision-making has been suggested as an endophenotype candidate for obsessive-compulsive disorder (OCD). However, few studies have examined whether decision-making under ambiguity is an endophenotype of OCD. This study aimed to investigate decision-making under ambiguity, as assessed by the Iowa Gambling Task (IGT), in patients with OCD and unaffected first-degree relatives (UFDR). Forty-seven non-medicated, non-co-morbid patients with OCD, 30 UFDR, and 47 healthy controls (HC) were compared in terms of decision-making using the IGT. The correlation between obsessive-compulsive symptoms and IGT performance was also investigated. Patients with OCD and UFDR performed worse than HC on the IGT. No correlation was found between obsessive-compulsive symptoms and IGT performance. A deficit in decision-making under ambiguity may be a trait and an endophenotype candidate for OCD.

Proactive and reactive construction of memory-based preferences

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

Context changes retrieval of prospective outcomes during decision deliberation

Foreseeing the future outcomes is the art of decision-making. Substantial evidence shows that, during choice deliberation, the brain can retrieve prospective decision outcomes. However, decisions are seldom made in a vacuum. Context carries information that can radically affect the outcomes of a choice. Nevertheless, most investigations of retrieval processes examined decisions in isolation, disregarding the context in which they occur. Here, we studied how context shapes prospective outcome retrieval during deliberation. We designed a decision-making task where participants were presented with object-context pairs and made decisions which led to a certain outcome. We show during deliberation, likely outcomes were retrieved in transient patterns of neural activity, as early as 3 s before participants decided. The strength of prospective outcome retrieval explains participants' behavioral efficiency, but only when context affects the decision outcome. Our results suggest context imparts strong constraints on retrieval processes and how neural representations are shaped during decision-making.

The affective gradient hypothesis: an affect-centered account of motivated behavior

Everyone agrees that feelings and actions are intertwined, but cannot agree how. According to dominant models, actions are directed by estimates of value and these values shape or are shaped by affect. I propose instead that affect is the only form of value that drives actions. Our mind constantly represents potential future states and how they would make us feel. These states collectively form a gradient reflecting feelings we could experience depending on actions we take. Motivated behavior reflects the process of traversing this affective gradient, towards desirable states and away from undesirable ones. This affective gradient hypothesis solves the puzzle of where values and goals come from, and offers a parsimonious account of apparent conflicts between emotion and cognition.

Topographically selective motor inhibition under threat of pain

ABSTRACT

Pain-related motor adaptations may be enacted predictively at the mere threat of pain, before pain occurrence. Yet, in humans, the neurophysiological mechanisms underlying motor adaptations in anticipation of pain remain poorly understood. We tracked the evolution of changes in corticospinal excitability (CSE) as healthy adults learned to anticipate the occurrence of lateralized, muscle-specific pain to the upper limb. Using a Pavlovian threat conditioning task, different visual stimuli predicted pain to the right or left forearm (experiment 1) or hand (experiment 2). During stimuli presentation before pain occurrence, single-pulse transcranial magnetic stimulation was applied over the left primary motor cortex to probe CSE and elicit motor evoked potentials from target right forearm and hand muscles. The correlation between participants' trait anxiety and CSE was also assessed. Results showed that threat of pain triggered corticospinal inhibition specifically in the limb where pain was expected. In addition, corticospinal inhibition was modulated relative to the threatened muscle, with threat of pain to the forearm inhibiting the forearm and hand muscles, whereas threat of pain to the hand inhibited the hand muscle only. Finally, stronger corticospinal inhibition correlated with greater trait anxiety. These results advance the mechanistic understanding of pain processes showing that pain-related motor adaptations are enacted at the mere threat of pain, as sets of anticipatory, topographically organized motor changes that are associated with the expected pain and are shaped by individual anxiety levels. Including such anticipatory motor changes into models of pain may lead to new treatments for pain-related disorders.

Dopamine dynamics are dispensable for movement but promote reward responses

Dopamine signalling modes differ in kinetics and spatial patterns of receptor activation1,2. How these modes contribute to motor function, motivation and learning has long been debated3-21. Here we show that action-potential-induced dopamine release is dispensable for movement initiation but supports reward-oriented behaviour. We generated mice with dopamine-neuron-specific knockout of the release site organizer protein RIM to disrupt action-potential-induced dopamine release. In these mice, rapid in vivo dopamine dynamics were strongly impaired, but baseline dopamine persisted and fully supported spontaneous movement. Conversely, reserpine-mediated dopamine depletion or blockade of dopamine receptors disrupted movement initiation. The dopamine precursor L-DOPA reversed reserpine-induced bradykinesia without restoring fast dopamine dynamics, a result that substantiated the conclusion that these dynamics are dispensable for movement initiation. In contrast to spontaneous movement, reward-oriented behaviour was impaired in dopamine-neuron-specific RIM knockout mice. In conditioned place preference and two-odour discrimination tasks, the mice effectively learned to distinguish the cues, which indicates that reward-based learning persists after RIM ablation. However, the performance vigour was reduced. During probabilistic cue-reward association, dopamine dynamics and conditioned responses assessed through anticipatory licking were disrupted. These results demonstrate that action-potential-induced dopamine release is dispensable for motor function and subsecond precision of movement initiation but promotes motivation and performance during reward-guided behaviours.

Cost-benefit Tradeoff Mediates the Rule- to Memory-based Processing Transition during Practice

Practice not only improves task performance but also changes task execution from rule- to memory-based processing by incorporating experiences from practice. However, how and when this change occurs is unclear. We test the hypothesis that strategy transitions in task learning can result from decision-making guided by cost-benefit analysis. Participants learn two task sequences and are then queried about the task type at a cued sequence and position. Behavioral improvement with practice can be accounted for by a computational model implementing cost-benefit analysis, and the model-predicted strategy transition points align with the observed behavioral slowing. Model comparisons using behavioral data show that strategy transitions are better explained by a cost-benefit analysis across alternative strategies rather than solely on memory strength. Model-guided fMRI findings suggest that the brain encodes a decision variable reflecting the cost-benefit analysis and that different strategy representations are double-dissociated. Further analyses reveal that strategy transitions are associated with activation patterns in the dorsolateral prefrontal cortex and increased pattern separation in the ventromedial prefrontal cortex. Together, these findings support cost-benefit analysis as a mechanism of practice-induced strategy shift.

Touchscreen response precision is sensitive to the explore/exploit tradeoff

The explore/exploit tradeoff is a fundamental property of choice selection during reward-guided decision making. In perceptual decision making, higher certainty decisions are more motorically precise, even when the decision does not require motor accuracy. However, while we can parametrically control uncertainty in perceptual tasks, we do not know what variables - if any - shape motor precision and reflect subjective certainty during reward-guided decision making. Touchscreens are increasingly used across species to measure choice, but provide no tactile feedback on whether an action is precise or not, and therefore provide a valuable opportunity to determine whether actions differ in precision due to explore/exploit state, reward, or individual variables. We find all three of these factors exert independent drives towards increased precision. During exploit states, successive touches to the same choice are closer together than those made in an explore state, consistent with exploit states reflecting higher certainty and/or motor stereotypy in responding. However, exploit decisions might be expected to be rewarded more frequently than explore decisions. We find that exploit choice precision is increased independently of a separate increase in precision due to immediate past reward, suggesting multiple mechanisms regulating choice precision. Finally, we see evidence that male mice in general are less precise in their interactions with the touchscreen than females, even when exploiting a choice. These results suggest that as exploit behavior emerges in reward-guided decision making, individuals become more motorically precise reflecting increased certainty, even when decision choice does not require additional motor accuracy, but this is influenced by individual differences and prior reward. These data uncover the hidden potential for touchscreen tasks in any species to uncover the latent neural states that unite cognition and movement.

Predictive Representations: Building Blocks of Intelligence

Adaptive behavior often requires predicting future events. The theory of reinforcement learning prescribes what kinds of predictive representations are useful and how to compute them. This review integrates these theoretical ideas with work on cognition and neuroscience. We pay special attention to the successor representation and its generalizations, which have been widely applied as both engineering tools and models of brain function. This convergence suggests that particular kinds of predictive representations may function as versatile building blocks of intelligence.

Sex differences in motivational biases over instrumental actions

Motivational (i.e., appetitive or aversive) cues can bias value-based decisions by affecting either direction and intensity of instrumental actions. Despite several findings describing important interindividual differences in these biases, whether biological sex can also play a role is still up to debate. By comparing females and males in both appetitive and aversive Pavlovian-to-Instrumental Transfer paradigms we found that, while motivational cues similarly bias the direction of instrumental actions in both sexes, the intensity of such actions is increased by the cue in male participants only. The present results constitute compelling evidence that a crucial motivational bias of daily actions directed to obtaining rewards or avoiding punishments is modulated by biological sex. This evidence sheds new light on the role of sex in motivational processes that underlie decision-making, highlighting the importance of considering sex as a crucial factor in future research on this topic.

Understanding dual process cognition via the minimum description length principle

Dual-process theories play a central role in both psychology and neuroscience, figuring prominently in domains ranging from executive control to reward-based learning to judgment and decision making. In each of these domains, two mechanisms appear to operate concurrently, one relatively high in computational complexity, the other relatively simple. Why is neural information processing organized in this way? We propose an answer to this question based on the notion of compression. The key insight is that dual-process structure can enhance adaptive behavior by allowing an agent to minimize the description length of its own behavior. We apply a single model based on this observation to findings from research on executive control, reward-based learning, and judgment and decision making, showing that seemingly diverse dual-process phenomena can be understood as domain-specific consequences of a single underlying set of computational principles.

A dual-pathway architecture enables chronic stress to disrupt agency and promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted systems neuroscience approach in male and female mice, we reveal a dual pathway, amygdala-striatal neuronal circuit architecture by which a recent history of chronic stress disrupts the action-outcome learning underlying adaptive agency and promotes the formation of inflexible habits. We found that the basolateral amygdala projection to the dorsomedial striatum is activated by rewarding events to support the action-outcome learning needed for flexible, goal-directed decision making. Chronic stress attenuates this to disrupt action-outcome learning and, therefore, agency. Conversely, the central amygdala projection to the dorsomedial striatum mediates habit formation. Following stress this pathway is progressively recruited to learning to promote the premature formation of inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to disrupt agency and promote habit. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and mental health conditions.

Model-free decision-making underlies motor errors in rapid sequential movements under threat

Our movements, especially sequential ones, are usually goal-directed, i.e., coupled with task-level goals. Consequently, cognitive strategies for decision-making and motor performance are likely to influence each other. However, evidence linking decision-making strategies and motor performance remains elusive. Here, we designed a modified version of the two-step task, named the two-step sequential movement task, where participants had to conduct rapid sequential finger movements to obtain rewards (n = 40). In the shock session, participants received an electrical shock if they made an erroneous or slow movement, while in the no-shock session, they only received zero reward. We found that participants who prioritised model-free decision-making committed more motor errors in the presence of the shock stimulus (shock sessions) than those who prioritised model-based decision-making. Using a mediation analysis, we also revealed a strong link between the balance of the model-based and the model-free learning strategies and sequential movement performances. These results suggested that model-free decision-making produces more motor errors than model-based decision-making in rapid sequential movements under the threat of stressful stimuli.

Aberrant neural computation of social controllability in nicotine-dependent humans

Social controllability, or the ability to exert control during social interactions, is crucial for optimal decision-making. Inability to do so might contribute to maladaptive behaviors such as smoking, which often takes place in social settings. Here, we examined social controllability in nicotine-dependent humans as they performed an fMRI task where they could influence the offers made by simulated partners. Computational modeling revealed that smokers under-estimated the influence of their actions and self-reported a reduced sense of control, compared to non-smokers. These findings were replicated in a large independent sample of participants recruited online. Neurally, smokers showed reduced tracking of forward projected choice values in the ventromedial prefrontal cortex, and impaired computation of social prediction errors in the midbrain. These results demonstrate that smokers were less accurate in estimating their personal influence when the social environment calls for control, providing a neurocomputational account for the social cognitive deficits in this population. Pre-registrations: OSF Registries|How interoceptive state interacts with value-based decision-making in addiction (fMRI study). OSF Registries|COVID-19: social cognition, mental health, and social distancing (online study).

The consequences of AI training on human decision-making

AI is now an integral part of everyday decision-making, assisting us in both routine and high-stakes choices. These AI models often learn from human behavior, assuming this training data is unbiased. However, we report five studies that show that people change their behavior to instill desired routines into AI, indicating this assumption is invalid. To show this behavioral shift, we recruited participants to play the ultimatum game, where they were asked to decide whether to accept proposals of monetary splits made by either other human participants or AI. Some participants were informed their choices would be used to train an AI proposer, while others did not receive this information. Across five experiments, we found that people modified their behavior to train AI to make fair proposals, regardless of whether they could directly benefit from the AI training. After completing this task once, participants were invited to complete this task again but were told their responses would not be used for AI training. People who had previously trained AI persisted with this behavioral shift, indicating that the new behavioral routine had become habitual. This work demonstrates that using human behavior as training data has more consequences than previously thought since it can engender AI to perpetuate human biases and cause people to form habits that deviate from how they would normally act. Therefore, this work underscores a problem for AI algorithms that aim to learn unbiased representations of human preferences.

Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity

The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.

Disorders of compulsivity: Deficits in arbitrating learning strategies

While previous research has shown that compulsivity is related to an imbalance between goal-directed and habitual learning systems, very little is known about whether this effect is due to the impairment of a single system or the impairment of the arbitration mechanism that determines which system controls behaviour at any given moment; the current study aims to address this disagreement. Nineteen alcohol use disorder, 30 obsessive-compulsive disorder (OCD) and 20 major depressive disorder patients and corresponding sex- and age-matched controls performed two-choice, three-stage Markov decision-making paradigm. Model-based and mode-free reinforcement learning models were used to independently fitted their behavioural data. Alcohol use disorder and OCD patients showed less model-based strategy choice than healthy controls in task conditions where the model-based strategy was optimal. Only OCD patients showed higher behavioural control system switching in task conditions where model-free use was optimal. Major depressive disorder patients did not differ from the matched control in both. These findings suggest that dysfunction in arbitration control between dual systems may be the basis for diverse disorders involving compulsivity.

Compulsive avoidance in youths and adults with OCD: an aversive pavlovian-to-instrumental transfer study

Compulsive behaviour may often be triggered by Pavlovian cues. Assessing how Pavlovian cues drive instrumental behaviour in obsessive-compulsive disorder (OCD) is therefore crucial to understand how compulsions develop and are maintained. An aversive Pavlovian-to-Instrumental transfer (PIT) paradigm, particularly one involving avoidance/cancellation of negative outcomes, can enable such investigation and has not previously been studied in clinical-OCD. Forty-one participants diagnosed with OCD (21 adults; 20 youths) and 44 controls (21 adults; 23 youths) completed an aversive PIT task. Participants had to prevent the delivery of unpleasant noises by moving a joystick in the correct direction. They could infer these correct responses by learning appropriate response-outcome (instrumental) and stimulus-outcome (Pavlovian) associations. We then assessed whether Pavlovian cues elicited specific instrumental avoidance responses (specific PIT) and induced general instrumental avoidance (general PIT). We investigated whether task learning and confidence indices influenced PIT strength differentially between groups. There was no overall group difference in PIT performance, although youths with OCD showed weaker specific PIT than youth controls. However, urge to avoid unpleasant noises and preference for safe over unsafe stimuli influenced specific and general PIT respectively in OCD, while PIT in controls was more influenced by confidence in instrumental and Pavlovian learning. Thus, in OCD, implicit motivational factors, but not learnt knowledge, may contribute to the successful integration of aversive Pavlovian and instrumental cues. This implies that compulsive avoidance may be driven by these automatic processes. Youths with OCD show deficits in specific PIT, suggesting cue integration impairments are only apparent in adolescence. These findings may be clinically relevant as they emphasise the importance of targeting such implicit motivational processes when treating OCD.

Bad habits-good goals? Meta-analysis and translation of the habit construct to alcoholism

Excessive alcohol consumption remains a global public health crisis, with millions suffering from alcohol use disorder (AUD, or simply "alcoholism"), leading to significantly reduced life expectancy. This review examines the interplay between habitual and goal-directed behaviors and the associated neurobiological changes induced by chronic alcohol exposure. Contrary to a strict habit-goal dichotomy, our meta-analysis of the published animal experiments combined with a review of human studies reveals a nuanced transition between these behavioral control systems, emphasizing the need for refined terminology to capture the probabilistic nature of decision biases in individuals with a history of chronic alcohol exposure. Furthermore, we distinguish habitual responding from compulsivity, viewing them as separate entities with diverse roles throughout the stages of the addiction cycle. By addressing species-specific differences and translational challenges in habit research, we provide insights to enhance future investigations and inform strategies for combatting AUD.

The Small G-Protein Rac1 in the Dorsomedial Striatum Promotes Alcohol-Dependent Structural Plasticity and Goal-Directed Learning in Mice

The small G-protein Ras-related C3 botulinum toxin substrate 1 (Rac1) promotes the formation of filamentous actin (F-actin). Actin is a major component of dendritic spines, and we previously found that alcohol alters actin composition and dendritic spine structure in the nucleus accumbens (NAc) and the dorsomedial striatum (DMS). To examine if Rac1 contributes to these alcohol-mediated adaptations, we measured the level of GTP-bound active Rac1 in the striatum of mice following 7 weeks of intermittent access to 20% alcohol. We found that chronic alcohol intake activates Rac1 in the DMS of male mice. In contrast, Rac1 is not activated by alcohol in the NAc and DLS of male mice or in the DMS of female mice. Similarly, closely related small G-proteins are not activated by alcohol in the DMS, and Rac1 activity is not increased in the DMS by moderate alcohol or natural reward. To determine the consequences of alcohol-dependent Rac1 activation in the DMS of male mice, we inhibited endogenous Rac1 by infecting the DMS of mice with an adeno-associated virus (AAV) expressing a dominant negative form of the small G-protein (Rac1-DN). We found that overexpression of AAV-Rac1-DN in the DMS inhibits alcohol-mediated Rac1 signaling and attenuates alcohol-mediated F-actin polymerization, which corresponded with a decrease in dendritic arborization and spine maturation. Finally, we provide evidence to suggest that Rac1 in the DMS plays a role in alcohol-associated goal-directed learning. Together, our data suggest that Rac1 in the DMS plays an important role in alcohol-dependent structural plasticity and aberrant learning.

The future of neuropsychology is digital, theory-driven, and Bayesian: a paradigmatic study of cognitive flexibility

Introduction

This study explores the transformative potential of digital, theory-driven, and Bayesian paradigms in neuropsychology by combining digital technologies, a commitment to evaluating theoretical frameworks, and Bayesian statistics. The study also examines theories of executive function and cognitive flexibility in a large sample of neurotypical individuals (N = 489).

Methods

We developed an internet-based Wisconsin Card-Sorting Task (iWCST) optimized for online assessment of perseveration errors (PE). Predictions of the percentage of PE, PE (%), in non-repetitive versus repetitive situations were derived from the established supervisory attention system (SAS) theory, non-repetitive PE (%) < repetitive PE (%), and the novel goal-directed instrumental control (GIC) theory, non-repetitive PE (%) > repetitive PE (%).

Results

Bayesian t-tests revealed the presence of a robust error suppression effect (ESE) indicating that PE are less likely in repetitive situations than in non-repetitive situations, contradicting SAS theory with posterior model probability p < 0.001 and confirming GIC theory with posterior model probability p > 0.999. We conclude that repetitive situations support cognitive set switching in the iWCST by facilitating the retrieval of goal-directed, instrumental memory that associates stimulus features, actions, and outcomes, thereby generating the ESE in neurotypical individuals. We also report exploratory data analyses, including a Bayesian network analysis of relationships between iWCST measures.

Discussion

Overall, this study serves as a paradigmatic model for combining digital technologies, theory-driven research, and Bayesian statistics in neuropsychology. It also provides insight into how this integrative, innovative approach can advance the understanding of executive function and cognitive flexibility and inform future research and clinical applications.

Decision-making under stress: A psychological and neurobiological integrative model

Understanding the impact of stress on cognitive processes, particularly decision-making, is crucial as it underpins behaviors essential for survival. However, research in this domain has yielded disparate results, with inconsistencies evident across stress-induction paradigms and drug administration protocols designed to investigate specific stress pathways or neuromodulators. Building upon empirical studies, this research identifies a multifaceted matrix of variables contributing to the divergent findings. This matrix encompasses factors such as the temporal proximity between stressors and decision tasks, the nature of stressors and decision contexts, individual characteristics including psychobiological profiles and affective states at the time of decision-making and even cultural influences. In response to these complexities, we propose a comprehensive model that integrates these relevant factors and their intricate interplay to elucidate the mechanisms governing decision-making during stressful events. By synthesizing these insights, our model not only refines existing paradigms but also provides a framework for future study designs, offering avenues for theoretical advancements and translational developments in the field of stress's impact on cognitive functions. This research contributes to a deeper understanding of the nuanced relationship between stress and decision-making, ultimately advancing our knowledge of cognitive processes under challenging conditions.

The Specification of a Computational Model of Physical Activity Habit

The influence of habit on physical activity is computationally modeled as the aggregated influence of past behavioral choices a person makes in a given context. We hypothesize that the influence of habit on behavior can be enhanced through engagement of the target behavior in a particular context or weakened through engagement of alternative behaviors in that context.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.