Individuals with problem gambling and obsessive-compulsive disorder learn through distinct reinforcement mechanisms

Recent advances in understanding how compulsivity is related to behavioural addictions over their timecourse

Purpose of Review

Behavioural addictions involve loss of control over initially rewarding behaviours, which continue despite adverse consequences. Theoretical models suggest that these patterns of behaviour evolve over time, with compulsive and habitual behaviours held to reflect a loss of behavioural control. Compulsivity can be broadly described as a propensity for (or engagement in) repetitive behaviours that are not aligned with overall goals. Here, we consider whether compulsivity is associated with behavioural addictions at different stages of their development, based on self-report and neurocognitive measures.

Recent Findings

This review found that there is initial evidence that compulsive traits might predispose individuals to engage in problematic behaviours, and that self-report and neurocognitive measures of compulsivity are associated with severity of problematic behaviours even in the early stages of behavioural addictions. In the later stages of behavioural addiction, there is strong evidence for an association of gambling disorder with cognitive inflexibility, but less evidence for an association between compulsivity and other types of behavioural addiction.

Summary

Moving forwards, well-powered longitudinal studies, including studies using ecological momentary assessment (EMA), will be important in robustly developing our understanding of how compulsivity is related to behavioural addictions over their timecourse.

Computational and Neural Evidence for Altered Fast and Slow Learning from Losses in Problem Gambling

Learning occurs across multiple timescales, with fast learning crucial for adapting to sudden environmental changes, and slow learning beneficial for extracting robust knowledge from multiple events. Here, we asked if miscalibrated fast vs slow learning can lead to maladaptive decision-making in individuals with problem gambling. We recruited participants with problem gambling (PG; N = 20; 9 female and 11 male) and a recreational gambling control group without any symptoms associated with PG (N = 20; 10 female and 10 male) from the community in Los Angeles, CA. Participants performed a decision-making task involving reward-learning and loss-avoidance while being scanned with fMRI. Using computational model fitting, we found that individuals in the PG group showed evidence for an excessive dependence on slow timescales and a reduced reliance on fast timescales during learning. fMRI data implicated the putamen, an area associated with habit, and medial prefrontal cortex (PFC) in slow loss-value encoding, with significantly more robust encoding in medial PFC in the PG group compared to controls. The PG group also exhibited stronger loss prediction error encoding in the insular cortex. These findings suggest that individuals with PG have an impaired ability to adjust their predictions following losses, manifested by a stronger influence of slow value learning. This impairment could contribute to the behavioral inflexibility of problem gamblers, particularly the persistence in gambling behavior typically observed in those individuals after incurring loss outcomes.

Learning and memory processes in behavioural addiction: A systematic review

Similar to addictive substances, addictive behaviours such as gambling and gaming are associated with maladaptive modulation of key brain areas and functional networks implicated in learning and memory. Therefore, this review sought to understand how different learning and memory processes relate to behavioural addictions and to unravel their underlying neural mechanisms. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically searched four databases - PsycINFO, PubMed, Scopus, and Web of Science using the agreed-upon search string. Findings suggest altered executive function-dependent learning processes and enhanced habit learning in behavioural addiction. Whereas the relationship between working memory and behavioural addiction is influenced by addiction type, working memory aspect, and task nature. Additionally, long-term memory is incoherent in individuals with addictive behaviours. Consistently, neurophysiological evidence indicates alterations in brain areas and networks implicated in learning and memory processes in behavioural addictions. Overall, the present review argues that, like substance use disorders, alteration in learning and memory processes may underlie the development and maintenance of behavioural addictions.

Computational analysis of value learning and value-driven detection of neutral faces by young and older adults

The rapid detection of neutral faces with emotional value plays an important role in social relationships for both young and older adults. Recent psychological studies have indicated that young adults show efficient value learning for neutral faces and the detection of "value-associated faces," while older adults show slightly different patterns of value learning and value-based detection of neutral faces. However, the mechanisms underlying these processes remain unknown. To investigate this, we applied hierarchical reinforcement learning and diffusion models to a value learning task and value-driven detection task that involved neutral faces; the tasks were completed by young and older adults. The results for the learning task suggested that the sensitivity of learning feedback might decrease with age. In the detection task, the younger adults accumulated information more efficiently than the older adults, and the perceptual time leading to motion onset was shorter in the younger adults. In younger adults only, the reward sensitivity during associative learning might enhance the accumulation of information during a visual search for neutral faces in a rewarded task. These results provide insight into the processing linked to efficient detection of faces associated with emotional values, and the age-related changes therein.

Learning and metacognition under volatility in GD: Lower learning rates and distorted coupling between action and confidence

Background and aims

Decisions and learning processes are under metacognitive control, where confidence in one's actions guides future behaviour. Indeed, studies have shown that being more confident results in less action updating and learning, and vice versa. This coupling between action and confidence can be disrupted, as has been found in individuals with high compulsivity symptoms. Patients with Gambling Disorder (GD) have been shown to exhibit both higher confidence and deficits in learning.

Methods

In this study, we tested the hypotheses that patients with GD display increased confidence, reduced action updating and lower learning rates. Additionally, we investigated whether the action-confidence coupling was distorted in patients with GD. To address this, 27 patients with GD and 30 control participants performed a predictive inference task designed to assess action and confidence dynamics during learning under volatility. Action-updating, confidence and their coupling were assessed and computational modeling estimated parameters for learning rates, error sensitivity, and sensitivity to environmental changes.

Results

Contrary to our expectations, results revealed no significant group differences in action updating or confidence levels. Nevertheless, GD patients exhibited a weakened coupling between confidence and action, as well as lower learning rates.

Discussion and conclusions

This suggests that patients with GD may underutilize confidence when steering future behavioral choices. Ultimately, these findings point to a disruption of metacognitive control in GD, without a general overconfidence bias in neutral, non-incentivized volatile learning contexts.

Opponent learning with different representations in the cortico-basal ganglia pathways can develop obsession-compulsion cycle

Obsessive-compulsive disorder (OCD) has been suggested to be associated with impairment of model-based behavioral control. Meanwhile, recent work suggested shorter memory trace for negative than positive prediction errors (PEs) in OCD. We explored relations between these two suggestions through computational modeling. Based on the properties of cortico-basal ganglia pathways, we modeled human as an agent having a combination of successor representation (SR)-based system that enables model-based-like control and individual representation (IR)-based system that only hosts model-free control, with the two systems potentially learning from positive and negative PEs in different rates. We simulated the agent's behavior in the environmental model used in the recent work that describes potential development of obsession-compulsion cycle. We found that the dual-system agent could develop enhanced obsession-compulsion cycle, similarly to the agent having memory trace imbalance in the recent work, if the SR- and IR-based systems learned mainly from positive and negative PEs, respectively. We then simulated the behavior of such an opponent SR+IR agent in the two-stage decision task, in comparison with the agent having only SR-based control. Fitting of the agents' behavior by the model weighing model-based and model-free control developed in the original two-stage task study resulted in smaller weights of model-based control for the opponent SR+IR agent than for the SR-only agent. These results reconcile the previous suggestions about OCD, i.e., impaired model-based control and memory trace imbalance, raising a novel possibility that opponent learning in model(SR)-based and model-free controllers underlies obsession-compulsion. Our model cannot explain the behavior of OCD patients in punishment, rather than reward, contexts, but it could be resolved if opponent SR+IR learning operates also in the recently revealed non-canonical cortico-basal ganglia-dopamine circuit for threat/aversiveness, rather than reward, reinforcement learning, and the aversive SR + appetitive IR agent could actually develop obsession-compulsion if the environment is modeled differently.