Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans

Distinct neural computations scale the violation of expected reward and emotion in social transgressions

Traditional decision-making models conceptualize humans as adaptive learners utilizing the differences between expected and actual rewards (prediction errors, PEs) to maximize outcomes, but rarely consider the influence of violations of emotional expectations (emotional PEs) and how it differs from reward PEs. Here, we conducted a fMRI experiment (n = 43) using a modified Ultimatum Game to examine how reward and emotional PEs affect punishment decisions in terms of rejecting unfair offers. Our results revealed that reward relative to emotional PEs exerted a stronger prediction to punishment decisions. On the neural level, the left dorsomedial prefrontal cortex (dmPFC) was strongly activated during reward receipt whereas the emotions engaged the bilateral anterior insula. Reward and emotional PEs were also encoded differently in brain-wide multivariate patterns, with a more sensitive neural signature observed within fronto-insular circuits for reward PE. We further identified a fronto-insular network encompassing the left anterior cingulate cortex, bilateral insula, left dmPFC and inferior frontal gyrus that encoded punishment decisions. In addition, a stronger fronto-insular pattern expression under reward PE predicted more punishment decisions. These findings underscore that reward and emotional violations interact to shape decisions in complex social interactions, while the underlying neurofunctional PEs computations are distinguishable.

A Method for Evaluating Hunger and Thirst in Monkeys by Measuring Blood Ghrelin and Osmolality Levels

Hunger and thirst drive animals' consumption behavior and regulate their decision-making concerning rewards. We previously assessed the thirst states of monkeys by measuring blood osmolality under controlled water access and examined how these thirst states influenced their risk-taking behavior in decisions involving fluid rewards. However, hunger assessment in monkeys remains poorly performed. Moreover, the lack of precise measures for hunger states leads to another issue regarding how hunger and thirst states interact with each other in each individual. Thus, when controlling food access to motivate performance, it remains unclear how these two physiological needs are satisfied in captive monkeys. Here, we measured blood ghrelin and osmolality levels to respectively assess hunger and thirst in four captive macaques. Using an enzyme-linked immunosorbent assay, we identified that the levels of blood ghrelin, a widely measured hunger-related peptide hormone in humans, were high after 20 h of no food access (with ad libitum water). This reflects a typical controlled food access condition. One hour after consuming a regular dry meal, the blood ghrelin levels in three out of four monkeys decreased to within their baseline range. Additionally, blood osmolality measured from the same blood sample, the standard hematological index of hydration status, increased after consuming the regular dry meal with no water access. Thus, ghrelin and osmolality may reflect the physiological states of individual monkeys regarding hunger and thirst, suggesting that these indices can be used as tools for monitoring hunger and thirst levels that mediate an animal's decision to consume rewards.

Recent Opioid Use Impedes Range Adaptation in Reinforcement Learning in Human Addiction

BACKGROUND

Drugs like opioids are potent reinforcers thought to co-opt value-based decisions by overshadowing other rewarding outcomes, but how this happens at a neurocomputational level remains elusive. Range adaptation is a canonical process of fine-tuning representations of value based on reward context. Here, we tested whether recent opioid exposure impacts range adaptation in opioid use disorder, potentially explaining why shifting decision making away from drug taking during this vulnerable period is so difficult.

METHODS

Participants who had recently (<90 days) used opioids (n = 34) or who had abstained from opioid use for ≥ 90 days (n = 20) and comparison control participants (n = 44) completed a reinforcement learning task designed to induce robust contextual modulation of value. Two models were used to assess the latent process that participants engaged while making their decisions: 1) a Range model that dynamically tracks context and 2) a standard Absolute model that assumes stationary, objective encoding of value.

RESULTS

Control participants and ≥90-days-abstinent participants with opioid use disorder exhibited choice patterns consistent with range-adapted valuation. In contrast, participants with recent opioid use were more prone to learn and encode value on an absolute scale. Computational modeling confirmed the behavior of most control participants and ≥90-days-abstinent participants with opioid use disorder (75%), but a minority in the recent use group (38%), was better fit by the Range model than the Absolute model. Furthermore, the degree to which participants relied on range adaptation correlated with duration of continuous abstinence and subjective craving/withdrawal.

CONCLUSIONS

Reduced context adaptation to available rewards could explain difficulty deciding about smaller (typically nondrug) rewards in the aftermath of drug exposure.

An opponent striatal circuit for distributional reinforcement learning

Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards - an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons - D1 and D2 MSNs - contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.