Gambling-like behavior in pigeons: 'jackpot' signals promote maladaptive risky choice

Hyperbolae

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable-interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.

Midsession reversal task with starlings: A quantitative test of the timing hypothesis

In the Mid-Session Reversal task (MSR), an animal chooses between two options, S1 and S2. Rewards follow S1 but not S2 from trials 1-40, and S2 but not S1 from trials 41-80. With pigeons, the psychometric function relating S1 choice proportion to trial number starts close to 1 and ends close to 0, with indifference (PSE) close to trial 40. Surprisingly, pigeons make anticipatory errors, choosing S2 before trial 41, and perseverative errors, choosing S1 after trial 40. These errors suggest that they use time into the session as the preference reversal cue. We tested this timing hypothesis with 10 Spotless starlings. After learning the MSR task with a T-s Inter-Trial Interval (ITI), they were exposed to either 2 T or T/2 ITIs during testing. Doubling the ITI should shift the psychometric function to the left and halve its PSE, whereas halving the ITI should shift the function to the right and double its PSE. When the starlings received one pellet per reward, the ITI manipulation was effective: The psychometric functions shifted in the direction and by the amount predicted by the timing hypothesis. However, non-temporal cues also influenced choice.

Space, feature, and risk sensitivity in homing pigeons (Columba livia): Broadening the conversation on the role of the avian hippocampus in memory

David Sherry has been a pioneer in investigating the avian hippocampal formation (HF) and spatial memory. Following on his work and observations that HF is sensitive to the occurrence of reward (food), we were interested in carrying out an exploratory study to investigate possible HF involvement in the representation goal value and risk. Control sham-lesioned and hippocampal-lesioned pigeons were trained in an open field to locate one food bowl containing a constant two food pellets on all trials, and two variable bowls with one containing five pellets on 75% (High Variable) and another on 25% (Low Variable) of their respective trials (High-Variable and Low-Variable bowls were never presented together). One pairing of pigeons learned bowl locations (space); another bowl colors (feature). Trained to color, hippocampal-lesioned pigeons performed as rational agents in their bowl choices and were indistinguishable from the control pigeons, a result consistent with HF regarded as unimportant for non-spatial memory. By contrast, when trained to location, hippocampal-lesioned pigeons differed from the control pigeons. They made more first-choice errors to bowls that never contained food, consistent with a role of HF in spatial memory. Intriguingly, the hippocampal-lesioned pigeons also made fewer first choices to both variable bowls, suggesting that hippocampal lesions resulted in the pigeons becoming more risk averse. Acknowledging that the results are preliminary and further research is needed, the data nonetheless support the general hypothesis that HF-dependent memory representations of space capture properties of reward value and risk, properties that contribute to decision making when confronted with a choice.

Evaluation of Risk Behavior in Gambling Addicted and Opioid Addicted Individuals

Evidence suggests that both opioid addicted and gambling addicted individuals are characterized by higher levels of risky behavior in comparison to healthy people. It has been shown that the administration of substitution drugs can reduce cravings for opioids and the risky decisions made by individuals addicted to opioids. Although it is suggested that the neurobiological foundations of addiction are similar, it is possible that risk behaviors in opioid addicts may differ in detail from those addicted to gambling. The aim of this work was to compare the level of risk behavior in individuals addicted to opioid, with that of individuals addicted to gambling, using the Iowa Gambling Task (IGT). The score and response time during the task were measured. It was also observed, in the basis of the whole IGT test, that individuals addicted to gambling make riskier decisions in comparison to healthy individuals from the control group but less riskier decisions in comparison to individuals addicted to opioids, before administration of methadone and without any statistically significant difference after administration of methadone-as there has been growing evidence that methadone administration is strongly associated with a significant decrease in risky behavior.

The midsession reversal task: A theoretical analysis

In the midsession reversal task, choice of one stimulus (S1) is correct for the first half of each session and choice of the other stimulus (S2) is correct for the last half of each session. Although humans and rats develop very close to what has been called a win-stay/lose-shift response strategy, pigeons do not. Pigeons start choosing S2 before the reversal, making anticipatory errors, and they keep choosing S1 after the reversal, making perseverative errors. Research suggests that the pigeons are timing the reversal from the start of the session. However, making the reversal unpredictable does not discourage the pigeons from timing. Curiously, pigeons' accuracy improves if one decreases the value of the S2 stimulus relative to the S1 stimulus. Another form of asymmetry between S1 and S2 can be found by varying, over trials, the number of S1 or S2 stimuli. Counterintuitively, if the number of S2 stimuli varies, it results in a large increase in anticipatory errors but little increase in perseverative errors. However, if the number of S1 stimuli varies over trials, it results in a large increase in perseverative errors but no increase in anticipatory errors. These last two effects suggest that in the original midsession reversal task, the pigeons are learning to reject S2 during the first half of each session and learning to reject S1 during the last half of each session. These results suggest that reject learning may also play an important role in the learning of simple simultaneous discriminations.

Pigeons can learn a difficult discrimination if reinforcement is delayed following choice

Delaying reinforcement typically has been thought to retard the rate of acquisition of an association, but there is evidence that it may facilitate acquisition of some difficult simultaneous discriminations. After describing several cases in which delaying reinforcement can facilitate acquisition, we suggest that under conditions in which the magnitude of reinforcement is difficult to discriminate, the introduction of a delay between choice and reinforcement can facilitate the discrimination. In the present experiment, we tested the hypothesis that the discrimination between one pellet of food for choice of one alternative and two pellets of food for choice of another may be a difficult discrimination when choice consists of a single peck. If a 10-s delay occurs between choice and reinforcement, however, the discrimination is significantly easier. It is suggested that when discrimination between the outcomes of a choice is difficult and impulsive choice leads to immediate reinforcement, acquisition may be retarded. Under these conditions, the introduction of a brief delay may facilitate acquisition.

Challenges and Opportunities in Animal Models of Gambling-like Behavior

In the past several years, there has been an explosion of interest in animal models of risk-based decision-making, a fundamental process associated with gambling disorder. While early work focused on establishing various tasks for assaying decision-making, current studies are determining the (subtle and not-so-subtle) influence of cues in driving risky decisions to better understand problem gambling. In addition, these decision-making paradigms are now being used to investigate comorbid conditions such as substance dependence or brain injury and replicating observations from human patients. These animal models have now developed to a point where therapeutic interventions may be assessed for not just gambling disorder, but also a number of other conditions which engender risky decision-making.

Midsession reversal learning by pigeons: Effect on accuracy of increasing the number of stimuli associated with one of the alternatives

The midsession reversal task involves a simultaneous discrimination in which choice of one stimulus (S1) is correct for the first 40 trials and choice of the other stimulus (S2) is correct for the last 40 trials of each 80-trial session. When pigeons are trained on the midsession reversal task, they appear to use the passage of time from the start of the session as a cue to reverse. As the reversal approaches, they begin to make anticipatory errors, choosing S2 early, and following the reversal they make perseverative errors, continuing to choose S1. Recent research suggests that anticipatory errors can be reduced (while not increasing perseverative errors) by reducing the probability of reinforcement for correct S2 choices from 100% to 20%. A similar effect can be found by increasing the response requirement for choice of S2 from one peck to ten pecks. In the present experiments, we asked if a similar effect could be attained by increasing the number of stimuli that, over trials, could serve as S2. Instead, in both experiments, we found that increasing the number of S2 stimuli actually increased the number of anticipatory errors. Several interpretations of this result are provided, including the possibility that attention to the variable S2 stimuli may have interfered with attention to the S1 stimulus.

Rats' optimal choice behavior in a gambling-like task

Among the different procedures that model gambling behavior in non-human animals, the "suboptimal choice procedure" has been extensively employed for analyzing the impact of environmental cues on choice behavior. It has been repeatedly demonstrated that pigeons prefer an alternative that infrequently presents a stimulus that signals a larger amount of reinforcement, than another alternative that always presents a stimulus associated with a smaller amount of reinforcement, even though the net rate of reinforcement is lower in the former. In the present study, we tested rats in the magnitude version of the suboptimal choice procedure. Eight rats were given a choice between two alternatives: a) one in which a stimulus predicting the delivery of ten pellets was presented with probability (p) = 0.2 and a stimulus predicting zero pellets was presented with p = 0.8, and b) one in which either of two stimuli predicted the delivery of three pellets with p = 1.0. Contrary to the consistent and robust suboptimal behavior of pigeons, rats preferred the optimal alternative. This effect occurred despite the high index of discrimination of the stimuli associated with the different outcomes shown by the rats. The relevance of this result to the development of animal models of gambling behavior is discussed.

Frequency and value both matter in the suboptimal choice procedure

Pigeons chose between two options on a concurrent-chains task with a single response requirement in the initial link. The suboptimal option ended with food 20% of the time whereas the optimal option ended with food 80% of the time. During a Sig-Both condition, terminal-link stimuli on both options signaled whether or not food would occur. During a Sig-Sub condition, terminal-link stimuli on the suboptimal option provided differential signals, but stimuli on the optimal option did not differentially signal the food and no food outcomes. Initial-link choices revealed a clear preference for the optimal option in the Sig-Both condition, but preference shifted toward suboptimality in the Sig-Sub condition. These findings show that pigeon suboptimal choice is not singularly driven by signal value, as has been suggested, but also by reinforcer frequency.

The role of 'jackpot' stimuli in maladaptive decision-making: dissociable effects of D1/D2 receptor agonists and antagonists

RATIONALE

Laboratory experiments often model risk through a choice between a large, uncertain (LU) reward against a small, certain (SC) reward as an index of an individual's risk tolerance. An important factor generally lacking from these procedures are reward-associated cues that may modulate risk preferences.

OBJECTIVE

We tested whether the addition of cues signaling 'jackpot' wins to LU choices would modulate risk preferences and if these cue effects were mediated by dopaminergic signaling.

METHODS

Three groups of rats chose between LU and SC rewards for which the LU probability of reward decreased across blocks. The unsignaled group received a non-informative stimulus of trial outcome. The signaled group received a jackpot signal prior to reward delivery and blackout on losses. The signaled-light group received a similar jackpot for wins, but a salient loss signal distinct from the win signal.

RESULTS

Presenting win signals decreased the discounting of LU value for both signaled groups regardless of loss signal, while the unsignaled group showed discounting similar to previous research without cues. Pharmacological challenges with D1/D2 agonists and antagonists revealed that D1 antagonism increased and decreased sensitives to the relative probability of reward for unsignaled and signaled groups, respectively, while D2 agonists decreased sensitivities to the relative magnitude of reward.

CONCLUSION

The results highlight how signals predictive of wins can promote maladaptive risk taking in individuals, while loss signals have reduced effect. Additionally, the presence of reward-predictive cues may change the underlying neurobehavioral mechanisms mediating decision-making under risk.