Sensitivity to changing contingencies in an impulsivity task

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Effects of methylphenidate on impulsive choice and delay aversion in Lewis rats

Attention-deficit/hyperactivity disorder (ADHD), a common behavioral disorder in children and young adults, is characterized by symptoms of impulsivity, inattention, and hyperactivity. The purpose of this study was to evaluate the Lewis rat strain as a model of ADHD by testing their impulsive choices. Lewis rats were compared to their source strain, the Wistar rat, on an impulsive choice task. Rats completed the tasks on and off methylphenidate, a commonly prescribed medication for ADHD. Off methylphenidate, Lewis rats made more impulsive choices than Wistar rats. Analyses of acquisition of choice behavior suggested that both strains were able to discriminate reward sizes, but Lewis rats still chose the smaller-sooner option more than the larger-later (LL) option when the delays to reward were the same. This may be due to an aversion to the LL lever, which was associated with the longest delays to reward. Higher doses of methylphenidate increased LL choices in Lewis rats but decreased LL choices in Wistar rats. Altogether, these results suggest Lewis rats may be a viable model for ADHD in individuals whose symptoms are characterized by impulsive choices.

Effects of the estrous cycle on impulsive choice and interval timing in female rats

Research in humans and animals shows differences in impulsive choice, which is a failure to wait for larger, delayed rewards, when comparing males and females. It is possible that fluctuations in sex hormones (estradiol and progesterone) across the reproductive cycle contribute to sex differences in impulsive choice. The current study delivered an impulsive choice task with peak interval trials to female rats while estrous cycles, the rodent reproductive cycle, were tracked over the course of the task. Female rats were more sensitive to changes in delay in the proestrus phase of the estrous cycle and made more larger-later choices when in estrus, particularly when the delay to the smaller reward was short. Estradiol increases dramatically during proestrus while progesterone peaks during estrus, suggesting that estradiol and progesterone may affect impulsive choice through mechanisms such as delay discounting, delay aversion, and/or timing processes. Analyses of timing of the choice task delays showed inconsistent effects of the estrous cycle across delays, suggesting that reward-timing interactions may have complicated how hormone fluctuations affected interval timing. Further research is needed to determine the mechanism underlying increased larger-later choices during the estrus phase, increased delay sensitivity during the proestrus phase, and variability in interval timing across delays and estrous cycle stages.

Learning when to wait and when to act

Choosing how long to wait in order to optimize reward is a complex decision. We embedded these decisions within a video-game environment in which the amount of reward smoothly increased the longer one waited. The availability of external cues varied in order to determine how they affected the decision to wait to achieve the goal of maximizing the reward rate. As a group, people were most optimal when they could directly observe the growth in reward, and this information overshadowed a static color cue that did not require extended observation. These results were considered within the context of improving the choice between acting versus waiting in order to maximize reward rates.

Sensitivity to Changing Environmental Conditions across Individuals with Subtype 2 Automatically Reinforced and Socially Reinforced Self-injury

Automatically reinforced Subtype 2 self-injurious behavior (ASIB) has been characterized as showing insensitivity to competing reinforcement contingencies in the contexts of both functional analyses and in treatment using reinforcement alone (Hagopian, Rooker, &Yenokyan, 2018). One question is whether this insensitivity is specific to Subtype 2 ASIB as response class in these contexts or whether it is represents a generalized response tendency of the individual that is evident across other response classes. To examine this question, we compared responding on a single-operant task under changing reinforcement schedules for three individuals with Subtype 2 ASIB, relative to a comparison group of three individuals with socially reinforced SIB (which is characterized by sensitivity to changes in reinforcement contingencies). As hypothesized, all individuals showed sensitivity to changes in contingencies. These results provide preliminary support that the insensitivity of Subtype 2 ASIB is a property specific to that response class in these contexts rather than a generalized response tendency of the individual.

Differential effects of social and novelty enrichment on individual differences in impulsivity and behavioral flexibility

Early life experience profoundly impacts behavior and cognitive functions in rats. The present study investigated how the presence of conspecifics and/or novel objects, could independently influence individual differences in impulsivity and behavioral flexibility. Twenty-four rats were reared in an isolated condition, an isolated condition with a novel object, a pair-housed social condition, or a pair-housed social condition with a novel object. The rats were then tested on an impulsive choice task, a behavioral flexibility task, and an impulsive action task. Novelty enrichment produced an overall increase in impulsive choice, while social enrichment decreased impulsive choice in the absence of novelty enrichment and also produced an overall increase in impulsive action. In the behavioral flexibility task, social enrichment increased regressive errors, whereas both social and novelty enrichment reduced never-reinforced errors. Individual differences analyses indicated a significant relationship between performance in the behavioral flexibility and impulsive action tasks, which may reflect a common psychological correlate of action inhibition. Moreover, there was a relationship between delay sensitivity in the impulsive choice task and performance on the DRL and behavioral flexibility tasks, suggesting a dual role for timing and inhibitory processes in driving the interrelationship between these tasks. Overall, these results indicate that social and novelty enrichment produce distinct effects on impulsivity and adaptability, suggesting the need to parse out the different elements of enrichment in future studies. Further research is warranted to better understand how individual differences in sensitivity to enrichment affect individuals' interactions with and the resulting consequences of the rearing environment.

Hot under the collar: The impact of heat on game play

High temperatures have been documented to affect behavior in a variety of ways depending on the nature of the task. We extended this prior research by examining the effects of dynamically changing temperature on various aspects of performance in a video game task. In the span of approximately an hour, temperature was gradually increased, stayed constant for a period of time, and gradually decreased to baseline. The gaming task was a variation on one used to assess impulsivity in participants thus allowing the possibility of assessing the effects of temperature on impulsive choice. Rather than heat increasing impulsivity and thus decreasing wait times, participants showed increases in wait times as temperature increased which either suggests that participants were becoming more self-controlled under heat or that the documented negative impact of heat on motor functioning was dominating their performance. Importantly, the participant's sensitivity to the changing task requirements was not affected by changes in temperature.

The effects of a time-based intervention on experienced middle-aged rats

Impulsive behavior is a common symptom in Attention Deficit Hyperactivity Disorder, schizophrenia, drug abuse, smoking, obesity and compulsive gambling. Stable levels of impulsive choice have been found in humans and rats and a recent study reported significant test-retest reliability of impulsive choice behavior after 1 and 5 months in rats. Time-based behavioral interventions have been successful in decreasing impulsive choices. These interventions led to improvements in the ability to time and respond more appropriately to adventitious choices. The current study examined the use of a time-based intervention in experienced, middle-aged rats. This intervention utilized a variable interval schedule previously found to be successful in improving timing and decreasing impulsive choice. This study found that the intervention led to a decrease in impulsive choices and there was a significant correlation between the improvement in self-control and post-intervention temporal precision in middle-aged rats. Although there were no overall group differences in bisection performance, individual differences were observed, suggesting an improvement in timing. This is an important contribution to the field because previous studies have utilized only young rats and because previous research indicates a decrease in general timing abilities with age.

Reward Contrast Effects on Impulsive Choice and Timing in Rats

Despite considerable interest in impulsive choice as a predictor of a variety of maladaptive behaviors, the mechanisms that drive choice behavior are still poorly understood. The present study sought to examine the influence of one understudied variable, reward magnitude contrast, on choice and timing behavior as changes in magnitude commonly occur within choice procedures. In addition, assessments of indirect effects on choice behavior through magnitude-timing interactions were assessed by measuring timing within the choice task. Rats were exposed to choice procedures composed of different pairs of magnitudes of rewards for either the smaller-sooner (SS) or larger-later (LL) option. In Phase 2, the magnitude of reward either increased or decreased by 1 pellet in different groups (LL increase = 1v1→1v2; SS decrease = 2v2 → 1v2; SS increase = 1v2 → 2v2), followed by a return to baseline in Phase 3. Choice behavior was affected by the initial magnitudes experienced in the task, demonstrating a strong anchor effect. The nature of the change in magnitude affected choice behavior as well. Timing behavior was also affected by the reward contrast manipulation albeit to a lesser degree and the timing and choice effects were correlated. The results suggest that models of choice behavior should incorporate reinforcement history, reward contrast elements, and magnitude-timing interactions, but that direct effects of reward contrast on choice should be given more weight than the indirect reward-timing interactions. A better understanding of the factors that contribute to choice behavior could supply key insights into this important individual differences variable.

Learning to wait for more likely or just more: greater tolerance to delays of reward with increasingly longer delays

Little research has focused on training greater tolerance to delays of rewards in the context of delayed gratification. In delayed gratification, waiting for a delayed outcome necessitates the ability to resist defection for a continuously available smaller, immediate outcome. The present research explored the use of a fading procedure for producing greater waiting in a video-game based, delayed gratification task. Participants were assigned to conditions in which either the reward magnitude, or the probability of receiving a reward, was a function of time waited and the delay to the maximum reward was gradually increased throughout this training. Waiting increased for all participants but less for those waiting for a greater reward magnitude than a greater reward probability. All participants showed a tendency to wait in a final testing phase, but training with probabilistic outcomes produced a significantly greater likelihood of waiting during testing. The behavioral requirements of delay discounting versus delay gratification are discussed, as well as the benefits of training greater self-control in a variety of contexts.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Training Tolerance to Delay Using the Escalating Interest Task

The present study examined the lasting effects of exposure to reinforcement that increased in magnitude as a function of time between responses in a first-person shooter video game preparation of the escalating interest task. When reinforcement density increased as a function of time, it encouraged participants to wait longer between responses (shots of a weapon). Participants exposed to such contingencies waited significantly longer to fire their weapons than participants who were exposed to linear growth, where long inter-response times were not differentially reinforced. Those with experience in conditions where reinforcement density increased as a function of time showed persistently longer wait times when the contingencies changed in the latter portion of the game where the disincentive to fire quickly was removed. The potential utility of such contingencies for training tolerance to delay of reinforcement and the broader implications of training self-control are discussed.

Gamification: The Intersection between Behavior Analysis and Game Design Technologies

Deterding et al. (Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, USA 15: 9-15, 2011) report a recent rise in popularity of video game inspired software designed to address issues in a variety of areas, including health, energy conservation, education, and business. These applications have been based on the concept of gamification, which involves a process by which nongame activities are designed to be more like a game. We provide examples of how gamification has been used to increase health-related behavior, energy consumption, academic performance, and other socially-significant behavior. We argue that behavior analytic research and practice stands to benefit from incorporating successful elements of game design. Lastly, we provide suggestions for behavior analysts regarding applied and basic research related to gamification.

Outcome probability versus magnitude: when waiting benefits one at the cost of the other

Using a continuous impulsivity and risk platform (CIRP) that was constructed using a video game engine, choice was assessed under conditions in which waiting produced a continuously increasing probability of an outcome with a continuously decreasing magnitude (Experiment 1) or a continuously increasing magnitude of an outcome with a continuously decreasing probability (Experiment 2). Performance in both experiments reflected a greater desire for a higher probability even though the corresponding wait times produced substantive decreases in overall performance. These tendencies are considered to principally reflect hyperbolic discounting of probability, power discounting of magnitude, and the mathematical consequences of different response rates. Behavior in the CIRP is compared and contrasted with that in the Balloon Analogue Risk Task (BART).