Impulsivity and sensitivity to amount and delay of reinforcement in an animal model of ADHD

Using rodent data to elucidate dopaminergic mechanisms of ADHD: Implications for human personality

An altered behavioral response to positive reinforcement has been proposed to be a core deficit in attention deficit hyperactivity disorder (ADHD). The spontaneously hypertensive rat (SHR), a congenic animal strain, displays a similarly altered response to reinforcement. The presence of this genetically determined phenotype in a rodent model allows experimental investigation of underlying neural mechanisms. Behaviorally, the SHR displays increased preference for immediate reinforcement, increased sensitivity to individual instances of reinforcement relative to integrated reinforcement history, and a steeper delay of reinforcement gradient compared to other rat strains. The SHR also shows less development of incentive to approach sensory stimuli, or cues, that predict reward after repeated cue-reward pairing. We consider the underlying neural mechanisms for these characteristics. It is well known that midbrain dopamine neurons are initially activated by unexpected reward and gradually transfer their responses to reward-predicting cues. This finding has inspired the dopamine transfer deficit (DTD) hypothesis, which predicts certain behavioral effects that would arise from a deficient transfer of dopamine responses from actual rewards to reward-predicting cues. We argue that the DTD predicts the altered responses to reinforcement seen in the SHR and individuals with ADHD. These altered responses to reinforcement in turn predict core symptoms of ADHD. We also suggest that variations in the degree of dopamine transfer may underlie variations in personality dimensions related to altered reinforcement sensitivity. In doing so, we highlight the value of rodent models to the study of human personality.

Timing and delay discounting in attention-deficit/hyperactivity disorder: A translational approach

Attention-deficit/hyperactivity disorder (ADHD) is a common neurodevelopmental disorder that often presents with abnormal time perception and increased impulsive choice behavior. The spontaneously hypertensive rat (SHR) is the most widely used preclinical model of the ADHD-Combined and ADHD-Hyperactive/Impulsive subtypes of the disorder. However, when testing the spontaneously hypertensive rat from Charles River (SHR/NCrl) on timing and impulsive choice tasks, the appropriate control strain is not clear, and it is possible that one of the possible control strains, the Wistar Kyoto from Charles River (WKY/NCrl), is an appropriate model for ADHD-Predominately Inattentive. Our goals were to test the SHR/NCrl, WKY/NCrl, and Wistar (WI; the progenitor strain for the SHR/NCrl and WKY/NCrl) strains on time perception and impulsive choice tasks to assess the validity of SHR/NCrl and WKY/NCrl as models of ADHD, and the validity of the WI strain as a control. We also sought to assess impulsive choice behavior in humans diagnosed with the three subtypes of ADHD and compare them with our findings from the preclinical models. We found SHR/NCrl rats timed faster and were more impulsive than WKY/NCrl and WI rats, and human participants diagnosed with ADHD were more impulsive compared to controls, but there were no differences between the three ADHD subtypes.

Previous experience with delays affects delay discounting in animal model of ADHD

BACKGROUND

ADHD is a disorder where a common symptom is impulsive behaviour, a broad term associated with making sub-optimal choices. One frequently used method to investigate impulsive behaviour is delay discounting, which involves choosing between a small, immediate reinforcer and a delayed, larger one. Choosing the small immediate reinforcer is by itself, however, not sufficient for terming the choice impulsive, as all organisms eventually switch to choosing the small, immediate reinforcer when the delay to the larger reinforcer becomes long. This switch can be termed impulsive only when it occurs more frequently, or at shorter LL delays, than typically observed in normal controls. A poorly understood aspect is how choice is influenced by previous experience with delays. Using an animal model of Attention-Deficit/Hyperactivity Disorder, the Spontaneously Hypertensive Rat, we manipulated the order of exposure to delays in a delay discounting task. Following a preference test, the Ascending group experienced gradually increasing delays between choice and reinforcer while the Descending group were exposed to these delays in reverse order.

RESULTS

The results showed that the Descending group chose the small, immediate reinforcer over the larger delayed to a much larger extent than the Ascending group, and continued to do so even when the delay component was ultimately removed. Strain effects were found in the Ascending group, with SHRs switching to the small, immediate reinforcer earlier than controls as the delay to the larger reinforcer increased.

CONCLUSION

The data suggests that delay discounting is affected by history of exposure to delayed consequences. When reinforcement contingencies are incrementally changed from having no response-reinforcer delay to a long delay, discounting of delayed consequences is gradual. However, a sudden change from no delay to a long delay, without intermediate training, results in a rapid switch to the small, immediate reinforcer option, and this behaviour is somewhat resilient to the shortening and eventual removal of the large reinforcer delay. The implication is that attempting to reduce already existing impulsive behaviour in children with ADHD will require gradual habituation and not sudden changes in reinforcement contingencies.

Prepubertal methylphenidate leads to sex-dependent differences in probabilistic discounting

Prescription psychostimulants, such as methylphenidate (MPH), have served as a first line treatment for ADHD and associated developmental disorders since 1961. Psychostimulants has been shown to improve attention, response inhibition, and reduce hyperactivity in patients with ADHD, as well as in non-clinical human populations and animals. While there is a considerable amount of preclinical research investigating the effects of stimulant medications on reward sensitivity and basic learning in male rats, less is understood about their effects in females. Further, there are competing theories on the long-term cognitive impact of MPH, specifically in children who do not have ADHD. To this end, Long-Evans female and male rats were exposed to methylphenidate (0, 2.5, 5 mg/kg, BID, IP) for 20 days during early development (PD10-29). After discontinuation of MPH into adulthood, rats (beginning PD 60) were trained and tested for risk-preference using a 2-choice probabilistic discounting task. For this task, rats were given an option between a 'large-risky' choice (3 sugar pellets delivered on a probabilistic VR schedule) and 'small-certain' choice (1 sugar pellet delivered on a FR schedule). Rats were subsequently tested on an open field conflict test. The results demonstrate that prepubertal exposure to MPH can have lasting effects on decision-making. Specifically, female rats treated with 2.5 mg/kg MPH displayed a decrease in preference for the risky option, whereas male rats treated with the same dose showed an overall increase in preference compared to sex-matched controls. Irrespective of sex, rats treated with 2.5 mg/kg MPH also demonstrated a decrease in anxiety/inhibitory behavior on the modified open field test compared to controls. These results were not due to differences in locomotor behavior. Overall, the study contributes to the growing body of evidence to suggest that MPH exposure early in development can have a sex-dependent impact on decision-making in adulthood.

Spontaneously hypertensive rats show higher impulsive action, but equal impulsive choice with both positive and aversive consequences

Both positive and aversive delayed consequences play an important role in decision making. However, most of research has studied the temporal discounting of the positive consequences, while the study of the aversives is scarce in general and null in some areas. This is the case of research on impulsivity in spontaneously hypertensive rats (SHR), an animal model of ADHD. To evaluate SHRs' sensitivity to aversive delayed consequences, we employed a choice procedure in which subjects chose between a smaller-amount alternative and a larger-amount alternative plus a shock; when preference for the smaller-amount alternative stabilized, five different delays to the shock were presented with the objective of analyzing the recovery of preference for the larger-amount alternative, which is related to the sensitivity to the delayed aversive consequence. To analyze the sensitivity to delayed positive consequences we employed a procedure that evaluated the preference between a smaller-amount alternative and a larger-amount alternative as a function of the delay to the later. Finally, to evaluate impulsive action we employed a DRL 10s schedule. In all tasks, Wistar rats were evaluated as control strain. The results indicated that choice impulsivity was equivalent between strains, both for positive and for aversive consequences. In contrast, we found a higher level of impulsive action in SHR.

Animal Models of ADHD?

To describe animals that express abnormal behaviors as a model of Attention-Deficit Hyperactivity Disorder (ADHD) implies that the abnormalities are analogous to those expressed by ADHD patients. The diagnostic features of ADHD comprise inattentiveness, impulsivity, and hyperactivity and so these behaviors are fundamental for validation of any animal model of this disorder. Several experimental interventions such as neurotoxic lesion of neonatal rats with 6-hydroxydopamine (6-OHDA), genetic alterations, or selective inbreeding of rodents have produced animals that express each of these impairments to some extent. This article appraises the validity of claims that these procedures have produced a model of ADHD, which is essential if they are to be used to investigate the underlying cause(s) of ADHD and its abnormal neurobiology.

Spontaneously Hypertensive Rat substrains show differences in model traits for addiction risk and cocaine self-administration: Implications for a novel rat reduced complexity cross

Forward genetic mapping of F2 crosses between closely related substrains of inbred rodents - referred to as a reduced complexity cross (RCC) - is a relatively new strategy for accelerating the pace of gene discovery for complex traits, such as drug addiction. RCCs to date were generated in mice, but rats are thought to be optimal for addiction genetic studies. Based on past literature, one inbred Spontaneously Hypertensive Rat substrain, SHR/NCrl, is predicted to exhibit a distinct behavioral profile as it relates to cocaine self-administration traits relative to another substrain, SHR/NHsd. Direct substrain comparisons are a necessary first step before implementing an RCC. We evaluated model traits for cocaine addiction risk and cocaine self-administration behaviors using a longitudinal within-subjects design. Impulsive-like and compulsive-like traits were greater in SHR/NCrl than SHR/NHsd, as were reactivity to sucrose reward, sensitivity to acute psychostimulant effects of cocaine, and cocaine use studied under fixed-ratio and tandem schedules of cocaine self-administration. Compulsive-like behavior correlated with the acute psychostimulant effects of cocaine, which in turn correlated with cocaine taking under the tandem schedule. Compulsive-like behavior also was the best predictor of cocaine seeking responses. Heritability estimates indicated that 22 %-40 % of the variances for the above phenotypes can be explained by additive genetic factors, providing sufficient genetic variance to conduct genetic mapping in F2 crosses of SHR/NCrl and SHR/NHsd. These results provide compelling support for using an RCC approach in SHR substrains to uncover candidate genes and variants that are of relevance to cocaine use disorders.

The irrelevancy of the inter-trial interval in delay-discounting experiments on an animal model of ADHD

Delay discounting involves choosing between a small, immediate reward, and a larger but delayed one. As the delay between choice and large reward gets longer, people with ADHD tend to become impulsive faster than controls, indicated by a switch in preference from the large to the smaller reward. Choosing the smaller reward when the larger is considered reward maximizing is labeled impulsive behaviour. It is well documented that increased delays between choice and reward affects choice preference in both humans and other animals. Other variables such as the inter-trial interval or trial length are observed to have an effect on human discounting, but their effect on discounting in other animals is largely assumed rather than tested. In the current experiment, we tested this assumption. One group of rats was exposed to increasing delays between choosing the large reward and receiving it, while another group experienced longer inter-trial intervals that were equal in length to the delays in the other group. This ensured that trial length was controlled for in delay discounting, but that the delay function and inter-trial intervals could be manipulated and measured separately. Results showed that while the delay between choice and reward caused impulsive behaviour in rats, the length of the inter-trial interval (and by extension trial length) had no impact on choice behaviour. A follow-up experiment found this to be the case even if the length of the inter-trial interval was signaled with audio cues. These results suggest that rats, and possibly animals in general, are insensitive to time between trials, and therefore cannot easily represent human counterparts on the task.

Analyzing the magnitude effect in spontaneously hypertensive (SHR) and wistar Kyoto (WKY) rats

We analyzed the magnitude effect in Spontaneously Hypertensive (SHR) and Wistar Kyoto (WKY) rats choosing between a smaller-sooner (SSF) and a larger-later food (LLF) in the initial link of a concurrent-chains procedure. The SSF was delivered immediately in one terminal link and the LLF delayed 0.01, 5, 10, 20, 40, or 80 s in the other terminal link. An ABABA design varied food amount, 1 vs. 4 and 3 vs. 8 food-pellets in conditions A and B, respectively. The SHRs made more impulsive choices than the WKYs. The hyperbolic-decay model and the Generalized Matching Law fitted the data well. Discounting rate (k) and the area under the discounting curve (AUC) for the choices made by the SHRs in conditions A, were like those in conditions B. For the choices that the individual WKYs made, k was slightly higher and AUC smaller in conditions B than in conditions A. For both strains sensitivity to the immediacy of the LLF (s) was slightly higher in conditions A than in conditions B. Thus, we found no conclusive, compelling evidence either supporting or discarding the magnitude effect in the SHRs and scarce evidence supporting an effect opposite to the magnitude effect in the WKYs.

Effect of Schedule-Induced Behavior on Responses of Spontaneously Hypertensive and Wistar-Kyoto Rats in a Delay-Discounting Task: A Preliminary Report

Delay discounting is the loss of the subjective value of an outcome as the time to its delivery increases. It has been suggested that organisms can become more tolerant of this delay when engaging in schedule-induced behaviors. Schedule-induced behaviors are those that develop at a high rate during intermittent reinforcement schedules without the need of arranged contingency to the reinforcer, and they have been considered as a model of compulsivity. There is evidence that relates compulsivity to greater delay discounting. The rate of delay discounting represents how impulsive the subject is, as the rate of discounting increases the higher the impulsivity. Thus, the main purpose of this study was to undertake a preliminary evaluation of whether developing schedule-induced behaviors affects performance in a delay-discounting task, by comparing spontaneously hypertensive rats (SHRs) and Wistar-Kyoto (WKY) rats. The rats were exposed to a task that consisted of presenting the subjects with two levers: one produced a small, immediate food reinforcer while the other one produced a larger, delayed reinforcer. During Condition A, the levers were presented, and a water bottle and a running wheel were available in the conditioning chambers; during Condition B, only the levers were presented. SHR and WKY rats developed schedule-induced behaviors during Condition A and showed no difference in discounting rates, contradicting previous reports. Lick allocation during response-reinforcer delays and the inter-trial interval (ITI) showed, respectively, pre- and post-food distributions. Discounting rates during Condition B (when rats could not engage in schedule-induced behaviors) did not reach statistical significance difference among strains of animals, although it was observed a tendency for WKY to behave more self-controlled. Likewise it was not found any effect of schedule-induced behavior on discounting rates, however, a tendency for WKY rats to behave more impulsive during access to drink and run seems to tentatively support the idea of schedule-induced behavior as a model of compulsivity in those rats, being impulsivity simply defined as an excess in behavior.

Examining the neurochemical underpinnings of animal models of risky choice: Methodological and analytic considerations

Because risky choice is associated with several psychiatric conditions, recent research has focused on examining the underlying neurochemical processes that control risk-based decision-making. Not surprisingly, several tasks have been developed to study the neural mechanisms involved in risky choice. The current review will briefly discuss the major tasks used to measure risky choice and will summarize the contribution of several major neurotransmitter systems to this behavior. To date, the most common measures of risky choice are the probability discounting task, the risky decision task, and the rat gambling task. Across these three tasks, the contribution of the dopaminergic system has been most studied, although the effects of serotonergic, adrenergic, cholinergic, and glutamatergic ligands will be discussed. Drug effects across these tasks have been inconsistent, which makes determining the precise role of neurotransmitter systems in risky choice somewhat difficult. Furthermore, procedural differences can modulate drug effects in these procedures, and the way data are analyzed can alter the interpretations one makes concerning pharmacological manipulations. By taking these methodological/analytic considerations into account, we may better elucidate the neurochemistry of risky decision-making. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Toward a Unifying Account of Impulsivity and the Development of Self-Control

Impulsivity has traditionally been thought to involve various behavioral traits that can be measured using different laboratory protocols. Whereas some authors regard different measures of impulsivity as reflecting fundamentally distinct and unrelated behavioral tendencies (fragmentation approach), others regard those different indexes as analogue forms of the same behavioral tendency, only superficially different (unification approach). Unifying accounts range from mere intuitions to more sophisticated theoretical systems. Some of the more complete attempts at unifying are intriguing but have validity weaknesses. We propose a new unifying attempt based on theoretical points posed by other authors and supplemented by theory and research on associative learning. We then apply these assumptions to characterize the paradigms used to study impulsivity in laboratory settings and evaluate their scope as an attempt at unification. We argue that our approach possesses a good balance of parsimony and empirical and theoretical grounding, as well as a more encompassing scope, and is more suitable for experimental testing than previous theoretical frameworks. In addition, the proposed approach is capable of generating a new definition of impulsivity and outlines a hypothesis of how self-control can be developed. Finally, we examine the fragmentation approach from a different perspective, emphasizing the importance of finding similarities among seemingly different phenomena.

Instrumental conditioning for food reinforcement in the spontaneously hypertensive rat model of attention deficit hyperactivity disorder

Background

The spontaneously hypertensive rat is thought to show good validity as a model of attention deficit hyperactivity disorder, in part because of impaired delayed reinforcement behaviour, corresponding to the dynamic developmental theory of the disorder. However, some previous studies may have been confounded use of fluid reward. Therefore, the objective of this study was to assess the spontaneously hypertensive rat and two comparison strains (Wistar and Wistar Kyoto) using a non-delayed food reinforcement paradigm in an attempt to advance knowledge of basic learnt behaviour in this strain, without potentially confounding reward sensitivity, which could impact on motivation to learn. Rats were trained on a fixed ratio 1 two choice discrimination schedule, extinction, reacquisition and reversal. We also tested non-reinforced spontaneous alternation to facilitate data interpretation.

Results

The spontaneously hypertensive rat displayed slower shaping and reduced on task activity during task acquisition, contrasting with previous results which indicate either enhanced responding and an impairment only when a delay is used; we suggest several reasons for this. In line with previous work, the same strain exhibited poor extinguishing of behaviour but were not impaired to the same extent on reversal of the discrimination. Finally, non-reinforced alternations on a Y-maze were also reduced in the spontaneously hypertensive rat.

Conclusions

In sum, the spontaneously hypertensive rat appear to show poor response inhibition in reinforced and non-reinforced contexts. However, impaired response inhibition was reduced during reversal when an opposite response produced food reward alongside presentation of the conditioned stimulus. We discuss the possibility of enhanced attribution of incentive salience to cues in this strain and highlight several points of caution for researchers conducting behavioural assessments using the spontaneously hypertensive rat and their associated comparison strains.

Electronic supplementary material

The online version of this article (10.1186/s13104-017-2857-5) contains supplementary material, which is available to authorized users.

Spontaneously hypertensive (SHR) rats choose more impulsively than Wistar-Kyoto (WKY) rats on a delay discounting task

Indications of Attention Deficit Hyperactivity Disorder (ADHD) in the spontaneously hypertensive rat (SHR) are not consistent across different tests of impulsivity, questioning the SHR's validity as a rodent model of ADHD. This study used a concurrent-chains procedure to examine possible differences in impulsive choice between SHRs and control-normotensive Wistar Kyoto (WKY) rats. The aim was to extend the generality of findings showing regularities between the hyperbolic-decay model and the generalized matching law fitting delay discounting data from nonhuman animals. The objectives were to: (1) examine differences in impulsive choice between SHRs and WKYs; (2) add evidence suggesting that the SHR is a suitable model of ADHD; (3) demonstrate that concurrent-chains procedures requiring locomotion detect differences in impulsive choice between SHRs and WKYs; (4) support the idea that impulsivity in nonhuman animals increases with training. The initial link used two non-independent random interval schedules arranging entries to the terminal links, where one fixed-time (FT) schedule delayed 1-food pellet and the other FT 4-food pellets. The FT delaying the former was kept constant at 0.1s and that delaying the latter changed after every 10 food deliveries, defining six delay components (0.1, 5, 10, 20, 40, and 80s) presented in random order each session. Results showed that the SHRs choose more impulsively than the WKYs, adding to the body of evidence suggesting that the SHR may be a suitable model of ADHD. Both models of choice fitted the impulsive choices of the SHRs and WKYs well; positive correlations between estimates of parameters k and s suggested compatibility between models of choice showing that impulsivity increases with training.

Methylphenidate and Atomoxetine-Responsive Prefrontal Cortical Genetic Overlaps in "Impulsive" SHR/NCrl and Wistar Rats

Impulsivity, the predisposition to act prematurely without foresight, is associated with a number of neuropsychiatric disorders, including attention-deficit/hyperactivity disorder (ADHD). Identifying genetic underpinnings of impulsive behavior may help decipher the complex etiology and neurobiological factors of disorders marked by impulsivity. To identify potential genetic factors of impulsivity, we examined common differentially expressed genes (DEGs) in the prefrontal cortex (PFC) of adolescent SHR/NCrl and Wistar rats, which showed marked decrease in preference for the large but delayed reward, compared with WKY/NCrl rats, in the delay discounting task. Of these DEGs, we examined drug-responsive transcripts whose mRNA levels were altered following treatment (in SHR/NCrl and Wistar rats) with drugs that alleviate impulsivity, namely, the ADHD medications methylphenidate and atomoxetine. Prefrontal cortical genetic overlaps between SHR/NCrl and Wistar rats in comparison with WKY/NCrl included genes associated with transcription (e.g., Btg2, Fos, Nr4a2), synaptic plasticity (e.g., Arc, Homer2), and neuron apoptosis (Grik2, Nmnat1). Treatment with methylphenidate and/or atomoxetine increased choice of the large, delayed reward in SHR/NCrl and Wistar rats and changed, in varying degrees, mRNA levels of Nr4a2, Btg2, and Homer2, genes with previously described roles in neuropsychiatric disorders characterized by impulsivity. While further studies are required, we dissected potential genetic factors that may influence impulsivity by identifying genetic overlaps in the PFC of "impulsive" SHR/NCrl and Wistar rats. Notably, these are also drug-responsive transcripts which may be studied further as biomarkers to predict response to ADHD drugs, and as potential targets for the development of treatments to improve impulsivity.

Optimal behavior by rats in a choice task is associated to a persistent conditioned inhibition effect

When given a choice between an alternative with a low probability of reinforcement and discriminative stimuli, and another with a higher probability of reinforcement and non-discriminative stimuli, pigeons show a clear preference for the former but rats clearly prefer the later. It has been reported that pigeon's suboptimal choice is associated to a diminishing effect of the stimulus correlated with non-reinforcement. In the present paper, we explored the possibility that rats' optimal choice is more strongly influenced than pigeons' by the stimulus associated to non-reinforcement and that the effects of it do not dissipate during training. We trained rats to choose between an alternative with 0.50 probability of reinforcement and discriminative stimuli, and an alternative with 0.75 probability of reinforcement and non-discriminative stimuli. We replicated the strong preference for the optimal alternative. Then, after several sessions of training, we presented summation trials in which both the stimulus associated to reinforcement and the stimulus associated to non-reinforcement were simultaneously presented. The results showed that the stimulus associated to non-reinforcement exerted a strong effect on choice, and, more importantly, that it did not seem to dissipate across training. These results suggest that the strong difference found between pigeons and rats in the suboptimal choice procedure is potentially related to differences in the impact of conditioned inhibitors.