Theory and method in the quantitative analysis of "impulsive choice" behaviour: implications for psychopharmacology

Theoretical note: Quantity and concentration as co-determinants of the reinforcing value of sucrose: A re-analysis of some previously published data

According to the Multiplicative Hyperbolic Model of reinforcer value (MHM), the overall value of a reinforcer may be defined by the multiplicative combination of a set of hyperbolic functions, each of which defines the impact of a particular feature of the reinforcer (e.g., quantity, immediacy of delivery). A previous experiment found that the relationship between the indifference volumes (qA(50)) of reinforcer A (a 0.4-M sucrose solution) and the fixed volume (qB) of reinforcer B (a 0.2-M sucrose solution: 32 - 256 μl) was consonant with this model. This paper describes a re-analysis of those data in an attempt to identify the nature of the effect of concentration on the two parameters of the size/value hyperbola (asymptote, ε, and sensitivity, Q). Comparison of two versions of the model in which (i) both parameters were free to vary as a function of qB and (ii) only ε was free to vary, showed that the latter model provided a satisfactory account of the data and that the inclusion of Q as an additional free parameter was not justified. Implications for the development of MHM are discussed.

Mechanisms of impulsive choice: Experiments to explore and models to map the empirical terrain

Impulsive choice is preference for a smaller-sooner (SS) outcome over a larger-later (LL) outcome when LL choices result in greater reinforcement maximization. Delay discounting is a model of impulsive choice that describes the decaying value of a reinforcer over time, with impulsive choice evident when the empirical choice-delay function is steep. Steep discounting is correlated with multiple diseases and disorders. Thus, understanding the processes underlying impulsive choice is a popular topic for investigation. Experimental research has explored the conditions that moderate impulsive choice, and quantitative models of impulsive choice have been developed that elegantly represent the underlying processes. This review spotlights experimental research in impulsive choice covering human and nonhuman animals across the domains of learning, motivation, and cognition. Contemporary models of delay discounting designed to explain the underlying mechanisms of impulsive choice are discussed. These models focus on potential candidate mechanisms, which include perception, delay and/or reinforcer sensitivity, reinforcement maximization, motivation, and cognitive systems. Although the models collectively explain multiple mechanistic phenomena, there are several cognitive processes, such as attention and working memory, that are overlooked. Future research and model development should focus on bridging the gap between quantitative models and empirical phenomena.

Further observations on the reinforcing value of sucrose solutions: Interaction between quantity and concentration

According to the Multiplicative Hyperbolic Model of reinforcer value (MHM), the value of a reinforcer is an increasing hyperbolic function of its size. In addition it is generally accepted that in the case of soluble reinforcers such as sucrose, value is an increasing function of molar concentration. The present experiment examined the interaction between size and concentration on the effectiveness of sucrose reinforcers in an operant choice paradigm. Rats were trained under an adjusting-magnitude schedule in which a response on lever B delivered a fixed volume of a sucrose solution, while a response on lever A delivered a 0.8-M sucrose solution, the volume of which, qA, was adjusted according to the rats' choices. When B was preferred in a given block of trials, qA was increased in the following block; when A was preferred, qA was reduced in the following block. The concentration and volume of B (cB, qB) were varied across six phases of the experiment and the corresponding indifference magnitudes of A (qA(50)) were measured. Log qA(50) increased and qB/qA(50) declined as approximately linear functions of log cB, consistent with the supposition that reinforcer value was determined by multiplicative combination of hyperbolic expressions representing concentration and volume.

Hyperbolic modeling and assessment of hypothetical health behaviors during a viral outbreak using crowdsourced samples

The COVID-19 pandemic provided an opportunity to investigate factors related to public response to public health measures, which could help better prepare implementation of similar measures for inevitable future pandemics. To understand individual and environmental factors that influence likelihood in engaging in personal and public health measures, three crowdsourced convenience samples from Amazon Mechanical Turk (MTurk) completed likelihood-discounting tasks of engaging in health behaviors given a variety of hypothetical viral outbreak scenarios. Experiment 1 assessed likelihood of mask wearing for a novel virus. Experiment 2 assessed vaccination likelihood based on efficacy and cost. Experiment 3 assessed likelihood of seeking health care based on number of symptoms and cost of treatment. Volume-based measures and three-dimensional modeling were used to analyze hypothetical decision making. Hypothetical public and personal health participation increased as viral fatality increased and generally followed a hyperbolic function. Public health participation was moderated by political orientation and trust in science, whereas treatment-seeking was only moderated by income. Analytic methods used in this cross-sectional study predicted population-level outcomes that occurred later in the pandemic and can be extended to various health behaviors.

Histone deacetylase inhibitors mitigate antipsychotic risperidone-induced motor side effects in aged mice and in a mouse model of Alzheimer's disease

Antipsychotic drugs are still widely prescribed to control various severe neuropsychiatric symptoms in the elderly and dementia patients although they are off-label use in the United States. However, clinical practice shows greater side effects and lower efficacy of antipsychotics for this vulnerable population and the mechanisms surrounding this aged-related sensitivity are not well understood. Our previous studies have shown that aging-induced epigenetic alterations may be involved in the increasing severity of typical antipsychotic haloperidol induced side effects in aged mice. Still, it is unknown if similar epigenetic mechanisms extend to atypical antipsychotics, which are most often prescribed to dementia patients combined with severe neuropsychiatric symptoms. In this study, we report that atypical antipsychotic risperidone also causes increased motor side effect behaviors in aged mice and 5xFAD mice. Histone deacetylase (HDAC) inhibitor Valproic Acid and Entinostat can mitigate the risperidone induced motor side effects. We further showed besides D2R, reduced expression of 5-HT2A, one of the primary atypical antipsychotic targets in the striatum of aged mice that are also mitigated by HDAC inhibitors. Finally, we demonstrate that specific histone acetylation mark H3K27 is hypoacetylated at the 5htr2a and Drd2 promoters in aged mice and can be reversed with HDAC inhibitors. Our work here establishes evidence for a mechanism where aging reduces expression of 5-HT2A and D2R, the key atypical antipsychotic drug targets through epigenetic alteration. HDAC inhibitors can restore 5-HT2A and D2R expression in aged mice and decrease the motor side effects in aged and 5xFAD mice.

Accumbal adenosine A2A receptor inactivation biases for large and costly rewards in the effort- but not delay-based decision making

The cost-benefit decision-making (CBDM) is critical to normal human activity and a diminished willingness to expend effort to obtain rewards is a prevalent/noted characteristic of neuropsychiatric disorders such as schizophrenia, Parkinson's disease. Numerous studies have identified nucleus accumbens (NAc) as an important locus for CBDM control but their neuromodulatory and behavioral mechanisms remain largely under-explored. Adenosine A_2A receptors (A_2ARs), which are highly concentrated in the striatopallidal neurons, can integrate glutamate and dopamine signals for controlling effort-related choice behaviors. While the involvement of A_2ARs in effort-based decision making is well documented, the role of other decision variables (reward discrimination) in effort-based decision making and the role of A_2AR in delay-based decision making are less clear. In this study, we have developed a well-controlled CBDM behavioral paradigm to manipulate effort/cost and reward independently or in combination, allowing a dissection of four behavioral elements: effort-based CBDM (E-CBDM), delay-based CBDM (D-CBDM), reward discrimination (RD), effort discrimination (ED), and determined the effect of genetic knockdown (KD) of NAc A_2AR on the four behavioral elements. We found that A_2AR KD in NAc increased the choice for larger, more costly reward in the E-CBDM, but not D-CBDM. Furthermore, this high-effort/high-reward bias was attributable to the increased willingness to engage in effort but not the effect of discrimination of reward magnitude. Our findings substantiate an important role of the NAc A_2AR in control of E-CBDM and support that pharmacologically targeting NAc A_2ARs would be a useful strategy for treating the aberrant effort-based decision making in neuropsychiatric disorders.

Hyperbolic discounting and exponentiated demand: Modeling demand for cigarettes in three dimensions

Behavioral economics has been a fruitful area of research in substance use. Mathematical descriptions of how individuals temporally discount the value of a commodity have been correlated with substance use and mathematical descriptions of drug consumption decreasing as a function of price (i.e., demand) predict maladaptive substance use. While there is a logical assumption that temporal factors affect demand for a drug, little has been done to merge these models. Thus, the purpose of this study was to combine models of discounting and demand, extending Howard Rachlin's work and contributions to novel areas of study. Data from 85 participants from Amazon Mechanical Turk (MTurk) who completed a hypothetical cigarette purchase task that included price of and delay to cigarettes were analyzed. Multilevel modeling was used to determine descriptive accuracy of combined additive and multiplicative models of discounting and demand. Of the discounting models used in conjunction with the exponentiated demand equation, the Rachlin hyperboloid best described the delay dimension of consumption. The multiplicative version of the Rachlin equation applied to both delay and price outperformed other models tested. Therefore, existing models of discounting and demand can be extended to modeling consumption data from complex multidimensional experimental arrangements.

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

Adulteration of sucrose with citric acid: Effect on reinforcing value, examined using an adjusting-magnitude schedule of reinforcement

According to the Multiplicative Hyperbolic Model (MHM), the value of a reinforcer is an increasing hyperbolic function of its size (q). Recently reported results indicated that dilution of a sucrose solution reduced its reinforcing value by increasing the 'size-sensitivity' parameter of this function and reducing its maximum. The present experiment examined whether adulterating a sucrose solution with citric acid would have a similar effect on the hyperbolic function. Rats were trained under an adjusting-magnitude schedule in which a response on lever B delivered a fixed volume of a sucrose/citric-acid mixture, while a response on lever A delivered a sucrose solution, the volume of which, q_A, was adjusted according to the rats' choices. When B was preferred in a given block of trials, q_A was increased in the following block; when A was preferred, q_A was reduced in the following block. q_B was varied across four phases of the experiment and the corresponding indifference magnitudes of A were measured. The results failed to support the hypothesis that adulteration with citric acid resulted in devaluation of sucrose. An alternative hypothesis derived from MHM, that the value of the mixture reflected the summation of positive (sucrose) and negative (citric acid) values, is discussed.

The orbitofrontal cortex in temporal cognition

One of the most important factors in decision-making is estimating the value of available options. Subregions of the prefrontal cortex, including the orbitofrontal cortex (OFC), have been deemed essential for this process. Value computations require a complex integration across numerous dimensions, including, reward magnitude, effort, internal state, and time. The importance of the temporal dimension is well illustrated by temporal discounting tasks, in which subjects select between smaller-sooner versus larger-later rewards. The specific role of OFC in telling time and integrating temporal information into decision-making remains unclear. Based on the current literature, in this review we reevaluate current theories of OFC function, accounting for the influence of time. Incorporating temporal information into value estimation and decision-making requires distinct, yet interrelated, forms of temporal information including the ability to tell time, represent time, create temporal expectations, and the ability to use this information for optimal decision-making in a wide range of tasks, including temporal discounting and wagering. We use the term "temporal cognition" to refer to the integrated use of these different aspects of temporal information. We suggest that the OFC may be a critical site for the integration of reward magnitude and delay, and thus important for temporal cognition. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Neural Dynamics of the Combined Discounting of Delay and Probability During the Evaluation of a Delayed Risky Reward

Delay discounting and probability discounting are two important processes, but in daily life there are many more situations that involve delayed risky outcomes. Although neuroscience research has extensively investigated delay and probability discounting in isolation, little research has explored the neural correlates of the combined discounting of delay and probability. Using the event-related brain potentials (ERPs) technique, we designed a novel paradigm to investigate neural processes related to the combined discounting of delay and probability during the evaluation of a delayed risky reward. ERP results suggested distinct temporal dynamics for delay and probability processing during combined discounting. Both the early frontal P200 and the N2 reflected only probability, not delay, while the parietal P300 was sensitive to both probability and delay. Furthermore, the late positive potential (LPP) was sensitive to probability, but insensitive to delay. These results suggest that probability has a prolonged modulatory effect on reward evaluation in the information processing stream. These findings contribute to an understanding of the neural processes underlying the combined discounting of delay and probability. The limitation of this study is to only consider four delay and probability combinations. Future studies can explore the combined discounting of more probability and delay combinations to further test the robustness of the conclusion.

Choice between different concentrations of sucrose in an adjusting-magnitude schedule: Evidence for reinforcer-specific value maxima

According to the Multiplicative Hyperbolic Model (MHM), an extension of Mazur's (1987) model of delay discounting, the value of a reinforcer is an increasing hyperbolic function of its size (q). This experiment tested a prediction based on a revised version of MHM (MHM-R), according to which the maximum value may differ between different reinforcers, a possibility not envisaged in MHM. Rats were trained under a discrete-trials adjusting-magnitude schedule in which a response on lever B delivered a dilute sucrose reinforcer (0.2 M) of fixed volume, q_B, while a response on lever A delivered a more concentrated sucrose solution (0.4 M) the volume of which, q_A, was adjusted according to the rats' choices. When B was preferred in a given block of trials, q_A was increased in the following block; when A was preferred, q_A was reduced in the following block. q_B was varied across four phases of the experiment and the corresponding indifference magnitudes of A (q_A(50)) were measured. In keeping with expectations based on both MHM and MHM-R, q_A(50) increased approximately linearly with q_B, and 1/q_A(50) with 1/q_B. The ratio q_B/q_A(50) increased linearly with q_B with a positive intercept, a finding that is consistent with MHM-R but not with MHM.

Comparison of multiplicative and additive hyperbolic and hyperboloid discounting models in delayed lotteries involving gains and losses

Many day-to-day decisions may involve risky outcomes that occur at some delay after a decision has been made. We refer to such scenarios as delayed lotteries. Despite human choice often involves delayed lotteries, past research has primarily focused on decisions with delayed or risky outcomes. Comparatively, less research has explored how delay and probability interact to influence decisions. Within research on delayed lotteries, rigorous comparisons of models that describe choice from the discounting framework have not been conducted. We performed two experiments to determine how gain or loss outcomes are devalued when delayed and risky. Experiment 1 used delay and probability ranges similar to past research on delayed lotteries. Experiment 2 used individually calibrated delay and probability ranges. Ten discounting models were fit to the data using a genetic algorithm. Candidate models were derived from past research on discounting delayed or probabilistic outcomes. We found that participants' behavior was best described primarily by a three-parameter multiplicative model. Measures based on information criteria pointed to a solution in which only delay and probability were psychophysically scaled. Absolute measures based on residuals pointed to a solution in which amount, delay, and probability are simultaneously scaled. Our research suggests that separate scaling parameters for different discounting factors may not be necessary with delayed lotteries.

Impulsivity and Suicidal Behavior

Suicide is the leading cause of injury mortality in the United States and the second-leading cause of death in people aged 10-34 years. While many long-term risk factors are known, the short-term prediction of suicidal behavior remains elusive. Many characteristics of suicidal behavior cut across diagnoses, but suicide is increased in recurrent psychiatric disorders, addictive disorders, and trauma-related disorders. Suicide results from the interaction of short-term and long-term behavioral regulation. The shorter the time-course of the mechanism, the closer it is to actual suicidal behavior, and the harder it is to prevent. We will discuss the manner in which impulsivity, a major determinant of short-term suicide risk, interacts with longer-term risk factors, especially sensitization to addictive or traumatic stimuli. Impulsivity predisposes to sensitization; in turn, impulsivity is a prominent component of sensitized behavior. Impulsivity can be described as a general pattern of behavior ("trait" impulsivity), as responses that are not conformed to their context (action-impulsivity), or as inability to delay reward or to take future consequences into account (choice-impulsivity). Each of these contributes to suicidal behavior. The neural mechanisms of impulsivity and sensitization are analogous, and sensitization can produce rapidly fluctuating patterns of impulsive behavior, arousal, and anhedonia. In order to recognize and prevent suicidal behavior, it is necessary to identify factors associated with susceptibility to bouts of impulsive behavior in people at elevated long-term risk.

Neural correlates of reward magnitude and delay during a probabilistic delay discounting task in alcohol use disorder

RATIONALE

Alcohol-use disorder (AUD) is associated with the propensity to choose smaller sooner options on the delay discounting task. It is unclear, however, how inherent risk underlies delay discounting behavior. As impulsive choice is a hallmark feature in AUD, it is important to understand the neural response to reward and delay while accounting for risk in impulsive decision-making.

OBJECTIVE

This study examined activation associated with delay and reward magnitude, while controlling for risk in a probabilistic delay discounting task in AUD and examined if differences in activation were associated with treatment outcomes.

METHODS

Thirty-nine recently abstinent alcohol-dependent volunteers and 46 controls completed a probabilistic delay discounting task paired with functional magnetic resonance imaging. Alcohol use was collected using a self-report journal for 3 months following baseline scan.

RESULTS

During delay stimulus presentations, Controls exhibited greater activation compared to the Alcohol group notably in the anterior insula, middle/dorsal anterior cingulate cortex (ACC), dorsolateral prefrontal cortex (PFC), and inferior parietal lobule. For magnitude, the Alcohol group exhibited greater activation than Controls primarily in medial PFC, rostral ACC, left posterior parietal cortex, and right precuneus. Within the Alcohol group, alcohol craving severity negatively correlated with right lateral PFC activation during reward magnitude in individuals who completed the 3-month study without relapse, while non-completers showed the opposite relationship.

CONCLUSIONS

The results of this study extend previous findings that alcohol use disorder is associated with differences in activation during an immediate or delayed choice by delineating activation associated with the parameters of impulsive choice.

In search of a definition of reinforcer value: Some successes and failures of the multiplicative hyperbolic model

The concept of 'value' has enjoyed a central position in many theoretical accounts of choice behaviour. Several definitions of 'value' are contrasted in this paper, and one particular approach is defended, whereby value is defined as a dimensionless intervening variable. This definition is a cornerstone of the multiplicative hyperbolic model of choice (MHM), which was proposed twenty years ago as a modification of Mazur's (1987) hyperbolic model of delay discounting. This paper reviews some of the merits and shortcomings of MHM, and suggests some ways in which MHM might be extended and improved. A formal link between 'value' and the related concept of 'response strength' is suggested, and revisions of the model are proposed which may enable it to accommodate several behavioural phenomena not considered in the original formulation. Broadening the scope of MHM comes at the cost of adding to its burden of free parameters, and it is emphasised that addition of any new parameters needs empirical justification. The status of value as a dimensionless intervening variable is upheld; however it is noted that a growing body of empirical evidence for links between neurobiological phenomena and value suggests that interpretation of value as a hypothetical construct may be warranted.

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Brain and Cognition for Addiction Medicine: From Prevention to Recovery Neural Substrates for Treatment of Psychostimulant-Induced Cognitive Deficits

Addiction to psychostimulants like cocaine, methamphetamine, and nicotine poses a continuing medical and social challenge both in the United States and all over the world. Despite a desire to quit drug use, return to drug use after a period of abstinence is a common problem among individuals dependent on psychostimulants. Recovery for psychostimulant drug-dependent individuals is particularly challenging because psychostimulant drugs induce significant changes in brain regions associated with cognitive functions leading to cognitive deficits. These cognitive deficits include impairments in learning/memory, poor decision making, and impaired control of behavioral output. Importantly, these drug-induced cognitive deficits often impact adherence to addiction treatment programs and predispose abstinent addicts to drug use relapse. Additionally, these cognitive deficits impact effective social and professional rehabilitation of abstinent addicts. The goal of this paper is to review neural substrates based on animal studies that could be pharmacologically targeted to reverse psychostimulant-induced cognitive deficits such as impulsivity and impairment in learning and memory. Further, the review will discuss neural substrates that could be used to facilitate extinction learning and thus reduce emotional and behavioral responses to drug-associated cues. Moreover, the review will discuss some non-pharmacological approaches that could be used either alone or in combination with pharmacological compounds to treat the above-mentioned cognitive deficits. Psychostimulant addiction treatment, which includes treatment for cognitive deficits, will help promote abstinence and allow for better rehabilitation and integration of abstinent individuals into society.

Preliminary evidence of altered neural response during intertemporal choice of losses in adult attention-deficit hyperactivity disorder

Impulsive behaviours are common symptoms of attention-deficit hyperactivity disorder (ADHD). Although previous studies have suggested functional models of impulsive behaviour, a full explanation of impulsivity in ADHD remains elusive. To investigate the detailed mechanisms behind impulsive behaviour in ADHD, we applied an economic intertemporal choice task involving gains and losses to adults with ADHD and healthy controls and measured brain activity by functional magnetic resonance imaging. In the intertemporal choice of future gains, we observed no behavioural or neural difference between the two groups. In the intertemporal choice of future losses, adults with ADHD exhibited higher discount rates than the control participants. Furthermore, a comparison of brain activity representing the sensitivity of future loss in the two groups revealed significantly lower activity in the striatum and higher activity in the amygdala in adults with ADHD than in controls. Our preliminary findings suggest that an altered size sensitivity to future loss is involved in apparent impulsive choice behaviour in adults with ADHD and shed light on the multifaceted impulsivity underlying ADHD.

Effects of sucrose concentration on choice in an adjusting-magnitude schedule

Rats were trained under a discrete-trials adjusting-magnitude schedule in which a response on lever A delivered either a larger or a smaller sucrose reinforcer (q_A1 = 8 μl, q_A2 = 64 μl) with equal probability, while a response on B delivered a reinforcer whose size q_B was adjusted according to the rats' choices. When A was preferred in a given block of trials, q_B was increased in the following block; when B was preferred, q_B was reduced in the following block. The oscillating changes in q_B, analysed by the Fourier transform, could be described by a power spectrum whose dominant frequency corresponded to a period of 40-50 trial blocks. The equilibrium value of q_B (q_B(50)) was inversely related to sucrose concentration; it significantly exceeded the arithmetic mean of q_A1 and q_A2 when the concentration was 0.2 or 0.4 M, but not when it was 0.8 or 1.6 M. Analysis by mixed-effects modelling revealed a trend for the power of oscillation of q_B to increase monotonically with sucrose concentration; the period of oscillation was not systematically related to sucrose concentration. These results are consistent with predictions derived from a revised version of the multiplicative hyperbolic model of intertemporal choice.

Impulsive Choice in Mice Lacking Paternal Expression of Grb10 Suggests Intragenomic Conflict in Behavior

The imprinted gene Grb10 is expressed in the brain from the paternal copy only. Here, Dent et al. show that paternal Grb10 regulates impulsive choices, i.e. whether an animal chooses a smaller food reward...

Imprinted genes are expressed from one parental allele only as a consequence of epigenetic events that take place in the mammalian germ line and are thought to have evolved through intragenomic conflict between parental alleles. We demonstrate, for the first time, oppositional effects of imprinted genes on brain and behavior. Specifically, we show that mice lacking paternal Grb10 make fewer impulsive choices, with no dissociable effects on a separate measure of impulsive action. Taken together with previous work showing that mice lacking maternal Nesp55 make more impulsive choices, this suggests that impulsive choice behavior is a substrate for the action of genomic imprinting. Moreover, the contrasting effect of these two genes suggests that impulsive choices are subject to intragenomic conflict and that maternal and paternal interests pull this behavior in opposite directions. Finally, these data may also indicate that an imbalance in expression of imprinted genes contributes to pathological conditions such as gambling and drug addiction, where impulsive behavior becomes maladaptive.

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Elevated outcome-anticipation and outcome-evaluation ERPs associated with a greater preference for larger-but-delayed rewards

Although waiting for a reward reduces or discounts its value, some people have a stronger tendency to wait for larger rewards and forgo smaller-but-immediate rewards. This ability to delay gratification is captured by individual differences in so-called intertemporal choices in which individuals are asked to choose between larger-but-delayed versus smaller-but-immediate rewards. The current study used event-related potentials (ERPs) to examine whether enhancement in two neurocognitive processes, outcome anticipation and outcome evaluation, modulate individual variability in intertemporal responses. After completing a behavioral intertemporal choice task, 34 participants performed an ERP gambling task. From this ERP task, we separately examined individual differences in outcome anticipation (stimulus-preceding negativity; SPN), early outcome valuation (feedback-related negativity; FRN), and late outcome evaluation (P3). We observed that both elevated outcome-anticipation (SPN) and late outcome-evaluation (P3) neural processes predicted a stronger preference toward larger-but-delayed rewards. No relationship was observed between intertemporal responses and early outcome evaluation (FRN), indicating that the relationship between outcome evaluation and intertemporal responses was specific to the late outcome-evaluation processing stream. Moreover, multiple regression analyses indicated that the SPN and P3 independently modulate individual differences in intertemporal responses, suggesting separate mechanisms underlie the relationship between these two neurocognitive processes and intertemporal responses. Accordingly, we identify two potential neurocognitive modulators of individual variability in intertemporal responses. We discuss the mechanisms underlying these modulators in terms of anticipation-related processing (SPN) and a saliency bias toward gain (compared to loss) outcomes (P3).

Nucleus accumbens core lesions induce sub-optimal choice and reduce sensitivity to magnitude and delay in impulsive choice tasks

The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making.

Role of impulsivity and reward in the anti-obesity actions of 5-HT2C receptor agonists

The selective 5-HT_2C receptor agonist lorcaserin entered clinical obesity trials with the prevalent view that satiety was a primary mechanism of action. Subsequent Phase II and III trials demonstrated efficacy in terms of weight loss, although the overall effect size (~3% placebo-corrected change) is considered modest. Lorcaserin has been approved by the FDA for the treatment of obesity with lifestyle modification, but since its introduction in 2013 its sales are in decline, probably due to its overall modest effect. However, in some individuals, lorcaserin has a much more clinically significant effect (i.e. >10% placebo-corrected change), although what common features, if any, define these high responders is presently unknown. In the present article we highlight the evidence that alternative mechanisms to satiety may contribute to the anti-obesity effect of lorcaserin, namely effects on constructs of primary and conditioned reward and impulsivity. This may better inform the clinical evaluation of lorcaserin (and any future 5-HT_2C receptor agonists) to subgroups of obese subjects characterized by overeating due to maladaptive impulsivity and reward mechanisms. One such population might be individuals diagnosed with binge eating disorder.

Effects of intra-accumbal administration of dopamine and ionotropic glutamate receptor drugs on delay discounting performance in rats

Nucleus accumbens core (NAcc) has been implicated in impulsive choice, as measured in delay discounting. The role of dopamine (DA) in impulsive choice has received considerable attention, whereas glutamate (Glu) has recently been shown to be an important mediator of discounting. However, research has not examined how DA or Glu receptors in NAcc mediate different aspects of delay discounting performance, that is, (a) sensitivity to reinforcer magnitude and (b) sensitivity to delayed reinforcement. Adult male Sprague-Dawley rats were first trained in a delay discounting task, in which the delay to a large magnitude food reinforcer increased across blocks of trials. Following behavioral training, rats received bilateral implantation of guide cannulas into NAcc. Half of the rats (n = 12) received infusions of the DA-selective ligands SKF 38393 (D1-like agonist: 0.03 or 0.1 μg), SCH 23390 (D1-like antagonist: 0.3 or 1.0 μg), quinpirole (D2-like agonist: 0.3 or 1.0 μg), and eticlopride (D2-like antagonist: 0.3 or 1.0 μg). The other half received infusions of the ionotropic Glu ligands MK-801 (NMDA uncompetitive antagonist: 0.3 or 1.0 μg), AP-5 (NMDA competitive antagonist: 0.3 or 1.0 μg), ifenprodil (noncompetitive antagonist at NR2B-containing NMDA receptors: 0.3 or 1.0 μg), and CNQX (AMPA competitive antagonist: 0.2 or 0.5 μg). Results showed that SCH 23390 (0.3 μg) decreased sensitivity to reinforcer magnitude without altering impulsive choice, whereas ifenprodil (1.0 μg) decreased sensitivity to delayed reinforcement (i.e., impulsive choice). The current results show that DA and NMDA receptors in NAcc mediate distinct aspects of discounting performance. (PsycINFO Database Record

Some properties of an adjusting-magnitude schedule of reinforcement: Implications for models of choice

Rats were trained under a discrete-trials adjusting-magnitude schedule in which a response on lever A delivered either a larger or a smaller reinforcer (q_A1 and q_A2) with equal probability, while a response on B delivered a reinforcer whose size q_B was adjusted according to the rats' choices. When A was preferred in a given block of trials, q_B was increased in the following block; when B was preferred, q_B was reduced in the following block. The oscillating changes in q_B, analysed by the Fourier transform, could be described by a power spectrum with a dominant period of about 50 trial blocks. With q_A1 held constant, the equilibrium value of q_B (q_B(50)) was monotonically related to q_A2, and exceeded the arithmetic mean of q_A1 and q_A2 when q_A1 was substantially greater than q_A2. A model derived from the multiplicative model of intertemporal choice provided a post hoc description of the data. Simulation of block-by-block changes in q_B derived from the model were generally consistent with the experimental data. Implications of the results for models of risky choice and for future use of the schedule in neurobehavioural experiments are discussed.

Effects of N-methyl-D-aspartate receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure

RATIONALE

The N-methyl-D-aspartate (NMDA) receptor has been recently identified as an important mediator of impulsive choice, as assessed in delay discounting. Although discounting is independently influenced by sensitivity to reinforcer magnitude and delayed reinforcement, few studies have examined how NMDA receptor ligands differentially affect these parameters.

OBJECTIVES

The current study examined the effects of various NMDA receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure.

METHODS

Following behavioral training, rats received treatments of the following NMDA receptor ligands: the uncompetitive antagonists ketamine (0, 1.0, 5.0, or 10.0 mg/kg; i.p.), MK-801 (0, 0.003, 0.01, or 0.03 mg/kg; s.c.), and memantine (0, 2.5, 5.0, or 10.0 mg/kg; i.p.), the competitive antagonist CGS 19755 (0, 5.0, 10.0, or 20.0 mg/kg; s.c.), the non-competitive NR2B subunit-selective antagonist ifenprodil (0, 1.0, 3.0, or 10.0 mg/kg; i.p), and the partial agonist D-cycloserine (0, 3.25, 15.0, or 30.0 mg/kg; s.c.).

RESULTS

When an exponential model was used to describe discounting, CGS 19755 (5.0 mg/kg) increased impulsive choice without altering sensitivity to reinforcer magnitude. Conversely, ketamine (10.0 mg/kg), memantine (5.0 mg/kg), and ifenprodil (10.0 mg/kg) decreased sensitivity to reinforcer magnitude without altering impulsive choice. MK-801 and D-cycloserine did not alter delay-discounting performance, although two-way ANOVA analyses indicated D-cycloserine (15.0 mg/kg) decreased impulsive choice.

CONCLUSIONS

The behavioral changes observed in delay discounting following administration of NMDA receptor antagonists do not always reflect an alteration in impulsive choice. These results emphasize the utility in employing quantitative methods to assess drug effects in delay discounting.

Effects of Group I metabotropic glutamate receptor antagonists on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting task in rats: Contribution of delay presentation order

Metabotropic glutamate receptor 1 (mGluR₁) blockade has been shown to decrease impulsive choice, as measured in delay discounting. However, several variables are known to influence an animal's discounting, including sensitivity to delayed reinforcement and sensitivity to reinforcer magnitude. The goal of this experiment was to determine the effects of mGluR₁, as well as mGluR₅, antagonism on these parameters. Forty Sprague Dawley rats were trained in delay discounting, in which consistently choosing a small, immediate reward reflects impulsive choice. For half of the rats, the delay to the large reinforcer increased across blocks of trials, whereas the delay decreased across the session for half of the rats. Following training, half of the rats received injections of the mGluR₁ antagonist JNJ 16259685 (JNJ; 0, 0.1, 0.3, or 1.0mg/kg; i.p), and half received injections of the mGluR₅ antagonist MPEP (0, 1.0, 3.0, or 10.0mg/kg; i.p.). Administration of JNJ increased sensitivity to delayed reinforcement (i.e., promoted impulsive choice), regardless of which schedule was used. However, the order in which delays were presented modulated the effects of JNJ on sensitivity to reinforcer magnitude. Specifically, JNJ decreased sensitivity to reinforcer magnitude in rats trained on the descending schedule only. MPEP did not alter sensitivity to reinforcer magnitude or sensitivity to delayed reinforcement. These results show that mGluR₁ is an important mediator of impulsive choice, and they provide further evidence that delay order presentation is an important variable that influences drug effects in delay discounting.

A framework for understanding and advancing intertemporal choice research using rodent models

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit. Even though great advances have been made, our understanding of the neurobiology of intertemporal choice is still in its infancy because of the complex and dynamic nature of this form of decision making. We close by briefly discussing recommendations for the future study of intertemporal choice research.

Effects of various cannabinoid ligands on choice behaviour in a rat model of gambling

It is estimated that 0.6-1% of the population in the USA and Canada fulfil the Diagnostic and Statistical Manual of Mental Disorders, 5th ed. (DSM-5) criteria for gambling disorders (GD). To date, there are no approved pharmacological treatments for GD. The rat gambling task (rGT) is a recently developed rodent analogue of the Iowa gambling task in which rats are trained to associate four response holes with different magnitudes and probabilities of food pellet rewards and punishing time-out periods. Similar to healthy human volunteers, most rats adopt the optimal strategies (optimal group). However, a subset of animals show preference for the disadvantageous options (suboptimal group), mimicking the choice pattern of patients with GD. Here, we explored for the first time the effects of various cannabinoid ligands (WIN 55,212-2, AM 4113, AM 630 and URB 597) on the rGT. Administration of the cannabinoid agonist CB1/CB2 WIN 55,212-2 improved choice strategy and increased choice latency in the suboptimal group, but only increased perseverative behaviour, when punished, in the optimal group. Blockade of CB1 or CB2 receptors or inhibition of fatty-acid amide hydrolase did not affect rGT performance. These results suggest that stimulation of cannabinoid receptors could affect gambling choice behaviours differentially in some subgroups of subjects.

Effects of NMDA receptor antagonists on probability discounting depend on the order of probability presentation

Risky decision making can be measured using a probability-discounting procedure, in which animals choose between a small, certain reinforcer and a large, uncertain reinforcer. Recent evidence has identified glutamate as a mediator of risky decision making, as blocking the N-methyl-d-aspartate (NMDA) receptor with MK-801 increases preference for a large, uncertain reinforcer. Because the order in which probabilities associated with the large reinforcer can modulate the effects of drugs on choice, the current study determined if NMDA receptor ligands alter probability discounting using ascending and descending schedules. Sixteen rats were trained in a probability-discounting procedure in which the odds against obtaining the large reinforcer increased (n=8) or decreased (n=8) across blocks of trials. Following behavioral training, rats received treatments of the NMDA receptor ligands MK-801 (uncompetitive antagonist; 0, 0.003, 0.01, or 0.03mg/kg), ketamine (uncompetitive antagonist; 0, 1.0, 5.0, or 10.0mg/kg), and ifenprodil (NR2B-selective non-competitive antagonist; 0, 1.0, 3.0, or 10.0mg/kg). Results showed discounting was steeper (indicating increased risk aversion) for rats on an ascending schedule relative to rats on the descending schedule. Furthermore, the effects of MK-801, ketamine, and ifenprodil on discounting were dependent on the schedule used. Specifically, the highest dose of each drug decreased risk taking in rats in the descending schedule, but only MK-801 (0.03mg/kg) increased risk taking in rats on an ascending schedule. These results show that probability presentation order modulates the effects of NMDA receptor ligands on risky decision making.

Motivation, Working Memory, and Decision Making: A Cognitive-Motivational Theory of Personality Vulnerability to Alcoholism

This article presents a cognitive-motivational theory (CMT) of the mechanisms associated with three basic dimensions of personality vulnerability to alcoholism, impulsivity/novelty seeking, harm avoidance, and excitement seeking. CMT describes the interrelationships between activity in basic motivational systems and attentional, decision-making, and working memory processes as the mechanisms associated with variation in each personality trait. Impulsivity/novelty seeking reflects activity in both appetitive and inhibitory motivational systems, greater attention to reward cues, and increased emotional reactivity to reward and frustration. Harm avoidance reflects individual differences in fearfulness and activity in specific inhibitory systems. Excitement seeking reflects the need to engage in appetitive behaviors in less predictable environments to experience positive affect. CMT also describes the impact of working memory and the specific motivational processes underlying each trait dimension on the dynamics ofdecision making from the perspective of decision field theory.

Drug Abuse as a Problem of Impaired Control: Current Approaches and Findings

The author presents a selective and integrative overview of cur-rent research examining acute effects of abused drugs on aspects of behavioral control in humans. The review discusses the concept of behavioral control and reviews research models that examine inhibitory influences as a central mechanism of control. Evidence for drug-induced impairment of inhibitory mechanisms is reviewed from current neuroanatomical, neuropharmacological, and behavioral perspectives. Model-based laboratory assessments of behavioral control are described in terms of their application to behavioral pharmacology and evidence for alcohol and psychostimulant effects on basic inhibitory mechanisms is reviewed. Drug effects on inhibitory mechanisms are also discussed in terms of their relevance to higher-order cognitive and behavioral functions, and in relation to impulsive behaviors, such as drug abuse. The review concludes by identifying important drug-related phenomenon (e.g., drug tolerance, cue reactivity) that might also be studied in behavioral control frameworks in future research.

The effects of physical activity on impulsive choice: Influence of sensitivity to reinforcement amount and delay

Impulsive choice is a diagnostic feature and/or complicating factor for several psychological disorders and may be examined in the laboratory using delay-discounting procedures. Recent investigators have proposed using quantitative measures of analysis to examine the behavioral processes contributing to impulsive choice. The purpose of this study was to examine the effects of physical activity (i.e., wheel running) on impulsive choice in a single-response, discrete-trial procedure using two quantitative methods of analysis. To this end, rats were assigned to physical activity or sedentary groups and trained to respond in a delay-discounting procedure. In this procedure, one lever always produced one food pellet immediately, whereas a second lever produced three food pellets after a 0, 10, 20, 40, or 80-s delay. Estimates of sensitivity to reinforcement amount and sensitivity to reinforcement delay were determined using (1) a simple linear analysis and (2) an analysis of logarithmically transformed response ratios. Both analyses revealed that physical activity decreased sensitivity to reinforcement amount and sensitivity to reinforcement delay. These findings indicate that (1) physical activity has significant but functionally opposing effects on the behavioral processes that contribute to impulsive choice and (2) both quantitative methods of analysis are appropriate for use in single-response, discrete-trial procedures.

Impulsivity and sensitivity to amount and delay of reinforcement in an animal model of ADHD

Previous research has been inconclusive about the degree of impulsivity displayed by spontaneously hypertensive rats (SHR), an animal model of Attention Deficit Hyperactivity Disorder (ADHD). In the present set of experiments, concurrent-chains schedules were employed in order to explore SHR's impulsivity, sensitivity to delay, and sensitivity to amount of reinforcement; Wistar rats (WIS) were used as comparison group. In the three experiments - performed with different subjects - non-independent variable interval 30s schedules were presented in the initial links; the difference between experiments was in the terminal links. For exploring impulsivity, one of the terminal links (SS) was associated to a short delay (2s) and a small reinforcer (1 pellet), whereas the other terminal link (LL) was associated to a longer delay (28s) and a larger reinforcer (4 pellets). The results indicated a remarkably higher impulsivity in SHR. Because this impulsivity may have as potential mechanisms an increased sensitivity to delay and/or a decreased sensitivity to the amount of reinforcement, in experiments 2 and 3 these possibilities were examined. For assessing sensitivity to delay, the following pairs of fixed interval (FI) schedules were used in the terminal links in five conditions: 2-28, 6-24, 15-15, 24-6, 28-2s; the magnitude of reinforcement was 1 pellet in all conditions for both alternatives. For assessing sensitivity to amount, in five conditions the alternatives were associated with different magnitudes of reinforcement: 1-5 pellets, 2-4, 3-3, 4-2 and 5-1 in left-right alternatives, respectively; the delay to reinforcement was controlled by a FI 15s in all conditions and for both alternatives. The sensitivity to delay and the sensitivity to amount were calculated according to the Generalized Matching Law. The results indicated a higher sensitivity to delay in SHR, and the same sensitivity to amount in SHR and WIS rats. These results suggest that the increased sensitivity to delay influences the high level of impulsivity observed in SHR.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Impulsivity in borderline personality disorder

BACKGROUND

Impulsivity is a core feature of borderline personality disorder (BPD) and is most frequently measured using self-rating scales. There is a need to find objective, valid and reliable measures of impulsivity. This study aimed to examine performance of participants with BPD compared with healthy controls on delay and probabilistic discounting tasks and the stop-signal task (SST), which are objective measures of choice and motor impulsivity, respectively.

METHOD

A total of 20 participants with BPD and 21 healthy control participants completed delay and probabilistic discounting tasks and the SST. They also completed the Barratt Impulsiveness Scale (BIS), a self-rating measure of impulsivity.

RESULTS

Participants with BPD showed significantly greater delay discounting than controls, manifest as a greater tendency to accept the immediately available lesser reward rather than waiting longer for a greater reward. Similarly they showed significantly greater discounting of rewards by the probability of payout, which correlated with past childhood trauma. Participants with BPD were found to choose the more certain and/or immediate rewards, irrespective of the value. On the SST the BPD and control groups did not differ significantly, demonstrating no difference in motor impulsivity. There was no significant difference between groups on self-reported impulsivity as measured by the BIS.

CONCLUSIONS

Measures of impulsivity show that while motor impulsivity was not significantly different in participants with BPD compared with controls, choice or reward-related impulsivity was significantly affected in those with BPD. This suggests that choice impulsivity but not motor impulsivity is a core feature of BPD.

Role of ionotropic glutamate receptors in delay and probability discounting in the rat

RATIONALE

Discounting of delayed and probabilistic reinforcement is linked to increased drug use and pathological gambling. Understanding the neurobiology of discounting is important for designing treatments for these disorders. Glutamate is considered to be involved in addiction-like behaviors; however, the role of ionotropic glutamate receptors (iGluRs) in discounting remains unclear.

OBJECTIVES

The current study examined the effects of N-methyl-D-aspartate (NMDA) and α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) glutamate receptor blockade on performance in delay and probability discounting tasks.

METHODS

Following training in either delay or probability discounting, rats (n = 12, each task) received pretreatments of the NMDA receptor antagonists MK-801 (0, 0.01, 0.03, 0.1, or 0.3 mg/kg, s.c.) or ketamine (0, 1.0, 5.0, or 10.0 mg/kg, i.p.), as well as the AMPA receptor antagonist CNQX (0, 1.0, 3.0, or 5.6 mg/kg, i.p.). Hyperbolic discounting functions were used to estimate sensitivity to delayed/probabilistic reinforcement and sensitivity to reinforcer amount.

RESULTS

An intermediate dose of MK-801 (0.03 mg/kg) decreased sensitivity to both delayed and probabilistic reinforcement. In contrast, ketamine did not affect the rate of discounting in either task but decreased sensitivity to reinforcer amount. CNQX did not alter sensitivity to reinforcer amount or delayed/probabilistic reinforcement.

CONCLUSIONS

These results show that blockade of NMDA receptors, but not AMPA receptors, decreases sensitivity to delayed/probabilistic reinforcement (MK-801) and sensitivity to reinforcer amount (ketamine). The differential effects of MK-801 and ketamine demonstrate that sensitivities to delayed/probabilistic reinforcement and reinforcer amount are pharmacologically dissociable.

Comparing hyperbolic, delay-amount sensitivity and present-bias models of delay discounting

Delay discounting is a widely studied phenomenon due to its ubiquity in psychopathological disorders. Several methods are well established to quantify the extent to which a delayed commodity is devalued as a function of the delay to its receipt. The most frequently used method is to fit a hyperbolic function and use an index of the gradient of the function, k, or to calculate the area under the discounting curve. The manuscript examines the behavior of these quantification indices for three different datasets, as well as provides information about potential limitations in their use. The primary limitation examined is the lack of mechanistic specificity provided by either method. Alternative formulations that are thought to provide some mechanistic information are examined for the three separate datasets: two variants of a hyperboloid model (Rachlin, 1989, Judgment, decision and choice. New York: W.H. Freeman) and the quasi-hyperbolic model (Laibson, 1997, Q. J. Econ., 112, 443-477). Examination of the parameters of each formulation suggests that the parameters derived from the quasi-hyperbolic model allows groups and conditions within the three datasets to be reliably distinguished more readily than the hyperboloid models. However, use of the quasi-hyperbolic model is complex and its limitations might offset its ability to discriminate within the datasets. "This article is part of a Special Issue entitled: SQAB 2014".

Partial inactivation of nucleus accumbens core decreases delay discounting in rats without affecting sensitivity to delay or magnitude

Increased preference for smaller, sooner rewards (delay discounting) is associated with several behavioral disorders, including ADHD and substance use disorders. However, delay discounting is a complex cognitive process and the relationship is unclear between the pathophysiology of the disorders and the component processes underlying delay discounting, including sensitivity to reinforcer delay and sensitivity to reinforcer magnitude. To investigate these processes, male Long Evans rats were trained in one of three tasks measuring sensitivity to delay, sensitivity to magnitude, or both (typical delay discounting task). After learning the task, animals were implanted with bilateral cannulae into either the nucleus accumbens core (AcbC) or the lateral orbitofrontal cortex (lOFC), both of which have been implicated in delay discounting. Upon recovering from the surgery, a baclofen/muscimol cocktail was infused to temporarily inactivate each of these two regions and task performance was assessed. Unlike previous studies showing that lesions of the AcbC increased delay discounting, partial inactivation of the AcbC decreased delay discounting, although it had no effects on the tasks independently assessing either sensitivity to delay or magnitude. The effects of AcbC inactivation were larger in animals that had low levels of delay discounting at baseline. Inactivation of the lOFC had no effects on behavior in any task. These findings suggest that the AcbC may act to promote impulsive choice in individuals with low impulsivity. Furthermore, the data suggest that the AcbC is able to modulate delay and magnitude sensitivity together, but not either of the two in isolation.

Discounting of monetary rewards that are both delayed and probabilistic: delay and probability combine multiplicatively, not additively

The value of an outcome is affected both by the delay until its receipt (delay discounting) and by the likelihood of its receipt (probability discounting). Despite being well-described by the same hyperboloid function, delay and probability discounting involve fundamentally different processes, as revealed, for example, by the differential effects of reward amount. Previous research has focused on the discounting of delayed and probabilistic rewards separately, with little research examining more complex situations in which rewards are both delayed and probabilistic. In 2 experiments, participants made choices between smaller rewards that were both immediate and certain and larger rewards that were both delayed and probabilistic. Analyses revealed significant interactions between delay and probability factors inconsistent with an additive model. In contrast, a hyperboloid discounting model in which delay and probability were combined multiplicatively provided an excellent fit to the data. These results suggest that the hyperboloid is a good descriptor of decision making in complicated monetary choice situations like those people encounter in everyday life.

Role of medial prefrontal and orbitofrontal monoamine transporters and receptors in performance in an adjusting delay discounting procedure

Performance in an adjusting delay discounting procedure is predictive of drug abuse vulnerability; however, the shared underlying specific prefrontal neural systems linking delay discounting and increased addiction-like behaviors are unclear. Rats received direct infusions of methylphenidate (MPH; 6.25, 25.0, or 100μg), amphetamine (AMPH; 0.25, 1.0, or 4.0μg), or atomoxetine (ATO; 1.0, 4.0, or 16.0μg) into either medial prefrontal cortex (mPFC) or orbitofrontal cortex (OFC) immediately prior to performance in an adjusting delay task. These drugs were examined because they are efficacious in treating impulse control disorders. Because dopamine (DA) and serotonin (5-HT) receptors are implicated in impulsive behavior, separate groups of rats received microinfusions of the DA receptor-selective drugs SKF 81297 (0.1 or 0.4µg), SCH 23390 (0.25 or 1.0µg), quinpirole (1.25 or 5.0µg), and eticlopride (0.25 or 1.0µg), or received microinfusions of the 5-HT receptor-selective drugs 8-OH-DPAT (0.025 or 0.1μg), WAY 100635 (0.01 or 0.04μg), DOI (2.5 or 10.0μg), and ketanserin (0.1 or 0.4μg). Impulsive choice was not altered significantly by MPH, AMPH, or ATO into either mPFC or OFC, indicating that neither of these prefrontal regions alone may mediate the systemic effect of ADHD medications on impulsive choice. However, quinpriole (1.25μg) and eticlopride infused into mPFC increased impulsive choice, whereas 8-OH-DPAT infused into OFC decreased impulsive choice. These latter results demonstrate that blockade of DA D2 receptors in mPFC or activation of 5-HT1A receptors in OFC increases impulsive choice in the adjusting delay procedure.