Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis

Not all discounts are created equal: Regional activity and brain networks in temporal and effort discounting

Reward outcomes associated with costs like time delay and effort investment are generally discounted in decision-making. Standard economic models predict rewards associated with different types of costs are devalued in a similar manner. However, our review of rodent lesion studies indicated partial dissociations between brain regions supporting temporal- and effort-based decision-making. Another debate is whether options involving low and high costs are processed in different brain substrates (dual-system) or in the same regions (single-system). This research addressed these issues using coordinate-based, connectivity-based, and activation network-based meta-analyses to identify overlapping and separable neural systems supporting temporal (39 studies) and effort (20 studies) discounting. Coordinate-based activation likelihood estimation and resting-state connectivity analyses showed immediate-small reward and delayed-large reward choices engaged distinct regions with unique connectivity profiles, but their activation network mapping was found to engage the default mode network. For effort discounting, salience and sensorimotor networks supported low-effort choices, while the frontoparietal network supported high-effort choices. There was little overlap between the temporal and effort networks. Our findings underscore the importance of differentiating different types of costs in decision-making and understanding discounting at both regional and network levels.

Decision Making: a Theoretical Review

Decision-making is a crucial skill that has a central role in everyday life and is necessary for adaptation to the environment and autonomy. It is the ability to choose between two or more options, and it has been studied through several theoretical approaches and by different disciplines. In this overview article, we contend a theoretical review regarding most theorizing and research on decision-making. Specifically, we focused on different levels of analyses, including different theoretical approaches and neuropsychological aspects. Moreover, common methodological measures adopted to study decision-making were reported. This theoretical review emphasizes multiple levels of analysis and aims to summarize evidence regarding this fundamental human process. Although several aspects of the field are reported, more features of decision-making process remain uncertain and need to be clarified. Further experimental studies are necessary for understanding this process better and for integrating and refining the existing theories.

In search of a definition of reinforcer value: Some successes and failures of the multiplicative hyperbolic model

The concept of 'value' has enjoyed a central position in many theoretical accounts of choice behaviour. Several definitions of 'value' are contrasted in this paper, and one particular approach is defended, whereby value is defined as a dimensionless intervening variable. This definition is a cornerstone of the multiplicative hyperbolic model of choice (MHM), which was proposed twenty years ago as a modification of Mazur's (1987) hyperbolic model of delay discounting. This paper reviews some of the merits and shortcomings of MHM, and suggests some ways in which MHM might be extended and improved. A formal link between 'value' and the related concept of 'response strength' is suggested, and revisions of the model are proposed which may enable it to accommodate several behavioural phenomena not considered in the original formulation. Broadening the scope of MHM comes at the cost of adding to its burden of free parameters, and it is emphasised that addition of any new parameters needs empirical justification. The status of value as a dimensionless intervening variable is upheld; however it is noted that a growing body of empirical evidence for links between neurobiological phenomena and value suggests that interpretation of value as a hypothetical construct may be warranted.

Separate Neural Networks for Gains and Losses in Intertemporal Choice

An important and unresolved question is how human brain regions process information and interact with each other in intertemporal choice related to gains and losses. Using psychophysiological interaction and dynamic causal modeling analyses, we investigated the functional interactions between regions involved in the decision-making process while participants performed temporal discounting tasks in both the gains and losses domains. We found two distinct intrinsic valuation systems underlying temporal discounting in the gains and losses domains: gains were specifically evaluated in the medial regions, including the medial prefrontal and orbitofrontal cortices, and losses were evaluated in the lateral dorsolateral prefrontal cortex. In addition, immediate reward or punishment was found to modulate the functional interactions between the dorsolateral prefrontal cortex and distinct regions in both the gains and losses domains: in the gains domain, the mesolimbic regions; in the losses domain, the medial prefrontal cortex, anterior cingulate cortex, and insula. These findings suggest that intertemporal choice of gains and losses might involve distinct valuation systems, and more importantly, separate neural interactions may implement the intertemporal choices of gains and losses. These findings may provide a new biological perspective for understanding the neural mechanisms underlying intertemporal choice of gains and losses.

Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex

Reward outcomes are available in many diverse situations and all involve choice. If there are multiple outcomes each rewarding, then decisions regarding relative value lead to choosing one over another. Important factors related to choice context should be encoded and utilized for this form of adaptive choosing. These factors can include the number of alternatives, the pacing of choice behavior and the possibility to reverse one's choice. An essential step in understanding if the context of choice is encoded is to directly compare choice with a context in which choice is absent. Neural activity in orbitofrontal cortex and striatum encodes potential value parameters related to reward quality and quantity as well as relative preference. We examined how neural activations in these brain regions are sensitive to choice situations and potentially involved in a prediction for the upcoming outcome selection. Neural activity was recorded and compared between a two-choice spatial delayed response task and an imperative 'one-option' task. Neural activity was obtained that extended from the instruction cue to the movement similar to previous work utilizing the identical imperative task. Orbitofrontal and striatal neural responses depended upon the decision about the choice of which reward to collect. Moreover, signals to predictive instruction cues that precede choice were selective for the choice situation. These neural responses could reflect chosen value with greater information on relative value of individual options as well as encode choice context itself embedded in the task as a part of the post-decision variable.

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Discounting: A practical guide to multilevel analysis of choice data

Multilevel modeling provides the ability to simultaneously evaluate the discounting of individuals and groups by examining choices between smaller sooner and larger later rewards. A multilevel logistic regression approach is advocated in which sensitivity to relative reward magnitude and relative delay are considered as separate contributors to choice. Examples of how to fit choice data using multilevel logistic models are provided to help researchers in the adoption of these methods.

Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as "OFC," and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Effects of intra-accumbal administration of dopamine and ionotropic glutamate receptor drugs on delay discounting performance in rats

Nucleus accumbens core (NAcc) has been implicated in impulsive choice, as measured in delay discounting. The role of dopamine (DA) in impulsive choice has received considerable attention, whereas glutamate (Glu) has recently been shown to be an important mediator of discounting. However, research has not examined how DA or Glu receptors in NAcc mediate different aspects of delay discounting performance, that is, (a) sensitivity to reinforcer magnitude and (b) sensitivity to delayed reinforcement. Adult male Sprague-Dawley rats were first trained in a delay discounting task, in which the delay to a large magnitude food reinforcer increased across blocks of trials. Following behavioral training, rats received bilateral implantation of guide cannulas into NAcc. Half of the rats (n = 12) received infusions of the DA-selective ligands SKF 38393 (D1-like agonist: 0.03 or 0.1 μg), SCH 23390 (D1-like antagonist: 0.3 or 1.0 μg), quinpirole (D2-like agonist: 0.3 or 1.0 μg), and eticlopride (D2-like antagonist: 0.3 or 1.0 μg). The other half received infusions of the ionotropic Glu ligands MK-801 (NMDA uncompetitive antagonist: 0.3 or 1.0 μg), AP-5 (NMDA competitive antagonist: 0.3 or 1.0 μg), ifenprodil (noncompetitive antagonist at NR2B-containing NMDA receptors: 0.3 or 1.0 μg), and CNQX (AMPA competitive antagonist: 0.2 or 0.5 μg). Results showed that SCH 23390 (0.3 μg) decreased sensitivity to reinforcer magnitude without altering impulsive choice, whereas ifenprodil (1.0 μg) decreased sensitivity to delayed reinforcement (i.e., impulsive choice). The current results show that DA and NMDA receptors in NAcc mediate distinct aspects of discounting performance. (PsycINFO Database Record

Fractionating impulsivity: neuropsychiatric implications

The ability to make decisions and act quickly without hesitation can be advantageous in many settings. However, when persistently expressed, impulsive decisions and actions are considered risky, maladaptive and symptomatic of such diverse brain disorders as attention-deficit hyperactivity disorder, drug addiction and affective disorders. Over the past decade, rapid progress has been made in the identification of discrete neural networks that underlie different forms of impulsivity - from impaired response inhibition and risky decision making to a profound intolerance of delayed rewards. Herein, we review what is currently known about the neural and psychological mechanisms of impulsivity, and discuss the relevance and application of these new insights to various neuropsychiatric disorders.

Effects of N-methyl-D-aspartate receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure

RATIONALE

The N-methyl-D-aspartate (NMDA) receptor has been recently identified as an important mediator of impulsive choice, as assessed in delay discounting. Although discounting is independently influenced by sensitivity to reinforcer magnitude and delayed reinforcement, few studies have examined how NMDA receptor ligands differentially affect these parameters.

OBJECTIVES

The current study examined the effects of various NMDA receptor ligands on sensitivity to reinforcer magnitude and delayed reinforcement in a delay-discounting procedure.

METHODS

Following behavioral training, rats received treatments of the following NMDA receptor ligands: the uncompetitive antagonists ketamine (0, 1.0, 5.0, or 10.0 mg/kg; i.p.), MK-801 (0, 0.003, 0.01, or 0.03 mg/kg; s.c.), and memantine (0, 2.5, 5.0, or 10.0 mg/kg; i.p.), the competitive antagonist CGS 19755 (0, 5.0, 10.0, or 20.0 mg/kg; s.c.), the non-competitive NR2B subunit-selective antagonist ifenprodil (0, 1.0, 3.0, or 10.0 mg/kg; i.p), and the partial agonist D-cycloserine (0, 3.25, 15.0, or 30.0 mg/kg; s.c.).

RESULTS

When an exponential model was used to describe discounting, CGS 19755 (5.0 mg/kg) increased impulsive choice without altering sensitivity to reinforcer magnitude. Conversely, ketamine (10.0 mg/kg), memantine (5.0 mg/kg), and ifenprodil (10.0 mg/kg) decreased sensitivity to reinforcer magnitude without altering impulsive choice. MK-801 and D-cycloserine did not alter delay-discounting performance, although two-way ANOVA analyses indicated D-cycloserine (15.0 mg/kg) decreased impulsive choice.

CONCLUSIONS

The behavioral changes observed in delay discounting following administration of NMDA receptor antagonists do not always reflect an alteration in impulsive choice. These results emphasize the utility in employing quantitative methods to assess drug effects in delay discounting.

A framework for understanding and advancing intertemporal choice research using rodent models

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit. Even though great advances have been made, our understanding of the neurobiology of intertemporal choice is still in its infancy because of the complex and dynamic nature of this form of decision making. We close by briefly discussing recommendations for the future study of intertemporal choice research.

Adolescent methylmercury exposure affects choice and delay discounting in mice

The developing fetus is vulnerable to low-level exposure to methylmercury (MeHg), an environmental neurotoxicant, but the consequences of exposure during the adolescent period remain virtually unknown. The current experiments were designed to assess the effects of low-level MeHg exposure during adolescence on delay discounting, preference for small, immediate reinforcers over large, delayed ones, using a mouse model. Thirty-six male C57BL/6n mice were exposed to 0, 0.3, or 3.0ppm mercury (as MeHg) via drinking water from postnatal day 21 through 59, encompassing the murine adolescent period. As adults, mice lever pressed for a 0.01-cc droplet of milk solution delivered immediately or four 0.01-cc droplets delivered after a delay. Delays ranged from 1.26 to 70.79s, and all were presented within a session. A model based on the Generalized Matching Law indicated that sensitivity to reinforcer magnitude was lower for MeHg-exposed mice relative to controls, indicating that responding in MeHg-exposed mice was relatively indifferent to the larger reinforcer. Sensitivity to reinforcer delay was reduced (delay discounting was decreased) in the 0.3-ppm group, but not in the 3.0-ppm group, compared to controls. Adolescence is a developmental period during which the brain and behavior may be vulnerable to MeHg exposure. As with gestational MeHg exposure, the effects are reflected in the impact of reinforcing stimuli.

No Effect of Subthalamic Deep Brain Stimulation on Intertemporal Decision-Making in Parkinson Patients

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is a widely used treatment for the motor symptoms of Parkinson's disease (PD). DBS or pharmacological treatment is believed to modulate the tendency to, or reverse, impulse control disorders. Several brain areas involved in impulsivity and reward valuation, such as the prefrontal cortex and striatum, are linked to the STN, and activity in these areas might be affected by STN-DBS. To investigate the effect of STN-DBS on one type of impulsive decision-making--delay discounting (i.e., the devaluation of reward with increasing delay until its receipt)--we tested 40 human PD patients receiving STN-DBS treatment and medication for at least 3 months. Patients were pseudo-randomly assigned to one of four groups to test the effects of DBS on/off states as well as medication on/off states on delay discounting. The delay-discounting task consisted of a series of choices among a smaller. sooner or a larger, later monetary reward. Despite considerable effects of DBS on motor performance, patients receiving STN-DBS did not choose more or less impulsively compared with those in the off-DBS group, as well as when controlling for risk attitude. Although null results have to be interpreted with caution, our findings are of significance to other researchers studying the effects of PD treatment on impulsive decision-making, and they are of clinical relevance for determining the therapeutic benefits of using STN-DBS.

Individual differences in impulsive action and dopamine transporter function in rat orbitofrontal cortex

Impulsivity, which can be subdivided into impulsive action and impulsive choice, is implicated as a factor underlying drug abuse vulnerability. Although previous research has shown that dopamine (DA) systems in prefrontal cortex are involved in impulsivity and substance abuse, it is not known if inherent variation in DA transporter (DAT) function contributes to impulsivity. The current study determined if individual differences in either impulsive action or impulsive choice are related to DAT function in orbitofrontal (OFC) and/or medial prefrontal cortex (mPFC). Rats were first tested both for impulsive action in a cued go/no-go task and for impulsive choice in a delay-discounting task. Following behavioral evaluation, in vitro [(3)H]DA uptake assays were performed in OFC and mPFC isolated from individual rats. Vmax in OFC, but not mPFC, was correlated with performance in the cued go/no-go task, with decreased OFC DAT function being associated with high impulsive action. In contrast, Vmax in OFC and mPFC was not correlated with performance in the delay-discounting task. The current results demonstrate that impulsive behavior in cued go/no-go performance is associated with decreased DAT function in OFC, suggesting that hyperdopaminergic tone in this prefrontal subregion mediates, at least in part, increased impulsive action.

Hippocampal interplay with the nucleus accumbens is critical for decisions about time

Human cognition depends upon the capacity to make decisions in the present that bear upon outcomes in the future. The nucleus accumbens, a recipient of direct projections from both the hippocampus and orbitofrontal cortex, is known to contribute to these aspects of decision-making. Here we demonstrate that interaction of the nucleus accumbens with the hippocampus, but not the orbitofrontal cortex, is critical in shaping decisions that involve time trade-offs. Compared with controls, rats with a disrupted hippocampal-accumbens interaction were strongly biased toward choosing stimuli that led to small and immediate food rewards over large and delayed ones. We show that this pattern of behavior cannot be ascribed to the impaired representation of stimulus value, the incapacity to wait, or a general disruption of decision-making. These results identify a hippocampal-accumbens circuit that may underlie a range of problems in which daily decisions are marked by a shift toward immediate gratification.

Impulsivity and sensitivity to amount and delay of reinforcement in an animal model of ADHD

Previous research has been inconclusive about the degree of impulsivity displayed by spontaneously hypertensive rats (SHR), an animal model of Attention Deficit Hyperactivity Disorder (ADHD). In the present set of experiments, concurrent-chains schedules were employed in order to explore SHR's impulsivity, sensitivity to delay, and sensitivity to amount of reinforcement; Wistar rats (WIS) were used as comparison group. In the three experiments - performed with different subjects - non-independent variable interval 30s schedules were presented in the initial links; the difference between experiments was in the terminal links. For exploring impulsivity, one of the terminal links (SS) was associated to a short delay (2s) and a small reinforcer (1 pellet), whereas the other terminal link (LL) was associated to a longer delay (28s) and a larger reinforcer (4 pellets). The results indicated a remarkably higher impulsivity in SHR. Because this impulsivity may have as potential mechanisms an increased sensitivity to delay and/or a decreased sensitivity to the amount of reinforcement, in experiments 2 and 3 these possibilities were examined. For assessing sensitivity to delay, the following pairs of fixed interval (FI) schedules were used in the terminal links in five conditions: 2-28, 6-24, 15-15, 24-6, 28-2s; the magnitude of reinforcement was 1 pellet in all conditions for both alternatives. For assessing sensitivity to amount, in five conditions the alternatives were associated with different magnitudes of reinforcement: 1-5 pellets, 2-4, 3-3, 4-2 and 5-1 in left-right alternatives, respectively; the delay to reinforcement was controlled by a FI 15s in all conditions and for both alternatives. The sensitivity to delay and the sensitivity to amount were calculated according to the Generalized Matching Law. The results indicated a higher sensitivity to delay in SHR, and the same sensitivity to amount in SHR and WIS rats. These results suggest that the increased sensitivity to delay influences the high level of impulsivity observed in SHR.

Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models

Over the past 20 years there has been a growing interest in the neural underpinnings of cost/benefit decision-making. Recent studies with animal models have made considerable advances in our understanding of how different prefrontal, striatal, limbic and monoaminergic circuits interact to promote efficient risk/reward decision-making, and how dysfunction in these circuits underlies aberrant decision-making observed in numerous psychiatric disorders. This review will highlight recent findings from studies exploring these questions using a variety of behavioral assays, as well as molecular, pharmacological, neurophysiological, and translational approaches. We begin with a discussion of how neural systems related to decision subcomponents may interact to generate more complex decisions involving risk and uncertainty. This is followed by an overview of interactions between prefrontal-amygdala-dopamine and habenular circuits in regulating choice between certain and uncertain rewards and how different modes of dopamine transmission may contribute to these processes. These data will be compared with results from other studies investigating the contribution of some of these systems to guiding decision-making related to rewards vs. punishment. Lastly, we provide a brief summary of impairments in risk-related decision-making associated with psychiatric disorders, highlighting recent translational studies in laboratory animals.

Activation of the kynurenine pathway and increased production of the excitotoxin quinolinic acid following traumatic brain injury in humans

Abstract

During inflammation, the kynurenine pathway (KP) metabolises the essential amino acid tryptophan (TRP) potentially contributing to excitotoxicity via the release of quinolinic acid (QUIN) and 3-hydroxykynurenine (3HK). Despite the importance of excitotoxicity in the development of secondary brain damage, investigations on the KP in TBI are scarce. In this study, we comprehensively characterised changes in KP activation by measuring numerous metabolites in cerebrospinal fluid (CSF) from TBI patients and assessing the expression of key KP enzymes in brain tissue from TBI victims. Acute QUIN levels were further correlated with outcome scores to explore its prognostic value in TBI recovery.

Methods

Twenty-eight patients with severe TBI (GCS ≤ 8, three patients had initial GCS = 9–10, but rapidly deteriorated to ≤8) were recruited. CSF was collected from admission to day 5 post-injury. TRP, kynurenine (KYN), kynurenic acid (KYNA), QUIN, anthranilic acid (AA) and 3-hydroxyanthranilic acid (3HAA) were measured in CSF. The Glasgow Outcome Scale Extended (GOSE) score was assessed at 6 months post-TBI. Post-mortem brains were obtained from the Australian Neurotrauma Tissue and Fluid Bank and used in qPCR for quantitating expression of KP enzymes (indoleamine 2,3-dioxygenase-1 (IDO1), kynurenase (KYNase), kynurenine amino transferase-II (KAT-II), kynurenine 3-monooxygenase (KMO), 3-hydroxyanthranilic acid oxygenase (3HAO) and quinolinic acid phosphoribosyl transferase (QPRTase) and IDO1 immunohistochemistry.

Results

In CSF, KYN, KYNA and QUIN were elevated whereas TRP, AA and 3HAA remained unchanged. The ratios of QUIN:KYN, QUIN:KYNA, KYNA:KYN and 3HAA:AA revealed that QUIN levels were significantly higher than KYN and KYNA, supporting increased neurotoxicity. Amplified IDO1 and KYNase mRNA expression was demonstrated on post-mortem brains, and enhanced IDO1 protein coincided with overt tissue damage. QUIN levels in CSF were significantly higher in patients with unfavourable outcome and inversely correlated with GOSE scores.

Conclusion

TBI induced a striking activation of the KP pathway with sustained increase of QUIN. The exceeding production of QUIN together with increased IDO1 activation and mRNA expression in brain-injured areas suggests that TBI selectively induces a robust stimulation of the neurotoxic branch of the KP pathway. QUIN’s detrimental roles are supported by its association to adverse outcome potentially becoming an early prognostic factor post-TBI.

Dissociable roles of dopamine and serotonin transporter function in a rat model of negative urgency

Negative urgency is a facet of impulsivity that reflects mood-based rash action and is associated with various maladaptive behaviors in humans. However, the underlying neural mechanisms of negative urgency are not fully understood. Several brain regions within the mesocorticolimbic pathway, as well as the neurotransmitters dopamine (DA) and serotonin (5-HT), have been implicated in impulsivity. Extracellular DA and 5-HT concentrations are regulated by DA transporters (DAT) and 5-HT transporters (SERT); thus, these transporters may be important molecular mechanisms underlying individual differences in negative urgency. The current study employed a reward omission task to model negative urgency in rats. During reward trials, a cue light signaled the non-contingent delivery of one sucrose pellet; immediately following the non-contingent reward, rats responded on a lever to earn sucrose pellets (operant phase). Omission trials were similar to reward trials, except that non-contingent sucrose was omitted following the cue light prior to the operant phase. As expected, contingent responding was higher following omission of expected reward than following delivery of expected reward, thus reflecting negative urgency. Upon completion of behavioral training, Vmax and Km were obtained from kinetic analysis of [(3)H]DA and [(3)H]5-HT uptake using synaptosomes prepared from nucleus accumbens (NAc), dorsal striatum (Str), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC) isolated from individual rats. Vmax for DAT in NAc and for SERT in OFC were positively correlated with negative urgency scores. The current findings suggest that mood-based impulsivity (negative urgency) is associated with enhanced DAT function in NAc and SERT function in OFC.

A dual-systems perspective on addiction: contributions from neuroimaging and cognitive training

Dual-systems theories explain lapses in self-control in terms of a conflict between automatic and deliberative modes of behavioral control. Numerous studies have now tested whether the brain areas that control behavior are organized in a manner consistent with dual-systems models. Brain regions directly associated with the mesolimbic dopamine system, the nucleus accumbens and ventromedial prefrontal cortex in particular, capture some of the features assumed by automatic processing. Regions in the lateral prefrontal cortex are more closely linked to deliberative processing and the exertion of self-control in the suppression of impulses. While identifying these regions crudely supports dual-systems theories, important modifications to what constitutes automatic and deliberative behavioral control are also suggested. Experiments have identified various means by which automatic processes may be sculpted. Additional work decomposes deliberative processes into component functions such as generalized working memory, reappraisal of emotional stimuli, and prospection. The importance of deconstructing dual-systems models into specific cognitive processes is clear for understanding and treating addiction. We discuss intervention possibilities suggested by recent research, and focus in particular on cognitive training approaches to bolster deliberative control processes that may aid quit attempts.

Partial inactivation of nucleus accumbens core decreases delay discounting in rats without affecting sensitivity to delay or magnitude

Increased preference for smaller, sooner rewards (delay discounting) is associated with several behavioral disorders, including ADHD and substance use disorders. However, delay discounting is a complex cognitive process and the relationship is unclear between the pathophysiology of the disorders and the component processes underlying delay discounting, including sensitivity to reinforcer delay and sensitivity to reinforcer magnitude. To investigate these processes, male Long Evans rats were trained in one of three tasks measuring sensitivity to delay, sensitivity to magnitude, or both (typical delay discounting task). After learning the task, animals were implanted with bilateral cannulae into either the nucleus accumbens core (AcbC) or the lateral orbitofrontal cortex (lOFC), both of which have been implicated in delay discounting. Upon recovering from the surgery, a baclofen/muscimol cocktail was infused to temporarily inactivate each of these two regions and task performance was assessed. Unlike previous studies showing that lesions of the AcbC increased delay discounting, partial inactivation of the AcbC decreased delay discounting, although it had no effects on the tasks independently assessing either sensitivity to delay or magnitude. The effects of AcbC inactivation were larger in animals that had low levels of delay discounting at baseline. Inactivation of the lOFC had no effects on behavior in any task. These findings suggest that the AcbC may act to promote impulsive choice in individuals with low impulsivity. Furthermore, the data suggest that the AcbC is able to modulate delay and magnitude sensitivity together, but not either of the two in isolation.

Role of medial prefrontal and orbitofrontal monoamine transporters and receptors in performance in an adjusting delay discounting procedure

Performance in an adjusting delay discounting procedure is predictive of drug abuse vulnerability; however, the shared underlying specific prefrontal neural systems linking delay discounting and increased addiction-like behaviors are unclear. Rats received direct infusions of methylphenidate (MPH; 6.25, 25.0, or 100μg), amphetamine (AMPH; 0.25, 1.0, or 4.0μg), or atomoxetine (ATO; 1.0, 4.0, or 16.0μg) into either medial prefrontal cortex (mPFC) or orbitofrontal cortex (OFC) immediately prior to performance in an adjusting delay task. These drugs were examined because they are efficacious in treating impulse control disorders. Because dopamine (DA) and serotonin (5-HT) receptors are implicated in impulsive behavior, separate groups of rats received microinfusions of the DA receptor-selective drugs SKF 81297 (0.1 or 0.4µg), SCH 23390 (0.25 or 1.0µg), quinpirole (1.25 or 5.0µg), and eticlopride (0.25 or 1.0µg), or received microinfusions of the 5-HT receptor-selective drugs 8-OH-DPAT (0.025 or 0.1μg), WAY 100635 (0.01 or 0.04μg), DOI (2.5 or 10.0μg), and ketanserin (0.1 or 0.4μg). Impulsive choice was not altered significantly by MPH, AMPH, or ATO into either mPFC or OFC, indicating that neither of these prefrontal regions alone may mediate the systemic effect of ADHD medications on impulsive choice. However, quinpriole (1.25μg) and eticlopride infused into mPFC increased impulsive choice, whereas 8-OH-DPAT infused into OFC decreased impulsive choice. These latter results demonstrate that blockade of DA D2 receptors in mPFC or activation of 5-HT1A receptors in OFC increases impulsive choice in the adjusting delay procedure.

Increased firing to cues that predict low-value reward in the medial orbitofrontal cortex

Anatomical, imaging, and lesion work have suggested that medial and lateral aspects of orbitofrontal cortex (OFC) play different roles in reward-guided decision-making, yet few single-neuron recording studies have examined activity in more medial parts of the OFC (mOFC) making it difficult to fully assess its involvement in motivated behavior. Previously, we have shown that neurons in lateral parts of the OFC (lOFC) selectively fire for rewards of different values. In that study, we trained rats to respond to different fluid wells for rewards of different sizes or delivered at different delays. Rats preferred large over small reward, and rewards delivered after short compared with long delays. Here, we recorded from single neurons in rat rostral mOFC as they performed the same task. Similar to the lOFC, activity was attenuated for rewards that were delivered after long delays and was enhanced for delivery of larger rewards. However, unlike lOFC, odor-responsive neurons in the mOFC were more active when cues predicted low-value outcomes. These data suggest that odor-responsive mOFC neurons signal the association between environmental cues and unfavorable outcomes during decision making.

Effect of orbitofrontal cortex lesions on temporal discounting in rats

Although choices of both humans and animals are more strongly influenced by immediate than delayed rewards, methodological limitations have made it difficult to estimate the precise form of temporal discounting in animals. In the present study, we sought to characterize temporal discounting in rats and to test the role of the orbitofrontal cortex (OFC) in this process. Rats were trained in a novel intertemporal choice task in which the sequence of delay durations was randomized across trials. The animals tended to choose a small immediate reward more frequently as the delay for a large reward increased, and, consistent with previous findings in other species, their choice behavior was better accounted for by hyperbolic than exponential discount functions. In addition, model comparisons showed that the animal's choice behavior was better accounted for by more complex discount functions with an additional parameter than a hyperbolic discount function. Following bilateral OFC lesions, rats extensively trained in this task showed no significant change in their intertemporal choice behavior. Our results suggest that the rodent OFC may not always play a role in temporal discounting when delays are randomized and/or after extensive training.

A review on the relationship between testosterone and life-course persistent antisocial behavior

Life-course persistent antisocial behavior is 10 to 14 times more prevalent in males and it has been suggested that testosterone levels could account for this gender bias. Preliminary studies with measures of fetal testosterone find inconsistent associations with antisocial behavior, especially studies that use the 2D:4D ratio as a proxy for fetal testosterone. However, circulating testosterone consistently shows positive associations with antisocial behaviors throughout childhood, adolescence, and adulthood, particularly in males. It is suggested that high fetal/circulating testosterone interactively influence the maturation and functionality of mesolimbic dopaminergic circuitry, right orbitofrontal cortex, and cortico-subcortical connectivity, resulting in a strong reward motivation, low social sensitivity, and dampened regulation of strong motivational/emotional processes. The link between these testosterone induced endophenotypes and actual display of antisocial behavior is strongly modulated by different social (e.g., social rejection, low SES) and genetic (e.g., MAOA, 5HTT) risk factors that can disturb socio-, psycho-, and biological development and interact with testosterone in shaping behavior. When these additional risk factors are present, the testosterone induced endophenotypes may increase the risk for a chronic antisocial lifestyle. However, behavioral endophenotypes induced by testosterone can also predispose towards socially adaptive traits such as a strong achievement motivation, leadership, fair bargaining behaviors, and social assertiveness. These adaptive traits are more likely to emerge when the high testosterone individual has positive social experiences that promote prosocial behaviors such as strong and secure attachments with his caregivers, affiliation with prosocial peers, and sufficient socioeconomic resources. A theoretical model is presented, various hypotheses are examined, and future venues for research are discussed.

Towards a general model of temporal discounting

Psychological models of temporal discounting have now successfully displaced classical economic theory due to the simple fact that many common behavior patterns, such as impulsivity, were unexplainable with classic models. However, the now dominant hyperbolic model of discounting is itself becoming increasingly strained. Numerous factors have arisen that alter discount rates with no means to incorporate the different influences into standard hyperbolic models. Furthermore, disparate literatures are emerging that propose theoretical constructs that are seemingly independent of hyperbolic discounting. We argue that, although hyperbolic discounting provides an eminently useful quantitative measure of discounting, it fails as a descriptive psychological model of the cognitive processes that produce intertemporal preferences. Instead, we propose that recent contributions from cognitive neuroscience indicate a path for developing a general model of time discounting. New data suggest a means by which neuroscience-based theory may both integrate the diverse empirical data on time preferences and merge seemingly disparate theoretical models that impinge on time preferences.

Dissociable contributions of the ventral hippocampus and orbitofrontal cortex to decision-making with a delayed or uncertain outcome

In this study, we examined how risk and delay influence rats' decision-making, and the role of the ventral hippocampus (VHC) and orbitofrontal cortex (OFC) in the valuation of these two factors. We used a touchscreen testing method in which rats with VHC lesions, OFC lesions and sham control surgery made choices in two decision-making tasks. In the delay discounting task, rats chose between two visual stimuli, one of which indicated a small, immediate reward, and the other of which indicated a large, delayed reward. In the probability discounting task, two stimuli indicated, instead, a small, certain reward or a large, uncertain reward. The two lesion groups showed a double dissociation with respect to the two tasks. Rats with VHC lesions were intolerant of delay, and were strongly biased towards the small, immediate reward. However, the same rats were indistinguishable from sham controls in the probability discounting task. The opposite pattern was observed for rats with OFC lesions; they performed normally in the delay discounting task, but showed a reduced tolerance for uncertainty as compared with sham-operated controls. These data support the conclusion that the VHC and OFC contribute differentially to decision-making that involves delayed or uncertain outcomes. This provides a means for understanding the neural basis of a range of neurological and psychiatric patients who show impaired decision-making and executive dysfunction.

How costs influence decision values for mixed outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.

Fos expression in the prefrontal cortex and ventral striatum after exposure to a free-operant timing schedule

It has been proposed that cortico-striato-thalamo-cortical circuits that incorporate the prefrontal cortex and corpus striatum regulate interval timing behaviour. In the present experiment regional Fos expression was compared between rats trained under an immediate timing schedule, the free-operant psychophysical procedure (FOPP), which entails temporally regulated switching between two operanda, and a yoked variable-interval (VI) schedule matched to the timing task for food deprivation level, reinforcement rate and overall response rate. The density of Fos-positive neurones (counts mm⁻²) in the orbital prefrontal cortex (OPFC) and the shell of the nucleus accumbens (AcbS) was greater in rats exposed to the FOPP than in rats exposed to the VI schedule, suggesting a greater activation of these areas during the performance of the former task. The enhancement of Fos expression in the OPFC is consistent with previous findings with both immediate and retrospective timing schedules. Enhanced Fos expression in the AcbS was previously found in retrospective timing schedules based on conditional discrimination tasks, but not in a single-operandum immediate timing schedule, the fixed-interval peak procedure. It is suggested that the ventral striatum may be engaged during performance on timing schedules that entail operant choice, irrespective of whether they belong to the immediate or retrospective categories.

Fos expression in the orbital prefrontal cortex after exposure to the fixed-interval peak procedure

It has been proposed that cortico-striato-thalamo-cortical circuits that incorporate the prefrontal cortex and dorsal striatum regulate interval timing behaviour. The present experiment examined whether performance on the fixed-interval peak procedure (FIPP), an immediate timing schedule, would induce neuronal activity in cortical and striatal areas, as revealed by enhanced expression of the Fos protein, a marker for neuronal activation. Regional Fos expression was compared between rats trained on the FIPP and rats trained on a variable-interval (VI) schedule matched to the FIPP for overall response rate and reinforcer delivery. Response rate in the peak trials of the FIPP conformed to a temporally differentiated pattern, which was well described by a modified Gaussian function; in agreement with previous findings, the peak time occurred close to the time at which the reinforcer was delivered in the fixed-interval trials, and the Weber fraction was within the range of values reported previously. The density of Fos-positive neurones (counts mm⁻²) in the orbital prefrontal cortex (OPFC) was greater in rats exposed to the FIPP than in rats exposed to the VI schedule, suggesting a greater activation of this area during the performance of the former task. This is consistent with the results of previous studies that have implicated the OPFC in interval timing behaviour. However, there was no significant difference between the levels of Fos expression in the dorsal or ventral striatum of the rats trained under the two schedules.

Nucleus accumbens and delay discounting in rats: evidence from a new quantitative protocol for analysing inter-temporal choice

RATIONALE

There is evidence that the core of the nucleus accumbens (AcbC) is involved in inter-temporal choice behaviour.

OBJECTIVE

A new behavioural protocol was used to examine the effect of destruction of the AcbC on delay discounting in inter-temporal choice schedules in rats.

METHOD

Rats with excitotoxic lesions of the AcbC or sham lesions made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately and a larger reinforcer (B) that was delivered after a delay which increased or decreased depending on the subject's choices. In two phases of the experiment, reinforcer sizes were selected which enabled theoretical parameters expressing delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays (i.e. the quasi-stable values of the adjusting delay seen after extended training) obtained in the two phases.

RESULTS

In both groups, indifference delays were shorter when the sizes of A and B were 14 and 25 μl than when they were 25 and 100 μl of a 0.6 M sucrose solution. Indifference delays were shorter in AcbC-lesioned than in sham-lesioned rats. Estimates of delay discounting rate based on the ratio of the indifference delays were lower in the AcbC-lesioned than in the sham-lesioned rats. The size sensitivity parameter did not differ between the groups. Adjusting delays in successive blocks of trials were analysed using Fourier transform. The period corresponding to the dominant frequency of the power spectrum and power within the dominant frequency band did not differ between the groups.

CONCLUSIONS

Destruction of the AcbC increased the rate of delay discounting.

Effects of selective dopaminergic compounds on a delay-discounting task

Impulsivity is widely regarded as a multidimensional trait that encompasses two or more distinct patterns of behavior, and dopaminergic systems are implicated in the expression of impulsive behavior in both humans and animal subjects. Impulsive choice, or the tendency to choose rewards associated with relatively little or no delay, has been extensively studied in humans and animal subjects using delay-discounting tasks. Here, delay-discounting procedures were used to assess the effects of receptor-selective dopaminergic agonists, antagonists, and dopamine transporter ligands on choices of immediate versus delayed sucrose pellets. The effects of d-amphetamine, GBR 12909, apomorphine, SKF 81297, sumanirole, pramipexole, ABT-724, SCH 23390, L-741,626, PG01037, and L-745,870 were assessed in 24 Sprague-Dawley rats. The only drugs to affect impulsive choice selectively without altering undelayed choice were the D1-like antagonist, SCH 23390 (0.01 mg/kg), and the D4 partial agonist, ABT-724 (3.2 mg/kg), which both increased impulsive choice. The shared effects of these compounds may be explained by their localization within the prefrontal cortex on different groups of neurons. None of the selective agonists and antagonists tested reduced impulsive choice, so further research is needed to determine if direct dopaminergic agonists or antagonists may be therapeutically useful in the treatment of impulse-control disorders.

Acute nicotine increases both impulsive choice and behavioural disinhibition in rats

RATIONALE

Heavy smokers exhibit greater levels of impulsive choice and behavioural disinhibition than non-smokers. To date, however, the relationship between nicotine use and differing dimensions of impulsivity has not been systematically assessed.

OBJECTIVES

A series of studies was designed to assess the acute dose-response effects of nicotine and the nicotinic receptor antagonist mecamylamine alone, and in combination with nicotine, on impulsive choice and behavioural disinhibition in rats.

METHODS

Separate groups of rats were trained on a symmetrically reinforced go/no-go task to measure levels of disinhibition and a systematic delayed reward task to measure levels of impulsive choice. Once trained, all animals in each task were treated acutely with nicotine (0.125, 0.25, 0.5 and 1.0 mg/kg), mecamylamine (0.1, 0.3 and 1.0 mg/kg) and varying doses of mecamylamine (0.1, 0.3 and 1.0 mg/kg) prior to nicotine (0.5 mg/kg). An additional experiment assessed the effects of alterations in primary motivation (presatiation and fasting) on performance in both tasks.

RESULTS

Acute nicotine increased both impulsive choice and behavioural disinhibition, effects that were blocked by pre-treatment with mecamylamine. Mecamylamine when administered alone did not alter impulsive behaviour. The lack of effect of presatiation on performance measures suggests that the observed nicotine-induced impulsivity cannot be attributed to the anorectic activity of the compound.

CONCLUSIONS

Present findings support the hypothesis that heightened impulsivity in smokers may in part be a consequence of the direct acute effects of nicotine. As such, drug-induced changes in impulsivity may play a critical role in the transition to and maintenance of nicotine dependence.

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 µl of a 0.6 M sucrose solution than when they were 25 and 100 µl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 µl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.

Dissociable effects of lesions to orbitofrontal cortex subregions on impulsive choice in the rat

The orbitofrontal cortex (OFC) is implicated in a variety of adaptive decision-making processes. Human studies suggest that there is a functional dissociation between medial and lateral OFC (mOFC and lOFC, respectively) subregions when performing certain choice procedures. However, little work has examined the functional consequences of manipulations of OFC subregions on decision making in rodents. In the present experiments, impulsive choice was assessed by evaluating intolerance to delayed, but economically optimal, reward options using a delay-discounting paradigm. Following initial delay-discounting training, rats received bilateral neurotoxic or sham lesions targeting whole OFC (wOFC) or restricted to either mOFC or lOFC subregions. A transient flattening of delay-discounting curves was observed in wOFC-lesioned animals relative to shams--differences that disappeared with further training. Stable, dissociable effects were found when lesions were restricted to OFC subregions; mOFC-lesioned rats showed increased, whereas lOFC-lesioned rats showed decreased, preference for the larger-delayed reward relative to sham-controls--a pattern that remained significant during retraining after all delays were removed. When locations of levers leading to small-immediate versus large-delayed rewards were reversed, wOFC- and lOFC-lesioned rats showed retarded, whereas mOFC-lesioned rats showed accelerated, trajectories for reversal of lever preference. These results provide the first direct evidence for dissociable functional roles of the mOFC and lOFC for impulsive choice in rodents. The findings are consistent with recent human functional imaging studies and suggest that functions of mOFC and lOFC subregions may be evolutionarily conserved and contribute differentially to decision-making processes.

Different functions in the cingulate cortex, a meta-analytic connectivity modeling study

The cingulate cortex is a structurally heterogeneous brain region involved in emotional, cognitive and motor tasks. With the aim of identifying which behavioral domains are associated with the activation of the cingulate cortex, we performed a structure based-meta-analysis using the activation likelihood estimation (ALE), which assesses statistical significant convergence of neuroimaging studies using the BrainMap database. To map the meta-analytic coactivation maps of the cingulate cortex (MACM), we subdivided the parenchyma along the rostro-caudal axis in 12 bilateral equispaced ROIs. ROIs were not chosen according to previously suggested subdivisions, as to obtain a completely data-driven result. Studies were included with one or more activation coordinates in at least one of the 12 pre-defined ROIs. The meta-analytic connectivity profile and behavioral domains profiles were identified for each ROI. Cluster analysis was then performed on the MACM and behavioral domains to group together ROIs with similar profiles. The results showed that the cingulate cortex can be divided in three clusters according to the MACM parcellation and in four according to the behavioral domain-based parcellation. In addition, a behavioral-domain based meta-analysis was conducted and the spatial consistency of functional connectivity patterns across different domain-related ALE results was evaluated by computing probabilistic maps. These maps identified some portions of the cingulate cortex as involved in several tasks. Our results showed the existence of a more specific functional characterization of some portions of the cingulate cortex but also a great multifunctionality of others. By analyzing a large number of studies, structure based meta-analysis can greatly contribute to new insights in the functional significance of brain activations and in the role of specific brain areas in behavior.

Nucleus accumbens neurons encode predicted and ongoing reward costs in rats

Efficient decision-making requires that animals consider both the benefits and the costs of potential actions, such as the amount of effort or temporal delay involved in reward seeking. The nucleus accumbens (NAc) has been implicated in the ability to choose between options with different costs and overcome high costs when necessary, but it is not clear how NAc processing contributes to this role. Here, neuronal activity in the rat NAc was monitored using multi-neuron electrophysiology during two cost-based decision tasks in which either reward effort or reward delay was manipulated. In each task, distinct visual cues predicted high-value (low effort/immediate) and low-value (high effort/delayed) rewards. After training, animals exhibited a behavioral preference for high-value rewards, yet overcame high costs when necessary to obtain rewards. Electrophysiological analysis indicated that a subgroup of NAc neurons exhibited phasic increases in firing rate during cue presentations. In the effort-based decision task (but not the delay-based task), this population reflected the cost-discounted value of the future response. In contrast, other subgroups of cells were activated during response initiation or reward delivery, but activity did not differ on the basis of reward cost. Finally, another population of cells exhibited sustained changes in firing rate while animals completed high-effort requirements or waited for delayed rewards. These findings are consistent with previous reports that implicate NAc function in reward prediction and behavioral allocation during reward-seeking behavior, and suggest a mechanism by which NAc activity contributes to both cost-based decisions and actual cost expenditure.

Selective ablations reveal that orbital and lateral prefrontal cortex play different roles in estimating predicted reward value

Subregions of prefrontal cortex are important for estimating reward values and using these values to guide behavior. The present studies directly tested whether orbital prefrontal cortex (O-PFC) and lateral prefrontal cortex (L-PFC) are necessary for evaluating trial-to-trial changes in the reward values predicted by visual cues. We have compared intact rhesus monkeys, those with bilateral O-PFC lesions (n = 3), and those with bilateral L-PFC lesions (n = 3). We used three versions of a visually cued color discrimination task: we varied reward size, delay to reward, or both. O-PFC lesions altered estimations of predicted reward value in all versions of the task. L-PFC lesions disrupted performance only when both reward size and delay to reward were varied together. Neither lesion directly affected basic internal drive states (satiation curves). Our results suggest that O-PFC is essential for establishing independent, context-specific scales with which predicted reward values are measured. L-PFC appears necessary for integration of predicted reward value across these different scales.

Choice between reinforcer delays versus choice between reinforcer magnitudes: differential Fos expression in the orbital prefrontal cortex and nucleus accumbens core

Lesions of the orbital prefrontal cortex (OPFC) and the nucleus accumbens core (AcbC) can disrupt performance in inter-temporal choice tasks, possibly by increasing the organism's sensitivity to delay and/or magnitude of reinforcement. This experiment examined whether exposure to an inter-temporal choice would induce neuronal activation in these areas, as indicated by enhanced expression of the Fos protein. Twelve rats were trained to press levers A and B under an adjusting-delay schedule in which a response on A delivered 50 microl of a sucrose reinforcer after 2 or 18s, whereas a response on B delivered the same reinforcer after a delay that was adjusted in accordance with the rat's choices. Another 12 rats were trained under a similar schedule in which a response on A delivered an immediate reinforcer of size 20 or 180 microl, whereas a response on B delivered an immediate reinforcer whose size was adjusted in accordance with the rat's choices. A third group received training under a schedule that did not entail variation of reinforcer size or delay, or choice between reinforcers, and a control group underwent food restriction without behavioural training. Exposure to the adjusting-delay schedule was associated with enhanced Fos expression in both the OPFC and AcbC, whereas exposure to the adjusting-magnitude schedule was associated with enhanced Fos expression in the OPFC but not the AcbC, compared to the control group. The results are consistent with previous findings that implicated the AcbC and OPFC in delay discounting, and the OPFC in sensitivity to reinforcer size.

Contributions of the orbitofrontal cortex to impulsive choice: interactions with basal levels of impulsivity, dopamine signalling, and reward-related cues

RATIONALE

Individual differences in impulsive decision-making may be critical determinants of vulnerability to impulse control disorders and substance abuse, yet little is known of their biological or behavioural basis. The orbitofrontal cortex (OFC) has been heavily implicated in the regulation of impulsive decision-making. However, lesions of the OFC in rats have both increased and decreased impulsivity in delay-discounting paradigms, where impulsive choice is defined as the selection of small immediate over larger delayed rewards.

OBJECTIVES

Reviewing the different methods used, we hypothesized that the effects of OFC inactivation on delay discounting may be critically affected by both subjects' baseline level of impulsive choice and the presence or absence of a cue to bridge the delay between selection and delivery of the large reward.

RESULTS

Here, we show that OFC inactivation increased impulsive choice in less impulsive rats when the delay was cued, but decreased impulsive choice in highly impulsive rats in an uncued condition.

CONCLUSIONS

Providing explicit environmental cues to signal the delay-to-reinforcement appears to change the way in which the OFC is recruited in the decision-making process in a baseline-dependent fashion. This change may reflect activation of the dopamine system, as intra-OFC infusions of dopamine receptor antagonists increased impulsive choice but only when the delay was cued.

Interactions between orbital prefrontal cortex and amygdala: advanced cognition, learned responses and instinctive behaviors

Recent research indicates that the orbital prefrontal cortex (PFo) represents stimulus valuations and that the amygdala updates these valuations. An exploration of how PFo and the amygdala interact could improve the understanding of both. PFo and the amygdala function cooperatively when monkeys choose objects associated with recently revalued foods. In other tasks, they function in opposition. PFo uses positive feedback to promote learning in object-reward reversal tasks, and PFo also promotes extinction learning. Amygdala function interferes with both kinds of learning. The amygdala underlies fearful responses to a rubber snake from the first exposure on, but PFo is necessary only after the initial exposure. The amygdala mediates an arousal response in anticipation of rewards, whereas PFo sometimes suppresses such arousal. A role for PFo in advanced cognition, for the amygdala in instinctive behavior, and for cortex-subcortex interactions in prioritizing behaviors provides one account for these findings.

Interactions between the prefrontal cortex and amygdala during delay discounting and reversal

Interactions between the prefrontal cortex and amygdala are thought to be critical for reward anticipation. Alterations in reward anticipation that lead to an inability to wait for rewards or a diminished capacity to change behavior when doing so would be optimal are often termed impulsivity and compulsivity, respectively. Distinct regions of the prefrontal cortex may support decreased impulsivity through self-control and decreased compulsivity through flexibility. However, both self-control and flexibility appear to involve the amygdala. Using a delay discounting paradigm, the current investigation found that inactivation and disconnection of the medial prefrontal cortex and basolateral amygdala led rats to become more impulsive by affecting preference for smaller immediate over larger delayed rewards. Conversely, inactivation and disconnection of the orbitofrontal cortex and amygdala led rats to become more compulsive as demonstrated by an inability to flexibly reverse stimulus-reward relationships in an odor reversal task. The current findings support a double dissociation between orbitofrontal cortex-amygdala interactions for odor reversal and medial prefrontal cortex-amygdala interactions for delay discounting.

Isolating behavioral mechanisms of intertemporal choice: nicotine effects on delay discounting and amount sensitivity

Many drugs of abuse produce changes in impulsive choice, that is, choice for a smaller-sooner reinforcer over a larger-later reinforcer. Because the alternatives differ in both delay and amount, it is not clear whether these drug effects are due to the differences in reinforcer delay or amount. To isolate the effects of delay, we used a titrating delay procedure. In phase 1, 9 rats made discrete choices between variable delays (1 or 19 s, equal probability of each) and a delay to a single food pellet. The computer titrated the delay to a single food pellet until the rats were indifferent between the two options. This indifference delay was used as the starting value for the titrating delay for all future sessions. We next evaluated the acute effects of nicotine (subcutaneous 1.0, 0.3, 0.1, and 0.03 mg/kg) on choice. If nicotine increases delay discounting, it should have increased preference for the variable delay. Instead, nicotine had very little effect on choice. In a second phase, the titrated delay alternative produced three food pellets instead of one, which was again produced by the variable delay (1 s or 19 s) alternative. Under this procedure, nicotine increased preference for the one pellet alternative. Nicotine-induced changes in impulsive choice are therefore likely due to differences in reinforcer amount rather than differences in reinforcer delay. In addition, it may be necessary to include an amount sensitivity parameter in any mathematical model of choice when the alternatives differ in reinforcer amount.

Structure-function relationships in the processing of regret in the orbitofrontal cortex

The influence of counterfactual thinking and regret on choice behavior has been widely acknowledged in economic science (Bell in Oper Res 30:961-981, 1982; Kahneman and Tversky in Judgment under uncertainty: heuristics and biases. Cambridge University Press, Cambridge, pp 201-210, 1982; Loomes and Sugden in Econ J 92:805-824, 1982). Neuroimaging studies have only recently begun to explore the neural correlates of this psychological factor and orbitofrontal cortex (OFC) activity was observed in several of them depending of the exact characteristics of the employed paradigm. This selective OFC involvement and, moreover, a consistently found dissociation of medial and lateral OFC activity clusters allow inferences to the function of this structure in counterfactual thinking and regret. Vice versa, the differential contribution of OFC subregions to these processes also adds evidence to the current debate on the function of this cortical structure in decision-making that attracted increasing attention in recent years.

Quantitative analysis of the effect of lesions of the subthalamic nucleus on intertemporal choice: further evidence for enhancement of the incentive value of food reinforcers

Recent evidence suggests that the subthalamic nucleus (STN) is involved in regulating the incentive value of food reinforcers. The objective of this study was to examine the effect of lesions of the STN on intertemporal choice (choice between reinforcers differing in size and delay). Rats with bilateral quinolinic acid-induced lesions of the STN (n = 15) or sham lesions (n = 14) were trained in a discrete-trials progressive delay schedule to press levers A and B for a sucrose solution. Responses on A delivered 50 microl of the solution after a delay d(A); responses on B delivered 100 microl after a delay d(B). d(B) increased across blocks of trials; d(A) was manipulated across phases of the experiment. Indifference delay, d(B(50)) (value of d(B) corresponding to 50% choice of B), was estimated for each rat in each phase, and linear indifference functions (d(B(50)) vs. d(A)) were derived. The STN-lesioned group showed a flatter slope of the indifference function (implying higher instantaneous reinforcer values) than the sham-lesioned group; the intercepts did not differ between the groups. The results agree with recent evidence for a role of the STN in incentive value. Unlike some earlier studies, these results do not indicate a role of the STN in delay discounting.

Effects of chronic administration of drugs of abuse on impulsive choice (delay discounting) in animal models

Drug-addicted individuals show high levels of impulsive choice, characterized by preference for small immediate over larger but delayed rewards. Although the causal relationship between chronic drug use and elevated impulsive choice in humans has been unclear, a small but growing body of literature over the past decade has shown that chronic drug administration in animal models can cause increases in impulsive choice, suggesting that a similar causal relationship may exist in human drug users. This article reviews this literature, with a particular focus on the effects of chronic cocaine administration, which have been most thoroughly characterized. The potential mechanisms of these effects are described in terms of drug-induced neural alterations in ventral striatal and prefrontal cortical brain systems. Some implications of this research for pharmacological treatment of drug-induced increases in impulsive choice are discussed, along with suggestions for future research in this area.