Rhesus monkeys with orbital prefrontal cortex lesions can learn to inhibit prepotent responses in the reversed reward contingency task

Optogenetic Inhibition of the Orbitofrontal Cortex Disrupts Inhibitory Control during Stop-Change Performance in Male Rats

Historically, the orbitofrontal cortex (OFC) has been implicated in a variety of behaviors ranging from reversal learning and inhibitory control to more complex representations of reward value and task space. While modern interpretations of the OFC's function have focused on a role in outcome evaluation, these cognitive processes often require an organism to inhibit a maladaptive response or strategy. Single-unit recordings from the OFC in rats performing a stop-change task show that the OFC responds strongly to STOP trials. To investigate the role that the OFC plays in stop-change performance, we expressed halorhodopsin (eNpHR3.0) in excitatory neurons in the OFC and tested rats on the stop-change task. Previous work suggests that the OFC differentiates between STOP trials based on trial sequence (i.e., gS trials: STOP trials preceded by a GO vs sS trials: STOP trials preceded by a STOP). We found that yellow light activation of the eNpHR3.0-expressing neurons significantly decreased accuracy only on STOP trials that followed GO trials (gS trials). Further, optogenetic inhibition of the OFC speeded reaction times on error trials. This suggests that the OFC plays a role in inhibitory control processes and that this role needs to be accounted for in modern interpretations of OFC function.

Environmental enrichment improves cognitive flexibility in rainbow trout in a visual discrimination task: first insights

Research on fish cognition provides strong evidence that fish are endowed with high level cognitive skills. However, most studies on cognitive flexibility and generalization abilities, two key adaptive traits for captive animals, focused on model species, and farmed fish received too little attention. Environmental enrichment was shown to improve learning abilities in various fish species, but its influence on cognitive flexibility and generalization abilities is still unknown. We studied farmed rainbow trout (Oncorhynchus mykiss) as an aquaculture model to study how environmental enrichment impacts their cognitive abilities. Using an operant conditioning device, allowing the expression of a motivated choice, we measured fish cognitive flexibility with serial reversal learning tests, after a successful acquisition phase based on two colors discrimination (2-alternative forced choice, 2-AFC), and their ability to generalize a rewarded color to any shape. Eight fish were divided into two groups: Condition E (fish reared from fry stages under enriched conditions with plants, rocks and pipes for ~9 months); Condition B (standard barren conditions). Only one fish (condition E) failed in the habituation phase of the device and one fish (condition B) failed in the 2-AFC task. We showed that after a successful acquisition phase in which the fish correctly discriminated two colors, they all succeeded in four reversal learnings, supporting evidence for cognitive flexibility in rainbow trout. They were all successful in the generalization task. Interestingly, fish reared in an enriched environment performed better in the acquisition phase and in the reversal learning (as evidenced by fewer trials needed to reach the learning criterion), but not in the generalization task. We assume that color-based generalization may be a simpler cognitive process than discriminative learning and cognitive flexibility, and does not seem to be influenced by environmental conditions. Given the small number of individuals tested, our results may be considered as first insights into cognitive flexibility in farmed fish using an operant conditioning device, but they pave the way for future studies. We conclude that farming conditions should take into account the cognitive abilities of fish, in particular their cognitive flexibility, by allowing them to live in an enriched environment.

Different methods of fear reduction are supported by distinct cortical substrates

Understanding how learned fear can be reduced is at the heart of treatments for anxiety disorders. Tremendous progress has been made in this regard through extinction training in which the aversive outcome is omitted. However, current progress almost entirely rests on this single paradigm, resulting in a very specialized knowledgebase at the behavioural and neural level of analysis. Here, we used a dual-paradigm approach to show that different methods that lead to reduction in learned fear in rats are dissociated in the cortex. We report that the infralimbic cortex has a very specific role in fear reduction that depends on the omission of aversive events but not on overexpectation. The orbitofrontal cortex, a structure generally overlooked in fear, is critical for downregulating fear when novel predictions about upcoming aversive events are generated, such as when fear is inflated or overexpected, but less so when an expected aversive event is omitted.

Activity in orbitofrontal neuronal ensembles reflects inhibitory control

Stopping, or inhibition, is a form of self-control that is a core element of flexible and adaptive behavior. Its neural origins remain unclear. Some views hold that inhibition decisions reflect the aggregation of widespread and diverse pieces of information, including information arising in ostensible core reward regions (i.e., outside the canonical executive system). We recorded activity of single neurons in the orbitofrontal cortex (OFC) of macaques, a region associated with economic decisions, and whose role in inhibition is debated. Subjects performed a classic inhibition task known as the stop signal task. Ensemble decoding analyses reveal a clear firing rate pattern that distinguishes successful from failed inhibition and that begins after the stop signal and before the stop signal reaction time (SSRT). We also found a different and orthogonal ensemble pattern that distinguishes successful from failed stopping before the beginning of the trial. These signals were distinct from, and orthogonal to, value encoding, which was also observed in these neurons. The timing of the early and late signals was, respectively, consistent with the idea that neuronal activity in OFC encodes inhibition both proactively and reactively.

Specializations for reward-guided decision-making in the primate ventral prefrontal cortex

The estimated values of choices, and therefore decision-making based on those values, are influenced by both the chance that the chosen items or goods can be obtained (availability) and their current worth (desirability) as well as by the ability to link the estimated values to choices (a process sometimes called credit assignment). In primates, the prefrontal cortex (PFC) has been thought to contribute to each of these processes; however, causal relationships between particular subdivisions of the PFC and specific functions have been difficult to establish. Recent lesion-based research studies have defined the roles of two different parts of the primate PFC - the orbitofrontal cortex (OFC) and the ventral lateral frontal cortex (VLFC) - and their subdivisions in evaluating each of these factors and in mediating credit assignment during reward-based decision-making.

Why has evolution not selected for perfect self-control?

Self-control refers to the ability to deliberately reject tempting options and instead select ones that produce greater long-term benefits. Although some apparent failures of self-control are, on closer inspection, reward maximizing, at least some self-control failures are clearly disadvantageous and non-strategic. The existence of poor self-control presents an important evolutionary puzzle because there is no obvious reason why good self-control should be more costly than poor self-control. After all, a rock is infinitely patient. I propose that self-control failures result from cases in which well-learned (and thus routinized) decision-making strategies yield suboptimal choices. These mappings persist in the decision-makers' repertoire because they result from learning processes that are adaptive in the broader context, either on the timescale of learning or of evolution. Self-control, then, is a form of cognitive control and the subjective feeling of effort likely reflects the true costs of cognitive control. Poor self-control, in this view, is ultimately a result of bounded optimality. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications.

Functional heterogeneity within the rodent lateral orbitofrontal cortex dissociates outcome devaluation and reversal learning deficits

The orbitofrontal cortex (OFC) is critical for updating reward-directed behaviours flexibly when outcomes are devalued or when task contingencies are reversed. Failure to update behaviour in outcome devaluation and reversal learning procedures are considered canonical deficits following OFC lesions in non-human primates and rodents. We examined the generality of these findings in rodents using lesions of the rodent lateral OFC (LO) in instrumental action-outcome and Pavlovian cue-outcome devaluation procedures. LO lesions disrupted outcome devaluation in Pavlovian but not instrumental procedures. Furthermore, although both anterior and posterior LO lesions disrupted Pavlovian outcome devaluation, only posterior LO lesions were found to disrupt reversal learning. Posterior but not anterior LO lesions were also found to disrupt the attribution of motivational value to Pavlovian cues in sign-tracking. These novel dissociable task- and subregion-specific effects suggest a way to reconcile contradictory findings between rodent and non-human primate OFC research.

Contributions of Lateral and Orbital Frontal Regions to Abstract Rule Acquisition and Reversal in Monkeys

The ability to learn and follow abstract rules relies on intact prefrontal regions including the lateral prefrontal cortex (LPFC) and the orbitofrontal cortex (OFC). Here, we investigate the specific roles of these brain regions in learning rules that depend critically on the formation of abstract concepts as opposed to simpler input-output associations. To this aim, we tested monkeys with bilateral removals of either LPFC or OFC on a rapidly learned task requiring the formation of the abstract concept of same vs. different. While monkeys with OFC removals were significantly slower than controls at both acquiring and reversing the concept-based rule, monkeys with LPFC removals were not impaired in acquiring the task, but were significantly slower at rule reversal. Neither group was impaired in the acquisition or reversal of a delayed visual cue-outcome association task without a concept-based rule. These results suggest that OFC is essential for the implementation of a concept-based rule, whereas LPFC seems essential for its modification once established.

Abstraction promotes creative problem-solving in rhesus monkeys

Abstraction allows us to discern regularities beyond the specific instances we encounter. It also promotes creative problem-solving by enabling us to consider unconventional problem solutions. However, the mechanisms by which this occurs are not well understood. Because it is often difficult to isolate human high-level cognitive processes, we utilized a nonhuman primate model, in which rhesus monkeys appear to use similar processes to consider an unconventional solution to the difficult reverse-reward problem: i.e., given the choice between a better and worse food option they must select the worse one to receive the better one. After solving this problem with only one specific example-one vs. four half-peanuts-three of four monkeys immediately transferred to novel cases: novel quantities, food items, non-food items, and to the choice between a larger, but inferior vegetable and a smaller, but superior food item (either grape or marshmallow), in which they selected the inferior vegetable to receive the superior option. Thus, we show that nonhuman animals have the capacity to comprehend abstract non-perceptual features, to infer them from one specific case, and to use them to override the natural preference to select the superior option. Critically, we also found that three monkeys had a large learning and performance advantage over the fourth monkey who showed less generalization from the original one and four half-peanuts. This difference suggests that abstraction promoted problem-solving via cascading activation from the two food item options to the relation between them, thus providing access to an initially nonapparent problem solution.

Differential effects of real versus hypothetical monetary reward magnitude on risk-taking behavior and brain activity

Human decisions are more easily affected by a larger amount of money than a smaller one. Although numerous studies have used hypothetical money as incentives to motivate human behavior, the validity of hypothetical versus real monetary rewards remains controversial. In the present study, we used event-related potential (ERP) with the balloon analogue risk task to investigate how magnitudes of real and hypothetical monetary rewards modulate risk-taking behavior and feedback-related negativity (FRN). Behavioral data showed that participants were more risk averse after negative feedback with increased magnitude of real monetary rewards, while no behavior differences were observed between large and small hypothetical monetary rewards. Similarly, ERP data showed a larger FRN in response to negative feedback during risk taking with large compared to small real monetary rewards, while no FRN differences were observed between large and small hypothetical monetary rewards. Moreover, FRN amplitude differences correlated with risk-taking behavior changes from small to large real monetary rewards, while such correlation was not observed for hypothetical monetary rewards. These findings suggest that the magnitudes of real and hypothetical monetary rewards have differential effects on risk-taking behavior and brain activity. Real and hypothetical money incentives may have different validity for modulating human decisions.

Lucky Rhythms in Orbitofrontal Cortex Bias Gambling Decisions in Humans

It is well established that emotions influence our decisions, yet the neural basis of this biasing effect is not well understood. Here we directly recorded local field potentials from the OrbitoFrontal Cortex (OFC) in five human subjects performing a financial decision-making task. We observed a striking increase in gamma-band (36-50 Hz) oscillatory activity that reflected subjects' decisions to make riskier choices. Additionally, these gamma rhythms were linked back to mismatched expectations or "luck" occurring in past trials. Specifically, when a subject expected to win but lost, the trial was defined as "unlucky" and when the subject expected to lose but won, the trial was defined as "lucky". Finally, a fading memory model of luck correlated to an objective measure of emotion, heart rate variability. Our findings suggest OFC may play a pivotal role in processing a subject's internal (emotional) state during financial decision-making, a particularly interesting result in light of the more recent "cognitive map" theory of OFC function.

Effects of stress on behavioral flexibility in rodents

Cognitive flexibility is the ability to switch between different rules or concepts and behavioral flexibility is the overt physical manifestation of these shifts. Behavioral flexibility is essential for adaptive responses and commonly measured by reversal learning and set-shifting performance in rodents. Both tasks have demonstrated vulnerability to stress with effects dependent upon stressor type and number of repetitions. This review compares the effects of stress on reversal learning and set-shifting to provide insight into the differential effect of stress on cognition. Acute and short-term repetition of stress appears to facilitate reversal learning whereas the longer term repetition of stress impairs reversal learning. Stress facilitated intradimensional set-shifting within a single, short-term stress protocol but otherwise generally impaired set-shifting performance in acute and repeated stress paradigms. Chronic unpredictable stress impairs reversal learning and set-shifting whereas repeated cold intermittent stress selectively impairs reversal learning and has no effect on set-shifting. In considering the mechanisms underlying the effects of stress on behavioral flexibility, pharmacological manipulations performed in conjunction with stress are also reviewed. Blocking corticosterone receptors does not affect the facilitation of reversal learning following acute stress but the prevention of corticosterone synthesis rescues repeated stress-induced set-shifting impairment. Enhancing post-synaptic norepinephrine function, serotonin availability, and dopamine receptor activation rescues and/or prevents behavioral flexibility performance following stress. While this review highlights a lack of a standardization of stress paradigms, some consistent effects are apparent. Future studies are necessary to specify the mechanisms underlying the stress-induced impairments of behavioral flexibility, which will aid in alleviating these symptoms in patients with some psychiatric disorders.

The neural basis of reversal learning: An updated perspective

Reversal learning paradigms are among the most widely used tests of cognitive flexibility and have been used as assays, across species, for altered cognitive processes in a host of neuropsychiatric conditions. Based on recent studies in humans, non-human primates, and rodents, the notion that reversal learning tasks primarily measure response inhibition, has been revised. In this review, we describe how cognitive flexibility is measured by reversal learning and discuss new definitions of the construct validity of the task that are serving as a heuristic to guide future research in this field. We also provide an update on the available evidence implicating certain cortical and subcortical brain regions in the mediation of reversal learning, and an overview of the principal neurotransmitter systems involved.

What the orbitofrontal cortex does not do

The number of papers about the orbitofrontal cortex (OFC) has grown from 1 per month in 1987 to a current rate of over 50 per month. This publication stream has implicated the OFC in nearly every function known to cognitive neuroscience and in most neuropsychiatric diseases. However, new ideas about OFC function are typically based on limited data sets and often ignore or minimize competing ideas or contradictory findings. Yet true progress in our understanding of an area's function comes as much from invalidating existing ideas as proposing new ones. Here we consider the proposed roles for OFC, critically examining the level of support for these claims and highlighting the data that call them into question.

Executive control signals in orbitofrontal cortex during response inhibition

Orbitofrontal cortex (OFC) lesions produce deficits in response inhibition and imaging studies suggest that activity in OFC is stronger on trials that require suppression of behavior, yet few studies have examined neural correlates at the single-unit level in a behavioral task that probes response inhibition without varying other factors, such as anticipated outcomes. Here we recorded from single neurons in lateral OFC in a task that required animals in the minority of trials to STOP or inhibit an ongoing movement and respond in the opposite direction. We found that population and single-unit firing was modulated primarily by response direction and movement speed, and that very few OFC neurons exhibited a response independent inhibition signal. Remarkably, the strength of the directional signal was not diminished on STOP trials and was actually stronger on STOP trials during conflict adaptation. Finally, directional signals were stronger during sessions in which rats had the most difficulty inhibiting behavior. These results suggest that "inhibition" deficits observed with OFC interference studies reflect deficits unrelated to signaling the need to inhibit behavior, but instead support a role for OFC in executive functions related to dissociating between two perceptually similar actions during response conflict.

Functions of gamma-band synchronization in cognition: from single circuits to functional diversity across cortical and subcortical systems

Gamma-band activity (30-90 Hz) and the synchronization of neural activity in the gamma-frequency range have been observed in different cortical and subcortical structures and have been associated with different cognitive functions. However, it is still unknown whether gamma-band synchronization subserves a single universal function or a diversity of functions across the full spectrum of cognitive processes. Here, we address this question reviewing the mechanisms of gamma-band oscillation generation and the functions associated with gamma-band activity across several cortical and subcortical structures. Additionally, we raise a plausible explanation of why gamma rhythms are found so ubiquitously across brain structures. Gamma band activity originates from the interplay between inhibition and excitation. We stress that gamma oscillations, associated with this interplay, originate from basic functional motifs that conferred advantages for low-level system processing and multiple cognitive functions throughout evolution. We illustrate the multifunctionality of gamma-band activity by considering its role in neural systems for perception, selective attention, memory, motivation and behavioral control. We conclude that gamma-band oscillations support multiple cognitive processes, rather than a single one, which, however, can be traced back to a limited set of circuit motifs which are found universally across species and brain structures.

Neurophysiology of performance monitoring and adaptive behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of one's actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

Mechanisms of reward circuit dysfunction in psychiatric illness: prefrontal-striatal interactions

The brain's "reward circuit" has been widely implicated in the pathophysiology of mental illness. Although there has been significant progress in identifying the functional characteristics of individual nodes within the circuit and linking dysfunction of these brain areas to various forms of psychopathology, there remains a substantial gap in understanding how the nodes of the circuit interact with one another, and how the growing neurobiological knowledge may be applied to improve psychiatric patient care. In this article, we summarize what is currently known about the functions and interactions of two key nodes of this circuit-the ventral striatum and the ventromedial prefrontal/orbital frontal cortex-in relation to mental illness.

Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies

Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum. Our simulations are consistent with the view that the amygdala maintains Pavlovian associations through incremental updating of synaptic strength and that the OFC supports flexibility by maintaining an activation-based working memory of the recent reward history. Our model provides a mechanistic explanation for electrophysiological evidence that cue-related firing in OFC neurons is nonselectively early after a contingency change and why this nonselective firing is critical for promoting plasticity in the amygdala. This ambiguous activation results from the simultaneous maintenance of recent outcomes and obsolete Pavlovian contingencies in working memory. Furthermore, at the beginning of reversal, the OFC is critical for supporting responses that are no longer inappropriate. This result is inconsistent with an exclusive inhibitory account of OFC function.

Tokens improve capuchin performance in the reverse-reward contingency task

In humans and apes, one of the most adaptive functions of symbols is to inhibit strong behavioural predispositions. However, to our knowledge, no study has yet investigated whether using symbols provides some advantage to non-ape primates. We aimed to trace the evolutionary roots of symbolic competence by examining whether tokens improve performance in the reverse-reward contingency task in capuchin monkeys, which diverged from the human lineage approximately 35 Ma. Eight capuchins chose between: (i) two food quantities, (ii) two quantities of 'low-symbolic distance tokens' (each corresponding to one unit of food), and (iii) two 'high-symbolic distance tokens' (each corresponding to a different amount of food). In all conditions, subjects had to select the smaller quantity to obtain the larger reward. No procedural modifications were employed. Tokens did improve performance: five subjects succeeded with high-symbolic distance tokens, though only one succeeded with food, and none succeeded with low-symbolic distance tokens. Moreover, two of the five subjects transferred the rule to novel token combinations. Learning effects or preference reversals could not account for the successful performance with high-symbolic distance tokens. This is, to our knowledge, the first demonstration that tokens do allow monkeys to inhibit strong behavioural predispositions, as occurs in chimpanzees and children.

Learning-associated gamma-band phase-locking of action-outcome selective neurons in orbitofrontal cortex

Gamma oscillations (30-100 Hz) correlate to a variety of neural functions, including sensory processing, attention, and action selection. However, they have barely been studied in relation to emotional processing and valuation of sensory signals and actions. We conducted multineuron and local field potential recordings in the orbitofrontal cortex (OFC) of rats performing a task in which they made go or no-go decisions based on two olfactory stimuli predicting appetitive or aversive outcomes. Gamma power was strongest during the late phase of odor sampling, just before go/no-go movement, and increased with behavioral learning. Learning speed was correlated to the slope of the gamma power increment. Spikes of OFC neurons were consistently timed to the gamma rhythm during odor sampling, regardless of the associated outcome. However, only a specific subgroup of cells showed consistent phase timing. These cells showed action-outcome selective activity, not during stimulus sampling but during subsequent movement responses. During sampling, this subgroup displayed a suppression in firing rate but a concurrent increment in the consistency of spike timing relative to gamma oscillations. In addition to gamma rhythm, OFC field potentials were characterized by theta oscillations during odor sampling. Neurons phase-locked to either theta or gamma rhythms but not to both, suggesting that they become associated with separate rhythmic networks involving OFC. Altogether, these results suggest that OFC gamma-band synchronization reflects inhibitory control over a subpopulation of neurons that express information about the emotional valence of actions after a motor decision, which suggests a novel mechanism for response inhibition.

A new perspective on the role of the orbitofrontal cortex in adaptive behaviour

The orbitofrontal cortex (OFC) is crucial for changing established behaviour in the face of unexpected outcomes. This function has been attributed to the role of the OFC in response inhibition or to the idea that the OFC is a rapidly flexible associative-learning area. However, recent data contradict these accounts, and instead suggest that the OFC is crucial for signalling outcome expectancies. We suggest that this function--signalling of expected outcomes--can also explain the crucial role of the OFC in changing behaviour in the face of unexpected outcomes.

Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with 'disinhibition' of responding for previously unrewarded cues

Orbitofrontal cortex (OFC) is critical for reversal learning. Reversal deficits are typically demonstrated in complex settings that combine Pavlovian and instrumental learning. Yet recent work has implicated the OFC specifically in behaviors guided by cues and the features of the specific outcomes they predict. To test whether the OFC is important for reversing such Pavlovian associations in the absence of confounding instrumental requirements, we trained rats on a simple Pavlovian task in which two auditory cues were presented, one paired with a food pellet reward and the other presented without reward. After learning, we reversed the cue-outcome associations. For half the rats, OFC was inactivated prior to each reversal session. Inactivation of OFC impaired the ability of the rats to reverse conditioned responding. This deficit reflected the inability of inactivated rats to develop normal responding for the previously unrewarded cue; inactivation of OFC had no impact on the ability of the rats to inhibit responding to the previously rewarded cue. These data show that OFC is critical to reversal of Pavlovian responding, and that the role of OFC in this behavior cannot be explained as a simple deficit in response inhibition. Furthermore, the contrast between the normal inhibition of responding, reported here, and impaired inhibition of responding during Pavlovian over-expectation, reported previously, suggests the novel hypothesis that OFC may be particularly critical for learning (or behavior) when it requires the subject to generate predictions about outcomes by bringing together or integrating disparate pieces of associative information.

Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions

Much of our behavior is guided by rules. Although human prefrontal cortex (PFC) and anterior cingulate cortex (ACC) are implicated in implementing rule-guided behavior, the crucial contributions made by different regions within these areas are not yet specified. In an attempt to bridge human neuropsychology and nonhuman primate neurophysiology, we report the effects of circumscribed lesions to macaque orbitofrontal cortex (OFC), principal sulcus (PS), superior dorsolateral PFC, ventrolateral PFC, or ACC sulcus, on separable cognitive components of a Wisconsin Card Sorting Test (WCST) analog. Only the PS lesions impaired maintenance of abstract rules in working memory; only the OFC lesions impaired rapid reward-based updating of representations of rule value; the ACC sulcus lesions impaired active reference to the value of recent choice-outcomes during rule-based decision-making.

The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

Humans and other animals change their behavior in response to unexpected outcomes. The orbitofrontal cortex (OFC) is implicated in such adaptive responding, based on evidence from reversal tasks. Yet these tasks confound using information about expected outcomes with learning when those expectations are violated. OFC is critical for the former function; here we show it is also critical for the latter. In a Pavlovian overexpectation task, inactivation of OFC prevented learning driven by unexpected outcomes, even when performance was assessed later. We propose this reflects a critical contribution of outcome signaling by OFC to encoding of reward prediction errors elsewhere. In accord with this proposal, we report that signaling of reward predictions by OFC neurons was related to signaling of prediction errors by dopamine neurons in ventral tegmental area (VTA). Furthermore, bilateral inactivation of VTA or contralateral inactivation of VTA and OFC disrupted learning driven by unexpected outcomes.

Ventrolateral prefrontal cortex is required for performance of a strategy implementation task but not reinforcer devaluation effects in rhesus monkeys

The ability to apply behavioral strategies to obtain rewards efficiently and make choices based on changes in the value of rewards is fundamental to the adaptive control of behavior. The extent to which different regions of the prefrontal cortex are required for specific kinds of decisions is not well understood. We tested rhesus monkeys with bilateral ablations of the ventrolateral prefrontal cortex on tasks that required the use of behavioral strategies to optimize the rate with which rewards were accumulated, or to modify choice behavior in response to changes in the value of particular rewards. Monkeys with ventrolateral prefrontal lesions were impaired in performing the strategy-based task, but not on value-based decision-making. In contrast, orbital prefrontal ablations produced the opposite impairments in the same tasks. These findings support the conclusion that independent neural systems within the prefrontal cortex are necessary for control of choice behavior based on strategies or on stimulus value.

Selective aspiration or neurotoxic lesions of orbital frontal areas 11 and 13 spared monkeys' performance on the object discrimination reversal task

Damage to the orbital frontal cortex (OFC) has long been associated with reversal learning deficits in several species. In monkeys, this impairment follows lesions that include several OFC subfields. However, the different connectional patterns of OFC subfields together with neuroimaging data in humans have suggested that specific OFC areas play distinctive roles in processing information necessary to guide behavior (Kringelbach and Rolls, 2004; Barbas, 2007; Price, 2007). More specifically, areas 11 and 13 contribute to a sensory network, whereas medial areas 10, 14, and 25 are heavily connected to a visceromotor network. To examine the contribution of areas 11 and 13 to reversal learning, we tested monkeys with selective damage to these two OFC areas on two versions of the ODR task using either one or five discrimination problems. We compared their performance with that of sham-operated controls and of animals with neurotoxic amygdala lesions, which served as operated controls. Neither damage to areas 11 and 13 nor damage to the amygdala affected performance on the ODR tasks. The results indicate that areas 11 and 13 do not critically contribute to reversal learning and that adjacent damage to OFC subfields (10, 12, 14, and 25) could account for the ODR deficits found in earlier lesion studies. This sparing of reversal learning will be discussed in relation to deficits found in the same animals on tasks that measure behavioral modulation when relative value of affective (positive and negative) stimuli was manipulated.

Its own reward: lessons to be drawn from the reversed-reward contingency paradigm

This is a review of the reversed-reward contingency (RRC) paradigm in animals and the cognitive functions on which it is founded. I shall present the RRC basic paradigm and the ensuing modifications it underwent, the animals tested, the results obtained and the analyses offered within the literature. Then I would the claim that RRC is a case of a compound cognitive behavior, one that is the result of interactions between three other cognitive functions: crude numerical assessment and economic choice (uniting value assignment and behavioral inhibition). I will present data concerning these three fields and will demonstrate how they are both affecting and affected by the findings of the RRC scheme. RRC is treated here as a test case for a broader type of analysis, one which, hopefully, will show that in order to fully understand composite and complex behavior we need to meticulously explore its building blocks and their dynamic interplay.

Pre-surgical training ameliorates orbitofrontal-mediated impairments in spatial reversal learning

We recently reported that orbitofrontal cortical (OFC) lesions impaired reversal learning of an instrumental two-lever spatial discrimination task, a deficit manifested as increased perseveration on the pre-potent response. Here we examine whether exposure to reversal learning test pre-operatively may have a beneficial effect for future reversal learning of OFC-lesioned animals. Rats were trained on a novel instrumental two-lever spatial discrimination and reversal learning task, measuring both 'cognitive flexibility' and constituent processes including response inhibition. Both levers were presented, only one of which was reinforced. The rat was required to respond on the reinforced lever under a fixed ratio 3 schedule of reinforcement. Following attainment of criterion, two reversals were introduced. Rats were then matched according to their reversal performance and subjected to bilateral excitotoxic OFC lesions. Following recovery, a series of four reversals was presented. OFC lesions impaired neither retention nor reversal phases. These data, together with the previously reported reversal deficit following OFC lesions, suggest that OFC is not needed when task experience has been gained but it is necessary when task demands are relatively high.

The role of the orbitofrontal cortex and medial striatum in the regulation of prepotent responses to food rewards

An impairment in learning to inhibit prepotent responses to positive stimuli is associated with damage to the orbitofrontal cortex (OFC) in rats, monkeys, and humans performing discrimination reversal, extinction, and detour reaching tasks. In contrast, a recent study showed that OFC-lesioned rhesus monkeys could learn to select the smaller of 2 quantities of food reward in order to receive the larger reward, at an equivalent rate to controls, despite the requirement to inhibit a prepotent response. Given this result, the aim of the present study was to further specify the contexts under which the OFC regulates responding and to identify additional components of limbic circuitry that contribute to such regulation. Marmosets with lesions of the OFC and medial striatum (MS), but not the amygdala, made more prepotent responses to a clear Perspex box containing high incentive food before learning to choose the box containing low incentive food, to obtain reward. However, having learned the incongruent incentive discrimination OFC- and MS-lesioned monkeys were impaired upon reversal of the reward contingencies, repeatedly selecting the previously rewarded low incentive object. These findings identify the critical contribution of the OFC and MS in the regulation of responding by affective cues.

Neural substrates of cognitive inflexibility after chronic cocaine exposure

Cognitive changes in addicts and animals exposed to addictive drugs have been extensively investigated over the past decades. One advantage of studying addiction using cognitive paradigms is that neural processing in addicts or drug-exposed animals can be compared to that in normal subjects. Tests of cognitive flexibility that measure the ability to change responding to a previously rewarded or punished stimulus are of potential interest in the study of addiction, because addiction can itself be viewed as an inability to change responding to stimuli previously associated with drug reward. One such test is reversal learning, which is impaired in cocaine addicts and animals that have chronically self-administered or been exposed to cocaine. A circuit including orbitofrontal cortex, basolateral amygdala and striatum subserves reversal learning. In rats that have been previously exposed to cocaine, neurons in these regions show selective and distinct changes in how they encode information during reversal learning. These changes suggest that in these rats, orbitofrontal cortex loses the ability to signal expected outcomes, and basolateral amygdala becomes inflexible in its encoding of cue significance. These changes could explain cocaine-induced impairments to cognitive flexibility and may have theoretical importance in addiction.

Guidance of instrumental behavior under reversal conditions requires dopamine D1 and D2 receptor activation in the orbitofrontal cortex

The orbitofrontal cortex (OFC) plays a critical role in learning a reversal of stimulus-reward contingencies. Dopamine (DA) neurons probably support reversal learning by emitting prediction error signals that indicate the discrepancy between the actually received reward and its prediction. However, the role of DA receptor-mediated signaling in the OFC to adapt behavior to changing stimulus-reward contingencies is largely unknown. Here we examined the effects of a selective D1 or D2 receptor blockade in the OFC on learning a reversal of previously acquired stimulus-reward magnitude contingencies. Rats were trained on a reaction time (RT) task demanding conditioned lever release with discriminative visual stimuli signaling in advance the upcoming reward magnitude (one or five food pellets). After acquisition, RTs were guided by stimulus-associated reward magnitudes, i.e. RTs of responses were significantly shorter for expected high versus low reward. Thereafter, stimulus-reward magnitude contingencies were reversed and learning was tested under reversal conditions for three blocks after pre-trial infusions of the selective D1 or D2 receptor antagonists R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepinhydrochloride (SCH23390), eticlopride, or vehicle. For comparisons, we included intra-OFC infusions of the selective N-methyl-D-aspartate receptor antagonist AP5. Results revealed that in animals subjected to intra-OFC infusions of SCH23390 or eticlopride learning a reversal of previously acquired stimulus reward-magnitude contingencies was impaired. Thus, in a visual discrimination task as used here, D1 and D2 receptor-mediated signaling in the OFC seems to be necessary to update the reward-predictive significance of stimuli.

Orbital prefrontal cortex is required for object-in-place scene memory but not performance of a strategy implementation task

The orbital prefrontal cortex is thought to be involved in behavioral flexibility in primates, and human neuroimaging studies have identified orbital prefrontal activation during episodic memory encoding. The goal of the present study was to ascertain whether deficits in strategy implementation and episodic memory that occur after ablation of the entire prefrontal cortex can be ascribed to damage to the orbital prefrontal cortex. Rhesus monkeys were preoperatively trained on two behavioral tasks, the performance of both of which is severely impaired by the disconnection of frontal cortex from inferotemporal cortex. In the strategy implementation task, monkeys were required to learn about two categories of objects, each associated with a different strategy that had to be performed to obtain food reward. The different strategies had to be applied flexibly to optimize the rate of reward delivery. In the scene memory task, monkeys learned 20 new object-in-place discrimination problems in each session. Monkeys were tested on both tasks before and after bilateral ablation of orbital prefrontal cortex. These lesions impaired new scene learning but had no effect on strategy implementation. This finding supports a role for the orbital prefrontal cortex in memory but places limits on the involvement of orbital prefrontal cortex in the representation and implementation of behavioral goals and strategies.

What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies

When Pat Goldman-Rakic described the circuitry and function of primate prefrontal cortex in her influential 1987 monograph (Goldman-Rakic, 1987), she included only a few short paragraphs on the orbitofrontal cortex (OFC). That year, there were only nine papers published containing the term "orbitofrontal," an average of less than one paper per month. Twenty years later, this rate has increased to 32 papers per month. This explosive growth is partly attributable to the remarkable similarities that exist in structure and function across species. These similarities suggest that OFC function can be usefully modeled in nonhuman and even nonprimate species. Here, we review some of these similarities.

Reconciling the roles of orbitofrontal cortex in reversal learning and the encoding of outcome expectancies

Damage to orbitofrontal cortex (OFC) has long been associated with decision-making deficits. Such deficits are epitomized by impairments in reversal learning. Historically, reversal learning deficits have been linked to a response inhibition function or to the rapid reversal of associative encoding in OFC neurons. However here we will suggest that OFC supports reversal learning not because its encoding is particularly flexible-indeed it actually is not-but rather because output from OFC is critical for flexible associative encoding downstream in basolateral amygdala (ABL). Consistent with this argument, we will show that reversal performance is actually inversely related to the flexibility of associative encoding in OFC (i.e., the better the reversal performance, the less flexible the encoding). Further, we will demonstrate that associative correlates in ABL are more flexible during reversal learning than in OFC, become less flexible after damage to OFC, and are required for the expression of the reversal deficit caused by OFC lesions. We will propose that OFC facilitates associative flexibility in downstream regions, such as ABL, for the same reason that it is critical for outcome-guided behavior in a variety of setting-namely that processing in OFC signals the value of expected outcomes. In addition to their role in guiding behavior, these outcome expectancies permit the rapid recognition of unexpected outcomes, thereby driving new learning.